The men, possibly painters, were left trapped and dangling 12 stories high after their scaffolding broke on Friday in Yonkers, just north of New York City.
Authorities said the men were only secured by safety belts.
A firefighter rappelled down the side of the building to rescue the men one at a time. Officials said it appeared the men were not injured.
The cause of the scaffold collapse was not immediately known.