The move brings long-awaited relief to the territory's Palestinian population and a significant achievement for its Hamas rulers.
The move also raises Israeli fears about militants gaining easier access in and out of Gaza.
Egypt and Israel have maintained a blockade over Gaza since 2007 to weaken Hamas following its seizure of the area.
Egypt's new military rulers decided to ease the blockade after president Hosni Mubarak was ousted back in February.
Hundreds of Gazans gathered early Saturday as the first bus load of passengers crossed the border at 9 a.m. Two Egyptian officers stood guard next to a large Egyptian flag atop the border gate as the vehicle rumbled through.
Until Saturday, the Rafah border terminal had functioned at a limited capacity. Only certain classes of people, such as students, businessmen or medical patients, were eligible to travel and the crossing was often subject to closures, leading to huge backlogs that forced people to wait for months.
The Associated Press contributed to this report.