The epicenter was located 12 miles east of San Jose on the Calaveras Fault, the USGS said.
The quake occurred at 11:42 a.m. and was measured at a depth of about 4 miles.
There were no immediate reports of any injuries or damage.
Tuesday's quake was the largest in the Bay Area since the 6.0 magnitude Napa earthquake on Aug. 24, 2014, seismologist Lucy Jones noted.
"What you have is that the same seismic stations that tell us what the earthquake is are now able to give you the information so quickly, you can give it a few seconds before the shaking gets to your site," Jones said, referring to California's earthquake early warning system. "So in the Bay Area in San Francisco, because the waves had to travel up from south of San Jose, they would have would have had a decent amount of warning. But of course, if you were actually in Alum Rock you're shaking is what allowed the others to know."
5.1 earthquake strikes near San Jose
In accordance with protocol, the Bay Area Rapid Transit system said trains would be holding for five minutes and will be followed by inspections. Riders were advised to expect delays.
Caltrain also said trains in the South Bay area of the region were slowing as a safety precaution.
"5.0+ earthquakes are pretty rare in the Bay Area," KGO-TV meteorologist Drew Tuma said on Twitter. "In the past 30 years, we've only had three others. Back in 2000, a magnitude 5.0 was reported in Yountville, in 2007, a magnitude 5.6 struck Alum rock, and the magnitude 6.0 in Napa in 2014."
The Santa Clara County Fire Department said that out of an abundance of caution it was performing a roll call of stations and units to ensure readiness, adding that the agency had not received any calls for service related to the earthquake.
How to make your own earthquake kit