San Diego police said officers responded to a report of a lost swimmer at Pacific Beach near Crystal Pier just before 2 a.m.
They searched the area but couldn't find him. About two hours later at 4:15 a.m., a body washed up.
The firefighter's identity has not been released, but he was reportedly in San Diego for the California Firefighter Summer Games.
News video from the scene showed several distraught people gathering at the scene Thursday morning. After sunrise, a dozen people escorted the firefighter's body -- placed on a gurney and covered with an American flag -- as it was wheeled to an awaiting coroner's van.
Fire Chief Colin Stowell of the San Diego Fire Department said he reached out to the chief of the Oakland Fire-Rescue Department, describing the conversation as "a very difficult call."
"All the fire service members, doesn't matter what agency, they're part of our family," Stowell said. "And so I knew from our own experience, having been through tragic accidents like this. I reached out to him and expressed my condolences, but also made sure that he knew that we would take care of his member just like one of our members down here."
Oakland fire officials were expected to drive down to San Diego.