It all started just before 4 p.m. when emergency crews were sent to the 1500 block of Delamare Drive in Avocado Heights after receiving a call about the animal.
A resident of the neighborhood spotted the dog hiding under a car around 11 a.m. and tried to get him out.
"He wouldn't," said Jeremy Thana. "I brought him some water, and then my roommates came out and we were trying to get him out and then he just bolted for this storm drain right here, and he dove right in!"
Another neighbor tried to help but realized the canine was scared.
"When I went to see him, he just had that look like, 'Help me!' said Gregory Marquez. "No barking, no yelping, just ... I need to get out."
Thana called animal control, but after two hours of waiting, he called the fire department.
AIR7 HD was above the scene as firefighters worked to retrieve the animal. The Los Angeles County Fire Department's Urban Search and Rescue team assisted in the rescue.
"We weren't able to get him out and he was scared," said Denice Lopez, a firefighter with USAR Station 103. "Every time we tried to grab him, he'd run back in the tunnel. We had to have the guys go down on the other manhole, make a lot of noise to kind of scare him back towards us. We were able to finally grab him and get him back out."
The animal did not appear to have any serious injuries.
It's unclear if the dog belongs to anyone. He was wearing a collar but it didn't have any identification.
"Being an animal lover myself, just to see them have that compassion for a strange dog ... they never stopped," said Marquez of the rescue crews.
The dog is now in the care of animal control services.