Crews with the Los Angeles County Fire Department and U.S. Coast Guard, along with the Los Angeles County Sheriff's Department, had resumed the search beginning at first light Friday after searching throughout the night Thursday.
On Friday night, the Coast Guard called off the effort "pending further updates" after searching more than 600 square miles.
The man, identified as 24-year-old Keion Dave, was reported to have fallen off the Jet Cat Express around 6 p.m. Thursday about three miles off the coast as the high-speed catamaran headed to Santa Catalina Island, authorities said.
Before the search was called off, his family expressed hope that he would be found alive.
"I hope he's safe and warm somewhere and just OK," said his sister Kiami Dave.
The boat is operated by Catalina Express. The company's president, Greg Bombard, told reporters that deckhands saw the man go overboard from the stern of the vessel.
"When they saw him hit the water, they threw a life ring over," he said.
Bombard said he didn't know if Dave jumped or fell off the boat.
"People saw him go over the side, so we'll look at the footage we've got, and we'll figure that out," Bombard said. "We've never had anybody jump over like that or fall over like that."
The ferry, which was carrying around 150 passengers, returned to Long Beach and passengers were transferred to another boat to continue the trip to the island.
Passengers set to board the ferry on Friday told Eyewitness News they were shocked to hear of the incident.
"I know the rails are so high, higher than waist high, so I was trying to figure out how did he fall over? Or maybe he was sitting on the rail, trying to take a picture and he fell over," said Wendy Griffin.
The Associated Press contributed to this report. The above video is from a previous report.