The fishermen reeled in the 4-foot shark off the pier at Huntington Beach on Monday.
A warden found the men in possession of the shark, but he thought it was a mako shark and he let the men go. A short time later, the warden realized it was a great white.
Chris Lowe, a marine biologist at California State University at Long Beach, saw video of the catch on YouTube and helped state authorities confirm that the animal was in fact a great white, probably no more than 3 months old.
Lowe said it's likely the shark would have survived had it been thrown back into the water.
Even when they grappled it to pull it out, my guess is that the fish probably could have been released alive," Lowe said.
Investigators are in possession of the carcass, but they have not released the identity of the fishermen in the video.
"One of the reasons why white sharks are protected many places including California is not because they're over-fished or they're endangered, it's because they're susceptible to that," Lowe said.
Southern California coastlines are breeding grounds for great whites and it's not uncommon to see the pups swimming close to shore from spring into late summer.
Great whites have been protected under California law since 1994. If the fisherman knew it was a great white shark, the catch would be illegal.
The Associated Press contributed to this story.