The survivors endured mild hypothermia, jellyfish stings and a rainy night in the pitch black ocean.
Authorities say the family set out on Saturday morning on a 22-foot fishing boat. With winds gusting 40 mph and swells up to 10 feet, the boat capsized shortly after noon.
"It was so nasty that we couldn't get any aircraft up," said Officer Robert Dube of the Florida Fish and Wildlife Conservation Commission.
In the minutes after the boat sank, an elderly woman, who was not wearing a life preserver, slipped away and drowned. The three men in the group clung to the boat, while three women and a 4-year-old girl clutching a cooler drifted away.
"We always preach, you know, stay with the boat because it's easier to find an overturned boat than it is a head bobbing out in the ocean," Dube said.
By sheer luck, a commercial boater discovered the men, and Coast Guard rescuers soon found the women. Wayne Crosby was one of the Good Samaritans who helped get the survivors to shore.
"They just all had that look on their face, like a thousand-yard stare," Crosby said.
Authorities said on Monday that everyone was wearing a life jacket expect the 80-year-old woman. The three women and the child had the life jackets on first, and then one of the men swam into the capsized boats to get vests for the men. It was earlier reported that no one was wearing life jackets.
The boaters were taken to a local hospital.