The group boarded the bus Thursday night to travel from the Aquarium of the Pacific to head to the Queen Mary's Dark Harbor event.
VIDEO: Wild shuttle ride to Queen Mary leaves passengers climbing out of windows
The trip, which was supposed to be just one mile, took a terrifying turn when the passengers said their bus driver refused to stop, driving for miles in the wrong direction.
The frightened passengers jumped from the windows in fear they were being kidnapped.
"I told him straight up, I'm like listen, at this point it's kidnapping. You can't hold us like this," said Brian Corbitt, a passenger. "I stood up with him and he just whipped around and nailed me in the stomach and knocked me back into a seat."
Fellow passenger Scott McCaslin described a chaotic scene.
"People start opening up the windows and everybody starts jumping out and clambering and just wedging in a panic," McCaslin said.
Police initially arrested the driver on suspicion of kidnapping. He was later released when investigators determined he was lost due to street closures in the area.
"At the moment, detectives are going to continue interviewing some of the patients on the bus that they're able to, and complete their investigation," said Officer Alvino Herrera, a spokesman for the Long Beach Police Department.
The incident is still being investigated for reports of battery between the driver and a passenger.
The Queen Mary is offering a full refund to each of the passengers involved.