The brawl occurred around 2:21 p.m. just north of the Pier near Tower 14.
When officers arrived, they found a man suffering from a stab wound, but police said it appeared to be non-life threatening. Another man may have broken his ankle during the altercation, according to police. Both victims were sent to local hospitals for treatment.
It's unclear what started the brawl, but according to witnesses who spoke with Eyewitness News, some of the men involved appeared to be street vendors. Police have not confirmed that information.
"So I make my way down that way and it turns out it was like a brawl, like 12-plus people," said witness Jose Gutierrez. "They were just going ham with poles and flags and umbrellas - whatever they could get their hands on. I saw a guy with a belt in his hand."
Police said three of the five men who were arrested were taken into custody for assault with a deadly weapon.
Another was arrested for battery and the fifth suspect was arrested for sexual battery, according to police.
Investigators didn't release further details. The incident remains under investigation.
Anyone with information is urged to contact the Santa Monica Police Department's Watch Commander at 310-458-8427 or non-emergency dispatch at 310-458-8491.
City News Service, Inc. contributed to this report.