The attack happened in the area of Colorado Boulevard and Holly Street around 11:30 p.m. Saturday.
The victim said she was walking alone after leaving a bar when she was approached from behind by the suspect.
"She didn't see the person come up to her, but that male put a sharp object to her back," said Lt. Tracey Ibarra of the Pasadena Police Department.
Investigators said the suspect sexually assaulted the woman after forcibly taking her to Legge Alley, a passageway between Union and Holly streets. He then fled the alley on foot.
"She did believe that he had a weapon, so that's why she complied with his orders to follow him. She initially thought that maybe it was going to be a robbery or something other than the sexual assault," said Ibarra.
Police said they have extra patrols assigned to Old Town Pasadena on weekends and that it is unusual to have a sexual assault in the area.
"It always feels really safe here, so that kind of shocks me a little bit and kind of makes me a little on edge right now," said Pasadena resident Jessica Elle.
Authorities are hoping someone saw something that can help them catch the attacker. The suspect was described as a Hispanic male, between 30-40 years old, about 5 feet 8 inches tall, with a medium build. He was last seen wearing dark clothing.
Anyone with any information on the assault was urged to the Pasadena Police Department at (626) 744-4241.