Dr. Patrick Sutton, who works at Huntington Hospital in Pasadena, is at the center of the allegations.
The Los Angeles Times reports that he made comments about a patient's appearance and sex life and that the complaint has been brought to the Medical Board of California.
The president of Huntington Hospital said she takes patient health and safety seriously and is closely following the investigation.
Sutton has been accused of sexual misconduct multiple times in the past.
He's practiced in Pasadena since 1989 and has reportedly delivered more than 6,000 babies.