With 100 percent of precincts reporting early Wednesday, Galperin had 56 percent of the votes and Zine had 44 percent.
Galperin, 49, sits on two volunteer city commissions, one focused on revenue efficiency and the other on quality and productivity. Both candidates had promised to root out abuse, fraud and waste as the city's official watchdog.
Zine, 65, spent the last 12 years on the City Council, representing a west San Fernando Valley district. He was also a Los Angeles Police Department officer for 33 years.
Galperin will take over the controller's job from Wendy Greuel, who lost to Eric Garcetti in her bid to become mayor.
City News Service contributed to this report.