The Association of Special Agents claims that Brown targeted 206 personnel for layoffs because the group endorsed Brown's opponent, Republican Meg Whitman, last year.
The layoffs were a result of the June budget, which cut $35 million in funding for the Bureau of Narcotic Enforcement. The agents call it political retaliation, but Brown's office says the cuts reflect the state's harsh fiscal realities and are not a vendetta.
"Those allegations are baseless and without merit. This is not a reduction that the governor proposed in his budge back in January. It's not a reduction that the governor proposed when he revised his budget back in May," said Deputy Director of Finance H.D. Palmer.
Palmer also said the legislature left Brown little choice, because it refused to extend sales taxes and vehicle license fees that would have lessened the need for cuts.
The Associated Press contributed to this report.
