The transportation community is especially livid because voters said they don't want that happening anymore.
"It would have a devastating effect on highway programs and transit programs across the state. We're just wondering what part of 'NO!' don't they get?" said transportation advocate Mark Watts.
The law requires any borrowing of those funds to be paid back within three years.
With the state so broke, Governor Schwarzenegger told the Los Angeles Times raiding those funds is a bad idea because it would probably force them to raise the sales tax. However, he has not taken the idea off the table.
"It's not necessarily a sales tax. I'm sure there's a number of bad things that could happen, whether it's a sales tax or some other kind of tax. But regardless, you have to pay that money back. You can't steal that money without paying it back," said the Governor's Press Secretary, Aaron McLear.
The Senate President also dislikes the idea because it eventually takes money from other programs if some kind of tax isn't approved to pay back the loan. Republicans have refused to support any tax hikes.
"Democrats will not stand for more borrowing. We've borrowed and borrowed. What happens is you have to borrow the money and you have to pay it back with school money, healthcare money and money for our elderly," said State Senate President Don Perata, (D) Oakland.
With Californians pinching pennies in this economy, the idea of a higher sales tax makes many roll their eyes.
"People are getting fired. People are losing their homes. It should be a time when the government is helping people. And they want to put more money out of our pockets? How is that fair?" asked Abraham Perez, who opposes higher taxes.
And stopping or delaying road projects is also unpopular because Californians want better commutes.
The borrowing plan hasn't been completely abandoned, despite opposition. It is one of the few budget-solving ideas Republicans support. At this point, the Governor and Democrats might be swayed just to get a budget in place. The budget is almost a month late.
