The so-called White Fire erupted Monday afternoon and forced some 6,000 campers out of the forest. It is threatening dozens of cabins and vacation homes.
The fire has burned 1,800 acres of dry chaparral, oak and pine and is still growing. It's about 10 percent contained.
A lull in winds helped firefighters through the midday, but the forecast later in the day was for wind gusts of up to 40 mph, Santa Barbara County fire Capt. David Sadecki said.
More than 550 firefighters were on the line, supported by a dozen aircraft.
"It's an out-of-control wildfire so it is a threat. There's a lot of dry vegetation in its path," Sadecki said. "It's still spring - it's not even summer - and it's burning like it's August or September."
The fire destroyed two vehicles and a U.S. Forest Service garage, and sent an enormous ashy plume over the mountains.
The cause of the blaze was under investigation.
The Associated Press contributed to this report.