Inside the walls of the Victorville Federal Correctional Complex are about 6,000 prisoners.
Don Shults, a representative of 3669 FCC Victorville, said he's worried about officer safety at the Victorville prison because the prison is overcrowded and understaffed.
"Our lives are on the line because of sequestration and being furloughed, and we need Congress to come together and reach some type of agreement," Shults said.
"With these cuts, it could raise the chances," of an attempted prison escape, Shults added.
Some nearby residents of the prison said they often think about the possibility of prisoners escaping.
"It concerns me a lot," said Michael Lasch of Victorville. "People don't realize what these guys are in there for. They're not in there because they were making cupcakes."
Victorville resident Dave Farley said he just feels bad for the officers who will have their pay cut.
"I don't see any problem on cutting back on that, but I hate to see that done," he said.
In a statement, Attorney General Eric Holder said in part, the "Bureau of Prisons would need to implement full or partial lockdowns and...reduce inmate reentry and training programs. This would leave inmates idle, increasing the likelihood of inmate misconduct, violence and other risks."
Holder said he has prevented daily furloughs of 3,570 federal prison staffers around the country, moving $150 million from other Justice Department accounts to stave off a serious threat to the lives and safety of correctional staff, inmates and the public.
Some 38,000 employees at the U.S. Bureau of Prisons supervise 176,000 inmates at 119 institutions, ensuring security and providing prisoners with needed programs.
The union said the furloughs would go into effect April 21.
The Associated Press contributed to this report.