His attorney, Brian Sun, said Carona's wife, Debi, was by his side as Carona surrendered Tuesday.
The Littleton prison is known as "The Camp." It's a minimum-security facility with little to no fencing and dorm-room-style housing.
Carona must serve at least 85 percent of his sentence of 5 1/2 years.
Carona sought to serve his time in California, but the Bureau of Prisons assigned him to the Colorado facility, said Sun.
Carona, 55, never shied away from the media spotlight, even after a grand jury indicted Orange County's former top cop in 2007.
"I look forward to my day in court," said Carona in 2007.
Carona has been free for the past two years while awaiting appeal after being convicted on a witness-tampering charge.
Carona secretly tape-recorded by his former friend and ex-assistant sheriff Don Haidl as he tried to convince the wealthy businessman to lie during an investigation into corruption in the /*Orange County Sheriff's Department*/.
Haidl cooperated with federal authorities after he pleaded guilty in 2007 to federal tax crimes that he allegedly gave Carona money and gifts.
The audiotape was key evidence that Carona's attorneys wanted barred. The 9th Circuit Court of Appeals ruled this month that Carona's appeal had no merit.
"The government acted ethically, appropriately and lawfully at all times," said Brett Sagel, assistant U.S. attorney in the Santa Ana office.
Once dubbed "America's Sheriff" for his determination to find the killer of 5-year-old Samantha Runnion in 2002, voters re-elected Carona sheriff three times.
The brief appeals court hearing ended with the judge saying "Justice has been done." Federal prosecutors agreed.
"At the end of the day, Michael Carona was removed from office, he's a convicted felon, and he's now in jail for five and a half years," said Sagel.
