Afghan officials said many of the 476 prisoners who escaped are Taliban insurgents.
It happened late Sunday night through a 1,000-foot-long tunnel dug from the outside by insurgents.
Officials at the prison said they only discovered the breach at about 4 a.m., a half hour after the Taliban said they had gotten all the prisoners out.
The militants began digging the tunnel about five months ago from a house within shooting distance of the prison guard towers. It was not immediately clear whether they lived in the house while they dug. They meticulously plotted the tunnel's course around police checkpoints and major roads, the insurgent group said in a statement.
The city's police mounted a massive search operation for the escaped convicts. They shot dead two inmates who tried to evade capture and re-arrested another 26, said Tooryalai Wesa, the provincial governor.
The Associated Press contributed to this report.