Bands of heavily armed gunmen attacked the prison and the guards from the outside, allowing inmates to escape in the southern port city of Mukalla.
Officials say those who escaped included fighters convicted on terror charges or awaiting trial.
The escape was described as carefully choreographed, as 57 al Qaeda-linked militants attacked their guards and seized their weapons before they made their way to freedom.
There are fears al Qaeda is exploiting the popular uprising against President Ali Abdullah Saleh to gain strength in southern Yemen.
Yemen's political crisis began when demonstrators inspired by successful uprisings in Egypt and Tunisia took to the streets in early February to demand Saleh's ouster. The largely peaceful movement gave way to heavy street fighting when tribal militias took up arms in late May.
Saleh, Yemen's president of nearly 33 years, was badly wounded in an attack on his Sanaa compound earlier this month and is undergoing medical treatment in Saudi Arabia. The head of Yemen's most powerful tribal confederation warned Tuesday in a letter to the Saudi king that Yemen could plunge into civil war if Saleh is allowed to return home.
The Associated Press contributed to this report.
