Before authorities announced his capture, several gunshots were heard near the bank. Regional police official Frederic Tamisier says the suspect was slightly injured during his capture. Authorities also said the gunman had psychiatric problems and claimed he was acting for religious reasons.
One French TV station said the bank's branch director was among the four hostages.
Toulouse is also where a man linked to Al Qaeda opened fire on a Jewish school back in March, killing four people. That man was killed after a 32-hour standoff with police.
The Associated Press contributed to this report.