Police were called shortly after 1:30 p.m. to the Atlantic Station Publix where the manager told them an armed man had entered the store and headed straight to the bathroom, police spokesman Officer Anthony Grant said.
"A witness observed the male and alerted store management, who then notified police," Grant told The Atlanta Journal-Constitution.
Officers saw the man leave the bathroom and quickly held him for questioning. According to police, his weapons included two long guns and three pistols, all of which were concealed.
Police identified the man as Rico Marley, 22, and said he has been charged with reckless conduct, the newspaper reported. Police are still working to determine why he had so many guns inside the store.
Magazine photographer, police officer, soon-to-be grandfather: What we know about Boulder shooting victims
Investigators said the man is undergoing a psychiatric evaluation.
Several police cars were parked at Atlantic Station as officers continued their investigation.
Atlantic Drive, a main street running in front of the store, was closed to traffic. Shoppers were being turned away as they approached the grocery store, but there were still plenty of people milling about the open-air mall.