"The major problem with the color-coded system is it didn't communicate information," said Homeland Security Secretary Janet Napolitano.
The replacement is called the National Terrorism Advisory System and will roll out in the next three months. It focuses on specific threats in geographic areas and will also have just two levels - imminent and elevated.
When agency officials think there is a threat the public should know about, they will issue an announcement and rely on news organizations and social media outlets to get the word out.
Critics said the old color codes did little to inform the public. Under that system, green, at the bottom, signals a low danger of attack; blue signals a general risk; yellow, a significant risk; orange, a high risk, and red, at the top, warns of a severe threat.
Since the outset, the nation has never been below the third threat level, yellow - an elevated or significant risk of terrorist attack.
The Associated Press contributed to this story.