This comes in the wake of al Qaeda terror threats that have shut down 19 U.S. embassies across the Middle East and Africa.
"We are concerned about a threat stream indicating the potential for terrorist attacks against U.S. persons or facilities overseas, especially emanating from the Arabian Peninsula," State Department spokeswoman Jen Psaki said in a statement. "The Department is taking appropriate steps to protect our employees, including local employees and visitors to our facilities."
The evacuated workers were loaded onto a C-17 plane and flown out of the country. Gerald Feierstein, the U.S. ambassador in Yemen, has arrived in Washington D.C. A small number of emergency staff will stay in Yemen.
An emergency message to U.S. citizens posted on the U.S. Embassy in Yemen's website described the terror threat as "extremely high."
"U.S. citizens currently in Yemen should depart," the message stated. "As staff levels at the Embassy are restricted, our ability to assist U.S. citizens in an emergency and provide routine consular services remains limited and may be further constrained by the fluid security situation."
There is also increased security at airports from Los Angeles to New York.
U.S. officials say al Qaeda's chief Ayman al-Zawahri sent secret messages from Pakistan to the leader of al Qaeda in Yemen about an attack.
Officials also uncovered communication between al Qaeda and someone in the United States via the U.S. mail.
U.S. officials reportedly intercepted secret messages between al Qaeda's top leaders about a potential attack. The reported goal is to load explosives onto trucks and target Westerners. Trucks are being searched on the ground in Yemen for explosives.
Agencies in the U.S. say they're tracking the threats.
"I can certainly tell you that all of our agencies are taking the threat very seriously and obviously we've taken broad reaction to the intelligence we've received," said Rep. Adam Schiff, D-Calif. "That doesn't happen for a garden variety intelligence, garden variety chatter. It has to be pretty high-level, it has to be corroborated, we have to have confidence in our sources."
One former U.S. counter intelligence official says al Qaeda's most dangerous plotters are now based in Yemen, and the country is now a haven for the terrorist group.
"This group is fairly ingenious, fairly bold and eager to cause damage, and there is some discussion that it's leadership has been elevated in the global organization for more responsible roles," said Daniel Benjamin, former ambassador-at-large.
The U.S. military is responding to these threats with drone strikes. Four alleged al Qaeda members were killed Tuesday in Yemen.
ABC News and the Associated Press contributed to this report.
