According to U.S. officials, the drone attack on Fahd al-Quso was conducted by the CIA.
Al-Quso was struck by a missile as he stepped out of his vehicle along with another al Qaeda operative in the southern Shabwa province, Yemeni military officials said.
The strike was authorized by the Yemeni government, which then made the announcement after the operation was complete. The CIA and U.S. military had conducted an extensive surveillance operation before the strike, officials said.
Al-Quso, 37, was on the FBI's most wanted list, with a $5 million reward for information leading to his capture. He was indicted in the U.S. for his role in the 2000 bombing of the USS Cole in the harbor of Aden, Yemen, in which 17 American sailors were killed and 39 injured.
He served more than five years in a Yemeni prison for his role in the attack and was released in 2007. He briefly escaped prison in 2003 but later turned himself in to serve the rest of his sentence.
The Associated Press contributed to this report.