The White House said Trump told Comey "that he has been terminated and removed from office."
Trump acted on the recommendations of Attorney General Jeff Sessions and Deputy Attorney General Rod Rosenstein, the White House said.
"The FBI is one of our Nation's most cherished and respected institutions and today will mark a new beginning for our crown jewel of law enforcement," Trump said in a statement released by the White House press secretary.
The White House said a search for a successor to Comey will begin immediately.
Comey had been scheduled to appear in Los Angeles Tuesday night for an agency recruiting event. He apparently learned of his firing by seeing it on television after he had already left for LA.
He did not appear at the recruiting event in Hollywood and instead was seen Tuesday night heading in a motorcade with a CHP escort to LAX and then boarding a private jet.
Trump wrote a letter to Comey informing him of the recommendations from Sessions and Rosenstein.
"I have accepted their recommendation and you are hereby terminated and removed from office, effective immediately," Trump wrote.
"While I greatly appreciate you informing me, on three separate occasions, that I am not under investigation, I nevertheless concur with the judgment of the Department of Justice that you are not able to effectively lead the Bureau."
"It is essential that we find new leadership for the FBI that restores public trust and confidence in its vital law enforcement mission."
"I wish you the best of luck in your future endeavors."
In Comey's absence, Andrew McCabe - Comey's deputy - is expected to serve as acting FBI director.
Sessions is expected to name an interim director during the search for a permanent replacement.
Former President Barack Obama appointed Comey to a 10-year term as FBI director in September 2013.
He found himself in the middle of several political controversies in the past year, including his role releasing information about the agency's investigation into then-candidate Hillary Clinton's emails.
More recently, he has testified to Congress about Russian interference in the election and possible connections between Russian officials and Trump's campaign.
Trump has been critical of Comey, including saying recently that he gave Hillary Clinton a "free pass for many bad deeds."
FBI Director Comey was the best thing that ever happened to Hillary Clinton in that he gave her a free pass for many bad deeds! The phony...— Donald J. Trump (@realDonaldTrump) May 3, 2017
Democratic lawmakers criticized Trump's decision, questioning if it was motivated by Comey's role in the Russia investigation.
"POTUS just fired FBI Director leading Trump/RussianHacking criminal investigation. Coverup?" tweeted Rep. Hakeem Jeffries, D-NY.
"Firing Comey has the foul stench of an attempt to stop an ongoing investigation into collusion between the Trump campaign and the Russians," tweeted Sen. Chris Van Hollen, D-Md.
A key Republican was also critical of the decision.
Sen. Richard Burr, R-N.C., who chairs the Senate Intelligence Committee, praised Comey and said he was "troubled" by his dismissal.
"I am troubled by the timing and reasoning of Director Comey's termination," Burr said. "I have found Director Comey to be a public servant of the highest order, and his dismissal further confuses an already difficult investigation by the Committee. In my interactions with the Director and with the Bureau under his leadership, he and the FBI have always been straightforward with our Committee. Director Comey has been more forthcoming with information than any FBI Director I can recall in my tenure on the congressional intelligence committees. His dismissal, I believe, is a loss for the Bureau and the nation."