An additional round of ammunition was found on the White House exterior. According to the Secret Service, the bullets were found Tuesday morning.
President Barack Obama was not at home at the time of the shooting.
The incident follows a report of a man firing an assault rifle on the Washington Mall near the White House on Friday. The suspect fled the scene after the incident.
Secret Service agents fanned out across the country looking for the suspect, and by Wednesday morning, Oscar Ramiro Ortega-Hernandez of Idaho was taken into custody at Pennsylvania Hampton Inn.
Officials had described him as a mentally ill man who may have deadly intentions.
"This is a man who discharged a rifle on downtown D.C. I think that speaks volumes," said Sgt. David Schlosser of the U.S. Park Police.
Authorities later found a car containing an assault rifle and empty shell casings, and witnesses reported hearing shots and saw two speeding vehicles in the area.
The Secret Service said it has not connected Friday's incident with the bullets found at the White House.
The Associated Press contributed to this report.
