As many as 11 agency employees are suspected of engaging prostitutes in Cartagena, Colombia, while setting up security for a visit by President Barack Obama. Three employees have been forced out so far, including two supervisors.
New York Congressman Peter King, who has been briefed on the investigation, says more firings may be imminent.
"I wouldn't be surprised if you saw more dismissals and more being forced out sooner rather than later," King said. "You may see a few more today or tomorrow."
Sen. Chuck Grassley, the senior Republican on the Senate Judiciary Committee, said he thought more people would be fired within "just a few days."
"'I expect there will be more, but that's what the investigation is all about," Grassley said.
An Air Force colonel and a military lawyer have gone to Colombia as part of the military probe. ABC News has learned that investigators have interviewed some of the prostitutes involved in the scandal.
On Thursday, the White House expressed renewed confidence in the director of the Secret Service.
Mr. Obama's chief spokesman, Jay Carney said the president's security in Cartagena was never compromised, and he asked for patience as official investigations continue.
The Associated Press contributed to this report.
