Most of that allegedly happened after 1999, when Romney says he left the company he founded.
"I went out to run the Olympics and said goodbye to my colleagues. They took over the business," he said.
But filings with the Securities and Exchange Commission list Romney as CEO and president of the company until 2002. Romney says he retained ownership shares, but that was all.
"I think most Americans figure if you are the chairman, CEO and president of a company that you are responsible for what that company does," said President Barack Obama.
Romney claims as a former businessman, he has the ability to create jobs. So the issue of whether he was involved in outsourcing jobs is a natural for the attack ads. But Romney says it's unfair and should be out of bounds.
"That's ridiculous and disturbing to come from their campaign and beneath the dignity of the president and his campaign. The president needs to take control of these people," said Romney.
In an interview Friday with ABC News, Romney had no comment on a report that Condoleezza Rice is a front runner in his search for a running mate. Rice had no comment either, but she has previously said she does not want the job.