Swift admitted to inflated deductions of more than $12.5 million.
Prosecutors say the number is extremely conservative and does not include losses suffered by the state of California.
When some of Swift's clients received audit notices from the IRS, Swift assisted his clients in avoiding payment of their tax liability and in obstructing the audits by fabricating false charitable contribution letters and Forms 1098, which detail mortgage interest paid.
Swift was already in custody after being convicted in an unrelated case.