More than 10,000 taxpayers wrongly received tax breaks for homes claimed by others, estimated in the the tens of millions of dollars, the report said.
"This is very troubling," George said. "Congress created and modified the homebuyer credit to stimulate the economy and help taxpayers achieve the American dream, not to line the pockets of wrongdoers."
The Internal Revenue Service is trying to get the money back. The IRS processed more than 2.6 million taxpayer claims worth $18.7 million through April, with only a few fraudulent claims.
"A very small number of payments were made to prisoners incorrectly, which the IRS is now taking all steps to recapture and to prevent going forward," the IRS said in a statement. "The IRS will follow up on every instance of an improper prisoner payment and take swift and appropriate enforcement actions."
So far, the IRS says it has stopped about 400,000 potentially fraudulent claims.
"These aggressive efforts have saved taxpayers more than $1 billion," the IRS said.
The report doesn't paint a rosy picture for a tax credit that was pushed by the National Association of Realtors and gave the housing market some life.
The first-time homebuyer tax credit was put into place in 2008, giving couples up to $7,500 that had to be repaid over the course of 15 years and was considered an interest-free loan.
The tax credit was changed last year, increasing the amount to as much as $8,000 and eliminating the repayment requirement. Starting in November, Congress also allowed previous homeowners to receive a credit of up to $6,500. The tax credit expired April 30.
Homebuyers must close by June 30 but Congress is pushing for a three-month extension until Sept. 30.
The report examines the tax credit under both programs.
This story was updated at 4:06 p.m.