"The problem here is two-fold: number one, there is a lack of outrage by the White House who could get to this a lot quicker if they were genuinely interested in cleaning up this mess," Rep. Aaron Schock (R-Ill.) told Chicago radio station 89 WLS on Wednesday.

Schock complained that the administration is still claiming that a few low-level staffers in an IRS office in Ohio were the only ones responsible for the focus on conservative groups.

"They're trying to downplay and still trying to suggest that this was just a few low-level staff in the Cincinnati office," Schock said. "Yeah, Lois Lerner, one of the top officials, now taking the Fifth. It's not low-level staff."

Lerner, the IRS official who oversees the agency's tax-exemption division, invoked her Fifth Amendment rights on Wednesday before the House Oversight Committee. Lerner said she had done nothing wrong, but had been advised by her attorneys to plead the Fifth.

President Obama has condemned the IRS’s actions and forced the agency’s acting director to resign last week.