Norquist points to a comment Hovde made as a CNBC analyst in 2009 that he personally had "no problem" being charged higher taxes before attacking President Obama and congressional Democrats on some of their tax moves. That attack has also been made by the fiscally conservative Club for Growth, which is backing former Rep. Mark Neumann (R-Wis.). Former Gov. Tommy Thompson (R) is also in the highly fluid race, which has gotten increasingly nasty in recent weeks.

Hovde's campaign was unapologetic.

"When Mr. Norquist says Eric has not made a written commitment opposing higher taxes, he's just plain wrong. Eric has signed his own tax pledge that will lower rates across the board, simplify the tax code and end corporate welfare," Hovde spokesman Sean Lansing said in response to Norquist. "Eric's plan leaves him beholden to the people of Wisconsin and nobody else."

In June, Hovde told The Hill he wanted lower overall tax rates but said tax loopholes should be closed, and called for an end to "corporate welfare." Norquist opposes any tax increases.

"It depends on how you get the revenue. I do not believe in increasing tax rates," he said when asked if he could support a deal to cut the national deficit with a deal that had 10 parts spending cuts to every one part revenue increases. "It doesn’t make sense to put taxes on the formation of capital. But I want to get rid of all this corporate welfare. If we can drive rates down and rip out this corporate welfare we'll have a burst of economic activity. As it comes down to exemptions, blow it out, get rid of it.

"Our tax policy is absurd, it really is," he continued. "Giant businesses benefit massively from our distorted tax system quash any effort to have meaningful reforms in our basic tax system. When small businesses are being taxed at a corporate rate of 35 percent and big businesses are being taxed at half that or almost nothing, you're making it impossible for small business to compete."

—This post was updated at 6:10 p.m.