Horowitz goes on to say that conservatives "don't need this black mark on our brand," and that Tennessee's 4th Congressional District gives them the "first opportunity to throw out a weak incumbent and replace him with one of our guys."

"While anyone would be better than the incumbent, we would be wise to use this opportunity to insert one more member who is not committed to playing ball with GOP leadership. I’ll be working at the Madison Project and with others to evaluate the candidates in the coming months. Let’s work together on this and make sure we don’t squander this opportunity," he adds.

It's further bad news for DesJarlais, after reports that he had pressured a former mistress to get an abortion broke during his campaign. The avowed anti-abortion-rights lawmaker was charged with hypocrisy by Democrats, but still went on to win his heavily Republican district by 12 points.

Following his reelection, however, further details emerged concerning a number of former mistresses and a rocky divorce in his past. Tennessee Republicans now expect DesJarlais to receive a number of primary challengers in 2014; Tennessee state Sen. Jim Tracy has already announced his intention to run.

DesJarlais actually received a relatively high legislative score from The Madison Project for 2011. He voted against the deal brokered to avoid the series of tax increases and budget cuts known as the "fiscal cliff," and the extension of aid for Hurricane Sandy victims.

But Horowitz told The Hill the group is looking for what he referred to as a "conservative rock star," someone who not only lived conservative values but who is also willing to "stand up to leadership."

He said DesJarlais's district, because of its deep-red lean — it voted for Republican Mitt Romney for president by nearly a 2-to-1 margin this cycle — is ripe for a Republican insurgent to challenge the incumbent. The group plans to spend money and time on grassroots organizing for the race, if a proper challenger can be found.

Horowitz said that the group is not yet ready to jump into the race, but will be watching to see what candidates enter the race.

"It's surprising that [DesJarlais] would even seek reelection. That's just something, as conservatives, we shouldn't have to deal with," he said.