South Carolina Gov. Nikki Haley (R) on Tuesday dealt a strong rebuke to GOP presidential candidate Newt Gingrich, accusing him of undercutting the House Republican budget plan. 

Haley, the governor of a key early primary state, said that Gingrich's criticism of Rep. Paul Ryan's (R-Wis.) budget was "absolutely unfortunate."

"Here you've got Representative Ryan trying to bring common sense to this world of insanity, and Newt absolutely cut him off at the knees," Haley told CNN

Haley is the latest conservative to pile on Gingrich, the former House Speaker, for calling Ryan's plan on Sunday an example of "right-wing social engineering."

But as governor of South Carolina, Haley's words could have a tremendous effect on the outcome of her state's primary, which is considered an important bellwether for GOP presidential candidates. 

Ryan's plan would transform Medicare into a so-called "premium support" system that would subsidize senior's purchases of private health insurance. Democrats have used the plan to attack Republicans, accusing them of attempting to privatize Medicare.

"With allies like that, who needs the left?" Ryan shot back at Gingrich on Monday.

Gingrich partially backed away from his comments on Monday, saying he used strong language in criticizing them. But he stood by his belief that Ryan's Medicare proposal could be politically tough. But Gingrich spokesman Rick Tyler said that there is "little daylight" between Ryan and Gingrich on Medicare.

But that did not stop Haley from rebuking Gingrich for criticizing the Ryan plan.

"When you have a conservative fighting for real change, the last thing we need is a presidential candidate cutting him off at the knees," she said.