"Freddie Mac really bothers me, you know when think about when you think about how much money he got from Fannie Mae and Freddie Mac, that’s really the taxpayers," Paul said. "Fannie Mae and Freddie Mac is a government agency because even though they weren’t getting the money directly, they were getting benefits and people made a lot of money…people should know that he is part of the problem, how can he be the solution if he lived off of Fannie Mae and Freddie Mac? How can he be wanting to clean up that mess if he’s received so much money from them?"

Gingrich has repeatedly denied lobbying on behalf of the mortgage company, but admitted to earning millions in consulting fees.

Pressed by Ingrahama about whether Gingrich was a conservative, Paul said he "wouldn't put him in that category."

"He was known to be from the left-wing of the party especially early in his years," Paul said. "He would come up with conservative viewpoints and all, but that doesn’t make him a conservative. No, I don’t put him in that category."

Paul released an ad Wednesday slamming Gingrich for "serial hypocrisy" and mocking Gingrich's past statements.