Newt Gingrich ended up to the left of his GOP rivals on immigration during a CNN debate on Tuesday, defending his position that the United States shouldn't deport every illegal immigrant while the other presidential hopefuls accused him of supporting magnets that attract illegal immigration.

"I don't see any reason to punish someone who came here at 3 years of age and wants to serve the United States of America," Gingrich said, adding that those whose parents brought them to the country illegally but want to serve in the U.S. military should be included in a program that lets foreigners earn citizenship through military service.

Gingrich said he found it hard to believe that Americans would support deporting millions of people and that visas should be made available for those with math and science skills.

But he also made it clear that flexibility would apply only to those who had been contributing positively to the nation for years, not to those continuing to flout U.S. immigration laws.

"I do believe if you've come here recently and you have no ties to the U.S., we should deport you," he said.

All of the candidates are keenly aware that Hispanics make up the fastest-growing voter bloc and that appearing anti-immigrant could hurt their candidacy. But they also know that the GOP base is hungry for tough talk on securing the border and preventing an influx of new workers when U.S. unemployment is so high.

Michele BachmannMichele Marie BachmannBoehner says he voted for Trump, didn't push back on election claims because he's retired Boehner: Trump 'stepped all over their loyalty' by lying to followers Boehner finally calls it as he sees it MORE said she thought it would be wrong to offer citizenship to 11 million people who had broken the law by entering the country, while Mitt Romney said amnesty was an illegal immigration magnet.

"This is a party that loves legal immigration. But we have to stop illegal immigration," Romney said.