Rep. Steve King (R-Iowa) believes the GOP's big loss with Hispanic voters isn't its biggest electoral problem.

When asked on Wednesday if it was time to move on comprehensive immigration reform, King reiterated his opposition to the idea — and said Republicans' problems came with other groups as well.

"We saw a marginal loss of the Hispanic vote, so people say we can't build a coalition again unless we make those accommodations," he said, according to Slate. "We should look at what happened with the conservatives who stay home. What happened with the biggest gender gap in history? What happened with the Ron Paul coalition, and the libertarians?"

King has long been an opponent of a broad-based immigration overhaul, and if Congress moves on one, many expect him to be a key player working against it.