Brady was responding to a question about President Obama's recent comment that his agenda would have an easier time going forward if Pelosi again ran a Democratic-controlled House.

Brady also spoke on the same day that the government released data saying the U.S. economy added just 88,000 jobs in March, far below the 200,000 that many economists expected. He said the employment number is more evidence that higher taxes, more regulation and uncertainty over implementation of the healthcare law hurts job growth.

"We know it doesn't work," Brady said. "Higher taxes, massive stimulus, spending higher, debts, more regulation. That's what this creates: a dismal economy for America."

When asked about gay marriage, Brady said he believes that Republicans who have said they now support that idea are "sincere," but he warned against going too far down the road of embracing concepts that most associate with Democrats.

"[F]or this party, being Democrat-light doesn't get us anywhere," he said.

And on gun control, he said House Republicans are ready to beat back whatever legislative proposal the Senate might to approve.

"In the House, we're not letting our guard down at all," he said. "We're prepared to knock these things down because they don't make us safe and they don't protect our Second Amendment rights."