The sequester would require $109 billion in cuts to defense and social programs in 2013. Ryan and other Republicans have said this would disproportionately cut from the Defense Department, and that Democrats are also seeking to dodge these cuts.

"The sequester, according to the secretary of Defense, the president himself, would hollow out our military when it kicks in on Jan. 2 next year," Ryan said.

The Republicans' Sequester Replacement Act, H.R. 5652, would maintain the Medicare cuts under the sequester, but make up another $19 billion through an overall reduction to discretionary spending cuts. That leaves another $72.5 billion in cuts, but H.R. 5652 would cut $315 billion, leading to an additional $242 billion in new deficit reduction.

Early in the debate, Democrats continued to attack the bill as one that does not seek any additional tax revenue from the wealthy, which they have proposed.

"They have signed a pledge that says, 'We're not going to ask for one penny of additional contribution from people making more than $1 million a year to help reduce our deficit, not one penny,' " Budget Committee ranking member Chris Van Hollen (D-Md.) said.

The House started two hours of debate at around 11 a.m., and afterwards was expected to pass the bill.