"We need to let these investigations go forward to try to figure out what we have here," he said on CNN's "Starting Point," but predicted that eventually, "it'll be one of those critical moments where we have to correct our course."
Other lawmakers have been more critical of the FBI's management of the scandal, which became public on Friday with Petraeus's resignation after he admitted to an extramarital affair.
Sen. Bernie Sanders (I-Vt.), Senate Intelligence Committee Chairwoman Dianne Feinstein (D-Calif.) and House Homeland Security Chairman Pete King (R-N.Y.) all said earlier this week that Congress should have been told sooner that Petraeus was under investigation.
Rep. Jason Chaffetz (R-Utah), in a separate interview on CNN, said he is more concerned that Feinstein did not know.
"You're not going to notify all of Congress," he said. "But those people [the chairmen of the House and Senate Intelligence committees] have to know."
Rep. Debbie Wasserman Schultz (D-Fla.), speaking on CNN in a separate interview, agreed that the leadership of the Intelligence committees should have been told before the rest of Congress.
"The leadership, the big four, are supposed to be notified," she said, referring to the chairmen and ranking members. "That's protocol."
The investigation into Petraeus, his mistress Paula Broadwell and Jill Kelley, the woman who originally triggered the investigation into Petraeus, has now extended to the top U.S. commander in Afghanistan, Gen. John Allen.
The FBI has uncovered 20,000 to 30,000 pages of "potentially inappropriate" communications between Allen and Kelley. Cummings said the high volume of correspondence under investigation suggests the investigation is "becoming quite serious."
Updated at 8:40 a.m.