Warner says he fears rising interest rates if no debt agreement reached this summer

"Every one point we add to our interest rate, we add over a trillion dollars to our debt, as well as what it would do in terms of job growth and job creation," he said. "That's just unacceptable."

Warner was speaking on the same day Vice President Joseph Biden's negotiating group on the debt and deficit was meeting with members of House and Senate that make up this group. When asked, Warner said he still believes the Five Guys group is relevant to the discussion.

"I believe it is," he said. "I still think the best chances for a long-term debt and deficit deal come with a bipartisan basis that starts in the Senate."

But Warner added, "I wish all the best to the Biden group, and if they get there first, go to it."

Warner did not discuss the possibility of expanding the Five Guys group, but did say the group would welcome back Tom Coburn (R-Okla.), who left the group in May.

He also downplayed the idea of another round of economic stimulus, which some have mentioned as a way to shore up disappointing economic data over the last few weeks that has led to renewed fears on Wall Street.

"In a perfect world, there would be need for additional stimulus," he said. "But we've kind of shot those bullets, and everything you do right now adds to that growing problem.

"Candidly, I think the best private stimulus we could have is putting in place a long-term debt and deficit deal that would get private capital off the sidelines," he added.