The government might not want to rush too much to divest of its investments in financial institutions, Sen. Mark WarnerMark Robert WarnerOvernight Cybersecurity: Equifax hit by earlier hack | What to know about Kaspersky controversy | Officials review EU-US privacy pact Overnight Tech: Equifax hit by earlier undisclosed hack | Facebook takes heat over Russian ads | Alt-right Twitter rival may lose domain Facebook under fire over Russian ads in election MORE (D-Va.) suggested Thursday.

While Warner urged the government to recover money from its Troubled Asset Relief Program (TARP) bailouts as quickly as reasonably possible, he encouraged a "good business decision" that could have taxpayers see a return on their investments.

"I think that we ought to make a good business decision. I want to make sure that the government gets out of these institutions as reasonably possible," Warner told Bloomberg News today. "But I also want to make sure that we who have taken the risk through these challenging times -- we, the taxpayers -- get some potential."

Warner said that Treasury Secretary Tim Geithner should make sure the banks don't pay back their loans too quickly.

"I think it's fully appropriate for the Treasury to evaluate before the banks are able to repay to make sure the banks they're not repaying too quickly if they're not fully out of the woods," he said.