The government might not want to rush too much to divest of its investments in financial institutions, Sen. Mark WarnerMark Robert WarnerHillicon Valley: Federal inquiry opened into Google health data deal | Facebook reports millions of post takedowns | Microsoft shakes up privacy debate | Disney plus tops 10M sign-ups in first day Microsoft embraces California law, shaking up privacy debate Google sparks new privacy fears over health care data MORE (D-Va.) suggested Thursday.

While Warner urged the government to recover money from its Troubled Asset Relief Program (TARP) bailouts as quickly as reasonably possible, he encouraged a "good business decision" that could have taxpayers see a return on their investments.

"I think that we ought to make a good business decision. I want to make sure that the government gets out of these institutions as reasonably possible," Warner told Bloomberg News today. "But I also want to make sure that we who have taken the risk through these challenging times -- we, the taxpayers -- get some potential."

Warner said that Treasury Secretary Tim Geithner should make sure the banks don't pay back their loans too quickly.

"I think it's fully appropriate for the Treasury to evaluate before the banks are able to repay to make sure the banks they're not repaying too quickly if they're not fully out of the woods," he said.