Releasing data from stress tests could undermine the beginnings of a financial sector recovery, Senate Budget Committee Ranking Member Judd Gregg (R-N.H.) declared Monday.

"I think we don't want to put a lot of negatives out there in the middle of what appears to be a tentative recovery in the financial recovery by releasing these stress tests," Gregg said this morning on CNBC.

Gregg said the Obama administration's promise to release the data on financial institutions' health was a "mistake," and ignores the fact that debt could shift quickly on banks' balance sheets as the economy continues to recover.

Some economists have worried that by releasing stress test data, banks which are perceived to be recovering could suffer tremendous market repercussions if the test results are negative, thereby exacerbating their problems.

Gregg said the troubled financial institutions should also be free to repay their government loans when they want, and should not be subject to repayment restrictions by the Treasury.

"If a bank decides to pay us back, the risk is that if they make a mistake, the leadership of that bank is going to be under severe stress from their stockholders and lawsuits by their stockholders," Gregg said. "Let's leave it up to the bank leadership: if they want to pay the government back, if they want to pay the taxpayers back their money, let them do it. Let's not have some official down at Treasury deciding if it's a good idea."

Watch a video of the Gregg interview below: