The United States may not be nationalizing banks, but they are certainly taking control, Senate Banking Committee Ranking Member Richard ShelbyRichard Craig ShelbyOvernight Finance: Trump floats entering Pacific trade pact he once called 'a disaster' | Senators worry over Mulvaney's power at consumer bureau | Battle for CFPB control heads to appeals court | House fails to pass balanced budget amendment Senators gripe over Mulvaney's power at consumer bureau Overnight Finance: Trump thanks Xi for promises on trade | McConnell urges GOP to call Trump about tariffs | China files WTO case against tariffs | GOP dismisses report tax law will add .9T to debt | What to expect from Mulvaney hearing on CFPB MORE (R-Ala.) said Wednesday.

Shelby, appearing on "Morning Joe" ahead of more hearings on financial regulation, said that when the government takes a 40 percent stake in banks, as it's expected to do with Citibank, "it's not nationalization, but it's control."

Shelby argued that some banks, who are sometimes alleged to be "too large to fail," should be allowed to do just that.

"They need to close some of these banks," Shelby added. "Some banks were too large to exist, so you can't be too big to fail."

The Banking committeeman said the most important thing to do now is to reform the financial regulatory system, adding that the Federal Reserve fell short in its oversight role leading up to the financial crisis.