The United States may not be nationalizing banks, but they are certainly taking control, Senate Banking Committee Ranking Member Richard ShelbyRichard Craig ShelbyOn The Money: Republicans expect Trump to pull controversial Fed nominee | Inside Judy Shelton's confirmation hearing | Trump extends emergency declaration at border Republicans expect Trump to withdraw controversial Fed nominee Pentagon transferring .8 billion to border wall MORE (R-Ala.) said Wednesday.

Shelby, appearing on "Morning Joe" ahead of more hearings on financial regulation, said that when the government takes a 40 percent stake in banks, as it's expected to do with Citibank, "it's not nationalization, but it's control."

Shelby argued that some banks, who are sometimes alleged to be "too large to fail," should be allowed to do just that.

"They need to close some of these banks," Shelby added. "Some banks were too large to exist, so you can't be too big to fail."

The Banking committeeman said the most important thing to do now is to reform the financial regulatory system, adding that the Federal Reserve fell short in its oversight role leading up to the financial crisis.