When Janet Yellen spoke on Friday at the Fed's symposium at Jackson Hole, Wyo. her remarks were quite frank. The Fed chair focused on the weak labor market and sluggish pace of growth.  But in my opinion, she was not frank enough.

Let's face it: most of us are scared. Wealthy investors watched their savings disintegrate six years ago, in a destruction of capital spurred by a severe economic downturn. Then, over the next few years, these same individuals saw their fortunes grow once again, even surpassing prior levels, thanks to the U.S. Federal Reserve’s easy money policy. 


During this process though, retail investors have been hammered. With little to no wage growth, high unemployment, and declining disposable income, it shouldn't come as any surprise that most Americans don't believe the economy has truly recovered.  To a great extent, policymakers share that belief. Here are five things that a central banker will never say, but actually should:

“The Great Depression still haunts us.” History teaches us that the world did everything wrong during that dreadful period. As a result of these painful memories, we are dead set on never making those same mistakes again.

Human beings live in fear (this emotion is more powerful than greed). This near constant worry means that, at times, it’s necessary for us to try to provide some level of security and order to markets.  Regulators, therefore, had to step in before irrational fear crushed the spirit of enterprise.  If markets are in a state of free fall, mom and pop will stop investing and dreaming of their future. Yet, the Fed’s assistance has continued longer than originally planned because of a slower-than-expected economic recovery.

“This long process has caused some collateral damage and some fraying of society.”  Since interest rates are the physical gravity of assets, it would make sense that the lower they go, the higher the assets will be worth.

This inverse relationship is great for the asset holders, but has little to no positive effects on the average American who has no such capital. Yet, pushing down rates was the only such tool at the Fed’s disposal and it was one they needed to deploy aggressively. 

While the byproduct has been massive wealth creation, the average investor is losing ground. We have a failed K-12 public education system, poor social, tax and budget policies emanating from Washington, and a growing wealth disparity - not only here in the United States but around the world.

Meanwhile, “the specter of Deflation is upon us.”  When the leverage balloon popped, it caused a disappearance of money. At the same time, the world is experiencing a surge in goods and services. In our world, inflation is tolerable as long as it doesn't get out of control. Deflation, however, is the true enemy of a modern economy.

We have to fight this decline in prices with all of the means at our disposal because such a period also means that money becomes more valuable. This is a disaster for societies that are debt laden.  Let’s say the average American earns $50,000 a year and carries $350,000 of debt. Deflation causes not only his or her salary to be cut in half, but also the prices for goods and services. Their debt is, in turn, doubled.

If this occurs, the poor and middle class will suffer the most. We are so worried about this possibility that we will accept all sorts of consumer based inflation (rises in food and energy prices) in an effort to stave off deflation.

“Our tools are quite limited to prevent this potential outcome and Congress and the president have stood by silently as we try to do all of the work.”:This is an indictment of both parties. Without proper action -- tax and entitlement reform and an improved budgetary process -- the economy is going to stagnate.

Corporate executives fear the future and are concerned about the lack of forward planning from the government. Without certain tax and healthcare liabilities, companies are hoarding cash, limiting their ability to make new investments. There has to be coordinated action between fiscal and monetary policy to really get things moving again and restore confidence.

"There is a way out of this mess but we can't tell you the playbook as it isn't our job."  The way out is fairly straightforward but is going to require a public private partnership on three fronts:

•          Infrastructure:  We need a private public infrastructure bank to help finance the rebuilding of roads, airports, the power grid and other essential services that a modern society takes for granted. Think of it as the mortgage you have on your home. It’s helping you afford your house today and as you decrease the debt, you increase your wealth.

•          Compensation and Asset Growth:  We don't like to share and when people climb to the top, they prefer to play the game King of Mountain as opposed to something more collective.  If private enterprises fail to come up with strategies to close the wealth divide, then guess what, the government could, through implementing bad policies, make things even worse. The top earners and thinkers need to spend more time  developing private sector opportunities for the less fortunate.

•          Education:  This is the toughest one and there  is no one-size-fits-all solution here, but educators know that the basics of education start in the home. We can pick and choose between public, private, and charter schools or tenured and non-tenured teachers, but without proper coordinated parent/teacher partnerships, children are at a disadvantage. Most of us who watched our uneducated grandparents work with their hands, know the benefits of a great education. If we need to "reset" anything in our society, it’s the promise to children that we can lay the framework of a solid education at their feet.

The Fed can't fix all our economic issues alone. Lawmakers need to stop acting like babies and start taking the steps necessary, along with some help from the private sector, to prevent deflation, re-establish trust and make America great again.

Scaramucci is founder and co-managing partner of SkyBridge Capital, a global asset management firm with about $11.2 billion in assets under management and advisement as of June 31.