Bankers are concerned that lenders would probably stop offering balloon loans if they don't come under QM safe harbor rules.
"The consequences are that if they are not in the safe harbor that banks will stop lending, they will not take the risk," Haynie said.
The importance of those loans were made clear in a recent survey where 75 percent of ICBA's membership said they offer balloon loans and that they make up between 45 and 50 percent of their portfolios, he said.
CFPB officials said Tuesday on Capitol Hill that they have tweaked the rural classification requirements that would allow for balloon loans to continue.
Kelly Cochran, assistant director for regulations at the CFPB, told the House Financial Services Committee that the agency has expanded the definition of "rural and underserved" to 9 percent of the population from 3 percent under QM.
But lawmakers and banking industry groups argue the change is not broad enough to encompass the breadth of the nation's rural community, which relies primarily on smaller lenders.
Several lawmakers during the hearing asked the CFPB officials to reconsider that percentage.
Haynie said his group is also pushing back on the percentage, especially considering that the U.S. Census Bureau says 20 percent of the nation's population lives in rural areas.
But a properly crafted category for smaller lenders in the QM rule, which includes balloon loans, could offset that lower rural percentage and allow community banks to meet the needs in those areas, Haynie said.
Yet the balloon loan issue underscores the problems that small- and medium-sized banks and credit unions are facing with the overall structure of the ability-to-pay rule, which they say will restrict lending.
Credit unions argue that they have maintained strict lending standards and have always taken into account a consumer's ability to repay.
The new requirements — including the a 43 percent debt-to-income ratio and points and fees rules — will make credit unions think twice about taking on the additional risk, Carrie Hunt, vice president of regulatory affairs and general counsel, of the National Association of Federal Credit Unions (NAFCU), told The Hill.
The ability-to-pay rule, which is designed to put the mortgage market back on firmer footing, would require lenders to look at a borrower's income, debt and assets when determining if they qualify for a loan. The rule has three completed categories and the CFPB is still working out the details of the QM rule that pertains to smaller lenders.
"We hope to issue it shortly, we want to get it out," Cochran said.
During the past several months — the QM rule was released in January — Cochran said that the agency has worked "very hard to structure a rule to accommodate and recognize small community banks and the critical access to credit they possess."
"We want to encourage that type of lending within scope of rule," she said.
Still, there are concerns that the agency won't craft the language to meet the industry's needs.
Hunt doesn't think the it will go "far enough" to address their concerns and while it could tamp them down, it is unlikely to provide the relief from the additional regulations that credit unions want. Credit unions have asked for an exemption from the QM rules.
Haynie agreed and said that while his group is encouraged that federal regulators are listening, he hopes that the final rule reflects a solution to those concerns.
Lawmakers also joined the chorus of those worrying that the rules will further constrain already tight lending in a gradually improving housing market and could, in turn, stymie the recovery.
Cochran acknowledged that the new ability-to-pay rule will affect business decisions in the short run but that, eventually, lenders will become more comfortable with the new regulations and will begin working outside the QM requirements.
Several Republican lawmakers argued that there is evidence that only about half of all the loans made in 2010 would fall under the QM rule and the majority are performing well, making it clear that lending could come to a halt once the rule is implemented in January.
Peter Carroll, assistant director for mortgage markets at the CFPB, said that according to a Federal Reserve study, the bulk of the QM rule covers 75 percent of mortgages made in 2011 and there are efforts to push coverage up to between 90 and 100 percent.
Rep. Gary Miller (R-Calif.) argued that, as currently written, the QM rule is unworkable and there is plenty of "bipartisan unhappiness" because it is more restrictive than expected, raising concerns that banks won't make loans outside of federal rules.
Miller urged them to modify the rule now and warned that it would not be good to "force us to legislatively change the rule."