Blackwell, who endorsed Rick Perry in October but is friends with Cain, added his voice to that of Jon Huntsman and Rep. Allen West (R-Fla.) who have called Cain's controversy a "distraction" to the more serious issues of the presidential race.

"He deserves his day in the court of public opinion. He should not admit guilt if there is no guilt, but he must forcefully answer this on a sustained and clear basis," Blackwell said, criticizing a lack of "clarity of message" from Cain's campaign on the allegations of sexual harassment and a 13 year affair that have beset Cain in the past weeks.

Cain is facing a "pattern" of allegations, Blackwell said. "He cannot sidestep the issue, he cannot talk about conspiracy theories, he cannot play the victim card."

He said he was "encouraged" that Cain has said he will "reassess" his campaign by discussing the ongoing controversy with his family and supporters.

Blackwell, a former Ohio secretary of state who ran against Michael Steele for RNC chairman, sits on the boards of directors for The Club for Growth, National Taxpayers Union, and the NRA. 

His call for Cain to get out of the race adds weight to indications that Cain might step down, although Cain has maintained he is not getting out of the race because he is not a "typical" politician and the allegations are false. He has begun to stumble in the polls and fundraising support.

Cain said Wednesday he will make a final decision within a week.