Rep. Allen West (R-Fla.) said Wednesday he would advise Herman Cain to let voters “move on” from the distraction of his campaign.

Cain told his campaign this week it is time to “reassess” in light of the resurfacing of old sexual harassment allegations and new accusations this week that he carried on a 13-year affair.

"Beyond reassessing his campaign, he probably needs to understand that he is a distracter for what's going on right now and we should move on," West told WMAL’s radio show “Morning Majority.”

West, a Tea Party favorite and the only Republican in the Congressional Black Caucus, suggested Cain is already out of the running and the GOP nomination field has come down to “a two-man race.”

He named Mitt Romney and Newt Gingrich, currently the front-runners in the polls, as the two most likely contenders. West said Romney, who has polled at around 24 percent nationally for months, has “hit a plateau” and called Gingrich, who came up from behind to become a front-runner within the last few weeks, “very viable” as a candidate.

“Speaker Gingrich is the next in line for conservatives who are looking for someone who is not Mitt Romney,” he said. "But he does have that 200-pound rucksack on his back with a lot of the personal baggage that people are going to come at him about.”

Gingrich reportedly mentioned West, along with Sen. Marco RubioMarco Antonio RubioOvernight Cybersecurity: What we learned from Carter Page's House Intel testimony | House to mark up foreign intel reform law | FBI can't access Texas shooter's phone | Sessions to testify at hearing amid Russia scrutiny Cornyn: Senate GOP tax plan to be released Thursday This week: GOP seeks to advance tax overhaul MORE (R-Fla.), earlier this week as possible vice presidential candidates on his hypothetical ticket.

West said he would have to pray about and discuss the opportunity with his family, but he promised: “If I can make a difference, then I'll answer that call.”