President Obama strayed from his prepared remarks at a Labor Day rally to accuse his opponents of talking about him "like a dog."

Obama was in Milwaukee to speak at the Milwaukee Laborfest, attended by Labor Secretary Hilda Solis, Transportation Secretary Ray LaHood and AFL-CIO President Richard Trumka. He used the speech to promote his $50 billion infrastructure plan and to criticize Republicans ahead of crucial midterm elections.

"Some powerful interests who had been dominating the agenda in Washington for a very long time and they're not always happy with me. They talk about me like a dog. That's not in my prepared remarks, but it's true," he told the union crowd.

The president isn't known to stray off prepared remarks and also took a more aggressive tone in the speech.

"They’re betting that between now and November, you’ll come down with a case of amnesia," Obama said. "They think you’ll forget what their agenda did to this country. They think you’ll just believe that they’ve changed. These are the folks whose policies helped devastate our middle class and drive our economy into a ditch. And now they’re asking you for the keys back."