Rush took off his suit jacket during a speech on the House floor Wednesday to reveal the hooded sweatshirt. Declared to be out of order, he was ejected from the floor. The presiding officer cited Rush's attire, which violated a House rule on wearing hats in the chamber.

Rush wore the hoodie to demonstrate his opposition to racial profiling, which he believes was involved in the shooting death last month of 17-year-old Trayvon Martin.

“Just because someone wears a hoodie does not make them a hoodlum,” Rush said in his speech.

Minority Leader Nancy Pelosi (D-Calif.) said this week that Rush showed “courage” in wearing the hoodie, but BoehnerJohn Andrew BoehnerFeehery: The next Republican wave is coming Rift widens between business groups and House GOP Juan Williams: Pelosi shows her power MORE said the rules were “evenly enforced.”

“I’ve asked members on both sides of the aisle to leave the floor myself,” Boehner said at a press conference on Thursday.

Asked about female members of Congress who occasionally wear hats on the House floor — also against the rules — Boehner said: “I’ve not witnessed that, and I think women members know that that’s in violation of the rules.”