Attorney General Eric HolderEric Himpton HolderHouse easily passes prison reform bill backed by Trump On Trump and DOJ, both liberals and conservatives are missing the point Holder: DOJ, FBI should reject Trump's requests MORE said he was careful not to allude to a racial component when criticizing Congress’s treatment of him last week. 

"I didn't say there was a racial component. I was very careful not to say that," Holder told The Huffington Post on Monday.  

Last week, Holder told a civil rights group he and President Obama’s administration have been subject to "unprecedented" and "ugly" attacks. He asked what attorney general or president in history had to deal with the same.  

Instead of race, Holder said he was alluding to the "breakdown in civility" and the ability to cultivate relationships in Washington. 

Holder questioned whether the 1964 Civil Rights Act would be able to pass in today's climate. 

"If we had a Congress or an executive branch-legislative branch relationship in the way that we now have one, where there's this lack of civility, I wonder whether or not you could have forged the necessary compromises, things that involved personal relationships, in order to get such a landmark piece of legislation passed," he said. 

He added: "And that's essentially what I was decrying — the fact that we can't somehow separate whatever our personal feelings are and focus on our functions as members of the executive branch or as legislators. I think that I've done a pretty good job in doing that, but it's frustrating at times."

Holder’s remarks during a speech last week came a day after testifying before a House panel where he faced a number of testy exchanges with Republican lawmakers, including Rep. Louie GohmertLouis (Louie) Buller GohmertHouse conservatives introduce resolution calling for second special counsel White House-backed prison reform bill advances in House GOP Rep. Zeldin to lead call for second special counsel MORE (R-Texas).

"Forget about me, you look at the way the attorney general of the United States was treated yesterday by a House committee — it has nothing to do with me, forget that," Holder said during unscripted remarks before the National Action Network convention. "What attorney general has ever had to deal with that kind of treatment? What president has ever had to deal with that kind of treatment?"

Gohmert was also quick to point out last week that Holder "didn't say race, and he was very careful about that." The Texas congressman, however, maintained past GOP attorneys general have faced much harsher criticism. 

During the interview Monday, Holder said he is “still not quite sure I understand” comments Gohmert made last year in reference to casting “aspersions on my asparagus.”

Nonetheless, it “sort of stuck” in Holder’s mind and he mocked Gohmert with the line during their exchange last week.