WATCH: Perry says he’s had ‘no conversation’ about taking over VA

Energy Secretary Rick PerryJames (Rick) Richard PerryThe enemy of my enemy is my friend — an alliance that may save the Middle East Celebrities, Rick Perry duped by viral Instagram hoax Instagram: No, old posts aren't being made public MORE shot down talk that he could move to the Department of Veteran Affairs (VA) if President Trump fires current VA Secretary David ShulkinDavid Jonathon ShulkinPress: Acosta, latest to walk the plank Senior Trump administration official to leave post next week Trump sent policy pitch from Mar-a-Lago member to VA secretary: report MORE

Asked if he has talked to the White House about making the move, Perry told reporters Wednesday: “Not offered, no conversation about it." 

Following President Trump's abrupt firing of Secretary of State Rex Tillerson on Tuesday and his comments that he almost has the Cabinet that he wants in place, speculation has swirled on Capitol Hill over which administration official may be replaced next. 

Multiple news outlets, led by The New York Times, reported Tuesday that President Trump was considering firing Shulkin and moving Perry into the job.

Lawmakers tell The Hill privately that the president seems displeased with Shulkin's performance and he may be the next official to go.