WATCH: Perry says he’s had ‘no conversation’ about taking over VA

Energy Secretary Rick PerryJames (Rick) Richard PerryOvernight Defense: Trump hails D-Day veterans in Normandy | Trump, Macron downplay rift on Iran | Trump mourns West Point cadet's death in accident | Pentagon closes review of deadly Niger ambush Trump hails D-Day veterans in Normandy: 'You are the pride of our nation' Trump hails D-Day veterans in Normandy: 'You are the pride of our nation' MORE shot down talk that he could move to the Department of Veteran Affairs (VA) if President Trump fires current VA Secretary David ShulkinDavid Jonathon ShulkinTrump sent policy pitch from Mar-a-Lago member to VA secretary: report Is a presidential appointment worth the risk? It’s time to end the scare tactics and get to work for our veterans MORE

Asked if he has talked to the White House about making the move, Perry told reporters Wednesday: “Not offered, no conversation about it." 

Following President Trump's abrupt firing of Secretary of State Rex Tillerson on Tuesday and his comments that he almost has the Cabinet that he wants in place, speculation has swirled on Capitol Hill over which administration official may be replaced next. 

Multiple news outlets, led by The New York Times, reported Tuesday that President Trump was considering firing Shulkin and moving Perry into the job.

Lawmakers tell The Hill privately that the president seems displeased with Shulkin's performance and he may be the next official to go.