President Obama stopped by for an unscheduled dinner on Monday night at the home of Sam Kass, a White House chef and adviser to first lady Michelle Obama.
A White House pool report said his deputy chief of staff, Anita Decker Breckenridge, also joined him for the dinner.
No other details were provided.
White House officials have been mum on the reason for the return.
Earlier on Monday, Obama met with his national security team and then separately with Attorney General Eric Holder, two meetings he could have been conceivably done from Martha's Vineyard.
This isn't the first time Obama has spent a little down time with Kass. The White House chef is a frequent golfing partner. He is also engaged to MSNBC host Alex Wagner.