Sen. Casey: Specter emphasized bipartisanship, humor

"I think you can still establish personal relationships so that you can work together," Casey said. "But we've got a long way to go. We need a lot more of that bipartisanship, but he worked at it his whole life."

Specter famously switched parties in 2009, from Republican to Democrat, after some 44 years as an elected official for the GOP. Casey said that decision took a toll on Specter, who counted many Republicans among his best friends.

"That was a tough decision," Casey said. "I knew he was doing it because he was thinking how he could best serve Pennsylvania."

Casey also said he would miss Specter's sense of humor; the senator would go on to pursue stand-up comedy after leaving the Senate in 2011.

"He had a great sense of humor," Casey said. "Sometimes someone who's that capable and that serious and that determined, you can forget the fact that they have a sense of humor, and he did. Not only as he exhibited as a stand-up comic, but also just in the interactions we would have. He would tell stories about campaigns and politics, and he was a delight to be around, especially on the floor when you could get him a couple of minutes to have him tell a war story or two."