This week marks five years since President Obama's executive order to close the detention center at Guantanamo Bay. Signed on his second day in office, the order reflected broad consensus that the facility should be shuttered for good. Both President Obama and Sen. John McCainJohn Sidney McCainThe Memo: Trump's strengths complicate election picture Mark Kelly: Arizona Senate race winner should be sworn in 'promptly' Cindy McCain: Trump allegedly calling war dead 'losers' was 'pretty much' last straw before Biden endorsement MORE (Ariz.), the Republican presidential nominee, supported the goal. Former President George W. Bush, Secretary of State Colin Powell, and others were on board. It seemed a foregone conclusion that the executive order would be realized. With 155 prisoners still at the island prison, what is the path forward?

Progress has been made on the path to closing Guantanamo, albeit much more slowly than anyone would have hoped. After years of legislating restrictions on transfers abroad and to the U.S., Congress reversed course in December when it passed a defense bill that clarified the foreign transfer rules for detainees, a near majority of whom have been cleared for transfer by our intelligence and defense agencies. Now that Congress has given the administration additional flexibility, the ball is in the president's court. And Obama has demonstrated renewed commitment to closing Guantanamo, from his speech at the National Defense University last May, to the appointment of two new special envoys in the State and Defense Departments charged with closing Guantanamo, to actually transferring detainees.

In the past year, the Obama administration has transferred 11 detainees, including three Uighurs, who, by all accounts, were simply in the wrong place at the wrong time. But if Obama wants to empty Guantanamo before he leaves office, he will need to carry out transfers at a much quicker pace. There are 77 detainees who are currently cleared for transfer; the administration must invest the time and resources to finalize security assurances with all host nations, and make the transfers. Some transfers should be relatively easy, like those for the five Tunisians now that their nation moves ever closer to democracy, or Shaker Aamer who is a resident of the United Kingdom and has a wife and kids awaiting him. Both the U.K. and Tunisian governments have asked for their countrymen back.

Other transfers will be more difficult, but can still be achieved. The U.S. will likely need to resettle the four Syrians cleared for transfer because of unrest in their nation. The United States should also start chipping away at the number of Yemeni detainees who account for 56 of the 77 detainees cleared for transfer. A significant number of Yemeni detainees were scheduled for release before the underwear bomber incident led President Obama to impose a moratorium on transfers back to Yemen. He has now lifted that ban, and the United States has an important security partner in President Hadi, whose government should be able to provide even better security assurances than the prior regime. It is time to start moving the Yemeni detainees.

The Defense Department should also step up the pace of the new Periodic Review Boards tasked with reviewing the cases of nearly 70 other detainees to determine if they are now eligible to be cleared for transfer. The first review in December cleared for transfer a Yemeni detainee who had been held for more than a decade without charge or trial. Doing one review board a month, though, will hardly dent the backlog. The administration should commit to completing all Periodic Review Boards this year.

Finally, Congress should take the next steps and lift the ban on transfers to the U.S. for prosecution, incarceration and emergency medical treatment. For some detainees who have been charged with crimes, Congress must face the prospect that they can either be sent home or sent to the United States for trial, because two federal appellate courts have ruled that the charges pending against them at Guantanamo were not internationally recognized war crimes at the time of offense, and therefore cannot be pursued in the legally problematic military commissions.

Earlier this week, a group of 31 retired generals and admirals, many of whom stood behind Obama when he signed the executive orders in 2009, urged the president to the complete the job. In a letter to Obama, they noted, " As long as it remains open, Guantanamo will undermine America’s security and status as a nation where human rights and the rule of law matter."

The consensus to close Guantanamo is again gathering momentum. It's time.

Osburn is director of Human Rights First's Law and Security Program.