"The brands know this," Miller said. "That's why I believe that they bear the ultimate responsibility for the horrendously unsafe working conditions in Bangladesh and elsewhere.

"Corporate leaders in the fashion industry have a moral imperative to ensure that these tragedies do not happen again."

Miller said the recent collapse of an eight-story building that was the home of five garment factories has left more than 650 workers dead and hundreds more injured. He said retailers like Wal-Mart, J.C. Penney, Mango, Benetton, H&M, the Children's Place, Gap, and the Dress Barn need to sign the voluntary Bangladesh Fire and Building Safety Agreement to ensure these accidents don't happen again.

Under that agreement, garment factories would have to meet fire safety, electrical and other codes, and retailers would have to terminate production contracts with factories that fail to meet these standards. Miller said Calvin Klein, Tommy Hilfiger, Van Heusen and Izod have already signed on to this agreement, but said other U.S. brands need to do so as well.

"It is now time for the major U.S. corporations, like the Gap, Wal-Mart and J.C. Penney, to join them," he said.