The FBI will use criminal mug shots to track suspects with facial recognition but won’t be collecting pictures of innocent civilians, Director James Comey said on Wednesday.

“We’re piloting the use of mug shots, along with our fingerprint database, to see if we can find bad guys by matching pictures with mug shots,” he said in a House Judiciary Committee hearing.

{mosads}Despite concerns that tens of millions of innocent people’s images would be included in the government’s database of facial images, Comey said the effort will just be focused on pictures taken while criminals were in custody.

“The explanation to me was the pilot [program] is mug shots because those are repeatable, we can count on the equality of them, and they are tied to criminal conduct, obviously,” the FBI chief said. “There was not a plan and there isn’t at present to add other, non-mug-shot photos.”

Earlier this year, the digital civil liberties group Electronic Frontier Foundation (EFF) received records under the Freedom of Information Act showing that the agency planned to have as many as 52 million pictures in its Next Generation Identification database by next year.

The group said that database could include photos of people who had never been charged with a crime but have to submit photos as part of a background check at their job.

“This means that, even if you have never been arrested for a crime, if your employer requires you to submit a photo as part of your background check, your face image could be searched — and you could be implicated as a criminal suspect — just by virtue of having that image in the non-criminal file,” EFF senior staff attorney Jennifer Lynch wrote in a blog post at the time. 

Comey said that’s not true. Though the FBI might get pictures of people in some sensitive jobs like drivers of school children, they would not be included in its Next Generation Identification system, and the agency also will not be mining local drivers’ license storehouses, he said.