We Built an ‘Unbelievable’ Facial Recognition Machine For Cheap

Our facial recognition system detected Dr. Richard Madonna walking through Bryant Park. Damon Winter/The New York Times

Most people pass through some type of public space in their daily routine — sidewalks, roads, train stations. Thousands walk through Bryant Park every day. But we generally think that a detailed log of our location, and a list of the people we’re with, is private. Facial recognition, applied to the web of cameras that already exists in most cities, is a threat to that privacy.

To demonstrate how easy it is to track people without their knowledge, we collected public images of people who worked near Bryant Park (available on their employers’ websites, for the most part) and ran one day of footage through Amazon’s commercial facial recognition service. Our system detected 2,750 faces from a nine-hour period (not necessarily unique people, since a person could be captured in multiple frames). It returned several possible identifications, including one frame matched to a head shot of Richard Madonna, a professor at the SUNY College of Optometry, with an 89 percent similarity score. The total cost: about $60.

“My first reaction was, ‘Oh my god, that is unbelievable,’” Dr. Madonna said, after we reached him and explained the experiment. “I was shocked at how readily it seems that it picked me up, because, really — it’s the side of my head.”

In our exercise, we built a database using only photos from public websites, and we obtained Dr. Madonna’s consent before publishing this story. We’ve deleted the images and data that we collected and are no longer monitoring the Bryant Park cameras.

Over decades, businesses and individuals have installed millions of cameras like the ones we used, inadvertently setting up the infrastructure for mass surveillance. In the past, a human would have to watch the video feed to identify people, making it impossible to comprehensively record everyone’s movements. But the accuracy and speed of modern facial recognition technology means that building a dragnet surveillance system is now feasible.

The law has not caught up. In the United States, the use of facial recognition is almost wholly unregulated.

“The technology has advanced faster than even I thought that it would,” said Jennifer Lynch, surveillance litigation director at the Electronic Frontier Foundation. She said that because of how quickly the technology has advanced, she would now support a wholesale ban on government use of facial recognition.

New York City is nowhere near China, where the government has installed approximately one surveillance camera for every seven citizens. But according to the A.C.L.U., police here have access to more than 9,000 camera feeds in Lower Manhattan alone.

Details are sparse, but there is evidence that those capabilities are formidable. The Police Department claims its Domain Awareness System, developed jointly with Microsoft (which also offers facial recognition software), “utilizes the largest network of cameras, license plate readers, and radiological sensors in the world.”

It’s unclear whether the Domain Awareness System currently uses facial recognition, though the Police Department experimented with it in 2012, according to Clare Garvie, an associate at the Center on Privacy and Technology at Georgetown Law School. The police have been reluctant to divulge details, and the center has sued the department for more information.

Amazon is one of several companies that sell facial recognition services to the public. The company has highlighted positive applications of the service we used, Rekognition, such as its ability to help find lost children. It insists that it requires customers comply with the law and respect others’ rights, but has been criticized for pushing its technology to law enforcement agencies.

Rekognition is already actively used by the sheriff’s office in Washington County, Ore., including to investigate minor crimes like shoplifting. The Orlando, Fla., Police Department is also using the technology in a pilot program.

Amazon notes that its service makes predictions, not decisions, and that the confidence level the service provides should be incorporated in a human review process. The company recommends using a threshold of at least 99 percent for applications of its facial recognition service that involve identification or public safety, though critics of the technology say that the scoring is opaque and that the company has no way of enforcing that threshold. None of the matches we obtained from the Bryant Park footage, correct or incorrect, met the threshold.

Read the complete article on the New York Times here.


By Ted

I'm a former member of the Radio Television News Directors Association and during the last 30 years I've written news stories, sports stories, stories for children, puzzles, and plays for puppets.

Many of my ebooks are color picture books for children, and include printable black and white pictures at the end of a story. Please visit my blog for the latest information on my ebooks, and any updates or changes or comments. Ted.

PS: Please help stop cruelty to animals and support your favorite humane society or organization.