What happens when digital eyes get the brains to match?
We usually think of surveillance cameras as digital eyes, watching over us or watching out for us, depending on your view. But really, they’re more like portholes: useful only when someone is looking through them. Sometimes that means a human watching live footage, usually from multiple video feeds. Most surveillance cameras are passive, however. They’re there as a deterrence, or to provide evidence if something goes wrong. Your car got stolen? Check the CCTV.
But this is changing — and fast. Artificial intelligence is giving surveillance cameras digital brains to match their eyes, letting them analyze live video with no humans necessary. This could be good news for public safety, helping police and first responders more easily spot crimes and accidents and have a range of scientific and industrial applications. But it also raises serious questions about the future of privacy and poses novel risks to social justice.
What happens when governments can track huge numbers of people using CCTV? When police can digitally tail you around a city just by uploading your mugshot into a database? Or when a biased algorithm is running on the cameras in your local mall, pinging the cops because it doesn’t like the look of a particular group of teens?
These scenarios are still a way off, but we’re already seeing the first fruits of combining artificial intelligence with surveillance. IC Realtime is one example. Its flagship product, unveiled last December, was billed as Google for CCTV. It’s an app and web platform named Ella that uses AI to analyze what’s happening in video feeds and make it instantly searchable. Ella can recognize hundreds of thousands of natural language queries, letting users search footage to find clips showing specific animals, people wearing clothes of a certain color, or even individual car makes and models.
In a web demo, IC Realtime CEO Matt Sailor showed The Verge a version of Ella hooked up to around 40 cameras surveilling an industrial park. He typed in various searches — “a man wearing red,” “UPS vans,” “police cars” — all of which brought up relevant footage in a few seconds. He then narrowed the results by time period and location and pointed out how users can give thumbs-up or thumbs-down to clips to improve the results — just like Netflix.