Enhancing surveillance apps with vision-based AI

We're seeing both a tantalizing potential and an underwhelming reality in the current automated surveillance technology. Consider, for example, the terrorist bombing at the finish line of the April 15, 2013 Boston Marathon. Bolyston Street was full of cameras—those permanently installed by law enforcement organisations and businesses and those carried by race spectators and participants. But none of them was able to detect the impending threat—represented by the intentionally abandoned backpacks, each containing a pressure cooker-implemented bomb—with sufficient advance notice to prevent the tragedy. Also, due to both the slow speed and low accuracy of the alternative computer-based image analysis algorithms, the flood of video footage was predominantly analysed by the eyes of the police department and FBI representatives attempting to identify and locate the perpetrators.

Consider, too, the military presence in Afghanistan and elsewhere, as well as the ongoing threat to U.S. embassies and other facilities around the world. Only a limited number of human surveillance personnel are available to look out for terrorist activities such as the installation of IEDs (improvised explosive devices) and other ordinances, the congregation and movement of enemy forces, and the like. These human surveillance assets are further hampered by fundamental human shortcomings such as distraction and fatigue.

Computers, on the other hand, don't get sidetracked, and they don't need sleep. More generally, an abundance of ongoing case studies, domestic and international alike, provide ideal opportunities to harness the analysis assistance that computer vision processing can deliver.

For example, automated analytics algorithms are able to sift through an abundance of security camera footage in order to pinpoint an object left at a scene and containing an explosive device, cash, contraband, or other contents of interest to investigators. After capturing facial features and other details of the person(s) who left the object, analytics algorithms can also index image databases both public (Facebook, Google Image Search, etc.) and private (CIA, FBI, etc.) in order to rapidly identify the suspect(s).

Unfortunately, left-object, facial recognition, and other related technologies haven't historically been sufficiently mature to be relied upon with high confidence, especially in non-ideal usage settings, such as when individuals aren't looking directly at the lens or are obscured by shadows, or other challenging lighting conditions. As a result, human eyes and brains were traditionally relied upon for video analysis instead of computer code, thereby delaying suspect identification and pursuit, as well as the possibility of error (false positives, missed opportunities, etc). Such automated surveillance technology shortcomings are rapidly being surmounted, however, as cameras (and the image sensors contained within them) become more feature rich, as the processors analysing the video outputs increase in performance, and as the associated software becomes more robust.

As these and other key system building blocks such as memory devices decrease in cost and power consumption, opportunities for surveillance applications are rapidly expanding beyond traditional law enforcement into new markets such as business analytics and consumer-tailored surveillance systems, as well as smart building and smart city initiatives. To facilitate these trends, an alliance of hardware and software component suppliers, product manufacturers, and system integrators has emerged to accelerate the availability and adoption of intelligent surveillance systems and other embedded vision processing opportunities.

How do artificial intelligence and embedded vision processing intersect? Answering this question begins with a few definitions. Computer vision is a broad, interdisciplinary field that attempts to extract useful information from visual inputs by analysing images and other raw sensor data.