“The world is drowning in imagery,” says Shashi Kant.
“We’ve generated more information in the last 48 hours than we had in all of human history leading up to 2004,” continues the founder of Netra Systems. “We are generating so much information, and the fact is 90% of that is imagery and most of it lies unexploited and inaccessible. That is a fundamental problem.”
Netra Systems is a visual search and tracking company designed to clean up the way we access our overwhelmingly large quantity of video and image-based content. Not just in our Instagram, Flickr and Facebook, but Netra Systems is hoping to alleviate the strain in numerous other sectors including retail, healthcare and security.
“When you think about it, Google just exploits text,” said Kant, “but imagine if we mine imagery in the same way and make it accessible and searchable.”
Netra Systems apply both machine-vision technology and a search engine style indexing. The algorithm breaks down a video by frames and tracks the blobs, which are the distinct objects within the frames. From there, Netra Systems identify the visual attributes of the blobs—should it be a vehicle, a cat, a human, etc.—it’ll note the colour, contour, texture, shape, etc.
Once the program recognizes the appearance characteristics of each object, it’ll give it a label in accordance to the most similar appearance. The “deep learning” will, for example, identify that within the image there is a blob that appears to be a cat, thus marking it as such. Then like Google, should a user search “cat” in the database, a list of plausible images will be presented in a search engine style result.
The capability of identifying imagery through an artificial neural network has a lot of potential for organization that focuses on little details. In recent scenarios as examples, Kant brought us back to the tragic events of the 2014 shooting at Parliament Hill, as well as the horrific incident involving the aftermath of the Boston Marathon bombing.
“One of our technology is what we call a multi-camera or cross-camera search,” said Kant. “In a retail store or in a surveillance camera, you typically only capture a small portion with a single view and there is no single camera that can cover the entire building. You need multiple cameras to follow the person and track from camera to camera. And that is the part where we really shine. And that is actually how we go and find the person.”
Organizations of all sizes invest a lot in cameras and machine vision, not only for security but also for visual data such as understanding high traffic areas within a store. By having a keen eye on a promotional item, retailers and suppliers can analyze customer’s engagement and propensity. The knowledge acquired will help merchants, managers and executives work together to make the best out of their retail space and the merchandise within.
When the topic of privacy implications arose, Kant responded with a question of his own: “Do you carry a cellphone? Between your cellphone and your credit card, if someone were to analyze that information they will know far more about you than looking at a camera and analyzing it.”
No doubt the visceral reaction to visual data is unlikely to change soon, however, we can clearly say that new technology and governance are being applied to keep us blobs safe when we are on the screen. Netra Systems offer a unique privacy protection that obfuscates images identified as humans in real-time. The original video will be recorded, saved, and retrieved only by those authorized, i.e. law officials.
Seeing the potential, investor, Mark Cuban signed on with Netra Systems in late 2013 after only 48 hours of email negotiation—no phone calls, no in-person meeting. “Machine vision is an area I have a lot of interest in,” Cuban wrote. “Its a big part of Motionloft.com and what we are doing there.”
When inquired about what excited him the most about Netra Systems, Cuban stated: “The ability to identify images with in a video from a connected database and their ability to apply artificial intelligence to video.”
“Plus the fact they are just damn smart,” he added.
Netra Systems is currently piloted and used by a number of major retailers and media and advertising agencies.