SentiVeillance SDK

Face/Object identification for video surveillance applications

SentiVeillance SDK is designed for developing software that performs biometric face identification and detects moving pedestrians or vehicles or other objects using live video streams from high-resolution digital surveillance cameras.
The SDK is used for passive identification – when passers-by do not make any efforts to be recognized. List of possible uses includes law enforcement, security, attendance control, visitor counting, traffic monitoring and other commercial applications.

• Real time face detection, template extraction and matching against watchlist database.
• Simultaneous tracking of multiple faces or objects in live video.
• Advanced moving objects detection and classification for pedestrians and vehicle tracking.
• Gender classification, age evaluation, detection of face expression, glasses and facial hair.
• Automatic operation logs and reports events, as well as enrolls new faces from video stream and add them to watchlist automatically.
• Large surveillance systems support by connecting up to 10 cameras to a single computer and quick synchronization between networked computers.
• Available as multiplatform SDK that supports multiple programming languages.
• Ready-to-use server for integration into video management systems (VMS) optionally available.
• Reasonable prices, flexible licensing and free customer support.

Watch the SentiVeillance (Current Version) biometric face recognition technology demonstration video on YouTube

SDK Contents

SentiVeillance (Current Version) SDK is based on the SentiVeillance (Current Version) technology that is specially designed for integrating biometric facial recognition into video surveillance systems. Face templates created with SentiVeillance SDK are fully compatible with VeriLook SDK and MegaMatcher SDK multi-biometric technology.

SentiVeillance (Current Version) SDK includes Device Manager library for Microsoft Windows and Linux that allows to perform simultaneous capture from multiple cameras.

SentiVeillance 2-camera component

The SentiVeillance 2-camera component is designed for using in small-scale video surveillance systems with several cameras. The component performs real-time detection and tracking of all detected faces from up to 2 live video streams. The component installations can be run on several computers distributed over network and synchronize data between themselves.

The SentiVeillance component is able to track multiple faces simultaneously and match them with faces from internal database (i.e. a watch-list of suspects or a list of company employees). New faces may be enrolled to the database either automatically or manually from image files. Person’s gender can be optionally determined for each person in a frame

The C# and VB .NET samples from the SDK shows how to use the component.

One SentiVeillance 2-camera component license is included with SentiVeillance (Current Version) SDK. More licenses for this component can be purchased any time by SentiVeillance (Current Version) SDK customers.

SentiVeillance 10-camera component

The SentiVeillance 10-camera component is designed for using in large surveillance systems with multiple cameras and has the same functionality as the SentiVeillance 2-camera component. It supports input from up to 10 live video streams on the same PC or server. The component installations can be run on several computers distributed over network and synchronize data between themselves.

The SentiVeillance 10-camera component licenses can be purchased any time by SentiVeillance (Current Version) SDK customers.

Face Extractor component

The Face Extractor creates face templates from face images. The component is intended for enrolling faces from still images into a surveillance system and provides its functionality for reasonable price.

The component extracts a single face template in 1.34 seconds. The specifi ed performance requires a PC or laptop with at least Intel Core 2 Q9400 (2.67 GHz) processor.

Face Extractor can generalize a face template from several images that include the same face to improve the template’s quality.

One Face Extractor license is included with SentiVeillance (Current Version) SDK. More licenses for this component can be purchased any time by SentiVeillance (Current Version) SDK customers.

Device Manager library

Device Manager library works under Microsoft Windows and Linux and provides functionality for simultaneous capture from multiple cameras. The library supports a range of high-resolution digital surveillance cameras and other cameras that provide DirectShow interface for Windows platform, or GStreamer interface for Linux platform.

The list of supported cameras is available below in the “System Requirements” section.

The Device Manager includes a plug-in framework that allows integrators to write plug-ins to support their cameras using the provided API. The SentiVeillance SDK documentation contains the detailed information and samples.

A video file can be also used as a data source for SentiVeillance. The input from the file is processed as coming from a virtual camera, thus the video is processed in real-time.

SentiVeillance (Current Version) Technology Capabilities

• Real time performance. SentiVeillance technology performs face, pedestrian or object recognition and tracking in real time. The technology is designed to run on multi-core processors to achieve fast performance.
• Two algorithms for surveillance systems. Depending on the surveillance system design, one of these algorithms may be used:
  1. Biometric face recognition algorithm is based on deep neural networks and provides these capabilities for surveillance systems:
  • Multiple face detection, features extraction and template matching with the internal database in real time.
  • Facial identification reliability enables using large watchlist databases
  • Face tracking is performed in all successive frames from the video source until they disappear from camera field of view. The face tracking algorithm uses dynamic face and motion prediction models that make it robust to occlusions like other objects or even other faces. The algorithm is able to continue tracking a face even when it re-appears after being fully covered by occlusions (like walls, furniture, posters etc).
  • Gender classification (optional) for each person in the frame.
  • Age determination (optional) for each person in the frame.
  • Smile, open-mouth, closed-eyes, glasses, dark-glasses, beard and mustache attributes detection (configurable).
• Motion detection and tracking algorithm performs advanced detection of moving objects in the scene, their classification and tracking until they disappear. These features are available for surveillance systems:
  • Object classification. After calibration SentiVeillance allows to perform object classification based on the size and movement speed. For example, users can configure a surveillance system to determine if a tracked object is a vehicle, a single pedestrian or group of pedestrians. See video below.
  • Restricted areas control. SentiVeillance algorithm can detect and report if people or objects enter, leave or stay in restricted areas. The events are triggered when people or objects cross pre-defined lines or enter polygon-shaped areas.
  • Tolerance to weather conditions. The algorithm ignores rain and snow, as well as trees and bushes, which are swayed by wind.
• Automatic operation. A system based on SentiVeillance (Current Version) SDK is able to log face, pedestrian or object appearance, disappearance and tracking. The detected faces are matched against the watchlist in the internal database and recognized faces are immediately reported to the system. The system uses face tracking for automatic enrollment from video stream and adding new facial templates to watch list on the fly.
• Large surveillance systems support. SentiVeillance (Current Version) SDK allows to integrate its technology into surveillance systems with multiple cameras and multiple data-processing nodes. A single computer can process video data from up to 10 cameras simultaneously. Multiple computers with running SentiVeillance software can quickly synchronize biometric and surveillance data between each other over the network. The synchronization can be customized as the SDK includes sample source code for using the communication and synchronization processes.
• Video files processing. SentiVeillance also accepts data from video files. The video files are processed in real time as coming from a virtual camera, therefore an hour-long video will be processed in one hour.

The list of supported cameras is available below in the “System Requirements” section.

The Device Manager includes a plug-in framework that allows integrators to write plug-ins to support their cameras using the provided API. The SentiVeillance SDK documentation contains the detailed information and samples.

A video file can be also used as a data source for SentiVeillance. The input from the file is processed as coming from a virtual camera, thus the video is processed in real-time.

Watch the vehicle and pedestrians tracking video on YouTube

Watch the pedestrians and restricted zone tracking video on YouTube

Tech Specs

The specifications are provided for the default values of the parameters.

640 x 480 pixels is the recommended minimal frame size for faces' detection. Face template extraction and matching with watch list database speeds are not dependent on the frame size.

32 pixels is the minimal distance between eyes for a face on video stream or image to perform reliable face tracking and template extraction.

40 x 40 pixels is the minimal moving object size for its detection and tracking.

Face, pedestrians or moving objects tracking performance is dependent on actual size of a face or an object in a frame, not on the size of the whole frame.

SentiVeillance has certain tolerance to face posture that assures face detection and tracking:

• head roll (tilt) – ±180 degrees (configurable);
  ±15 degrees default value is the fastest setting which is usually sufficient for most near-frontal face images.
• head pitch (nod) – ±15 degrees from frontal position.
• head yaw (bobble) – ±45 degrees from frontal position (configurable).
  ±15 degrees default value is the fastest setting which is usually sufficient for most near-frontal face images.

See also the whole list of recommendations and constraints for SentiVeillance usage.

7 kilobytes facial template size assures best accuracy and good performance of the facial recognition algorithm. 4 and 5 kilobytes template sizes are also available.

At least 2 processor cores are required to process surveillance data from one camera with moderate number of faces in a frame. A PC, which has a processor with 4 cores, can be used to process data from 2 cameras almost without performance decrease. If large number of faces in a frame is expected, data processing will require to utilize more processor cores or to use more powerful processor.

The performance specifications are provided for Intel Core i7-4771 processor, running at 3.5 GHz clock rate, and 1920 x 1080 pixels videos.