Learning Patterns of Activity Using Real-Time Tracking

The authors aim to build a visual monitoring system that passively observes moving objects in a site and learns patterns of activity from those observations, noting that extended sites require multiple cameras and thus motion tracking, camera coordination, activity classification, and event detection. This paper concentrates on motion tracking. Motion segmentation is based on an adaptive background subtraction method that models each pixel as a mixture of Gaussians and uses an online approximation to update the model, evaluating which Gaussian distributions most likely arise from the background process. The result is a stable, real-time outdoor tracker that reliably handles lighting changes, repetitive motions from clutter, and long-term scene changes.

Although the tracker does not know the identity of any object, that identity stays consistent for an entire tracking sequence, and the system exploits this by accumulating joint co-occurrences of representations within a sequence. These joint co-occurrence statistics are then used to create a hierarchical binary-tree classification of the representations, which is useful for classifying whole sequences as well as individual instances of activities in a site. The mixture-of-Gaussians background model became a widely adopted foundation for real-time surveillance and tracking.