Spatial Analysis and Video Insights

Spatial Analysis and Video Insights are two distinct but complementary Azure services under the Computer Vision umbrella. Video Insights is primarily delivered via Azure Video Indexer (formerly Video Analyzer for Media), which extracts rich metadata from video files—including transcripts, faces, emotions, objects, and keyframes. Spatial Analysis, on the other hand, is a real-time video analysis capability that uses Azure Cognitive Services Computer Vision to detect and track people in a video stream, and trigger events based on their spatial relationships (e.g., entering a zone, crossing a line, maintaining distance).

For the AI-900 exam, you must understand that: - Azure Video Indexer is for processing recorded video or live streams to generate insights like OCR, speech-to-text, sentiment, and object detection. It is a SaaS solution. - Spatial Analysis is a containerized service (deployed on the edge or cloud) that analyzes live video feeds to count people, track movement, and detect specific spatial events. It is designed for real-time scenarios like retail analytics, workplace safety, and access control.

Spatial Analysis is deployed as a Docker container that ingests video frames from an RTSP (Real-Time Streaming Protocol) camera or a video file. The container runs on an Azure IoT Edge device or a virtual machine with a GPU. The processing pipeline follows these steps:

- Operations: Spatial Analysis supports several operations. Each operation has specific parameters: - personCrossingLine: Detects when a person crosses a line. Parameters: Line (list of points defining the line), CrossingDirection (e.g., leftToRight, rightToLeft, both). Default: both. - personZoneEnterExit: Detects when a person enters or exits a zone. Parameters: Zone (list of points defining polygon), EventType (enter, exit, both). Default: both. - personCount: Counts the number of people in a zone. No event emission; returns count at intervals. - personDistance: Detects if two people are within a specified distance. Parameters: MinimumDistanceThreshold in pixels (default 100). - Configuration via JSON: All operations are configured in a JSON file passed to the container at startup. Example:

{
  "operations": [
    {
      "operation": "personCrossingLine",
      "parameters": {
        "Line": [[0.1, 0.5], [0.9, 0.5]],
        "CrossingDirection": "both"
      }
    }
  ],
  "cameraConfiguration": {
    "videoUrl": "rtsp://camera:554/stream",
    "frameRate": 15
  }
}

Azure Video Indexer is a cloud service that analyzes video and audio to extract insights. It supports both upload and real-time streaming via the API. Key capabilities tested on AI-900:

Video Indexer uses pre-trained models and does not require custom training for general insights. You can customize by uploading a custom language model or face model.

Spatial Analysis and Video Indexer often integrate with: - Azure IoT Hub: For receiving events from Spatial Analysis containers. - Azure Event Grid: For routing events to functions or logic apps. - Azure Logic Apps: For triggering workflows (e.g., send email when person enters restricted zone). - Azure Blob Storage: For storing video files and insights. - Azure Media Services: For encoding and streaming video.

To verify Spatial Analysis is running:

For Video Indexer, you can use the Azure portal or REST API to view indexed insights. The portal provides a timeline with detected entities.

| Feature | Spatial Analysis | Video Indexer | |---------|------------------|---------------| | Deployment | Container (edge/cloud) | SaaS (cloud) | | Use Case | Real-time spatial events | Post-processing insights | | Output | Event messages | JSON metadata, timeline | | Customization | Parameters only | Custom models (face, language) | | Hardware | GPU required | No GPU needed |

Both services are part of the Computer Vision category in Azure Cognitive Services. The exam expects you to know which service to use for a given scenario: real-time vs. batch, edge vs. cloud, spatial vs. general insights.

A large retail chain deploys Spatial Analysis in 500 stores to count foot traffic and measure dwell times. Each store has 4 ceiling-mounted cameras covering aisles and entrances. The operations used are personCount for each zone and personCrossingLine for entrance/exit lines. The container runs on an NVIDIA Jetson AGX Orin device at each store. Events are sent to Azure IoT Hub, which feeds a Power BI dashboard showing real-time occupancy. The system processes 15 fps per camera, handling up to 30 people per frame. Common misconfiguration: setting frameRate too high (30 fps) causes GPU overload and dropped frames. The solution is to reduce to 10 fps for static scenes. Another issue: camera angle too oblique reduces detection accuracy – overhead angles (45-60 degrees) are optimal. The chain saves 20% on staffing costs by aligning schedules with traffic patterns.

A manufacturing plant uses Spatial Analysis to enforce social distancing. Cameras monitor assembly lines and break rooms. The personDistance operation is configured with MinimumDistanceThreshold=150 pixels (about 6 feet at typical camera distance). When two workers are too close, an alert is sent to a safety supervisor via Logic Apps and SMS. The system also uses personZoneEnterExit to detect entry into hazardous areas. The container runs on an Azure Stack Edge device with a T4 GPU. Performance consideration: the distance calculation is computationally intensive – each pair of people requires a comparison. With 20 people in frame, that's 190 comparisons per frame. To optimize, the container skips comparisons if the bounding boxes are far apart (using a spatial hash grid). The plant reduced safety incidents by 40%.

An office building uses Spatial Analysis to grant access to authorized personnel. Cameras at entrances detect people and trigger personCrossingLine. The event is sent to an Azure Function that checks a database of authorized employees via face recognition (using Face API). If authorized, the function opens the door via an IoT relay. The system processes 5 entrances simultaneously. A key challenge: lighting changes at different times of day cause false negatives. The solution is to adjust the confidence threshold dynamically based on time of day (0.6 during day, 0.4 at night). The building also uses Video Indexer to analyze lobby footage for security audits, indexing 24 hours of video daily. The combination of real-time spatial analysis and batch video indexing provides both immediate response and historical review.