ML-TN-005 — Real-time Social Distancing and video streaming on Orca SBC

From DAVE Developer's Wiki
Revision as of 15:03, 6 December 2021 by U0001 (talk | contribs)
Jump to: navigation, search
Info Box
NeuralNetwork.png Applies to Machine Learning


History[edit | edit source]

Version Date Notes
1.0.0 December 2021 First public release

Introduction[edit | edit source]

This Technical Note (TN for short) illustrates an interesting use case of the i.MX8M Plus-powered Orca Single Board Computer. In a nutshell, this example demonstrates the following functionalities, which are typical for many video processing applications:

  • Capturing live video streams from MIPI CSI-2 cameras
  • Hardware-accelerated color space conversion (GPU-powered)
  • Hardware-accelerated real-time inferencing (NPU-powered)
  • Hardware-accelerated video encoding (two streams)
  • Hardware-accelerated video decoding (two streams)
  • Hardware-accelerated GUI (GPU)

Testbed[edit | edit source]

Basically, the testbed consists of two Orca SBC's connected as shown in the following diagram.

TBD

Orca SBC #1 is also interfaced to the following camera modules by Basler:

  • daa2500-60mc
  • daa3840-30mc

Both SBC's run Yocto Linux based on the 5.4.70 BSP by NXP.

Implementation[edit | edit source]

Camera modules interfacing[edit | edit source]

The camera modules are connected to the MIPI CSI-2 ports of the i.MX8M Plus. Each one exploits 4 MIPI lanes.

The camera modules are configured TBD

  • daa2500-60mc
    • resolution:
    • frame rate:
  • daa3840-30mc
    • resolution:
    • frame rate:


640x360@15fps

Video streams processing[edit | edit source]

Social Distancing algorithm[edit | edit source]

The Social Distancing algorithm is described in detail in ML-TN-002_-_Real-time_Social_Distancing_estimation this document.

GUI[edit | edit source]

A GUI application runs on the Orca SBC #1 too. This application is engineered with Crank Storyboard 6 and shows some parameters of the Social Distancing algorithm.