MISC-TN-015: Proof-of-Concept of an industrial, high-frame-rate video recording/streaming system

Info Box Applies to Machine Learning

History

Version	Date	Notes
1.0.0	August 2020	First public release

Introduction

This Technical Note (TN for short) illustrates a Proof-of-Concept (PoC) that DAVE Embedded Systems made for a customer operating in the industrial automation market. The goal was to build a prototype of a high-frame-rate video recording/streaming system. In a typical scenario, illustrated in the following picture, this device would be used in fast automatic manufacturing lines for two purposes:

• remote monitoring
• detailed off-line "post-mortem" failure analysis.

Typical scenario of utilization

In essence, the system consists of a high-frame-rate image sensor (*) shooting a specific area of the line. The frames captured by the sensors are addressed to a processing platform, which implements the functionalities detailed in the following sections.

(*) Resolution and frame rate of this stream have to be determined in function of the characteristics of the scene to shoot, first and foremost the speed of moving objects framed by the sensor and its lens. In the case under discussion, the customer specified a resolution of 1280x720 and a frame rate of 300 fps, and the use of a global shutter.

Functionalities

Streaming capability is used to monitor the production line remotely. Under normal operation, this is enough for the human operators to get an overview of the line while it is working. For this purpose, a simple low-frame-rate video stream (25 fps or something) over a local area network does the job.

The most interesting functionality is related to recording capability, however. As shown in the previous image, the production line is governed by a Programmable Logic Controller (PLC), which is interfaced to several actuators and sensors. Of course, the line may also be subject to different kinds of faults. The most severe—for instance, a major mechanical failure—may lead to the automatic stop of the line. Thanks to the aforementioned sensors, the PLC is aware of such faulty conditions. In the occurrence of these events, it raises an alarm signal directed to the video recording system. Whenever an alarm is detected, the recording system's aim is to store on a mass storage device high-frame-rate footage showing what happened right before and right after the alarm event.

Alarm mode

During normal operation, the high-frame-rate stream (indicated by the red flow in the previous picture) is stored

which, in turn, enters "alarm mode." This mode is used to

The typical scenario in which this device would be used Under normal conditions, this camera streams

   compresses the 60/30fps stream and streams it over Ethernet

Normal mode allows the human operator to remotely supervise the industrial machine where the device is installed

Future work

ML

Credits

• Image sources:
  • Technology vector created by macrovector - www.freepik.com