XUELK-AN-007: USB Camera input on i.MX6UL-based platforms
| HOME | SOMs | SBCs | ToloMEO Embedded Assistant |  GET A QUOTE
|
 ONLINE HELPDESK
|
| Roadmap | IoT Services | ML/AI services | Embedded Design Services |
Info Box
|
|
This application note was validated against specific versions of the kit only. It may not work with other versions. Supported versions are listed in the History section. |
|
Contents
History[edit | edit source]
| Version | Date | XUELK version | Notes |
|---|---|---|---|
| July 2017 | 1.1.3 | First release |
Introduction[edit | edit source]
As known, i.MX6UL SoC has not an IPU cell allowing strong performances on video acquisition. However, using an USB camera it is possible to capture a raw data video playing it on LCD controller.
This application note describes some tests that were performed on an AXELULite-based platform to evaluate the capabilities and the performances of the i.MXUL when it captures and displays a camera source on LCD panel. The test bed was based on Yocto distribution using smart package manager for adding required software component.
USB camera driver[edit | edit source]
Common USB cameras can be used with a standard USB Video Class driver. Linux kernel must be compiled with this driver built-in or the kernel module can be dynamically loaded (in this case it has to be available on /lib/modules root file system directory).
Once plugged on USB host connector, the kernel driver recognizes the device and create the device node /dev/video1
[ 6.989549] uvcvideo: Found UVC 1.00 device USB2.0 Camera (1871:0141) [ 7.041576] input: USB2.0 Camera as /devices/soc0/soc.0/2100000.aips-bus/2184200.usb/ci_hdrc.1/usb2/2-1/2-1:1.0/input/input5 [ 7.138819] usbcore: registered new interface driver uvcvideo [ 7.191864] USB Video Class driver (1.1.1) [ 7.439394] usb_audio: Warning! Unlikely big volume range (=6144), cval->res is probably wrong. [ 7.489486] usb_audio: [3] FU [Mic Capture Volume] ch = 1, val = 5120/11264/1 [ 7.546073] usbcore: registered new interface driver snd-usb-audio
gstreamer[edit | edit source]
Starting from original Yocto based root file system provided here, more packages need to be added for using the well known multimedia framework gstreamer
More info about gstreamer can be found here and on Github gstreamer server
We assume that network interface has been already configured for Internet access. In any case, simple configuration can be done directly on serial console with the following commands:
root@sbc-lynx:~# ifconfig eth0 192.168.0.94 root@sbc-lynx:~# route add default gw 192.168.0.254 root@sbc-lynx:~# echo "nameserver 8.8.8.8" > /etc/resolv.conf
where 192.168.0.254 is the gateway address and 8.8.8.8 is the DNS server (in this case Google's one).
smart packages[edit | edit source]
Firts of all, SMART channels must be added for accessing DAVE's server for Yocto pre-built packages, using the following commands:
root@sbc-lynx:~# smart channel --add cortexa7hf_vfp_neon type=rpm-md baseurl=http://yocto.dave.eu/xuelk-latest/cortexa7hf_vfp_neon root@sbc-lynx:~# smart channel --add cortexa7hf_vfp_neon_mx6ul type=rpm-md baseurl=http://yocto.dave.eu/xuelk-latest/cortexa7hf_vfp_neon_mx6ul root@sbc-lynx:~# smart channel --add sbc_lynx type=rpm-md baseurl=http://yocto.dave.eu/xuelk-latest/sbc_lynx
and finally updating the smart cache checking for the new available packages:
root@sbc-lynx:~# smart update Updating cache... ######################################## [100%] Fetching information for 'cortexa7hf_vfp_neon_mx6ul'... -> http://yocto.dave.eu/xuelk-latest/cortexa7hf_vfp_neon_mx6ul/.../repomd.xml repomd.xml ######################################## [ 16%] -> http://yocto.dave.eu/xuelk-latest/.../filelists.xml.gz filelists.xml.gz ######################################## [ 25%] -> http://yocto.dave.eu/xuelk-latest/.../primary.xml.gz primary.xml.gz ######################################## [ 33%] Fetching information for 'cortexa7hf_vfp_neon'... -> http://yocto.dave.eu/xuelk-latest/cortexa7hf_vfp_neon/repodata/repomd.xml repomd.xml ######################################## [ 50%] -> http://yocto.dave.eu/xuelk-latest/cortexa7hf_vfp_neon/.../filelists.xml.gz filelists.xml.gz ######################################## [ 58%] -> http://yocto.dave.eu/xuelk-latest/cortexa7hf_vfp_neon/.../primary.xml.gz primary.xml.gz ######################################## [ 66%] Fetching information for 'sbc_lynx'... -> http://yocto.dave.eu/xuelk-latest/sbc_lynx/repodata/repomd.xml repomd.xml ######################################## [ 83%] -> http://yocto.dave.eu/xuelk-latest/sbc_lynx/repodata/filelists.xml.gz -> http://yocto.dave.eu/xuelk-latest/sbc_lynx/repodata/primary.xml.gz primary.xml.gz ######################################## [ 91%] filelists.xml.gz ######################################## [100%] Updating cache... ######################################## [100%] Channels have 11874 new packages. Saving cache... root@sbc-lynx:~#
Adding these three channels there will be 11874 new available packages.
Adding gstreamer packages[edit | edit source]
It is possibile to install the gstreamer 1.0 tool using the following command:
root@sbc-lynx:~# smart install gstreamer1.0-1.4.5-r0 Loading cache... Updating cache... ######################################## [100%] Computing transaction... Installing packages (1): gstreamer1.0-1.4.5-r0@cortexa7hf_vfp_neon 763.6kB of package files are needed. 1.8MB will be used. Confirm changes? (Y/n): y Fetching packages... -> http://yocto.dave.eu/.../gstreamer1.0-1.4.5-r0.cortexa7hf_vfp_neon.rpm gstreamer1.0-1.4.5-r0.cortexa.. ######################################## [100%] Committing transaction... Preparing... ######################################## [ 0%] 1:Installing gstreamer1.0 ######################################## [100%] root@sbc-lynx:~#
Then, it is enough to install the other gstreamer plugins, for example:
gstreamer1.0-plugins-good-1.4.5-r0 gstreamer1.0-plugins-base-1.4.5-r0 gstreamer1.0-plugins-bad-1.4.5-r0
These packages will install about:
plugins-good: 5.5MB of package files are needed. 11.7MB will be used. plugins-base: 1.8MB of package files are needed. 3.7MB will be used. plugins-bad: 2.1MB of package files are needed. 3.9MB will be used.
Adding fsl gstreamer plugin[edit | edit source]
For video sink on framebuffer, the NXP gstreamer plugin is required. This can be installed from fsl package:
root@sbc-lynx:~# smart install gst1.0-fsl-plugin-4.0.8-r0 Loading cache... Updating cache... ######################################## [100%] Computing transaction... Installing packages (7): gst1.0-fsl-plugin-4.0.8-r0@sbc_lynx lib-oggvorbis-dec-arm11-elinux2-4.0.8-r0@cortexa7hf_vfp_neon_mx6ul libfslcodec-aac-4.0.8-r0@cortexa7hf_vfp_neon_mx6ul libfslcodec-mp3-4.0.8-r0@cortexa7hf_vfp_neon_mx6ul libfslparser-4.0.8-r0@cortexa7hf_vfp_neon_mx6ul libgstfsl-1.0-0-4.0.8-r0@sbc_lynx libpxp0-1:5.3-r0@sbc_lynx 1.6MB of package files are needed. 5.9MB will be used. Confirm changes? (Y/n): y Fetching packages... -> http://yocto.dave.eu/.../libfslcodec-mp3-4.0.8-r0.cortexa7hf_vfp_neon_mx6ul.rpm -> http://yocto.dave.eu/.../lib-oggvorbis-dec-arm11-elinux2-4.0.8-r0.cortexa7hf_vfp_neon_mx6ul.rpm lib-oggvorbis-dec-arm11-elinu.. ######################################## [ 14%] -> http://yocto.dave.eu/xuelk-latest/.../libgstfsl-1.0-0-4.0.8-r0.sbc_lynx.rpm libgstfsl-1.0-0-4.0.8-r0.sbc_.. ######################################## [ 28%] -> http://yocto.dave.eu/.../libfslcodec-aac-4.0.8-r0.cortexa7hf_vfp_neon_mx6ul.rpm libfslcodec-aac-4.0.8-r0.cort.. ######################################## [ 42%] libfslcodec-mp3-4.0.8-r0.cort.. ######################################## [ 57%] -> http://yocto.dave.eu/.../gst1.0-fsl-plugin-4.0.8-r0.sbc_lynx.rpm -> http://yocto.dave.eu/xuelk-latest/sbc_lynx/libpxp0-5.3-r0.sbc_lynx.rpm libpxp0-5.3-r0.sbc_lynx.rpm ######################################## [ 71%] -> http://yocto.dave.eu/.../libfslparser-4.0.8-r0.cortexa7hf_vfp_neon_mx6ul.rpm libfslparser-4.0.8-r0.cortexa.. ######################################## [ 85%] gst1.0-fsl-plugin-4.0.8-r0.sb.. ######################################## [100%] Committing transaction... Preparing... ######################################## [ 0%] 1:Installing libfslparser ######################################## [ 14%] 2:Installing libfslcodec-mp3 ######################################## [ 28%] 3:Installing libfslcodec-aac ######################################## [ 42%] 4:Installing libpxp0 ######################################## [ 57%] 5:Installing lib-oggvorbis.. ######################################## [ 71%] 6:Installing libgstfsl-1.0-0 ######################################## [ 85%] 7:Installing gst1.0-fsl-pl.. ######################################## [100%] root@sbc-lynx:/lib/modules#
gstreamer pipeline[edit | edit source]
The pipeline used is the easiest one for capturing a raw data stream from a standard USB camera
gst-launch-1.0 -v v4l2src device=/dev/video1 ! imxv4l2sink
Test bed[edit | edit source]
- iMX6UL cpu is running at 528MHz
root@arm:~# cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq 528000
- system DRAM is 256MB
- lcd panel is 800x480
Results[edit | edit source]
Use case #1: Video capture 640x480@30fps[edit | edit source]
- pipeline:
root@sbc-lynx:~# gst-launch-1.0 -v v4l2src device=/dev/video1 ! 'video/x-raw, format=YUY2, width=640, height=480, framerate=30/1' ! imxv4l2sink force-aspect-ratio=true ====== IMXV4L2SINK: 4.0.8 build on Aug 25 2016 18:01:57. ====== Setting pipeline to PAUSED ... display(/dev/fb0) resolution is (800x480). Pipeline is live and does not need PREROLL ... Setting pipeline to PLAYING ... New clock: GstSystemClock /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)YUY2\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1" /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)YUY2\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1" /GstPipeline:pipeline0/GstImxV4l2Sink:imxv4l2sink0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)YUY2\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1" /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)YUY2\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1" v4l2sink need allocate 3 buffers.
- top process shows that cpu is running at 20%
Mem: 81752K used, 165412K free, 356K shrd, 9084K buff, 18600K cached
CPU: 2% usr 13% sys 0% nic 64% idle 0% io 0% irq 19% sirq
Load average: 0.13 0.09 0.05 1/64 452
PID PPID USER STAT VSZ %VSZ %CPU COMMAND
432 386 root S 35364 14% 22% gst-launch-1.0 -v v4l2src device=/dev/
447 386 root R 2828 1% 2% top -d 1
71 2 root SW 0 0% 1% [pxp_dispatch]
3 2 root SW 0 0% 1% [ksoftirqd/0]
382 1 root S 5028 2% 0% /usr/sbin/tcf-agent -d -L- -l0
...
Use case #2: Video capture 640x480@30fps scaled to 320x240[edit | edit source]
This use case is similar to the previous one, but the video data has been scaled to 1/4 using the imxv4l2sink plugin overlay capabilities
root@sbc-lynx:~# gst-launch-1.0 -v v4l2src device=/dev/video1 ! 'video/x-raw, format=YUY2, width=320, height=240, framerate=30/1' ! imxv4l2sink overlay-top=120 overlay-left=240 overlay-width=320 overlay-height=240 force-aspect-ratio=true ====== IMXV4L2SINK: 4.0.8 build on Aug 25 2016 18:01:57. ====== Setting pipeline to PAUSED ... display(/dev/fb0) resolution is (800x480). Pipeline is live and does not need PREROLL ... Setting pipeline to PLAYING ... New clock: GstSystemClock /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)YUY2\,\ width\=\(int\)320\,\ height\=\(int\)240\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1" /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)YUY2\,\ width\=\(int\)320\,\ height\=\(int\)240\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1" /GstPipeline:pipeline0/GstImxV4l2Sink:imxv4l2sink0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)YUY2\,\ width\=\(int\)320\,\ height\=\(int\)240\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1" /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)YUY2\,\ width\=\(int\)320\,\ height\=\(int\)240\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1" v4l2sink need allocate 3 buffers.
- top process shows that cpu is running at 13%
Mem: 176956K used, 70208K free, 356K shrd, 3896K buff, 116504K cached CPU: 0% usr 9% sys 0% nic 71% idle 0% io 0% irq 19% sirq Load average: 0.27 0.35 0.36 1/64 683 PID PPID USER STAT VSZ %VSZ %CPU COMMAND 676 378 root S 35512 14% 13% gst-launch-1.0 -v v4l2src device=/dev/
Use case #3: Video capture 640x480@30fps with rotation[edit | edit source]
In this use case the PXP iMX6UL hardware cell is performing an image rotation on the screen using the imxvideoconvert_pxp NXP gstreamer plugin
root@sbc-lynx:~# gst-launch-1.0 -v v4l2src device=/dev/video1 ! capsfilter caps="video/x-raw, width=640, height=480, framerate=30/1" ! imxvideoconvert_pxp rotation=1 ! imxv4l2sink force-aspect-ratio=true ====== IMXV4L2SINK: 4.0.8 build on Aug 25 2016 18:01:57. ====== Setting pipeline to PAUSED ... display(/dev/fb0) resolution is (800x480). Pipeline is live and does not need PREROLL ... Setting pipeline to PLAYING ... New clock: GstSystemClock /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)YUY2\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1" /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)YUY2\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1" /GstPipeline:pipeline0/imxvideoconvert_pxp:imxvideoconvert_pxp0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)BGRx\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ framerate\=\(fraction\)30/1\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive" /GstPipeline:pipeline0/GstImxV4l2Sink:imxv4l2sink0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)BGRx\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ framerate\=\(fraction\)30/1\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive" /GstPipeline:pipeline0/imxvideoconvert_pxp:imxvideoconvert_pxp0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)YUY2\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1" /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)YUY2\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1" v4l2sink need allocate 3 buffers.
