DESK-XZ7-L-AN-0004: Using Python with BORA

From DAVE Developer's Wiki
Jump to: navigation, search
Info Box


200px-Emblem-important.svg.png

This application note has been validated using the kit version in the History table.

History[edit | edit source]

Version Date Development Kit version

1.0.0

Oct 2022 DESK-XZ7-L 1.0.0-rc1
1.0.1 Jan 2024 DESK-XZ7-L 1.0.1

Introduction[edit | edit source]

As found on Python official website, Python is a programming language that lets you work more quickly and integrate your systems more effectively and also Python can be easy to pick up whether you're a first-time programmer or you're experienced with other languages.

These sentences got a real confirmation if you have a look at the IEEE Top Programming Languages 2021 ranking. As you can see in the following picture Python reached the Top of the ranking even more than the native C language used since the beginning in Embedded systems programming:


IEEE Top programming languages 2021 - Embedded


As reported in this article The C/C++ programming languages dominate embedded systems programming, though they have a number of disadvantages. Python, on the other hand, has many strengths that make it a great language for embedded systems.

This application note provides some examples of software packages installation that can be used for building a Python development system adding python3 libraries to the BORA platform.

Python on DESK[edit | edit source]

python3 application is already present on DESK-XZ7-L petalinux root file system.

Installing python packages[edit | edit source]

First of all check for the default installed python/pip version in the DESK-XZ7 root file system:

root@bora:~# python3 --version
Python 3.8.5
root@bora:~#

pip3 can be installed using python and the related script get-pip-py:

root@bora:~# wget https://bootstrap.pypa.io/get-pip.py
Connecting to bootstrap.pypa.io (151.101.0.175:443)
wget: note: TLS certificate validation not implemented
saving to 'get-pip.py'
get-pip.py           100% |********************************************************************************************| 2573k  0:00:00 ETA
'get-pip.py' saved
root@bora:~# python3 get-pip.py
Collecting pip
  Downloading pip-23.3.2-py3-none-any.whl.metadata (3.5 kB)
Collecting setuptools
  Downloading setuptools-69.0.3-py3-none-any.whl.metadata (6.3 kB)
Collecting wheel
  Downloading wheel-0.42.0-py3-none-any.whl.metadata (2.2 kB)
Downloading pip-23.3.2-py3-none-any.whl (2.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 2.1 MB/s eta 0:00:00
Downloading setuptools-69.0.3-py3-none-any.whl (819 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 819.5/819.5 kB 1.9 MB/s eta 0:00:00
Downloading wheel-0.42.0-py3-none-any.whl (65 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.4/65.4 kB 427.3 kB/s eta 0:00:00
DEPRECATION: gpg 1.14.0-unknown has a non-standard version number. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of gpg or contact the author to suggest that they release a version with a conforming version number. Discussion can be found at https://github.com/pypa/pip/issues/12063
Installing collected packages: wheel, setuptools, pip
Successfully installed pip-23.3.2 setuptools-69.0.3 wheel-0.42.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
root@bora:~# 
  • then, pip version can be checked
root@bora:~# pip3 --version
pip 23.3.2 from /usr/lib/python3.8/site-packages/pip (python 3.8)
root@bora:~#

Virtual environments[edit | edit source]

As explained on python3 12. Virtual Environments and Packages tutorial it is better to "create a virtual environment, a self-contained directory tree that contains a Python installation for a particular version of Python, plus a number of additional packages"

This avoids the problem related to Applications that sometimes need a specific version of a library. In this way, it is possible to install the required packages (with their specific version required) only in this virtual environment (which may differ from the root installation).

First of all the venv module should be installed:

root@bora:~# python3 -m pip install --user virtualenv
Collecting virtualenv
  Downloading virtualenv-20.25.0-py3-none-any.whl.metadata (4.5 kB)
Collecting distlib<1,>=0.3.7 (from virtualenv)
  Downloading distlib-0.3.8-py2.py3-none-any.whl.metadata (5.1 kB)
Collecting filelock<4,>=3.12.2 (from virtualenv)
  Downloading filelock-3.13.1-py3-none-any.whl.metadata (2.8 kB)
Collecting platformdirs<5,>=3.9.1 (from virtualenv)
  Downloading platformdirs-4.1.0-py3-none-any.whl.metadata (11 kB)
Downloading virtualenv-20.25.0-py3-none-any.whl (3.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.8/3.8 MB 1.9 MB/s eta 0:00:00
Downloading distlib-0.3.8-py2.py3-none-any.whl (468 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 468.9/468.9 kB 2.2 MB/s eta 0:00:00
Downloading filelock-3.13.1-py3-none-any.whl (11 kB)
Downloading platformdirs-4.1.0-py3-none-any.whl (17 kB)
DEPRECATION: gpg 1.14.0-unknown has a non-standard version number. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of gpg or contact the author to suggest that they release a version with a conforming version number. Discussion can be found at https://github.com/pypa/pip/issues/12063
Installing collected packages: distlib, platformdirs, filelock, virtualenv
  WARNING: The script virtualenv is installed in '/home/root/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed distlib-0.3.8 filelock-3.13.1 platformdirs-4.1.0 virtualenv-20.25.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
root@bora:~#
  • the virtual environment named desk can be created:
root@bora:~# python3 -m venv desk
root@bora:~# 
  • activate the virtual environment
root@bora:~# source desk/bin/activate
(desk) root@bora:~# 
  • upgrade pip3 in the virtual environment:
(desk) root@bora:~# pip3 install --upgrade pip
Collecting pip
  Downloading pip-23.3.2-py3-none-any.whl (2.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1 MB 167 kB/s
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 20.1.1
    Uninstalling pip-20.1.1:
      Successfully uninstalled pip-20.1.1
Successfully installed pip-23.3.2
(desk) root@bora:~#
  • setuptools can be upgraded too:
(desk) root@bora:~# pip3 install --upgrade setuptools
Requirement already satisfied: setuptools in ./desk/lib/python3.8/site-packages (47.1.0)
Collecting setuptools
  Using cached setuptools-69.0.3-py3-none-any.whl.metadata (6.3 kB)
Using cached setuptools-69.0.3-py3-none-any.whl (819 kB)
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 47.1.0
    Uninstalling setuptools-47.1.0:
      Successfully uninstalled setuptools-47.1.0
Successfully installed setuptools-69.0.3
(desk) root@bora:~#

Afterword, the wheel package is worth to be installed for further package installation:

(desk) root@bora:~# pip3 install wheel
Collecting wheel
  Using cached wheel-0.42.0-py3-none-any.whl.metadata (2.2 kB)
Using cached wheel-0.42.0-py3-none-any.whl (65 kB)
Installing collected packages: wheel
Successfully installed wheel-0.42.0
(desk) root@bora:~#

packages installation[edit | edit source]

For the purposes of this Application Note, pip3 has been used to install some python packages typically used in a controller equipment.

In this example, the following packages are installed:

  • asyncua (OPCUA client and server library)
  • bcrypt (password hashing)
  • can (color animator)
  • cbor (Concise Binary Object Representation - CBOR is comparable to JSON, has a superset of JSON’s ability)
  • netifaces (network interface info)
  • ntplib (Python NTP)
  • pymodbus (Modbus stack protocol)
  • pyserial (Serial Port extension)
  • python-daemon (Standard daemon process library)
  • python-dateutil (Python extension dateutil)
  • python-prctl (C extension for system call)
  • requests (Python HTTP)
(desk) root@bora:~# pip3 install netifaces
Collecting netifaces
  Using cached netifaces-0.11.0.tar.gz (30 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: netifaces
  Building wheel for netifaces (setup.py) ... done
  Created wheel for netifaces: filename=netifaces-0.11.0-cp38-cp38-linux_armv7l.whl size=35205 sha256=e405dea3379d15e715be65b57f651beef13df5df815013bb8332960a481f58ac
  Stored in directory: /home/root/.cache/pip/wheels/04/e2/b4/b811799a082cacaeff801af32c856fbefe6279ba96554d9a21
Successfully built netifaces
Installing collected packages: netifaces
Successfully installed netifaces-0.11.0
(desk) root@bora:~# pip3 install can
Collecting can
  Downloading can-0.0.0.tar.gz (1.0 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: can
  Building wheel for can (setup.py) ... done
  Created wheel for can: filename=can-0.0.0-py3-none-any.whl size=1266 sha256=b210b09ceb4ddf6c322bd2b7b249976cd45312b0919b7044158698ca604d9a00
  Stored in directory: /home/root/.cache/pip/wheels/45/b4/e7/1f023b01e25425d5b9f3d2d537af7d7bf632e5d7a66ee703ed
Successfully built can
Installing collected packages: can
Successfully installed can-0.0.0
(desk) root@bora:~# pip3 install cbor
Collecting cbor
  Downloading cbor-1.0.0.tar.gz (20 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: cbor
  Building wheel for cbor (setup.py) ... done
  Created wheel for cbor: filename=cbor-1.0.0-py3-none-any.whl size=10029 sha256=36179cc461113f665dd3e948c56e5928726b83f01efc6107fe08030bf49ed8b2
  Stored in directory: /home/root/.cache/pip/wheels/31/3a/8a/9d1a5e6f83f620826578089812b8ac236b86eaf4fbc3b36384
Successfully built cbor
Installing collected packages: cbor
Successfully installed cbor-1.0.0
(desk) root@bora:~# pip3 install requests
Collecting requests
  Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
Collecting charset-normalizer<4,>=2 (from requests)
  Downloading charset_normalizer-3.3.2-py3-none-any.whl.metadata (33 kB)
Collecting idna<4,>=2.5 (from requests)
  Downloading idna-3.6-py3-none-any.whl.metadata (9.9 kB)
Collecting urllib3<3,>=1.21.1 (from requests)
  Downloading urllib3-2.1.0-py3-none-any.whl.metadata (6.4 kB)
Collecting certifi>=2017.4.17 (from requests)
  Downloading certifi-2023.11.17-py3-none-any.whl.metadata (2.2 kB)
Downloading requests-2.31.0-py3-none-any.whl (62 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.6/62.6 kB 557.9 kB/s eta 0:00:00
Downloading certifi-2023.11.17-py3-none-any.whl (162 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 162.5/162.5 kB 1.2 MB/s eta 0:00:00
Downloading charset_normalizer-3.3.2-py3-none-any.whl (48 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.5/48.5 kB 424.1 kB/s eta 0:00:00
Downloading idna-3.6-py3-none-any.whl (61 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.6/61.6 kB 542.1 kB/s eta 0:00:00
Downloading urllib3-2.1.0-py3-none-any.whl (104 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 104.6/104.6 kB 959.3 kB/s eta 0:00:00
Installing collected packages: urllib3, idna, charset-normalizer, certifi, requests
Successfully installed certifi-2023.11.17 charset-normalizer-3.3.2 idna-3.6 requests-2.31.0 urllib3-2.1.0
(desk) root@bora:~# pip3 install pymodbus
Collecting pymodbus
  Downloading pymodbus-3.6.3-py3-none-any.whl.metadata (14 kB)
Downloading pymodbus-3.6.3-py3-none-any.whl (205 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 205.2/205.2 kB 1.1 MB/s eta 0:00:00
Installing collected packages: pymodbus
Successfully installed pymodbus-3.6.3
(desk) root@bora:~# pip3 install ntplib
Collecting ntplib
  Downloading ntplib-0.4.0-py2.py3-none-any.whl (6.8 kB)
Installing collected packages: ntplib
Successfully installed ntplib-0.4.0
(desk) root@bora:~# pip3 install python-daemon
Collecting python-daemon
  Downloading python_daemon-3.0.1-py3-none-any.whl (31 kB)
Collecting docutils (from python-daemon)
  Downloading docutils-0.20.1-py3-none-any.whl.metadata (2.8 kB)
Collecting lockfile>=0.10 (from python-daemon)
  Downloading lockfile-0.12.2-py2.py3-none-any.whl (13 kB)
Requirement already satisfied: setuptools>=62.4.0 in ./desk/lib/python3.8/site-packages (from python-daemon) (69.0.3)
Downloading docutils-0.20.1-py3-none-any.whl (572 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 572.7/572.7 kB 452.5 kB/s eta 0:00:00
Installing collected packages: lockfile, docutils, python-daemon
Successfully installed docutils-0.20.1 lockfile-0.12.2 python-daemon-3.0.1
(desk) root@bora:~# pip3 install python-dateutil
Collecting python-dateutil
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting six>=1.5 (from python-dateutil)
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: six, python-dateutil
Successfully installed python-dateutil-2.8.2 six-1.16.0
(desk) root@bora:~# pip3 install python-prctl
Collecting python-prctl
  Using cached python-prctl-1.8.1.tar.gz (28 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: python-prctl
  Building wheel for python-prctl (setup.py) ... done
  Created wheel for python-prctl: filename=python_prctl-1.8.1-cp38-cp38-linux_armv7l.whl size=25369 sha256=369d991812ce1d758297332ae01c9d725f0bd5bc480c382bf8b4a917a5ebd6c0
  Stored in directory: /home/root/.cache/pip/wheels/70/90/f1/b786edccb1b2d9290281dad80c8aa0e93ab5c1f845a8d404aa
Successfully built python-prctl
Installing collected packages: python-prctl
Successfully installed python-prctl-1.8.1
(desk) root@bora:~# pip3 install pyserial
Collecting pyserial
  Downloading pyserial-3.5-py2.py3-none-any.whl (90 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 90.6/90.6 kB 86.5 kB/s eta 0:00:00
Installing collected packages: pyserial
Successfully installed pyserial-3.5
(desk) root@bora:~# 

asyncua[edit | edit source]

Some packages may require Rust compiler for building the (for example) wheel:

  • download the rust Linux installer
wget https://sh.rustup.rs
  • execute the installer script:
./rustup-init.sh
root@bora:~# 

Once installed the related environment should be initialized:

(desk) root@bora:~# source "$HOME/.cargo/env"

For building wheel some package are required (like python3-dev) and this can be installed using dnf:

root@bora:~# dnf install python3-dev
OE Remote Repo: sswreleases rel-v2021.2 generic 1.4 kB/s | 257  B     00:00
OE Remote Repo: sswreleases rel-v2021.2 generic 3.7 kB/s | 257  B     00:00
OE Remote Repo: sswreleases rel-v2021.2 generic 1.2 MB/s | 139 kB     00:00
OE Remote Repo: sswreleases rel-v2021.2 generic 4.7 kB/s | 257  B     00:00
OE Remote Repo: sswreleases rel-v2021.2 generic 4.3 kB/s | 257  B     00:00
...
Dependencies resolved.
================================================================================
 Package              Architecture      Version       Repository           Size
================================================================================
Installing:
 python3-dev          cortexa9t2hf_neon 3.8.5-r0.0    oe-remote-repo-sswreleases-rel-v2021.2-generic-rpm-cortexa9t2hf_neon

...
...
Transaction Summary
================================================================================
Install  80 Packages

Total download size: 3.7 M
Installed size: 19 M
Is this ok [y/N]: y
Downloading Packages:
...
...
root@bora:~#

In this way, the asyncua package can be built/installed:

(desk) root@bora:~# pip3 install asyncua
Collecting asyncua
  Using cached asyncua-1.0.6-py3-none-any.whl.metadata (9.3 kB)
Collecting aiofiles (from asyncua)
  Using cached aiofiles-23.2.1-py3-none-any.whl.metadata (9.7 kB)
Collecting aiosqlite (from asyncua)
  Using cached aiosqlite-0.19.0-py3-none-any.whl (15 kB)
Requirement already satisfied: python-dateutil in ./desk/lib/python3.8/site-packages (from asyncua) (2.8.2)
Collecting pytz (from asyncua)
  Using cached pytz-2023.4-py2.py3-none-any.whl.metadata (22 kB)
Collecting cryptography>40.0.1 (from asyncua)
  Downloading cryptography-42.0.2.tar.gz (672 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 672.8/672.8 kB 559.1 kB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting sortedcontainers (from asyncua)
  Downloading sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting pyOpenSSL (from asyncua)
  Downloading pyOpenSSL-24.0.0-py3-none-any.whl.metadata (12 kB)
Collecting typing-extensions (from asyncua)
  Downloading typing_extensions-4.9.0-py3-none-any.whl.metadata (3.0 kB)
Collecting cffi>=1.12 (from cryptography>40.0.1->asyncua)
  Using cached cffi-1.16.0-cp38-cp38-linux_armv7l.whl
Requirement already satisfied: six>=1.5 in ./desk/lib/python3.8/site-packages (from python-dateutil->asyncua) (1.16.0)
Collecting pycparser (from cffi>=1.12->cryptography>40.0.1->asyncua)
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Downloading asyncua-1.0.6-py3-none-any.whl (773 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 773.0/773.0 kB 2.2 MB/s eta 0:00:00
Downloading aiofiles-23.2.1-py3-none-any.whl (15 kB)
Downloading pyOpenSSL-24.0.0-py3-none-any.whl (58 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.6/58.6 kB 521.5 kB/s eta 0:00:00
Downloading pytz-2023.4-py2.py3-none-any.whl (506 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 506.5/506.5 kB 480.3 kB/s eta 0:00:00
Downloading typing_extensions-4.9.0-py3-none-any.whl (32 kB)
Building wheels for collected packages: cryptography
  Building wheel for cryptography (pyproject.toml) ... done
  Created wheel for cryptography: filename=cryptography-42.0.2-cp38-cp38-linux_armv7l.whl size=2206468 sha256=27ef627b5636e640bc179ff1ad13948ac029ad3c709d1e137670f6c5fe79c131
  Stored in directory: /home/root/.cache/pip/wheels/b8/2d/51/9a40e9f8548e174065e9988ad0eece7edf49ece9b2251e0790
Successfully built cryptography
Installing collected packages: sortedcontainers, pytz, typing-extensions, pycparser, aiosqlite, aiofiles, cffi, cryptography, pyOpenSSL, asyncua
Successfully installed aiofiles-23.2.1 aiosqlite-0.19.0 asyncua-1.0.6 cffi-1.16.0 cryptography-42.0.2 pyOpenSSL-24.0.0 pycparser-2.21 pytz-2023.4 sortedcontainers-2.4.0 typing-extensions-4.9.0
(desk) root@bora:~#

check for installed packages and version[edit | edit source]

At the end, the package installed can be listed:

(desk) root@bora:~# pip3 list
Package            Version
------------------ ----------
(desk) root@bora:~#
aiofiles           23.2.1
aiosqlite          0.19.0
asyncua            1.0.6
bcrypt             4.1.2
can                0.0.0
cbor               1.0.0
certifi            2023.11.17
cffi               1.16.0
charset-normalizer 3.3.2
cryptography       42.0.2
docutils           0.20.1
idna               3.6
lockfile           0.12.2
netifaces          0.11.0
ntplib             0.4.0
pip                23.3.2
pycparser          2.21
pymodbus           3.6.3
PyNaCl             1.2.0
pyOpenSSL          24.0.0
pyserial           3.5
python-daemon      3.0.1
python-dateutil    2.8.2
python-prctl       1.8.1
pytz               2023.4
requests           2.31.0
setuptools         69.0.3
six                1.16.0
sortedcontainers   2.4.0
typing_extensions  4.9.0
urllib3            2.1.0
wheel              0.42.0
(desk) root@bora:~# 

package import[edit | edit source]

The installed packages can be imported and used:

(env) root@bora:~# python3
Python 3.8.5 (default, Jul 20 2020, 13:26:22)
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncua
>>> import can
>>> import cbor
>>> import daemon
>>> import dateutil
>>> import netifaces
>>> import ntplib
>>> import pymodbus
>>> import prctl
>>> import serial
>>> import requests
>>>