DESK-XZ7-L-AN-0004: Using Python with BORA

Info Box Applies to BORA Application Notes Applies to BORA Xpress Application Notes Applies to BORA Lite Application Notes

This application note has been validated using the kit version in the History table.

History

Version	Date	Development Kit version
1.0.0	Oct 2022	DESK-XZ7-L 1.0.0-rc1

Introduction

As found on Python official website, Python is a programming language that lets you work more quickly and integrate your systems more effectively and also Python can be easy to pick up whether you're a first-time programmer or you're experienced with other languages.

These sentences got a real confirmation if you have a look at the IEEE Top Programming Languages 2021 ranking. As you can see in the following picture Python reached the Top of the ranking even more than the native C language used since the beginning in Embedded systems programming:

IEEE Top programming languages 2021 - Embedded

As reported in this article The C/C++ programming languages dominate embedded systems programming, though they have a number of disadvantages. Python, on the other hand, has many strengths that make it a great language for embedded systems.

This application note provides some examples of software packages installation that can be used for building a Python development system adding python3 libraries to the BORA platform.

Python on DESK

python3 application is already present on DESK-XZ7-L petalinux root file system.

Installing python packages

First of all check for the default installed python/pip version in the DESK-XZ7 root file system:

root@bora:~# python3 --version
Python 3.8.5
root@bora:~#

pip3 version can be upgraded too:

root@bora:~# pip3 --version
pip 20.0.2 from /usr/lib/python3.8/site-packages/pip (python 3.8)
root@bora:~# pip3 install --upgrade pip
Collecting pip
  Downloading pip-22.3-py3-none-any.whl (2.1 MB)
     |████████████████████████████████| 2.1 MB 289 kB/s 
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 20.0.2
    Uninstalling pip-20.0.2:
      Successfully uninstalled pip-20.0.2
Successfully installed pip-22.3
root@bora:~# 

Virtual environments

As explained on python3 12. Virtual Environments and Packages tutorial it is better to "create a virtual environment, a self-contained directory tree that contains a Python installation for a particular version of Python, plus a number of additional packages"

This avoids the problem related to Applications that sometimes need a specific version of a library. In this way, it is possible to install the required packages (with their specific version required) only in this virtual environment (which may differ from the root installation).

First of all the venv module should be installed:

root@bora:~# python3 -m pip install --user virtualenv
Collecting virtualenv
  Downloading virtualenv-20.16.5-py3-none-any.whl (8.8 MB)
     |████████████████████████████████|  8.8/8.8 MB 2.5 MB/s eta 0:00:00
Collecting filelock<4,>=3.4.1
  Downloading filelock-3.8.0-py3-none-any.whl (10 kB)
Collecting distlib<1,>=0.3.5
  Downloading distlib-0.3.6-py2.py3-none-any.whl (468 kB)
     |████████████████████████████████|  468.5/468.5 kB 2.2 MB/s eta 0:00:00
Collecting platformdirs<3,>=2.4
  Downloading platformdirs-2.5.2-py3-none-any.whl (14 kB)
Installing collected packages: distlib, platformdirs, filelock, virtualenv
  WARNING: The script virtualenv is installed in '/home/root/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed distlib-0.3.6 filelock-3.8.0 platformdirs-2.5.2 virtualenv-20.16.5
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
root@bora:~# 

then the virtual environment named desk can be created:

root@bora:~# python3 -m venv desk
root@bora:~# 

• activate the virtual environment

root@bora:~# source env/bin/activate
(desk) root@bora:~# 

• upgrade pip3 in the virtual environment:

(desk) root@bora:~# pip3 install --upgrade pip
Collecting pip
  Using cached pip-22.3-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 20.1.1
    Uninstalling pip-20.1.1:
      Successfully uninstalled pip-20.1.1
Successfully installed pip-22.3
(desk) root@bora:~# pip3 --version
pip 22.3 from /home/root/env/lib/python3.8/site-packages/pip (python 3.8)
(desk) root@bora:~#

• setuptools can be upgraded too:

(desk) root@bora:~# pip3 install --upgrade setuptools
Requirement already satisfied: setuptools in ./env/lib/python3.8/site-packages (47.1.0)
Collecting setuptools
  Downloading setuptools-65.5.0-py3-none-any.whl (1.2 MB)
     |████████████████████████████████|  1.2/1.2 MB 2.3 MB/s eta 0:00:00
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 47.1.0
    Uninstalling setuptools-47.1.0:
      Successfully uninstalled setuptools-47.1.0
Successfully installed setuptools-65.5.0
(desk) root@bora:~# 

Afterword, the wheel package is worth to be installed for further package installation:

(desk) root@bora:~# pip3 install wheel
Collecting wheel
  Downloading wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Installing collected packages: wheel
Successfully installed wheel-0.37.1
(desk) root@bora:~# 

packages installation

For the purposes of this Application Note, pip3 has been used to install some python packages typically used in a controller equipment.

In this example, the following packages are installed:

• asyncua (OPCUA client and server library)
• can (color animator)
• cbor (Concise Binary Object Representation - CBOR is comparable to JSON, has a superset of JSON’s ability)
• netifaces (network interface info)
• ntplib (Python NTP)
• paramiko (SSH2 protocol library)
• psutil (process and system monitoring)
• pymodbus (Modbus stack protocol)
• python-daemon (Standard daemon process library)
• python-prctl (C extension for system call)
• requests (Python HTTP)

(desk) root@bora:~# pip3 install cbor
Collecting cbor
  Downloading cbor-1.0.0.tar.gz (20 kB)
  Preparing metadata (setup.py) ... - \ done
Building wheels for collected packages: cbor
  Building wheel for cbor (setup.py) ... - \ | / - \ | / - done
  Created wheel for cbor: filename=cbor-1.0.0-cp38-cp38-linux_armv7l.whl size=51722 sha256=056a9213e339003ff735d05fec04fa008ce05284211db1bc8c38083efb20013d
  Stored in directory: /home/root/.cache/pip/wheels/62/39/af/70e4f2f1154ae74c7689a115210902b4adcf8967f94253f878
Successfully built cbor
Installing collected packages: cbor
Successfully installed cbor-1.0.0
(desk) root@bora:~# pip3 install python-prctl
Collecting python-prctl
  Downloading python-prctl-1.8.1.tar.gz (28 kB)
  Preparing metadata (setup.py) ... - \ done
Building wheels for collected packages: python-prctl
  Building wheel for python-prctl (setup.py) ... - \ | / - \ | done
  Created wheel for python-prctl: filename=python_prctl-1.8.1-cp38-cp38-linux_armv7l.whl size=25341 sha256=c117e57bb506c338071debe433cc8f9ce559400a7f4c6d68fca9aea33812a5d1
  Stored in directory: /home/root/.cache/pip/wheels/d2/28/a0/76bcff726d677b487a8c926f51f4e672380ff8ca78c7d8d619
Successfully built python-prctl
Installing collected packages: python-prctl
Successfully installed python-prctl-1.8.1
(desk) root@bora:~# pip3 install netifaces
Collecting netifaces
  Downloading netifaces-0.11.0.tar.gz (30 kB)
  Preparing metadata (setup.py) ... - \ done
Building wheels for collected packages: netifaces
  Building wheel for netifaces (setup.py) ... - \ | / - \ | / - \ | / - \ | / - done
  Created wheel for netifaces: filename=netifaces-0.11.0-cp38-cp38-linux_armv7l.whl size=35204 sha256=e49fb06ca220a45c58c3c7df3c311eebd8968fd13e8a45038ad82620a3365b7a
  Stored in directory: /home/root/.cache/pip/wheels/f1/2a/15/7d0abf7b60244bd5d7b32699837118eac09cb3f9f305164a71
Successfully built netifaces
Installing collected packages: netifaces
Successfully installed netifaces-0.11.0
(desk) root@bora:~# pip3 install psutil
Collecting psutil
  Downloading psutil-5.9.3.tar.gz (483 kB)
     |████████████████████████████████|  483.6/483.6 kB 2.3 MB/s eta 0:00:00
  Installing build dependencies ... - \ | done
  Getting requirements to build wheel ... - \ | done
  Installing backend dependencies ... - \ | done
  Preparing metadata (pyproject.toml) ... - \ | / done
Building wheels for collected packages: psutil
  Building wheel for psutil (pyproject.toml) ... - \ | / - \ | / - \ | / - \ | / done
  Created wheel for psutil: filename=psutil-5.9.3-cp38-cp38-linux_armv7l.whl size=289747 sha256=95645d458a09ba46fff0a52d5957947310afc52520588259038f75ab2cb60012
  Stored in directory: /home/root/.cache/pip/wheels/8d/fc/1f/072e3caf8d3a4d56c497287c7741ad4c483d4adc65c9109abe
Successfully built psutil
Installing collected packages: psutil
Successfully installed psutil-5.9.3
(desk) root@bora:~# pip3 install requests
Collecting requests
  Downloading requests-2.28.1-py3-none-any.whl (62 kB)
     |████████████████████████████████|  62.8/62.8 kB 887.7 kB/s eta 0:00:00
Collecting idna<4,>=2.5
  Downloading idna-3.4-py3-none-any.whl (61 kB)
     |████████████████████████████████|  61.5/61.5 kB 830.4 kB/s eta 0:00:00
Collecting certifi>=2017.4.17
  Downloading certifi-2022.9.24-py3-none-any.whl (161 kB)
     |████████████████████████████████|  161.1/161.1 kB 1.2 MB/s eta 0:00:00
Collecting charset-normalizer<3,>=2
  Downloading charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
     |████████████████████████████████|  140.4/140.4 kB 2.1 MB/s eta 0:00:00
Installing collected packages: urllib3, idna, charset-normalizer, certifi, requests
Successfully installed certifi-2022.9.24 charset-normalizer-2.1.1 idna-3.4 requests-2.28.1 urllib3-1.26.12
(desk) root@bora:~# 

asyncua

Some packages may require Rust compiler for building the (for example) wheel:

• download the rust Linux installer

wget https://sh.rustup.rs

• execute the installer script:

root@bora:~# ./rustup-init.sh
info: downloading installer
Warning: using the BusyBox version of wget.  Not enforcing strong cipher suites for TLS or TLS v1.2, this is potentially less secure
Connecting to static.rust-lang.org (99.86.159.71:443)
wget: note: TLS certificate validation not implemented
saving to '/tmp/tmp.crML5188Vt/rustup-init'
rustup-init          100% |********************************| 13.5M  0:00:00 ETA
'/tmp/tmp.crML5188Vt/rustup-init' saved
$<2>
Welcome to Rust!
$<2>
This will download and install the official compiler for the Rust
programming language, and its package manager, Cargo.
...
...

$<2>
Rust is installed now. Great!
$<2>
To get started you may need to restart your current shell.
This would reload your $<2>PATH$<2> environment variable to include
Cargo's bin directory ($HOME/.cargo/bin).

To configure your current shell, run:
source "$HOME/.cargo/env"
root@bora:~# 

Once installed the related environment should be initialized:

root@bora:~# source "$HOME/.cargo/env"
root@bora:~# 

and again the Virtual environment activated (for installing the package inside the env):

root@bora:~# source env/bin/activate
root@bora:~# 

In this way, the asyncua package can be built/installed:

(desk) root@bora:~# python3 -m pip install asyncua
Collecting asyncua
  Downloading asyncua-1.0.0-py3-none-any.whl (747 kB)
     |████████████████████████████████| 747.6/747.6 kB 2.4 MB/s eta 0:00:00
Collecting cryptography
  Downloading cryptography-38.0.1.tar.gz (599 kB)
     |████████████████████████████████| 599.4/599.4 kB 2.2 MB/s eta 0:00:00
  Installing build dependencies ... - \ | / - \ | / - \ | / - \ | / done
  Getting requirements to build wheel ... - \ | / done
  Preparing metadata (pyproject.toml) ... - \ | / - done
Collecting pytz
  Downloading pytz-2022.5-py2.py3-none-any.whl (500 kB)
     |████████████████████████████████| 500.7/500.7 kB 2.3 MB/s eta 0:00:00
Collecting aiosqlite
  Downloading aiosqlite-0.17.0-py3-none-any.whl (15 kB)
Collecting python-dateutil
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
     |████████████████████████████████| 247.7/247.7 kB 940.5 kB/s eta 0:00:00
Collecting sortedcontainers
  Downloading sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting aiofiles
  Downloading aiofiles-22.1.0-py3-none-any.whl (14 kB)
Collecting typing_extensions>=3.7.2
  Using cached typing_extensions-4.4.0-py3-none-any.whl (26 kB)
Collecting cffi>=1.12
  Using cached cffi-1.15.1-cp38-cp38-linux_armv7l.whl
Collecting six>=1.5
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting pycparser
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Building wheels for collected packages: cryptography
  Building wheel for cryptography (pyproject.toml) ... | / - \ done
  Created wheel for cryptography: filename=cryptography-38.0.1-cp38-cp38-linux_armv7l.whl size=1768831 sha256=1cfd149510ee62605549bde5a24cb6bbd2f959094906ec6d22c4850fd2b6268d
  Stored in directory: /home/root/.cache/pip/wheels/fb/dd/30/687c60465a7b76d3b461f7eb89da28d4f68edc182d625a3ccf
Successfully built cryptography
Installing collected packages: sortedcontainers, pytz, typing_extensions, six, pycparser, aiofiles, python-dateutil, cffi, aiosqlite, cryptography, asyncua
Successfully installed aiofiles-22.1.0 aiosqlite-0.17.0 asyncua-1.0.0 cffi-1.15.1 cryptography-38.0.1 pycparser-2.21 python-dateutil-2.8.2 pytz-2022.5 six-1.16.0 sortedcontainers-2.4.0 typing_extensions-4.4.0
(desk) root@bora:~# 

paramiko

Some package may have some troubles too about the versioning of sub-package. For example, this is the case where paramiko cannot be installed due to some build errors, see Paramiko error while building pynacl.

This issue can be overcome installing a previous version for PyNaCl:

It occurs only with PyNaCl==1.5.0, so you can specify PyNaCl==1.4.0. It's OK because paramiko's requirement is PyNacl>=1.0.1

For example, the version 1.2.0 can be succesfully installed:

(desk) root@bora:~# pip3 install PyNaCl==1.2.0
Collecting PyNaCl==1.2.0
  Downloading PyNaCl-1.2.0.tar.gz (3.3 MB)
     |████████████████████████████████| 3.3/3.3 MB 2.4 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Requirement already satisfied: six in ./env/lib/python3.8/site-packages (from PyNaCl==1.2.0) (1.16.0)
Requirement already satisfied: cffi>=1.4.1 in ./env/lib/python3.8/site-packages (from PyNaCl==1.2.0) (1.15.1)
Requirement already satisfied: pycparser in ./env/lib/python3.8/site-packages (from cffi>=1.4.1->PyNaCl==1.2.0) (2.21)
Building wheels for collected packages: PyNaCl
  Building wheel for PyNaCl (setup.py) ... done
  Created wheel for PyNaCl: filename=PyNaCl-1.2.0-cp38-cp38-linux_armv7l.whl size=465695 sha256=b9d81dd6c93cd1b1527683500d2e6efbefaef1fadeddff5bef799cbba5d6dd47
  Stored in directory: /home/root/.cache/pip/wheels/c8/a7/4c/f03548f24ea252a853b28a9281ac3d3bb59107432fa5cd0a4e
Successfully built PyNaCl
Installing collected packages: PyNaCl
Successfully installed PyNaCl-1.2.0
(desk) root@bora:~# 

and the paramiko module is then installed:

(desk) root@bora:~# pip3 install paramiko
Collecting paramiko
  Downloading paramiko-2.11.0-py2.py3-none-any.whl (212 kB)
     |████████████████████████████████| 212.9/212.9 kB 1.5 MB/s eta 0:00:00
Collecting bcrypt>=3.1.3
  Downloading bcrypt-4.0.1.tar.gz (25 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: six in ./env/lib/python3.8/site-packages (from paramiko) (1.16.0)
Requirement already satisfied: pynacl>=1.0.1 in ./env/lib/python3.8/site-packages (from paramiko) (1.2.0)
Requirement already satisfied: cryptography>=2.5 in ./env/lib/python3.8/site-packages (from paramiko) (38.0.1)
Requirement already satisfied: cffi>=1.12 in ./env/lib/python3.8/site-packages (from cryptography>=2.5->paramiko) (1.15.1)
Requirement already satisfied: pycparser in ./env/lib/python3.8/site-packages (from cffi>=1.12->cryptography>=2.5->paramiko) (2.21)
Building wheels for collected packages: bcrypt
  Building wheel for bcrypt (pyproject.toml) ... done
  Created wheel for bcrypt: filename=bcrypt-4.0.1-cp38-cp38-linux_armv7l.whl size=597512 sha256=cc3884972a18b64377fab94c738fc4f2abf0c78ca4d6cb1eec41848fde710078
  Stored in directory: /home/root/.cache/pip/wheels/5c/32/05/1f093a4b4e1de66f1cff0a05ef844adbe89dcc027163712ee4
Successfully built bcrypt
Installing collected packages: bcrypt, paramiko
Successfully installed bcrypt-4.0.1 paramiko-2.11.0
(desk) root@bora:~# 

check for installed packages and version

At the end, the package installed can be listed:

(desk) root@bora:~# pip3 list
Package            Version
------------------ ---------
aiofiles           22.1.0
aiosqlite          0.17.0
asyncua            1.0.0
bcrypt             4.0.1
can                0.0.0
cbor               1.0.0
certifi            2022.9.24
cffi               1.15.1
charset-normalizer 2.1.1
cryptography       38.0.1
docutils           0.19
idna               3.4
lockfile           0.12.2
netifaces          0.11.0
ntplib             0.4.0
paramiko           2.11.0
pip                22.3
psutil             5.9.3
pycparser          2.21
pymodbus           3.0.0
PyNaCl             1.2.0
python-daemon      2.3.2
python-dateutil    2.8.2
python-prctl       1.8.1
pytz               2022.5
requests           2.28.1
setuptools         65.5.0
six                1.16.0
sortedcontainers   2.4.0
typing_extensions  4.4.0
urllib3            1.26.12
wheel              0.37.1
(desk) root@bora:~#

package import

The installed packages can be imported and used:

(env) root@bora:~# python3
Python 3.8.5 (default, Jul 20 2020, 13:26:22)
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncua
>>> import can
>>> import cbor
>>> import daemon
>>> import netifaces
>>> import ntplib
>>> import paramiko
>>> import psutil
>>> import pymodbus
>>> import prctl
>>> import requests
>>>