MISC-TN-025: Running Phoronix Test Suite on SBCSPG IIoT gateway

From DAVE Developer's Wiki
Jump to: navigation, search
Info Box
SBCSPG-1.jpg Applies to SBCSPG
Cloud-computing-banner.jpg Applies to IoT
Warning-icon.png This technical note was validated against specific versions of hardware and software. It may not work with other versions. Warning-icon.png


History[edit | edit source]

Version Date Notes
1.0.0 January 2023 First public release

Introduction[edit | edit source]

This Technical Note shows how to run Phoronix Test Suite on the SBCSBP gateway. For the sake of simplicity, the Debian Stretch distribution was used.

This test suite is convenient to estimate the performance of a target platform or to compare different platforms. The suite is extremely modular and tons of tests are available.

Setting up the target[edit | edit source]

The test suite is released as a architecture-independent package. As such, the installation is straightforward once the required PHP packages are installed onto the target.

The installation package is available here.

Results[edit | edit source]

For the sake of simplicity, the smallpt suite was used to compare the results depending on the selected governor.

Governor: interactive[edit | edit source]

armbian@sbcspg:~$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors
conservative ondemand userspace powersave interactive performance 
armbian@sbcspg:~$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
198000 396000 528000 
armbian@sbcspg:~$ /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
-bash: /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor: Permission denied
armbian@sbcspg:~$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
interactive

armbian@sbcspg:~$ phoronix-test-suite benchmark smallpt


NOTICE: The following PHP extensions are OPTIONAL but recommended:

GD        The GD library is recommended for improved graph rendering.
Bzip2     The bzcompress/bzip2 support can be used for greater file compression.
SQLite3   SQLite3 is required when running a Phoromatic server.
CURL      CURL is recommended for an enhanced download experience.

Updated OpenBenchmarking.org Repository Index
pts: 510 Distinct Tests, 2169 Test Versions, 56 Suites
Available Changes From 3 July 2022 To 12 January
Updated Test:   pts/ai-benchmark        v1.0.2  AI Benchmark Alpha                  
Updated Test:   pts/aircrack-ng         v1.3.0  Aircrack-ng                         
Updated Test:   pts/aom-av1             v3.5.0  AOM AV1                             
Updated Test:   pts/astcenc             v1.4.0  ASTC Encoder                        
Updated Test:   pts/avifenc             v1.3.0  libavif avifenc                     
Updated Test:   pts/batman-knight       v1.0.1  Batman: Arkham Knight               
Updated Test:   pts/blender             v3.4.0  Blender                             
Updated Test:   pts/blosc               v1.2.0  C-Blosc                             
Updated Test:   pts/brl-cad             v1.4.0  BRL-CAD                             
Updated Test:   pts/build-erlang        v1.2.0  Timed Erlang/OTP Compilation        
Updated Test:   pts/build-linux-kernel  v1.15.0 Timed Linux Kernel Compilation      
Updated Test:   pts/build-nodejs        v1.2.0  Timed Node.js Compilation           
Updated Test:   pts/build-php           v1.6.0  Timed PHP Compilation               
New Test:       pts/build-python        v1.0.0  Timed CPython Compilation           
Updated Test:   pts/build-wasmer        v1.2.0  Timed Wasmer Compilation            
Updated Test:   pts/clickhouse          v1.2.0  ClickHouse                          
Updated Test:   pts/cloudsuite-da       v1.1.0  CloudSuite Data Analytics           
Updated Test:   pts/cloudsuite-ga       v1.0.1  CloudSuite Graph Analytics          
Updated Test:   pts/cloudsuite-ma       v1.0.1  CloudSuite In-Memory Analytics      
Updated Test:   pts/cloudsuite-ms       v1.0.1  CloudSuite Media Streaming          
New Test:       pts/cockroach           v1.0.2  CockroachDB                         
Updated Test:   pts/compress-7zip       v1.10.0 7-Zip Compression                   
Updated Test:   pts/couchdb             v1.2.0  Apache CouchDB                      
Updated Test:   pts/cpuminer-opt        v1.6.0  Cpuminer-Opt                        
Updated Test:   pts/csgo                v1.7.2  Counter-Strike: Global Offensive    
New Test:       pts/cyberpunk2077       v1.0.1  Cyberpunk 2077                      
Updated Test:   pts/ddnet               v1.3.0  DDraceNetwork                       
New Test:       pts/deepsparse          v1.0.1  Neural Magic DeepSparse             
New Test:       pts/dragonflydb         v1.0.0  Dragonflydb                         
Updated Test:   pts/encode-flac         v1.8.1  FLAC Audio Encoding                 
New Test:       pts/encodec             v1.0.1  EnCodec                             
New Test:       pts/etcd                v1.0.0  etcd                                
New Test:       pts/f122                v1.0.1  F1 22                               
Updated Test:   pts/ffmpeg              v3.0.0  FFmpeg                              
New Test:       pts/fluidx3d            v1.0.0  FluidX3D                            
Updated Test:   pts/graphics-magick     v2.1.0  GraphicsMagick                      
Updated Test:   pts/gravitymark         v1.7.0  GravityMark                         
New Test:       pts/gta5                v1.0.0  Grand Theft Auto V                  
Updated Test:   pts/hammerdb-mariadb    v1.1.0  HammerDB - MariaDB                  
Updated Test:   pts/hammerdb-postgresql v1.1.1  HammerDB - PostgreSQL               
Updated Test:   pts/hbase               v1.1.0  Apache HBase                        
Updated Test:   pts/hitman              v1.0.2  HITMAN                              
Updated Test:   pts/hitman3             v1.0.1  HITMAN 3                            
Updated Test:   pts/intel-mlc           v1.1.0  Intel Memory Latency Checker        
Updated Test:   pts/jpegxl              v1.5.0  JPEG XL libjxl                      
Updated Test:   pts/jpegxl-decode       v1.5.0  JPEG XL Decoding libjxl             
Updated Test:   pts/kvazaar             v1.2.0  Kvazaar                             
Updated Test:   pts/l4d2                v1.0.1  Left 4 Dead 2                       
Updated Test:   pts/lammps              v1.4.0  LAMMPS Molecular Dynamics Simulator 
Updated Test:   pts/libplacebo          v1.1.0  Libplacebo                          
New Test:       pts/memcached           v1.0.0  Memcached                           
Updated Test:   pts/memtier-benchmark   v1.4.1  memtier_benchmark                   
New Test:       pts/minibude            v1.0.0  miniBUDE                            
Updated Test:   pts/mnn                 v2.1.0  Mobile Neural Network               
Updated Test:   pts/natron              v1.1.0  Natron                              
Updated Test:   pts/ncnn                v1.4.0  NCNN                                
New Test:       pts/nekrs               v1.0.0  nekRS                               
Updated Test:   pts/nginx               v3.0.0  nginx                               
Updated Test:   pts/node-web-tooling    v1.0.1  Node.js V8 Web Tooling Benchmark    
Updated Test:   pts/numenta-nab         v1.1.1  Numenta Anomaly Benchmark           
Updated Test:   pts/onednn              v3.0.0  oneDNN                              
Updated Test:   pts/openfoam            v1.2.0  OpenFOAM                            
New Test:       pts/openradioss         v1.0.0  OpenRadioss                         
Updated Test:   pts/openvino            v1.2.0  OpenVINO                            
Updated Test:   pts/openvkl             v1.3.0  OpenVKL                             
Updated Test:   pts/pgbench             v1.13.0 PostgreSQL                          
Updated Test:   pts/portal2             v1.1.2  Portal 2                            
Updated Test:   pts/primesieve          v1.9.0  Primesieve                          
New Test:       pts/quadray             v1.0.0  QuadRay                             
Updated Test:   pts/rav1e               v1.7.0  rav1e                               
Updated Test:   pts/redis               v1.4.0  Redis                               
Updated Test:   pts/rocksdb             v1.3.0  Facebook RocksDB                    
Updated Test:   pts/schbench            v1.1.0  Schbench                            
Updated Test:   pts/scikit-learn        v1.2.0  Scikit-Learn                        
Updated Test:   pts/smhasher            v1.1.0  SMHasher                            
New Test:       pts/spacy               v1.0.0  spaCy                               
New Test:       pts/spark               v1.0.0  Apache Spark                        
Updated Test:   pts/spec-jbb2015        v1.1.0  SPECjbb 2015                        
Updated Test:   pts/srsran              v1.2.0  srsRAN                              
Updated Test:   pts/stargate            v1.1.0  Stargate Digital Audio Workstation  
Updated Test:   pts/strange-brigade     v1.0.1  Strange Brigade                     
Updated Test:   pts/stream              v1.3.4  Stream                              
Updated Test:   pts/stress-ng           v1.6.0  Stress-NG                           
Updated Test:   pts/svt-av1             v2.7.0  SVT-AV1                             
Updated Test:   pts/tensorflow          v2.0.0  TensorFlow                          
Updated Test:   pts/tf2                 v1.2.4  Team Fortress 2                     
Updated Test:   pts/tww3                v1.0.1  Total War: WARHAMMER III            
Updated Test:   pts/unigine-super       v1.0.8  Unigine Superposition               
Updated Test:   pts/unpack-linux        v1.2.0  Unpacking The Linux Kernel          
Updated Test:   pts/unvanquished        v1.7.0  Unvanquished                        
New Test:       pts/uvg266              v1.0.0  uvg266                              
Updated Test:   pts/webp                v1.2.0  WebP Image Encode                   
Updated Test:   pts/webp2               v1.2.0  WebP2 Image Encode                  
New Test:       pts/will-it-scale       v1.0.0  will-it-scale                       
Updated Test:   pts/xmrig               v1.1.0  Xmrig                               
New Test:       pts/xplane12            v1.0.1  X-Plane                             
Updated Test:   pts/y-cruncher          v1.2.0  Y-Cruncher                          
Updated Suite:  pts/compilation         v1.2.8  Timed Code Compilation              
Updated Suite:  pts/database            v1.3.7  Database Test Suite                 
Updated Suite:  pts/hpc                 v1.1.7  HPC - High Performance Computing    
Updated Suite:  pts/machine-learning    v1.3.5  Machine Learning                    
Updated Suite:  pts/opencl              v1.1.1  OpenCL                              
Updated Suite:  pts/raytracing          v1.0.3  Raytracing                          
Updated Suite:  pts/steam               v1.0.9  Steam                               
Updated Suite:  pts/video-encoding      v1.3.3  Video Encoding                      
Updated OpenBenchmarking.org Repository Index
system: 41 Distinct Tests, 126 Test Versions
Available Changes From 3 July 2022 To 12 January
Updated Test:  system/inkscape v1.0.1  Inkscape 
Updated Test:  system/selenium v1.0.32 Selenium 
Updated OpenBenchmarking.org Repository Index
git: 8 Distinct Tests, 11 Test Versions
    Evaluating External Test Dependencies .........................................................................................

Phoronix Test Suite v10.8.4

    To Install:    pts/smallpt-1.2.1

    Determining File Requirements .................................................................................................
    Searching Download Caches .....................................................................................................

    1 Test To Install
        1 File To Download [0.01MB]
        1MB Of Disk Space Is Needed
        4 Seconds Estimated Install Time

    pts/smallpt-1.2.1:
        Test Installation 1 of 1
        1 File Needed [0.01 MB]
        Downloading: smallpt-1.tar.gz                                                                                      [0.01MB]
        Downloading ...............................................................................................................
        Approximate Install Size: 0.1 MB
        Estimated Install Time: 4 Seconds
        Installing Test @ 13:05:03


System Information


  PROCESSOR:              ARMv7 Cortex-A7 @ 0.53GHz
    Core Count:           1                         
    Scaling Driver:       imx6q-cpufreq interactive 

  GRAPHICS:               

  MOTHERBOARD:            Freescale i.MX6 Ultralite

  MEMORY:                 494MB

  DISK:                   8GB SA08G
    File-System:          ext4                     
    Mount Options:        data=ordered relatime rw 
    Disk Details:         Block Size: 4096         

  OPERATING SYSTEM:       Debian 9.9
    Kernel:               4.1.15-rialto-1.1.3 (armv7l) 
    Compiler:             GCC 6.3.0 20170516           

    Would you like to save these test results (Y/n): 
    Enter a name for the result file: benchmark smallp
    Enter a unique name to describe this test run / configuration: 20230112 benchmark smallp

If desired, enter a new description below to better describe this result set / system configuration under test.
Press ENTER to proceed without changes.

Current Description: ARMv7 Cortex-A7 testing on Debian 9.9 via the Phoronix Test Suite.

New Description: 

        [Performance Tip] The CPU scaling governor is currently not set to performance. It's possible to obtain greater performance
        if using the performance governor.

        To change behavior, run: 

        echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

        Reference: https://openbenchmarking.org/result/1706268-TR-CPUGOVERN32


        To stop showing performance tips, run: phoronix-test-suite unload-module perf_tips

        Continuing in 5 seconds or press CTRL-C to stop the testing process.

Smallpt 1.0:
    pts/smallpt-1.2.1
    Test 1 of 1
    Estimated Trial Run Count:    3                     
    Estimated Time To Completion: 5 Minutes [13:10 CET] 
        Started Run 1 @ 13:06:25
        Started Run 2 @ 14:13:36
        Started Run 3 @ 15:20:46

    Global Illumination Renderer; 128 Samples:
        4027.47
        4025.968
        4028.022

    Average: 4027.153 Seconds
    Deviation: 0.03%

    Do you want to view the text results of the testing (Y/n): 
benchmark smallp
ARMv7 Cortex-A7 testing on Debian 9.9 via the Phoronix Test Suite.


20230112 benchmark smallp: 

	Processor: ARMv7 Cortex-A7 @ 0.53GHz (1 Core), Motherboard: Freescale i.MX6 Ultralite (Device Tree) UltraLite CB0046 SBCSPG rev.A, Memory: 494MB, Disk: 8GB SA08G

	OS: Debian 9.9, Kernel: 4.1.15-rialto-1.1.3 (armv7l), Compiler: GCC 6.3.0 20170516, File-System: ext4


    Smallpt 1.0
    Global Illumination Renderer; 128 Samples
    Seconds < Lower Is Better
    20230112 benchmark smallp . 4027.15 |==============================================================================================

    Would you like to upload the results to OpenBenchmarking.org (y/n): n

Governor: performance[edit | edit source]

...
Do you want to view the text results of the testing (Y/n): 
benchmark-smallp-governor-performance
ARMv7 Cortex-A7 testing on Debian 9.9 via the Phoronix Test Suite.


20230113-benchmark-smallp-governor-performance: 

	Processor: ARMv7 Cortex-A7 @ 0.53GHz (1 Core), Motherboard: Freescale i.MX6 Ultralite (Device Tree) UltraLite CB0046 SBCSPG rev.A, Memory: 494MB, Disk: 8GB SA08G

	OS: Debian 9.9, Kernel: 4.1.15-rialto-1.1.3 (armv7l), Compiler: GCC 6.3.0 20170516, File-System: ext4


    Smallpt 1.0
    Global Illumination Renderer; 128 Samples
    Seconds < Lower Is Better
    20230113-benchmark-smallp-governor-performance . 4073.91

Governor: userspace[edit | edit source]

armbian@sbcspg:~$ phoronix-test-suite benchmark smallpt

    Evaluating External Test Dependencies .........................................................................................

Phoronix Test Suite v10.8.4

    Installed:     pts/smallpt-1.2.1

System Information


  PROCESSOR:              ARMv7 Cortex-A7 @ 0.53GHz
    Core Count:           1                       
    Scaling Driver:       imx6q-cpufreq userspace 

  GRAPHICS:               

  MOTHERBOARD:            Freescale i.MX6 Ultralite

  MEMORY:                 494MB

  DISK:                   8GB SA08G
    File-System:          ext4                     
    Mount Options:        data=ordered relatime rw 
    Disk Details:         Block Size: 4096         

  OPERATING SYSTEM:       Debian 9.9
    Kernel:               4.1.15-rialto-1.1.3 (armv7l) 
    Compiler:             GCC 6.3.0 20170516           

    Would you like to save these test results (Y/n): 

    Recently Saved Test Results:
       benchmark-smallp-governor-performance   [Today]
       benchmark-smallp                        [Today]

    Enter a name for the result file: benchmark-smallp-governor-userspace-528MHz
    Enter a unique name to describe this test run / configuration: 20230113-benchmark-smallp-governor-userspace-528MHz

If desired, enter a new description below to better describe this result set / system configuration under test.
Press ENTER to proceed without changes.

Current Description: ARMv7 Cortex-A7 testing on Debian 9.9 via the Phoronix Test Suite.

New Description: 

        [Performance Tip] The CPU scaling governor is currently not set to performance. It's possible to obtain greater performance
        if using the performance governor.

        To change behavior, run: 

        echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

        Reference: https://openbenchmarking.org/result/1706268-TR-CPUGOVERN32


        To stop showing performance tips, run: phoronix-test-suite unload-module perf_tips

        Continuing in 5 seconds or press CTRL-C to stop the testing process.

Smallpt 1.0:
    pts/smallpt-1.2.1
    Test 1 of 1
    Estimated Trial Run Count:    3                               
    Estimated Time To Completion: 3 Hours, 23 Minutes [18:30 CET] 
        Started Run 1 @ 15:08:20
        Started Run 2 @ 16:15:33
        Started Run 3 @ 17:22:37

    Global Illumination Renderer; 128 Samples:
        4028.008
        4020.172
        4019.183

    Average: 4022.454 Seconds
    Deviation: 0.12%

    Do you want to view the text results of the testing (Y/n): 
benchmark-smallp-governor-userspace-528MHz
ARMv7 Cortex-A7 testing on Debian 9.9 via the Phoronix Test Suite.


20230113-benchmark-smallp-governor-userspace-528MHz: 

	Processor: ARMv7 Cortex-A7 @ 0.53GHz (1 Core), Motherboard: Freescale i.MX6 Ultralite (Device Tree) UltraLite CB0046 SBCSPG rev.A, Memory: 494MB, Disk: 8GB SA08G

	OS: Debian 9.9, Kernel: 4.1.15-rialto-1.1.3 (armv7l), Compiler: GCC 6.3.0 20170516, File-System: ext4


    Smallpt 1.0
    Global Illumination Renderer; 128 Samples
    Seconds < Lower Is Better
    20230113-benchmark-smallp-governor-userspace-528MHz . 4022.45 |====================================================================

    Would you like to upload the results to OpenBenchmarking.org (y/n): n

Conclusion[edit | edit source]

As detailed in the following table, the type of governor has basically no influence on the result.

Governor Result
interactive (default) 4027.15
performance 4073.91
userspace (528 MHz) 4022.45