XELK-AN-009: Comparing JVMs (Java Virtual Machines) on i.MX6 family processors

Info Box Applies to Axel Ultra Applies to AXEL ESATTA Applies to Axel Lite

This application note has been validated using the kit version in the History table.

History

Processor	Date	Kit version	Notes
i.MX6 Cortex-A9	Nov 2019	XELK 4.0.0	i.MX6 DualLite SOC version
i.MX6UL Cortex-A7	Nov 2019	XUELK 2.0.3	i.MX6 Ultra Lite SOC version

Introduction

Nowadays the Java programming language is more and more used for creating embedded application taking advantages from the popular and powerful programming language like Java.

As well known, from wikipedia description:

Java is a general-purpose programming language that is class-based, object-oriented, and designed to have as few implementation dependencies as possible. It is intended to let application developers write once, run anywhere: Java code can run on all platforms that support Java without the need for recompilation.
Java applications are typically compiled to bytecode that can run on any Java virtual machine (JVM) regardless of the underlying computer architecture.

Java Virtual Machine

There are many different JVM that can run on ARM platform - like - let you to run your Java application on DAVE's Embedded platforms.

In this Application Note, we would like to explore some of the most popular Java Virtual Machines and measure their performances on the embedded platform. The different JVMs performance results let the user to target which JVM can be used for running its Java code in the better way.

ZeroVM

The ZeroVM JDK version has to be taken into account for some JVM version: this version - used for example on OpenJDK 8 and AdoptOpenJDK 8 give output results are really less performing then the others VMs.

It should be considered - infact - that OpenJDK Zero VM is an interpreter only JVM. It's easier to port because it has no architecture specific assembly code but, on the other hand, it's not performant because it has no architecture specific assembly code.

Benchmarks

Two standard Java code benchmark tools are used to compare the JVMs:

• SciMark 2.0
• CaffeineMark 3.0

We have to comply to the proper license agreements in order to run the benchmarks:

• SciMark: please read the credits page about National Institute of Standards and Technology (NIST) GPL license
• CaffeineMark: please read carefully the license agreement for running the Pendragon Software's CaffeineMark™ ver. 3.0

JVMs

The following Java Virtual Machines have been selected for the main Java player in the market. Obviously the Oracle JDK is the reference one due to the original Java project started by Sun Microsystems and aquired by Oracle on 2010.

The Java license agreement should be accomplished in order to use and evaluate the Oracle

• Oracle JDK SE 8

Virtual Machines

Here below the list of JVMs compared in the tests and the download sites:

Java Virtual Machine	JDK version	VM version	Notes
Oracle JDK	8u211	Java HotSpot(TM) Client VM (build 25.211-b12, mixed mode)
OpenJDK	1.8.0_102	OpenJDK Zero VM (build 25.102-b14, interpreted mode)	ZeroVM
AdoptOpenJDK	1.8.0_232	OpenJDK Zero VM (AdoptOpenJDK)(build 25.232-b09, interpreted mode)	ZeroVM
ZuluCommunity	8u232	OpenJDK Client VM (Zulu8.42.0.195-CA-linux_aarch32hf) (build 25.232-b195, mixed mode, Evaluation)
AdoptOpenJDK	11.0.5	OpenJDK Server VM AdoptOpenJDK (build 11.0.5+10, mixed mode)
LibericaJDK	11.0.5-BellSoft	OpenJDK 32-Bit Server VM (build 11.0.5-BellSoft+11, mixed mode)
ZuluCommunity	11.0.4	OpenJDK Client VM Zulu11.33+21-CA (build 11.0.4+10-LTS, mixed mode)

Benchmarking results

The benchmarks have been run on the same platform and same cpu: the different JVMs was decompressed in the /home/root/java directory of the target and started manually.

Here below an example of SciMark benchmark started for OracleJDK:

root@imx6qxelk:~/java# ./jdk1.8.0_211/bin/java -cp ./scimark2lib.jar jnt.scimark2.commandline
SciMark 2.0a

Composite Score: 69.37463986126829
FFT (1024): 69.63592899407837
SOR (100x100):   123.68132332709474
Monte Carlo : 17.99165245491594
Sparse matmult (N=1000, nz=5000): 50.866191316811516
LU (100x100): 84.69810321344092

java.vendor: Oracle Corporation
java.version: 1.8.0_211
os.arch: arm
os.name: Linux
os.version: 4.9.11-xelk-4.0.0
root@imx6qxelk:~/java#

Then, an example of Caffeine benchmark:

root@imx6qxelk:~/java/benchmark/Caffeine# ../../jdk1.8.0_211/bin/java CaffeineMarkEmbeddedApp
Sieve score = 12204 (98)
Loop score = 46042 (2017)
Logic score = 22955 (0
String score = 29983 (708)
Float score = 21900 (185)
Method score = 16041 (166650)
Overall score = 22673
root@imx6qxelk:~/java/benchmark/Caffeine#

List of test results

Java Virtual Machine	SciMark (Composite Score)	CaffeineMark (Overall score)	Notes
Oracle JDK	69.37	22673
OpenJDK 8	3.62	605	The interpreted JVM version explains the very poor performance
AdoptOpenJDK 8	3.44	569	The interpreted JVM version explains the very poor performance
ZuluCommunity 8	69.14	17237
AdoptOpenJDK 11	82.39	78111	Top performance JVM
LibericaJDK 11	82.55	77083	Top performance JVM
ZuluCommunity 11	69.61	17060