DESK-MX6-L-AN-0009: Comparing Java Virtual Machines

Info Box Applies to AXEL Lite Application Notes

History

Date	Development Kit version
September 2023	DESK-MX6L-L 4.0.1

Introduction

Nowadays the Java programming language has been mainly used to develop business server side application.

On the latest years it can be used on embedded devices too, thanks both to the constantly increasing processing power of embedded microprocessors and to the optimization of Java Virtual Machines

Java Virtual Machine

There are many different JVM that can run on ARM platform: the JVM let you to run your Java application on an Embedded platform like the DAVE Embedded Systems' SBC-AXEL platforms.

In this Application Note, we would like to explore some of the most popular Java Virtual Machines and measure their performances on the embedded platform. The different JVMs performance results let the user to target which JVM can be the best selection for running its Java code.

Available Java Virtual Machines

There's tons of JVM available in the market, both commercials and free/opensource ones: the JVM listed below have been selected from the main Java players in the market.

Obviously the Oracle JDK is the reference one due to the original Java project started by Sun Microsystems and acquired by Oracle on 2010.

Historically Sun/Oracle provides different flavors of their JVM, depending on the target host (desktop vs server vs embedded).

Oracle Java SE Embedded was the flavor targetting embbedded market, but Oracle announced that Java 8 is the final major Oracle Java SE Embedded Product.

Here the blog announce:

Accordingly, starting with JDK 9, Oracle doesn't plan to offer a separate "Oracle Java SE Embedded" download. In other words, Oracle Java SE 8 Embedded is the final release series of the "Oracle Java SE Embedded" product. Java SE 8 Embedded is now on restricted availability intended for existing embedded support customers only.

For the The Oracle Java License, please find more information on Oracle Java web site.

Newest JVMs based on Java 11-LTS, 17-LTS and 21 can be found from the OpenJDK JVM players.

Selection of embedded Virtual Machine

Here below is the list of JVMs compared in the tests and the binary download sites:

Java Virtual Machine	JDK version	runtime version	VM version
Oracle JDK	java version "1.8.0_371"	Java(TM) SE Runtime Environment (build 1.8.0_371-b11)	Java HotSpot(TM) Client VM (build 25.371-b11, mixed mode)
Eclipse Temurin OpenJDK	openjdk version "17.0.8.1" 2023-08-24	OpenJDK Runtime Environment Temurin-17.0.8.1+1 (build 17.0.8.1+1)	OpenJDK Server VM Temurin-17.0.8.1+1 (build 17.0.8.1+1, mixed mode, sharing)
BellSoft Liberica Standard JDK	openjdk version "17.0.8.1" 2023-08-24 LTS	OpenJDK Runtime Environment (build 17.0.8.1+1-LTS)	OpenJDK 32-Bit Server VM (build 17.0.8.1+1-LTS, mixed mode)
Azul Zulu	openjdk version "17.0.8.1" 2023-08-24	OpenJDK Runtime Environment Zulu17.44+53-CA (build 17.0.8.1+1)	OpenJDK Server VM Zulu17.44+53-CA (build 17.0.8.1+1, mixed mode)

Licensing

Each JVM has a proper licensing model and this has to be taken into account:

• Oracle: the Java license agreement should be accomplished in order to use and evaluate it
• OpenJDK, Eclipse Temurin OpenJDK: GPLv2
• Zulu OpenJDK: see Azul builds of OpenJDK (Zulu Builds of OpenJDK is copyrighted software based on OpenJDK (http://openjdk.java.net/) and is 100% open source, cit)
• BellSoft Liberica: Liberica JDK EULA (Liberica JDK is copyrighted software based on OpenJDK and is 100% Open Source Software, cit.)

Benchmarks

Two standard Java code benchmark tools are used to compare the JVMs:

• SciMark 2.0
• CaffeineMark 3.0

Licensing

We have to comply to the proper license agreements in order to run the benchmarks:

• SciMark: please read the credits page about National Institute of Standards and Technology (NIST) GPL license
• CaffeineMark: according to the Pendragon Software's CaffeineMark™ ver. 3.0
  • the test was performed without independent verification by Pendragon Software and that Pendragon Software makes no representations or warranties as to the result of the test

Test results

The benchmarks have been run on the same platform and same cpu: the different JVMs was decompressed in the /home/root/java directory of the target and started manually.

DESK-MX6-L-4.0.1

The following tests have been performed on SBC-AXEL equipped with:

• AXEL Lite DXLM0193C3R SOM
• i.MX6 Dual Lite processor @ 792MHz
• 1GB DDR
• boot from SD card (u-boot, kernel and root file system)

Java Virtual Machine	SciMark 2.0 (Composite Score)	CaffeineMark 3.0 (Float score) (*)
Oracle JDK 8	69.73	16014
Temurin OpenJDK	83.47	26362
LibericaJDK	82.82	26413
Zulu	82.88	26380

(*) Float score has been used because of the Overall score for some JVM hasn't been correctly calculated

Here below an example of SciMark benchmark started for Adoptium Temurin OpenJDK on:

root@desk-mx6:~# ./Temurin-jdk-17.0.8.1+1/bin/java -cp ./benchmark/scimark2lib.jar jnt.scimark2.commandline

SciMark 2.0a

Composite Score: 83.46940387379361
FFT (1024): 71.79126909610153
SOR (100x100):   146.29175660834022
Monte Carlo : 52.44928769038194
Sparse matmult (N=1000, nz=5000): 60.77151649169519
LU (100x100): 86.04318948244916

java.vendor: Eclipse Adoptium
java.version: 17.0.8.1
os.arch: arm
os.name: Linux
os.version: 5.15.71-desk-mx6-l-4.0.1+g29cc6fd23cd1
root@desk-mx6:~#

Then, an example of Caffeine benchmark:

root@desk-mx6:~# ./Temurin-jdk-17.0.8.1+1/bin/java -cp ./benchmark/Caffeine CaffeineMarkEmbeddedApp
Sieve score = 18224 (98)
Loop score = 87635 (2017)
Logic score = 10137743 (0)
String score = 38923 (708)
Float score = 33960 (185)
Method score = 26362 (166650)
Overall score = 90901
root@desk-mx6:~#

OpenJDK 11

As a comparison, an OpenJDK 11 has been used for understanding if a major difference has been introduced on OpenJDK 17:

root@desk-mx6:~# ./Temurin-jdk-11.0.20.1+1/bin/java -cp ./benchmark/scimark2lib.jar jnt.scimark2.commandline

SciMark 2.0a

Composite Score: 82.46389830912622
FFT (1024): 69.23787500115992
SOR (100x100):   145.10203665259215
Monte Carlo : 53.43062241875944
Sparse matmult (N=1000, nz=5000): 60.43526450567323
LU (100x100): 84.11369296744635

java.vendor: Eclipse Adoptium
java.version: 11.0.20.1
os.arch: arm
os.name: Linux
os.version: 5.15.71-desk-mx6-l-4.0.1+g29cc6fd23cd1
root@desk-mx6:~#

As depicted above, there are no important differences on the JVM performance (using the benchmarks selected).

Copyright and Trademark Information

Java™ and HotJava™ are trademarks of Sun Microsystems, Inc. and refer to Sun's Java programming language and HotJava browser technologies.

Java™ and OpenJDK™ are trademarks or registered trademarks of Oracle and/or its affiliates.

Azul Zulu™ is a trademark and Copyright © of Azul Systems

Liberica is a Copyright © of BellSoft Corporation

Adoptium OpenJDK is a Copyright © of Eclipse Foundation.

The CaffeineMark™ benchmark is a trademark and Copyright © of Pendragon Software