Difference between revisions of "XELK-AN-009: Comparing JVMs (Java Virtual Machines) on i.MX6 family processors"

From DAVE Developer's Wiki
Jump to: navigation, search
(Created page with "{{InfoBoxTop}} {{AppliesToAxel}} {{AppliesToAxelEsatta}} {{AppliesToAxelLite}} {{InfoBoxBottom}} {{ImportantMessage|text=This application note has been validated using the ''...")
(43 intermediate revisions by 3 users not shown)
Line 3: Line 3:
 
{{AppliesToAxelEsatta}}
 
{{AppliesToAxelEsatta}}
 
{{AppliesToAxelLite}}
 
{{AppliesToAxelLite}}
{{AppliesToAXEL Lite AN}}
 
{{AppliesToSMARX AN}}
 
{{AppliesToAXELULite}}
 
{{AppliesToAXEL ULite AN}}
 
{{AppliesToSBCLynx}}
 
{{AppliesTo SBC Lynx AN}}
 
{{AppliesTo RIALTO SBC AN}}
 
 
{{InfoBoxBottom}}
 
{{InfoBoxBottom}}
  
Line 16: Line 9:
  
 
{| class="wikitable" border="1"
 
{| class="wikitable" border="1"
!Version
+
!Processor
 
!Date
 
!Date
!Development Kit version
+
!Kit version
 +
!Notes
 
|-
 
|-
| 1.0.0
+
|i.MX6 Cortex-A9
| Nov 2019
+
|Nov 2019
|[[Axel_Embedded_Linux_Kit_(XELK)#XELK_4.0.0|XELK 4.0.0]] , [[AXEL_ULite_and_SBC_Lynx_Embedded_Linux_Kit_(XUELK)#XUELK_2.0.1.2F2.0.2.2F2.0.3|XUELK 2.0.3]]
+
|[[Axel_Embedded_Linux_Kit_(XELK)#XELK_4.0.0|XELK 4.0.0]]
 +
|i.MX6 DualLite SOC version
 +
|-
 +
|i.MX6UL Cortex-A7
 +
|Nov 2019
 +
|[[AXEL_ULite_and_SBC_Lynx_Embedded_Linux_Kit_(XUELK)#XUELK_2.0.1.2F2.0.2.2F2.0.3|XUELK 2.0.3]]
 +
| i.MX6 Ultra Lite SOC version
 
|-
 
|-
 
|}
 
|}
  
 
==Introduction==
 
==Introduction==
Nowadays the [https://en.wikipedia.org/wiki/Java_(programming_language) Java programming language] has been mainly used to develop business server side application.
+
Nowadays the Java programming language is more and more used for creating embedded application taking advantages from the popular and powerful programming language like [https://en.wikipedia.org/wiki/Java_(programming_language) Java].
 
 
On the latest years it can be used on embedded devices too, thanks both to the constantly increasing processing power of embedded microprocessors and to the optimization of Java Virtual Machines
 
 
 
=== Java Virtual Machine ===
 
There are many different JVM that can run on ARM platform: the JVM let you to run your Java application on an Embedded platform like the DAVE's [[:Category:SBC-AXEL | SBC-AXEL]] and [[:Category:SBC_Lynx | SBC-Lynx]] platforms.
 
  
In this Application Note, we would like to explore some of the most popular Java Virtual Machines and measure their performances on the embedded platform.
+
As well known, from [https://en.wikipedia.org/wiki/Java_(programming_language) wikipedia] description:
The different JVMs performance results let the user to target which JVM can be the best selection for running its Java code.
 
 
 
==== ZeroVM====
 
The [https://openjdk.java.net/projects/zero/ ZeroVM] JDK version has to be taken into account if used on some JVM: this version - used for example on ''OpenJDK 8'' and ''AdoptOpenJDK 8'' - gives output results '''really less performing''' then the others VMs.
 
 
 
It should be considered - infact - that ''OpenJDK Zero VM is an interpreter only JVM''. It's easier to port because it has no architecture specific assembly code but, on the other hand,
 
it's not performant because it has no architecture specific assembly code.
 
 
 
== Available Java Virtual Machines ==
 
There's tons of JVM available in the market, both commercials and free/opensource ones: the JVM listed below have been selected from the main Java players in the market.
 
 
 
Obviously the <b>Oracle JDK</b> is the reference one due to the original Java project started by [https://en.wikipedia.org/wiki/Java_(programming_language) Sun Microsystems] and acquired by [https://en.wikipedia.org/wiki/Sun_acquisition_by_Oracle Oracle] on 2010.
 
 
 
Historically Sun/Oracle provides different flavors of their JVM, depending on the target host (desktop vs server vs embedded.
 
 
 
[https://www.oracle.com/java/technologies/javase-embedded-downloads.html Oracle Java SE Embedded] was the flavor targetting embbedded market, but Oracle announced that '''Java 8''' is ''the final major Oracle Java SE Embedded Product''.
 
 
 
[https://blogs.oracle.com/java-platform-group/convergence-of-oracle-java-se-embedded-with-oracle-jdk Here] the blog announce:
 
  
 
<pre>
 
<pre>
Accordingly, starting with JDK 9, Oracle doesn't plan to offer a separate "Oracle Java SE Embedded" download. In other words, Oracle Java SE 8 Embedded is the final release series of the "Oracle Java SE Embedded" product.  New features in JDK 9 obviate the need for it to be a separate product.
+
Java is a general-purpose programming language that is class-based, object-oriented, and designed to have as few implementation dependencies as possible. It is intended to let application developers write once, run anywhere: Java code can run on all platforms that support Java without the need for recompilation.
 +
Java applications are typically compiled to bytecode that can run on any Java virtual machine (JVM) regardless of the underlying computer architecture.
 
</pre>
 
</pre>
  
For the ''The Oracle Java License'', please find more information on Oracle [https://java.com/en/download/release_notice.jsp Java web site].
+
=== Java Virtual Machine ===
 +
There are many different JVM that can run on ARM platform - like - let you to run your Java application on DAVE's Embedded platforms.  
  
Newest JVM based on Java 9, 10, 11, 12 and 13 can be found from the OpenJDK JVM players.
+
In this Application Note, we would like to explore some of the most popular Java Virtual Machines and measure their performances on the embedded platform. The different JVMs performance results let the user to target which JVM can be used for running its Java code in the better way.
 
 
=== Selection of embedded Virtual Machine ===
 
 
 
Here below the list of JVMs compared in the tests and the binary download sites:
 
 
 
{| class="wikitable" border="1"
 
! Java Virtual Machine
 
! JDK version
 
! VM version
 
!Notes
 
|-
 
| [https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html Oracle JDK]
 
| 8u211
 
| Java HotSpot(TM) Client VM (build 25.211-b12, mixed mode)
 
|
 
|-
 
| OpenJDK (*)
 
| 1.8.0_102
 
| OpenJDK Zero VM (build 25.102-b14, interpreted mode)
 
| [https://openjdk.java.net/projects/zero/ ZeroVM]
 
|-
 
| [https://github.com/AdoptOpenJDK/openjdk8-binaries/releases/download/jdk8u232-b09/OpenJDK8U-jdk_arm_linux_hotspot_8u232b09.tar.gz AdoptOpenJDK]
 
| 1.8.0_232
 
| OpenJDK Zero VM (AdoptOpenJDK)(build 25.232-b09, interpreted mode)
 
| [https://openjdk.java.net/projects/zero/ ZeroVM]
 
|-
 
| [https://cdn.azul.com/zulu-embedded/bin/zulu8.42.0.195-ca-jdk1.8.0_232-linux_aarch32hf.tar.gz ZuluCommunity]
 
| 8u232
 
| OpenJDK Client VM (Zulu8.42.0.195-CA-linux_aarch32hf) (build 25.232-b195, mixed mode, Evaluation)
 
|
 
|-
 
| [https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.5%2B10/OpenJDK11U-jdk_arm_linux_hotspot_11.0.5_10.tar.gz AdoptOpenJDK]
 
| 11.0.5
 
| OpenJDK Server VM AdoptOpenJDK (build 11.0.5+10, mixed mode)
 
|
 
|-
 
| [https://download.bell-sw.com/java/11.0.5+11/bellsoft-jdk11.0.5+11-linux-arm32-vfp-hflt.tar.gz LibericaJDK]
 
| 11.0.5-BellSoft
 
| OpenJDK 32-Bit Server VM (build 11.0.5-BellSoft+11, mixed mode)
 
|
 
|-
 
| [https://cdn.azul.com/zulu-embedded/bin/zulu11.33.21-ca-jdk11.0.4-linux_aarch32hf.tar.gz ZuluCommunity]
 
| 11.0.4
 
| OpenJDK Client VM Zulu11.33+21-CA (build 11.0.4+10-LTS, mixed mode)
 
|
 
|-
 
|}
 
(*) We do not provide a download URL for OpenJDK 1.8.x because it has been build with standard recipe provided with DAVE Development Kits
 
 
 
=== Licensing ===
 
Each JVM has a proper licensing model and this has to be taken into account:
 
 
 
* Oracle: the [https://www.oracle.com/downloads/licenses/javase-license1.html Java license] agreement should be accomplished in order to use and evaluate it
 
* OpenJDK, AdoptOpenJDK: [https://adoptopenjdk.net/about.html GPLv2]
 
* Zulu: see [https://www.azul.com/products/zulu-community/ Zulu Community] (''free download and uses without restrictions'', cit'')''
 
* BellSoft: [https://bell-sw.com/pages/java-11.0.5%20for%20Embedded/ Liberica JDK for Embedded] (''provided with no license restrictions'', cit.'')''
 
  
 
== Benchmarks ==
 
== Benchmarks ==
Line 123: Line 46:
 
* [http://www.benchmarkhq.ru/cm30/ CaffeineMark 3.0]
 
* [http://www.benchmarkhq.ru/cm30/ CaffeineMark 3.0]
  
=== Licensing ===
 
 
We have to comply to the proper license agreements in order to run the benchmarks:
 
We have to comply to the proper license agreements in order to run the benchmarks:
* <b><i>SciMark</i></b>: please read the [https://math.nist.gov/scimark2/credits.html credits] page about <b>National Institute of Standards and Technology (NIST)</b> GPL license
+
* SciMark: please read the [https://math.nist.gov/scimark2/credits.html credits] page about <b>National Institute of Standards and Technology (NIST)</b> GPL license
* <b><i>CaffeineMark</i></b>: according to the <b>Pendragon Software's CaffeineMark'''™''' ver. 3.0</b>
+
* CaffeineMark: please read carefully the [http://www.benchmarkhq.ru/cm30/disclaim.html#license license agreement] for running the <b>Pendragon Software's CaffeineMark'''''' ver. 3.0</b>
** the test was performed without independent verification by Pendragon Software and that Pendragon Software makes no representations or warranties as to the result of the test
 
 
 
=== Test results ===
 
The benchmarks have been run on the same platform and same cpu: the different JVMs was decompressed in the ''/home/root/java'' directory of the target and started manually.
 
 
 
==== XELK ====
 
The following tests have been performed on [[:Category:SBC-AXEL | SBC-AXEL]] equipped with:
 
* '''DXLM0193''' SOM
 
* i.MX6 Dual Lite processor @ 792MHz
 
* 1GB DDR
 
* boot from SD card (u-boot, kernel and root file system)
 
 
 
{| class="wikitable" border="1"
 
! Java Virtual Machine
 
! SciMark 2.0 (Composite Score)
 
! CaffeineMark 3.0 (Overall score)
 
|-
 
| Oracle JDK 8
 
| 69.37
 
| 22673
 
|-
 
| OpenJDK 8 (*)
 
| ''3.62''
 
| ''605''
 
|-
 
| AdoptOpenJDK 8 (*)
 
| ''3.44''
 
| ''569''
 
|-
 
| ZuluCommunity 8
 
| 69.14
 
| 17237
 
|-
 
| AdoptOpenJDK 11
 
| '''82.39'''
 
| '''78111'''
 
|-
 
| LibericaJDK 11
 
| '''82.55'''
 
| '''77083'''
 
|-
 
| ZuluCommunity 11
 
| 69.61
 
| 17060
 
|-
 
|}
 
 
 
(*) The ''interpreted'' [[XELK-AN-009:_Comparing_JVMs_(Java_Virtual_Machines)_on_i.MX6_family_processors#ZeroVM | ZeroVM]] JVM version explains the very poor performance
 
 
 
==== XUELK ====
 
The following tests have been performed on [[:Category:SBC_Lynx | SBC-Lynx]] equipped with:
 
* i.MX6 Ultra Lite processor @ 528MHz
 
* 512MB DDR
 
* boot from SD card (u-boot, kernel and root file system)
 
 
 
{| class="wikitable" border="1"
 
! Java Virtual Machine
 
! SciMark 2.0 (Composite Score)
 
! CaffeineMark 3.0 (Overall score)
 
|-
 
| Oracle JDK 8
 
| 28.75
 
| 9426
 
|-
 
| OpenJDK 8 (*)
 
| ''1.68''
 
| ''257''
 
|-
 
| AdoptOpenJDK 8 (*)
 
| ''1.55''
 
| ''243''
 
|-
 
| ZuluCommunity 8
 
| 29.26
 
| 9482
 
|-
 
| AdoptOpenJDK 11
 
| '''54.23'''
 
| '''30268'''
 
|-
 
| LibericaJDK 11
 
| '''54.46'''
 
| '''27261'''
 
|-
 
| ZuluCommunity 11
 
| 29.16
 
| 9200
 
|-
 
|}
 
 
 
(*) The ''interpreted'' [[XELK-AN-009:_Comparing_JVMs_(Java_Virtual_Machines)_on_i.MX6_family_processors#ZeroVM | ZeroVM]] JVM version explains the very poor performance
 
 
 
For a detailed test command usage, please click on the ''Expand'' link:
 
 
 
<div class="mw-collapsible mw-collapsed">
 
 
 
<pre class="board-terminal">
 
root@sbc-lynx:~/java# ls -la
 
drwxr-xr-x  10 root    root          4096 Jan  8 22:29 .
 
drwx------    4 root    root          4096 Jan  8 21:41 ..
 
drwxr-xr-x  10 1000    1000          4096 Jan  8 22:15 adopt-jdk11.0.5
 
drwxr-xr-x    5 1000    1000          4096 Jan  8 21:45 adopt-jdk8u232
 
drwxrwxr-x    7 1000    1000          4096 Jan  8 21:55 bellsoft-jdk11.0.5
 
drwxr-xr-x    3 root    root          4096 Jan  8 21:53 benchmark
 
drwxr-xr-x  10 1000    1000          4096 Jan  8 21:52 jdk1.8.0_211
 
drwxrwxr-x    9 111      122          4096 Jan  8 21:49 zulu-jdk1.8.0_232
 
drwxrwxr-x  10 111      122          4096 Jan  8 22:01 zulu-jdk11.0.4
 
root@sbc-lynx:~/java# ls -la benchmark/
 
drwxr-xr-x    3 root    root          4096 Jan  8 21:53 .
 
drwxr-xr-x  10 root    root          4096 Jan  8 22:29 ..
 
drwxr-xr-x    2 root    root          4096 Jan  8 21:43 Caffeine
 
-rwxr-xr-x    1 root    root        34229 Jan  8 21:53 scimark2lib.jar
 
root@sbc-lynx:~/java#
 
</pre>
 
 
 
Here below an example of SciMark benchmark started for OracleJDK on '''i.MX6DL''':
 
 
 
<pre class="board-terminal">
 
root@imx6qxelk:~/java# ./jdk1.8.0_211/bin/java -cp ./benchmark/scimark2lib.jar jnt.scimark2.commandline
 
SciMark 2.0a
 
 
 
Composite Score: 69.37463986126829
 
FFT (1024): 69.63592899407837
 
SOR (100x100):  123.68132332709474
 
Monte Carlo : 17.99165245491594
 
Sparse matmult (N=1000, nz=5000): 50.866191316811516
 
LU (100x100): 84.69810321344092
 
 
 
java.vendor: Oracle Corporation
 
java.version: 1.8.0_211
 
os.arch: arm
 
os.name: Linux
 
os.version: 4.9.11-xelk-4.0.0
 
root@imx6qxelk:~/java#
 
</pre>
 
 
 
and the same JVM on the '''i.MX6UL''':
 
 
 
<pre class="board-terminal">
 
root@sbc-lynx:~/java# ./jdk1.8.0_211/bin/java -cp ./benchmark/scimark2lib.jar jnt.scimark2.commandline
 
 
 
SciMark 2.0a
 
 
 
Composite Score: 28.752359792101156
 
FFT (1024): 24.74240503907121
 
SOR (100x100):  52.136229862882686
 
Monte Carlo : 7.4006247447847375
 
Sparse matmult (N=1000, nz=5000): 24.794189606277396
 
LU (100x100): 34.68834970748977
 
 
 
java.vendor: Oracle Corporation
 
java.version: 1.8.0_211
 
os.arch: arm
 
os.name: Linux
 
os.version: 4.1.15-xuelk-2.0.3
 
root@sbc-lynx:~/java#
 
</pre>
 
 
 
Then, an example of Caffeine benchmark on '''i.MX6DL''':
 
 
 
<pre class="board-terminal">
 
root@imx6qxelk:~/java# ./jdk1.8.0_211/bin/java -cp ./benchmark/Caffeine CaffeineMarkEmbeddedApp
 
Sieve score = 12204 (98)
 
Loop score = 46042 (2017)
 
Logic score = 22955 (0
 
String score = 29983 (708)
 
Float score = 21900 (185)
 
Method score = 16041 (166650)
 
Overall score = 22673
 
root@imx6qxelk:~/java#
 
</pre>
 
 
 
and the same JVM on the '''i.MX6UL''':
 
 
 
<pre class="board-terminal">
 
root@sbc-lynx:~/java# ./jdk1.8.0_211/bin/java -cp ./benchmark/Caffeine CaffeineMarkEmbeddedApp
 
Sieve score = 6359 (98)
 
Loop score = 14088 (2017)
 
Logic score = 11336 (0)
 
String score = 11944 (708)
 
Float score = 7008 (185)
 
Method score = 8253 (166650)
 
Overall score = 9426
 
root@sbc-lynx:~/java#
 
</pre>
 
 
 
</div>
 
 
 
== Copyright and Trademark Information ==
 
 
 
Java™ and HotJava™ are trademarks of Sun Microsystems, Inc. and refer to Sun's Java programming language and HotJava browser technologies.
 
 
 
Java™ and OpenJDK™ are trademarks or registered trademarks of Oracle and/or its affiliates.
 
 
 
Zulu Community™ is a trademark and Copyright © of Azul Systems, Inc.
 
  
Liberica is a Copyright of © BellSoft LTD
+
== JVMs ==
 +
The following Java Virtual Machines have been selected for the main Java player in the market. Obviously the <b>Oracle JDK</b> is the reference one due to the
  
The CaffeineMark™ benchmark is a trademark and Copyright © of Pendragon Software
+
The [https://www.oracle.com/downloads/licenses/javase-license1.html Java license] agreement should be accomplished in order to use and evaluate the Oracle
  
AdoptOpenJDK is a community of Java User Group (JUG) members, Java developers and vendors including Azul, Amazon, GoDaddy, IBM, jClarity, Microsoft, New Relic, Pivotal and Red Hat.
+
* [https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html  Oracle JDK SE 8]

Revision as of 16:00, 18 November 2019

Info Box
Axel-04.png Applies to Axel Ultra
Axel-02.png Applies to AXEL ESATTA
Axel-lite 02.png Applies to Axel Lite


200px-Emblem-important.svg.png

This application note has been validated using the kit version in the History table.

History[edit | edit source]

Processor Date Kit version Notes
i.MX6 Cortex-A9 Nov 2019 XELK 4.0.0 i.MX6 DualLite SOC version
i.MX6UL Cortex-A7 Nov 2019 XUELK 2.0.3 i.MX6 Ultra Lite SOC version

Introduction[edit | edit source]

Nowadays the Java programming language is more and more used for creating embedded application taking advantages from the popular and powerful programming language like Java.

As well known, from wikipedia description:

Java is a general-purpose programming language that is class-based, object-oriented, and designed to have as few implementation dependencies as possible. It is intended to let application developers write once, run anywhere: Java code can run on all platforms that support Java without the need for recompilation.
Java applications are typically compiled to bytecode that can run on any Java virtual machine (JVM) regardless of the underlying computer architecture.

Java Virtual Machine[edit | edit source]

There are many different JVM that can run on ARM platform - like - let you to run your Java application on DAVE's Embedded platforms.

In this Application Note, we would like to explore some of the most popular Java Virtual Machines and measure their performances on the embedded platform. The different JVMs performance results let the user to target which JVM can be used for running its Java code in the better way.

Benchmarks[edit | edit source]

Two standard Java code benchmark tools are used to compare the JVMs:

We have to comply to the proper license agreements in order to run the benchmarks:

  • SciMark: please read the credits page about National Institute of Standards and Technology (NIST) GPL license
  • CaffeineMark: please read carefully the license agreement for running the Pendragon Software's CaffeineMark ver. 3.0

JVMs[edit | edit source]

The following Java Virtual Machines have been selected for the main Java player in the market. Obviously the Oracle JDK is the reference one due to the

The Java license agreement should be accomplished in order to use and evaluate the Oracle