Difference between revisions of "XELK-AN-009: Comparing JVMs (Java Virtual Machines) on i.MX6 family processors"
(→List of test results) |
|||
| (31 intermediate revisions by 3 users not shown) | |||
| Line 3: | Line 3: | ||
{{AppliesToAxelEsatta}} | {{AppliesToAxelEsatta}} | ||
{{AppliesToAxelLite}} | {{AppliesToAxelLite}} | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
{{InfoBoxBottom}} | {{InfoBoxBottom}} | ||
| Line 16: | Line 9: | ||
{| class="wikitable" border="1" | {| class="wikitable" border="1" | ||
| − | ! | + | !Processor |
!Date | !Date | ||
| − | ! | + | !Kit version |
| + | !Notes | ||
|- | |- | ||
| − | | | + | |i.MX6 Cortex-A9 |
| − | | Nov 2019 | + | |Nov 2019 |
| − | |[[Axel_Embedded_Linux_Kit_(XELK)#XELK_4.0.0|XELK 4.0.0]] | + | |[[Axel_Embedded_Linux_Kit_(XELK)#XELK_4.0.0|XELK 4.0.0]] |
| + | |i.MX6 DualLite SOC version | ||
| + | |- | ||
| + | |i.MX6UL Cortex-A7 | ||
| + | |Nov 2019 | ||
| + | |[[AXEL_ULite_and_SBC_Lynx_Embedded_Linux_Kit_(XUELK)#XUELK_2.0.1.2F2.0.2.2F2.0.3|XUELK 2.0.3]] | ||
| + | | i.MX6 Ultra Lite SOC version | ||
|- | |- | ||
|} | |} | ||
==Introduction== | ==Introduction== | ||
| − | Nowadays the [https://en.wikipedia.org/wiki/Java_(programming_language) Java | + | Nowadays the Java programming language is more and more used for creating embedded application taking advantages from the popular and powerful programming language like [https://en.wikipedia.org/wiki/Java_(programming_language) Java]. |
| − | + | As well known, from [https://en.wikipedia.org/wiki/Java_(programming_language) wikipedia] description: | |
| + | |||
| + | <pre> | ||
| + | Java is a general-purpose programming language that is class-based, object-oriented, and designed to have as few implementation dependencies as possible. It is intended to let application developers write once, run anywhere: Java code can run on all platforms that support Java without the need for recompilation. | ||
| + | Java applications are typically compiled to bytecode that can run on any Java virtual machine (JVM) regardless of the underlying computer architecture. | ||
| + | </pre> | ||
=== Java Virtual Machine === | === Java Virtual Machine === | ||
| − | There are many different JVM that can run on ARM platform | + | There are many different JVM that can run on ARM platform - like - let you to run your Java application on DAVE's Embedded platforms. |
| − | In this Application Note, we would like to explore some of the most popular Java Virtual Machines and measure their performances on the embedded platform. | + | In this Application Note, we would like to explore some of the most popular Java Virtual Machines and measure their performances on the embedded platform. The different JVMs performance results let the user to target which JVM can be used for running its Java code in the better way. |
| − | The different JVMs performance results let the user to target which JVM can be | ||
| − | + | === ZeroVM=== | |
The [https://openjdk.java.net/projects/zero/ ZeroVM] JDK version has to be taken into account if used on some JVM: this version - used for example on ''OpenJDK 8'' and ''AdoptOpenJDK 8'' - gives output results '''really less performing''' then the others VMs. | The [https://openjdk.java.net/projects/zero/ ZeroVM] JDK version has to be taken into account if used on some JVM: this version - used for example on ''OpenJDK 8'' and ''AdoptOpenJDK 8'' - gives output results '''really less performing''' then the others VMs. | ||
| Line 43: | Line 47: | ||
it's not performant because it has no architecture specific assembly code. | it's not performant because it has no architecture specific assembly code. | ||
| − | == | + | == Benchmarks == |
| − | + | Two standard Java code benchmark tools are used to compare the JVMs: | |
| + | * [https://math.nist.gov/scimark2/ SciMark 2.0] | ||
| + | * [http://www.benchmarkhq.ru/cm30/ CaffeineMark 3.0] | ||
| − | + | We have to comply to the proper license agreements in order to run the benchmarks: | |
| + | * <b><i>SciMark</i></b>: please read the [https://math.nist.gov/scimark2/credits.html credits] page about <b>National Institute of Standards and Technology (NIST)</b> GPL license | ||
| + | * <b><i>CaffeineMark</i></b>: please read carefully the [http://www.benchmarkhq.ru/cm30/disclaim.html#license license agreement] for running the <b>Pendragon Software's CaffeineMark'''™''' ver. 3.0</b> | ||
| − | + | == Java Virtual Machines == | |
| + | The Java Virtual Machines listed below, have been selected from the main Java players in the market. Obviously the <b>Oracle JDK</b> is the reference one due to the original Java project started by [https://en.wikipedia.org/wiki/Java_(programming_language) Sun Microsystems] and aquired by [https://en.wikipedia.org/wiki/Sun_acquisition_by_Oracle Oracle] on 2010. | ||
| − | + | === Licensing === | |
| + | Each JVM has a proper licensing model and this has to be taken into account: | ||
| − | [https:// | + | * Oracle: the [https://www.oracle.com/downloads/licenses/javase-license1.html Java license] agreement should be accomplished in order to use and evaluate it |
| + | * AdoptOpenJDK: the [https://adoptopenjdk.net/about.html GPLv2] license is used for the OpenJDK code | ||
| + | * Zulu: the [https://www.azul.com/products/zulu-community/ Zulu Community] version has free download and uses without restrictions | ||
| + | * BellSoft: the [https://bell-sw.com/pages/java-11.0.5%20for%20Embedded/ Liberica JDK for Embedded] is provided with no license restrictions | ||
| − | + | === Virtual Machine version === | |
| − | |||
| − | |||
| − | |||
| − | |||
| − | + | Here below the list of JVMs compared in the tests and the download sites: | |
| − | |||
| − | |||
| − | |||
| − | Here below the list of JVMs compared in the tests and the | ||
{| class="wikitable" border="1" | {| class="wikitable" border="1" | ||
| Line 77: | Line 82: | ||
| | | | ||
|- | |- | ||
| − | | OpenJDK | + | | OpenJDK |
| 1.8.0_102 | | 1.8.0_102 | ||
| OpenJDK Zero VM (build 25.102-b14, interpreted mode) | | OpenJDK Zero VM (build 25.102-b14, interpreted mode) | ||
| Line 108: | Line 113: | ||
|- | |- | ||
|} | |} | ||
| − | |||
| − | == | + | == Benchmarking results == |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
The benchmarks have been run on the same platform and same cpu: the different JVMs was decompressed in the ''/home/root/java'' directory of the target and started manually. | The benchmarks have been run on the same platform and same cpu: the different JVMs was decompressed in the ''/home/root/java'' directory of the target and started manually. | ||
| − | + | Here below an example of SciMark benchmark started for OracleJDK: | |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
<pre class="board-terminal"> | <pre class="board-terminal"> | ||
| − | + | root@imx6qxelk:~/java# ./jdk1.8.0_211/bin/java -cp ./scimark2lib.jar jnt.scimark2.commandline | |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | root@imx6qxelk:~/java# ./jdk1.8.0_211/bin/java -cp . | ||
SciMark 2.0a | SciMark 2.0a | ||
| Line 263: | Line 138: | ||
</pre> | </pre> | ||
| − | + | Then, an example of Caffeine benchmark: | |
<pre class="board-terminal"> | <pre class="board-terminal"> | ||
| − | root@ | + | root@imx6qxelk:~/java/benchmark/Caffeine# ../../jdk1.8.0_211/bin/java CaffeineMarkEmbeddedApp |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
Sieve score = 12204 (98) | Sieve score = 12204 (98) | ||
Loop score = 46042 (2017) | Loop score = 46042 (2017) | ||
| Line 296: | Line 149: | ||
Method score = 16041 (166650) | Method score = 16041 (166650) | ||
Overall score = 22673 | Overall score = 22673 | ||
| − | root@imx6qxelk:~/java# | + | root@imx6qxelk:~/java/benchmark/Caffeine# |
</pre> | </pre> | ||
| − | + | === List of test results === | |
| − | + | {| class="wikitable" border="1" | |
| − | + | ! Java Virtual Machine | |
| − | + | ! SciMark (Composite Score) | |
| − | + | ! CaffeineMark (Overall score) | |
| − | + | ! Notes | |
| − | + | |- | |
| − | + | | Oracle JDK 8 | |
| − | + | | 69.37 | |
| − | + | | 22673 | |
| − | + | | | |
| − | + | |- | |
| − | + | | OpenJDK 8 | |
| − | + | | 3.62 | |
| − | + | | 605 | |
| − | + | | The ''interpreted'' JVM version explains the very poor performance | |
| − | + | |- | |
| − | + | | AdoptOpenJDK 8 | |
| − | + | | 3.44 | |
| − | + | | 569 | |
| − | + | | The ''interpreted'' JVM version explains the very poor performance | |
| − | + | |- | |
| − | + | | ZuluCommunity 8 | |
| − | + | | 69.14 | |
| − | + | | 17237 | |
| − | + | | | |
| − | + | |- | |
| − | + | | AdoptOpenJDK 11 | |
| + | | '''82.39''' | ||
| + | | '''78111''' | ||
| + | | | ||
| + | |- | ||
| + | | LibericaJDK 11 | ||
| + | | '''82.55''' | ||
| + | | '''77083''' | ||
| + | | | ||
| + | |- | ||
| + | | ZuluCommunity 11 | ||
| + | | 69.61 | ||
| + | | 17060 | ||
| + | | | ||
| + | |- | ||
| + | |} | ||
Revision as of 21:05, 18 November 2019
| HOME | SOMs | SBCs | ToloMEO Embedded Assistant |  GET A QUOTE
|
 ONLINE HELPDESK
|
| Roadmap | IoT Services | ML/AI services | Embedded Design Services |
Info Box
|
 This application note has been validated using the kit version in the History table. |
Contents
History[edit | edit source]
| Processor | Date | Kit version | Notes |
|---|---|---|---|
| i.MX6 Cortex-A9 | Nov 2019 | XELK 4.0.0 | i.MX6 DualLite SOC version |
| i.MX6UL Cortex-A7 | Nov 2019 | XUELK 2.0.3 | i.MX6 Ultra Lite SOC version |
Introduction[edit | edit source]
Nowadays the Java programming language is more and more used for creating embedded application taking advantages from the popular and powerful programming language like Java.
As well known, from wikipedia description:
Java is a general-purpose programming language that is class-based, object-oriented, and designed to have as few implementation dependencies as possible. It is intended to let application developers write once, run anywhere: Java code can run on all platforms that support Java without the need for recompilation. Java applications are typically compiled to bytecode that can run on any Java virtual machine (JVM) regardless of the underlying computer architecture.
Java Virtual Machine[edit | edit source]
There are many different JVM that can run on ARM platform - like - let you to run your Java application on DAVE's Embedded platforms.
In this Application Note, we would like to explore some of the most popular Java Virtual Machines and measure their performances on the embedded platform. The different JVMs performance results let the user to target which JVM can be used for running its Java code in the better way.
ZeroVM[edit | edit source]
The ZeroVM JDK version has to be taken into account if used on some JVM: this version - used for example on OpenJDK 8 and AdoptOpenJDK 8 - gives output results really less performing then the others VMs.
It should be considered - infact - that OpenJDK Zero VM is an interpreter only JVM. It's easier to port because it has no architecture specific assembly code but, on the other hand, it's not performant because it has no architecture specific assembly code.
Benchmarks[edit | edit source]
Two standard Java code benchmark tools are used to compare the JVMs:
We have to comply to the proper license agreements in order to run the benchmarks:
- SciMark: please read the credits page about National Institute of Standards and Technology (NIST) GPL license
- CaffeineMark: please read carefully the license agreement for running the Pendragon Software's CaffeineMark™ ver. 3.0
Java Virtual Machines[edit | edit source]
The Java Virtual Machines listed below, have been selected from the main Java players in the market. Obviously the Oracle JDK is the reference one due to the original Java project started by Sun Microsystems and aquired by Oracle on 2010.
Licensing[edit | edit source]
Each JVM has a proper licensing model and this has to be taken into account:
- Oracle: the Java license agreement should be accomplished in order to use and evaluate it
- AdoptOpenJDK: the GPLv2 license is used for the OpenJDK code
- Zulu: the Zulu Community version has free download and uses without restrictions
- BellSoft: the Liberica JDK for Embedded is provided with no license restrictions
Virtual Machine version[edit | edit source]
Here below the list of JVMs compared in the tests and the download sites:
| Java Virtual Machine | JDK version | VM version | Notes |
|---|---|---|---|
| Oracle JDK | 8u211 | Java HotSpot(TM) Client VM (build 25.211-b12, mixed mode) | |
| OpenJDK | 1.8.0_102 | OpenJDK Zero VM (build 25.102-b14, interpreted mode) | ZeroVM |
| AdoptOpenJDK | 1.8.0_232 | OpenJDK Zero VM (AdoptOpenJDK)(build 25.232-b09, interpreted mode) | ZeroVM |
| ZuluCommunity | 8u232 | OpenJDK Client VM (Zulu8.42.0.195-CA-linux_aarch32hf) (build 25.232-b195, mixed mode, Evaluation) | |
| AdoptOpenJDK | 11.0.5 | OpenJDK Server VM AdoptOpenJDK (build 11.0.5+10, mixed mode) | |
| LibericaJDK | 11.0.5-BellSoft | OpenJDK 32-Bit Server VM (build 11.0.5-BellSoft+11, mixed mode) | |
| ZuluCommunity | 11.0.4 | OpenJDK Client VM Zulu11.33+21-CA (build 11.0.4+10-LTS, mixed mode) |
Benchmarking results[edit | edit source]
The benchmarks have been run on the same platform and same cpu: the different JVMs was decompressed in the /home/root/java directory of the target and started manually.
Here below an example of SciMark benchmark started for OracleJDK:
root@imx6qxelk:~/java# ./jdk1.8.0_211/bin/java -cp ./scimark2lib.jar jnt.scimark2.commandline SciMark 2.0a Composite Score: 69.37463986126829 FFT (1024): 69.63592899407837 SOR (100x100): 123.68132332709474 Monte Carlo : 17.99165245491594 Sparse matmult (N=1000, nz=5000): 50.866191316811516 LU (100x100): 84.69810321344092 java.vendor: Oracle Corporation java.version: 1.8.0_211 os.arch: arm os.name: Linux os.version: 4.9.11-xelk-4.0.0 root@imx6qxelk:~/java#
Then, an example of Caffeine benchmark:
root@imx6qxelk:~/java/benchmark/Caffeine# ../../jdk1.8.0_211/bin/java CaffeineMarkEmbeddedApp Sieve score = 12204 (98) Loop score = 46042 (2017) Logic score = 22955 (0 String score = 29983 (708) Float score = 21900 (185) Method score = 16041 (166650) Overall score = 22673 root@imx6qxelk:~/java/benchmark/Caffeine#
List of test results[edit | edit source]
| Java Virtual Machine | SciMark (Composite Score) | CaffeineMark (Overall score) | Notes |
|---|---|---|---|
| Oracle JDK 8 | 69.37 | 22673 | |
| OpenJDK 8 | 3.62 | 605 | The interpreted JVM version explains the very poor performance |
| AdoptOpenJDK 8 | 3.44 | 569 | The interpreted JVM version explains the very poor performance |
| ZuluCommunity 8 | 69.14 | 17237 | |
| AdoptOpenJDK 11 | 82.39 | 78111 | |
| LibericaJDK 11 | 82.55 | 77083 | |
| ZuluCommunity 11 | 69.61 | 17060 |
