Benchmarking embedded Java

As I'm working an a Java processor I'm interested to benchmark embedded
Java systems. However, the usual SPECJvm98 is far too large for many
embedded system.
After searching around on the net to find embeddded benchmarks in Java
I've decided to write a new benchmark suit.

It contains several micro-benchmarks to evaluate CPI for single bytecodes
or short sequences of bytecodes, one synthetic benchmark (the Sieve of
Eratosthenes) and two application benchmarks.

To provide a realistic workload for embedded systems a real-time
application was adapted to create the first application benchmark (Kfl).
The application is from one node of a distributed motor control system. A
simulation of the environment (sensors and actors) and the communication
system (commands from the master station) forms part of the benchmark for
simulating the real-world workload.

The second application benchmark is an adaption of a tiny TCP/IP stack
(Ejip) for embedded Java. The benchmark contains two UDP server/clients
exchanging messages via a loopback device.

There is a great variation in processing power on different embedded
systems. To handle these variation all benchmarks are `self adjusting'.
Each benchmarks consists of an aspect that is benchmarked in a loop and
an `overhead' loop that contains any overhead from the benchmark that
should be subtracted from the result (this feature is for the micro
benchmarks). The loop count adapts until the benchmark runs for more than
a second. The number of iterations per second is calculated which means
that higher values indicate better performance.

The benchmark framework needs only two system functions: One to measure
time in millisecond resolution and one to print the results.  These
functions are encapsulated in LowLevel.java and can be adapted to
environments where the full Java library is not available. For example,
the leJOS system has very limited output capabilities and and a special
LowLevel.java exists in the lejos subdirectory. The benchmark is
straight-forward to use. To compile and run all benchmarks on a standard
JVM simple type:

        javac jbe/DoAll.java
        java jbe.DoAll

I've put the results of JOP, leJOS, JStamp (aJ80), EJC and, for a
reference only, different JVM versions (Suns JVM and gcj) running on a
Pentium MMX 266 at my website:

http://www.jopdesign.com/perf.jsp

If someone has access to other embedded Java solutions it would be nice
to add benchmark results to the table. You can find the source of the
embedded benchmark at the website above.

I would be especially interested in following platforms: the aJ100, TINI,
Cjip, Lightfood in an FPGA, Moon and the PSC1000 (now called Ignite).

Martin
----------------------------------------------
JOP - a Java Processor core for FPGAs:
http://www.jopdesign.com/

Thank's to Tim Simpson results for a TINI at 40MHz is added to the list.

Martin
--
----------------------------------------------
JOP - a Java Processor core for FPGAs:
http://www.jopdesign.com/

"Martin Schoeberl" <martin.schoeberl@chello.at> schrieb im Newsbeitrag
news:bpT4d.308165$vG5.160804@news.chello.at...
> As I'm working an a Java processor I'm interested to benchmark embedded
> Java systems. However, the usual SPECJvm98 is far too large for many
> embedded system.
> After searching around on the net to find embeddded benchmarks in Java
> I've decided to write a new benchmark suit.
>
> It contains several micro-benchmarks to evaluate CPI for single
bytecodes
> or short sequences of bytecodes, one synthetic benchmark (the Sieve of
> Eratosthenes) and two application benchmarks.
>
> To provide a realistic workload for embedded systems a real-time
> application was adapted to create the first application benchmark
(Kfl).
> The application is from one node of a distributed motor control system.
A
> simulation of the environment (sensors and actors) and the
communication
> system (commands from the master station) forms part of the benchmark
for
> simulating the real-world workload.
>
> The second application benchmark is an adaption of a tiny TCP/IP stack
> (Ejip) for embedded Java. The benchmark contains two UDP server/clients
> exchanging messages via a loopback device.
>
> There is a great variation in processing power on different embedded
> systems. To handle these variation all benchmarks are `self adjusting'.
> Each benchmarks consists of an aspect that is benchmarked in a loop and
> an `overhead' loop that contains any overhead from the benchmark that
> should be subtracted from the result (this feature is for the micro
> benchmarks). The loop count adapts until the benchmark runs for more
than
> a second. The number of iterations per second is calculated which means
> that higher values indicate better performance.
>
> The benchmark framework needs only two system functions: One to measure
> time in millisecond resolution and one to print the results.  These
> functions are encapsulated in LowLevel.java and can be adapted to
> environments where the full Java library is not available. For example,
> the leJOS system has very limited output capabilities and and a special
> LowLevel.java exists in the lejos subdirectory. The benchmark is
> straight-forward to use. To compile and run all benchmarks on a
standard
> JVM simple type:
>
>         javac jbe/DoAll.java
>         java jbe.DoAll
>
> I've put the results of JOP, leJOS, JStamp (aJ80), EJC and, for a
> reference only, different JVM versions (Suns JVM and gcj) running on a
> Pentium MMX 266 at my website:
>
> http://www.jopdesign.com/perf.jsp
>
> If someone has access to other embedded Java solutions it would be nice
> to add benchmark results to the table. You can find the source of the
> embedded benchmark at the website above.
>
> I would be especially interested in following platforms: the aJ100,
TINI,
> Cjip, Lightfood in an FPGA, Moon and the PSC1000 (now called Ignite).
>
> Martin
> ----------------------------------------------
> JOP - a Java Processor core for FPGAs:
> http://www.jopdesign.com/
>
>
>