How to use NestedVM

If you do a google search for "call native code from java" then you'll get results for the Java Native Interface (JNI) and a few about Java Native Access (JNA). These tools combine native code and Java code in a way that makes the JVM susceptible to crashing from memory access errors in native code.

Nothing comes up in a search for "JNI alternatives", but what if we could simply emulate a CPU with Java and run binary code? Or can binary code be converted to Java bytecode?

Turns out that the problem has already been solved by a project called NestedVM. Their website is un-assuming and gives no indication as to how well it works other than the statement that "any application written in C, C++, Fortran, or any other language supported by GCC can be run in 100% pure Java with no source changes". Buried in the footnotes of their acedemic whitepaper is the vast number of libraries that they have compiled and have working using NestedVM. Additionally, some research shows that they are used by an older Sqlite JDBC driver by xerial (see also this StackOverflow post).

We've tried their software and can confirm that it works extremely well. The only thing lacking is documentation. The git repository has a sparse readme file, links to a wiki have gone down, leaving only archive.org and their mailing list/google groups. One comment on Google Groups states "I can not believe this tool hasn't been more hyped up," to which its creator replied "We're really bad at promoting our software :)". Indeed, there are thousands of people seeking to use such a tool as an alternative to JNI, but its not widely known and hard to find.

How does it work?

NestedVM provides a compiler and a runtime (just like the runtime converter does, although we call it a converter since it produces souces) as well as a modified version of gcc that can be used to compile for the MIPS architecture. MIPS is an obscure CPU instruction set from the 1980s. It was used for the Nintendo 64 and PS2 game consoles, and still has some use today in embedded systems such as routers.

MIPS was chosen for its simple instruction set which the NestedVM then translates into Java bytecode. In addition, they provide a runtime that emulates the operating system component of a binary application (primarily I/O).

Usage

The ibex.org version only supports GCC 3.3.6, but some contributors have forked the project and updated it to GCC 4.8.5. A link is available here: https://github.com/bgould/nestedvm

First step is to install a virtual machine with CentOS 7 and setup the repository according to the readme instructions. Also do `sudo make install`.

Then download the sources that you want to compile and run the following script (with modifications as necessary). Note that you will need an executable target. If you don't have one, you will need link the MIPS library into an executable with an empty main() function.

set -e export PATH=$PATH:/usr/local/nestedvm/bin export CC="/usr/local/nestedvm/bin/mips-unknown-elf-gcc" export CPP="/usr/local/nestedvm/bin/mips-unknown-elf-gcc -E" export CFLAGS="-O3 -mmemcpy -ffunction-sections -fdata-sections -falign-functions=512 -fno-rename-registers -fno-schedule-insns -fno-delayed-branch -march=mips1 -specs=/usr/local/nestedvm/mips-unknown-elf/lib/crt0-override.spec " export CPPFLAGS=$CFLAGS export LDFLAGS="-march=mips1 -specs=/usr/local/nestedvm/mips-unknown-elf/lib/crt0-override.spec --static -Wl,--gc-sections" ./configure --host=mips-unknown-elf make CC=/usr/local/nestedvm/bin/mips-unknown-elf-gcc LD=/usr/local/nestedvm/bin/mips-unknown-elf-gcc

The flags are mentioned in the whitepaper and are for performance and compatibility reasons. GCC and LD are replaced with "mips-unknown-elf-gcc". The flags are referenced also from the provided Makefile which compiles the C test sources to mips.

Then, you will need to run in the sources directory:

java -jar compiler.jar -outfile myclass.class -o supportCall com.test.myclass {mips file}

For {mips file} you will need to use the executable created by the make process. It won't be named .mips, but like all unix executables, has no extension.

Lastly, you will need to make a JAR file after putting your class in its proper package root:

jar -cf myclass.jar com/test/myclass.class

Copy myclass.jar and unix_runtime.jar off the VM and include it in a Java project.

Java Usage

The NestedVM project is large and mostly undocumented. To understand its usage you can review the test scripts ex. Call.java. Once you build your class, you can create an instance of that class. Multiple instances can be created and they will be isolated from each other.

Call the method "run" or "start" to run the application like a normal application.

If you look at Test.c, you will see two extern declarations: "_pause" and "_call_java". These declarations not only allow you to interact with the Java caller, they are needed to run arbitrary calls from the JVM. To run arbitrary calls, replace the body of your "main" method with a call to "_pause". Then, after starting your application, you can run arbitrary calls using instance.call(...) methods.

Other methods

The other Runtime class methods are equally undocumented. It is useful to remember that everything is defined in memory as an integer array and will reduce to either a 32-bit pointer or a 32-bit integer (even other memory types like char).

Other C to Java Converter Reviews

We've looked through several converters listed on http://www.thefullwiki.org/C_to_Java_byte-code_compiler and can provide some additional information about some of these projects after reviewing them:

Cybil - website is down, but can be found at https://github.com/SimonKagstrom/cibyl. Looks very similar to NestedVM. Last commit 2011.

LLJVM - abandoned at the "work in progress" stage. It ran a few limited test cases at one time.

C2J - The sources for this project don't seem to be available. Runs as a Windows GUI program. Appears to have been abandoned.

Axiomatic Multi-Platform C - also appears abandoned.

Java Backend for GCC - might refer to an old GNU Java implementation

Javum - sounds similar to NestedVM but not available

egcs-jvm - abandoned

Another option is "graalvm" which is an experimental project by Oracle to replace Java bytecode with LLVM bitcode, thus gaining binary compatibility with all languages that can compile to bitcode. It is not an option for the current JVM, but rather an experimental JVM implementation.

We also have an experimental "C to Java" converter of our own that we may publish soon.

Conclusion

NestedVM is a project that was developed to completion more than 10 years ago. It is not well known and poorly documented, but it still works today. One reason why it is not more popular is that so much open source Java code has been written since NestedVM first began development in 2004 that there is little reason for most developers to use native code other than for performance reasons. For the Runtime Converter it is exactly what we needed since PHP relies a few very large and unique C libraries for some of its functions.