Accelerating the Android build process

Building a C++ engine on Android

For the last couple of months, my focus has been on Android. Android is the primary target platform for Pilkapel but also, one of the hardest to develop for and debug. In this post, I’ll talk a bit about the problems I’ve faced creating a native C++ game engine that runs on the platform, specifically the ‘build process’. This isn’t going to involve much game specific stuff and no funky graphics, this is pure build system grunt work but I thought this might be interesting to some people so I’ve attempted to write the process up.

When I first achieved a working build of Anthracite on Android a few years ago, it was a great feeling - every new platform is like this, each time I have the code running on a different machine, it’s a nice warm glow of achievement and the promise of ‘write once, run everywhere’.

Now, as any seasoned Java developer will tell you, that little phrase is big on promise but in reality, it’s usually ‘write once, debug everywhere’ and as programmers spend a lot of our time trying to figure out what’s going wrong, that’s means that the cross platform promise isn’t a big a win as it sounds. On top of this, Android is a rapidly moving target and google’s tools are geared towards developing Android specific apps, not cross-platform apps (ironic given the fact that so much on Android is Java based). This means that this style of programming, writing code in cross-platform C++ and building from the ground up isn’t as well supported by google as their ‘preferred’ methods, using their UI libraries and the Java ecosystem. Since starting development on Anthracite, Android has shifted IDE and build systems several times. Every update has meant weeks of work for me and nothing ever ‘just works’. Learning new things is fun but not so much when you just want to get on with a job - then it’s a roadblock.

It’s got to the point where I’ve been dreading the semi-regular process of ensuring that Anthracite still runs on Android. I’d open up Android Studio, it’d inform me that it wants to update and that foolish part of my mind that would say ‘go on, what harm can it do’, I’d press the button and then I’d be in for a week of figuring out cryptic error messages or trying to relearn a piece of software that’d morphed in to something new, again. I said the last time that if I ever had to repeat this process, I’d write my own damned build system. Of course, this did happen again and Android Studio now looks entirely different: I’m sure it’s much better and the team responsible did a great job but now I had no idea where anything was and what I want to do is finish my game, not relearn a dev environment. After a miserable couple of nights prodding my old project with a virtual stick, trying to get it to build anything, I realised I’d probably have to make a new project and start afresh, joylessly attempting to find the correct dialog box to inform the system where my various libraries live and even once I’d got it working, I’d have a slow build system that didn’t quite fit my needs, all with the full knowledge that I’d have to repeat the process next time.

I’d had enough, time to do it the hard way to make my life easier.

What is a build system?

Firstly though, you might wonder what a build system even is. Build systems are a kind of program that knows how to call other programs that do various tasks related to creating software and orchestrates those programs to create all those files in the right order. Build systems usually have some notion of minimizing work, not repeating themselves re-doing things that have already been done and don’t need updating but also capable of figuring out ‘cascading’ changes - i.e. if you change file A and files B,C & D reference that file, they get built too. Popular build systems include the venerable ‘make’ system (used a lot for C/C++ code, it’s been around since 1976!), Ant, Gradle etc. Android currently relies on ‘Gradle’ which is known for being a tad obtuse and impenetrable. Gradle is used to call the programs that build Java projects, download packages, process the Android UI scripts and so on. However, for Anthracite, 99% of the work is based on custom C & C++ code (which uses ‘make’) and data pre-processing steps which are written in python. In Anthracite, each platform is abstracted away by a thin layer and for Android that means a tiny Java file and a bit of C++ scaffolding - Anthracite doesn’t make a lot of use of Android’s UI system or the localisation strings - all of that is cross platform and handled by the Anthracite engine. So, Gradle and Android Studio are really overkill then for this task, it’s expecting a lot of things that just aren’t there and places restrictions on where things should be that slowed the process down and made it hard to keep the project tidy.

Avoiding Roadblocks

I decided to try and build an APK (Android pack file) using just the commands that Gradle would call, with the goal of having a single command that would use those programs to build the project and deploy it to the device. I figured that even if I wasn’t successful, I’d learn a lot more about how APKs are built and that would help me debug Gradle projects in the future.

Tasty APK building recipe

So, what is in an APK file and how do we build them from scratch? As I mentioned earlier, despite its complexity and obtuseness, all Gradle does is decide to run a bunch of other programs in order. You can use lots of simpler alternatives to do that - make files, batch files, python scripts, you name it. I’ve decided to use python as that’s a good fit for my existing data building tools and it’s cross platform (batch files and shell scripts do the same thing but aren’t cross platform). Plus, you can debug python line-by-line in vs code if you want to.

The basic steps I follow are:

• Set up environment variables to point to Android NDK and SDK
• Build C library
• Build Java code
• Convert java binary to ‘Dalvik’ format
• Run AAPT to process Android data files and create APK file
• Run Anthracite script to pre-process data and pack it in to APK file
• Sign APK
• Align APK

Build C Library

Firstly, I ensure that the Anthracite C++ libraries are all correctly built by calling ‘ndk-build’, which is related to the ‘make’ command (but with Android related extras) and uses a file called ‘Android.mk’ to describe how this should be done. This is a step I already had to do before this project and previously, I’d run this make program and then finish the build in Android Studio. The ‘Android.mk’ file is almost identical to the make files Anthracite uses on Linux, WASM, MacOS and optionally on windows (Visual studio’s build system can be used as well). Once this completes successfully, a binary file with an ‘.so’ extension is created - this is similar to ‘.dll’ files on windows, it’s a shared library file and there are two copies, one release and one debug. The debug library is much larger and contains helpful debugging information.

The command to do this is as follows:

ndk-build -j8

And to remove extra debugging information (it takes a lot of space) run

strip libAnthracite.so

Build Java

After this, the java ‘glue’ files that allow the C++ library to talk to the Android system are compiled. This is a small java file, a thin wrapper around the anthracite core. This is compiled using a program called ‘javac’ and this creates an (almost) executable file in what’s called ‘java bytecode’. This runs on a ‘virtual machine’ that has abstracted instructions that look a lot like machine code (the native language of any given type of chip- they’re all very different from each other) but not for any real machine, just something close to them. Java will then compile this down to machine code when you run it on a particular device - this is java’s (and C#‘s, which is a bit of a java ripoff) main strength - it’s a cross platform language which means you can make a java app and so long as the user has a Java player, you can run that app on that machine. That’s the idea, anyway. In reality there are often problems. For Anthracite, the call to Javac looks like this:

javac -d obj -classpath %ANDROID_SDK_ROOT%\platforms\android-32\android.jar;%ANDROID_SDK_ROOT%\platforms\android-32\core-for-system-modules.jar;%PROJ_DIR%\lib\appcompat-1.4.1.aar;%PROJ_DIR%\lib\android-support-v4.jar;%PROJ_DIR%\obj;%PROJ_DIR%\libs -sourcepath src src/com/totga/anthracite/*.java

Convert to Dalvik

However, Android doesn’t really do things quite like that, it takes the java byte code and converts it into its own thing called ‘Dalvik’, which is a sort of bytecode but not the same as Java’s own. The Dalvik runtime has apparently been superseded by ‘ART’ but still uses the same bytecode, so hey, doesn’t matter to us. You can convert from java ‘.class’ files to Dalvik ‘.dex’ files using a program called ‘mydex’. That it works, is all I care about!

The Android SDK provides a batch file called ‘mydex’ which wraps around a java program called ‘d8’ that dexifies all the relevant files in a directory - you can use wildcards like this:

mydex obj\totga\anthracite\*

Run AAPT

Next I call when building the anthracite APK is ‘AAPT’ which is a program provided by the Android SDK. AAPT stands for ‘Android Asset Packaging Tool’ and I’m still using the original version - there’s a new one (AAPT2) but as I’m really lightly touching on the Android ecosystem, I don’t have much use for it yet. AAPT takes the ‘AndroidManifest.xml’ file that all Android projects have and describe the permissions the program requires (If I want to use the camera, network connection or whatever, this needs to be stated in the manifest file) and requires the programmer to supply references to libraries, tell it where the output files are and so on.

You can call AAPT multiple times. For Anthraite, a call, it looks something like this:

AAPT add mypackage.apk.unaligned classes.dex 
AAPT add mypackage.apk.unaligned lib\arm64-v8a\libAnthracite.so 

Now what AAPT is really doing under the hood is largely just building a zip archive - the APK file is really just a zip file. You can drop it in to any zip unpacker and it’ll let you root around all the files. Now, some of the restrictions that I mentioned earlier regarding where data ought to live are reflected in AAPT but the great thing about this being just a zip file is that we can just bung whatever files in there however we like and that’s what I did - I used anthracite’s existing data building tools which traverse a given scene and work out what other files are being used by that scene and I modified this to pack those files in to the APK, with the added bonus that I can reuse this system for other platforms. So, my data build tool is working alongside AAPT to build the Android archive.

It was also important to make sure the compiled C++ library files (which have a ‘.so’ extension) were in the right place and this is something that always confused me with Gradle - half the time I had no idea where the thing was looking for my ‘.so’ files, it seemed to be fairly mysterious and ‘mysterious’ isn’t a good adjective to have when developing software. Taking control of this meant I now know exactly where my files are going and where they’re coming from. Also, it’s about 20 times faster than Gradle and faster is good.

Sign here please

Next up, you have to sign your jar file so that it’ll be accepted on the device and on the Android store - you can do this using a program called ‘jarsigner’ which digitally signs your package using a crypto key that you can create using a program like ‘putty’ or ‘ssh-keygen’.

jarsigner -verbose -sigalg SHA1withRSA -digestalg SHA1 -keystore release.keystore mypackage.apk.unaligned release_key -storepass "something" -keypass "something"

Align all the things

The last step in building the APK to run a program called ‘zipalign’ which I aligns the zip file contents so that they can be efficiently accessed in memory (many chip architectures like things to be aligned on 4 (or other powers of 2) byte boundaries.)

ZIPALIGN -f 4 mypackage.apk.unaligned mypackage.apk

After all that, voila, you should have an APK file which can be delivered to the device(s) using the ‘ADB’ command.

ADB -s <your_device> install -r -d mypackage.apk

But did it work and was it worth it?

Fortunately, It did! One of the best improvements is raw speed: I now have a very fast build process on Android - 20-30 seconds from start to finish in release mode, which is much better than the up-to-10 minutes Gradle could take! So, from turnaround time alone this has been a big win.

The beauty of all this is that’s super fast, reliable, very flexible, space optimized (I can easily strip out unused files) it can be run from any software with no UI intervention. Eventually, I’d like to run this from Anthracite’s editor/modeller, ClayWorks: make a scene, press a button, have it on your Android device.

With luck, I’m also now slightly future proof. However, I already know that google has iterated on their base tools (I can get away with using AAPT for now but perhaps I’ll have to upgrade later) but at least now I am familiar with these, I can learn the new tools which is less of a bother than trying to figure out what on earth Gradle is up to.

Unresolved Issues Packages - bit of a pain

One wrinkle I did have was in downloading Android packages - they have recently made this more difficult and in the end, I simply didn’t use some libraries. I wrote a program to download Android library jars but some of them have been fragmented in to smaller files and its not straightforward to see how these hang together. Not knowing where else to get the package list from, I scraped the details from a website. I developed this script from a shell script called ‘tiny-android-template.sh’. If you’re interested in developing APK files the hard way, feel free to take that code and improve upon it.

Conclusion and next steps All this is great but the goal is to get a game finished and released on Android. It’s been a long time now since the last Android build and Anthracite has moved on quite a bit and there are new bugs on the platform, which brings me to the next problem with native Android development - debugging!

In order to make that easier, I went down another fun rabbit hole but more on that next time.

If you’re a developer using Gradle or trying to go a bit under the hood, please feel free to talk about this below the line.