Extracting textures from Defold .arcd files

The Defold game engine is able to generate applications that run on Desktop, smartphones and HTML5 web browsers. Just for fun I built an application that extracts textures and Lua scripts from the generated binary data files.

Problem​

A game would not run on my computer, and I thought it would be worthwile to at least extract its image assets. It had a 1.5 GiB file called game.arcd that probably contained all the data, and I just had to obtain them from there.

Identification​

I had no idea what tool was used to create the game.

Searching for .arcd file type did not give me any results (looking for game.arcd would have helped, but I only learned that later). Instead I ran strings on the executable file to see messages are inside it:

$ strings -n 10 executable |wc -l
5799
 
$ strings -n 10 executable |grep -iE 'error|fail'
...
ERROR:DLIB: dmLog already initialized
...
 
$ strings -n 10 executable
...
dmGraphics::ValidateAsyncJobProcessing
dmGraphics::OpenGLClear
dmGraphics::OpenGLFlip
...

I wanted at least 10 characters because otherwise it would just be too much output - 5800 lines were enough already. Then I looked for non-generic error messages and non-standard API strings, in the hope that that would tell me which engine was in use.

Searching for both dmLog and dmGraphics told me that the Defold game engine was used here.

Existing tools​

I looked for existing tools that could unpack the game.arcd file and found Unfold, a unpacker written in Defold itself. Unfortunately it did not work.

Archive format​

The Defold documentation talks about the game archive format, and it did not seem to be very complicated.

When inspecting the Defold source code I saw that there were .proto files, which are specifications for "Protocol buffers", a binary serialization format. This proto files can be compiled into Java, C, C++, PHP and other languages. That's enough to read all the files.

Let's get coding​

I also found a ArchiveReader.java file that already implemented a file extractor! So I downloaded IntelliJ idea, set up a new Java project and wrote a tool that utilized the reader.

Working on a 1.5GiB file would be slow, so I checked out the Defold games showcase and downloaded a game that looked small: At first I used the browser's network inspector to obtain archive_files.json and then downloaded all files linked from it:

$ jq -r .content[].pieces[].name < archive_files.json | xargs -L1 I{} wget http://example.org/archive_files_dir/{}

The files extracted by my tool were of no use. After some tinkering I found out that files can be compressed and encrypted, and both features were not implemented in the ArchiveReader class.

An hour later I had a BetterArchiveReader with support for both compression and decryption, and it was able to extract .luac and .texturec files.

The .luac files has the Lua source code, but also some binary before and after the actual script code. This means the files were again packed protobufs, and needed to be unpacked.

Texture format​

.texturec files may contain multiple texture formats, and each of them had meta data about the actual image format. The files I was interested in had Compression: basis UASTC, which turned out to be a format suitable for directly loading onto the graphics card.

Luckily the basis_universal basisu command line tool already contains an unpacker that is able to convert .basis texture files back into .png!

Prettification​

The tool itself worked, but had hard-coded paths and I had to comment out code to switch between extraction and listing.

I decided to add a nice command line interface so others would not have to go through the same hoops as I. After evaluating Java command line parsing libraries I decided on JCommander.

After the initial work of about 5 hours, I spent 1.5 days to build the cli interface. It even has a pretty help screen:

$ java -jar dist/arcdEx-full-0.2.0.jar
Usage: arcdex [options] [command] [command options]
  Options:
    -h, --help
 
  Commands:
    archive      Inspect a Defold .arcd archive
      Usage: archive [options] <.arcd data file> [files to extract]
        Options:
          --arci
            .arci index file path
          --dmanifest
            .dmanifest file path
          --extract-lua
            Extract Lua scripts from .luac files
            Default: false
          --extract-textures
            Extract textures from .texturec files
            Default: false
          -f, --filter
            File extension filter
          -l, --list
            List archive contents
            Default: false
          --outdir
            Directory to extract files to
          -v, --verbose
            Show names of extracted files
            Default: false
 
    lua      Inspect or extract a .luac file
      Usage: lua [options] <.luac file path>
        Options:
          -v, --verbose
            Show names of extracted files
            Default: false
          -i
            Show .luac file information. Do not extract.
            Default: false
 
    texture      Inspect or extract a texture file
      Usage: texture [options] <.texturec file path>
        Options:
          -v, --verbose
            Show names of extracted files
            Default: false
          -i
            Show texture information. Do not extract.
            Default: false

Download​

The tool is called arcdEx, is open source and can be downloaded from codeberg.org/cweiske/arcdEx.