Work on my own server for the abandoned PlayJam GameStick micro console continued, and two weeks ago I deemed it good enough to make it public for others to test.
To make sure everything works fine I removed the GameStick host name entries from my local DNS server, factory reset the GameStick and started the initial setup. It all worked fine as expected, but after completing setup the screen went black and stayed that way. Only notifications that a Gamepad was connected/disconnected or the internet connection was established were shown in the bottom right corner. The GameStick did not react to gamepad nor keyboard input.
Since had reinstalled the 2071 firmware as part of my test, I had no access to adb - for that I had to install and start the TOFU media player, install and run my "start adb" plugin. None of this was an option since the main UI ("Console") did not load.
Git backups
Flashing firmware 2058 gave the same black screen that 2071 showed.
I suspected something to be off with my API and changed the connect API responses to known-to-work versions from git history of the playjam gamestick API code, but that did not help. Minimizing them did not help either.
Next I tried to replace the profile API with a version that I used when I got it working for the first time, but that did also not help.
I also suspected the session IDs to have invalid characters, but changing them did not yield any results. The black screen stayed.
Another thing that the official connect API responses had were UI translation strings, so I integrated them - but no avail.
The first three days of poking in the dark were over.
Cache files
During development I had collected a number of server responses that were cached on different GameSticks (Toast, Cataphoresis, Ryo, Kazdan, Lee Chapman). I let my server return those files, but the screen still stayed black.
Since I knew that the UI did not show the latest data when they were downloaded from the server, I had to boot the GameStick once to the black screen, wait for it to download all API data, and rebooted again to see if the new data made any difference.
Nothing.
I also replaced the small .jpg profile images with original large .png files.
From previous experiments I knew that OOBE in firmware 2071 did not show the profile image, while 2058 does. So I tried to find out what changed between those two firmware versions: 2058 used the custom JSON handler, while 2071 had completely switched to a Gson to parse and hydrate the API responses.
There were no differences to find in the parsed properties. Day 4 over.
Android emulator
My main problem was that I did not have error logs because the PlayJam developers had disabled adbd on the GameStick. My next idea was to install all the GameStick .apk files inside the Android emulator and see the error in its logs.
Fortunately I could configure a 4.1 Android system for the emulator, start it and install all com.playjam.* apk files that were part of the 2071 firmware. I could start OOBE, but it would hang in the 5th step, the activation. Logs said that it could not obtain the hardware ID which is needed in requests to the API server.
I dived deep into the decompiled code, found out how the GameStick loads its hardware ID, built an Android app that provides a system service that starts on boot and sends out the com.playjam.SYSTEM_INFO intent with a fake hardware ID and firmware version information.
I learned the hard way that building an app with only a service and without any user interface will never be marked as "activated", and thus its services will never be started by the Android system. So I had to build a dummy UI activity that started the service once.
In the end, my service worked and the GameStick apks could fetch the correct API from the server, and OOBE finished.
The Console UI had some errors because certain files and folders were missing in the /data user partition. I copied them over from the firmware image, and the GameStick intro video played!
Unfortunately the UI crashed while the spinner was rotating, because playing a sound did not work in the emulator:
W/AudioPolicyManagerBase( 654): getOutput() could not find output for stream 3, samplingRate 0,format 0, channels 3, flags 0 E/AudioTrack-Java( 2246): [ android.media.AudioTrack ] getMinBufferSize(): error querying hardware W/dalvikvm( 2246): JNI WARNING: JNI method called with exception pending W/dalvikvm( 2246): in Lcom/ideaworks3d/marmalade/LoaderThread;.runOnOSTickNative:()V (GetObjectClass) W/dalvikvm( 2246): Pending exception is: I/dalvikvm( 2246): java.lang.NullPointerException: I/dalvikvm( 2246): at com.ideaworks3d.marmalade.SoundPlayer.start(SoundPlayer.java:75) I/dalvikvm( 2246): at com.ideaworks3d.marmalade.LoaderThread.soundStart(LoaderThread.java:856)
But this was a kind of success: The intro video was not visible on my real factory-reset GameStick.
Day 6 over.
GameStick CFW
So my GameStick stays completely black but the emulator shows the intro video: I needed to get back to real hardware. One thing I had not yet tried was CFW 1.4, the Custom FirmWare by shanti (GameStickers.net: [ROM] CFW v1.3 - Updated 1.4 is out :D, archived version). It hopefully has adbd enabled, which would give me logs.
I flashed it onto the GameStick, and it booted into a standard Android user interface (the whole point of the firmware was to get an usable Android without any restrictions). Then I installed all the com.playjam.*.apk files and populated /data as I had when using the emulator.
The results were good: OOBE worked, intro video plays, the spinner is visible and then the normal UI starts and is usable - only game images did not load.
But while it was nice that I got the official UI working on custom firmware, I did not find out why the official firmware stayed black.
Day 7 was gone.
Custom firmware
When inspecting the CFW 1.4 files I noticed /system/build.prop which contained three additional lines:
persist.service.adb.enable=1 persist.service.debuggable=1 persist.sys.usb.config=mass_storage,adb
If the official firmware had this, I'd get proper error logs and could see the error messages.
I learned that the GameStick firmware .img files are standard Android OTA .zip files and not some custom format.
I also learned that firmware files need to be signed, so I used the standard java method for signing:
$ signapk -a 4 --min-sdk-version 16 --disable-v2 certs/certificate.pem certs/key.pk8 image.tmp.zip image.signed.img
.. but when trying to flash that image I got:
Verifying update package...
Installation aborted.
I tried different parameter combinations (align/noalign, v1/v2) but all failed.
Then I read all 28 pages of the archived CFW thread on gamestickers.net and found someone who asked how the image was signed. shanti had answered:
here is the program I used: "Sign-em! 2.0"
http://forum.xda-developers.com/showthread.php?t=1966007
I got the linux version and found it used signapk.jar. After finding that I saw that it used some internal old Sun Java class sun.misc.BASE64Encoder that was not available anymore in any recent OpenJDK :(
Android system recovery normally only installs firmware images
that have been signed by one of a number of white-listed keys.
The GameStick's recovery seems to allow all properly signed images,
regardless which signing key was used.
This is the reason the CFW was possible at all
- on OUYA, custom firmwares could only be installed when using a
custom bootloader,
but not with the stock recovery that requires firmwares to be signed
by a key that only OUYA possessed.
Then I found the next XDA thread post that linked to HemanthJabalpuri/signapk which contained MinSignApk 1.0 that ran on current Java versions!
I signed my firmware image with MinSignApk, and could flash it!
Update 2023-06: An adbd-enabling firmware image is available at codeberg.org/gamestick-fans/firmware-adb-enabler/
Errors
Now that I had flashed my custom firmware 2071+adb I finally saw the errors:
Download directory
I/ActivityManager( 3509): START {act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] flg=0x10000000 pkg=com.playjam.gamestick.console cmp=com.playjam.gamestick.console/.Main u=0} from pid 3754
D/GameStick( 4262): ****************************************
D/GameStick( 4262): Failed to create download directory
The directory existed, so I removed it:
$ adb shell rmdir /data/data/com.playjam.consoledev/files/Downloads
Next boot, the GameStick would create the directory, and the following it would complain again. So this was not the problem.
readAsciiLine
I/ActivityManager( 4479): Displayed com.playjam.gamestick.console/.Main: +313ms (total +1s416ms) W/System.err( 4628): java.io.EOFException W/System.err( 4628): at libcore.io.Streams.readAsciiLine(Streams.java:203)
The firmware update check needs to return its JSON on a single line, otherwise parsing would fail. I modified the server.
OpenTextureFile
D/PlayjamKeyboard( 3685): Starting for type : 0/0 I/marmalade( 4262): PJException error has occurred : 62 c:/_work/dev/marmalade_main/menu_head/src/PNGFile.cpp OpenTextureFile open png Error : empty file name
This finally looked like the real problem. I removed all game information from the API in case the game images were the culprit, as well as the profile images. Still the same error.
Day 8 was over.
The next day I tried more image related changes:
- Use .png profile images
- Provide a lastAchievementImageUrl as .png
- Remove profile images again
A race condition
While booting the GameStick I came across a very special error:
W/DatabaseService:ConnectDownloader( 3794): Failed to duplicate connect data for console : /data/GameStick/ConsoleResources/315f7b66-fae7-4ab3-9d46-bf41942694b6.json => /data/GameStickCache/reg_server_response.json
Reading through the source code led to me understanding that process:
- GameStick user interface requests the game data in mode cache_then_fresh. This means that cached data should be returned immediately, but a background request to get fresh data would be started afterwards.
- The cached response data is returned gets parsed by ConnectDownloader
- In the background, the new data are fetched from the server.
- New data download finishes and the old cached data file gets deleted.
- ConnectDownloader finishes parsing the old file and wants to copy it to a different place, so that the "console" UI process can use it.
The last step failed because downloading fresh data (500 kiB!) was faster than parsing cached data.
This could only happen on my local setup: GameStick's WiFi reception is so bad that I could not use my normal access points. Instead I enable an access point on my laptop whenever I do GameStick development, and the GameStick only needs to transmit data some 30cm to my laptop. Since the API server is running on the same machine, data transmission is much faster than usual, leading to that race condition.
This problem was fixed with a sleep(2) call to slow down the API.
/data
The GameStick still gave the OpenTextureFile error with no hint about the file it tried to read.
I tried my luck, built a 2058 firmware with adbd and did get a more verbose error:
I/marmalade( 5452): PJException error has occurred : 97 c:/_work/dev/marmalade_main/menu_head/src/PNGFile.cpp OpenTextureFile open png Error : Cannot open file raw:///data/GameStickCache/Assets/textures/placeholder3.png
That was very strange. I knew that the firmware image contains nearly 1000 files in the /data folder, and confirmed that this particular file also exists in both 2058 and 2071 firmware files.
It turned out that /data/GameStickCache/Assets/ was completely missing on my device, as well as /data/GameStickCache/Resources/. What the heck?
Now I remembered that when flashing firmware via system recovery, I always do two steps to get a nice clean system:
- Apply update(*.img)
- Wipe data/factory reset
But the GameStick firmware installation process already populates the user data partition /data/! Wiping user data after installation removes all the asset files that are needed by the console user interface, leaving a broken system.
Now it also made sense that I did not see this problems on the emulator and the CFW setup: I knew those files were missing and had manually copied them from the firmware update into /data.
Day 9 had finally brought relief.
Fin
The key takaway is that the PlayJam developers did not follow standard Android conventions and put necessary system data onto the user partition.
When flashing a GameStick, always wipe user data first, and then flash the firmware .img file. Never the other way round.