The latest posts in full-text for feed readers.
I'm on the quest to make PlayJam GameStick micro consoles usable again. Building a replacement server is underway, and collecting game information is nearly finished - but not much progress has been made obtaining the actual .apk files.
37 apks could be downloaded from the PlayJam Amazon AWS S3 bucket, but the rest of them are not accessible although we know the URLs. They are probably only accessible with a temporary access token that the GameStick server once generated.
We contacted a number of developers and asked for the apks, but most don't have them anymore - they are lost. Some developers don't even remember having programmed for the GameStick :) And many don't react on e-mails.
The last resort is to get the .apk files off the GameSticks that have them installed. PlayJam wanted to prevent that, so they locked the GameStick down, preventing access to the Android settings, disabling the adb daemon and even removing the functionality to sideload games (which was available in early firmwares).
Simply starting the Android settings or adbd via the advanced launcher TOFU plugin did not work. I needed to find a way to inspect the GameStick and run applications on them to see what worked. Because adb access was not available, I decided to build my own telnet server that would run as TOFU plugin on the GameStick.
I found a nice Python 2 socket server example and used that as base to build my own server that I could connect to with a telnet client. Development was easy because I could run the server locally. Only at the end I built a TOFU plugin that simply started the telnet server script.
One thing I learned was that the shell (my server) needs to take care of storing the current working directory, and also needs to implement change directory commands - those cannot be delegated to the system. After 2.5 hours I had a telnet server that ran on the GameStick, and I could finally inspect the file system and run programs:
cweiske:~/dev/gamestick/sync/tofu-plugin> telnet gamestick 5023
Trying 192.168.3.41...
Connected to gamestick.
Escape character is '^]'.
/> ls
acct
backup
boot
cache
[...]
/> whoami
u0_a40
/> am start -n com.android.settings/.Settings
[Errno 13] Permission denied
Now I could try to open the Android settings (which still fails) and start adbd - it turned out that running programs on the GameStick is only possible as root:
$ telnet gamestick 5023
Trying 192.168.3.41...
Connected to gamestick.
Escape character is '^]'.
/> su -c start adbd
/> quit
Connection closed by foreign host.
cweiske:~/dev/gamestick/sync/tofu-plugin> adb connect gamestick
already connected to gamestick:5555
cweiske:~/dev/gamestick/sync/tofu-plugin> adb shell
root@android:/ # pwd
/
root@android:/ # ls /data
GameStick
GameStickCache
The result of this endeavour are two TOFU plugins:
Update 2023-06: An adbd-enabling firmware image is available at codeberg.org/gamestick-fans/firmware-adb-enabler/ - see GameStick: Black screen after OOBE.
Published on 2023-05-07 in gamestick
Work on my own server for the abandoned PlayJam GameStick micro console continued, and two weeks ago I deemed it good enough to make it public for others to test.
To make sure everything works fine I removed the GameStick host name entries from my local DNS server, factory reset the GameStick and started the initial setup. It all worked fine as expected, but after completing setup the screen went black and stayed that way. Only notifications that a Gamepad was connected/disconnected or the internet connection was established were shown in the bottom right corner. The GameStick did not react to gamepad nor keyboard input.
Since had reinstalled the 2071 firmware as part of my test, I had no access to adb - for that I had to install and start the TOFU media player, install and run my "start adb" plugin. None of this was an option since the main UI ("Console") did not load.
Flashing firmware 2058 gave the same black screen that 2071 showed.
I suspected something to be off with my API and changed the connect API responses to known-to-work versions from git history of the playjam gamestick API code, but that did not help. Minimizing them did not help either.
Next I tried to replace the profile API with a version that I used when I got it working for the first time, but that did also not help.
I also suspected the session IDs to have invalid characters, but changing them did not yield any results. The black screen stayed.
Another thing that the official connect API responses had were UI translation strings, so I integrated them - but no avail.
The first three days of poking in the dark were over.
During development I had collected a number of server responses that were cached on different GameSticks (Toast, Cataphoresis, Ryo, Kazdan, Lee Chapman). I let my server return those files, but the screen still stayed black.
Since I knew that the UI did not show the latest data when they were downloaded from the server, I had to boot the GameStick once to the black screen, wait for it to download all API data, and rebooted again to see if the new data made any difference.
Nothing.
I also replaced the small .jpg profile images with original large .png files.
From previous experiments I knew that OOBE in firmware 2071 did not show the profile image, while 2058 does. So I tried to find out what changed between those two firmware versions: 2058 used the custom JSON handler, while 2071 had completely switched to a Gson to parse and hydrate the API responses.
There were no differences to find in the parsed properties. Day 4 over.
My main problem was that I did not have error logs because the PlayJam developers had disabled adbd on the GameStick. My next idea was to install all the GameStick .apk files inside the Android emulator and see the error in its logs.
Fortunately I could configure a 4.1 Android system for the emulator, start it and install all com.playjam.* apk files that were part of the 2071 firmware. I could start OOBE, but it would hang in the 5th step, the activation. Logs said that it could not obtain the hardware ID which is needed in requests to the API server.
I dived deep into the decompiled code, found out how the GameStick loads its hardware ID, built an Android app that provides a system service that starts on boot and sends out the com.playjam.SYSTEM_INFO intent with a fake hardware ID and firmware version information.
I learned the hard way that building an app with only a service and without any user interface will never be marked as "activated", and thus its services will never be started by the Android system. So I had to build a dummy UI activity that started the service once.
In the end, my service worked and the GameStick apks could fetch the correct API from the server, and OOBE finished.
The Console UI had some errors because certain files and folders were missing in the /data user partition. I copied them over from the firmware image, and the GameStick intro video played!
Unfortunately the UI crashed while the spinner was rotating, because playing a sound did not work in the emulator:
W/AudioPolicyManagerBase( 654): getOutput() could not find output for stream 3, samplingRate 0,format 0, channels 3, flags 0 E/AudioTrack-Java( 2246): [ android.media.AudioTrack ] getMinBufferSize(): error querying hardware W/dalvikvm( 2246): JNI WARNING: JNI method called with exception pending W/dalvikvm( 2246): in Lcom/ideaworks3d/marmalade/LoaderThread;.runOnOSTickNative:()V (GetObjectClass) W/dalvikvm( 2246): Pending exception is: I/dalvikvm( 2246): java.lang.NullPointerException: I/dalvikvm( 2246): at com.ideaworks3d.marmalade.SoundPlayer.start(SoundPlayer.java:75) I/dalvikvm( 2246): at com.ideaworks3d.marmalade.LoaderThread.soundStart(LoaderThread.java:856)
But this was a kind of success: The intro video was not visible on my real factory-reset GameStick.
Day 6 over.
So my GameStick stays completely black but the emulator shows the intro video: I needed to get back to real hardware. One thing I had not yet tried was CFW 1.4, the Custom FirmWare by shanti (GameStickers.net: [ROM] CFW v1.3 - Updated 1.4 is out :D, archived version). It hopefully has adbd enabled, which would give me logs.
I flashed it onto the GameStick, and it booted into a standard Android user interface (the whole point of the firmware was to get an usable Android without any restrictions). Then I installed all the com.playjam.*.apk files and populated /data as I had when using the emulator.
The results were good: OOBE worked, intro video plays, the spinner is visible and then the normal UI starts and is usable - only game images did not load.
But while it was nice that I got the official UI working on custom firmware, I did not find out why the official firmware stayed black.
Day 7 was gone.
When inspecting the CFW 1.4 files I noticed /system/build.prop which contained three additional lines:
persist.service.adb.enable=1 persist.service.debuggable=1 persist.sys.usb.config=mass_storage,adb
If the official firmware had this, I'd get proper error logs and could see the error messages.
I learned that the GameStick firmware .img files are standard Android OTA .zip files and not some custom format.
I also learned that firmware files need to be signed, so I used the standard java method for signing:
$ signapk -a 4 --min-sdk-version 16 --disable-v2 certs/certificate.pem certs/key.pk8 image.tmp.zip image.signed.img
.. but when trying to flash that image I got:
Verifying update package...
Installation aborted.
I tried different parameter combinations (align/noalign, v1/v2) but all failed.
Then I read all 28 pages of the archived CFW thread on gamestickers.net and found someone who asked how the image was signed. shanti had answered:
here is the program I used: "Sign-em! 2.0"
http://forum.xda-developers.com/showthread.php?t=1966007
I got the linux version and found it used signapk.jar. After finding that I saw that it used some internal old Sun Java class sun.misc.BASE64Encoder that was not available anymore in any recent OpenJDK :(
Android system recovery normally only installs firmware images
that have been signed by one of a number of white-listed keys.
The GameStick's recovery seems to allow all properly signed images,
regardless which signing key was used.
This is the reason the CFW was possible at all
- on OUYA, custom firmwares could only be installed when using a
custom bootloader,
but not with the stock recovery that requires firmwares to be signed
by a key that only OUYA possessed.
Then I found the next XDA thread post that linked to HemanthJabalpuri/signapk which contained MinSignApk 1.0 that ran on current Java versions!
I signed my firmware image with MinSignApk, and could flash it!
Update 2023-06: An adbd-enabling firmware image is available at codeberg.org/gamestick-fans/firmware-adb-enabler/
Now that I had flashed my custom firmware 2071+adb I finally saw the errors:
I/ActivityManager( 3509): START {act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] flg=0x10000000 pkg=com.playjam.gamestick.console cmp=com.playjam.gamestick.console/.Main u=0} from pid 3754 D/GameStick( 4262): **************************************** D/GameStick( 4262): Failed to create download directory
The directory existed, so I removed it:
$ adb shell rmdir /data/data/com.playjam.consoledev/files/Downloads
Next boot, the GameStick would create the directory, and the following it would complain again. So this was not the problem.
I/ActivityManager( 4479): Displayed com.playjam.gamestick.console/.Main: +313ms (total +1s416ms) W/System.err( 4628): java.io.EOFException W/System.err( 4628): at libcore.io.Streams.readAsciiLine(Streams.java:203)
The firmware update check needs to return its JSON on a single line, otherwise parsing would fail. I modified the server.
D/PlayjamKeyboard( 3685): Starting for type : 0/0 I/marmalade( 4262): PJException error has occurred : 62 c:/_work/dev/marmalade_main/menu_head/src/PNGFile.cpp OpenTextureFile open png Error : empty file name
This finally looked like the real problem. I removed all game information from the API in case the game images were the culprit, as well as the profile images. Still the same error.
Day 8 was over.
The next day I tried more image related changes:
While booting the GameStick I came across a very special error:
W/DatabaseService:ConnectDownloader( 3794): Failed to duplicate connect data for console : /data/GameStick/ConsoleResources/315f7b66-fae7-4ab3-9d46-bf41942694b6.json => /data/GameStickCache/reg_server_response.json
Reading through the source code led to me understanding that process:
The last step failed because downloading fresh data (500 kiB!) was faster than parsing cached data.
This could only happen on my local setup: GameStick's WiFi reception is so bad that I could not use my normal access points. Instead I enable an access point on my laptop whenever I do GameStick development, and the GameStick only needs to transmit data some 30cm to my laptop. Since the API server is running on the same machine, data transmission is much faster than usual, leading to that race condition.
This problem was fixed with a sleep(2) call to slow down the API.
The GameStick still gave the OpenTextureFile error with no hint about the file it tried to read.
I tried my luck, built a 2058 firmware with adbd and did get a more verbose error:
I/marmalade( 5452): PJException error has occurred : 97 c:/_work/dev/marmalade_main/menu_head/src/PNGFile.cpp OpenTextureFile open png Error : Cannot open file raw:///data/GameStickCache/Assets/textures/placeholder3.png
That was very strange. I knew that the firmware image contains nearly 1000 files in the /data folder, and confirmed that this particular file also exists in both 2058 and 2071 firmware files.
It turned out that /data/GameStickCache/Assets/ was completely missing on my device, as well as /data/GameStickCache/Resources/. What the heck?
Now I remembered that when flashing firmware via system recovery, I always do two steps to get a nice clean system:
But the GameStick firmware installation process already populates the user data partition /data/! Wiping user data after installation removes all the asset files that are needed by the console user interface, leaving a broken system.
Now it also made sense that I did not see this problems on the emulator and the CFW setup: I knew those files were missing and had manually copied them from the firmware update into /data.
Day 9 had finally brought relief.
The key takaway is that the PlayJam developers did not follow standard Android conventions and put necessary system data onto the user partition.
When flashing a GameStick, always wipe user data first, and then flash the firmware .img file. Never the other way round.
Published on 2023-06-19 in bigsuck, gamestick
My replacement server for the PlayJam GameStick micro console is ready for semi-public testing. It allows you to finish the initial setup (OOBE), list all the games, and download + play 130 of them.
It's not complete yet: We know of games that cannot be controlled properly because button mappings are missing, and achievements are not yet stored on the server. Firmware updates are not distributed. Probably many other issues are waiting to be found.
It is semi-public because GameStick devices have to be white-listed by me.
When your GameStick is stuck in the initial setup, attach a keyboard. On the language selection screen, type "hardware" and your GameStick's hardware ID will be shown in the top left corner.
Send that hardware ID by mail to me via cweiske+gamestick@cweiske.de or join the GameStick Fans Discord chat server and contact me there.
If the hardware ID does not appear, continue until WiFi setup and complete it. The GameStick will contact my server automatically and I can read the ID from the logs. Open ip.cweiske.de in your browser and send me that IP address, so I can correlate IP and hardware ID.
The GameStick setup in screenshots:
The Gamestick OOBE lets you select the language, setup WiFi and shows a verification code:
Then you need to continue in your browser: Enter a name, select a profile image and choose the age rating for games.
When you finish setup in the browser, the GameStick will show your "Gamer tag" (name) and the profile image (only on firmware 2058, non on 2071):
Afterwards the boot video will play and you'll the the main user interface that you can use to download + play games:
Games names beginning with !!
indicate that they cannot be downloaded.
We currently have .apk files for 131 of the 188 games.
Published on 2023-06-19 in gamestick
The PlayJam GameStick was funded on Kickstarter in 2013, around the same time as the OUYA. It had the same fate as the OUYA as well: In 2016 the GameStick servers became unresponsive, and 2017 PlayJam announced the official death of the micro console.
After rescuing the OUYA by building a replacement server I needed a new hobby and bought a PlayJam GameStick.
Unlike the OUYA, the GameStick did not have a strong community that raced to backup everything when the server shutdown was announced, and that left me with .. not much. No GameStick game meta data, no game apks, no API documentation, and even the "gamestickers.net" forum had long been shut off in 2023.
What I had were two firmware update files, GameStick-Software-v2058.img and GameStick-Software-v2071.img.
So equipped with the venerable jadx-gui, mitmproxy and a lot of unix command line tools I began to look at the GameStick's core applications. Luckily the source code had not been obfuscated, which helped tremendously during reverse engineering.
The first problem was the initial setup: After language selection, screen setup and WiFi connection the OOBE checked for firmware updates and failed. Without a proper response to the update request, it would not go any further and just hang.
mitmproxy told me that the GameStick made a HTTP POST request to http://update.gamestickservices.net/check.php, and by searching for that URL in the decompiled source code I found the place the response was parsed. I only had to write a text file that told the GameStick that no update was available:
{"available": false}
.. and the update check could be bypassed.
The last step in the initial setup was the registration: It told you to point your browser to the non-existing URL wwww.gamestick.tv/activate and follow the instructions. The GameStick would show a registration code that had to be entered in the browser, and in the end everything would be fine.
Some more debugging and guessing let me get around that by providing two files: A device information and a player profile file. Success!
The GameStick user interface was open! I saw that I had a very early firmware 0.1.53 and updated it to the latest 0.9.2071. Unfortunately, the registration did not work anymore because the new firmware expected some different fields in the JSON server responses. After figuring that out, the GameStick UI stayed black.
Downgrading to 0.9.2058 and playing around with the JSON files finally brought the UI back.
In the GameStick Fans discord chat server @Toast gave me a copy of cache files that were stored on his GameStick, and from there I could extract the meta data of 31 games, and download the .apk of 11 games. Not the 180 that were once available, but a start.
It took another week of tinkering until I had the first game showing up on the "featured" screen. Details with video and screenshots worked, but it would not download.
Today, an evening later, I added a missing "version" property to the app's "download" object and the game installed. This is the first time in 6 years that a game had been downloaded and installed on a PlayJam GameStick.
Now we need to get game data off a GameStick that was online in 2015 or later, so that all the games can be made available. Also, the rest of the API needs to be reverse engineered, documented and a proper community replacement server needs to be written. Fun times ahead!
Published on 2023-04-17 in gamestick
After building a minimal server for my PlayJam GameStick micro gaming console, I sought a way to get cached game data off existing GameSticks.
Nobody had cared enough to obtain copies of the server API responses before the official server had been shut down in 2017, and now we had nearly no information about the games that could once be bought, downloaded, installed and played on a GameStick. I already had the data for 31 games, but none for 157 others that were once available.
One day GameStick Fans discord member alex.shorts wrote:
@cweiske I have a GameStick with the tofu media player, and used it in the days before the servers were shut down
This was the jackpot! All game data would be cached on it, we just had to get the files.
GameSticks came pre-installed with the "TOFU Media Center", which was a fork of XBMC 12 tailored for the GameStick. XBMC (now Kodi) has support for plugins, and using such a plugin to enable Android's debug bridge adb was the most promising way to reach that goal: We wanted to run am start -n com.android.settings/.Settings to open standard Android settings and enable adb via the developer settings.
alex.shorts tried the gamestickers rooting tutorial but failed because the "Advanced Launcher" TOFU plugin was not even able to create a start script - it failed with an error that others had seen before:
Cannot create launchers.xml file
Using another "Advanced Launcher" version yielded the same error.
When we cannot open the settings to enable adb, then we need a TOFU plugin that copies the cache files onto a SD card.
When programming, the program (here the plugin) needs to be tested. Testing on GameStick's TOFU means:
This process takes ~2 minutes, which is way too long - when changing a line of code in the editor, I want feedback quickly.
Next I tried to compile XBMC 12 on my Linux laptop. After investing 1 hour to obtain and install all required dependencies, the "make" script failed because the 2023 C compiler does not like to compile code that the 2013 C compiler had no problems with.
Since my C skills are sub-par I ditched that idea.
swig libboost-dev python2-dev libass-dev libmpeg2-4-dev libmad0-dev libsamplerate-dev libvorbis-dev libmodplug-dev libcurl4-gnutls-dev libflac-dev libbz2-dev liblzo2-dev libyajl-dev libtinyxml-dev libcdio-dev libsdl1.2-dev libsdl-image1.2-dev libsdl-mixer1.2-dev libxmu-dev libssh-dev libsmbclient-dev gperf libiso9660-dev yasm
When compilation fails, use a pre-compiled XBMC. There were no pre-compiled XBMC 12 to find for Linux, but for Windows!
So I installed wine and could finally start XBMC.
With a local XBMC running, I could install a minimal plugin version and the continue editing the python file in ~/.wine/drive_c/users/cweiske/Application Data/XBMC/addons/. That allowed me to quickly test changes, even if I could not use it to test the actual GameStick directory copying code.
As soon as the Windows plugin version worked, I adjusted the paths to be GameStick-compatible, zipped the files and installed it in TOFU - and it worked!
The script.copygamestickcache code and ready-to-install plugin files are available.
A couple of days later, alex.shorts used the plugin to get the cache files off his GameStick and sent them to me.
In the meantime, I had built a schema for game data files as well as some scripts to import the cache files and download the available .apk, movie and image files. We now have complete data of 183 games!
5 games had been pulled from the store before shutdown, and so we still need some donor who can provide cache files from 2015.
Only 33 .apk files could be downloaded from S3, the others are blocked - probably only accessible with temporary access tokens that once were generated by the GameStick server.
The next step is to build a TOFU plugin that copies all .apk files off the GameStick.
Published on 2023-05-02 in gamestick