questshock/README.md

# Questshock

![questshock](res/logo_small.png)

An open source project to play the classic 1994 System Shock on a VR
headset, built on top of [Shockolate](https://github.com/Interrupt/systemshock),
a cross-platform port of the original game.

## 1. Design principles

- **Standalone-headset-only.** The game must run entirely on the headset
  itself - no PC required, whether tethered or streamed (not PCVR). Any
  feature or dependency that assumes a host PC is out of scope.
- **OpenXR-first.** Favor OpenXR-standard APIs over vendor-specific ones
  (e.g. Meta's Oculus Mobile SDK) wherever there's a choice, so the port
  isn't locked to Meta Quest and can work across other standalone,
  Android-based OpenXR headsets (e.g. Pico) too.

## 2. Layout

- `engine/` - a vendored snapshot of the Shockolate engine source. Built
  via Docker; see "4. Desktop build" below.
- `build-image/` - the Dockerfile (and supporting scripts) for the engine
  build environment. Every third-party dependency the engine needs to
  compile (SDL2, SDL2_mixer, the fluidsynth-lite MIDI synth, a MIDI
  soundfont) is fetched and built once into this image - compiling
  `engine/` itself needs no network access.
- `build-image.sh` / `run-image.sh` / `upload-image.sh` - build the image,
  run it to compile the engine, and push it to a registry, respectively.
- `res/assets/` - where you place your own purchased copy of the game (see
  "3. Game assets" below); `res/assets/extract_assets.sh` extracts it into
  `ss_ee/`.
- `res/run.sh` - the launcher script, copied into `dist/` on build.
- `Makefile` - targets:
  - `all` (default) - alias for `dist`.
  - `build-image` - builds the Docker build-image (`./build-image.sh`).
    Only needed once, or after `build-image/` changes.
  - `engine` - compiles `engine/` (the vendored Shockolate snapshot) via
    the build-image, offline. Always re-run by the targets below, so
    `dist`/`package`/`apk` stay in sync with the current `engine/` source.
  - `assets` - preflight check only: fails with a pointer to
    `res/assets/extract_assets.sh` if the purchased game assets (see "3.
    Game assets" below) haven't been extracted yet.
  - `dist` - assembles `dist/`, a self-contained runnable copy of the
    game, out of the compiled engine and the extracted assets (depends on
    `engine` + `assets`). See "4. Desktop build" below.
  - `package` - builds a versioned, redistributable
    `dist/shockolate-<version>-linux-<arch>.tar.gz` that omits the
    proprietary game assets (ships `res/GET_ASSETS.txt` in their place).
    See "4.1. Packaging a distributable build" below.
  - `apk` - builds `dist/questshock-<version>-android-arm64.apk` for
    Android-based OpenXR headsets. Unlike every other target, this needs
    network access at build time (Gradle/AGP's own dependency
    resolution). See "5. Android / Quest build" below.
  - `android-studio` - stages `engine/`/`android/gl4es-src/` (scratch,
    patched copies) and the Android prebuilts with host-resolvable paths,
    so Android Studio can compile and deploy `android/` natively instead
    of inside the build-image. See "5.2. Building natively in Android
    Studio" below.
  - `clean` - removes all build output (`dist/`, `build/`, compiled
    engine artifacts, and the staged Android project files).
- `android/` - the Quest app (Java `SDLActivity` glue, Gradle project).
  `android/engine-patches/` holds the patches needed to build `engine/`
  as an Android shared library instead of a desktop executable - applied
  to a scratch copy at build time; `engine/` itself is never modified.
  `android/gl4es-src/` similarly vendors GL4ES (see "6. License" below), with
  `android/gl4es-patches/` applied to a scratch copy at build time (same
  as `engine/`) and compiled from source alongside it, not prebuilt. See
  "5. Android / Quest build" below for what each patch does.

## 3. Game assets

System Shock's game data is not included in this repository and cannot
be redistributed - you need to own a copy. Buy **System Shock: Enhanced
Edition** on [gog.com](https://www.gog.com/), download the offline
installer (a `.exe`), and drop it into `res/assets/`. Then run
`res/assets/extract_assets.sh`, which pulls the classic game's data and
sound files out of the installer (it's an Inno Setup package; the actual
game data lives inside it in a zip-format `sshock.kpf`) into
`res/assets/ss_ee/`. That script needs `innoextract` and `unzip`; if
they aren't installed locally it falls back to running the extraction in
a throwaway Docker container instead.

If you already have the game installed instead (on Windows, or via
Wine/Proton on Linux), you don't need the installer or the script at
all - just copy its `res/data/` and `res/sound/` folders directly into
`res/assets/ss_ee/data/` and `res/assets/ss_ee/sound/`. That's exactly
the same layout `extract_assets.sh` produces, so `make dist`/`make
package`/`make apk` pick it up the same way either way. This one copy of
game assets feeds both the desktop build and the Android/Quest build
below.

## 4. Desktop build

Builds and runs Questshock natively on Linux (your dev machine, or any
Linux box) - useful for local development and testing without a VR
headset at all. For the VR-headset build, see "5. Android / Quest build"
below instead.

```sh
# 1. Build the engine build-image (once, or after build-image/ changes)
./build-image.sh

# 2. Get your own copy of the game data (see "3. Game assets" above), then:
res/assets/extract_assets.sh

# 3. Compile the engine and assemble dist/
make dist

# 4. Play
dist/run.sh
```

`make dist` always recompiles the engine from the current `engine/`
source (via `run-image.sh`), so a fresh build-image plus a re-run of
`make dist` is all that's needed after pulling engine changes.

### 4.1. Packaging a distributable build

`make package` builds `dist/shockolate-<version>-linux-<arch>.tar.gz`: the
compiled binary, its runtime libraries, shaders, a default MIDI
soundfont, license information, and `res/GET_ASSETS.txt` in place of the
actual game data (which the tarball never includes). Version comes from
the current git tag (push a `vX.Y.Z` tag to drive a release); without one
it builds an untagged `0.0.0-dev+<sha>` placeholder. `make apk` (see
"5. Android / Quest build" below) is versioned identically, via the same
`build-image/version.sh`.

A Gitea Actions workflow (`.gitea/workflows/build.yml`) builds this
package on every push, using the build-image as its container (so no
extra setup is needed in CI beyond the image itself), and publishes the
resulting tarball to dl.ladkau.de.

## 5. Android / Quest build

`make apk` builds `dist/questshock-<version>-android-arm64.apk` - an
immersive OpenXR app (see `android/app/src/main/cpp/xr_session.c`) that
can be sideloaded onto any Android-based VR headset with OpenXR support
(Meta Quest, Pico, etc. - see "1. Design principles" above), not just one
vendor's store. The game's own rendering is unchanged (still a flat, 2D
render, no stereo 3D scene), but instead of running as a Home-hosted 2D
panel it's now shown as a single head-tracked quad floating in front of
the viewer, with a separate menu quad toggled by a controller button and
driven by a laser-pointer-style aim ray from each hand
(`android/app/src/main/cpp/xr_input.c`) - point and pull the trigger to
interact with it, same as the game's own Bluetooth mouse/keyboard input
otherwise works unchanged.

The menu launcher (`MenuOverlay.java` - just a "Keyboard" button so far)
and the keyboard (`KeyboardOverlay.java`) are two fully independent quads,
each its own `XrOverlay` instance (`android/app/src/main/cpp/xr_overlay.c`
- a generic "off-screen Android View rendered into an OpenXR quad,
hit-tested via a laser pointer" module shared by both, and by whatever
similar panel comes next) with its own swapchain, visibility, and
position, so they can be shown/hidden/moved independently - a design
specifically meant to let the keyboard stay open while also picking
something from the (still-to-be-built-out) menu. The controller's menu
button only ever opens/closes the launcher; clicking its "Keyboard" button
only opens the keyboard (never touching the launcher's own visibility) -
the keyboard's own title bar holds the only way to close it again.

The keyboard is a hand-built, full US-layout key grid (there's no system
IME to borrow once immersive) that forwards each key press straight to the
game as real input - non-printable keys (Esc/Enter/Backspace/Tab/arrows/
F1-F12) as a `SDLActivity.onNativeKeyDown()`/`onNativeKeyUp()` pair, the
same one a physical Bluetooth keyboard's presses already go through, and
printable keys (letters, digits, punctuation, Space) via a synthesized
`SDL_TEXTINPUT` event pushed directly from native code
(`questshock_native.c`). A Shift key toggles the whole grid between
lowercase/numbers and uppercase/symbols (doubling as Caps Lock - it stays
toggled until pressed again), and Ctrl/Alt arm as one-shot modifiers,
consumed by whichever key is pressed next - meant as a general stand-in
for keyboard-driven functionality that isn't (yet, or ever) mapped onto
the controllers, not just a future config screen's text entry.

Once opened, the keyboard stays up as a standing input panel while
actually playing rather than a modal you open and close. Its title bar
doubles as a drag handle (grab-and-drag with the trigger, handled entirely
on the native side in `xr_input.c` before it ever reaches
`KeyboardOverlay`'s own touch dispatch) for repositioning it, and holds
the Close button that hides it. It's also rendered semi-transparent
(`XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT` in `xr_session.c`,
set only on the keyboard's layer - the menu launcher stays fully opaque)
so whatever's behind it - the game quad, mid-play - stays visible while
it's up.

While the menu launcher or keyboard is open, each hand also gets a thin,
colored laser-beam quad (cyan left, amber right) from the controller to
wherever its ray currently crosses that panel's plane - a second,
billboarded `XrCompositionLayerQuad` per hand (`xr_input_build_beam()` in
`xr_input.c`), oriented so it reads as a line from the viewer's eye
regardless of angle, since the flat cross-shaped reticle used while
aiming at the game quad is drawn directly into the game quad's own
texture and can't represent a ray traversing real 3D space. Aiming at the
game quad itself still uses that flat reticle, unchanged. (Only tested on
Meta Quest so far -
the steps below use Quest-specific tool names where relevant, but the
same `adb install` flow applies to any Android headset with USB
debugging enabled.)

### 5.1. Installing and playing

1. Install the APK with [SideQuest](https://sidequestvr.com/) (or `adb
   install`).
2. Launch it once. It'll ask for storage permission, then create
   `/sdcard/questshock/` and extract its own bundled files (shaders, a
   default MIDI soundfont) there - `res/data/` and `res/sound/` are
   deliberately left missing, since that's the proprietary game data.
3. With the headset connected to a PC, use SideQuest's file browser (or
   any MTP file manager) to copy your own `res/data/` and `res/sound/`
   (see `/sdcard/questshock/GET_ASSETS_QUEST.txt`, extracted in step 2,
   for exactly what's needed and where it comes from) into
   `/sdcard/questshock/res/`.
4. Launch it again.

### 5.2. Building natively in Android Studio

`make apk` always compiles the engine and links the APK inside the
Docker build-image - convenient for CI/CLI builds, but Android Studio
can't attach a debugger to (or get IDE code-intelligence for) a build
that happens inside a container it isn't running.

To have Android Studio compile and deploy `android/` itself instead:

```sh
make android-studio
```

(equivalent to `./run-image.sh bash build-image/prepare-android-project.sh
--host-paths` directly.) This stages everything `make apk` normally stages
(scratch, patched copies of `engine/` and `android/gl4es-src/`; the Android
SDL2/SDL2_mixer/fluidsynth-lite/openxr prebuilts; bundled assets) - the
same as `build-apk.sh`'s own prep step - except it writes
`android/engine.properties` with paths that resolve on your host
filesystem, and additionally exports the prebuilt libraries (otherwise
only present inside the image, at `/opt/prebuilt/android`) to
`build/android-prebuilt/` so they're visible outside the container too.
gl4es itself is compiled from its scratch copy as part of Android
Studio's own native build (a CMake subdirectory of `engine/`'s build, not
a separate prebuilt step), so `android/gl4es-src/`/`android/gl4es-patches/`
changes are picked up by a normal rebuild here - no need to re-run
`./build-image.sh` first, unlike `build-image/Dockerfile` changes. Re-run
`make android-studio` whenever `engine/`, `android/engine-patches/`,
`android/gl4es-src/`, `android/gl4es-patches/`, or the prebuilt-library
versions in `build-image/Dockerfile` change (the Gradle build already
does this automatically for you via its `stageEngine` task, so this
manual re-run is mainly useful for confirming staging succeeded on its
own) - and always after `make clean`, which deletes
`android/engine.properties` along with everything else.

Then, in Android Studio, use File > Open and select the `android/`
directory itself (the one containing `settings.gradle` - not the repo
root, and not `android/app/`) to open it as a project (JDK 17, NDK
`26.1.10909125`, and SDK Platform/Build-Tools 34 installed, matching
`build-image/Dockerfile` and `android/app/build.gradle`) and build/run
normally - no Docker involved for this part.

### 5.3. Testing cycles in Android Studio

Once the project above is open, iterating against a real headset works
the same as any other Android Studio project:

1. Enable Developer Mode on the headset (via its companion phone app -
   for Quest, the Meta Horizon app's Devices/Developer Mode toggle) and
   turn on USB debugging once prompted. Connect the headset to your
   development machine with a USB cable (Wi-Fi debugging via `adb
   connect` works too, once paired once over USB).
2. Confirm it's visible with `adb devices` - it should also show up in
   Android Studio's device dropdown.
3. Press Run (or Debug, to attach breakpoints in both Java and, via
   Android Studio's "Dual"/"Native" debugger, the C code under
   `android/app/src/main/cpp/` and the staged `engine/` sources) -
   Android Studio builds, installs, and launches the app on the headset
   directly. No manual `adb install` or SideQuest step needed for this
   loop.
4. `/sdcard/questshock/` (game assets, extracted shaders/soundfont)
   persists across reinstalls from Android Studio, so this loop doesn't
   require re-copying `res/data/`/`res/sound/` on every iteration - only
   a full uninstall or wiping that directory clears it.
5. Use Android Studio's Logcat pane to watch output live; both the Java
   side and the engine's own logging (see
   `android/engine-patches/08-android-logcat-output.patch`) are tagged
   `QuestShock`, so filtering Logcat by that tag shows everything
   questshock-specific without gl4es/OpenXR loader/system noise (drop
   the filter to see those too).
6. Only re-run `make android-studio` manually (see above) after
   changing `engine/`, `android/engine-patches/`, `android/gl4es-src/`,
   `android/gl4es-patches/`, or the prebuilt-library versions in
   `build-image/Dockerfile` - the Gradle build's `stageEngine` task does
   this automatically otherwise, so plain Java/C++ edits under
   `android/app/src/main/` just need another Run/Debug press.

### 5.4. Engine/gl4es patch reference

`android/engine-patches/` (applied to a scratch copy of `engine/`) and
`android/gl4es-patches/` (applied to a scratch copy of `android/gl4es-src/`)
are both plain, numbered patch files applied in order at build time -
`engine/` and `android/gl4es-src/` themselves are never modified. What
each one does:

`android/engine-patches/`:

- **`01-android-shared-lib.patch`** - Android has no standalone
  executables (Java loads a shared library via JNI), so this builds
  `CMakeLists.txt`'s `main` target as a `SHARED` library on Android
  instead of the desktop `systemshock` executable, folding in the
  Android-native glue sources (`ANDROID_EXTRA_SOURCES`).
- **`02-android-opengl-es.patch`** - replaces the desktop
  `find_package(OpenGL)` with `add_subdirectory()`-building gl4es from
  source (translates the engine's desktop GL calls to GLES/EGL), so it
  rebuilds incrementally alongside the engine instead of needing a full
  build-image rebuild per gl4es change.
- **`03-android-gles-context.patch`** - requests a GLES 3.2 context
  instead of a desktop GL core profile (Quest's Adreno GPU supports it,
  and 3.2 has `GL_UNPACK_ROW_LENGTH`/`GL_CLAMP_TO_BORDER` as core,
  avoiding workarounds needed on GLES 2.0).
- **`04-android-opengl-es-render.patch`** - swaps a few gl4es rough edges
  (its shader-rewrite-based `glPointSize`/alpha-test/point-sprite
  emulation) for real GLES-native equivalents: a custom `pointSize`
  uniform, a shader-side alpha discard, and native point-sprite
  rasterization.
- **`05-android-audio-driver.patch`** - forces SDL2's older OpenSL ES
  audio backend instead of AAudio, because AAudio only allows one open
  playback device and the engine opens two (cutscene audio plus
  `Mix_OpenAudio` for SFX/MIDI).
- **`06-android-resize-event.patch`** - makes the engine react to
  `SDL_WINDOWEVENT_RESIZED`, not just `SIZE_CHANGED`, since Android's SDL
  backend only ever sends the former for surface-driven resizes.
- **`07-android-logcat-link.patch`** - links Android's `log` library.
- **`08-android-logcat-output.patch`** - routes the engine's own `log.c`
  (INFO/DEBUG/WARN/ERROR) through `__android_log_vprint` so it reaches
  logcat, instead of stdio (which Android never captures).
- **`09-android-no-window-resize.patch`** - skips the desktop-only
  `SDL_SetWindowFullscreen`/`SetWindowSize`/`SetWindowPosition` calls on
  Android, since there's no real desktop-style window to resize and
  calling them desyncs SDL's cached window size from the real surface.
- **`10-android-openxr-cmake.patch`** - wires up the OpenXR loader plus
  EGL as link/include dependencies for the CMake build.
- **`11-android-openxr-present.patch`** - the core OpenXR present-path
  hook: redirects `SDLDraw()`/`opengl_swap_and_restore()` to submit into
  the XR swapchain (via `xr_frame_begin()`/`xr_frame_end()`) instead of
  the normal window, for both the GL-rendered 3D path and the plain
  software-composited path (splash screen/cutscenes/menus).

`android/gl4es-patches/`:

- **`01-android-cmake-subdirectory.patch`** - since gl4es is now
  `add_subdirectory()`'d straight into the engine's own CMake configure
  (via `engine-patches/02` above) rather than built standalone, this
  skips gl4es's desktop-style output-directory/`link_directories()` setup
  and its `libGL.so.1` SONAME versioning on Android, since AGP's native
  packaging needs a plain `libGL.so` and CMake's own default naming
  already produces that correctly.

### 5.5. Debugging notes

#### 5.5.1. The menu quad rendering the game instead of itself

While building the OpenXR menu (`android/app/src/main/cpp/xr_overlay.c` -
`xr_menu.c` at the time - and `MenuOverlay.java`), the menu's
composition-layer quad consistently showed
the game's own live rendering instead of the menu's content, even though
every diagnostic (FBO bindings, swapchain/layer submission, texture
upload, viewport/scissor state) checked out correct in isolation.

**Root cause:** the menu's blit was a normal `glDrawArrays` call routed
through gl4es (the GLES/EGL translation layer the engine's desktop-style
OpenGL calls go through on Quest). gl4es's fixed-pipeline emulation
(`fpe.c`) decides whether to substitute its own generated shader for
whatever program is bound by checking `fpe_IsEmpty()` - an all-zero-bytes
check over its internal fixed-function state struct. But
`fpe_ReleventState()` unconditionally sets one field of that struct,
`alphafunc`, to a nonzero sentinel (`FPE_ALWAYS`) whenever alpha testing
is disabled - which is true for essentially every draw call in this
codebase. That means `fpe_IsEmpty()` could basically never be true, so
gl4es was *always* substituting its own shader (reproducing the game's
last-used texture/fixed-function state) for the menu's draw call,
regardless of which program, textures, or GL state the menu code set up
around it.

**Fix:** stop routing the menu's blit through gl4es's draw pipeline
entirely. It's now a real (non-gl4es) GLES3 `glBlitFramebuffer()` - a
pure hardware pixel copy with no shader/program/vertex-array stage at
all - copying directly from the menu's own offscreen texture into the
swapchain image. With no draw call and no shader involved, gl4es's `fpe.c`
has nothing to intercept.

Two earlier, more plausible-looking theories were investigated and ruled
out first: forcing the real (non-gl4es-shadowed) GL program to bind
(`real_glUseProgram`), and disabling `GL_TEXTURE_2D` on every texture
unit around the draw (an earlier, incomplete read of the `fpe_IsEmpty()`
condition). Both were red herrings - neither touches the `alphafunc`
field that was actually keeping the substitution permanently active.

Finding this took about a week of calendar time (2026-07-26 to
2026-08-01) spread across several sessions, mostly because each
hypothesis required a full edit-rebuild-deploy-retest cycle on real Quest
hardware to falsify (there's no way to reproduce gl4es's Android-only
codepath on desktop). Most of that time went into working through gl4es
itself - which is not this project's code - since the bug's actual
trigger (a single always-nonzero struct field) sits well below the
project's own layer boundary and doesn't show up in any of the local GL
state that the menu's own code controls.

## 6. License

The original tooling in this repository (the Docker build image, build
scripts, Makefile, asset extraction script, and the Quest app in
`android/` - aside from `org/libsdl/app/`, see below) is licensed under
the [MIT License](LICENSE).

`android/app/src/main/java/org/libsdl/app/` is copied from
[SDL2](https://www.libsdl.org/)'s own android-project template and is
zlib-licensed, same as SDL2 itself.

The Android build also bundles [GL4ES](https://github.com/ptitSeb/gl4es)
(`lib/arm64-v8a/libGL.so` in the APK, compiled at APK build time from the
vendored snapshot in `android/gl4es-src/`, patched via
`android/gl4es-patches/` the same way `engine/` is - see below), which
translates the engine's desktop-style OpenGL calls into GLES/EGL and is
MIT-licensed.

It also bundles the Khronos Group's own [OpenXR-SDK loader](
https://github.com/KhronosGroup/OpenXR-SDK) (`lib/arm64-v8a/libopenxr_loader.so`,
prebuilt unmodified into the build image), which is Apache 2.0-licensed.

The vendored engine snapshot in `engine/` is
[Shockolate](https://github.com/Interrupt/systemshock), which is licensed
under the **GNU GPLv3** (see `engine/LICENSE`) - it is included unchanged
and is *not* relicensed by this project's MIT license. Any build or
distribution of the compiled engine must comply with the GPLv3.

The game assets extracted into `res/assets/ss_ee/` are proprietary,
copyrighted game data owned by their respective rightsholders - they are
never committed to this repository (see `.gitignore`) and must be
supplied by each user from their own legitimate purchase.

`NOTICE.txt` at the repository root summarizes the above and is bundled
into the release tarball by `make package` (see "4.1. Packaging a
distributable build" above) - keep the two in sync.