matthias/questshock
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add controller input and a laser-pointer-driven menu quad
build / build (push) Successful in 2m26s
Details

Browse Source 
- xr_input.c/h: an OpenXR action set (aim pose, trigger/select click, a
  menu-toggle button) plus ray/quad hit-testing, and a visible
  laser-pointer line drawn from the controller toward the hit point.
- xr_menu.c/h + MenuOverlay.java: a second composition-layer quad,
  toggled by a controller button, showing an off-screen, never-attached
  Android View tree (one "Keyboard" button so far) driven by synthetic
  touch events computed from the laser ray's hit UV.
- Migrate gl4es from a prebuilt binary baked into the Docker build-image
  to a vendored source snapshot (android/gl4es-src/) compiled from
  source at APK build time via CMake add_subdirectory(), patched through
  a new android/gl4es-patches/ (mirrors the existing engine-patches/
  pattern). This was needed to track down and fix a bug hit while
  building the menu: gl4es's fixed-pipeline emulation (fpe.c) was
  unconditionally substituting its own shader onto the menu's draw call
  (fpe_ReleventState() always sets alphafunc to a nonzero sentinel, so
  its fpe_IsEmpty() check could never see the state as empty), making
  the menu quad show the game's own rendering instead of its own
  content. Fixed by routing the menu's blit through a real
  glBlitFramebuffer() call instead of a shader-based draw, which gl4es's
  fpe.c has nothing to intercept - documented in README.md's new
  "Debugging notes" section.
- Renumber android/engine-patches/ to close the gap left by removing an
  unrelated diagnostic-only patch, and strip investigation-journal
  comments and dead diagnostic code (temporary tracing, env-var probes)
  left over from finding the bug above.
- Restructure README.md into numbered sections, document every
  engine/gl4es patch, add Android Studio dev/testing-cycle instructions,
  and generalize Quest-specific wording to any OpenXR headset. Update
  NOTICE.txt to match (gl4es is now vendored/patched source, not a
  prebuilt binary; add the OpenXR-SDK loader).
This commit is contained in:
ml
2026-08-02 06:27:49 +02:00
parent 2300d081c3
commit d2dc58391b
310 changed files with 141458 additions and 226 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+267 -65
View File
@@ -6,7 +6,7 @@ An open source project to play the classic 1994 System Shock on a VR
headset, built on top of [Shockolate](https://github.com/Interrupt/systemshock),
a cross-platform port of the original game.
## Design principles
## 1. Design principles
- **Standalone-headset-only.** The game must run entirely on the headset
itself - no PC required, whether tethered or streamed (not PCVR). Any
@@ -16,10 +16,10 @@ a cross-platform port of the original game.
isn't locked to Meta Quest and can work across other standalone,
Android-based OpenXR headsets (e.g. Pico) too.
## Layout
## 2. Layout
- `engine/` - a vendored snapshot of the Shockolate engine source. Built
via Docker; see below.
via Docker; see "4. Desktop build" below.
- `build-image/` - the Dockerfile (and supporting scripts) for the engine
build environment. Every third-party dependency the engine needs to
compile (SDL2, SDL2_mixer, the fluidsynth-lite MIDI synth, a MIDI
@@ -28,40 +28,42 @@ a cross-platform port of the original game.
- `build-image.sh` / `run-image.sh` / `upload-image.sh` - build the image,
run it to compile the engine, and push it to a registry, respectively.
- `res/assets/` - where you place your own purchased copy of the game (see
below); `res/assets/extract_assets.sh` extracts it into `ss_ee/`.
"3. Game assets" below); `res/assets/extract_assets.sh` extracts it into
`ss_ee/`.
- `res/run.sh` - the launcher script, copied into `dist/` on build.
- `Makefile` - assembles `dist/`, a self-contained runnable copy of the
game, out of the compiled engine and the extracted assets. Also builds
`dist/shockolate-<version>-linux-<arch>.tar.gz`, a redistributable
package that omits the proprietary game assets (`make package`), and
`dist/questshock-<version>-android-arm64.apk` for the Quest (`make apk`).
- `Makefile` - targets:
- `all` (default) - alias for `dist`.
- `build-image` - builds the Docker build-image (`./build-image.sh`).
Only needed once, or after `build-image/` changes.
- `engine` - compiles `engine/` (the vendored Shockolate snapshot) via
the build-image, offline. Always re-run by the targets below, so
`dist`/`package`/`apk` stay in sync with the current `engine/` source.
- `assets` - preflight check only: fails with a pointer to
`res/assets/extract_assets.sh` if the purchased game assets (see "3.
Game assets" below) haven't been extracted yet.
- `dist` - assembles `dist/`, a self-contained runnable copy of the
game, out of the compiled engine and the extracted assets (depends on
`engine` + `assets`). See "4. Desktop build" below.
- `package` - builds a versioned, redistributable
`dist/shockolate-<version>-linux-<arch>.tar.gz` that omits the
proprietary game assets (ships `res/GET_ASSETS.txt` in their place).
See "4.1. Packaging a distributable build" below.
- `apk` - builds `dist/questshock-<version>-android-arm64.apk` for
Android-based OpenXR headsets. Unlike every other target, this needs
network access at build time (Gradle/AGP's own dependency
resolution). See "5. Android / Quest build" below.
- `clean` - removes all build output (`dist/`, `build/`, compiled
engine artifacts, and the staged Android project files).
- `android/` - the Quest app (Java `SDLActivity` glue, Gradle project).
`android/engine-patches/` holds the one small patch needed to build
`engine/` as an Android shared library instead of a desktop executable
- applied to a scratch copy at build time; `engine/` itself is never
modified.
`android/engine-patches/` holds the patches needed to build `engine/`
as an Android shared library instead of a desktop executable - applied
to a scratch copy at build time; `engine/` itself is never modified.
`android/gl4es-src/` similarly vendors GL4ES (see "6. License" below), with
`android/gl4es-patches/` applied to a scratch copy at build time (same
as `engine/`) and compiled from source alongside it, not prebuilt. See
"5. Android / Quest build" below for what each patch does.
## Building
```sh
# 1. Build the engine build-image (once, or after build-image/ changes)
./build-image.sh
# 2. Get your own copy of the game data (see "Game assets" below), then:
res/assets/extract_assets.sh
# 3. Compile the engine and assemble dist/
make dist
# 4. Play
dist/run.sh
```
`make dist` always recompiles the engine from the current `engine/`
source (via `run-image.sh`), so a fresh build-image plus a re-run of
`make dist` is all that's needed after pulling engine changes.
## Game assets
## 3. Game assets
System Shock's game data is not included in this repository and cannot
be redistributed - you need to own a copy. Buy **System Shock: Enhanced
@@ -79,33 +81,69 @@ Wine/Proton on Linux), you don't need the installer or the script at
all - just copy its `res/data/` and `res/sound/` folders directly into
`res/assets/ss_ee/data/` and `res/assets/ss_ee/sound/`. That's exactly
the same layout `extract_assets.sh` produces, so `make dist`/`make
package`/`make apk` pick it up the same way either way.
package`/`make apk` pick it up the same way either way. This one copy of
game assets feeds both the desktop build and the Android/Quest build
below.
## Packaging
## 4. Desktop build
Builds and runs Questshock natively on Linux (your dev machine, or any
Linux box) - useful for local development and testing without a VR
headset at all. For the VR-headset build, see "5. Android / Quest build"
below instead.
```sh
# 1. Build the engine build-image (once, or after build-image/ changes)
./build-image.sh
# 2. Get your own copy of the game data (see "3. Game assets" above), then:
res/assets/extract_assets.sh
# 3. Compile the engine and assemble dist/
make dist
# 4. Play
dist/run.sh
```
`make dist` always recompiles the engine from the current `engine/`
source (via `run-image.sh`), so a fresh build-image plus a re-run of
`make dist` is all that's needed after pulling engine changes.
### 4.1. Packaging a distributable build
`make package` builds `dist/shockolate-<version>-linux-<arch>.tar.gz`: the
compiled binary, its runtime libraries, shaders, a default MIDI
soundfont, license information, and `res/GET_ASSETS.txt` in place of the
actual game data (which the tarball never includes). Version comes from
the current git tag (push a `vX.Y.Z` tag to drive a release); without one
it builds an untagged `0.0.0-dev+<sha>` placeholder. `make apk` (below)
is versioned identically, via the same `build-image/version.sh`.
it builds an untagged `0.0.0-dev+<sha>` placeholder. `make apk` (see
"5. Android / Quest build" below) is versioned identically, via the same
`build-image/version.sh`.
A Gitea Actions workflow (`.gitea/workflows/build.yml`) builds this
package on every push, using the build-image as its container (so no
extra setup is needed in CI beyond the image itself), and publishes the
resulting tarball to dl.ladkau.de.
## Playing on Meta Quest
## 5. Android / Quest build
`make apk` builds `dist/questshock-<version>-android-arm64.apk` - an
immersive OpenXR app (see `android/app/src/main/cpp/xr_session.c`): the
game's own rendering is unchanged (still a flat, 2D render, no stereo 3D
scene), but instead of running as a Home-hosted 2D panel it's now shown as
a single head-tracked quad floating in front of the viewer in an otherwise
empty space. There's no controller-driven interaction yet (that's ongoing
work - a laser-pointer-driven menu and on-screen keyboard); for now, play
with a Bluetooth mouse/keyboard connected to the headset same as before.
immersive OpenXR app (see `android/app/src/main/cpp/xr_session.c`) that
can be sideloaded onto any Android-based VR headset with OpenXR support
(Meta Quest, Pico, etc. - see "1. Design principles" above), not just one
vendor's store. The game's own rendering is unchanged (still a flat, 2D
render, no stereo 3D scene), but instead of running as a Home-hosted 2D
panel it's now shown as a single head-tracked quad floating in front of
the viewer in an otherwise empty space. There's no controller-driven
interaction yet (that's ongoing work - a laser-pointer-driven menu and
on-screen keyboard); for now, play with a Bluetooth mouse/keyboard
connected to the headset same as before. (Only tested on Meta Quest so
far - the steps below use Quest-specific tool names where relevant, but
the same `adb install` flow applies to any Android headset with USB
debugging enabled.)
### 5.1. Installing and playing
1. Install the APK with [SideQuest](https://sidequestvr.com/) (or `adb
install`).
@@ -113,14 +151,14 @@ with a Bluetooth mouse/keyboard connected to the headset same as before.
`/sdcard/questshock/` and extract its own bundled files (shaders, a
default MIDI soundfont) there - `res/data/` and `res/sound/` are
deliberately left missing, since that's the proprietary game data.
3. With the Quest connected to a PC, use SideQuest's file browser (or any
MTP file manager) to copy your own `res/data/` and `res/sound/` (see
`/sdcard/questshock/GET_ASSETS_QUEST.txt`, extracted in step 2, for
exactly what's needed and where it comes from) into
3. With the headset connected to a PC, use SideQuest's file browser (or
any MTP file manager) to copy your own `res/data/` and `res/sound/`
(see `/sdcard/questshock/GET_ASSETS_QUEST.txt`, extracted in step 2,
for exactly what's needed and where it comes from) into
`/sdcard/questshock/res/`.
4. Launch it again.
### Building natively in Android Studio
### 5.2. Building natively in Android Studio
`make apk` always compiles the engine and links the APK inside the
Docker build-image - convenient for CI/CLI builds, but Android Studio
@@ -133,22 +171,180 @@ To have Android Studio compile and deploy `android/` itself instead:
./run-image.sh bash build-image/prepare-android-project.sh --host-paths
```
This stages everything `make apk` normally stages (a scratch, patched
copy of `engine/`; the Android SDL2/SDL2_mixer/fluidsynth-lite/gl4es/openxr
prebuilts; bundled assets) - the same as `build-apk.sh`'s own prep step -
except it writes `android/engine.properties` with paths that resolve on
your host filesystem, and additionally exports the prebuilt libraries
(otherwise only present inside the image, at `/opt/prebuilt/android`) to
This stages everything `make apk` normally stages (scratch, patched copies
of `engine/` and `android/gl4es-src/`; the Android
SDL2/SDL2_mixer/fluidsynth-lite/openxr prebuilts; bundled assets) - the
same as `build-apk.sh`'s own prep step - except it writes
`android/engine.properties` with paths that resolve on your host
filesystem, and additionally exports the prebuilt libraries (otherwise
only present inside the image, at `/opt/prebuilt/android`) to
`build/android-prebuilt/` so they're visible outside the container too.
Re-run it whenever `engine/`, `android/engine-patches/`, or the
prebuilt-library versions in `build-image/Dockerfile` change.
gl4es itself is compiled from its scratch copy as part of Android
Studio's own native build (a CMake subdirectory of `engine/`'s build, not
a separate prebuilt step), so `android/gl4es-src/`/`android/gl4es-patches/`
changes are picked up by a normal rebuild here - no need to re-run
`./build-image.sh` first, unlike `build-image/Dockerfile` changes. Re-run
this script whenever `engine/`, `android/engine-patches/`,
`android/gl4es-src/`, `android/gl4es-patches/`, or the prebuilt-library
versions in `build-image/Dockerfile` change (the Gradle build already
does this automatically for you via its `stageEngine` task, so this
manual re-run is mainly useful for confirming staging succeeded on its
own).
Then open `android/` as a project in Android Studio (JDK 17, NDK
Then, in Android Studio, use File > Open and select the `android/`
directory itself (the one containing `settings.gradle` - not the repo
root, and not `android/app/`) to open it as a project (JDK 17, NDK
`26.1.10909125`, and SDK Platform/Build-Tools 34 installed, matching
`build-image/Dockerfile` and `android/app/build.gradle`) and build/run
normally - no Docker involved for this part.
## License
### 5.3. Testing cycles in Android Studio
Once the project above is open, iterating against a real headset works
the same as any other Android Studio project:
1. Enable Developer Mode on the headset (via its companion phone app -
for Quest, the Meta Horizon app's Devices/Developer Mode toggle) and
turn on USB debugging once prompted. Connect the headset to your
development machine with a USB cable (Wi-Fi debugging via `adb
connect` works too, once paired once over USB).
2. Confirm it's visible with `adb devices` - it should also show up in
Android Studio's device dropdown.
3. Press Run (or Debug, to attach breakpoints in both Java and, via
Android Studio's "Dual"/"Native" debugger, the C code under
`android/app/src/main/cpp/` and the staged `engine/` sources) -
Android Studio builds, installs, and launches the app on the headset
directly. No manual `adb install` or SideQuest step needed for this
loop.
4. `/sdcard/questshock/` (game assets, extracted shaders/soundfont)
persists across reinstalls from Android Studio, so this loop doesn't
require re-copying `res/data/`/`res/sound/` on every iteration - only
a full uninstall or wiping that directory clears it.
5. Use Android Studio's Logcat pane to watch output live; both the Java
side and the engine's own logging (see
`android/engine-patches/08-android-logcat-output.patch`) are tagged
`QuestShock`, so filtering Logcat by that tag shows everything
questshock-specific without gl4es/OpenXR loader/system noise (drop
the filter to see those too).
6. Only re-run `prepare-android-project.sh` manually (see above) after
changing `engine/`, `android/engine-patches/`, `android/gl4es-src/`,
`android/gl4es-patches/`, or the prebuilt-library versions in
`build-image/Dockerfile` - the Gradle build's `stageEngine` task does
this automatically otherwise, so plain Java/C++ edits under
`android/app/src/main/` just need another Run/Debug press.
### 5.4. Engine/gl4es patch reference
`android/engine-patches/` (applied to a scratch copy of `engine/`) and
`android/gl4es-patches/` (applied to a scratch copy of `android/gl4es-src/`)
are both plain, numbered patch files applied in order at build time -
`engine/` and `android/gl4es-src/` themselves are never modified. What
each one does:
`android/engine-patches/`:
- **`01-android-shared-lib.patch`** - Android has no standalone
executables (Java loads a shared library via JNI), so this builds
`CMakeLists.txt`'s `main` target as a `SHARED` library on Android
instead of the desktop `systemshock` executable, folding in the
Android-native glue sources (`ANDROID_EXTRA_SOURCES`).
- **`02-android-opengl-es.patch`** - replaces the desktop
`find_package(OpenGL)` with `add_subdirectory()`-building gl4es from
source (translates the engine's desktop GL calls to GLES/EGL), so it
rebuilds incrementally alongside the engine instead of needing a full
build-image rebuild per gl4es change.
- **`03-android-gles-context.patch`** - requests a GLES 3.2 context
instead of a desktop GL core profile (Quest's Adreno GPU supports it,
and 3.2 has `GL_UNPACK_ROW_LENGTH`/`GL_CLAMP_TO_BORDER` as core,
avoiding workarounds needed on GLES 2.0).
- **`04-android-opengl-es-render.patch`** - swaps a few gl4es rough edges
(its shader-rewrite-based `glPointSize`/alpha-test/point-sprite
emulation) for real GLES-native equivalents: a custom `pointSize`
uniform, a shader-side alpha discard, and native point-sprite
rasterization.
- **`05-android-audio-driver.patch`** - forces SDL2's older OpenSL ES
audio backend instead of AAudio, because AAudio only allows one open
playback device and the engine opens two (cutscene audio plus
`Mix_OpenAudio` for SFX/MIDI).
- **`06-android-resize-event.patch`** - makes the engine react to
`SDL_WINDOWEVENT_RESIZED`, not just `SIZE_CHANGED`, since Android's SDL
backend only ever sends the former for surface-driven resizes.
- **`07-android-logcat-link.patch`** - links Android's `log` library.
- **`08-android-logcat-output.patch`** - routes the engine's own `log.c`
(INFO/DEBUG/WARN/ERROR) through `__android_log_vprint` so it reaches
logcat, instead of stdio (which Android never captures).
- **`09-android-no-window-resize.patch`** - skips the desktop-only
`SDL_SetWindowFullscreen`/`SetWindowSize`/`SetWindowPosition` calls on
Android, since there's no real desktop-style window to resize and
calling them desyncs SDL's cached window size from the real surface.
- **`10-android-openxr-cmake.patch`** - wires up the OpenXR loader plus
EGL as link/include dependencies for the CMake build.
- **`11-android-openxr-present.patch`** - the core OpenXR present-path
hook: redirects `SDLDraw()`/`opengl_swap_and_restore()` to submit into
the XR swapchain (via `xr_frame_begin()`/`xr_frame_end()`) instead of
the normal window, for both the GL-rendered 3D path and the plain
software-composited path (splash screen/cutscenes/menus).
`android/gl4es-patches/`:
- **`01-android-cmake-subdirectory.patch`** - since gl4es is now
`add_subdirectory()`'d straight into the engine's own CMake configure
(via `engine-patches/02` above) rather than built standalone, this
skips gl4es's desktop-style output-directory/`link_directories()` setup
and its `libGL.so.1` SONAME versioning on Android, since AGP's native
packaging needs a plain `libGL.so` and CMake's own default naming
already produces that correctly.
### 5.5. Debugging notes
#### 5.5.1. The menu quad rendering the game instead of itself
While building the OpenXR menu (`android/app/src/main/cpp/xr_menu.c`,
`MenuOverlay.java`), the menu's composition-layer quad consistently showed
the game's own live rendering instead of the menu's content, even though
every diagnostic (FBO bindings, swapchain/layer submission, texture
upload, viewport/scissor state) checked out correct in isolation.
**Root cause:** the menu's blit was a normal `glDrawArrays` call routed
through gl4es (the GLES/EGL translation layer the engine's desktop-style
OpenGL calls go through on Quest). gl4es's fixed-pipeline emulation
(`fpe.c`) decides whether to substitute its own generated shader for
whatever program is bound by checking `fpe_IsEmpty()` - an all-zero-bytes
check over its internal fixed-function state struct. But
`fpe_ReleventState()` unconditionally sets one field of that struct,
`alphafunc`, to a nonzero sentinel (`FPE_ALWAYS`) whenever alpha testing
is disabled - which is true for essentially every draw call in this
codebase. That means `fpe_IsEmpty()` could basically never be true, so
gl4es was *always* substituting its own shader (reproducing the game's
last-used texture/fixed-function state) for the menu's draw call,
regardless of which program, textures, or GL state the menu code set up
around it.
**Fix:** stop routing the menu's blit through gl4es's draw pipeline
entirely. It's now a real (non-gl4es) GLES3 `glBlitFramebuffer()` - a
pure hardware pixel copy with no shader/program/vertex-array stage at
all - copying directly from the menu's own offscreen texture into the
swapchain image. With no draw call and no shader involved, gl4es's `fpe.c`
has nothing to intercept.
Two earlier, more plausible-looking theories were investigated and ruled
out first: forcing the real (non-gl4es-shadowed) GL program to bind
(`real_glUseProgram`), and disabling `GL_TEXTURE_2D` on every texture
unit around the draw (an earlier, incomplete read of the `fpe_IsEmpty()`
condition). Both were red herrings - neither touches the `alphafunc`
field that was actually keeping the substitution permanently active.
Finding this took about a week of calendar time (2026-07-26 to
2026-08-01) spread across several sessions, mostly because each
hypothesis required a full edit-rebuild-deploy-retest cycle on real Quest
hardware to falsify (there's no way to reproduce gl4es's Android-only
codepath on desktop). Most of that time went into working through gl4es
itself - which is not this project's code - since the bug's actual
trigger (a single always-nonzero struct field) sits well below the
project's own layer boundary and doesn't show up in any of the local GL
state that the menu's own code controls.
## 6. License
The original tooling in this repository (the Docker build image, build
scripts, Makefile, asset extraction script, and the Quest app in
@@ -159,10 +355,12 @@ the [MIT License](LICENSE).
[SDL2](https://www.libsdl.org/)'s own android-project template and is
zlib-licensed, same as SDL2 itself.
The Quest build also bundles [GL4ES](https://github.com/ptitSeb/gl4es)
(`lib/arm64-v8a/libGL.so` in the APK, prebuilt unmodified into the build
image), which translates the engine's desktop-style OpenGL calls into
GLES/EGL and is MIT-licensed.
The Android build also bundles [GL4ES](https://github.com/ptitSeb/gl4es)
(`lib/arm64-v8a/libGL.so` in the APK, compiled at APK build time from the
vendored snapshot in `android/gl4es-src/`, patched via
`android/gl4es-patches/` the same way `engine/` is - see below), which
translates the engine's desktop-style OpenGL calls into GLES/EGL and is
MIT-licensed.
It also bundles the Khronos Group's own [OpenXR-SDK loader](
https://github.com/KhronosGroup/OpenXR-SDK) (`lib/arm64-v8a/libopenxr_loader.so`,
@@ -178,3 +376,7 @@ The game assets extracted into `res/assets/ss_ee/` are proprietary,
copyrighted game data owned by their respective rightsholders - they are
never committed to this repository (see `.gitignore`) and must be
supplied by each user from their own legitimate purchase.
`NOTICE.txt` at the repository root summarizes the above and is bundled
into the release tarball by `make package` (see "4.1. Packaging a
distributable build" above) - keep the two in sync.