The game quad's laser beam was purely visual - aiming at the game's own
UI (inventory, menus, dialogs) had no way to interact with it. Add a
real mouse pointer: absolute cursor position plus a single left click,
mirroring a plain point-and-click mouse (no mouselook, no right-click,
no drag).
New xr_mouse.c/h bridges the aim-ray hit-test to the vendored engine's
own public mouse API - mouse_put_xy() for position, a synthetic
SDL_MOUSEBUTTONDOWN/UP (SDL_BUTTON_LEFT) via SDL_PushEvent for clicks,
the same technique nativeSendPrintableChar() already uses for
synthetic keyboard input. No engine patch needed. Deliberately bypasses
SDLActivity.onNativeMouse()/SDL_MOUSEMOTION/SetMouseXY(), whose
physical-window-size scaling is unreliable in this headless immersive
build.
xr_input_try_game_quad() now reports the hit u,v; a new
g_game_touch_active[] tracks a held click on the game quad so it
survives the ray straying onto the keyboard/menu before release,
gating those overlays' claims the same way pending overlay touches
already do.
Expand the on-screen keyboard to a full US layout (letters, digits,
punctuation, Shift layer, Tab, arrows, F1-F12, one-shot Ctrl/Alt
modifiers), make it a persistent, movable, closeable panel via a
title-bar drag handle, and lower its opacity.
Previously the keyboard was just a second panel swapped into the same
quad as the menu launcher, which made the launcher translucent too and
made the keyboard and menu mutually exclusive. Extract the shared
swapchain/JNI/touch-dispatch plumbing into two generic, reusable
modules - xr_swapchain.c (swapchain + per-image FBO setup) and
xr_overlay.c (an off-screen Android View rendered into its own OpenXR
quad, hit-tested via a laser pointer) - and make the menu launcher and
keyboard two independent XrOverlay instances instead of one. The
controller's menu button now only opens/closes the launcher; the
launcher's "Keyboard" button only opens the keyboard - so the keyboard
can stay up while picking something from the menu, and only the
keyboard's own Close button hides it.
xr_menu.c/.h are gone, fully absorbed into xr_overlay.c. On the Java
side, OverlayPanel.java carries the shared off-screen-render/touch/
cursor plumbing that MenuOverlay and the new KeyboardOverlay both
subclass.
Replaces the single shared, side-by-side swapchain (game + menu content
packed into one image, needing viewport/scissor juggling to keep the
menu's own glClear from bleeding into the game's region) with two fully
independent XrSwapchainState instances, each sized to exactly its own
content and acquired/released on its own within the same
xrBeginFrame/xrEndFrame pair - an ordinary multi-layer OpenXR setup.
The shared-swapchain design was originally adopted to work around what
looked like a Horizon OS compositor limitation with independent
swapchains, but that was diagnosed before the real root cause of the
"menu shows the game's content" bug was found (gl4es's fpe.c
unconditionally substituting its own shader onto the menu's draw call -
see README's Debugging notes). Since the actual fix routes the menu's
blit through a real, non-gl4es glBlitFramebuffer() call - orthogonal to
how many swapchains exist - splitting back onto two swapchains works
fine and removes the GL_SCISSOR_TEST workaround and sub-rectangle
offset math entirely. Confirmed on-device: both quads render correctly,
hit-testing and 90 FPS unaffected.
Also adds a visible cursor to the menu quad itself: xr_input.c now
tracks whichever hand's aim ray hits the menu each frame and forwards it
to a new MenuOverlay.nativeUpdateCursor(), which draws a small dot at
that position (composited through the same Bitmap the menu's own
content already goes through) - previously there was no visual feedback
at all showing where you were pointing before pulling the trigger.
- xr_input.c/h: an OpenXR action set (aim pose, trigger/select click, a
menu-toggle button) plus ray/quad hit-testing, and a visible
laser-pointer line drawn from the controller toward the hit point.
- xr_menu.c/h + MenuOverlay.java: a second composition-layer quad,
toggled by a controller button, showing an off-screen, never-attached
Android View tree (one "Keyboard" button so far) driven by synthetic
touch events computed from the laser ray's hit UV.
- Migrate gl4es from a prebuilt binary baked into the Docker build-image
to a vendored source snapshot (android/gl4es-src/) compiled from
source at APK build time via CMake add_subdirectory(), patched through
a new android/gl4es-patches/ (mirrors the existing engine-patches/
pattern). This was needed to track down and fix a bug hit while
building the menu: gl4es's fixed-pipeline emulation (fpe.c) was
unconditionally substituting its own shader onto the menu's draw call
(fpe_ReleventState() always sets alphafunc to a nonzero sentinel, so
its fpe_IsEmpty() check could never see the state as empty), making
the menu quad show the game's own rendering instead of its own
content. Fixed by routing the menu's blit through a real
glBlitFramebuffer() call instead of a shader-based draw, which gl4es's
fpe.c has nothing to intercept - documented in README.md's new
"Debugging notes" section.
- Renumber android/engine-patches/ to close the gap left by removing an
unrelated diagnostic-only patch, and strip investigation-journal
comments and dead diagnostic code (temporary tracing, env-var probes)
left over from finding the bug above.
- Restructure README.md into numbered sections, document every
engine/gl4es patch, add Android Studio dev/testing-cycle instructions,
and generalize Quest-specific wording to any OpenXR headset. Update
NOTICE.txt to match (gl4es is now vendored/patched source, not a
prebuilt binary; add the OpenXR-SDK loader).
- New android/app/src/main/cpp/xr_session.c (+.h): owns the OpenXR
instance/session/local-space/swapchain and the per-frame
xrWaitFrame/xrBeginFrame/xrEndFrame loop, submitting the game's
existing flat render as a single head-tracked XrCompositionLayerQuad.
No stereo rendering or controller input yet - that's steps B/C/D of
the plan.
- 11-android-openxr-cmake.patch: links the OpenXR loader as a build
dependency, staged into jniLibs the same way as gl4es/SDL2.
build-image/Dockerfile and prepare-android-project.sh build and stage
it.
- 12-android-openxr-present.patch: redirects OpenGL.cc's final present
step (opengl_swap_and_restore / SDLDraw's software path) into the XR
swapchain FBO instead of the window, on Android only. Two real gl4es
bugs had to be worked around to get pixels on screen at all:
- gl4es's own glBindFramebuffer errors on an FBO id it didn't create
itself, even though the real driver-level bind succeeds - fixed by
creating the swapchain FBO container via gl4es's own
glGenFramebuffers/glBindFramebuffer, and only using the real
(dlsym'd) driver call for attaching OpenXR's foreign swapchain
texture, which gl4es's own attach can't handle.
- gl4es's glBegin/glVertexAttrib/glVertex3f immediate-mode emulation
only captures a fresh per-vertex value for attribute 0 (position,
driven directly by glVertex3f) - custom attributes like this
shader's texcoords/light are GLES2's *constant*-attribute API and
applied once for the whole draw, not per vertex, silently
collapsing the UI-overlay quad to a single sampled texel. Fixed by
switching that one draw call to real vertex arrays
(glVertexAttribPointer/glDrawArrays).
Also flips the V texcoord to match SDL's top-down row order against
GL's texture convention, and reuses a persistent texture object
instead of a fresh gen/upload/delete every frame.
- AndroidManifest.xml: declares the immersive-HMD intent category and
focus-aware metadata, drops the 2D-panel layout hint.
- README: documents the new OpenXR immersive mode.
ml