Split the keyboard onto its own translucent OpenXR quad

Expand the on-screen keyboard to a full US layout (letters, digits,
punctuation, Shift layer, Tab, arrows, F1-F12, one-shot Ctrl/Alt
modifiers), make it a persistent, movable, closeable panel via a
title-bar drag handle, and lower its opacity.

Previously the keyboard was just a second panel swapped into the same
quad as the menu launcher, which made the launcher translucent too and
made the keyboard and menu mutually exclusive. Extract the shared
swapchain/JNI/touch-dispatch plumbing into two generic, reusable
modules - xr_swapchain.c (swapchain + per-image FBO setup) and
xr_overlay.c (an off-screen Android View rendered into its own OpenXR
quad, hit-tested via a laser pointer) - and make the menu launcher and
keyboard two independent XrOverlay instances instead of one. The
controller's menu button now only opens/closes the launcher; the
launcher's "Keyboard" button only opens the keyboard - so the keyboard
can stay up while picking something from the menu, and only the
keyboard's own Close button hides it.

xr_menu.c/.h are gone, fully absorbed into xr_overlay.c. On the Java
side, OverlayPanel.java carries the shared off-screen-render/touch/
cursor plumbing that MenuOverlay and the new KeyboardOverlay both
subclass.

README.md

@@ -140,12 +140,57 @@ can be sideloaded onto any Android-based VR headset with OpenXR support
 vendor's store. The game's own rendering is unchanged (still a flat, 2D
 render, no stereo 3D scene), but instead of running as a Home-hosted 2D
 panel it's now shown as a single head-tracked quad floating in front of
-the viewer in an otherwise empty space. There's no controller-driven
+the viewer, with a separate menu quad toggled by a controller button and
-interaction yet (that's ongoing work - a laser-pointer-driven menu and
+driven by a laser-pointer-style aim ray from each hand
-on-screen keyboard); for now, play with a Bluetooth mouse/keyboard
+(`android/app/src/main/cpp/xr_input.c`) - point and pull the trigger to
-connected to the headset same as before. (Only tested on Meta Quest so
+interact with it, same as the game's own Bluetooth mouse/keyboard input
-far - the steps below use Quest-specific tool names where relevant, but
+otherwise works unchanged.
-the same `adb install` flow applies to any Android headset with USB
+
+The menu launcher (`MenuOverlay.java` - just a "Keyboard" button so far)
+and the keyboard (`KeyboardOverlay.java`) are two fully independent quads,
+each its own `XrOverlay` instance (`android/app/src/main/cpp/xr_overlay.c`
+- a generic "off-screen Android View rendered into an OpenXR quad,
+hit-tested via a laser pointer" module shared by both, and by whatever
+similar panel comes next) with its own swapchain, visibility, and
+position, so they can be shown/hidden/moved independently - a design
+specifically meant to let the keyboard stay open while also picking
+something from the (still-to-be-built-out) menu. The controller's menu
+button only ever opens/closes the launcher; clicking its "Keyboard" button
+only opens the keyboard (never touching the launcher's own visibility) -
+the keyboard's own title bar holds the only way to close it again.
+
+The keyboard is a hand-built, full US-layout key grid (there's no system
+IME to borrow once immersive) that forwards each key press straight to the
+game as real input - non-printable keys (Esc/Enter/Backspace/Tab/arrows/
+F1-F12) as a `SDLActivity.onNativeKeyDown()`/`onNativeKeyUp()` pair, the
+same one a physical Bluetooth keyboard's presses already go through, and
+printable keys (letters, digits, punctuation, Space) via a synthesized
+`SDL_TEXTINPUT` event pushed directly from native code
+(`questshock_native.c`). A Shift key toggles the whole grid between
+lowercase/numbers and uppercase/symbols (doubling as Caps Lock - it stays
+toggled until pressed again), and Ctrl/Alt arm as one-shot modifiers,
+consumed by whichever key is pressed next - meant as a general stand-in
+for keyboard-driven functionality that isn't (yet, or ever) mapped onto
+the controllers, not just a future config screen's text entry.
+
+Once opened, the keyboard stays up as a standing input panel while
+actually playing rather than a modal you open and close. Its title bar
+doubles as a drag handle (grab-and-drag with the trigger, handled entirely
+on the native side in `xr_input.c` before it ever reaches
+`KeyboardOverlay`'s own touch dispatch) for repositioning it, and holds
+the Close button that hides it. It's also rendered semi-transparent
+(`XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT` in `xr_session.c`,
+set only on the keyboard's layer - the menu launcher stays fully opaque)
+so whatever's behind it - the game quad, mid-play - stays visible while
+it's up.
+
+The laser
+pointer/cursor is currently only visible while
+actually aiming at the game or menu quad respectively - there's no visual
+feedback yet while aiming at empty space between them. (Only tested on
+Meta Quest so far -
+the steps below use Quest-specific tool names where relevant, but the
+same `adb install` flow applies to any Android headset with USB
 debugging enabled.)
 
 ### 5.1. Installing and playing
@@ -306,8 +351,9 @@ each one does:
 
 #### 5.5.1. The menu quad rendering the game instead of itself
 
-While building the OpenXR menu (`android/app/src/main/cpp/xr_menu.c`,
+While building the OpenXR menu (`android/app/src/main/cpp/xr_overlay.c` -
-`MenuOverlay.java`), the menu's composition-layer quad consistently showed
+`xr_menu.c` at the time - and `MenuOverlay.java`), the menu's
+composition-layer quad consistently showed
 the game's own live rendering instead of the menu's content, even though
 every diagnostic (FBO bindings, swapchain/layer submission, texture
 upload, viewport/scissor state) checked out correct in isolation.

android/app/build.gradle

@@ -171,7 +171,7 @@ android {
                     "-DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=BOTH", \
                     "-DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=BOTH", \
                     "-DCMAKE_SHARED_LINKER_FLAGS=-Wl,-z,max-page-size=16384", \
-                    "-DANDROID_EXTRA_SOURCES=${projectDir}/src/main/cpp/questshock_native.c;${projectDir}/src/main/cpp/xr_session.c;${projectDir}/src/main/cpp/xr_input.c;${projectDir}/src/main/cpp/xr_menu.c"
+                    "-DANDROID_EXTRA_SOURCES=${projectDir}/src/main/cpp/questshock_native.c;${projectDir}/src/main/cpp/xr_session.c;${projectDir}/src/main/cpp/xr_input.c;${projectDir}/src/main/cpp/xr_overlay.c;${projectDir}/src/main/cpp/xr_swapchain.c"
                 abiFilters 'arm64-v8a'
             }
         }

android/app/src/main/cpp/questshock_native.c

@@ -7,9 +7,53 @@
 #include <jni.h>
 #include <unistd.h>
 
+#include <SDL.h>
+
+#include "xr_overlay.h"
+#include "xr_session.h"
+
 JNIEXPORT void JNICALL
 Java_de_ladkau_questshock_QuestShockActivity_nativeChdir(JNIEnv *env, jclass clazz, jstring path) {
     const char *cpath = (*env)->GetStringUTFChars(env, path, NULL);
     chdir(cpath);
     (*env)->ReleaseStringUTFChars(env, path, cpath);
 }
+
+// KeyboardOverlay's hand-built on-screen keyboard (see
+// KeyboardOverlay.commitPrintableChar()) has no real Android IME session
+// behind it, so printable characters can't reach the game via Android's
+// usual IME path - instead this builds and pushes an SDL_TEXTINPUT event
+// directly, the same event type/shape SDL's own Android backend pushes for
+// typed text, using only the public SDL_Event/SDL_PushEvent API. The
+// engine's pump_events() (sdl_events.c) only picks up printable characters
+// from this event, not from SDL_KEYDOWN (see handleKeyPress() in
+// KeyboardOverlay.java for why).
+JNIEXPORT void JNICALL
+Java_de_ladkau_questshock_KeyboardOverlay_nativeSendPrintableChar(JNIEnv *env, jclass clazz,
+                                                                   jchar c) {
+    SDL_Event event;
+    SDL_zero(event);
+    event.type = SDL_TEXTINPUT;
+    event.text.timestamp = SDL_GetTicks();
+    event.text.windowID = 0;
+    event.text.text[0] = (char)c;
+    event.text.text[1] = '\0';
+    SDL_PushEvent(&event);
+}
+
+// KeyboardOverlay's title-bar Close button - the keyboard is a fully
+// independent overlay quad from the menu launcher (see xr_overlay.h), so
+// this is the only way to hide it once it's open; the controller's menu
+// button only ever affects the menu launcher.
+JNIEXPORT void JNICALL
+Java_de_ladkau_questshock_KeyboardOverlay_nativeRequestClose(JNIEnv *env, jclass clazz) {
+    xr_overlay_set_visible(xr_session_get_keyboard_overlay(), false);
+}
+
+// MenuOverlay's "Keyboard" button - only opens the keyboard overlay, never
+// touches the menu launcher's own visibility, so both can be shown
+// together.
+JNIEXPORT void JNICALL
+Java_de_ladkau_questshock_MenuOverlay_nativeShowKeyboard(JNIEnv *env, jclass clazz) {
+    xr_overlay_set_visible(xr_session_get_keyboard_overlay(), true);
+}

android/app/src/main/cpp/xr_input.c

@@ -7,7 +7,7 @@
 
 #include <GLES3/gl3.h>
 
-#include "xr_menu.h"
+#include "xr_overlay.h"
 #include "xr_session.h"
 
 #define TAG "QuestShock"
@@ -17,6 +17,24 @@
 #define LEFT 0
 #define RIGHT 1
 
+static const char *kHandName[2] = {"left", "right"};
+static const float kHandColor[2][4] = {
+    {0.2f, 1.0f, 1.0f, 1.0f}, // left: cyan
+    {1.0f, 0.85f, 0.1f, 1.0f}, // right: amber
+};
+
+// The keyboard overlay's title bar (drag handle + Close button) is handled
+// entirely here, before a hit would normally be dispatched to
+// KeyboardOverlay as a touch - these fractions must agree with
+// KeyboardOverlay.java's TITLE_BAR_HEIGHT/CLOSE_BUTTON_WIDTH constants (in
+// pixels, out of KeyboardOverlay.WIDTH/HEIGHT there). A hit in the top
+// TITLE_BAR_V_FRACTION of the quad, at or left of CLOSE_BUTTON_U_FRACTION,
+// is the drag handle; right of that (still within the title bar) is the
+// Close button, which is dispatched as an ordinary click instead. The menu
+// launcher overlay has no title bar - it's a plain click-only panel.
+#define TITLE_BAR_V_FRACTION (90.0f / 768.0f)
+#define CLOSE_BUTTON_U_FRACTION (1.0f - 160.0f / 1024.0f)
+
 static XrInstance g_instance = XR_NULL_HANDLE;
 static XrSession g_session = XR_NULL_HANDLE;
 static XrActionSet g_action_set = XR_NULL_HANDLE;
@@ -27,14 +45,29 @@ static XrPath g_hand_path[2];
 static XrSpace g_aim_space[2] = {XR_NULL_HANDLE, XR_NULL_HANDLE};
 
 // Edge-detection state, so touch dispatch and logging only fire on actual
-// state changes, not every frame.
+// state changes, not every frame. Each interaction target (game quad, menu
+// launcher, keyboard) tracks its own "was this hand's ray hitting it last
+// frame" independently, since the menu and keyboard overlays can now both
+// be visible at once.
 static bool g_prev_select[2] = {false, false};
 static bool g_prev_menu = false;
-static bool g_prev_hit[2] = {false, false};
+static bool g_game_prev_hit[2] = {false, false};
-// Tracks which hand currently has an in-flight synthetic touch down on the
+static bool g_menu_prev_hit[2] = {false, false};
-// menu (so a later trigger-up edge only dispatches a matching touch-up if
-// a touch-down was actually sent for that hand).
 static bool g_menu_touch_active[2] = {false, false};
+static bool g_keyboard_prev_hit[2] = {false, false};
+static bool g_keyboard_touch_active[2] = {false, false};
+
+// Drag state for repositioning the keyboard overlay via its title bar (see
+// TITLE_BAR_V_FRACTION/CLOSE_BUTTON_U_FRACTION above) - the menu launcher
+// has no title bar, so it's never draggable. g_keyboard_drag_hand is -1
+// when g_keyboard_dragging is false. The offset is the vector from the
+// quad's center to the grabbed point at drag-start, kept constant for the
+// whole gesture so the quad follows the hand rigidly rather than
+// re-centering under the ray each frame.
+static bool g_keyboard_dragging = false;
+static int g_keyboard_drag_hand = -1;
+static float g_keyboard_drag_offset_x = 0.0f;
+static float g_keyboard_drag_offset_y = 0.0f;
 
 static GLuint g_reticle_program = 0;
 static GLint g_reticle_color_loc = -1;
@@ -220,6 +253,106 @@ bool xr_input_init(XrInstance instance, XrSession session) {
     return true;
 }
 
+// Hit-tests/dispatches this hand's ray against one overlay quad - shared
+// by the menu launcher and keyboard overlays in xr_input_sync_and_draw()
+// below. dragCapable enables the keyboard-only title-bar/drag-handle
+// handling (see TITLE_BAR_V_FRACTION/CLOSE_BUTTON_U_FRACTION); the menu
+// launcher has no title bar, so it's always false there and every hit is a
+// plain click. Returns true if this overlay is visible - claiming the
+// hand's ray processing for this frame, regardless of whether the ray
+// actually hits it - so the caller should stop trying other targets (menu/
+// keyboard/game quad are mutually exclusive per hand, per frame).
+static bool xr_input_try_overlay(XrOverlay *overlay, bool dragCapable, int hand,
+                                  const XrSpaceLocation *location, float fx, float fy, float fz,
+                                  bool selectDownEdge, bool selectUpEdge, bool *touchActive,
+                                  bool *prevHit, const char *quadName, float *outCursorU,
+                                  float *outCursorV, bool *outCursorHit) {
+    if (!xr_overlay_is_visible(overlay))
+        return false;
+
+    float quadCenterX, quadCenterY, distance, halfWidth, halfHeight;
+    xr_overlay_get_quad_extent(overlay, &quadCenterX, &quadCenterY, &distance, &halfWidth,
+                                &halfHeight);
+
+    // The quad's plane is z = -distance (facing the viewer along -Z) - a
+    // ray parallel to it (fz ~ 0) never crosses. worldX/Y is the raw
+    // world-space intersection point, needed as-is for dragging (which can
+    // legitimately move the quad to where the ray isn't currently hitting
+    // it); cx/cy are that same point in the quad's own [-1,1] local space;
+    // hit/u/v (the equivalent 0..1, top-left-origin coordinates the Java
+    // view's pixel grid and the on-quad hit logs use) are only meaningful
+    // within the quad's current bounds, unlike planeHit.
+    bool planeHit = false, hit = false;
+    float u = 0.0f, v = 0.0f, worldX = 0.0f, worldY = 0.0f;
+    if (fabsf(fz) > 1e-5f) {
+        float t = (-distance - location->pose.position.z) / fz;
+        if (t > 0.0f) {
+            planeHit = true;
+            worldX = location->pose.position.x + t * fx;
+            worldY = location->pose.position.y + t * fy;
+            float cx = (worldX - quadCenterX) / halfWidth;
+            float cy = (worldY - quadCenterY) / halfHeight;
+            hit = fabsf(cx) <= 1.0f && fabsf(cy) <= 1.0f;
+            u = (cx + 1.0f) * 0.5f;
+            v = (1.0f - cy) * 0.5f;
+        }
+    }
+
+    if (hit != *prevHit) {
+        LOGI("XR: %s aim ray %s %s quad (u=%.2f v=%.2f)", kHandName[hand],
+             hit ? "entered" : "left", quadName, u, v);
+        *prevHit = hit;
+    }
+    if (hit) {
+        *outCursorHit = true;
+        *outCursorU = u;
+        *outCursorV = v;
+    }
+
+    if (!dragCapable) {
+        if (selectDownEdge && hit) {
+            xr_overlay_touch(overlay, u, v, true);
+            *touchActive = true;
+        } else if (selectUpEdge && *touchActive) {
+            xr_overlay_touch(overlay, u, v, false);
+            *touchActive = false;
+        }
+        return true;
+    }
+
+    bool onTitleBar = hit && v < TITLE_BAR_V_FRACTION;
+    bool onDragHandle = onTitleBar && u <= CLOSE_BUTTON_U_FRACTION;
+
+    if (selectDownEdge) {
+        if (onDragHandle) {
+            g_keyboard_dragging = true;
+            g_keyboard_drag_hand = hand;
+            g_keyboard_drag_offset_x = worldX - quadCenterX;
+            g_keyboard_drag_offset_y = worldY - quadCenterY;
+        } else if (hit) {
+            xr_overlay_touch(overlay, u, v, true);
+            *touchActive = true;
+        }
+    } else if (selectUpEdge) {
+        if (g_keyboard_dragging && g_keyboard_drag_hand == hand) {
+            g_keyboard_dragging = false;
+            g_keyboard_drag_hand = -1;
+        } else if (*touchActive) {
+            xr_overlay_touch(overlay, u, v, false);
+            *touchActive = false;
+        }
+    }
+
+    // Re-intersects against the same fixed-distance plane every frame the
+    // drag continues, so the quad tracks the hand in real time rather than
+    // only jumping on the down/up edges above.
+    if (g_keyboard_dragging && g_keyboard_drag_hand == hand && planeHit) {
+        xr_overlay_set_position(overlay, worldX - g_keyboard_drag_offset_x,
+                                 worldY - g_keyboard_drag_offset_y);
+    }
+    return true;
+}
+
 void xr_input_sync_and_draw(XrSpace baseSpace, XrTime time, bool draw) {
     if (g_action_set == XR_NULL_HANDLE)
         return;
@@ -230,30 +363,38 @@ void xr_input_sync_and_draw(XrSpace baseSpace, XrTime time, bool draw) {
     syncInfo.activeActionSets = &activeSet;
     xr_check(xrSyncActions(g_session, &syncInfo), "xrSyncActions");
 
-    bool menuVisible = xr_menu_is_visible();
+    XrOverlay *menuOverlay = xr_session_get_menu_overlay();
+    XrOverlay *keyboardOverlay = xr_session_get_keyboard_overlay();
+    bool menuVisible = xr_overlay_is_visible(menuOverlay);
+    bool keyboardVisible = xr_overlay_is_visible(keyboardOverlay);
 
-    float quadCenterX = 0.0f, quadCenterY = 0.0f, distance, halfWidth, halfHeight;
+    // A drag can't get stuck active across a hide/reopen - e.g. Close
-    if (menuVisible)
+    // being hit on one hand mid-drag on the other, or the session ending.
-        xr_menu_get_quad_extent(&quadCenterX, &quadCenterY, &distance, &halfWidth, &halfHeight);
+    if (!keyboardVisible) {
-    else
+        g_keyboard_dragging = false;
-        xr_get_game_quad_extent(&distance, &halfWidth, &halfHeight);
+        g_keyboard_drag_hand = -1;
+    }
 
-    static const char *kHandName[2] = {"left", "right"};
+    // The GL reticle is drawn directly into whatever framebuffer is
-    static const float kHandColor[2][4] = {
+    // currently bound - the game quad's swapchain image (see xr_frame_end(),
-        {0.2f, 1.0f, 1.0f, 1.0f}, // left: cyan
+    // which calls this while that image is still bound). That only makes
-        {1.0f, 0.85f, 0.1f, 1.0f}, // right: amber
+    // sense while aiming at the game quad; the menu/keyboard overlays' own
-    };
+    // cursors are drawn by their Java views instead (see
-
+    // xr_overlay_update_cursor() below), composited into their Bitmaps the
-    // The reticle is drawn directly into whatever framebuffer is currently
+    // same way their other content is.
-    // bound - the game quad's swapchain image (see xr_frame_end(), which
+    bool drawReticle = draw && !menuVisible && !keyboardVisible;
-    // calls this while that image is still bound). That only makes sense
-    // while aiming at the game quad; the menu quad's content comes from
-    // MenuOverlay's rendered Bitmap instead (xr_menu.c), uploaded on a
-    // separate swapchain, so no reticle is drawn for it here.
-    bool drawReticle = draw && !menuVisible;
     if (drawReticle)
         glUseProgram(g_reticle_program);
 
+    // Fed to xr_overlay_update_cursor() after the loop below - whichever
+    // hand's ray hits a given overlay last wins if both do, good enough
+    // for a single on-quad cursor per overlay (only ever updated when hit
+    // is true, so a later hand that misses doesn't hide an earlier hand's
+    // hit).
+    bool menuCursorHit = false, keyboardCursorHit = false;
+    float menuCursorU = 0.0f, menuCursorV = 0.0f;
+    float keyboardCursorU = 0.0f, keyboardCursorV = 0.0f;
+
     for (int hand = 0; hand < 2; hand++) {
         XrActionStateGetInfo selectInfo = {XR_TYPE_ACTION_STATE_GET_INFO};
         selectInfo.action = g_select_click_action;
@@ -277,13 +418,14 @@ void xr_input_sync_and_draw(XrSpace baseSpace, XrTime time, bool draw) {
             bool menuDown = menuState.isActive && menuState.currentState;
             if (menuDown && !g_prev_menu) {
                 LOGI("XR: menu-toggle button DOWN");
-                xr_menu_toggle_visible();
+                // Only ever affects the menu launcher - the keyboard is
-                // Avoid a stale hit/touch-active carrying over from
+                // fully independent (its own Close button is the only way
-                // whichever quad was being tested against before the
+                // to hide it, see KeyboardOverlay.java).
-                // toggle - the next frame re-evaluates against the new
+                if (menuOverlay != NULL) {
-                // target from a clean state.
+                    xr_overlay_toggle_visible(menuOverlay);
-                g_prev_hit[LEFT] = g_prev_hit[RIGHT] = false;
+                    g_menu_prev_hit[LEFT] = g_menu_prev_hit[RIGHT] = false;
-                g_menu_touch_active[LEFT] = g_menu_touch_active[RIGHT] = false;
+                    g_menu_touch_active[LEFT] = g_menu_touch_active[RIGHT] = false;
+                }
             } else if (!menuDown && g_prev_menu) {
                 LOGI("XR: menu-toggle button UP");
             }
@@ -309,46 +451,40 @@ void xr_input_sync_and_draw(XrSpace baseSpace, XrTime time, bool draw) {
         float fx, fy, fz;
         quat_rotate_vec(&location.pose.orientation, 0.0f, 0.0f, -1.0f, &fx, &fy, &fz);
 
-        // The quad's plane is z = -distance (facing the viewer along -Z,
+        // Keyboard first (it's the more likely target while it's up), then
-        // see QUAD_DISTANCE_METERS/xr_menu.c's MENU_DISTANCE_METERS) - a
+        // the menu launcher, then - only if neither is visible - the game
-        // ray parallel to it (fz ~ 0) never crosses. cx/cy are the hit
+        // quad's own reticle below. A single ray only ever interacts with
-        // point in the quad's own [-1,1] local space (the same space
+        // one target per hand per frame.
-        // android_draw_surface_as_quad() draws its own vertex positions
+        if (xr_input_try_overlay(keyboardOverlay, true, hand, &location, fx, fy, fz,
-        // in); u/v are the equivalent 0..1, top-left-origin coordinates
+                                  selectDownEdge, selectUpEdge, &g_keyboard_touch_active[hand],
-        // MenuOverlay's pixel grid and the on-quad hit logs both use.
+                                  &g_keyboard_prev_hit[hand], "keyboard", &keyboardCursorU,
+                                  &keyboardCursorV, &keyboardCursorHit))
+            continue;
+        if (xr_input_try_overlay(menuOverlay, false, hand, &location, fx, fy, fz, selectDownEdge,
+                                  selectUpEdge, &g_menu_touch_active[hand], &g_menu_prev_hit[hand],
+                                  "menu", &menuCursorU, &menuCursorV, &menuCursorHit))
+            continue;
+
+        float distance, halfWidth, halfHeight;
+        xr_get_game_quad_extent(&distance, &halfWidth, &halfHeight);
+
         bool hit = false;
         float u = 0.0f, v = 0.0f, cx = 0.0f, cy = 0.0f;
         if (fabsf(fz) > 1e-5f) {
             float t = (-distance - location.pose.position.z) / fz;
             if (t > 0.0f) {
-                cx = (location.pose.position.x + t * fx - quadCenterX) / halfWidth;
+                cx = (location.pose.position.x + t * fx) / halfWidth;
-                cy = (location.pose.position.y + t * fy - quadCenterY) / halfHeight;
+                cy = (location.pose.position.y + t * fy) / halfHeight;
                 hit = fabsf(cx) <= 1.0f && fabsf(cy) <= 1.0f;
                 u = (cx + 1.0f) * 0.5f;
                 v = (1.0f - cy) * 0.5f;
             }
         }
 
-        if (menuVisible) {
+        if (hit != g_game_prev_hit[hand]) {
-            if (hit != g_prev_hit[hand]) {
-                LOGI("XR: %s aim ray %s menu quad (u=%.2f v=%.2f)", kHandName[hand],
-                     hit ? "entered" : "left", u, v);
-                g_prev_hit[hand] = hit;
-            }
-            if (selectDownEdge && hit) {
-                xr_menu_touch(u, v, true);
-                g_menu_touch_active[hand] = true;
-            } else if (selectUpEdge && g_menu_touch_active[hand]) {
-                xr_menu_touch(u, v, false);
-                g_menu_touch_active[hand] = false;
-            }
-            continue;
-        }
-
-        if (hit != g_prev_hit[hand]) {
             LOGI("XR: %s aim ray %s game quad (u=%.2f v=%.2f)", kHandName[hand],
                  hit ? "entered" : "left", u, v);
-            g_prev_hit[hand] = hit;
+            g_game_prev_hit[hand] = hit;
         }
 
         if (!hit || !drawReticle)
@@ -364,6 +500,12 @@ void xr_input_sync_and_draw(XrSpace baseSpace, XrTime time, bool draw) {
         glDrawArrays(GL_LINES, 0, 4);
         glDisableVertexAttribArray(RETICLE_POSITION_LOC);
     }
+
+    if (menuVisible)
+        xr_overlay_update_cursor(menuOverlay, menuCursorU, menuCursorV, menuCursorHit);
+    if (keyboardVisible)
+        xr_overlay_update_cursor(keyboardOverlay, keyboardCursorU, keyboardCursorV,
+                                  keyboardCursorHit);
 }
 
 void xr_input_shutdown(void) {
@@ -386,7 +528,12 @@ void xr_input_shutdown(void) {
     g_instance = XR_NULL_HANDLE;
     g_session = XR_NULL_HANDLE;
     memset(g_prev_select, 0, sizeof(g_prev_select));
-    memset(g_prev_hit, 0, sizeof(g_prev_hit));
+    memset(g_game_prev_hit, 0, sizeof(g_game_prev_hit));
+    memset(g_menu_prev_hit, 0, sizeof(g_menu_prev_hit));
     memset(g_menu_touch_active, 0, sizeof(g_menu_touch_active));
+    memset(g_keyboard_prev_hit, 0, sizeof(g_keyboard_prev_hit));
+    memset(g_keyboard_touch_active, 0, sizeof(g_keyboard_touch_active));
     g_prev_menu = false;
+    g_keyboard_dragging = false;
+    g_keyboard_drag_hand = -1;
 }

android/app/src/main/cpp/xr_input.h

@@ -1,11 +1,13 @@
 // Controller input for questshock's immersive Quest build: one OpenXR
 // action set (aim pose + trigger click per hand, a menu-toggle button on
-// the left controller) plus ray/quad hit-testing. While the menu quad
+// the left controller) plus ray/quad hit-testing. While neither the menu
-// (xr_menu.c) is hidden, this tests against the game quad xr_session.c
+// launcher nor keyboard overlay (see xr_overlay.h, xr_session.c) is
-// submits and draws a small reticle where each hand's aim ray crosses it;
+// visible, this tests against the game quad xr_session.c submits and draws
-// while the menu is visible, it tests against the menu quad instead and
+// a small reticle where each hand's aim ray crosses it; while either
-// forwards trigger edges as synthetic touches (no reticle - the menu's
+// overlay is visible, it tests against that overlay's quad instead (the
-// own content comes from MenuOverlay's rendered Bitmap).
+// keyboard is tried first) and forwards trigger edges as synthetic touches
+// (no reticle - each overlay's own content comes from its Java view's
+// rendered Bitmap).
 #ifndef QUESTSHOCK_XR_INPUT_H
 #define QUESTSHOCK_XR_INPUT_H
 

android/app/src/main/cpp/xr_menu.c

@@ -1,356 +0,0 @@
-#include "xr_menu.h"
-
-#include <dlfcn.h>
-#include <stdlib.h>
-
-#include <android/log.h>
-#include <jni.h>
-
-#include <EGL/egl.h>
-#include <GLES3/gl3.h>
-
-#include <SDL.h>
-
-#define TAG "QuestShock"
-#define LOGE(...) __android_log_print(ANDROID_LOG_ERROR, TAG, __VA_ARGS__)
-#define LOGI(...) __android_log_print(ANDROID_LOG_INFO, TAG, __VA_ARGS__)
-
-#define MENU_WIDTH 1024
-#define MENU_HEIGHT 768
-
-// Below the game quad (QUAD_DISTANCE_METERS/QUAD_WIDTH_METERS in
-// xr_session.c, 2m out) so both are visible together without overlapping.
-#define MENU_DISTANCE_METERS 1.5f
-#define MENU_WIDTH_METERS 0.8f
-#define MENU_CENTER_X_METERS 0.0f
-#define MENU_CENTER_Y_METERS -0.45f
-
-static bool g_visible = false;
-
-static jclass g_menu_overlay_class = NULL;
-static jmethodID g_native_init_method = NULL;
-static jmethodID g_take_pixels_method = NULL;
-static jmethodID g_dispatch_touch_method = NULL;
-
-// The menu overlay's content lives in this texture, uploaded by
-// xr_menu_upload_source_texture_if_dirty() below. It's created through
-// gl4es (glGenTextures/glBindTexture/glTexImage2D), like any other texture
-// the engine itself creates, so ordinary glTexSubImage2D uploads against it
-// work the normal way.
-static GLuint g_source_texture = 0;
-// A framebuffer with g_source_texture as its only color attachment, used as
-// the read source for xr_menu_render_if_visible()'s glBlitFramebuffer()
-// call. Created via the real (non-gl4es) GLES entry points below: that blit
-// bypasses gl4es entirely (see xr_menu_render_if_visible() for why), so its
-// source framebuffer has to be a real GL object rather than one gl4es
-// tracks.
-static GLuint g_source_fbo = 0;
-
-typedef void (*PFNQSGETINTEGERV)(GLenum, GLint *);
-typedef void (*PFNQSGENFRAMEBUFFERS)(GLsizei, GLuint *);
-typedef void (*PFNQSDELETEFRAMEBUFFERS)(GLsizei, const GLuint *);
-typedef void (*PFNQSBINDFRAMEBUFFER)(GLenum, GLuint);
-typedef void (*PFNQSFRAMEBUFFERTEXTURE2D)(GLenum, GLenum, GLenum, GLuint, GLint);
-typedef void (*PFNQSBLITFRAMEBUFFER)(GLint, GLint, GLint, GLint, GLint, GLint, GLint, GLint,
-                                      GLbitfield, GLenum);
-
-// The menu's source framebuffer and its glBlitFramebuffer() copy into the
-// swapchain image go through the real GLES driver rather than gl4es: gl4es's
-// fixed-pipeline emulation unconditionally substitutes its own shader onto
-// any gl4es-routed draw call, which would silently replace the menu's
-// content with whatever the game itself last rendered (see
-// xr_menu_render_if_visible()). A framebuffer blit has no shader stage at
-// all, so going through the real driver for it sidesteps the problem
-// entirely.
-static PFNQSGETINTEGERV real_glGetIntegerv;
-static PFNQSGENFRAMEBUFFERS real_glGenFramebuffers;
-static PFNQSDELETEFRAMEBUFFERS real_glDeleteFramebuffers;
-static PFNQSBINDFRAMEBUFFER real_glBindFramebuffer;
-static PFNQSFRAMEBUFFERTEXTURE2D real_glFramebufferTexture2D;
-// glBlitFramebuffer is a GLES 3.0 addition; resolved via eglGetProcAddress
-// rather than dlsym, since Android's driver dispatch doesn't guarantee ES3+
-// symbols are dlsym-able by name from libGLESv2.so, unlike the GLES2-core
-// functions above.
-static PFNQSBLITFRAMEBUFFER real_glBlitFramebuffer;
-
-static bool xr_menu_load_real_gles(void) {
-    void *lib = dlopen("libGLESv2.so", RTLD_NOW | RTLD_LOCAL);
-    if (lib == NULL) {
-        LOGE("XR: menu dlopen(libGLESv2.so) failed: %s", dlerror());
-        return false;
-    }
-    real_glGetIntegerv = (PFNQSGETINTEGERV)dlsym(lib, "glGetIntegerv");
-    real_glGenFramebuffers = (PFNQSGENFRAMEBUFFERS)dlsym(lib, "glGenFramebuffers");
-    real_glDeleteFramebuffers = (PFNQSDELETEFRAMEBUFFERS)dlsym(lib, "glDeleteFramebuffers");
-    real_glBindFramebuffer = (PFNQSBINDFRAMEBUFFER)dlsym(lib, "glBindFramebuffer");
-    real_glFramebufferTexture2D = (PFNQSFRAMEBUFFERTEXTURE2D)dlsym(lib, "glFramebufferTexture2D");
-    real_glBlitFramebuffer = (PFNQSBLITFRAMEBUFFER)eglGetProcAddress("glBlitFramebuffer");
-    return real_glGetIntegerv && real_glGenFramebuffers && real_glDeleteFramebuffers &&
-           real_glBindFramebuffer && real_glFramebufferTexture2D && real_glBlitFramebuffer;
-}
-
-static uint32_t g_pixel_generation = 0;
-static uint32_t g_source_uploaded_generation = (uint32_t)-1;
-static uint8_t *g_pixel_cache = NULL; // MENU_WIDTH*MENU_HEIGHT*4 bytes
-
-void xr_menu_get_content_size(int *width, int *height) {
-    *width = MENU_WIDTH;
-    *height = MENU_HEIGHT;
-}
-
-// FindClass() from this thread (SDL's native thread, attached to the JVM
-// via AttachCurrentThread rather than spawned from Java) resolves against
-// the bootstrap classloader, which only knows framework classes - it can't
-// see de.ladkau.questshock.MenuOverlay at all. Routing through the
-// activity's own classloader is the standard, documented workaround.
-static jclass xr_menu_find_class(JNIEnv *env, jobject activity, const char *name) {
-    jclass activityClass = (*env)->GetObjectClass(env, activity);
-    jmethodID getClassLoader =
-        (*env)->GetMethodID(env, activityClass, "getClassLoader", "()Ljava/lang/ClassLoader;");
-    jobject classLoader = (*env)->CallObjectMethod(env, activity, getClassLoader);
-    jclass classLoaderClass = (*env)->FindClass(env, "java/lang/ClassLoader");
-    jmethodID loadClass = (*env)->GetMethodID(env, classLoaderClass, "loadClass",
-                                               "(Ljava/lang/String;)Ljava/lang/Class;");
-    jstring className = (*env)->NewStringUTF(env, name);
-    jclass result = (jclass)(*env)->CallObjectMethod(env, classLoader, loadClass, className);
-    (*env)->DeleteLocalRef(env, className);
-    return result;
-}
-
-static bool xr_menu_init_jni(void) {
-    JNIEnv *env = (JNIEnv *)SDL_AndroidGetJNIEnv();
-    jobject activity = (jobject)SDL_AndroidGetActivity();
-    if (env == NULL || activity == NULL) {
-        LOGE("XR: menu - no JNIEnv/Activity from SDL");
-        return false;
-    }
-
-    jclass localClass = xr_menu_find_class(env, activity, "de.ladkau.questshock.MenuOverlay");
-    if (localClass == NULL) {
-        LOGE("XR: menu - could not find MenuOverlay class");
-        return false;
-    }
-    g_menu_overlay_class = (jclass)(*env)->NewGlobalRef(env, localClass);
-
-    g_native_init_method =
-        (*env)->GetStaticMethodID(env, g_menu_overlay_class, "nativeInit", "(Landroid/app/Activity;)V");
-    g_take_pixels_method =
-        (*env)->GetStaticMethodID(env, g_menu_overlay_class, "nativeTakePixelsIfDirty", "()[B");
-    g_dispatch_touch_method =
-        (*env)->GetStaticMethodID(env, g_menu_overlay_class, "nativeDispatchTouch", "(FFZ)V");
-    if (!g_native_init_method || !g_take_pixels_method || !g_dispatch_touch_method) {
-        LOGE("XR: menu - could not resolve MenuOverlay JNI methods");
-        return false;
-    }
-
-    (*env)->CallStaticVoidMethod(env, g_menu_overlay_class, g_native_init_method, activity);
-    // CallStaticVoidMethod doesn't surface Java exceptions on its own - if
-    // MenuOverlay's constructor throws (it runs its View measure/layout/
-    // draw calls on this native render thread rather than the Android UI
-    // thread, a plausible crash vector), the exception would otherwise be
-    // left silently pending and corrupt whatever JNI call runs next.
-    if ((*env)->ExceptionCheck(env)) {
-        LOGE("XR: menu MenuOverlay.nativeInit() threw a pending Java exception:");
-        (*env)->ExceptionDescribe(env);
-        (*env)->ExceptionClear(env);
-        return false;
-    }
-    return true;
-}
-
-static bool xr_menu_create_source_texture(void) {
-    glGenTextures(1, &g_source_texture);
-    glBindTexture(GL_TEXTURE_2D, g_source_texture);
-    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
-    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
-    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
-    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
-    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, MENU_WIDTH, MENU_HEIGHT, 0, GL_RGBA, GL_UNSIGNED_BYTE,
-                 NULL);
-
-    GLint prevFbo = 0;
-    real_glGetIntegerv(GL_FRAMEBUFFER_BINDING, &prevFbo);
-    real_glGenFramebuffers(1, &g_source_fbo);
-    real_glBindFramebuffer(GL_FRAMEBUFFER, g_source_fbo);
-    real_glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D,
-                                 g_source_texture, 0);
-    real_glBindFramebuffer(GL_FRAMEBUFFER, (GLuint)prevFbo);
-
-    return g_source_texture != 0;
-}
-
-bool xr_menu_init(XrInstance instance, XrSession session) {
-    (void)instance;
-    (void)session;
-
-    if (!xr_menu_load_real_gles()) {
-        LOGE("XR: menu couldn't resolve real GLES functions via dlsym/eglGetProcAddress");
-        return false;
-    }
-    if (!xr_menu_create_source_texture())
-        return false;
-    if (!xr_menu_init_jni())
-        return false;
-
-    g_pixel_cache = (uint8_t *)malloc((size_t)MENU_WIDTH * MENU_HEIGHT * 4);
-    if (g_pixel_cache == NULL)
-        return false;
-
-    LOGI("XR: menu overlay ready (%dx%d)", MENU_WIDTH, MENU_HEIGHT);
-    return true;
-}
-
-void xr_menu_toggle_visible(void) {
-    g_visible = !g_visible;
-    LOGI("XR: menu quad now %s", g_visible ? "visible" : "hidden");
-}
-
-bool xr_menu_is_visible(void) { return g_visible; }
-
-void xr_menu_get_quad_extent(float *center_x_m, float *center_y_m, float *distance_m,
-                              float *half_width_m, float *half_height_m) {
-    *center_x_m = MENU_CENTER_X_METERS;
-    *center_y_m = MENU_CENTER_Y_METERS;
-    *distance_m = MENU_DISTANCE_METERS;
-    *half_width_m = MENU_WIDTH_METERS * 0.5f;
-    *half_height_m = MENU_WIDTH_METERS * 0.5f * (float)MENU_HEIGHT / (float)MENU_WIDTH;
-}
-
-void xr_menu_touch(float u, float v, bool down) {
-    if (g_menu_overlay_class == NULL)
-        return;
-    JNIEnv *env = (JNIEnv *)SDL_AndroidGetJNIEnv();
-    if (env == NULL)
-        return;
-    // Passed via CallStaticVoidMethodA/jvalue rather than the variadic
-    // Call*Method form - deliberately sidesteps relying on JNI
-    // implementations correctly un-doing C's float-to-double default
-    // argument promotion for varargs float parameters (a well-known JNI
-    // footgun; Android's ART handles it correctly, but there's no reason
-    // to depend on that when the jvalue form is unambiguous either way).
-    jvalue args[3];
-    args[0].f = u;
-    args[1].f = v;
-    args[2].z = (jboolean)down;
-    (*env)->CallStaticVoidMethodA(env, g_menu_overlay_class, g_dispatch_touch_method, args);
-}
-
-// Pulls MenuOverlay's latest pixels (if it redrew since the last check)
-// into g_pixel_cache and bumps g_pixel_generation - called once per visible
-// frame, before deciding whether g_source_texture needs a fresh upload.
-static void xr_menu_refresh_pixel_cache(void) {
-    JNIEnv *env = (JNIEnv *)SDL_AndroidGetJNIEnv();
-    if (env == NULL)
-        return;
-    jbyteArray pixels =
-        (jbyteArray)(*env)->CallStaticObjectMethod(env, g_menu_overlay_class, g_take_pixels_method);
-    // CallStaticObjectMethod also doesn't surface Java exceptions on its
-    // own - if nativeTakePixelsIfDirty() itself throws, it would otherwise
-    // silently look identical to "nothing changed yet" (a null return).
-    if ((*env)->ExceptionCheck(env)) {
-        LOGE("XR: menu nativeTakePixelsIfDirty() threw a pending Java exception:");
-        (*env)->ExceptionDescribe(env);
-        (*env)->ExceptionClear(env);
-        return;
-    }
-    if (pixels == NULL)
-        return;
-    (*env)->GetByteArrayRegion(env, pixels, 0, MENU_WIDTH * MENU_HEIGHT * 4, (jbyte *)g_pixel_cache);
-    (*env)->DeleteLocalRef(env, pixels);
-    g_pixel_generation++;
-}
-
-// Uploads g_pixel_cache into g_source_texture via plain (gl4es-routed)
-// glBindTexture/glTexSubImage2D, matching how the texture was created.
-// Skipped when nothing changed since the last upload. Texture-unit-0 state
-// is saved and restored around the upload, since this call is gl4es-routed
-// and the engine's own next-frame rendering assumes nothing touched its
-// texture bindings since it last drew.
-static void xr_menu_upload_source_texture_if_dirty(void) {
-    if (g_source_uploaded_generation == g_pixel_generation)
-        return;
-
-    GLint prevActiveTexture = GL_TEXTURE0;
-    glGetIntegerv(GL_ACTIVE_TEXTURE, &prevActiveTexture);
-    glActiveTexture(GL_TEXTURE0);
-    GLint prevTexture2D = 0;
-    glGetIntegerv(GL_TEXTURE_BINDING_2D, &prevTexture2D);
-
-    glBindTexture(GL_TEXTURE_2D, g_source_texture);
-    glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, MENU_WIDTH, MENU_HEIGHT, GL_RGBA, GL_UNSIGNED_BYTE,
-                     g_pixel_cache);
-    g_source_uploaded_generation = g_pixel_generation;
-
-    glBindTexture(GL_TEXTURE_2D, (GLuint)prevTexture2D);
-    if (prevActiveTexture != GL_TEXTURE0)
-        glActiveTexture((GLenum)prevActiveTexture);
-}
-
-void xr_menu_render_if_visible(void) {
-    if (g_source_fbo == 0)
-        return;
-
-    xr_menu_refresh_pixel_cache();
-    xr_menu_upload_source_texture_if_dirty();
-
-    // Copies g_source_texture (via g_source_fbo) directly into the menu's
-    // sub-rectangle of the currently-bound shared swapchain framebuffer,
-    // using the real GLES3 hardware blit (glBlitFramebuffer) rather than a
-    // shader-based full-screen-quad draw. A blit has no vertex/fragment
-    // shading stage, no shader program, and no texture units involved at
-    // all, so gl4es's fixed-pipeline-emulation layer - which unconditionally
-    // substitutes a customized shader (reproducing the game's own
-    // last-bound texture/fixed-function state) onto any gl4es-routed draw
-    // call - has nothing to intercept here. It also never touches gl4es's
-    // own tracked program/vertex-array/texture-binding shadow state, so the
-    // only piece of state that needs save/restore is the READ framebuffer
-    // binding; binding only GL_READ_FRAMEBUFFER, rather than the combined
-    // GL_FRAMEBUFFER target, leaves the DRAW side (the swapchain image
-    // itself) untouched throughout.
-    //
-    // Y is flipped between source and destination: g_source_texture's texel
-    // row v=0 holds MenuOverlay's Bitmap row 0 (the top of the rendered
-    // content, standard top-row-first raster order), and reading that same
-    // texture via an attached FBO, framebuffer y=0 accesses that identical
-    // row. The destination swapchain framebuffer follows the opposite
-    // convention - its own y=0 is its bottom edge - so swapping dstY0/dstY1
-    // keeps the menu content right-side up (glBlitFramebuffer supports
-    // inverted src/dst rects for exactly this).
-    GLint prevReadFbo = 0;
-    real_glGetIntegerv(GL_READ_FRAMEBUFFER_BINDING, &prevReadFbo);
-    GLint viewport[4] = {0, 0, 0, 0};
-    real_glGetIntegerv(GL_VIEWPORT, viewport);
-
-    real_glBindFramebuffer(GL_READ_FRAMEBUFFER, g_source_fbo);
-    real_glBlitFramebuffer(0, 0, MENU_WIDTH, MENU_HEIGHT, viewport[0], viewport[1] + viewport[3],
-                            viewport[0] + viewport[2], viewport[1], GL_COLOR_BUFFER_BIT,
-                            GL_LINEAR);
-    real_glBindFramebuffer(GL_READ_FRAMEBUFFER, (GLuint)prevReadFbo);
-}
-
-void xr_menu_shutdown(void) {
-    if (g_source_texture != 0)
-        glDeleteTextures(1, &g_source_texture);
-    g_source_texture = 0;
-    g_source_uploaded_generation = (uint32_t)-1;
-
-    if (g_source_fbo != 0 && real_glDeleteFramebuffers != NULL)
-        real_glDeleteFramebuffers(1, &g_source_fbo);
-    g_source_fbo = 0;
-
-    if (g_menu_overlay_class != NULL) {
-        JNIEnv *env = (JNIEnv *)SDL_AndroidGetJNIEnv();
-        if (env != NULL)
-            (*env)->DeleteGlobalRef(env, g_menu_overlay_class);
-    }
-    g_menu_overlay_class = NULL;
-    g_native_init_method = NULL;
-    g_take_pixels_method = NULL;
-    g_dispatch_touch_method = NULL;
-
-    free(g_pixel_cache);
-    g_pixel_cache = NULL;
-    g_pixel_generation = 0;
-
-    g_visible = false;
-}

android/app/src/main/cpp/xr_menu.h

@@ -1,70 +0,0 @@
-// Menu quad for questshock's immersive Quest build: MenuOverlay.java's
-// off-screen-rendered View tree (currently just a "Keyboard" button - see
-// the plan's step D for the on-screen keyboard that goes behind it),
-// toggled by xr_input.c's menu_toggle action and hit-tested by the same ray
-// xr_input.c already computes per hand.
-//
-// The menu's content shares xr_session.c's single OpenXR swapchain (see
-// that file for why: two independently-created swapchains submitted as
-// separate composition layers make the Horizon OS compositor show the
-// game's own content on the menu quad instead of the menu's - a
-// compositor-level limitation, not anything under this app's control).
-// xr_session.c sizes its shared swapchain to fit both the game's and the
-// menu's content side by side, and calls into this file to render the
-// menu's own sub-rectangle each visible frame.
-#ifndef QUESTSHOCK_XR_MENU_H
-#define QUESTSHOCK_XR_MENU_H
-
-#include <stdbool.h>
-
-#define XR_USE_PLATFORM_ANDROID 1
-#define XR_USE_GRAPHICS_API_OPENGL_ES 1
-#include <openxr/openxr.h>
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-// Pixel size of the menu's own content region - xr_session.c needs this
-// before it creates its shared swapchain, to size it to fit both the
-// game's and the menu's content. Has no OpenXR/GL dependency, safe to call
-// before xr_menu_init().
-void xr_menu_get_content_size(int *width, int *height);
-
-// Call once, after xr_session.c has created its shared swapchain - sets up
-// the MenuOverlay Java-side singleton and the gl4es-owned texture its
-// pixels get uploaded into. Returns false (logged, non-fatal - the caller
-// just has no menu) on failure.
-bool xr_menu_init(XrInstance instance, XrSession session);
-
-// Flips menu-quad visibility - called by xr_input.c on the menu_toggle
-// action's rising edge.
-void xr_menu_toggle_visible(void);
-bool xr_menu_is_visible(void);
-
-// Local-space pose/size of the menu quad, valid regardless of visibility -
-// shared with xr_input.c's ray/quad hit-testing, the same way
-// xr_get_game_quad_extent() (xr_session.h) is for the game quad.
-void xr_menu_get_quad_extent(float *center_x_m, float *center_y_m, float *distance_m,
-                              float *half_width_m, float *half_height_m);
-
-// Forwards a hit on the menu quad (in the quad's own 0..1 u/v, top-left
-// origin) to the MenuOverlay Java view as a synthetic touch down/up -
-// called by xr_input.c on the select_click action's edges, only while a
-// hand's aim ray currently hits the menu quad.
-void xr_menu_touch(float u, float v, bool down);
-
-// Refreshes MenuOverlay's pixels from Java if they changed since the last
-// call, and blits them into whichever framebuffer is currently bound -
-// xr_session.c binds its shared swapchain image and sets the
-// viewport/scissor to the menu's own sub-rectangle within it before calling
-// this. Only called while xr_menu_is_visible().
-void xr_menu_render_if_visible(void);
-
-void xr_menu_shutdown(void);
-
-#ifdef __cplusplus
-}
-#endif
-
-#endif

android/app/src/main/cpp/xr_overlay.c

@@ -0,0 +1,444 @@
+#include "xr_overlay.h"
+
+#include <dlfcn.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <android/log.h>
+#include <jni.h>
+
+#include <EGL/egl.h>
+#include <GLES3/gl3.h>
+
+#include <SDL.h>
+
+#include "xr_swapchain.h"
+
+#define TAG "QuestShock"
+#define LOGE(...) __android_log_print(ANDROID_LOG_ERROR, TAG, __VA_ARGS__)
+#define LOGI(...) __android_log_print(ANDROID_LOG_INFO, TAG, __VA_ARGS__)
+
+struct XrOverlay {
+    XrInstance instance; // for error logging only
+    int width, height;
+    float distance_m, half_width_m, half_height_m;
+    float center_x_m, center_y_m;
+    bool visible;
+
+    XrSwapchainState swapchain;
+
+    jclass java_class;
+    jmethodID native_init_method;
+    jmethodID take_pixels_method;
+    jmethodID dispatch_touch_method;
+    jmethodID update_cursor_method;
+
+    // This overlay's content lives in this texture, uploaded by
+    // xr_overlay_upload_source_texture_if_dirty() below. It's created
+    // through gl4es (glGenTextures/glBindTexture/glTexImage2D), like any
+    // other texture the engine itself creates, so ordinary
+    // glTexSubImage2D uploads against it work the normal way.
+    GLuint source_texture;
+    // A framebuffer with source_texture as its only color attachment, used
+    // as the read source for the blit in
+    // xr_overlay_render_and_build_layer() below. Created via the real
+    // (non-gl4es) GLES entry points below: that blit bypasses gl4es
+    // entirely (see xr_overlay_render_and_build_layer() for why), so its
+    // source framebuffer has to be a real GL object rather than one gl4es
+    // tracks.
+    GLuint source_fbo;
+
+    uint32_t pixel_generation;
+    uint32_t source_uploaded_generation;
+    uint8_t *pixel_cache; // width*height*4 bytes
+};
+
+// This file's other GL calls otherwise resolve to gl4es (the only GL
+// symbol provider linked into this binary - see android/engine-patches/
+// 02-android-opengl-es.patch), fine for anything shared with the engine's
+// own gl4es-routed rendering. But the source FBO's blit (see
+// xr_overlay_render_and_build_layer()) needs the real driver directly:
+// gl4es's fixed-pipeline emulation unconditionally substitutes its own
+// shader onto any gl4es-routed draw call, which would silently replace an
+// overlay's content with whatever the game itself last rendered. A
+// framebuffer blit has no shader stage at all, so going through the real
+// driver for it sidesteps the problem entirely - resolved once here and
+// shared by every XrOverlay instance, since they're process-wide function
+// pointers regardless of how many overlays exist.
+typedef void (*PFNQSGETINTEGERV)(GLenum, GLint *);
+typedef void (*PFNQSGENFRAMEBUFFERS)(GLsizei, GLuint *);
+typedef void (*PFNQSDELETEFRAMEBUFFERS)(GLsizei, const GLuint *);
+typedef void (*PFNQSBINDFRAMEBUFFER)(GLenum, GLuint);
+typedef void (*PFNQSFRAMEBUFFERTEXTURE2D)(GLenum, GLenum, GLenum, GLuint, GLint);
+typedef void (*PFNQSBLITFRAMEBUFFER)(GLint, GLint, GLint, GLint, GLint, GLint, GLint, GLint,
+                                      GLbitfield, GLenum);
+
+static PFNQSGETINTEGERV real_glGetIntegerv;
+static PFNQSGENFRAMEBUFFERS real_glGenFramebuffers;
+static PFNQSDELETEFRAMEBUFFERS real_glDeleteFramebuffers;
+static PFNQSBINDFRAMEBUFFER real_glBindFramebuffer;
+static PFNQSFRAMEBUFFERTEXTURE2D real_glFramebufferTexture2D;
+// glBlitFramebuffer is a GLES 3.0 addition; resolved via eglGetProcAddress
+// rather than dlsym, since Android's driver dispatch doesn't guarantee ES3+
+// symbols are dlsym-able by name from libGLESv2.so, unlike the GLES2-core
+// functions above.
+static PFNQSBLITFRAMEBUFFER real_glBlitFramebuffer;
+static bool g_real_gles_loaded = false;
+
+static bool xr_overlay_load_real_gles(void) {
+    if (g_real_gles_loaded)
+        return true;
+    void *lib = dlopen("libGLESv2.so", RTLD_NOW | RTLD_LOCAL);
+    if (lib == NULL) {
+        LOGE("XR: overlay dlopen(libGLESv2.so) failed: %s", dlerror());
+        return false;
+    }
+    real_glGetIntegerv = (PFNQSGETINTEGERV)dlsym(lib, "glGetIntegerv");
+    real_glGenFramebuffers = (PFNQSGENFRAMEBUFFERS)dlsym(lib, "glGenFramebuffers");
+    real_glDeleteFramebuffers = (PFNQSDELETEFRAMEBUFFERS)dlsym(lib, "glDeleteFramebuffers");
+    real_glBindFramebuffer = (PFNQSBINDFRAMEBUFFER)dlsym(lib, "glBindFramebuffer");
+    real_glFramebufferTexture2D = (PFNQSFRAMEBUFFERTEXTURE2D)dlsym(lib, "glFramebufferTexture2D");
+    real_glBlitFramebuffer = (PFNQSBLITFRAMEBUFFER)eglGetProcAddress("glBlitFramebuffer");
+    g_real_gles_loaded = real_glGetIntegerv && real_glGenFramebuffers &&
+                          real_glDeleteFramebuffers && real_glBindFramebuffer &&
+                          real_glFramebufferTexture2D && real_glBlitFramebuffer;
+    return g_real_gles_loaded;
+}
+
+// FindClass() from this thread (SDL's native thread, attached to the JVM
+// via AttachCurrentThread rather than spawned from Java) resolves against
+// the bootstrap classloader, which only knows framework classes - it can't
+// see app classes like de.ladkau.questshock.KeyboardOverlay at all.
+// Routing through the activity's own classloader is the standard,
+// documented workaround.
+static jclass xr_overlay_find_class(JNIEnv *env, jobject activity, const char *name) {
+    jclass activityClass = (*env)->GetObjectClass(env, activity);
+    jmethodID getClassLoader =
+        (*env)->GetMethodID(env, activityClass, "getClassLoader", "()Ljava/lang/ClassLoader;");
+    jobject classLoader = (*env)->CallObjectMethod(env, activity, getClassLoader);
+    jclass classLoaderClass = (*env)->FindClass(env, "java/lang/ClassLoader");
+    jmethodID loadClass = (*env)->GetMethodID(env, classLoaderClass, "loadClass",
+                                               "(Ljava/lang/String;)Ljava/lang/Class;");
+    jstring className = (*env)->NewStringUTF(env, name);
+    jclass result = (jclass)(*env)->CallObjectMethod(env, classLoader, loadClass, className);
+    (*env)->DeleteLocalRef(env, className);
+    return result;
+}
+
+static bool xr_overlay_init_jni(XrOverlay *overlay, const char *java_class_name) {
+    JNIEnv *env = (JNIEnv *)SDL_AndroidGetJNIEnv();
+    jobject activity = (jobject)SDL_AndroidGetActivity();
+    if (env == NULL || activity == NULL) {
+        LOGE("XR: overlay - no JNIEnv/Activity from SDL");
+        return false;
+    }
+
+    jclass localClass = xr_overlay_find_class(env, activity, java_class_name);
+    if (localClass == NULL) {
+        LOGE("XR: overlay - could not find class %s", java_class_name);
+        return false;
+    }
+    overlay->java_class = (jclass)(*env)->NewGlobalRef(env, localClass);
+
+    overlay->native_init_method = (*env)->GetStaticMethodID(env, overlay->java_class, "nativeInit",
+                                                              "(Landroid/app/Activity;)V");
+    overlay->take_pixels_method =
+        (*env)->GetStaticMethodID(env, overlay->java_class, "nativeTakePixelsIfDirty", "()[B");
+    overlay->dispatch_touch_method =
+        (*env)->GetStaticMethodID(env, overlay->java_class, "nativeDispatchTouch", "(FFZ)V");
+    overlay->update_cursor_method =
+        (*env)->GetStaticMethodID(env, overlay->java_class, "nativeUpdateCursor", "(FFZ)V");
+    if (!overlay->native_init_method || !overlay->take_pixels_method ||
+        !overlay->dispatch_touch_method || !overlay->update_cursor_method) {
+        LOGE("XR: overlay - could not resolve %s JNI methods", java_class_name);
+        return false;
+    }
+
+    (*env)->CallStaticVoidMethod(env, overlay->java_class, overlay->native_init_method, activity);
+    // CallStaticVoidMethod doesn't surface Java exceptions on its own - if
+    // the constructor throws (it runs its View measure/layout/draw calls
+    // on this native render thread rather than the Android UI thread, a
+    // plausible crash vector), the exception would otherwise be left
+    // silently pending and corrupt whatever JNI call runs next.
+    if ((*env)->ExceptionCheck(env)) {
+        LOGE("XR: overlay %s.nativeInit() threw a pending Java exception:", java_class_name);
+        (*env)->ExceptionDescribe(env);
+        (*env)->ExceptionClear(env);
+        return false;
+    }
+    return true;
+}
+
+static bool xr_overlay_create_source_texture(XrOverlay *overlay) {
+    glGenTextures(1, &overlay->source_texture);
+    glBindTexture(GL_TEXTURE_2D, overlay->source_texture);
+    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
+    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
+    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
+    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
+    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, overlay->width, overlay->height, 0, GL_RGBA,
+                 GL_UNSIGNED_BYTE, NULL);
+
+    GLint prevFbo = 0;
+    real_glGetIntegerv(GL_FRAMEBUFFER_BINDING, &prevFbo);
+    real_glGenFramebuffers(1, &overlay->source_fbo);
+    real_glBindFramebuffer(GL_FRAMEBUFFER, overlay->source_fbo);
+    real_glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D,
+                                 overlay->source_texture, 0);
+    real_glBindFramebuffer(GL_FRAMEBUFFER, (GLuint)prevFbo);
+
+    return overlay->source_texture != 0;
+}
+
+XrOverlay *xr_overlay_create(XrInstance instance, XrSession session, int64_t swapchain_format,
+                              const XrOverlayConfig *config) {
+    if (!xr_overlay_load_real_gles()) {
+        LOGE("XR: overlay couldn't resolve real GLES functions via dlsym/eglGetProcAddress");
+        return NULL;
+    }
+
+    XrOverlay *overlay = (XrOverlay *)calloc(1, sizeof(XrOverlay));
+    if (overlay == NULL)
+        return NULL;
+
+    overlay->instance = instance;
+    overlay->width = config->width;
+    overlay->height = config->height;
+    overlay->distance_m = config->distance_m;
+    overlay->half_width_m = config->width_m * 0.5f;
+    overlay->half_height_m = config->width_m * 0.5f * (float)config->height / (float)config->width;
+    overlay->center_x_m = config->default_center_x_m;
+    overlay->center_y_m = config->default_center_y_m;
+
+    if (!xr_swapchain_create(instance, session, swapchain_format, config->width, config->height,
+                              &overlay->swapchain)) {
+        LOGE("XR: overlay %s swapchain setup failed", config->java_class_name);
+        free(overlay);
+        return NULL;
+    }
+
+    if (!xr_overlay_create_source_texture(overlay)) {
+        xr_swapchain_destroy(&overlay->swapchain);
+        free(overlay);
+        return NULL;
+    }
+
+    if (!xr_overlay_init_jni(overlay, config->java_class_name)) {
+        xr_swapchain_destroy(&overlay->swapchain);
+        free(overlay);
+        return NULL;
+    }
+
+    overlay->pixel_cache = (uint8_t *)malloc((size_t)config->width * config->height * 4);
+    if (overlay->pixel_cache == NULL) {
+        xr_swapchain_destroy(&overlay->swapchain);
+        free(overlay);
+        return NULL;
+    }
+    overlay->source_uploaded_generation = (uint32_t)-1;
+
+    LOGI("XR: overlay %s ready (%dx%d)", config->java_class_name, config->width, config->height);
+    return overlay;
+}
+
+void xr_overlay_destroy(XrOverlay *overlay) {
+    if (overlay == NULL)
+        return;
+
+    if (overlay->source_texture != 0)
+        glDeleteTextures(1, &overlay->source_texture);
+    if (overlay->source_fbo != 0 && real_glDeleteFramebuffers != NULL)
+        real_glDeleteFramebuffers(1, &overlay->source_fbo);
+
+    if (overlay->java_class != NULL) {
+        JNIEnv *env = (JNIEnv *)SDL_AndroidGetJNIEnv();
+        if (env != NULL)
+            (*env)->DeleteGlobalRef(env, overlay->java_class);
+    }
+
+    xr_swapchain_destroy(&overlay->swapchain);
+    free(overlay->pixel_cache);
+    free(overlay);
+}
+
+// Every function below tolerates overlay == NULL (a no-op, or the
+// obvious "not visible"/zeroed-extent default) - xr_overlay_create() can
+// fail (e.g. its backing Java class couldn't be resolved), and unlike the
+// single static-global menu quad this replaced, an overlay instance is
+// otherwise a heap pointer callers (xr_session.c, questshock_native.c) can
+// hold onto and use across frames without re-checking for failure every
+// time.
+void xr_overlay_toggle_visible(XrOverlay *overlay) {
+    if (overlay != NULL)
+        overlay->visible = !overlay->visible;
+}
+
+void xr_overlay_set_visible(XrOverlay *overlay, bool visible) {
+    if (overlay != NULL)
+        overlay->visible = visible;
+}
+
+bool xr_overlay_is_visible(const XrOverlay *overlay) { return overlay != NULL && overlay->visible; }
+
+void xr_overlay_get_quad_extent(const XrOverlay *overlay, float *center_x_m, float *center_y_m,
+                                 float *distance_m, float *half_width_m, float *half_height_m) {
+    if (overlay == NULL) {
+        *center_x_m = *center_y_m = *distance_m = *half_width_m = *half_height_m = 0.0f;
+        return;
+    }
+    *center_x_m = overlay->center_x_m;
+    *center_y_m = overlay->center_y_m;
+    *distance_m = overlay->distance_m;
+    *half_width_m = overlay->half_width_m;
+    *half_height_m = overlay->half_height_m;
+}
+
+void xr_overlay_set_position(XrOverlay *overlay, float center_x_m, float center_y_m) {
+    if (overlay == NULL)
+        return;
+    overlay->center_x_m = center_x_m;
+    overlay->center_y_m = center_y_m;
+}
+
+void xr_overlay_touch(XrOverlay *overlay, float u, float v, bool down) {
+    if (overlay == NULL)
+        return;
+    JNIEnv *env = (JNIEnv *)SDL_AndroidGetJNIEnv();
+    if (env == NULL)
+        return;
+    // Passed via CallStaticVoidMethodA/jvalue rather than the variadic
+    // Call*Method form - deliberately sidesteps relying on JNI
+    // implementations correctly un-doing C's float-to-double default
+    // argument promotion for varargs float parameters (a well-known JNI
+    // footgun; Android's ART handles it correctly, but there's no reason
+    // to depend on that when the jvalue form is unambiguous either way).
+    jvalue args[3];
+    args[0].f = u;
+    args[1].f = v;
+    args[2].z = (jboolean)down;
+    (*env)->CallStaticVoidMethodA(env, overlay->java_class, overlay->dispatch_touch_method, args);
+}
+
+void xr_overlay_update_cursor(XrOverlay *overlay, float u, float v, bool visible) {
+    if (overlay == NULL)
+        return;
+    JNIEnv *env = (JNIEnv *)SDL_AndroidGetJNIEnv();
+    if (env == NULL)
+        return;
+    // Same jvalue-form rationale as xr_overlay_touch() above.
+    jvalue args[3];
+    args[0].f = u;
+    args[1].f = v;
+    args[2].z = (jboolean)visible;
+    (*env)->CallStaticVoidMethodA(env, overlay->java_class, overlay->update_cursor_method, args);
+}
+
+// Pulls the Java view's latest pixels (if it redrew since the last check)
+// into pixel_cache and bumps pixel_generation - called once per rendered
+// frame, before deciding whether source_texture needs a fresh upload.
+static void xr_overlay_refresh_pixel_cache(XrOverlay *overlay) {
+    JNIEnv *env = (JNIEnv *)SDL_AndroidGetJNIEnv();
+    if (env == NULL)
+        return;
+    jbyteArray pixels = (jbyteArray)(*env)->CallStaticObjectMethod(
+        env, overlay->java_class, overlay->take_pixels_method);
+    // CallStaticObjectMethod also doesn't surface Java exceptions on its
+    // own - if nativeTakePixelsIfDirty() itself throws, it would otherwise
+    // silently look identical to "nothing changed yet" (a null return).
+    if ((*env)->ExceptionCheck(env)) {
+        LOGE("XR: overlay nativeTakePixelsIfDirty() threw a pending Java exception:");
+        (*env)->ExceptionDescribe(env);
+        (*env)->ExceptionClear(env);
+        return;
+    }
+    if (pixels == NULL)
+        return;
+    (*env)->GetByteArrayRegion(env, pixels, 0, overlay->width * overlay->height * 4,
+                                (jbyte *)overlay->pixel_cache);
+    (*env)->DeleteLocalRef(env, pixels);
+    overlay->pixel_generation++;
+}
+
+// Uploads pixel_cache into source_texture via plain (gl4es-routed)
+// glBindTexture/glTexSubImage2D, matching how the texture was created.
+// Skipped when nothing changed since the last upload. Texture-unit-0 state
+// is saved and restored around the upload, since this call is gl4es-routed
+// and the engine's own next-frame rendering assumes nothing touched its
+// texture bindings since it last drew.
+static void xr_overlay_upload_source_texture_if_dirty(XrOverlay *overlay) {
+    if (overlay->source_uploaded_generation == overlay->pixel_generation)
+        return;
+
+    GLint prevActiveTexture = GL_TEXTURE0;
+    glGetIntegerv(GL_ACTIVE_TEXTURE, &prevActiveTexture);
+    glActiveTexture(GL_TEXTURE0);
+    GLint prevTexture2D = 0;
+    glGetIntegerv(GL_TEXTURE_BINDING_2D, &prevTexture2D);
+
+    glBindTexture(GL_TEXTURE_2D, overlay->source_texture);
+    glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, overlay->width, overlay->height, GL_RGBA,
+                     GL_UNSIGNED_BYTE, overlay->pixel_cache);
+    overlay->source_uploaded_generation = overlay->pixel_generation;
+
+    glBindTexture(GL_TEXTURE_2D, (GLuint)prevTexture2D);
+    if (prevActiveTexture != GL_TEXTURE0)
+        glActiveTexture((GLenum)prevActiveTexture);
+}
+
+bool xr_overlay_render_and_build_layer(XrOverlay *overlay, XrCompositionLayerQuad *out_quad) {
+    if (overlay == NULL || !overlay->visible)
+        return false;
+    if (!xr_swapchain_acquire(overlay->instance, &overlay->swapchain))
+        return false;
+
+    xr_overlay_refresh_pixel_cache(overlay);
+    xr_overlay_upload_source_texture_if_dirty(overlay);
+
+    // Copies source_texture (via source_fbo) directly into the swapchain
+    // image xr_swapchain_acquire() just bound - sized to exactly
+    // width x height, so the destination is always the whole image - using
+    // the real GLES3 hardware blit (glBlitFramebuffer) rather than a
+    // shader-based full-screen-quad draw. A blit has no vertex/fragment
+    // shading stage, no shader program, and no texture units involved at
+    // all, so gl4es's fixed-pipeline-emulation layer - which
+    // unconditionally substitutes a customized shader (reproducing the
+    // game's own last-bound texture/fixed-function state) onto any
+    // gl4es-routed draw call - has nothing to intercept here. It also
+    // never touches gl4es's own tracked program/vertex-array/
+    // texture-binding shadow state, so the only piece of state that needs
+    // save/restore is the READ framebuffer binding; binding only
+    // GL_READ_FRAMEBUFFER, rather than the combined GL_FRAMEBUFFER target,
+    // leaves the DRAW side (the swapchain image itself) untouched
+    // throughout.
+    //
+    // Y is flipped between source and destination: source_texture's texel
+    // row v=0 holds the Java view's Bitmap row 0 (the top of the rendered
+    // content, standard top-row-first raster order), and reading that same
+    // texture via an attached FBO, framebuffer y=0 accesses that identical
+    // row. The destination swapchain framebuffer follows the opposite
+    // convention - its own y=0 is its bottom edge - so swapping dstY0/dstY1
+    // keeps the content right-side up (glBlitFramebuffer supports inverted
+    // src/dst rects for exactly this).
+    GLint prevReadFbo = 0;
+    real_glGetIntegerv(GL_READ_FRAMEBUFFER_BINDING, &prevReadFbo);
+
+    real_glBindFramebuffer(GL_READ_FRAMEBUFFER, overlay->source_fbo);
+    real_glBlitFramebuffer(0, 0, overlay->width, overlay->height, 0, overlay->height,
+                            overlay->width, 0, GL_COLOR_BUFFER_BIT, GL_LINEAR);
+    real_glBindFramebuffer(GL_READ_FRAMEBUFFER, (GLuint)prevReadFbo);
+
+    xr_swapchain_release(overlay->instance, &overlay->swapchain);
+
+    memset(out_quad, 0, sizeof(*out_quad));
+    out_quad->type = XR_TYPE_COMPOSITION_LAYER_QUAD;
+    out_quad->subImage.swapchain = overlay->swapchain.swapchain;
+    out_quad->subImage.imageRect.extent.width = overlay->swapchain.width;
+    out_quad->subImage.imageRect.extent.height = overlay->swapchain.height;
+    out_quad->pose.orientation.w = 1.0f;
+    out_quad->pose.position.x = overlay->center_x_m;
+    out_quad->pose.position.y = overlay->center_y_m;
+    out_quad->pose.position.z = -overlay->distance_m;
+    out_quad->size.width = overlay->half_width_m * 2.0f;
+    out_quad->size.height = overlay->half_height_m * 2.0f;
+    return true;
+}

android/app/src/main/cpp/xr_overlay.h

@@ -0,0 +1,96 @@
+// Generic "off-screen Android View rendered into its own OpenXR quad,
+// hit-tested/touched via a laser pointer" module - each instance owns its
+// own swapchain (via xr_swapchain.h), its own visible/position state, and
+// the JNI glue to a backing Java class (menu launcher, keyboard, or any
+// future such panel) that must expose these static methods, matching the
+// pattern MenuOverlay.java already established:
+//   static void nativeInit(Activity activity)
+//   static byte[] nativeTakePixelsIfDirty()
+//   static void nativeDispatchTouch(float u, float v, boolean down)
+//   static void nativeUpdateCursor(float u, float v, boolean visible)
+//
+// xr_input.c drives touch/cursor dispatch and (for panels that support it,
+// like the keyboard's title bar) dragging via xr_overlay_set_position();
+// xr_session.c owns instance lifetime and calls
+// xr_overlay_render_and_build_layer() once per frame per instance.
+#ifndef QUESTSHOCK_XR_OVERLAY_H
+#define QUESTSHOCK_XR_OVERLAY_H
+
+#include <stdbool.h>
+#include <stdint.h>
+
+#define XR_USE_PLATFORM_ANDROID 1
+#define XR_USE_GRAPHICS_API_OPENGL_ES 1
+#include <openxr/openxr.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+typedef struct XrOverlay XrOverlay;
+
+typedef struct {
+    // Fully-qualified Java class name backing this overlay, e.g.
+    // "de.ladkau.questshock.KeyboardOverlay" - resolved via the activity's
+    // own ClassLoader (see xr_overlay_find_class() in xr_overlay.c).
+    const char *java_class_name;
+    // Pixel size of the overlay's own content/swapchain.
+    int width;
+    int height;
+    // Initial local-space pose (meters, in front of the viewer along -Z) -
+    // see xr_overlay_get_quad_extent(). width_m is the quad's physical
+    // width; its height is derived to preserve width/height's aspect.
+    float distance_m;
+    float width_m;
+    float default_center_x_m;
+    float default_center_y_m;
+} XrOverlayConfig;
+
+// instance/session are used to create this overlay's own swapchain
+// (format shared with every other quad this app submits - see
+// xr_session.c's chosenFormat). Returns NULL (logged) on failure - the
+// caller just has no such panel.
+XrOverlay *xr_overlay_create(XrInstance instance, XrSession session, int64_t swapchain_format,
+                              const XrOverlayConfig *config);
+void xr_overlay_destroy(XrOverlay *overlay);
+
+void xr_overlay_toggle_visible(XrOverlay *overlay);
+void xr_overlay_set_visible(XrOverlay *overlay, bool visible);
+bool xr_overlay_is_visible(const XrOverlay *overlay);
+
+// Local-space pose/size of the quad, valid regardless of visibility -
+// shared with xr_input.c's ray/quad hit-testing.
+void xr_overlay_get_quad_extent(const XrOverlay *overlay, float *center_x_m, float *center_y_m,
+                                 float *distance_m, float *half_width_m, float *half_height_m);
+
+// Repositions the quad (world-space X/Y offset, same units/space as
+// xr_overlay_get_quad_extent()'s center_x_m/center_y_m) - e.g. driven by
+// xr_input.c while the user drags a panel's title bar. Distance from the
+// viewer isn't adjustable this way - only left/right/up/down
+// repositioning, not push/pull.
+void xr_overlay_set_position(XrOverlay *overlay, float center_x_m, float center_y_m);
+
+// Forwards a hit on the quad (in the quad's own 0..1 u/v, top-left origin)
+// to the backing Java view as a synthetic touch down/up.
+void xr_overlay_touch(XrOverlay *overlay, float u, float v, bool down);
+
+// Updates the on-quad cursor the Java view draws at the current aim hit
+// point (same u/v convention as xr_overlay_touch()) - called once per
+// frame regardless of click state, so the cursor tracks the ray
+// continuously. visible=false hides it.
+void xr_overlay_update_cursor(XrOverlay *overlay, float u, float v, bool visible);
+
+// If visible: acquires this overlay's next swapchain image, blits the
+// Java view's latest rendered pixels into it, releases the image, and
+// fills *out_quad's subImage/pose/size. space/eyeVisibility/layerFlags are
+// left for the caller to set (shared/policy choices across every layer
+// this app submits, e.g. translucency - not this module's concern).
+// Returns false (out_quad untouched) if not visible or something failed -
+// the caller should skip submitting a layer for it this frame.
+bool xr_overlay_render_and_build_layer(XrOverlay *overlay, XrCompositionLayerQuad *out_quad);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif

android/app/src/main/cpp/xr_session.c

@@ -1,6 +1,5 @@
 #include "xr_session.h"
 
-#include <dlfcn.h>
 #include <stdlib.h>
 #include <string.h>
 
@@ -18,7 +17,8 @@
 #include <SDL.h>
 
 #include "xr_input.h"
-#include "xr_menu.h"
+#include "xr_overlay.h"
+#include "xr_swapchain.h"
 
 #define TAG "QuestShock"
 #define LOGE(...) __android_log_print(ANDROID_LOG_ERROR, TAG, __VA_ARGS__)
@@ -33,9 +33,26 @@
 #define QUAD_DISTANCE_METERS 2.0f
 #define QUAD_WIDTH_METERS 1.6f
 
-// Up to this many swapchain images/FBOs - real runtimes report small counts
+// The menu launcher (just a "Keyboard" button - see MenuOverlay.java) is
-// (2-4); this is just a fixed upper bound for the cache arrays below.
+// small and sits above where the keyboard defaults to; the keyboard keeps
-#define MAX_SWAPCHAIN_IMAGES 8
+// its previous size/position. They're independent XrOverlay instances (see
+// xr_overlay.h) so both can be shown/hidden/dragged separately - deliberately
+// positioned apart by default so they don't start out overlapping.
+#define MENU_JAVA_CLASS "de.ladkau.questshock.MenuOverlay"
+#define MENU_WIDTH 512
+#define MENU_HEIGHT 256
+#define MENU_DISTANCE_METERS 1.3f
+#define MENU_WIDTH_METERS 0.35f
+#define MENU_DEFAULT_CENTER_X_METERS 0.0f
+#define MENU_DEFAULT_CENTER_Y_METERS 0.15f
+
+#define KEYBOARD_JAVA_CLASS "de.ladkau.questshock.KeyboardOverlay"
+#define KEYBOARD_WIDTH 1024
+#define KEYBOARD_HEIGHT 768
+#define KEYBOARD_DISTANCE_METERS 1.5f
+#define KEYBOARD_WIDTH_METERS 0.8f
+#define KEYBOARD_DEFAULT_CENTER_X_METERS 0.0f
+#define KEYBOARD_DEFAULT_CENTER_Y_METERS -0.45f
 
 static XrInstance g_instance = XR_NULL_HANDLE;
 static XrSystemId g_system_id = XR_NULL_SYSTEM_ID;
@@ -54,55 +71,13 @@ static XrSessionState g_session_state = XR_SESSION_STATE_UNKNOWN;
 // later state.
 static bool g_session_running = false;
 
-// A single shared swapchain, wide enough to hold the game's own content
+static XrSwapchainState g_game_swapchain;
-// (left) and the menu's content (right) side by side - see xr_frame_end()
+static XrOverlay *g_menu_overlay = NULL;
-// for how each gets its own imageRect/viewport/scissor sub-rectangle within
+static XrOverlay *g_keyboard_overlay = NULL;
-// it. Deliberately not two independent swapchains: submitting two
-// separately-created swapchains as two XrCompositionLayerQuads makes the
-// Horizon OS compositor show the game's own content on the menu quad
-// instead of the menu's - a compositor-level limitation with that
-// configuration, not something fixable from app-side GL/OpenXR code.
-// Sharing one swapchain avoids it entirely.
-static XrSwapchain g_swapchain = XR_NULL_HANDLE;
-static int g_game_width = 0;
-static int g_game_height = 0;
-static int g_menu_width = 0;
-static int g_menu_height = 0;
-static GLuint g_swapchain_fbos[MAX_SWAPCHAIN_IMAGES];
-static uint32_t g_swapchain_image_count = 0;
 
 static XrTime g_predicted_display_time = 0;
 static bool g_frame_should_render = false;
-static bool g_have_acquired_image = false;
+static bool g_have_acquired_game_image = false;
-
-// This file's GL calls otherwise resolve to gl4es (the only GL symbol
-// provider linked into this binary - see android/engine-patches/
-// 02-android-opengl-es.patch), which is fine for anything shared with the
-// engine's own gl4es-routed rendering. But the swapchain images OpenXR
-// hands us are real driver texture objects gl4es never created itself,
-// and gl4es's own glFramebufferTexture2D can't attach a texture it has no
-// tracked metadata for. So the FBO *container* is created via gl4es's own
-// glGenFramebuffers/glBindFramebuffer (so gl4es recognizes the id as its
-// own and its own per-frame glBindFramebuffer succeeds), while the
-// texture-attach step - the specifically foreign part - goes through the
-// real driver directly, via dlsym against libGLESv2.so.
-typedef void (*PFNQSFRAMEBUFFERTEXTURE2D)(GLenum, GLenum, GLenum, GLuint, GLint);
-typedef GLenum (*PFNQSCHECKFRAMEBUFFERSTATUS)(GLenum);
-
-static PFNQSFRAMEBUFFERTEXTURE2D real_glFramebufferTexture2D;
-static PFNQSCHECKFRAMEBUFFERSTATUS real_glCheckFramebufferStatus;
-
-static bool xr_load_real_gles(void) {
-    void *lib = dlopen("libGLESv2.so", RTLD_NOW | RTLD_LOCAL);
-    if (lib == NULL) {
-        LOGE("XR: dlopen(libGLESv2.so) failed: %s", dlerror());
-        return false;
-    }
-    real_glFramebufferTexture2D = (PFNQSFRAMEBUFFERTEXTURE2D)dlsym(lib, "glFramebufferTexture2D");
-    real_glCheckFramebufferStatus =
-        (PFNQSCHECKFRAMEBUFFERSTATUS)dlsym(lib, "glCheckFramebufferStatus");
-    return real_glFramebufferTexture2D && real_glCheckFramebufferStatus;
-}
 
 static bool xr_check(XrResult result, const char *what) {
     if (XR_SUCCEEDED(result))
@@ -202,10 +177,12 @@ static bool xr_create_instance_and_session(int game_width, int game_height) {
     // Required by spec before creating an OpenGL ES-backed session, even
     // though we don't gate on the returned min/max API version here.
     PFN_xrGetOpenGLESGraphicsRequirementsKHR getGLESRequirements =
-        (PFN_xrGetOpenGLESGraphicsRequirementsKHR)xr_get_proc("xrGetOpenGLESGraphicsRequirementsKHR");
+        (PFN_xrGetOpenGLESGraphicsRequirementsKHR)xr_get_proc(
+            "xrGetOpenGLESGraphicsRequirementsKHR");
     if (getGLESRequirements == NULL)
         return false;
-    XrGraphicsRequirementsOpenGLESKHR glesRequirements = {XR_TYPE_GRAPHICS_REQUIREMENTS_OPENGL_ES_KHR};
+    XrGraphicsRequirementsOpenGLESKHR glesRequirements = {
+        XR_TYPE_GRAPHICS_REQUIREMENTS_OPENGL_ES_KHR};
     if (!xr_check(getGLESRequirements(g_instance, g_system_id, &glesRequirements),
                   "xrGetOpenGLESGraphicsRequirementsKHR"))
         return false;
@@ -246,7 +223,9 @@ static bool xr_create_instance_and_session(int game_width, int game_height) {
     // GL_RGBA with no decode step anywhere in this path, matching the
     // desktop SDL_RenderCopy path this replaces - an sRGB swapchain format
     // would auto-gamma-encode on write and double-encode already-encoded
-    // data.
+    // data. Every swapchain below (game quad, menu/keyboard overlays)
+    // shares this one choice - the compositor's supported format set
+    // doesn't depend on swapchain size.
     uint32_t formatCount = 0;
     xrEnumerateSwapchainFormats(g_session, 0, &formatCount, NULL);
     int64_t *formats = (int64_t *)malloc(sizeof(int64_t) * formatCount);
@@ -261,82 +240,38 @@ static bool xr_create_instance_and_session(int game_width, int game_height) {
     }
     free(formats);
 
-    g_game_width = game_width;
+    if (!xr_swapchain_create(g_instance, g_session, chosenFormat, game_width, game_height,
-    g_game_height = game_height;
+                              &g_game_swapchain)) {
-    xr_menu_get_content_size(&g_menu_width, &g_menu_height);
+        LOGE("XR: game swapchain setup failed");
-    // The game's own content occupies (0,0)-(game_width,game_height); the
-    // menu's occupies (game_width,0)-(game_width+menu_width,menu_height) -
-    // side by side in one image, wide enough and tall enough for both.
-    uint32_t sharedWidth = (uint32_t)(g_game_width + g_menu_width);
-    uint32_t sharedHeight = (uint32_t)(g_game_height > g_menu_height ? g_game_height : g_menu_height);
-
-    XrSwapchainCreateInfo swapchainInfo = {XR_TYPE_SWAPCHAIN_CREATE_INFO};
-    swapchainInfo.usageFlags = XR_SWAPCHAIN_USAGE_COLOR_ATTACHMENT_BIT | XR_SWAPCHAIN_USAGE_SAMPLED_BIT;
-    swapchainInfo.format = chosenFormat;
-    swapchainInfo.sampleCount = 1;
-    swapchainInfo.width = sharedWidth;
-    swapchainInfo.height = sharedHeight;
-    swapchainInfo.faceCount = 1;
-    swapchainInfo.arraySize = 1;
-    swapchainInfo.mipCount = 1;
-    if (!xr_check(xrCreateSwapchain(g_session, &swapchainInfo, &g_swapchain),
-                  "xrCreateSwapchain"))
-        return false;
-
-    uint32_t imageCount = 0;
-    xrEnumerateSwapchainImages(g_swapchain, 0, &imageCount, NULL);
-    if (imageCount > MAX_SWAPCHAIN_IMAGES) {
-        LOGE("XR: swapchain reports %u images, only room for %d", imageCount,
-             MAX_SWAPCHAIN_IMAGES);
-        return false;
-    }
-    // Zero-initialized, not just `.type` set per element - these structs
-    // also carry a `next` field the runtime may read, and an
-    // uninitialized stack array would leave it as garbage.
-    XrSwapchainImageOpenGLESKHR images[MAX_SWAPCHAIN_IMAGES] = {0};
-    for (uint32_t i = 0; i < imageCount; i++)
-        images[i].type = XR_TYPE_SWAPCHAIN_IMAGE_OPENGL_ES_KHR;
-    if (!xr_check(xrEnumerateSwapchainImages(g_swapchain, imageCount, &imageCount,
-                                              (XrSwapchainImageBaseHeader *)images),
-                  "xrEnumerateSwapchainImages"))
-        return false;
-    g_swapchain_image_count = imageCount;
-
-    if (!xr_load_real_gles()) {
-        LOGE("XR: couldn't resolve real GLES FBO functions via dlsym");
         return false;
     }
 
-    // Wraps each swapchain-provided texture in its own framebuffer, matching
+    // Both non-fatal - the game keeps rendering with no such panel if
-    // how OpenGL.cc's own CreateFrameBuffer() wraps backupBuffer - just
+    // either fails (e.g. the Java class couldn't be resolved via JNI).
-    // without a depth/stencil attachment, since the final composite draw
+    XrOverlayConfig menuConfig = {
-    // (see opengl_swap_and_restore) never needs one. The Gen/Bind calls are
+        .java_class_name = MENU_JAVA_CLASS,
-    // gl4es's own (linked, not dlsym'd) - see the comment above
+        .width = MENU_WIDTH,
-    // real_glFramebufferTexture2D for why.
+        .height = MENU_HEIGHT,
-    bool all_complete = true;
+        .distance_m = MENU_DISTANCE_METERS,
-    for (uint32_t i = 0; i < imageCount; i++) {
+        .width_m = MENU_WIDTH_METERS,
-        glGenFramebuffers(1, &g_swapchain_fbos[i]);
+        .default_center_x_m = MENU_DEFAULT_CENTER_X_METERS,
-        glBindFramebuffer(GL_FRAMEBUFFER, g_swapchain_fbos[i]);
+        .default_center_y_m = MENU_DEFAULT_CENTER_Y_METERS,
-        real_glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D,
+    };
-                                     images[i].image, 0);
+    g_menu_overlay = xr_overlay_create(g_instance, g_session, chosenFormat, &menuConfig);
-        GLenum status = real_glCheckFramebufferStatus(GL_FRAMEBUFFER);
-        if (status != GL_FRAMEBUFFER_COMPLETE) {
-            LOGE("XR: swapchain FBO %u incomplete: 0x%x", i, status);
-            all_complete = false;
-        }
-    }
-    glBindFramebuffer(GL_FRAMEBUFFER, 0);
-    if (!all_complete)
-        return false;
 
-    LOGI("XR: instance/session/swapchain ready (%ux%u shared, %dx%d game + %dx%d menu, %u images)",
+    XrOverlayConfig keyboardConfig = {
-         sharedWidth, sharedHeight, g_game_width, g_game_height, g_menu_width, g_menu_height,
+        .java_class_name = KEYBOARD_JAVA_CLASS,
-         imageCount);
+        .width = KEYBOARD_WIDTH,
-
+        .height = KEYBOARD_HEIGHT,
-    // Also non-fatal - the game keeps rendering with no menu quad if this
+        .distance_m = KEYBOARD_DISTANCE_METERS,
-    // fails (e.g. MenuOverlay couldn't be resolved via JNI).
+        .width_m = KEYBOARD_WIDTH_METERS,
-    xr_menu_init(g_instance, g_session);
+        .default_center_x_m = KEYBOARD_DEFAULT_CENTER_X_METERS,
+        .default_center_y_m = KEYBOARD_DEFAULT_CENTER_Y_METERS,
+    };
+    g_keyboard_overlay = xr_overlay_create(g_instance, g_session, chosenFormat, &keyboardConfig);
 
+    LOGI("XR: instance/session ready (game %dx%d, menu %dx%d, keyboard %dx%d)", game_width,
+         game_height, MENU_WIDTH, MENU_HEIGHT, KEYBOARD_WIDTH, KEYBOARD_HEIGHT);
     return true;
 }
 
@@ -349,6 +284,9 @@ bool xr_init(int game_width, int game_height) {
     return true;
 }
 
+XrOverlay *xr_session_get_menu_overlay(void) { return g_menu_overlay; }
+XrOverlay *xr_session_get_keyboard_overlay(void) { return g_keyboard_overlay; }
+
 void xr_poll_events(void) {
     if (g_instance == XR_NULL_HANDLE)
         return;
@@ -369,7 +307,8 @@ void xr_poll_events(void) {
             if (g_session_state == XR_SESSION_STATE_READY) {
                 XrSessionBeginInfo beginInfo = {XR_TYPE_SESSION_BEGIN_INFO};
                 beginInfo.primaryViewConfigurationType = XR_VIEW_CONFIGURATION_TYPE_PRIMARY_STEREO;
-                g_session_running = xr_check(xrBeginSession(g_session, &beginInfo), "xrBeginSession");
+                g_session_running =
+                    xr_check(xrBeginSession(g_session, &beginInfo), "xrBeginSession");
             } else if (g_session_state == XR_SESSION_STATE_STOPPING) {
                 xr_check(xrEndSession(g_session), "xrEndSession");
                 g_session_running = false;
@@ -384,7 +323,7 @@ void xr_poll_events(void) {
 bool xr_is_session_running(void) { return g_session_running; }
 
 bool xr_frame_begin(void) {
-    g_have_acquired_image = false;
+    g_have_acquired_game_image = false;
     if (!xr_is_session_running())
         return false;
 
@@ -402,113 +341,78 @@ bool xr_frame_begin(void) {
     if (!g_frame_should_render)
         return false;
 
-    uint32_t imageIndex = 0;
+    g_have_acquired_game_image = xr_swapchain_acquire(g_instance, &g_game_swapchain);
-    XrSwapchainImageAcquireInfo acquireInfo = {XR_TYPE_SWAPCHAIN_IMAGE_ACQUIRE_INFO};
+    return g_have_acquired_game_image;
-    if (!xr_check(xrAcquireSwapchainImage(g_swapchain, &acquireInfo, &imageIndex),
-                  "xrAcquireSwapchainImage"))
-        return false;
-
-    XrSwapchainImageWaitInfo waitImageInfo = {XR_TYPE_SWAPCHAIN_IMAGE_WAIT_INFO};
-    waitImageInfo.timeout = XR_INFINITE_DURATION;
-    if (!xr_check(xrWaitSwapchainImage(g_swapchain, &waitImageInfo), "xrWaitSwapchainImage"))
-        return false;
-
-    // gl4es's own (linked, not dlsym'd) bind - this targets an FBO id
-    // gl4es itself created (see xr_create_instance_and_session()), so its
-    // own "current FBO" bookkeeping updates correctly and its immediate-
-    // mode draw calls in android_draw_surface_as_quad() land in the right
-    // place. Scissor-limited to the game's own sub-rectangle of the shared
-    // image (see g_swapchain's comment) - glViewport alone would already
-    // keep the engine's draw calls within that rectangle, but glClear
-    // ignores viewport and would otherwise wipe the menu's own region of
-    // the same image whenever the engine clears before drawing.
-    glBindFramebuffer(GL_FRAMEBUFFER, g_swapchain_fbos[imageIndex]);
-    glViewport(0, 0, g_game_width, g_game_height);
-    glEnable(GL_SCISSOR_TEST);
-    glScissor(0, 0, g_game_width, g_game_height);
-
-    g_have_acquired_image = true;
-    return true;
 }
 
 void xr_frame_end(void) {
     if (g_session == XR_NULL_HANDLE)
         return;
 
-    // Draw the laser-pointer reticle into the still-bound swapchain
+    // Draw the laser-pointer reticle into the still-bound game swapchain
-    // framebuffer before releasing it, so it composites on top of
+    // framebuffer before releasing it, so it composites on top of whatever
-    // whatever this frame's game content already drew there. Syncing
+    // this frame's game content already drew there. Syncing actions
-    // actions happens even when there's nothing to draw (no acquired
+    // happens even when there's nothing to draw (no acquired image this
-    // image this frame), so edge detection (trigger/menu-button clicks)
+    // frame), so edge detection (trigger/menu-button clicks) doesn't miss a
-    // doesn't miss a frame. Only ever draws while the menu isn't visible
+    // frame. Only ever draws while neither overlay is visible (see
-    // (see xr_input.c), so it never needs to know about the menu's own
+    // xr_input.c) - the menu/keyboard render into their own independent
-    // sub-rectangle below.
+    // swapchain images below.
     if (xr_is_session_running())
-        xr_input_sync_and_draw(g_local_space, g_predicted_display_time, g_have_acquired_image);
+        xr_input_sync_and_draw(g_local_space, g_predicted_display_time,
+                                g_have_acquired_game_image);
 
-    bool menuVisible = g_have_acquired_image && xr_menu_is_visible();
+    if (g_have_acquired_game_image)
-    if (menuVisible) {
+        xr_swapchain_release(g_instance, &g_game_swapchain);
-        // Switch viewport/scissor to the menu's own sub-rectangle of the
-        // same shared image (to the right of the game's own region, see
-        // g_swapchain's comment) and let xr_menu.c draw its content there -
-        // still the same framebuffer bound in xr_frame_begin(), just a
-        // different sub-rectangle of it.
-        glViewport(g_game_width, 0, g_menu_width, g_menu_height);
-        glScissor(g_game_width, 0, g_menu_width, g_menu_height);
-        xr_menu_render_if_visible();
-    }
-
-    if (g_have_acquired_image) {
-        glDisable(GL_SCISSOR_TEST);
-        XrSwapchainImageReleaseInfo releaseInfo = {XR_TYPE_SWAPCHAIN_IMAGE_RELEASE_INFO};
-        xr_check(xrReleaseSwapchainImage(g_swapchain, &releaseInfo), "xrReleaseSwapchainImage");
-    }
 
     if (!xr_is_session_running())
         return;
 
+    // Up to 3 layers: the game quad (if a frame was actually rendered),
+    // and, while visible, the menu launcher and keyboard overlays - each
+    // gets its own acquire/render/release cycle against its own swapchain
+    // (see xr_overlay_render_and_build_layer()) any time before xrEndFrame,
+    // unlike the game quad there's no per-frame engine rendering to wrap
+    // around here, just each overlay's own blit.
     XrCompositionLayerQuad gameQuad = {XR_TYPE_COMPOSITION_LAYER_QUAD};
     gameQuad.space = g_local_space;
     gameQuad.eyeVisibility = XR_EYE_VISIBILITY_BOTH;
-    gameQuad.subImage.swapchain = g_swapchain;
+    gameQuad.subImage.swapchain = g_game_swapchain.swapchain;
-    gameQuad.subImage.imageRect.offset.x = 0;
+    gameQuad.subImage.imageRect.extent.width = g_game_swapchain.width;
-    gameQuad.subImage.imageRect.offset.y = 0;
+    gameQuad.subImage.imageRect.extent.height = g_game_swapchain.height;
-    gameQuad.subImage.imageRect.extent.width = g_game_width;
-    gameQuad.subImage.imageRect.extent.height = g_game_height;
     gameQuad.pose.orientation.w = 1.0f;
     gameQuad.pose.position.z = -QUAD_DISTANCE_METERS;
     gameQuad.size.width = QUAD_WIDTH_METERS;
-    gameQuad.size.height = QUAD_WIDTH_METERS * (float)g_game_height / (float)g_game_width;
+    gameQuad.size.height =
+        QUAD_WIDTH_METERS * (float)g_game_swapchain.height / (float)g_game_swapchain.width;
 
-    // Up to 2 layers: the game quad (if a frame was actually rendered) and,
+    const XrCompositionLayerBaseHeader *layers[3];
-    // while toggled on, the menu quad in front of it - see xr_input.c's
-    // menu_toggle handling. Both reference the SAME g_swapchain, just
-    // different imageRect sub-rectangles within it.
-    const XrCompositionLayerBaseHeader *layers[2];
     uint32_t layerCount = 0;
-    if (g_have_acquired_image)
+    if (g_have_acquired_game_image)
         layers[layerCount++] = (XrCompositionLayerBaseHeader *)&gameQuad;
 
-    XrCompositionLayerQuad menuQuad = {XR_TYPE_COMPOSITION_LAYER_QUAD};
+    XrCompositionLayerQuad menuQuad;
-    if (menuVisible) {
+    if (g_frame_should_render && xr_overlay_render_and_build_layer(g_menu_overlay, &menuQuad)) {
-        float centerX, centerY, distance, halfWidth, halfHeight;
-        xr_menu_get_quad_extent(&centerX, &centerY, &distance, &halfWidth, &halfHeight);
         menuQuad.space = g_local_space;
         menuQuad.eyeVisibility = XR_EYE_VISIBILITY_BOTH;
-        menuQuad.subImage.swapchain = g_swapchain;
-        menuQuad.subImage.imageRect.offset.x = g_game_width;
-        menuQuad.subImage.imageRect.offset.y = 0;
-        menuQuad.subImage.imageRect.extent.width = g_menu_width;
-        menuQuad.subImage.imageRect.extent.height = g_menu_height;
-        menuQuad.pose.orientation.w = 1.0f;
-        menuQuad.pose.position.x = centerX;
-        menuQuad.pose.position.y = centerY;
-        menuQuad.pose.position.z = -distance;
-        menuQuad.size.width = halfWidth * 2.0f;
-        menuQuad.size.height = halfHeight * 2.0f;
         layers[layerCount++] = (XrCompositionLayerBaseHeader *)&menuQuad;
     }
 
+    XrCompositionLayerQuad keyboardQuad;
+    if (g_frame_should_render &&
+        xr_overlay_render_and_build_layer(g_keyboard_overlay, &keyboardQuad)) {
+        keyboardQuad.space = g_local_space;
+        keyboardQuad.eyeVisibility = XR_EYE_VISIBILITY_BOTH;
+        // Lets the keyboard sit on screen translucently while playing
+        // rather than fully hiding whatever's behind it - KeyboardOverlay
+        // writes real sub-255 alpha into the swapchain texture (see
+        // OverlayPanel.compositeAndPublish()), already in Android's
+        // default premultiplied format, which is what this flag's absence
+        // of UNPREMULTIPLIED_ALPHA_BIT assumes. The menu launcher above
+        // deliberately doesn't get this flag - it stays fully opaque.
+        keyboardQuad.layerFlags = XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT;
+        layers[layerCount++] = (XrCompositionLayerBaseHeader *)&keyboardQuad;
+    }
+
     XrFrameEndInfo endInfo = {XR_TYPE_FRAME_END_INFO};
     endInfo.displayTime = g_predicted_display_time;
     endInfo.environmentBlendMode = XR_ENVIRONMENT_BLEND_MODE_OPAQUE;
@@ -518,15 +422,7 @@ void xr_frame_end(void) {
 }
 
 void xr_shutdown(void) {
-    for (uint32_t i = 0; i < g_swapchain_image_count; i++) {
+    xr_swapchain_destroy(&g_game_swapchain);
-        if (g_swapchain_fbos[i] != 0)
-            glDeleteFramebuffers(1, &g_swapchain_fbos[i]);
-    }
-    g_swapchain_image_count = 0;
-
-    if (g_swapchain != XR_NULL_HANDLE)
-        xrDestroySwapchain(g_swapchain);
-    g_swapchain = XR_NULL_HANDLE;
 
     if (g_local_space != XR_NULL_HANDLE)
         xrDestroySpace(g_local_space);
@@ -534,7 +430,10 @@ void xr_shutdown(void) {
 
     // Before the session/instance they were created from.
     xr_input_shutdown();
-    xr_menu_shutdown();
+    xr_overlay_destroy(g_menu_overlay);
+    g_menu_overlay = NULL;
+    xr_overlay_destroy(g_keyboard_overlay);
+    g_keyboard_overlay = NULL;
 
     if (g_session != XR_NULL_HANDLE)
         xrDestroySession(g_session);
@@ -551,5 +450,6 @@ void xr_shutdown(void) {
 void xr_get_game_quad_extent(float *distance_m, float *half_width_m, float *half_height_m) {
     *distance_m = QUAD_DISTANCE_METERS;
     *half_width_m = QUAD_WIDTH_METERS * 0.5f;
-    *half_height_m = QUAD_WIDTH_METERS * 0.5f * (float)g_game_height / (float)g_game_width;
+    *half_height_m =
+        QUAD_WIDTH_METERS * 0.5f * (float)g_game_swapchain.height / (float)g_game_swapchain.width;
 }

android/app/src/main/cpp/xr_session.h

@@ -1,28 +1,43 @@
 // Minimal OpenXR bring-up for questshock's immersive Quest build. Keeps the
 // game's own rendering untouched (see android/engine-patches/
 // 11-android-openxr-present.patch) - this module only owns the OpenXR
-// instance/session/swapchain and the final "submit a quad layer instead of
+// instance/session/swapchains and the final "submit quad layers instead of
-// presenting to a window" step. No stereo rendering, no controller input
+// presenting to a window" step. No stereo rendering - the game composite is
-// yet (see the plan's steps B/C/D for those) - the game composite is shown
+// shown as a single flat quad floating in front of the viewer.
-// as a single flat quad floating in front of the viewer.
 #ifndef QUESTSHOCK_XR_SESSION_H
 #define QUESTSHOCK_XR_SESSION_H
 
 #include <stdbool.h>
 
+#include "xr_overlay.h"
+
 #ifdef __cplusplus
 extern "C" {
 #endif
 
 // Call once, right after init_opengl() (OpenGL.cc) has created and made
 // current the GL context SDL/gl4es already use - creates the OpenXR
-// instance/session sharing that same EGL display/context, plus one
+// instance/session sharing that same EGL display/context, plus the game
-// swapchain sized to the game's logical resolution (game_width/height, i.e.
+// quad's own swapchain, sized to the game's logical resolution
-// grd_cap->w/h - see Shock.c's InitSDL()). Returns false if OpenXR bring-up
+// (game_width/height, i.e. grd_cap->w/h - see Shock.c's InitSDL()). Also
-// failed (e.g. no runtime installed) - callers should fall back to the
+// creates the menu launcher and keyboard overlay quads (see xr_overlay.h) -
-// existing window-present path in that case.
+// two independent XrOverlay instances, each with their own swapchain,
+// visibility, and position, retrievable via xr_session_get_menu_overlay()/
+// xr_session_get_keyboard_overlay() below. Returns false if OpenXR
+// bring-up failed (e.g. no runtime installed) - callers should fall back
+// to the existing window-present path in that case.
 bool xr_init(int game_width, int game_height);
 
+// The menu launcher (small "Keyboard" button panel, toggled by the
+// controller's menu button) and keyboard (the on-screen key grid, shown/
+// hidden independently via MenuOverlay's "Keyboard" button and
+// KeyboardOverlay's own Close button) overlay quads - used by xr_input.c
+// for ray/quad hit-testing and touch/cursor dispatch, and by
+// questshock_native.c's JNI glue (nativeShowKeyboard()/
+// nativeRequestClose()). NULL if xr_init() failed before creating them.
+XrOverlay *xr_session_get_menu_overlay(void);
+XrOverlay *xr_session_get_keyboard_overlay(void);
+
 // Pumps XR session-state events. Call once per frame, before
 // xr_frame_begin(). Must still be called even when xr_init() returned
 // false (no-op in that case).
@@ -34,17 +49,21 @@ void xr_poll_events(void);
 bool xr_is_session_running(void);
 
 // Begins the XR frame (xrWaitFrame/xrBeginFrame) and, if the runtime wants
-// this frame rendered, acquires the next swapchain image and binds its
+// this frame rendered, acquires the game quad's next swapchain image and
-// framebuffer as the current render target - ready for the caller to draw
+// binds its framebuffer as the current render target - ready for the
-// into exactly as it would have drawn to the default framebuffer. Returns
+// caller to draw into exactly as it would have drawn to the default
-// true if the caller should draw this frame; xr_frame_end() must be called
+// framebuffer. Returns true if the caller should draw this frame;
-// unconditionally afterward either way (a begun XR frame must always be
+// xr_frame_end() must be called unconditionally afterward either way (a
-// ended, rendered or not).
+// begun XR frame must always be ended, rendered or not).
 bool xr_frame_begin(void);
 
-// Releases the swapchain image (if one was acquired this frame) and
+// Releases the game quad's swapchain image (if one was acquired this
-// submits it as a single XrCompositionLayerQuad positioned in front of the
+// frame), then does the same acquire/render/release cycle for the menu
-// local reference space's origin, then ends the XR frame.
+// launcher and keyboard overlay quads via xr_overlay_render_and_build_layer()
+// for whichever of them is currently visible, and submits whichever of the
+// three quads actually rendered this frame as composition layers
+// positioned in front of the local reference space's origin, then ends the
+// XR frame.
 void xr_frame_end(void);
 
 void xr_shutdown(void);

android/app/src/main/cpp/xr_swapchain.c

@@ -0,0 +1,160 @@
+#include "xr_swapchain.h"
+
+#include <dlfcn.h>
+
+#include <android/log.h>
+
+#define TAG "QuestShock"
+#define LOGE(...) __android_log_print(ANDROID_LOG_ERROR, TAG, __VA_ARGS__)
+
+static bool xr_check(XrInstance instance, XrResult result, const char *what) {
+    if (XR_SUCCEEDED(result))
+        return true;
+    char resultString[XR_MAX_RESULT_STRING_SIZE] = {0};
+    if (instance != XR_NULL_HANDLE)
+        xrResultToString(instance, result, resultString);
+    LOGE("XR: %s failed: %s (%d)", what, resultString[0] ? resultString : "?", (int)result);
+    return false;
+}
+
+// This file's other GL calls otherwise resolve to gl4es (the only GL
+// symbol provider linked into this binary - see android/engine-patches/
+// 02-android-opengl-es.patch), which is fine for anything shared with the
+// engine's own gl4es-routed rendering. But the swapchain images OpenXR
+// hands us are real driver texture objects gl4es never created itself,
+// and gl4es's own glFramebufferTexture2D can't attach a texture it has no
+// tracked metadata for. So the FBO *container* is created via gl4es's own
+// glGenFramebuffers/glBindFramebuffer (so gl4es recognizes the id as its
+// own and its own per-frame glBindFramebuffer succeeds), while the
+// texture-attach step - the specifically foreign part - goes through the
+// real driver directly, via dlsym against libGLESv2.so.
+typedef void (*PFNQSFRAMEBUFFERTEXTURE2D)(GLenum, GLenum, GLenum, GLuint, GLint);
+typedef GLenum (*PFNQSCHECKFRAMEBUFFERSTATUS)(GLenum);
+
+static PFNQSFRAMEBUFFERTEXTURE2D real_glFramebufferTexture2D;
+static PFNQSCHECKFRAMEBUFFERSTATUS real_glCheckFramebufferStatus;
+static bool g_real_gles_loaded = false;
+
+static bool xr_swapchain_load_real_gles(void) {
+    if (g_real_gles_loaded)
+        return true;
+    void *lib = dlopen("libGLESv2.so", RTLD_NOW | RTLD_LOCAL);
+    if (lib == NULL) {
+        LOGE("XR: swapchain dlopen(libGLESv2.so) failed: %s", dlerror());
+        return false;
+    }
+    real_glFramebufferTexture2D = (PFNQSFRAMEBUFFERTEXTURE2D)dlsym(lib, "glFramebufferTexture2D");
+    real_glCheckFramebufferStatus =
+        (PFNQSCHECKFRAMEBUFFERSTATUS)dlsym(lib, "glCheckFramebufferStatus");
+    g_real_gles_loaded = real_glFramebufferTexture2D && real_glCheckFramebufferStatus;
+    return g_real_gles_loaded;
+}
+
+bool xr_swapchain_create(XrInstance instance, XrSession session, int64_t format, int width,
+                          int height, XrSwapchainState *out) {
+    if (!xr_swapchain_load_real_gles()) {
+        LOGE("XR: swapchain couldn't resolve real GLES FBO functions via dlsym");
+        return false;
+    }
+
+    out->width = width;
+    out->height = height;
+
+    XrSwapchainCreateInfo swapchainInfo = {XR_TYPE_SWAPCHAIN_CREATE_INFO};
+    swapchainInfo.usageFlags =
+        XR_SWAPCHAIN_USAGE_COLOR_ATTACHMENT_BIT | XR_SWAPCHAIN_USAGE_SAMPLED_BIT;
+    swapchainInfo.format = format;
+    swapchainInfo.sampleCount = 1;
+    swapchainInfo.width = (uint32_t)width;
+    swapchainInfo.height = (uint32_t)height;
+    swapchainInfo.faceCount = 1;
+    swapchainInfo.arraySize = 1;
+    swapchainInfo.mipCount = 1;
+    if (!xr_check(instance, xrCreateSwapchain(session, &swapchainInfo, &out->swapchain),
+                  "xrCreateSwapchain"))
+        return false;
+
+    uint32_t imageCount = 0;
+    xrEnumerateSwapchainImages(out->swapchain, 0, &imageCount, NULL);
+    if (imageCount > XR_SWAPCHAIN_MAX_IMAGES) {
+        LOGE("XR: swapchain reports %u images, only room for %d", imageCount,
+             XR_SWAPCHAIN_MAX_IMAGES);
+        return false;
+    }
+    // Zero-initialized, not just `.type` set per element - these structs
+    // also carry a `next` field the runtime may read, and an uninitialized
+    // stack array would leave it as garbage.
+    XrSwapchainImageOpenGLESKHR images[XR_SWAPCHAIN_MAX_IMAGES] = {0};
+    for (uint32_t i = 0; i < imageCount; i++)
+        images[i].type = XR_TYPE_SWAPCHAIN_IMAGE_OPENGL_ES_KHR;
+    if (!xr_check(instance,
+                  xrEnumerateSwapchainImages(out->swapchain, imageCount, &imageCount,
+                                              (XrSwapchainImageBaseHeader *)images),
+                  "xrEnumerateSwapchainImages"))
+        return false;
+    out->image_count = imageCount;
+
+    // Wraps each swapchain-provided texture in its own framebuffer, matching
+    // how OpenGL.cc's own CreateFrameBuffer() wraps backupBuffer - just
+    // without a depth/stencil attachment, since the final composite draw
+    // (see opengl_swap_and_restore) never needs one. The Gen/Bind calls are
+    // gl4es's own (linked, not dlsym'd) - see the comment above
+    // real_glFramebufferTexture2D for why.
+    bool all_complete = true;
+    for (uint32_t i = 0; i < imageCount; i++) {
+        glGenFramebuffers(1, &out->fbos[i]);
+        glBindFramebuffer(GL_FRAMEBUFFER, out->fbos[i]);
+        real_glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D,
+                                     images[i].image, 0);
+        GLenum status = real_glCheckFramebufferStatus(GL_FRAMEBUFFER);
+        if (status != GL_FRAMEBUFFER_COMPLETE) {
+            LOGE("XR: swapchain FBO %u incomplete: 0x%x", i, status);
+            all_complete = false;
+        }
+    }
+    glBindFramebuffer(GL_FRAMEBUFFER, 0);
+    return all_complete;
+}
+
+void xr_swapchain_destroy(XrSwapchainState *state) {
+    for (uint32_t i = 0; i < state->image_count; i++) {
+        if (state->fbos[i] != 0)
+            glDeleteFramebuffers(1, &state->fbos[i]);
+    }
+    state->image_count = 0;
+    if (state->swapchain != XR_NULL_HANDLE)
+        xrDestroySwapchain(state->swapchain);
+    state->swapchain = XR_NULL_HANDLE;
+}
+
+// Acquires the next image of the given swapchain, waits for the runtime to
+// finish with it, and binds its framebuffer (sized to exactly fill it) as
+// the current render target.
+bool xr_swapchain_acquire(XrInstance instance, XrSwapchainState *state) {
+    uint32_t imageIndex = 0;
+    XrSwapchainImageAcquireInfo acquireInfo = {XR_TYPE_SWAPCHAIN_IMAGE_ACQUIRE_INFO};
+    if (!xr_check(instance, xrAcquireSwapchainImage(state->swapchain, &acquireInfo, &imageIndex),
+                  "xrAcquireSwapchainImage"))
+        return false;
+
+    XrSwapchainImageWaitInfo waitImageInfo = {XR_TYPE_SWAPCHAIN_IMAGE_WAIT_INFO};
+    waitImageInfo.timeout = XR_INFINITE_DURATION;
+    if (!xr_check(instance, xrWaitSwapchainImage(state->swapchain, &waitImageInfo),
+                  "xrWaitSwapchainImage"))
+        return false;
+
+    // gl4es's own (linked, not dlsym'd) bind - this targets an FBO id gl4es
+    // itself created (see xr_swapchain_create()), so its own "current FBO"
+    // bookkeeping updates correctly and its immediate-mode draw calls
+    // (android_draw_surface_as_quad(), the laser reticle) land in the right
+    // place.
+    glBindFramebuffer(GL_FRAMEBUFFER, state->fbos[imageIndex]);
+    glViewport(0, 0, state->width, state->height);
+    return true;
+}
+
+void xr_swapchain_release(XrInstance instance, XrSwapchainState *state) {
+    XrSwapchainImageReleaseInfo releaseInfo = {XR_TYPE_SWAPCHAIN_IMAGE_RELEASE_INFO};
+    xr_check(instance, xrReleaseSwapchainImage(state->swapchain, &releaseInfo),
+              "xrReleaseSwapchainImage");
+}

android/app/src/main/cpp/xr_swapchain.h

@@ -0,0 +1,55 @@
+// Generic OpenXR swapchain + per-image GL framebuffer helper, shared by
+// every quad this app submits - the game quad (xr_session.c) and every
+// xr_overlay.c instance (menu launcher, keyboard) - since they all need
+// exactly the same setup, just at their own size/format.
+#ifndef QUESTSHOCK_XR_SWAPCHAIN_H
+#define QUESTSHOCK_XR_SWAPCHAIN_H
+
+#include <stdbool.h>
+#include <stdint.h>
+
+#include <EGL/egl.h>
+#include <GLES3/gl3.h>
+#include <jni.h>
+
+#define XR_USE_PLATFORM_ANDROID 1
+#define XR_USE_GRAPHICS_API_OPENGL_ES 1
+#include <openxr/openxr.h>
+#include <openxr/openxr_platform.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+// Up to this many swapchain images/FBOs - real runtimes report small counts
+// (2-4); this is just a fixed upper bound for the cache arrays below.
+#define XR_SWAPCHAIN_MAX_IMAGES 8
+
+typedef struct {
+    XrSwapchain swapchain;
+    GLuint fbos[XR_SWAPCHAIN_MAX_IMAGES];
+    uint32_t image_count;
+    int width;
+    int height;
+} XrSwapchainState;
+
+// Creates the swapchain and wraps each of its images in its own GL
+// framebuffer, ready to bind and render into directly. instance is only
+// used to log a human-readable error string on failure.
+bool xr_swapchain_create(XrInstance instance, XrSession session, int64_t format, int width,
+                          int height, XrSwapchainState *out);
+
+void xr_swapchain_destroy(XrSwapchainState *state);
+
+// Acquires the next image, waits for the runtime to finish with it, and
+// binds its framebuffer (sized to exactly fill it) as the current render
+// target.
+bool xr_swapchain_acquire(XrInstance instance, XrSwapchainState *state);
+
+void xr_swapchain_release(XrInstance instance, XrSwapchainState *state);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif

android/app/src/main/java/de/ladkau/questshock/KeyboardOverlay.java

@@ -0,0 +1,474 @@
+package de.ladkau.questshock;
+
+import android.app.Activity;
+import android.util.TypedValue;
+import android.view.Gravity;
+import android.view.KeyEvent;
+import android.view.View;
+import android.view.ViewGroup;
+import android.widget.Button;
+import android.widget.LinearLayout;
+import android.widget.TextView;
+import java.util.ArrayList;
+import java.util.List;
+import org.libsdl.app.SDLActivity;
+
+/**
+ * The on-screen keyboard quad (see OverlayPanel for the shared off-screen-
+ * render/touch/cursor plumbing this builds on) - a hand-built, full
+ * US-layout key grid (there's no system IME to borrow once immersive) plus
+ * a text field echoing what's been sent. Each key press is forwarded
+ * straight to the game as real input (see handleKeyPress()/
+ * handleCharKeyPress()) - non-printable keys as an
+ * SDLActivity.onNativeKeyDown()/onNativeKeyUp() pair, the same one a
+ * physical Bluetooth keyboard's presses already drive, and printable keys
+ * via a synthesized SDL_TEXTINPUT event pushed from native code (see
+ * commitPrintableChar()/nativeSendPrintableChar() in questshock_native.c) -
+ * except while a Ctrl/Alt "armed" modifier is active, when they go through
+ * the same onNativeKeyDown()/onNativeKeyUp() path instead, so the modifier
+ * carries across (see toggleModifier()). Shift is a purely local layer
+ * toggle (uppercase/symbols vs lowercase/numbers), never itself sent to the
+ * game.
+ *
+ * This is a fully independent overlay quad from MenuOverlay's small
+ * launcher - opened via that launcher's "Keyboard" button
+ * (nativeShowKeyboard()) but otherwise self-contained: once open, the
+ * controller's menu button (which only ever affects the menu launcher, see
+ * xr_input.c) doesn't hide it, so it stays up as a standing input panel
+ * while actually playing rather than a modal you open and close. The only
+ * way to hide it is its own title bar's Close button (buildTitleBar()),
+ * which also doubles as a drag handle for repositioning it (handled
+ * entirely on the native side - see xr_input.c - since it never reaches
+ * this class's own touch dispatch). contentAlpha() renders the whole panel
+ * semi-transparent so whatever's behind it stays visible while it's up.
+ */
+public class KeyboardOverlay extends OverlayPanel {
+    // Matches xr_session.c's KEYBOARD_WIDTH/KEYBOARD_HEIGHT swapchain size -
+    // fixed, not tied to any real display metric, since this tree is never
+    // shown in a real window.
+    private static final int WIDTH = 1024;
+    private static final int HEIGHT = 768;
+
+    private static final int TYPED_TEXT_HEIGHT = 100;
+
+    // Height of the title bar (buildTitleBar()) and width of its Close
+    // button - xr_input.c's TITLE_BAR_V_FRACTION/CLOSE_BUTTON_U_FRACTION
+    // must be kept in sync with these (as fractions of WIDTH/HEIGHT above),
+    // since the drag-handle-vs-Close-button-vs-rest-of-the-keyboard
+    // decision happens entirely on the native side, before a hit ever
+    // reaches this class's own touch dispatch.
+    private static final int TITLE_BAR_HEIGHT = 90;
+    private static final int CLOSE_BUTTON_WIDTH = 160;
+
+    // ~70% opaque - readable but lets whatever's behind the panel (the game
+    // quad, or empty space) show through while it's left up during play.
+    private static final int PANEL_ALPHA = 180;
+
+    private static KeyboardOverlay instance;
+
+    private TextView typedTextView;
+    private final StringBuilder typedText = new StringBuilder();
+
+    // Shift is a local-only layer toggle (never itself sent to the game -
+    // see updateShiftVisual()); Ctrl/Alt are one-shot "armed" modifiers,
+    // sent to the game the moment they're pressed and released again as
+    // soon as the next key consumes them (see toggleModifier()/
+    // releaseArmedModifiers()) - there's no press-and-hold gesture with a
+    // laser pointer + trigger click, so armed-then-consumed stands in for
+    // holding the key down.
+    private boolean shiftActive = false;
+    private boolean ctrlArmed = false;
+    private boolean altArmed = false;
+    private Button shiftButton;
+    private Button ctrlButton;
+    private Button altButton;
+
+    // Every CharKey created by addRow(), so updateShiftVisual() can relabel
+    // all of them at once when Shift toggles. Assigned (not a field
+    // initializer) at the top of buildContent() - see OverlayPanel's
+    // constructor note on why a field a subclass's buildContent() depends
+    // on can't rely on normal field-initializer timing.
+    private List<CharKey> charKeys;
+
+    // A key whose printed character (and, while unmodified, whose game
+    // input) depends on the Shift layer - covers letters, digits, and
+    // punctuation uniformly (e.g. 'q'/'Q', '1'/'!', '-'/'_'). androidKeyCode
+    // is only used for the Ctrl/Alt-combo path (see handleCharKeyPress()) -
+    // the unmodified path goes through commitPrintableChar() instead, which
+    // doesn't need an Android keycode at all.
+    private static final class CharKey {
+        final char base;
+        final char shifted;
+        final int androidKeyCode;
+        Button button;
+
+        CharKey(char base, char shifted, int androidKeyCode) {
+            this.base = base;
+            this.shifted = shifted;
+            this.androidKeyCode = androidKeyCode;
+        }
+    }
+
+    private static CharKey charKey(char base, char shifted, int androidKeyCode) {
+        return new CharKey(base, shifted, androidKeyCode);
+    }
+
+    // Space is a CharKey like any other (see addRow()), but a literal
+    // space character makes for an unreadable, blank-looking button - keep
+    // showing the word "Space" instead, same as before it became a CharKey.
+    private static String charKeyLabel(CharKey key, boolean shiftActive) {
+        if (key.base == ' ') {
+            return "Space";
+        }
+        return String.valueOf(shiftActive ? key.shifted : key.base);
+    }
+
+    private KeyboardOverlay(Activity activity) {
+        super(activity, WIDTH, HEIGHT);
+    }
+
+    @Override
+    protected int contentAlpha() {
+        return PANEL_ALPHA;
+    }
+
+    @Override
+    protected View buildContent() {
+        charKeys = new ArrayList<>();
+
+        LinearLayout panel = new LinearLayout(activity);
+        panel.setOrientation(LinearLayout.VERTICAL);
+        panel.setBackgroundColor(0xFF202020);
+
+        panel.addView(buildTitleBar(),
+            new LinearLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT, TITLE_BAR_HEIGHT));
+
+        typedTextView = new TextView(activity);
+        typedTextView.setTextColor(0xFFFFFFFF);
+        typedTextView.setTextSize(TypedValue.COMPLEX_UNIT_SP, 28);
+        typedTextView.setBackgroundColor(0xFF303030);
+        typedTextView.setPadding(20, 20, 20, 20);
+        panel.addView(typedTextView, new LinearLayout.LayoutParams(
+            ViewGroup.LayoutParams.MATCH_PARENT, TYPED_TEXT_HEIGHT));
+
+        // Rows follow real keyboard geometry where it helps (Tab starting
+        // the QWERTY row, Enter ending the home row, Shift starting the
+        // bottom-letter row, a compact arrow cluster bottom-right) so the
+        // layout reads as a familiar keyboard rather than an arbitrary grid.
+        addRow(panel, "Esc", "F1", "F2", "F3", "F4", "F5", "F6", "F7", "F8", "F9", "F10",
+            "F11", "F12", "Back");
+        addRow(panel,
+            charKey('`', '~', KeyEvent.KEYCODE_GRAVE),
+            charKey('1', '!', KeyEvent.KEYCODE_1),
+            charKey('2', '@', KeyEvent.KEYCODE_2),
+            charKey('3', '#', KeyEvent.KEYCODE_3),
+            charKey('4', '$', KeyEvent.KEYCODE_4),
+            charKey('5', '%', KeyEvent.KEYCODE_5),
+            charKey('6', '^', KeyEvent.KEYCODE_6),
+            charKey('7', '&', KeyEvent.KEYCODE_7),
+            charKey('8', '*', KeyEvent.KEYCODE_8),
+            charKey('9', '(', KeyEvent.KEYCODE_9),
+            charKey('0', ')', KeyEvent.KEYCODE_0),
+            charKey('-', '_', KeyEvent.KEYCODE_MINUS),
+            charKey('=', '+', KeyEvent.KEYCODE_EQUALS));
+        addRow(panel, "Tab",
+            charKey('q', 'Q', KeyEvent.KEYCODE_Q), charKey('w', 'W', KeyEvent.KEYCODE_W),
+            charKey('e', 'E', KeyEvent.KEYCODE_E), charKey('r', 'R', KeyEvent.KEYCODE_R),
+            charKey('t', 'T', KeyEvent.KEYCODE_T), charKey('y', 'Y', KeyEvent.KEYCODE_Y),
+            charKey('u', 'U', KeyEvent.KEYCODE_U), charKey('i', 'I', KeyEvent.KEYCODE_I),
+            charKey('o', 'O', KeyEvent.KEYCODE_O), charKey('p', 'P', KeyEvent.KEYCODE_P),
+            charKey('[', '{', KeyEvent.KEYCODE_LEFT_BRACKET),
+            charKey(']', '}', KeyEvent.KEYCODE_RIGHT_BRACKET),
+            charKey('\\', '|', KeyEvent.KEYCODE_BACKSLASH));
+        addRow(panel, "Ctrl",
+            charKey('a', 'A', KeyEvent.KEYCODE_A), charKey('s', 'S', KeyEvent.KEYCODE_S),
+            charKey('d', 'D', KeyEvent.KEYCODE_D), charKey('f', 'F', KeyEvent.KEYCODE_F),
+            charKey('g', 'G', KeyEvent.KEYCODE_G), charKey('h', 'H', KeyEvent.KEYCODE_H),
+            charKey('j', 'J', KeyEvent.KEYCODE_J), charKey('k', 'K', KeyEvent.KEYCODE_K),
+            charKey('l', 'L', KeyEvent.KEYCODE_L),
+            charKey(';', ':', KeyEvent.KEYCODE_SEMICOLON),
+            charKey('\'', '"', KeyEvent.KEYCODE_APOSTROPHE),
+            "Enter");
+        addRow(panel, "Shift",
+            charKey('z', 'Z', KeyEvent.KEYCODE_Z), charKey('x', 'X', KeyEvent.KEYCODE_X),
+            charKey('c', 'C', KeyEvent.KEYCODE_C), charKey('v', 'V', KeyEvent.KEYCODE_V),
+            charKey('b', 'B', KeyEvent.KEYCODE_B), charKey('n', 'N', KeyEvent.KEYCODE_N),
+            charKey('m', 'M', KeyEvent.KEYCODE_M),
+            charKey(',', '<', KeyEvent.KEYCODE_COMMA),
+            charKey('.', '>', KeyEvent.KEYCODE_PERIOD),
+            charKey('/', '?', KeyEvent.KEYCODE_SLASH),
+            "Up");
+        addRow(panel, "Alt", charKey(' ', ' ', KeyEvent.KEYCODE_SPACE), "Left", "Down", "Right");
+        return panel;
+    }
+
+    // The title bar: a decorative "drag here" label (not itself
+    // interactive - see the class doc, dragging is handled entirely by
+    // xr_input.c before a hit ever reaches here) plus a real Close button,
+    // which is dispatched as an ordinary click like any other key. Its
+    // height and the Close button's width must stay in sync with
+    // xr_input.c's TITLE_BAR_V_FRACTION/CLOSE_BUTTON_U_FRACTION.
+    private LinearLayout buildTitleBar() {
+        LinearLayout bar = new LinearLayout(activity);
+        bar.setOrientation(LinearLayout.HORIZONTAL);
+        bar.setBackgroundColor(0xFF3A3A3A);
+
+        TextView label = new TextView(activity);
+        label.setText("Keyboard  -  drag here to move");
+        label.setTextColor(0xFFFFFFFF);
+        label.setTextSize(TypedValue.COMPLEX_UNIT_SP, 22);
+        label.setGravity(Gravity.CENTER_VERTICAL);
+        label.setPadding(24, 0, 0, 0);
+        bar.addView(label, new LinearLayout.LayoutParams(
+            0, ViewGroup.LayoutParams.MATCH_PARENT, 1f));
+
+        Button close = new Button(activity);
+        close.setText("Close");
+        close.setOnClickListener(v -> handleKeyPress("Close"));
+        close.setOnTouchListener(CLICK_ON_TOUCH_UP);
+        bar.addView(close,
+            new LinearLayout.LayoutParams(CLOSE_BUTTON_WIDTH, ViewGroup.LayoutParams.MATCH_PARENT));
+
+        return bar;
+    }
+
+    // Each row fills the panel's remaining height evenly (weight 1 on the
+    // row itself). Accepts a mix of String (a fixed-action control key,
+    // handled by handleKeyPress()) and CharKey (a Shift-sensitive character
+    // key, handled by handleCharKeyPress()) per item, so a row can match
+    // real keyboard geometry (e.g. "Tab" followed by a run of CharKeys).
+    // Within a row, CharKeys and most control keys share width equally
+    // except Space (wider, like a real keyboard) and Esc/Back/Enter/Tab/
+    // Shift/Ctrl/Alt (narrower, see keyWeight()).
+    private void addRow(LinearLayout panel, Object... items) {
+        LinearLayout row = new LinearLayout(activity);
+        row.setOrientation(LinearLayout.HORIZONTAL);
+        for (Object item : items) {
+            Button key = new Button(activity);
+            key.setOnTouchListener(CLICK_ON_TOUCH_UP);
+            float weight;
+            if (item instanceof CharKey) {
+                CharKey charKey = (CharKey) item;
+                charKey.button = key;
+                key.setText(charKeyLabel(charKey, shiftActive));
+                key.setOnClickListener(v -> handleCharKeyPress(charKey));
+                charKeys.add(charKey);
+                // Space is a CharKey too (base == shifted == ' '), but keeps
+                // its traditional wide key like the other control keys do.
+                weight = (charKey.base == ' ') ? 4f : 1f;
+            } else {
+                String label = (String) item;
+                key.setText(label);
+                key.setOnClickListener(v -> handleKeyPress(label));
+                weight = keyWeight(label);
+                if (label.equals("Shift")) {
+                    shiftButton = key;
+                } else if (label.equals("Ctrl")) {
+                    ctrlButton = key;
+                } else if (label.equals("Alt")) {
+                    altButton = key;
+                }
+            }
+            row.addView(key, new LinearLayout.LayoutParams(
+                0, ViewGroup.LayoutParams.MATCH_PARENT, weight));
+        }
+        panel.addView(row, new LinearLayout.LayoutParams(
+            ViewGroup.LayoutParams.MATCH_PARENT, 0, 1f));
+    }
+
+    private static float keyWeight(String label) {
+        if (label.equals("Back") || label.equals("Esc") || label.equals("Enter")
+                || label.equals("Tab") || label.equals("Shift") || label.equals("Ctrl")
+                || label.equals("Alt")) {
+            return 1.5f;
+        }
+        return 1f;
+    }
+
+    // Maps a control key's label to the Android keycode forwarded via
+    // SDLActivity.onNativeKeyDown()/onNativeKeyUp() - the same pair a
+    // physical Bluetooth keyboard's key events already drive (see
+    // SDLActivity.handleKeyEvent()). F-keys are contiguous in
+    // android.view.KeyEvent (KEYCODE_F1..KEYCODE_F12), so "F1".."F12" are
+    // computed rather than listed individually.
+    private static int controlKeyCode(String label) {
+        switch (label) {
+            case "Back": return KeyEvent.KEYCODE_DEL;
+            case "Esc": return KeyEvent.KEYCODE_ESCAPE;
+            case "Enter": return KeyEvent.KEYCODE_ENTER;
+            case "Tab": return KeyEvent.KEYCODE_TAB;
+            case "Up": return KeyEvent.KEYCODE_DPAD_UP;
+            case "Down": return KeyEvent.KEYCODE_DPAD_DOWN;
+            case "Left": return KeyEvent.KEYCODE_DPAD_LEFT;
+            case "Right": return KeyEvent.KEYCODE_DPAD_RIGHT;
+            default:
+                if (label.charAt(0) == 'F') {
+                    int n = Integer.parseInt(label.substring(1));
+                    return KeyEvent.KEYCODE_F1 + (n - 1);
+                }
+                throw new IllegalArgumentException("no keycode for " + label);
+        }
+    }
+
+    // "Close" (the title bar's button - see buildTitleBar()) hides this
+    // overlay's quad entirely (see nativeRequestClose()) and never reaches
+    // the game. Shift/Ctrl/Alt are handled separately below (Shift is a
+    // local layer toggle; Ctrl/Alt are one-shot armed modifiers) since
+    // neither sends a plain key event of its own the way every other
+    // control key here does. Everything else (Back/Esc/Enter/Tab/arrows/
+    // F1-F12) is non-printable, so it's forwarded as a real Android key
+    // event via controlKeyCode() above, then releases any armed Ctrl/Alt
+    // modifier (this key just consumed it). This all matters beyond just
+    // the visible typedTextView below: the engine's cutscene skip handler
+    // (cutsloop.c's cutscene_key_handler()) only reacts to Esc/Enter/Space,
+    // so those need to reach the game as actual game input, not just be
+    // echoed locally.
+    private void handleKeyPress(String label) {
+        if (label.equals("Close")) {
+            nativeRequestClose();
+            return;
+        }
+        if (label.equals("Shift")) {
+            shiftActive = !shiftActive;
+            updateShiftVisual();
+            return;
+        }
+        if (label.equals("Ctrl") || label.equals("Alt")) {
+            toggleModifier(label);
+            return;
+        }
+
+        int keyCode = controlKeyCode(label);
+        SDLActivity.onNativeKeyDown(keyCode);
+        SDLActivity.onNativeKeyUp(keyCode);
+        releaseArmedModifiers();
+
+        if (label.equals("Back")) {
+            if (typedText.length() > 0) {
+                typedText.setLength(typedText.length() - 1);
+            }
+        } else {
+            typedText.append('[').append(label).append(']');
+        }
+        typedTextView.setText(typedText.toString());
+    }
+
+    // A CharKey's printed character depends only on the Shift layer
+    // (updateShiftVisual() keeps its button's label in sync). Which game
+    // input it produces depends on whether a modifier is armed: unmodified,
+    // it goes through commitPrintableChar() below like a normal typed
+    // character; with Ctrl/Alt armed, it instead goes through the same
+    // onNativeKeyDown()/onNativeKeyUp() path handleKeyPress() uses for
+    // control keys, since that's the only path that carries
+    // ev.key.keysym.mod through to the engine (sdl_events.c's pump_events()
+    // reads Ctrl/Alt only off that path, never off SDL_TEXTINPUT).
+    private void handleCharKeyPress(CharKey key) {
+        char c = shiftActive ? key.shifted : key.base;
+        if (ctrlArmed || altArmed) {
+            SDLActivity.onNativeKeyDown(key.androidKeyCode);
+            SDLActivity.onNativeKeyUp(key.androidKeyCode);
+            releaseArmedModifiers();
+        } else {
+            commitPrintableChar(c);
+        }
+        typedText.append(c);
+        typedTextView.setText(typedText.toString());
+    }
+
+    // Ctrl/Alt are "armed" rather than held: pressing one immediately sends
+    // its keydown (so SDL's own modifier tracking picks it up for whatever
+    // key comes next - see handleCharKeyPress()/handleKeyPress()) and
+    // brackets its label for feedback; pressing the same key again before
+    // it's been used cancels it (sends the matching keyup, un-brackets).
+    // The normal case - actually being consumed by the next key press - is
+    // handled by releaseArmedModifiers() below, not here.
+    private void toggleModifier(String label) {
+        boolean ctrl = label.equals("Ctrl");
+        Button button = ctrl ? ctrlButton : altButton;
+        int keyCode = ctrl ? KeyEvent.KEYCODE_CTRL_LEFT : KeyEvent.KEYCODE_ALT_LEFT;
+        boolean nowArmed = ctrl ? !ctrlArmed : !altArmed;
+        if (ctrl) {
+            ctrlArmed = nowArmed;
+        } else {
+            altArmed = nowArmed;
+        }
+        if (nowArmed) {
+            SDLActivity.onNativeKeyDown(keyCode);
+        } else {
+            SDLActivity.onNativeKeyUp(keyCode);
+        }
+        button.setText(nowArmed ? "[" + label + "]" : label);
+    }
+
+    // Called after any key actually reaches the game (a control key in
+    // handleKeyPress(), or a CharKey in handleCharKeyPress()) so an armed
+    // Ctrl/Alt only ever applies to the very next key, then releases -
+    // matching a real Ctrl/Alt+key combo's keyup once the combo is done.
+    private void releaseArmedModifiers() {
+        if (ctrlArmed) {
+            SDLActivity.onNativeKeyUp(KeyEvent.KEYCODE_CTRL_LEFT);
+            ctrlArmed = false;
+            ctrlButton.setText("Ctrl");
+        }
+        if (altArmed) {
+            SDLActivity.onNativeKeyUp(KeyEvent.KEYCODE_ALT_LEFT);
+            altArmed = false;
+            altButton.setText("Alt");
+        }
+    }
+
+    // Shift never itself reaches the game (see class doc) - it only flips
+    // which character each CharKey shows/sends, including its own label.
+    // Note this means a Ctrl+Shift+key chord won't carry Shift to the
+    // engine (Ctrl/Alt are real forwarded modifier keys, Shift here isn't) -
+    // not worth solving unless it actually comes up.
+    private void updateShiftVisual() {
+        shiftButton.setText(shiftActive ? "[Shift]" : "Shift");
+        for (CharKey key : charKeys) {
+            key.button.setText(charKeyLabel(key, shiftActive));
+        }
+    }
+
+    // Synthesizes the SDL_TEXTINPUT event pump_events() (sdl_events.c)
+    // requires for printable characters (see handleCharKeyPress() above),
+    // by building and pushing it directly in native code (see
+    // nativeSendPrintableChar() in questshock_native.c) rather than through
+    // Android's IME (there's no real IME session behind this off-screen,
+    // never-attached grid for that plumbing to hook into).
+    private static void commitPrintableChar(char c) {
+        nativeSendPrintableChar(c);
+    }
+
+    private static native void nativeSendPrintableChar(char c);
+
+    // Hides this overlay's quad (xr_overlay_set_visible(..., false)) - the
+    // only way to do so, since the controller's menu button doesn't affect
+    // the keyboard (see this class's doc comment and xr_input.c).
+    private static native void nativeRequestClose();
+
+    // Called from xr_overlay.c's xr_overlay_init_jni(), on the render
+    // thread, right after it resolves this class via the activity's own
+    // ClassLoader (plain FindClass() can't see app classes from a thread
+    // that was attached to the JVM rather than spawned from Java).
+    public static void nativeInit(Activity activity) {
+        if (instance == null) {
+            instance = new KeyboardOverlay(activity);
+        }
+    }
+
+    public static byte[] nativeTakePixelsIfDirty() {
+        return instance == null ? null : instance.takePixelsIfDirty();
+    }
+
+    public static void nativeDispatchTouch(final float u, final float v, final boolean down) {
+        if (instance != null) {
+            instance.dispatchTouch(u, v, down);
+        }
+    }
+
+    public static void nativeUpdateCursor(final float u, final float v, final boolean visible) {
+        if (instance != null) {
+            instance.dispatchUpdateCursor(u, v, visible);
+        }
+    }
+}

android/app/src/main/java/de/ladkau/questshock/MenuOverlay.java

@@ -1,132 +1,76 @@
 package de.ladkau.questshock;
 
 import android.app.Activity;
-import android.graphics.Bitmap;
-import android.graphics.Canvas;
-import android.os.SystemClock;
-import android.util.Log;
 import android.view.Gravity;
-import android.view.MotionEvent;
 import android.view.View;
 import android.widget.Button;
 import android.widget.FrameLayout;
-import java.nio.ByteBuffer;
 
 /**
- * An off-screen, never-attached-to-a-window View tree for questshock's
+ * The small main menu launcher quad (see OverlayPanel for the shared off-
- * OpenXR menu quad (see android/app/src/main/cpp/xr_menu.c, which owns the
+ * screen-render/touch/cursor plumbing this builds on) - just a "Keyboard"
- * quad's swapchain and drives this class entirely via JNI - construction,
+ * button for now. Toggled by the controller's menu button (see
- * touch input, and pixel readback). Rendering to a Bitmap and dispatching
+ * xr_input.c); its "Keyboard" click only opens the (fully independent)
- * synthetic MotionEvents into an unattached hierarchy both work the same
+ * keyboard overlay quad via nativeShowKeyboard() - it never touches this
- * way they would for an attached View - draw(Canvas)/dispatchTouchEvent()
+ * panel's own visibility, so both can be shown together (see
- * don't require a ViewRootImpl/window, just a measured+laid-out tree.
+ * KeyboardOverlay for the keyboard itself).
  */
-public class MenuOverlay {
+public class MenuOverlay extends OverlayPanel {
-    private static final String TAG = "QuestShock";
+    // Matches xr_session.c's MENU_WIDTH/MENU_HEIGHT swapchain size - fixed,
-
-    // Matches xr_menu.c's MENU_WIDTH/MENU_HEIGHT swapchain size - fixed,
     // not tied to any real display metric, since this tree is never shown
     // in a real window.
-    static final int WIDTH = 1024;
+    private static final int WIDTH = 512;
-    static final int HEIGHT = 768;
+    private static final int HEIGHT = 256;
 
     private static MenuOverlay instance;
 
-    private final Activity activity;
-    private final FrameLayout root;
-    private final Button keyboardButton;
-    private final Bitmap bitmap;
-    private final Canvas canvas;
-
-    private final Object pixelLock = new Object();
-    private byte[] pendingPixels;
-
     private MenuOverlay(Activity activity) {
-        this.activity = activity;
+        super(activity, WIDTH, HEIGHT);
-
-        root = new FrameLayout(activity);
-        root.setBackgroundColor(0xFF202020);
-
-        keyboardButton = new Button(activity);
-        keyboardButton.setText("Keyboard");
-        keyboardButton.setOnClickListener(v -> Log.i(TAG, "MenuOverlay: Keyboard button clicked"));
-        FrameLayout.LayoutParams lp = new FrameLayout.LayoutParams(420, 140);
-        lp.gravity = Gravity.CENTER;
-        root.addView(keyboardButton, lp);
-
-        bitmap = Bitmap.createBitmap(WIDTH, HEIGHT, Bitmap.Config.ARGB_8888);
-        canvas = new Canvas(bitmap);
-
-        int widthSpec = View.MeasureSpec.makeMeasureSpec(WIDTH, View.MeasureSpec.EXACTLY);
-        int heightSpec = View.MeasureSpec.makeMeasureSpec(HEIGHT, View.MeasureSpec.EXACTLY);
-        root.measure(widthSpec, heightSpec);
-        root.layout(0, 0, WIDTH, HEIGHT);
-
-        forceRedraw();
     }
 
-    // Called from xr_menu.c's xr_menu_init(), on the render thread, right
+    @Override
-    // after it resolves this class via the activity's own ClassLoader
+    protected View buildContent() {
-    // (plain FindClass() can't see app classes from a thread that was
+        FrameLayout panel = new FrameLayout(activity);
-    // attached to the JVM rather than spawned from Java - see that file's
+        panel.setBackgroundColor(0xFF202020);
-    // xr_menu_find_class()).
+        Button keyboardButton = new Button(activity);
+        keyboardButton.setText("Keyboard");
+        keyboardButton.setOnClickListener(v -> nativeShowKeyboard());
+        keyboardButton.setOnTouchListener(CLICK_ON_TOUCH_UP);
+        FrameLayout.LayoutParams lp = new FrameLayout.LayoutParams(300, 100);
+        lp.gravity = Gravity.CENTER;
+        panel.addView(keyboardButton, lp);
+        return panel;
+    }
+
+    // Opens the keyboard overlay quad (questshock_native.c ->
+    // xr_overlay_set_visible(xr_session_get_keyboard_overlay(), true)) -
+    // see KeyboardOverlay for the panel itself and its own Close button,
+    // the only way to hide it again.
+    private static native void nativeShowKeyboard();
+
+    // Called from xr_overlay.c's xr_overlay_init_jni(), on the render
+    // thread, right after it resolves this class via the activity's own
+    // ClassLoader (plain FindClass() can't see app classes from a thread
+    // that was attached to the JVM rather than spawned from Java).
     public static void nativeInit(Activity activity) {
         if (instance == null) {
             instance = new MenuOverlay(activity);
         }
     }
 
-    // Polled once per frame while the menu quad is visible (see
-    // xr_menu_render_if_visible()) - returns the current pixels only once
-    // per redraw (null otherwise), so the native side knows when it can
-    // skip re-uploading a texture that hasn't actually changed.
     public static byte[] nativeTakePixelsIfDirty() {
-        if (instance == null) {
+        return instance == null ? null : instance.takePixelsIfDirty();
-            return null;
-        }
-        synchronized (instance.pixelLock) {
-            byte[] pixels = instance.pendingPixels;
-            instance.pendingPixels = null;
-            return pixels;
-        }
     }
 
-    // u/v are the menu quad's own hit-test coordinates (0..1, top-left
-    // origin) - computed by xr_input.c's ray/quad intersection against
-    // xr_menu.c's reported quad geometry, forwarded here as a synthetic
-    // tap. Runs on the UI thread since the View/Bitmap/Canvas objects here
-    // are otherwise only ever touched from there.
     public static void nativeDispatchTouch(final float u, final float v, final boolean down) {
-        if (instance == null) {
+        if (instance != null) {
-            return;
+            instance.dispatchTouch(u, v, down);
         }
-        instance.activity.runOnUiThread(() -> instance.handleTouch(u, v, down));
     }
 
-    private void handleTouch(float u, float v, boolean down) {
+    public static void nativeUpdateCursor(final float u, final float v, final boolean visible) {
-        float x = u * WIDTH;
+        if (instance != null) {
-        float y = v * HEIGHT;
+            instance.dispatchUpdateCursor(u, v, visible);
-        long time = SystemClock.uptimeMillis();
-        MotionEvent event = MotionEvent.obtain(
-            time, time, down ? MotionEvent.ACTION_DOWN : MotionEvent.ACTION_UP, x, y, 0);
-        try {
-            root.dispatchTouchEvent(event);
-        } finally {
-            event.recycle();
-        }
-        forceRedraw();
-    }
-
-    private void forceRedraw() {
-        canvas.drawColor(0xFF202020);
-        root.draw(canvas);
-
-        byte[] pixels = new byte[WIDTH * HEIGHT * 4];
-        // ARGB_8888's actual in-memory byte order is R,G,B,A - matches
-        // GL_RGBA/GL_UNSIGNED_BYTE on the native side with no swizzling.
-        bitmap.copyPixelsToBuffer(ByteBuffer.wrap(pixels));
-        synchronized (pixelLock) {
-            pendingPixels = pixels;
         }
     }
 }

android/app/src/main/java/de/ladkau/questshock/OverlayPanel.java

@@ -0,0 +1,237 @@
+package de.ladkau.questshock;
+
+import android.app.Activity;
+import android.graphics.Bitmap;
+import android.graphics.Canvas;
+import android.graphics.Paint;
+import android.os.SystemClock;
+import android.view.MotionEvent;
+import android.view.View;
+import java.nio.ByteBuffer;
+
+/**
+ * Shared machinery for an off-screen, never-attached-to-a-window View tree
+ * rendered into its own OpenXR quad (see android/app/src/main/cpp/
+ * xr_overlay.c, which owns the quad's swapchain and drives whichever
+ * concrete subclass via JNI - construction, touch input, and pixel
+ * readback). Rendering to a Bitmap and dispatching synthetic MotionEvents
+ * into an unattached hierarchy both work the same way they would for an
+ * attached View - draw(Canvas)/dispatchTouchEvent() don't require a
+ * ViewRootImpl/window, just a measured+laid-out tree.
+ *
+ * Subclasses (MenuOverlay, KeyboardOverlay) provide their own content via
+ * buildContent() and, if they want translucency, contentAlpha() - and, since
+ * Java has no static virtual dispatch and each overlay needs its own
+ * singleton, each subclass still declares its own thin nativeInit()/
+ * nativeTakePixelsIfDirty()/nativeDispatchTouch()/nativeUpdateCursor()
+ * static methods (matching what xr_overlay.c's JNI glue resolves by class
+ * name) that just delegate into the instance methods here
+ * (takePixelsIfDirty()/dispatchTouch()/dispatchUpdateCursor()).
+ */
+abstract class OverlayPanel {
+    private static final float CURSOR_RADIUS = 10f;
+
+    // Button's own click handling (View.onTouchEvent()'s ACTION_UP case)
+    // calls View.post(mPerformClick) rather than invoking performClick()
+    // directly; post() queues the runnable to run once the view is attached
+    // to a window and returns true immediately even when unattached, so on
+    // this permanently-unattached tree that queued click silently never
+    // fires - dispatchTouchEvent() still delivers the down/up events
+    // correctly, only the click callback is swallowed. Registering this as
+    // each button's OnTouchListener bypasses that path entirely: returning
+    // true here skips View's internal onTouchEvent() (see
+    // ViewGroup/View#dispatchTouchEvent), so performClick() (which still
+    // runs any OnClickListener set via setOnClickListener()) is called
+    // directly instead.
+    protected static final View.OnTouchListener CLICK_ON_TOUCH_UP = (v, event) -> {
+        switch (event.getAction()) {
+            case MotionEvent.ACTION_DOWN:
+                v.setPressed(true);
+                break;
+            case MotionEvent.ACTION_UP:
+                v.setPressed(false);
+                v.performClick();
+                break;
+            case MotionEvent.ACTION_CANCEL:
+                v.setPressed(false);
+                break;
+        }
+        return true;
+    };
+
+    protected final Activity activity;
+    private final int width;
+    private final int height;
+    private final View root;
+
+    // contentBitmap holds just the panel's own rendered content (no cursor),
+    // redrawn only when that content actually changes (redrawContent(), via
+    // handleTouch() or a subclass's own key handling) via root.draw() - a
+    // full off-screen View-tree traversal that gets noticeably more
+    // expensive on a busier content tree. bitmap is what's actually
+    // published to native (see takePixelsIfDirty()): compositeAndPublish()
+    // cheaply blits contentBitmap plus the cursor circle into it. Splitting
+    // these two apart matters because updateCursor() below fires every
+    // single frame (~90Hz) while aiming at the panel - if it called the
+    // expensive root.draw() path every time (as an earlier version did), a
+    // busy content tree made each redraw slow enough that runOnUiThread()
+    // posts piled up faster than the UI thread could drain them, so the
+    // cursor visibly lagged behind the controller's actual aim instead of
+    // tracking it.
+    private final Bitmap contentBitmap;
+    private final Canvas contentCanvas;
+    private final Bitmap bitmap;
+    private final Canvas canvas;
+    private final Paint cursorPaint;
+    // Draws contentBitmap at contentAlpha() - see compositeAndPublish().
+    private final Paint contentAlphaPaint;
+
+    private final Object pixelLock = new Object();
+    private byte[] pendingPixels;
+
+    // Only ever touched on the UI thread (both dispatchUpdateCursor() and
+    // compositeAndPublish() run/are posted there) - no lock needed, unlike
+    // pendingPixels above.
+    private boolean cursorVisible = false;
+    private float cursorX = 0f;
+    private float cursorY = 0f;
+
+    protected OverlayPanel(Activity activity, int width, int height) {
+        this.activity = activity;
+        this.width = width;
+        this.height = height;
+
+        root = buildContent();
+        int widthSpec = View.MeasureSpec.makeMeasureSpec(width, View.MeasureSpec.EXACTLY);
+        int heightSpec = View.MeasureSpec.makeMeasureSpec(height, View.MeasureSpec.EXACTLY);
+        root.measure(widthSpec, heightSpec);
+        root.layout(0, 0, width, height);
+
+        contentBitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
+        contentCanvas = new Canvas(contentBitmap);
+        bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
+        canvas = new Canvas(bitmap);
+
+        cursorPaint = new Paint();
+        cursorPaint.setColor(0xFFFFFFFF);
+        cursorPaint.setAntiAlias(true);
+
+        // contentAlpha() must not depend on any subclass instance state -
+        // it's called here, from the superclass constructor, before the
+        // subclass's own field initializers/constructor body have run (the
+        // usual Java construction order: this class's fields/constructor
+        // first, then the subclass's). A static constant is the only thing
+        // that's safe to return.
+        contentAlphaPaint = new Paint();
+        contentAlphaPaint.setAlpha(contentAlpha());
+
+        redrawContent();
+    }
+
+    // Builds this overlay's content View tree - called once, from this
+    // class's own constructor (see the note on contentAlpha() above: don't
+    // rely on subclass instance fields being initialized yet here; if a
+    // subclass needs its own mutable state available while building its
+    // content, e.g. a list of buttons to later relabel, initialize that
+    // field's value at the top of this method instead of via a field
+    // initializer).
+    protected abstract View buildContent();
+
+    // 0-255; defaults to fully opaque. Override for translucency (see the
+    // constructor's note on what's safe to depend on here).
+    protected int contentAlpha() {
+        return 255;
+    }
+
+    // Polled once per frame while this overlay's quad is visible (see
+    // xr_overlay_render_and_build_layer()) - returns the current pixels
+    // only once per redraw (null otherwise), so the native side knows when
+    // it can skip re-uploading a texture that hasn't actually changed.
+    protected final byte[] takePixelsIfDirty() {
+        synchronized (pixelLock) {
+            byte[] pixels = pendingPixels;
+            pendingPixels = null;
+            return pixels;
+        }
+    }
+
+    // u/v are this quad's own hit-test coordinates (0..1, top-left origin)
+    // - computed by xr_input.c's ray/quad intersection against xr_overlay.c's
+    // reported quad geometry, forwarded here as a synthetic tap. Runs on the
+    // UI thread since the View/Bitmap/Canvas objects here are otherwise only
+    // ever touched from there.
+    protected final void dispatchTouch(final float u, final float v, final boolean down) {
+        activity.runOnUiThread(() -> handleTouch(u, v, down));
+    }
+
+    // u/v are the same hit-test coordinates dispatchTouch() uses, but
+    // polled once per frame regardless of click state rather than only on
+    // click edges - lets the cursor track the aim ray continuously instead
+    // of only jumping when a trigger is pressed. visible=false (no hand's
+    // ray currently on the quad) hides it.
+    protected final void dispatchUpdateCursor(final float u, final float v,
+                                               final boolean visible) {
+        activity.runOnUiThread(() -> updateCursor(u, v, visible));
+    }
+
+    private void updateCursor(float u, float v, boolean visible) {
+        float x = u * width;
+        float y = v * height;
+        if (visible == cursorVisible && x == cursorX && y == cursorY) {
+            return;
+        }
+        cursorVisible = visible;
+        cursorX = x;
+        cursorY = y;
+        compositeAndPublish();
+    }
+
+    private void handleTouch(float u, float v, boolean down) {
+        float x = u * width;
+        float y = v * height;
+        long time = SystemClock.uptimeMillis();
+        MotionEvent event = MotionEvent.obtain(
+            time, time, down ? MotionEvent.ACTION_DOWN : MotionEvent.ACTION_UP, x, y, 0);
+        try {
+            root.dispatchTouchEvent(event);
+        } finally {
+            event.recycle();
+        }
+        redrawContent();
+    }
+
+    // Re-runs the full off-screen View-tree draw (expensive - see the
+    // contentBitmap field comment above) - call only when the panel's
+    // actual content changed, not for the cursor-only updates
+    // compositeAndPublish() below handles on its own.
+    protected final void redrawContent() {
+        contentCanvas.drawColor(0xFF202020);
+        root.draw(contentCanvas);
+        compositeAndPublish();
+    }
+
+    // Cheap per-frame path: blits the last-rendered contentBitmap (no View
+    // traversal) plus the cursor circle into bitmap and publishes it.
+    // contentBitmap itself is fully opaque (redrawContent() draws an opaque
+    // background first); contentAlpha() translucency is applied right here
+    // instead, via contentAlphaPaint - which needs bitmap cleared to fully
+    // transparent first, since otherwise each frame's alpha-blended draw
+    // would blend against whatever was left over in bitmap from the
+    // previous frame rather than against nothing, compounding into a
+    // ghosting trail across frames.
+    private void compositeAndPublish() {
+        bitmap.eraseColor(0);
+        canvas.drawBitmap(contentBitmap, 0, 0, contentAlphaPaint);
+        if (cursorVisible) {
+            canvas.drawCircle(cursorX, cursorY, CURSOR_RADIUS, cursorPaint);
+        }
+
+        byte[] pixels = new byte[width * height * 4];
+        // ARGB_8888's actual in-memory byte order is R,G,B,A - matches
+        // GL_RGBA/GL_UNSIGNED_BYTE on the native side with no swizzling.
+        bitmap.copyPixelsToBuffer(ByteBuffer.wrap(pixels));
+        synchronized (pixelLock) {
+            pendingPixels = pixels;
+        }
+    }
+}