Split the keyboard onto its own translucent OpenXR quad

Expand the on-screen keyboard to a full US layout (letters, digits,
punctuation, Shift layer, Tab, arrows, F1-F12, one-shot Ctrl/Alt
modifiers), make it a persistent, movable, closeable panel via a
title-bar drag handle, and lower its opacity.

Previously the keyboard was just a second panel swapped into the same
quad as the menu launcher, which made the launcher translucent too and
made the keyboard and menu mutually exclusive. Extract the shared
swapchain/JNI/touch-dispatch plumbing into two generic, reusable
modules - xr_swapchain.c (swapchain + per-image FBO setup) and
xr_overlay.c (an off-screen Android View rendered into its own OpenXR
quad, hit-tested via a laser pointer) - and make the menu launcher and
keyboard two independent XrOverlay instances instead of one. The
controller's menu button now only opens/closes the launcher; the
launcher's "Keyboard" button only opens the keyboard - so the keyboard
can stay up while picking something from the menu, and only the
keyboard's own Close button hides it.

xr_menu.c/.h are gone, fully absorbed into xr_overlay.c. On the Java
side, OverlayPanel.java carries the shared off-screen-render/touch/
cursor plumbing that MenuOverlay and the new KeyboardOverlay both
subclass.

README.md

@@ -144,17 +144,47 @@ the viewer, with a separate menu quad toggled by a controller button and
 driven by a laser-pointer-style aim ray from each hand
 (`android/app/src/main/cpp/xr_input.c`) - point and pull the trigger to
 interact with it, same as the game's own Bluetooth mouse/keyboard input
-otherwise works unchanged. The menu's "Keyboard" button swaps to a
+otherwise works unchanged.
-hand-built on-screen key grid (`MenuOverlay.java` - there's no system IME
+
-to borrow once immersive) that forwards each key press straight to the
+The menu launcher (`MenuOverlay.java` - just a "Keyboard" button so far)
-game as real input - non-printable keys (Esc/Enter/Backspace) as a
+and the keyboard (`KeyboardOverlay.java`) are two fully independent quads,
-`SDLActivity.onNativeKeyDown()`/`onNativeKeyUp()` pair, the same one a
+each its own `XrOverlay` instance (`android/app/src/main/cpp/xr_overlay.c`
-physical Bluetooth keyboard's presses already go through, and printable
+- a generic "off-screen Android View rendered into an OpenXR quad,
-keys (letters, Space) via a synthesized `SDL_TEXTINPUT` event pushed
+hit-tested via a laser pointer" module shared by both, and by whatever
-directly from native code (`questshock_native.c`) - meant as a general
+similar panel comes next) with its own swapchain, visibility, and
-stand-in for keyboard-driven functionality that isn't (yet, or ever)
+position, so they can be shown/hidden/moved independently - a design
-mapped onto the controllers, not just a future config screen's text
+specifically meant to let the keyboard stay open while also picking
-entry. The laser
+something from the (still-to-be-built-out) menu. The controller's menu
+button only ever opens/closes the launcher; clicking its "Keyboard" button
+only opens the keyboard (never touching the launcher's own visibility) -
+the keyboard's own title bar holds the only way to close it again.
+
+The keyboard is a hand-built, full US-layout key grid (there's no system
+IME to borrow once immersive) that forwards each key press straight to the
+game as real input - non-printable keys (Esc/Enter/Backspace/Tab/arrows/
+F1-F12) as a `SDLActivity.onNativeKeyDown()`/`onNativeKeyUp()` pair, the
+same one a physical Bluetooth keyboard's presses already go through, and
+printable keys (letters, digits, punctuation, Space) via a synthesized
+`SDL_TEXTINPUT` event pushed directly from native code
+(`questshock_native.c`). A Shift key toggles the whole grid between
+lowercase/numbers and uppercase/symbols (doubling as Caps Lock - it stays
+toggled until pressed again), and Ctrl/Alt arm as one-shot modifiers,
+consumed by whichever key is pressed next - meant as a general stand-in
+for keyboard-driven functionality that isn't (yet, or ever) mapped onto
+the controllers, not just a future config screen's text entry.
+
+Once opened, the keyboard stays up as a standing input panel while
+actually playing rather than a modal you open and close. Its title bar
+doubles as a drag handle (grab-and-drag with the trigger, handled entirely
+on the native side in `xr_input.c` before it ever reaches
+`KeyboardOverlay`'s own touch dispatch) for repositioning it, and holds
+the Close button that hides it. It's also rendered semi-transparent
+(`XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT` in `xr_session.c`,
+set only on the keyboard's layer - the menu launcher stays fully opaque)
+so whatever's behind it - the game quad, mid-play - stays visible while
+it's up.
+
+The laser
 pointer/cursor is currently only visible while
 actually aiming at the game or menu quad respectively - there's no visual
 feedback yet while aiming at empty space between them. (Only tested on
@@ -321,8 +351,9 @@ each one does:
 
 #### 5.5.1. The menu quad rendering the game instead of itself
 
-While building the OpenXR menu (`android/app/src/main/cpp/xr_menu.c`,
+While building the OpenXR menu (`android/app/src/main/cpp/xr_overlay.c` -
-`MenuOverlay.java`), the menu's composition-layer quad consistently showed
+`xr_menu.c` at the time - and `MenuOverlay.java`), the menu's
+composition-layer quad consistently showed
 the game's own live rendering instead of the menu's content, even though
 every diagnostic (FBO bindings, swapchain/layer submission, texture
 upload, viewport/scissor state) checked out correct in isolation.

android/app/build.gradle

@@ -171,7 +171,7 @@ android {
                     "-DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=BOTH", \
                     "-DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=BOTH", \
                     "-DCMAKE_SHARED_LINKER_FLAGS=-Wl,-z,max-page-size=16384", \
-                    "-DANDROID_EXTRA_SOURCES=${projectDir}/src/main/cpp/questshock_native.c;${projectDir}/src/main/cpp/xr_session.c;${projectDir}/src/main/cpp/xr_input.c;${projectDir}/src/main/cpp/xr_menu.c"
+                    "-DANDROID_EXTRA_SOURCES=${projectDir}/src/main/cpp/questshock_native.c;${projectDir}/src/main/cpp/xr_session.c;${projectDir}/src/main/cpp/xr_input.c;${projectDir}/src/main/cpp/xr_overlay.c;${projectDir}/src/main/cpp/xr_swapchain.c"
                 abiFilters 'arm64-v8a'
             }
         }

android/app/src/main/cpp/questshock_native.c

@@ -9,6 +9,9 @@
 
 #include <SDL.h>
 
+#include "xr_overlay.h"
+#include "xr_session.h"
+
 JNIEXPORT void JNICALL
 Java_de_ladkau_questshock_QuestShockActivity_nativeChdir(JNIEnv *env, jclass clazz, jstring path) {
     const char *cpath = (*env)->GetStringUTFChars(env, path, NULL);
@@ -16,17 +19,18 @@ Java_de_ladkau_questshock_QuestShockActivity_nativeChdir(JNIEnv *env, jclass cla
     (*env)->ReleaseStringUTFChars(env, path, cpath);
 }
 
-// MenuOverlay's hand-built on-screen keyboard (see
+// KeyboardOverlay's hand-built on-screen keyboard (see
-// MenuOverlay.commitPrintableChar()) has no real Android IME session behind
+// KeyboardOverlay.commitPrintableChar()) has no real Android IME session
-// it, so printable characters can't reach the game via Android's usual IME
+// behind it, so printable characters can't reach the game via Android's
-// path - instead this builds and pushes an SDL_TEXTINPUT event directly,
+// usual IME path - instead this builds and pushes an SDL_TEXTINPUT event
-// the same event type/shape SDL's own Android backend pushes for typed
+// directly, the same event type/shape SDL's own Android backend pushes for
-// text, using only the public SDL_Event/SDL_PushEvent API. The engine's
+// typed text, using only the public SDL_Event/SDL_PushEvent API. The
-// pump_events() (sdl_events.c) only picks up printable characters from
+// engine's pump_events() (sdl_events.c) only picks up printable characters
-// this event, not from SDL_KEYDOWN (see handleKeyPress() in MenuOverlay.java
+// from this event, not from SDL_KEYDOWN (see handleKeyPress() in
-// for why).
+// KeyboardOverlay.java for why).
 JNIEXPORT void JNICALL
-Java_de_ladkau_questshock_MenuOverlay_nativeSendPrintableChar(JNIEnv *env, jclass clazz, jchar c) {
+Java_de_ladkau_questshock_KeyboardOverlay_nativeSendPrintableChar(JNIEnv *env, jclass clazz,
+                                                                   jchar c) {
     SDL_Event event;
     SDL_zero(event);
     event.type = SDL_TEXTINPUT;
@@ -36,3 +40,20 @@ Java_de_ladkau_questshock_MenuOverlay_nativeSendPrintableChar(JNIEnv *env, jclas
     event.text.text[1] = '\0';
     SDL_PushEvent(&event);
 }
+
+// KeyboardOverlay's title-bar Close button - the keyboard is a fully
+// independent overlay quad from the menu launcher (see xr_overlay.h), so
+// this is the only way to hide it once it's open; the controller's menu
+// button only ever affects the menu launcher.
+JNIEXPORT void JNICALL
+Java_de_ladkau_questshock_KeyboardOverlay_nativeRequestClose(JNIEnv *env, jclass clazz) {
+    xr_overlay_set_visible(xr_session_get_keyboard_overlay(), false);
+}
+
+// MenuOverlay's "Keyboard" button - only opens the keyboard overlay, never
+// touches the menu launcher's own visibility, so both can be shown
+// together.
+JNIEXPORT void JNICALL
+Java_de_ladkau_questshock_MenuOverlay_nativeShowKeyboard(JNIEnv *env, jclass clazz) {
+    xr_overlay_set_visible(xr_session_get_keyboard_overlay(), true);
+}

android/app/src/main/cpp/xr_input.c

@@ -7,7 +7,7 @@
 
 #include <GLES3/gl3.h>
 
-#include "xr_menu.h"
+#include "xr_overlay.h"
 #include "xr_session.h"
 
 #define TAG "QuestShock"
@@ -17,6 +17,24 @@
 #define LEFT 0
 #define RIGHT 1
 
+static const char *kHandName[2] = {"left", "right"};
+static const float kHandColor[2][4] = {
+    {0.2f, 1.0f, 1.0f, 1.0f}, // left: cyan
+    {1.0f, 0.85f, 0.1f, 1.0f}, // right: amber
+};
+
+// The keyboard overlay's title bar (drag handle + Close button) is handled
+// entirely here, before a hit would normally be dispatched to
+// KeyboardOverlay as a touch - these fractions must agree with
+// KeyboardOverlay.java's TITLE_BAR_HEIGHT/CLOSE_BUTTON_WIDTH constants (in
+// pixels, out of KeyboardOverlay.WIDTH/HEIGHT there). A hit in the top
+// TITLE_BAR_V_FRACTION of the quad, at or left of CLOSE_BUTTON_U_FRACTION,
+// is the drag handle; right of that (still within the title bar) is the
+// Close button, which is dispatched as an ordinary click instead. The menu
+// launcher overlay has no title bar - it's a plain click-only panel.
+#define TITLE_BAR_V_FRACTION (90.0f / 768.0f)
+#define CLOSE_BUTTON_U_FRACTION (1.0f - 160.0f / 1024.0f)
+
 static XrInstance g_instance = XR_NULL_HANDLE;
 static XrSession g_session = XR_NULL_HANDLE;
 static XrActionSet g_action_set = XR_NULL_HANDLE;
@@ -27,14 +45,29 @@ static XrPath g_hand_path[2];
 static XrSpace g_aim_space[2] = {XR_NULL_HANDLE, XR_NULL_HANDLE};
 
 // Edge-detection state, so touch dispatch and logging only fire on actual
-// state changes, not every frame.
+// state changes, not every frame. Each interaction target (game quad, menu
+// launcher, keyboard) tracks its own "was this hand's ray hitting it last
+// frame" independently, since the menu and keyboard overlays can now both
+// be visible at once.
 static bool g_prev_select[2] = {false, false};
 static bool g_prev_menu = false;
-static bool g_prev_hit[2] = {false, false};
+static bool g_game_prev_hit[2] = {false, false};
-// Tracks which hand currently has an in-flight synthetic touch down on the
+static bool g_menu_prev_hit[2] = {false, false};
-// menu (so a later trigger-up edge only dispatches a matching touch-up if
-// a touch-down was actually sent for that hand).
 static bool g_menu_touch_active[2] = {false, false};
+static bool g_keyboard_prev_hit[2] = {false, false};
+static bool g_keyboard_touch_active[2] = {false, false};
+
+// Drag state for repositioning the keyboard overlay via its title bar (see
+// TITLE_BAR_V_FRACTION/CLOSE_BUTTON_U_FRACTION above) - the menu launcher
+// has no title bar, so it's never draggable. g_keyboard_drag_hand is -1
+// when g_keyboard_dragging is false. The offset is the vector from the
+// quad's center to the grabbed point at drag-start, kept constant for the
+// whole gesture so the quad follows the hand rigidly rather than
+// re-centering under the ray each frame.
+static bool g_keyboard_dragging = false;
+static int g_keyboard_drag_hand = -1;
+static float g_keyboard_drag_offset_x = 0.0f;
+static float g_keyboard_drag_offset_y = 0.0f;
 
 static GLuint g_reticle_program = 0;
 static GLint g_reticle_color_loc = -1;
@@ -220,6 +253,106 @@ bool xr_input_init(XrInstance instance, XrSession session) {
     return true;
 }
 
+// Hit-tests/dispatches this hand's ray against one overlay quad - shared
+// by the menu launcher and keyboard overlays in xr_input_sync_and_draw()
+// below. dragCapable enables the keyboard-only title-bar/drag-handle
+// handling (see TITLE_BAR_V_FRACTION/CLOSE_BUTTON_U_FRACTION); the menu
+// launcher has no title bar, so it's always false there and every hit is a
+// plain click. Returns true if this overlay is visible - claiming the
+// hand's ray processing for this frame, regardless of whether the ray
+// actually hits it - so the caller should stop trying other targets (menu/
+// keyboard/game quad are mutually exclusive per hand, per frame).
+static bool xr_input_try_overlay(XrOverlay *overlay, bool dragCapable, int hand,
+                                  const XrSpaceLocation *location, float fx, float fy, float fz,
+                                  bool selectDownEdge, bool selectUpEdge, bool *touchActive,
+                                  bool *prevHit, const char *quadName, float *outCursorU,
+                                  float *outCursorV, bool *outCursorHit) {
+    if (!xr_overlay_is_visible(overlay))
+        return false;
+
+    float quadCenterX, quadCenterY, distance, halfWidth, halfHeight;
+    xr_overlay_get_quad_extent(overlay, &quadCenterX, &quadCenterY, &distance, &halfWidth,
+                                &halfHeight);
+
+    // The quad's plane is z = -distance (facing the viewer along -Z) - a
+    // ray parallel to it (fz ~ 0) never crosses. worldX/Y is the raw
+    // world-space intersection point, needed as-is for dragging (which can
+    // legitimately move the quad to where the ray isn't currently hitting
+    // it); cx/cy are that same point in the quad's own [-1,1] local space;
+    // hit/u/v (the equivalent 0..1, top-left-origin coordinates the Java
+    // view's pixel grid and the on-quad hit logs use) are only meaningful
+    // within the quad's current bounds, unlike planeHit.
+    bool planeHit = false, hit = false;
+    float u = 0.0f, v = 0.0f, worldX = 0.0f, worldY = 0.0f;
+    if (fabsf(fz) > 1e-5f) {
+        float t = (-distance - location->pose.position.z) / fz;
+        if (t > 0.0f) {
+            planeHit = true;
+            worldX = location->pose.position.x + t * fx;
+            worldY = location->pose.position.y + t * fy;
+            float cx = (worldX - quadCenterX) / halfWidth;
+            float cy = (worldY - quadCenterY) / halfHeight;
+            hit = fabsf(cx) <= 1.0f && fabsf(cy) <= 1.0f;
+            u = (cx + 1.0f) * 0.5f;
+            v = (1.0f - cy) * 0.5f;
+        }
+    }
+
+    if (hit != *prevHit) {
+        LOGI("XR: %s aim ray %s %s quad (u=%.2f v=%.2f)", kHandName[hand],
+             hit ? "entered" : "left", quadName, u, v);
+        *prevHit = hit;
+    }
+    if (hit) {
+        *outCursorHit = true;
+        *outCursorU = u;
+        *outCursorV = v;
+    }
+
+    if (!dragCapable) {
+        if (selectDownEdge && hit) {
+            xr_overlay_touch(overlay, u, v, true);
+            *touchActive = true;
+        } else if (selectUpEdge && *touchActive) {
+            xr_overlay_touch(overlay, u, v, false);
+            *touchActive = false;
+        }
+        return true;
+    }
+
+    bool onTitleBar = hit && v < TITLE_BAR_V_FRACTION;
+    bool onDragHandle = onTitleBar && u <= CLOSE_BUTTON_U_FRACTION;
+
+    if (selectDownEdge) {
+        if (onDragHandle) {
+            g_keyboard_dragging = true;
+            g_keyboard_drag_hand = hand;
+            g_keyboard_drag_offset_x = worldX - quadCenterX;
+            g_keyboard_drag_offset_y = worldY - quadCenterY;
+        } else if (hit) {
+            xr_overlay_touch(overlay, u, v, true);
+            *touchActive = true;
+        }
+    } else if (selectUpEdge) {
+        if (g_keyboard_dragging && g_keyboard_drag_hand == hand) {
+            g_keyboard_dragging = false;
+            g_keyboard_drag_hand = -1;
+        } else if (*touchActive) {
+            xr_overlay_touch(overlay, u, v, false);
+            *touchActive = false;
+        }
+    }
+
+    // Re-intersects against the same fixed-distance plane every frame the
+    // drag continues, so the quad tracks the hand in real time rather than
+    // only jumping on the down/up edges above.
+    if (g_keyboard_dragging && g_keyboard_drag_hand == hand && planeHit) {
+        xr_overlay_set_position(overlay, worldX - g_keyboard_drag_offset_x,
+                                 worldY - g_keyboard_drag_offset_y);
+    }
+    return true;
+}
+
 void xr_input_sync_and_draw(XrSpace baseSpace, XrTime time, bool draw) {
     if (g_action_set == XR_NULL_HANDLE)
         return;
@@ -230,36 +363,37 @@ void xr_input_sync_and_draw(XrSpace baseSpace, XrTime time, bool draw) {
     syncInfo.activeActionSets = &activeSet;
     xr_check(xrSyncActions(g_session, &syncInfo), "xrSyncActions");
 
-    bool menuVisible = xr_menu_is_visible();
+    XrOverlay *menuOverlay = xr_session_get_menu_overlay();
+    XrOverlay *keyboardOverlay = xr_session_get_keyboard_overlay();
+    bool menuVisible = xr_overlay_is_visible(menuOverlay);
+    bool keyboardVisible = xr_overlay_is_visible(keyboardOverlay);
 
-    float quadCenterX = 0.0f, quadCenterY = 0.0f, distance, halfWidth, halfHeight;
+    // A drag can't get stuck active across a hide/reopen - e.g. Close
-    if (menuVisible)
+    // being hit on one hand mid-drag on the other, or the session ending.
-        xr_menu_get_quad_extent(&quadCenterX, &quadCenterY, &distance, &halfWidth, &halfHeight);
+    if (!keyboardVisible) {
-    else
+        g_keyboard_dragging = false;
-        xr_get_game_quad_extent(&distance, &halfWidth, &halfHeight);
+        g_keyboard_drag_hand = -1;
-
+    }
-    static const char *kHandName[2] = {"left", "right"};
-    static const float kHandColor[2][4] = {
-        {0.2f, 1.0f, 1.0f, 1.0f}, // left: cyan
-        {1.0f, 0.85f, 0.1f, 1.0f}, // right: amber
-    };
 
     // The GL reticle is drawn directly into whatever framebuffer is
     // currently bound - the game quad's swapchain image (see xr_frame_end(),
     // which calls this while that image is still bound). That only makes
-    // sense while aiming at the game quad; the menu quad's own cursor is
+    // sense while aiming at the game quad; the menu/keyboard overlays' own
-    // drawn by MenuOverlay itself instead (see xr_menu_update_cursor()
+    // cursors are drawn by their Java views instead (see
-    // below), composited into its Bitmap the same way its other content is.
+    // xr_overlay_update_cursor() below), composited into their Bitmaps the
-    bool drawReticle = draw && !menuVisible;
+    // same way their other content is.
+    bool drawReticle = draw && !menuVisible && !keyboardVisible;
     if (drawReticle)
         glUseProgram(g_reticle_program);
 
-    // Fed to xr_menu_update_cursor() after the loop below - whichever hand's
+    // Fed to xr_overlay_update_cursor() after the loop below - whichever
-    // ray hits the menu quad last wins if both do, good enough for a single
+    // hand's ray hits a given overlay last wins if both do, good enough
-    // on-quad cursor (only ever updated when hit is true, so a later hand
+    // for a single on-quad cursor per overlay (only ever updated when hit
-    // that misses doesn't hide an earlier hand's hit).
+    // is true, so a later hand that misses doesn't hide an earlier hand's
-    bool menuCursorHit = false;
+    // hit).
+    bool menuCursorHit = false, keyboardCursorHit = false;
     float menuCursorU = 0.0f, menuCursorV = 0.0f;
+    float keyboardCursorU = 0.0f, keyboardCursorV = 0.0f;
 
     for (int hand = 0; hand < 2; hand++) {
         XrActionStateGetInfo selectInfo = {XR_TYPE_ACTION_STATE_GET_INFO};
@@ -284,13 +418,14 @@ void xr_input_sync_and_draw(XrSpace baseSpace, XrTime time, bool draw) {
             bool menuDown = menuState.isActive && menuState.currentState;
             if (menuDown && !g_prev_menu) {
                 LOGI("XR: menu-toggle button DOWN");
-                xr_menu_toggle_visible();
+                // Only ever affects the menu launcher - the keyboard is
-                // Avoid a stale hit/touch-active carrying over from
+                // fully independent (its own Close button is the only way
-                // whichever quad was being tested against before the
+                // to hide it, see KeyboardOverlay.java).
-                // toggle - the next frame re-evaluates against the new
+                if (menuOverlay != NULL) {
-                // target from a clean state.
+                    xr_overlay_toggle_visible(menuOverlay);
-                g_prev_hit[LEFT] = g_prev_hit[RIGHT] = false;
+                    g_menu_prev_hit[LEFT] = g_menu_prev_hit[RIGHT] = false;
                     g_menu_touch_active[LEFT] = g_menu_touch_active[RIGHT] = false;
+                }
             } else if (!menuDown && g_prev_menu) {
                 LOGI("XR: menu-toggle button UP");
             }
@@ -316,51 +451,40 @@ void xr_input_sync_and_draw(XrSpace baseSpace, XrTime time, bool draw) {
         float fx, fy, fz;
         quat_rotate_vec(&location.pose.orientation, 0.0f, 0.0f, -1.0f, &fx, &fy, &fz);
 
-        // The quad's plane is z = -distance (facing the viewer along -Z,
+        // Keyboard first (it's the more likely target while it's up), then
-        // see QUAD_DISTANCE_METERS/xr_menu.c's MENU_DISTANCE_METERS) - a
+        // the menu launcher, then - only if neither is visible - the game
-        // ray parallel to it (fz ~ 0) never crosses. cx/cy are the hit
+        // quad's own reticle below. A single ray only ever interacts with
-        // point in the quad's own [-1,1] local space (the same space
+        // one target per hand per frame.
-        // android_draw_surface_as_quad() draws its own vertex positions
+        if (xr_input_try_overlay(keyboardOverlay, true, hand, &location, fx, fy, fz,
-        // in); u/v are the equivalent 0..1, top-left-origin coordinates
+                                  selectDownEdge, selectUpEdge, &g_keyboard_touch_active[hand],
-        // MenuOverlay's pixel grid and the on-quad hit logs both use.
+                                  &g_keyboard_prev_hit[hand], "keyboard", &keyboardCursorU,
+                                  &keyboardCursorV, &keyboardCursorHit))
+            continue;
+        if (xr_input_try_overlay(menuOverlay, false, hand, &location, fx, fy, fz, selectDownEdge,
+                                  selectUpEdge, &g_menu_touch_active[hand], &g_menu_prev_hit[hand],
+                                  "menu", &menuCursorU, &menuCursorV, &menuCursorHit))
+            continue;
+
+        float distance, halfWidth, halfHeight;
+        xr_get_game_quad_extent(&distance, &halfWidth, &halfHeight);
+
         bool hit = false;
         float u = 0.0f, v = 0.0f, cx = 0.0f, cy = 0.0f;
         if (fabsf(fz) > 1e-5f) {
             float t = (-distance - location.pose.position.z) / fz;
             if (t > 0.0f) {
-                cx = (location.pose.position.x + t * fx - quadCenterX) / halfWidth;
+                cx = (location.pose.position.x + t * fx) / halfWidth;
-                cy = (location.pose.position.y + t * fy - quadCenterY) / halfHeight;
+                cy = (location.pose.position.y + t * fy) / halfHeight;
                 hit = fabsf(cx) <= 1.0f && fabsf(cy) <= 1.0f;
                 u = (cx + 1.0f) * 0.5f;
                 v = (1.0f - cy) * 0.5f;
             }
         }
 
-        if (menuVisible) {
+        if (hit != g_game_prev_hit[hand]) {
-            if (hit != g_prev_hit[hand]) {
-                LOGI("XR: %s aim ray %s menu quad (u=%.2f v=%.2f)", kHandName[hand],
-                     hit ? "entered" : "left", u, v);
-                g_prev_hit[hand] = hit;
-            }
-            if (hit) {
-                menuCursorHit = true;
-                menuCursorU = u;
-                menuCursorV = v;
-            }
-            if (selectDownEdge && hit) {
-                xr_menu_touch(u, v, true);
-                g_menu_touch_active[hand] = true;
-            } else if (selectUpEdge && g_menu_touch_active[hand]) {
-                xr_menu_touch(u, v, false);
-                g_menu_touch_active[hand] = false;
-            }
-            continue;
-        }
-
-        if (hit != g_prev_hit[hand]) {
             LOGI("XR: %s aim ray %s game quad (u=%.2f v=%.2f)", kHandName[hand],
                  hit ? "entered" : "left", u, v);
-            g_prev_hit[hand] = hit;
+            g_game_prev_hit[hand] = hit;
         }
 
         if (!hit || !drawReticle)
@@ -378,7 +502,10 @@ void xr_input_sync_and_draw(XrSpace baseSpace, XrTime time, bool draw) {
     }
 
     if (menuVisible)
-        xr_menu_update_cursor(menuCursorU, menuCursorV, menuCursorHit);
+        xr_overlay_update_cursor(menuOverlay, menuCursorU, menuCursorV, menuCursorHit);
+    if (keyboardVisible)
+        xr_overlay_update_cursor(keyboardOverlay, keyboardCursorU, keyboardCursorV,
+                                  keyboardCursorHit);
 }
 
 void xr_input_shutdown(void) {
@@ -401,7 +528,12 @@ void xr_input_shutdown(void) {
     g_instance = XR_NULL_HANDLE;
     g_session = XR_NULL_HANDLE;
     memset(g_prev_select, 0, sizeof(g_prev_select));
-    memset(g_prev_hit, 0, sizeof(g_prev_hit));
+    memset(g_game_prev_hit, 0, sizeof(g_game_prev_hit));
+    memset(g_menu_prev_hit, 0, sizeof(g_menu_prev_hit));
     memset(g_menu_touch_active, 0, sizeof(g_menu_touch_active));
+    memset(g_keyboard_prev_hit, 0, sizeof(g_keyboard_prev_hit));
+    memset(g_keyboard_touch_active, 0, sizeof(g_keyboard_touch_active));
     g_prev_menu = false;
+    g_keyboard_dragging = false;
+    g_keyboard_drag_hand = -1;
 }

android/app/src/main/cpp/xr_input.h

@@ -1,11 +1,13 @@
 // Controller input for questshock's immersive Quest build: one OpenXR
 // action set (aim pose + trigger click per hand, a menu-toggle button on
-// the left controller) plus ray/quad hit-testing. While the menu quad
+// the left controller) plus ray/quad hit-testing. While neither the menu
-// (xr_menu.c) is hidden, this tests against the game quad xr_session.c
+// launcher nor keyboard overlay (see xr_overlay.h, xr_session.c) is
-// submits and draws a small reticle where each hand's aim ray crosses it;
+// visible, this tests against the game quad xr_session.c submits and draws
-// while the menu is visible, it tests against the menu quad instead and
+// a small reticle where each hand's aim ray crosses it; while either
-// forwards trigger edges as synthetic touches (no reticle - the menu's
+// overlay is visible, it tests against that overlay's quad instead (the
-// own content comes from MenuOverlay's rendered Bitmap).
+// keyboard is tried first) and forwards trigger edges as synthetic touches
+// (no reticle - each overlay's own content comes from its Java view's
+// rendered Bitmap).
 #ifndef QUESTSHOCK_XR_INPUT_H
 #define QUESTSHOCK_XR_INPUT_H
 

android/app/src/main/cpp/xr_menu.c

@@ -1,374 +0,0 @@
-#include "xr_menu.h"
-
-#include <dlfcn.h>
-#include <stdlib.h>
-
-#include <android/log.h>
-#include <jni.h>
-
-#include <EGL/egl.h>
-#include <GLES3/gl3.h>
-
-#include <SDL.h>
-
-#define TAG "QuestShock"
-#define LOGE(...) __android_log_print(ANDROID_LOG_ERROR, TAG, __VA_ARGS__)
-#define LOGI(...) __android_log_print(ANDROID_LOG_INFO, TAG, __VA_ARGS__)
-
-#define MENU_WIDTH 1024
-#define MENU_HEIGHT 768
-
-// Below the game quad (QUAD_DISTANCE_METERS/QUAD_WIDTH_METERS in
-// xr_session.c, 2m out) so both are visible together without overlapping.
-#define MENU_DISTANCE_METERS 1.5f
-#define MENU_WIDTH_METERS 0.8f
-#define MENU_CENTER_X_METERS 0.0f
-#define MENU_CENTER_Y_METERS -0.45f
-
-static bool g_visible = false;
-
-static jclass g_menu_overlay_class = NULL;
-static jmethodID g_native_init_method = NULL;
-static jmethodID g_take_pixels_method = NULL;
-static jmethodID g_dispatch_touch_method = NULL;
-static jmethodID g_update_cursor_method = NULL;
-
-// The menu overlay's content lives in this texture, uploaded by
-// xr_menu_upload_source_texture_if_dirty() below. It's created through
-// gl4es (glGenTextures/glBindTexture/glTexImage2D), like any other texture
-// the engine itself creates, so ordinary glTexSubImage2D uploads against it
-// work the normal way.
-static GLuint g_source_texture = 0;
-// A framebuffer with g_source_texture as its only color attachment, used as
-// the read source for xr_menu_render_if_visible()'s glBlitFramebuffer()
-// call. Created via the real (non-gl4es) GLES entry points below: that blit
-// bypasses gl4es entirely (see xr_menu_render_if_visible() for why), so its
-// source framebuffer has to be a real GL object rather than one gl4es
-// tracks.
-static GLuint g_source_fbo = 0;
-
-typedef void (*PFNQSGETINTEGERV)(GLenum, GLint *);
-typedef void (*PFNQSGENFRAMEBUFFERS)(GLsizei, GLuint *);
-typedef void (*PFNQSDELETEFRAMEBUFFERS)(GLsizei, const GLuint *);
-typedef void (*PFNQSBINDFRAMEBUFFER)(GLenum, GLuint);
-typedef void (*PFNQSFRAMEBUFFERTEXTURE2D)(GLenum, GLenum, GLenum, GLuint, GLint);
-typedef void (*PFNQSBLITFRAMEBUFFER)(GLint, GLint, GLint, GLint, GLint, GLint, GLint, GLint,
-                                      GLbitfield, GLenum);
-
-// The menu's source framebuffer and its glBlitFramebuffer() copy into the
-// swapchain image go through the real GLES driver rather than gl4es: gl4es's
-// fixed-pipeline emulation unconditionally substitutes its own shader onto
-// any gl4es-routed draw call, which would silently replace the menu's
-// content with whatever the game itself last rendered (see
-// xr_menu_render_if_visible()). A framebuffer blit has no shader stage at
-// all, so going through the real driver for it sidesteps the problem
-// entirely.
-static PFNQSGETINTEGERV real_glGetIntegerv;
-static PFNQSGENFRAMEBUFFERS real_glGenFramebuffers;
-static PFNQSDELETEFRAMEBUFFERS real_glDeleteFramebuffers;
-static PFNQSBINDFRAMEBUFFER real_glBindFramebuffer;
-static PFNQSFRAMEBUFFERTEXTURE2D real_glFramebufferTexture2D;
-// glBlitFramebuffer is a GLES 3.0 addition; resolved via eglGetProcAddress
-// rather than dlsym, since Android's driver dispatch doesn't guarantee ES3+
-// symbols are dlsym-able by name from libGLESv2.so, unlike the GLES2-core
-// functions above.
-static PFNQSBLITFRAMEBUFFER real_glBlitFramebuffer;
-
-static bool xr_menu_load_real_gles(void) {
-    void *lib = dlopen("libGLESv2.so", RTLD_NOW | RTLD_LOCAL);
-    if (lib == NULL) {
-        LOGE("XR: menu dlopen(libGLESv2.so) failed: %s", dlerror());
-        return false;
-    }
-    real_glGetIntegerv = (PFNQSGETINTEGERV)dlsym(lib, "glGetIntegerv");
-    real_glGenFramebuffers = (PFNQSGENFRAMEBUFFERS)dlsym(lib, "glGenFramebuffers");
-    real_glDeleteFramebuffers = (PFNQSDELETEFRAMEBUFFERS)dlsym(lib, "glDeleteFramebuffers");
-    real_glBindFramebuffer = (PFNQSBINDFRAMEBUFFER)dlsym(lib, "glBindFramebuffer");
-    real_glFramebufferTexture2D = (PFNQSFRAMEBUFFERTEXTURE2D)dlsym(lib, "glFramebufferTexture2D");
-    real_glBlitFramebuffer = (PFNQSBLITFRAMEBUFFER)eglGetProcAddress("glBlitFramebuffer");
-    return real_glGetIntegerv && real_glGenFramebuffers && real_glDeleteFramebuffers &&
-           real_glBindFramebuffer && real_glFramebufferTexture2D && real_glBlitFramebuffer;
-}
-
-static uint32_t g_pixel_generation = 0;
-static uint32_t g_source_uploaded_generation = (uint32_t)-1;
-static uint8_t *g_pixel_cache = NULL; // MENU_WIDTH*MENU_HEIGHT*4 bytes
-
-void xr_menu_get_content_size(int *width, int *height) {
-    *width = MENU_WIDTH;
-    *height = MENU_HEIGHT;
-}
-
-// FindClass() from this thread (SDL's native thread, attached to the JVM
-// via AttachCurrentThread rather than spawned from Java) resolves against
-// the bootstrap classloader, which only knows framework classes - it can't
-// see de.ladkau.questshock.MenuOverlay at all. Routing through the
-// activity's own classloader is the standard, documented workaround.
-static jclass xr_menu_find_class(JNIEnv *env, jobject activity, const char *name) {
-    jclass activityClass = (*env)->GetObjectClass(env, activity);
-    jmethodID getClassLoader =
-        (*env)->GetMethodID(env, activityClass, "getClassLoader", "()Ljava/lang/ClassLoader;");
-    jobject classLoader = (*env)->CallObjectMethod(env, activity, getClassLoader);
-    jclass classLoaderClass = (*env)->FindClass(env, "java/lang/ClassLoader");
-    jmethodID loadClass = (*env)->GetMethodID(env, classLoaderClass, "loadClass",
-                                               "(Ljava/lang/String;)Ljava/lang/Class;");
-    jstring className = (*env)->NewStringUTF(env, name);
-    jclass result = (jclass)(*env)->CallObjectMethod(env, classLoader, loadClass, className);
-    (*env)->DeleteLocalRef(env, className);
-    return result;
-}
-
-static bool xr_menu_init_jni(void) {
-    JNIEnv *env = (JNIEnv *)SDL_AndroidGetJNIEnv();
-    jobject activity = (jobject)SDL_AndroidGetActivity();
-    if (env == NULL || activity == NULL) {
-        LOGE("XR: menu - no JNIEnv/Activity from SDL");
-        return false;
-    }
-
-    jclass localClass = xr_menu_find_class(env, activity, "de.ladkau.questshock.MenuOverlay");
-    if (localClass == NULL) {
-        LOGE("XR: menu - could not find MenuOverlay class");
-        return false;
-    }
-    g_menu_overlay_class = (jclass)(*env)->NewGlobalRef(env, localClass);
-
-    g_native_init_method =
-        (*env)->GetStaticMethodID(env, g_menu_overlay_class, "nativeInit", "(Landroid/app/Activity;)V");
-    g_take_pixels_method =
-        (*env)->GetStaticMethodID(env, g_menu_overlay_class, "nativeTakePixelsIfDirty", "()[B");
-    g_dispatch_touch_method =
-        (*env)->GetStaticMethodID(env, g_menu_overlay_class, "nativeDispatchTouch", "(FFZ)V");
-    g_update_cursor_method =
-        (*env)->GetStaticMethodID(env, g_menu_overlay_class, "nativeUpdateCursor", "(FFZ)V");
-    if (!g_native_init_method || !g_take_pixels_method || !g_dispatch_touch_method ||
-        !g_update_cursor_method) {
-        LOGE("XR: menu - could not resolve MenuOverlay JNI methods");
-        return false;
-    }
-
-    (*env)->CallStaticVoidMethod(env, g_menu_overlay_class, g_native_init_method, activity);
-    // CallStaticVoidMethod doesn't surface Java exceptions on its own - if
-    // MenuOverlay's constructor throws (it runs its View measure/layout/
-    // draw calls on this native render thread rather than the Android UI
-    // thread, a plausible crash vector), the exception would otherwise be
-    // left silently pending and corrupt whatever JNI call runs next.
-    if ((*env)->ExceptionCheck(env)) {
-        LOGE("XR: menu MenuOverlay.nativeInit() threw a pending Java exception:");
-        (*env)->ExceptionDescribe(env);
-        (*env)->ExceptionClear(env);
-        return false;
-    }
-    return true;
-}
-
-static bool xr_menu_create_source_texture(void) {
-    glGenTextures(1, &g_source_texture);
-    glBindTexture(GL_TEXTURE_2D, g_source_texture);
-    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
-    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
-    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
-    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
-    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, MENU_WIDTH, MENU_HEIGHT, 0, GL_RGBA, GL_UNSIGNED_BYTE,
-                 NULL);
-
-    GLint prevFbo = 0;
-    real_glGetIntegerv(GL_FRAMEBUFFER_BINDING, &prevFbo);
-    real_glGenFramebuffers(1, &g_source_fbo);
-    real_glBindFramebuffer(GL_FRAMEBUFFER, g_source_fbo);
-    real_glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D,
-                                 g_source_texture, 0);
-    real_glBindFramebuffer(GL_FRAMEBUFFER, (GLuint)prevFbo);
-
-    return g_source_texture != 0;
-}
-
-bool xr_menu_init(XrInstance instance, XrSession session) {
-    (void)instance;
-    (void)session;
-
-    if (!xr_menu_load_real_gles()) {
-        LOGE("XR: menu couldn't resolve real GLES functions via dlsym/eglGetProcAddress");
-        return false;
-    }
-    if (!xr_menu_create_source_texture())
-        return false;
-    if (!xr_menu_init_jni())
-        return false;
-
-    g_pixel_cache = (uint8_t *)malloc((size_t)MENU_WIDTH * MENU_HEIGHT * 4);
-    if (g_pixel_cache == NULL)
-        return false;
-
-    LOGI("XR: menu overlay ready (%dx%d)", MENU_WIDTH, MENU_HEIGHT);
-    return true;
-}
-
-void xr_menu_toggle_visible(void) {
-    g_visible = !g_visible;
-    LOGI("XR: menu quad now %s", g_visible ? "visible" : "hidden");
-}
-
-bool xr_menu_is_visible(void) { return g_visible; }
-
-void xr_menu_get_quad_extent(float *center_x_m, float *center_y_m, float *distance_m,
-                              float *half_width_m, float *half_height_m) {
-    *center_x_m = MENU_CENTER_X_METERS;
-    *center_y_m = MENU_CENTER_Y_METERS;
-    *distance_m = MENU_DISTANCE_METERS;
-    *half_width_m = MENU_WIDTH_METERS * 0.5f;
-    *half_height_m = MENU_WIDTH_METERS * 0.5f * (float)MENU_HEIGHT / (float)MENU_WIDTH;
-}
-
-void xr_menu_touch(float u, float v, bool down) {
-    if (g_menu_overlay_class == NULL)
-        return;
-    JNIEnv *env = (JNIEnv *)SDL_AndroidGetJNIEnv();
-    if (env == NULL)
-        return;
-    // Passed via CallStaticVoidMethodA/jvalue rather than the variadic
-    // Call*Method form - deliberately sidesteps relying on JNI
-    // implementations correctly un-doing C's float-to-double default
-    // argument promotion for varargs float parameters (a well-known JNI
-    // footgun; Android's ART handles it correctly, but there's no reason
-    // to depend on that when the jvalue form is unambiguous either way).
-    jvalue args[3];
-    args[0].f = u;
-    args[1].f = v;
-    args[2].z = (jboolean)down;
-    (*env)->CallStaticVoidMethodA(env, g_menu_overlay_class, g_dispatch_touch_method, args);
-}
-
-void xr_menu_update_cursor(float u, float v, bool visible) {
-    if (g_menu_overlay_class == NULL)
-        return;
-    JNIEnv *env = (JNIEnv *)SDL_AndroidGetJNIEnv();
-    if (env == NULL)
-        return;
-    // Same jvalue-form rationale as xr_menu_touch() above.
-    jvalue args[3];
-    args[0].f = u;
-    args[1].f = v;
-    args[2].z = (jboolean)visible;
-    (*env)->CallStaticVoidMethodA(env, g_menu_overlay_class, g_update_cursor_method, args);
-}
-
-// Pulls MenuOverlay's latest pixels (if it redrew since the last check)
-// into g_pixel_cache and bumps g_pixel_generation - called once per visible
-// frame, before deciding whether g_source_texture needs a fresh upload.
-static void xr_menu_refresh_pixel_cache(void) {
-    JNIEnv *env = (JNIEnv *)SDL_AndroidGetJNIEnv();
-    if (env == NULL)
-        return;
-    jbyteArray pixels =
-        (jbyteArray)(*env)->CallStaticObjectMethod(env, g_menu_overlay_class, g_take_pixels_method);
-    // CallStaticObjectMethod also doesn't surface Java exceptions on its
-    // own - if nativeTakePixelsIfDirty() itself throws, it would otherwise
-    // silently look identical to "nothing changed yet" (a null return).
-    if ((*env)->ExceptionCheck(env)) {
-        LOGE("XR: menu nativeTakePixelsIfDirty() threw a pending Java exception:");
-        (*env)->ExceptionDescribe(env);
-        (*env)->ExceptionClear(env);
-        return;
-    }
-    if (pixels == NULL)
-        return;
-    (*env)->GetByteArrayRegion(env, pixels, 0, MENU_WIDTH * MENU_HEIGHT * 4, (jbyte *)g_pixel_cache);
-    (*env)->DeleteLocalRef(env, pixels);
-    g_pixel_generation++;
-}
-
-// Uploads g_pixel_cache into g_source_texture via plain (gl4es-routed)
-// glBindTexture/glTexSubImage2D, matching how the texture was created.
-// Skipped when nothing changed since the last upload. Texture-unit-0 state
-// is saved and restored around the upload, since this call is gl4es-routed
-// and the engine's own next-frame rendering assumes nothing touched its
-// texture bindings since it last drew.
-static void xr_menu_upload_source_texture_if_dirty(void) {
-    if (g_source_uploaded_generation == g_pixel_generation)
-        return;
-
-    GLint prevActiveTexture = GL_TEXTURE0;
-    glGetIntegerv(GL_ACTIVE_TEXTURE, &prevActiveTexture);
-    glActiveTexture(GL_TEXTURE0);
-    GLint prevTexture2D = 0;
-    glGetIntegerv(GL_TEXTURE_BINDING_2D, &prevTexture2D);
-
-    glBindTexture(GL_TEXTURE_2D, g_source_texture);
-    glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, MENU_WIDTH, MENU_HEIGHT, GL_RGBA, GL_UNSIGNED_BYTE,
-                     g_pixel_cache);
-    g_source_uploaded_generation = g_pixel_generation;
-
-    glBindTexture(GL_TEXTURE_2D, (GLuint)prevTexture2D);
-    if (prevActiveTexture != GL_TEXTURE0)
-        glActiveTexture((GLenum)prevActiveTexture);
-}
-
-void xr_menu_render_if_visible(void) {
-    if (g_source_fbo == 0)
-        return;
-
-    xr_menu_refresh_pixel_cache();
-    xr_menu_upload_source_texture_if_dirty();
-
-    // Copies g_source_texture (via g_source_fbo) directly into the
-    // currently-bound framebuffer - the menu's own dedicated swapchain
-    // image, sized to exactly MENU_WIDTH x MENU_HEIGHT (see xr_session.c),
-    // so the destination is always the whole image - using the real GLES3
-    // hardware blit (glBlitFramebuffer) rather than a shader-based
-    // full-screen-quad draw. A blit has no vertex/fragment shading stage,
-    // no shader program, and no texture units involved at all, so gl4es's
-    // fixed-pipeline-emulation layer - which unconditionally substitutes a
-    // customized shader (reproducing the game's own last-bound
-    // texture/fixed-function state) onto any gl4es-routed draw call - has
-    // nothing to intercept here. It also never touches gl4es's own tracked
-    // program/vertex-array/texture-binding shadow state, so the only piece
-    // of state that needs save/restore is the READ framebuffer binding;
-    // binding only GL_READ_FRAMEBUFFER, rather than the combined
-    // GL_FRAMEBUFFER target, leaves the DRAW side (the swapchain image
-    // itself) untouched throughout.
-    //
-    // Y is flipped between source and destination: g_source_texture's texel
-    // row v=0 holds MenuOverlay's Bitmap row 0 (the top of the rendered
-    // content, standard top-row-first raster order), and reading that same
-    // texture via an attached FBO, framebuffer y=0 accesses that identical
-    // row. The destination swapchain framebuffer follows the opposite
-    // convention - its own y=0 is its bottom edge - so swapping dstY0/dstY1
-    // keeps the menu content right-side up (glBlitFramebuffer supports
-    // inverted src/dst rects for exactly this).
-    GLint prevReadFbo = 0;
-    real_glGetIntegerv(GL_READ_FRAMEBUFFER_BINDING, &prevReadFbo);
-
-    real_glBindFramebuffer(GL_READ_FRAMEBUFFER, g_source_fbo);
-    real_glBlitFramebuffer(0, 0, MENU_WIDTH, MENU_HEIGHT, 0, MENU_HEIGHT, MENU_WIDTH, 0,
-                            GL_COLOR_BUFFER_BIT, GL_LINEAR);
-    real_glBindFramebuffer(GL_READ_FRAMEBUFFER, (GLuint)prevReadFbo);
-}
-
-void xr_menu_shutdown(void) {
-    if (g_source_texture != 0)
-        glDeleteTextures(1, &g_source_texture);
-    g_source_texture = 0;
-    g_source_uploaded_generation = (uint32_t)-1;
-
-    if (g_source_fbo != 0 && real_glDeleteFramebuffers != NULL)
-        real_glDeleteFramebuffers(1, &g_source_fbo);
-    g_source_fbo = 0;
-
-    if (g_menu_overlay_class != NULL) {
-        JNIEnv *env = (JNIEnv *)SDL_AndroidGetJNIEnv();
-        if (env != NULL)
-            (*env)->DeleteGlobalRef(env, g_menu_overlay_class);
-    }
-    g_menu_overlay_class = NULL;
-    g_native_init_method = NULL;
-    g_take_pixels_method = NULL;
-    g_dispatch_touch_method = NULL;
-    g_update_cursor_method = NULL;
-
-    free(g_pixel_cache);
-    g_pixel_cache = NULL;
-    g_pixel_generation = 0;
-
-    g_visible = false;
-}

android/app/src/main/cpp/xr_menu.h

@@ -1,71 +0,0 @@
-// Menu quad for questshock's immersive Quest build: MenuOverlay.java's
-// off-screen-rendered View tree (a "Keyboard" button that swaps to a
-// hand-built on-screen key grid - there's no system IME to borrow once
-// immersive), toggled by xr_input.c's menu_toggle action and hit-tested by
-// the same ray xr_input.c already computes per hand.
-//
-// The menu has its own independent OpenXR swapchain, sized to exactly
-// xr_menu_get_content_size()'s dimensions (see xr_session.c, which creates
-// it and calls into this file to render its content each visible frame).
-#ifndef QUESTSHOCK_XR_MENU_H
-#define QUESTSHOCK_XR_MENU_H
-
-#include <stdbool.h>
-
-#define XR_USE_PLATFORM_ANDROID 1
-#define XR_USE_GRAPHICS_API_OPENGL_ES 1
-#include <openxr/openxr.h>
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-// Pixel size of the menu's own content region - xr_session.c needs this
-// before it creates the menu's own swapchain, to size it correctly. Has no
-// OpenXR/GL dependency, safe to call before xr_menu_init().
-void xr_menu_get_content_size(int *width, int *height);
-
-// Call once, after xr_session.c has created the menu's own swapchain - sets
-// up the MenuOverlay Java-side singleton and the gl4es-owned texture its
-// pixels get uploaded into. Returns false (logged, non-fatal - the caller
-// just has no menu) on failure.
-bool xr_menu_init(XrInstance instance, XrSession session);
-
-// Flips menu-quad visibility - called by xr_input.c on the menu_toggle
-// action's rising edge.
-void xr_menu_toggle_visible(void);
-bool xr_menu_is_visible(void);
-
-// Local-space pose/size of the menu quad, valid regardless of visibility -
-// shared with xr_input.c's ray/quad hit-testing, the same way
-// xr_get_game_quad_extent() (xr_session.h) is for the game quad.
-void xr_menu_get_quad_extent(float *center_x_m, float *center_y_m, float *distance_m,
-                              float *half_width_m, float *half_height_m);
-
-// Forwards a hit on the menu quad (in the quad's own 0..1 u/v, top-left
-// origin) to the MenuOverlay Java view as a synthetic touch down/up -
-// called by xr_input.c on the select_click action's edges, only while a
-// hand's aim ray currently hits the menu quad.
-void xr_menu_touch(float u, float v, bool down);
-
-// Updates the on-quad cursor MenuOverlay draws at the current aim hit point
-// (in the quad's own 0..1 u/v, top-left origin) - called once per frame
-// while xr_menu_is_visible(), regardless of click state, so the cursor
-// tracks the ray continuously rather than only jumping on click edges like
-// xr_menu_touch() above. visible=false hides it (no hand's ray currently
-// hits the quad).
-void xr_menu_update_cursor(float u, float v, bool visible);
-
-// Refreshes MenuOverlay's pixels from Java if they changed since the last
-// call, and blits them into whichever framebuffer is currently bound -
-// xr_session.c binds the menu's own swapchain image before calling this.
-// Only called while xr_menu_is_visible().
-void xr_menu_render_if_visible(void);
-
-void xr_menu_shutdown(void);
-
-#ifdef __cplusplus
-}
-#endif
-
-#endif

android/app/src/main/cpp/xr_overlay.c

@@ -0,0 +1,444 @@
+#include "xr_overlay.h"
+
+#include <dlfcn.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <android/log.h>
+#include <jni.h>
+
+#include <EGL/egl.h>
+#include <GLES3/gl3.h>
+
+#include <SDL.h>
+
+#include "xr_swapchain.h"
+
+#define TAG "QuestShock"
+#define LOGE(...) __android_log_print(ANDROID_LOG_ERROR, TAG, __VA_ARGS__)
+#define LOGI(...) __android_log_print(ANDROID_LOG_INFO, TAG, __VA_ARGS__)
+
+struct XrOverlay {
+    XrInstance instance; // for error logging only
+    int width, height;
+    float distance_m, half_width_m, half_height_m;
+    float center_x_m, center_y_m;
+    bool visible;
+
+    XrSwapchainState swapchain;
+
+    jclass java_class;
+    jmethodID native_init_method;
+    jmethodID take_pixels_method;
+    jmethodID dispatch_touch_method;
+    jmethodID update_cursor_method;
+
+    // This overlay's content lives in this texture, uploaded by
+    // xr_overlay_upload_source_texture_if_dirty() below. It's created
+    // through gl4es (glGenTextures/glBindTexture/glTexImage2D), like any
+    // other texture the engine itself creates, so ordinary
+    // glTexSubImage2D uploads against it work the normal way.
+    GLuint source_texture;
+    // A framebuffer with source_texture as its only color attachment, used
+    // as the read source for the blit in
+    // xr_overlay_render_and_build_layer() below. Created via the real
+    // (non-gl4es) GLES entry points below: that blit bypasses gl4es
+    // entirely (see xr_overlay_render_and_build_layer() for why), so its
+    // source framebuffer has to be a real GL object rather than one gl4es
+    // tracks.
+    GLuint source_fbo;
+
+    uint32_t pixel_generation;
+    uint32_t source_uploaded_generation;
+    uint8_t *pixel_cache; // width*height*4 bytes
+};
+
+// This file's other GL calls otherwise resolve to gl4es (the only GL
+// symbol provider linked into this binary - see android/engine-patches/
+// 02-android-opengl-es.patch), fine for anything shared with the engine's
+// own gl4es-routed rendering. But the source FBO's blit (see
+// xr_overlay_render_and_build_layer()) needs the real driver directly:
+// gl4es's fixed-pipeline emulation unconditionally substitutes its own
+// shader onto any gl4es-routed draw call, which would silently replace an
+// overlay's content with whatever the game itself last rendered. A
+// framebuffer blit has no shader stage at all, so going through the real
+// driver for it sidesteps the problem entirely - resolved once here and
+// shared by every XrOverlay instance, since they're process-wide function
+// pointers regardless of how many overlays exist.
+typedef void (*PFNQSGETINTEGERV)(GLenum, GLint *);
+typedef void (*PFNQSGENFRAMEBUFFERS)(GLsizei, GLuint *);
+typedef void (*PFNQSDELETEFRAMEBUFFERS)(GLsizei, const GLuint *);
+typedef void (*PFNQSBINDFRAMEBUFFER)(GLenum, GLuint);
+typedef void (*PFNQSFRAMEBUFFERTEXTURE2D)(GLenum, GLenum, GLenum, GLuint, GLint);
+typedef void (*PFNQSBLITFRAMEBUFFER)(GLint, GLint, GLint, GLint, GLint, GLint, GLint, GLint,
+                                      GLbitfield, GLenum);
+
+static PFNQSGETINTEGERV real_glGetIntegerv;
+static PFNQSGENFRAMEBUFFERS real_glGenFramebuffers;
+static PFNQSDELETEFRAMEBUFFERS real_glDeleteFramebuffers;
+static PFNQSBINDFRAMEBUFFER real_glBindFramebuffer;
+static PFNQSFRAMEBUFFERTEXTURE2D real_glFramebufferTexture2D;
+// glBlitFramebuffer is a GLES 3.0 addition; resolved via eglGetProcAddress
+// rather than dlsym, since Android's driver dispatch doesn't guarantee ES3+
+// symbols are dlsym-able by name from libGLESv2.so, unlike the GLES2-core
+// functions above.
+static PFNQSBLITFRAMEBUFFER real_glBlitFramebuffer;
+static bool g_real_gles_loaded = false;
+
+static bool xr_overlay_load_real_gles(void) {
+    if (g_real_gles_loaded)
+        return true;
+    void *lib = dlopen("libGLESv2.so", RTLD_NOW | RTLD_LOCAL);
+    if (lib == NULL) {
+        LOGE("XR: overlay dlopen(libGLESv2.so) failed: %s", dlerror());
+        return false;
+    }
+    real_glGetIntegerv = (PFNQSGETINTEGERV)dlsym(lib, "glGetIntegerv");
+    real_glGenFramebuffers = (PFNQSGENFRAMEBUFFERS)dlsym(lib, "glGenFramebuffers");
+    real_glDeleteFramebuffers = (PFNQSDELETEFRAMEBUFFERS)dlsym(lib, "glDeleteFramebuffers");
+    real_glBindFramebuffer = (PFNQSBINDFRAMEBUFFER)dlsym(lib, "glBindFramebuffer");
+    real_glFramebufferTexture2D = (PFNQSFRAMEBUFFERTEXTURE2D)dlsym(lib, "glFramebufferTexture2D");
+    real_glBlitFramebuffer = (PFNQSBLITFRAMEBUFFER)eglGetProcAddress("glBlitFramebuffer");
+    g_real_gles_loaded = real_glGetIntegerv && real_glGenFramebuffers &&
+                          real_glDeleteFramebuffers && real_glBindFramebuffer &&
+                          real_glFramebufferTexture2D && real_glBlitFramebuffer;
+    return g_real_gles_loaded;
+}
+
+// FindClass() from this thread (SDL's native thread, attached to the JVM
+// via AttachCurrentThread rather than spawned from Java) resolves against
+// the bootstrap classloader, which only knows framework classes - it can't
+// see app classes like de.ladkau.questshock.KeyboardOverlay at all.
+// Routing through the activity's own classloader is the standard,
+// documented workaround.
+static jclass xr_overlay_find_class(JNIEnv *env, jobject activity, const char *name) {
+    jclass activityClass = (*env)->GetObjectClass(env, activity);
+    jmethodID getClassLoader =
+        (*env)->GetMethodID(env, activityClass, "getClassLoader", "()Ljava/lang/ClassLoader;");
+    jobject classLoader = (*env)->CallObjectMethod(env, activity, getClassLoader);
+    jclass classLoaderClass = (*env)->FindClass(env, "java/lang/ClassLoader");
+    jmethodID loadClass = (*env)->GetMethodID(env, classLoaderClass, "loadClass",
+                                               "(Ljava/lang/String;)Ljava/lang/Class;");
+    jstring className = (*env)->NewStringUTF(env, name);
+    jclass result = (jclass)(*env)->CallObjectMethod(env, classLoader, loadClass, className);
+    (*env)->DeleteLocalRef(env, className);
+    return result;
+}
+
+static bool xr_overlay_init_jni(XrOverlay *overlay, const char *java_class_name) {
+    JNIEnv *env = (JNIEnv *)SDL_AndroidGetJNIEnv();
+    jobject activity = (jobject)SDL_AndroidGetActivity();
+    if (env == NULL || activity == NULL) {
+        LOGE("XR: overlay - no JNIEnv/Activity from SDL");
+        return false;
+    }
+
+    jclass localClass = xr_overlay_find_class(env, activity, java_class_name);
+    if (localClass == NULL) {
+        LOGE("XR: overlay - could not find class %s", java_class_name);
+        return false;
+    }
+    overlay->java_class = (jclass)(*env)->NewGlobalRef(env, localClass);
+
+    overlay->native_init_method = (*env)->GetStaticMethodID(env, overlay->java_class, "nativeInit",
+                                                              "(Landroid/app/Activity;)V");
+    overlay->take_pixels_method =
+        (*env)->GetStaticMethodID(env, overlay->java_class, "nativeTakePixelsIfDirty", "()[B");
+    overlay->dispatch_touch_method =
+        (*env)->GetStaticMethodID(env, overlay->java_class, "nativeDispatchTouch", "(FFZ)V");
+    overlay->update_cursor_method =
+        (*env)->GetStaticMethodID(env, overlay->java_class, "nativeUpdateCursor", "(FFZ)V");
+    if (!overlay->native_init_method || !overlay->take_pixels_method ||
+        !overlay->dispatch_touch_method || !overlay->update_cursor_method) {
+        LOGE("XR: overlay - could not resolve %s JNI methods", java_class_name);
+        return false;
+    }
+
+    (*env)->CallStaticVoidMethod(env, overlay->java_class, overlay->native_init_method, activity);
+    // CallStaticVoidMethod doesn't surface Java exceptions on its own - if
+    // the constructor throws (it runs its View measure/layout/draw calls
+    // on this native render thread rather than the Android UI thread, a
+    // plausible crash vector), the exception would otherwise be left
+    // silently pending and corrupt whatever JNI call runs next.
+    if ((*env)->ExceptionCheck(env)) {
+        LOGE("XR: overlay %s.nativeInit() threw a pending Java exception:", java_class_name);
+        (*env)->ExceptionDescribe(env);
+        (*env)->ExceptionClear(env);
+        return false;
+    }
+    return true;
+}
+
+static bool xr_overlay_create_source_texture(XrOverlay *overlay) {
+    glGenTextures(1, &overlay->source_texture);
+    glBindTexture(GL_TEXTURE_2D, overlay->source_texture);
+    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
+    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
+    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
+    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
+    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, overlay->width, overlay->height, 0, GL_RGBA,
+                 GL_UNSIGNED_BYTE, NULL);
+
+    GLint prevFbo = 0;
+    real_glGetIntegerv(GL_FRAMEBUFFER_BINDING, &prevFbo);
+    real_glGenFramebuffers(1, &overlay->source_fbo);
+    real_glBindFramebuffer(GL_FRAMEBUFFER, overlay->source_fbo);
+    real_glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D,
+                                 overlay->source_texture, 0);
+    real_glBindFramebuffer(GL_FRAMEBUFFER, (GLuint)prevFbo);
+
+    return overlay->source_texture != 0;
+}
+
+XrOverlay *xr_overlay_create(XrInstance instance, XrSession session, int64_t swapchain_format,
+                              const XrOverlayConfig *config) {
+    if (!xr_overlay_load_real_gles()) {
+        LOGE("XR: overlay couldn't resolve real GLES functions via dlsym/eglGetProcAddress");
+        return NULL;
+    }
+
+    XrOverlay *overlay = (XrOverlay *)calloc(1, sizeof(XrOverlay));
+    if (overlay == NULL)
+        return NULL;
+
+    overlay->instance = instance;
+    overlay->width = config->width;
+    overlay->height = config->height;
+    overlay->distance_m = config->distance_m;
+    overlay->half_width_m = config->width_m * 0.5f;
+    overlay->half_height_m = config->width_m * 0.5f * (float)config->height / (float)config->width;
+    overlay->center_x_m = config->default_center_x_m;
+    overlay->center_y_m = config->default_center_y_m;
+
+    if (!xr_swapchain_create(instance, session, swapchain_format, config->width, config->height,
+                              &overlay->swapchain)) {
+        LOGE("XR: overlay %s swapchain setup failed", config->java_class_name);
+        free(overlay);
+        return NULL;
+    }
+
+    if (!xr_overlay_create_source_texture(overlay)) {
+        xr_swapchain_destroy(&overlay->swapchain);
+        free(overlay);
+        return NULL;
+    }
+
+    if (!xr_overlay_init_jni(overlay, config->java_class_name)) {
+        xr_swapchain_destroy(&overlay->swapchain);
+        free(overlay);
+        return NULL;
+    }
+
+    overlay->pixel_cache = (uint8_t *)malloc((size_t)config->width * config->height * 4);
+    if (overlay->pixel_cache == NULL) {
+        xr_swapchain_destroy(&overlay->swapchain);
+        free(overlay);
+        return NULL;
+    }
+    overlay->source_uploaded_generation = (uint32_t)-1;
+
+    LOGI("XR: overlay %s ready (%dx%d)", config->java_class_name, config->width, config->height);
+    return overlay;
+}
+
+void xr_overlay_destroy(XrOverlay *overlay) {
+    if (overlay == NULL)
+        return;
+
+    if (overlay->source_texture != 0)
+        glDeleteTextures(1, &overlay->source_texture);
+    if (overlay->source_fbo != 0 && real_glDeleteFramebuffers != NULL)
+        real_glDeleteFramebuffers(1, &overlay->source_fbo);
+
+    if (overlay->java_class != NULL) {
+        JNIEnv *env = (JNIEnv *)SDL_AndroidGetJNIEnv();
+        if (env != NULL)
+            (*env)->DeleteGlobalRef(env, overlay->java_class);
+    }
+
+    xr_swapchain_destroy(&overlay->swapchain);
+    free(overlay->pixel_cache);
+    free(overlay);
+}
+
+// Every function below tolerates overlay == NULL (a no-op, or the
+// obvious "not visible"/zeroed-extent default) - xr_overlay_create() can
+// fail (e.g. its backing Java class couldn't be resolved), and unlike the
+// single static-global menu quad this replaced, an overlay instance is
+// otherwise a heap pointer callers (xr_session.c, questshock_native.c) can
+// hold onto and use across frames without re-checking for failure every
+// time.
+void xr_overlay_toggle_visible(XrOverlay *overlay) {
+    if (overlay != NULL)
+        overlay->visible = !overlay->visible;
+}
+
+void xr_overlay_set_visible(XrOverlay *overlay, bool visible) {
+    if (overlay != NULL)
+        overlay->visible = visible;
+}
+
+bool xr_overlay_is_visible(const XrOverlay *overlay) { return overlay != NULL && overlay->visible; }
+
+void xr_overlay_get_quad_extent(const XrOverlay *overlay, float *center_x_m, float *center_y_m,
+                                 float *distance_m, float *half_width_m, float *half_height_m) {
+    if (overlay == NULL) {
+        *center_x_m = *center_y_m = *distance_m = *half_width_m = *half_height_m = 0.0f;
+        return;
+    }
+    *center_x_m = overlay->center_x_m;
+    *center_y_m = overlay->center_y_m;
+    *distance_m = overlay->distance_m;
+    *half_width_m = overlay->half_width_m;
+    *half_height_m = overlay->half_height_m;
+}
+
+void xr_overlay_set_position(XrOverlay *overlay, float center_x_m, float center_y_m) {
+    if (overlay == NULL)
+        return;
+    overlay->center_x_m = center_x_m;
+    overlay->center_y_m = center_y_m;
+}
+
+void xr_overlay_touch(XrOverlay *overlay, float u, float v, bool down) {
+    if (overlay == NULL)
+        return;
+    JNIEnv *env = (JNIEnv *)SDL_AndroidGetJNIEnv();
+    if (env == NULL)
+        return;
+    // Passed via CallStaticVoidMethodA/jvalue rather than the variadic
+    // Call*Method form - deliberately sidesteps relying on JNI
+    // implementations correctly un-doing C's float-to-double default
+    // argument promotion for varargs float parameters (a well-known JNI
+    // footgun; Android's ART handles it correctly, but there's no reason
+    // to depend on that when the jvalue form is unambiguous either way).
+    jvalue args[3];
+    args[0].f = u;
+    args[1].f = v;
+    args[2].z = (jboolean)down;
+    (*env)->CallStaticVoidMethodA(env, overlay->java_class, overlay->dispatch_touch_method, args);
+}
+
+void xr_overlay_update_cursor(XrOverlay *overlay, float u, float v, bool visible) {
+    if (overlay == NULL)
+        return;
+    JNIEnv *env = (JNIEnv *)SDL_AndroidGetJNIEnv();
+    if (env == NULL)
+        return;
+    // Same jvalue-form rationale as xr_overlay_touch() above.
+    jvalue args[3];
+    args[0].f = u;
+    args[1].f = v;
+    args[2].z = (jboolean)visible;
+    (*env)->CallStaticVoidMethodA(env, overlay->java_class, overlay->update_cursor_method, args);
+}
+
+// Pulls the Java view's latest pixels (if it redrew since the last check)
+// into pixel_cache and bumps pixel_generation - called once per rendered
+// frame, before deciding whether source_texture needs a fresh upload.
+static void xr_overlay_refresh_pixel_cache(XrOverlay *overlay) {
+    JNIEnv *env = (JNIEnv *)SDL_AndroidGetJNIEnv();
+    if (env == NULL)
+        return;
+    jbyteArray pixels = (jbyteArray)(*env)->CallStaticObjectMethod(
+        env, overlay->java_class, overlay->take_pixels_method);
+    // CallStaticObjectMethod also doesn't surface Java exceptions on its
+    // own - if nativeTakePixelsIfDirty() itself throws, it would otherwise
+    // silently look identical to "nothing changed yet" (a null return).
+    if ((*env)->ExceptionCheck(env)) {
+        LOGE("XR: overlay nativeTakePixelsIfDirty() threw a pending Java exception:");
+        (*env)->ExceptionDescribe(env);
+        (*env)->ExceptionClear(env);
+        return;
+    }
+    if (pixels == NULL)
+        return;
+    (*env)->GetByteArrayRegion(env, pixels, 0, overlay->width * overlay->height * 4,
+                                (jbyte *)overlay->pixel_cache);
+    (*env)->DeleteLocalRef(env, pixels);
+    overlay->pixel_generation++;
+}
+
+// Uploads pixel_cache into source_texture via plain (gl4es-routed)
+// glBindTexture/glTexSubImage2D, matching how the texture was created.
+// Skipped when nothing changed since the last upload. Texture-unit-0 state
+// is saved and restored around the upload, since this call is gl4es-routed
+// and the engine's own next-frame rendering assumes nothing touched its
+// texture bindings since it last drew.
+static void xr_overlay_upload_source_texture_if_dirty(XrOverlay *overlay) {
+    if (overlay->source_uploaded_generation == overlay->pixel_generation)
+        return;
+
+    GLint prevActiveTexture = GL_TEXTURE0;
+    glGetIntegerv(GL_ACTIVE_TEXTURE, &prevActiveTexture);
+    glActiveTexture(GL_TEXTURE0);
+    GLint prevTexture2D = 0;
+    glGetIntegerv(GL_TEXTURE_BINDING_2D, &prevTexture2D);
+
+    glBindTexture(GL_TEXTURE_2D, overlay->source_texture);
+    glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, overlay->width, overlay->height, GL_RGBA,
+                     GL_UNSIGNED_BYTE, overlay->pixel_cache);
+    overlay->source_uploaded_generation = overlay->pixel_generation;
+
+    glBindTexture(GL_TEXTURE_2D, (GLuint)prevTexture2D);
+    if (prevActiveTexture != GL_TEXTURE0)
+        glActiveTexture((GLenum)prevActiveTexture);
+}
+
+bool xr_overlay_render_and_build_layer(XrOverlay *overlay, XrCompositionLayerQuad *out_quad) {
+    if (overlay == NULL || !overlay->visible)
+        return false;
+    if (!xr_swapchain_acquire(overlay->instance, &overlay->swapchain))
+        return false;
+
+    xr_overlay_refresh_pixel_cache(overlay);
+    xr_overlay_upload_source_texture_if_dirty(overlay);
+
+    // Copies source_texture (via source_fbo) directly into the swapchain
+    // image xr_swapchain_acquire() just bound - sized to exactly
+    // width x height, so the destination is always the whole image - using
+    // the real GLES3 hardware blit (glBlitFramebuffer) rather than a
+    // shader-based full-screen-quad draw. A blit has no vertex/fragment
+    // shading stage, no shader program, and no texture units involved at
+    // all, so gl4es's fixed-pipeline-emulation layer - which
+    // unconditionally substitutes a customized shader (reproducing the
+    // game's own last-bound texture/fixed-function state) onto any
+    // gl4es-routed draw call - has nothing to intercept here. It also
+    // never touches gl4es's own tracked program/vertex-array/
+    // texture-binding shadow state, so the only piece of state that needs
+    // save/restore is the READ framebuffer binding; binding only
+    // GL_READ_FRAMEBUFFER, rather than the combined GL_FRAMEBUFFER target,
+    // leaves the DRAW side (the swapchain image itself) untouched
+    // throughout.
+    //
+    // Y is flipped between source and destination: source_texture's texel
+    // row v=0 holds the Java view's Bitmap row 0 (the top of the rendered
+    // content, standard top-row-first raster order), and reading that same
+    // texture via an attached FBO, framebuffer y=0 accesses that identical
+    // row. The destination swapchain framebuffer follows the opposite
+    // convention - its own y=0 is its bottom edge - so swapping dstY0/dstY1
+    // keeps the content right-side up (glBlitFramebuffer supports inverted
+    // src/dst rects for exactly this).
+    GLint prevReadFbo = 0;
+    real_glGetIntegerv(GL_READ_FRAMEBUFFER_BINDING, &prevReadFbo);
+
+    real_glBindFramebuffer(GL_READ_FRAMEBUFFER, overlay->source_fbo);
+    real_glBlitFramebuffer(0, 0, overlay->width, overlay->height, 0, overlay->height,
+                            overlay->width, 0, GL_COLOR_BUFFER_BIT, GL_LINEAR);
+    real_glBindFramebuffer(GL_READ_FRAMEBUFFER, (GLuint)prevReadFbo);
+
+    xr_swapchain_release(overlay->instance, &overlay->swapchain);
+
+    memset(out_quad, 0, sizeof(*out_quad));
+    out_quad->type = XR_TYPE_COMPOSITION_LAYER_QUAD;
+    out_quad->subImage.swapchain = overlay->swapchain.swapchain;
+    out_quad->subImage.imageRect.extent.width = overlay->swapchain.width;
+    out_quad->subImage.imageRect.extent.height = overlay->swapchain.height;
+    out_quad->pose.orientation.w = 1.0f;
+    out_quad->pose.position.x = overlay->center_x_m;
+    out_quad->pose.position.y = overlay->center_y_m;
+    out_quad->pose.position.z = -overlay->distance_m;
+    out_quad->size.width = overlay->half_width_m * 2.0f;
+    out_quad->size.height = overlay->half_height_m * 2.0f;
+    return true;
+}

android/app/src/main/cpp/xr_overlay.h

@@ -0,0 +1,96 @@
+// Generic "off-screen Android View rendered into its own OpenXR quad,
+// hit-tested/touched via a laser pointer" module - each instance owns its
+// own swapchain (via xr_swapchain.h), its own visible/position state, and
+// the JNI glue to a backing Java class (menu launcher, keyboard, or any
+// future such panel) that must expose these static methods, matching the
+// pattern MenuOverlay.java already established:
+//   static void nativeInit(Activity activity)
+//   static byte[] nativeTakePixelsIfDirty()
+//   static void nativeDispatchTouch(float u, float v, boolean down)
+//   static void nativeUpdateCursor(float u, float v, boolean visible)
+//
+// xr_input.c drives touch/cursor dispatch and (for panels that support it,
+// like the keyboard's title bar) dragging via xr_overlay_set_position();
+// xr_session.c owns instance lifetime and calls
+// xr_overlay_render_and_build_layer() once per frame per instance.
+#ifndef QUESTSHOCK_XR_OVERLAY_H
+#define QUESTSHOCK_XR_OVERLAY_H
+
+#include <stdbool.h>
+#include <stdint.h>
+
+#define XR_USE_PLATFORM_ANDROID 1
+#define XR_USE_GRAPHICS_API_OPENGL_ES 1
+#include <openxr/openxr.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+typedef struct XrOverlay XrOverlay;
+
+typedef struct {
+    // Fully-qualified Java class name backing this overlay, e.g.
+    // "de.ladkau.questshock.KeyboardOverlay" - resolved via the activity's
+    // own ClassLoader (see xr_overlay_find_class() in xr_overlay.c).
+    const char *java_class_name;
+    // Pixel size of the overlay's own content/swapchain.
+    int width;
+    int height;
+    // Initial local-space pose (meters, in front of the viewer along -Z) -
+    // see xr_overlay_get_quad_extent(). width_m is the quad's physical
+    // width; its height is derived to preserve width/height's aspect.
+    float distance_m;
+    float width_m;
+    float default_center_x_m;
+    float default_center_y_m;
+} XrOverlayConfig;
+
+// instance/session are used to create this overlay's own swapchain
+// (format shared with every other quad this app submits - see
+// xr_session.c's chosenFormat). Returns NULL (logged) on failure - the
+// caller just has no such panel.
+XrOverlay *xr_overlay_create(XrInstance instance, XrSession session, int64_t swapchain_format,
+                              const XrOverlayConfig *config);
+void xr_overlay_destroy(XrOverlay *overlay);
+
+void xr_overlay_toggle_visible(XrOverlay *overlay);
+void xr_overlay_set_visible(XrOverlay *overlay, bool visible);
+bool xr_overlay_is_visible(const XrOverlay *overlay);
+
+// Local-space pose/size of the quad, valid regardless of visibility -
+// shared with xr_input.c's ray/quad hit-testing.
+void xr_overlay_get_quad_extent(const XrOverlay *overlay, float *center_x_m, float *center_y_m,
+                                 float *distance_m, float *half_width_m, float *half_height_m);
+
+// Repositions the quad (world-space X/Y offset, same units/space as
+// xr_overlay_get_quad_extent()'s center_x_m/center_y_m) - e.g. driven by
+// xr_input.c while the user drags a panel's title bar. Distance from the
+// viewer isn't adjustable this way - only left/right/up/down
+// repositioning, not push/pull.
+void xr_overlay_set_position(XrOverlay *overlay, float center_x_m, float center_y_m);
+
+// Forwards a hit on the quad (in the quad's own 0..1 u/v, top-left origin)
+// to the backing Java view as a synthetic touch down/up.
+void xr_overlay_touch(XrOverlay *overlay, float u, float v, bool down);
+
+// Updates the on-quad cursor the Java view draws at the current aim hit
+// point (same u/v convention as xr_overlay_touch()) - called once per
+// frame regardless of click state, so the cursor tracks the ray
+// continuously. visible=false hides it.
+void xr_overlay_update_cursor(XrOverlay *overlay, float u, float v, bool visible);
+
+// If visible: acquires this overlay's next swapchain image, blits the
+// Java view's latest rendered pixels into it, releases the image, and
+// fills *out_quad's subImage/pose/size. space/eyeVisibility/layerFlags are
+// left for the caller to set (shared/policy choices across every layer
+// this app submits, e.g. translucency - not this module's concern).
+// Returns false (out_quad untouched) if not visible or something failed -
+// the caller should skip submitting a layer for it this frame.
+bool xr_overlay_render_and_build_layer(XrOverlay *overlay, XrCompositionLayerQuad *out_quad);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif

android/app/src/main/cpp/xr_session.c

@@ -1,6 +1,5 @@
 #include "xr_session.h"
 
-#include <dlfcn.h>
 #include <stdlib.h>
 #include <string.h>
 
@@ -18,7 +17,8 @@
 #include <SDL.h>
 
 #include "xr_input.h"
-#include "xr_menu.h"
+#include "xr_overlay.h"
+#include "xr_swapchain.h"
 
 #define TAG "QuestShock"
 #define LOGE(...) __android_log_print(ANDROID_LOG_ERROR, TAG, __VA_ARGS__)
@@ -33,9 +33,26 @@
 #define QUAD_DISTANCE_METERS 2.0f
 #define QUAD_WIDTH_METERS 1.6f
 
-// Up to this many swapchain images/FBOs - real runtimes report small counts
+// The menu launcher (just a "Keyboard" button - see MenuOverlay.java) is
-// (2-4); this is just a fixed upper bound for the cache arrays below.
+// small and sits above where the keyboard defaults to; the keyboard keeps
-#define MAX_SWAPCHAIN_IMAGES 8
+// its previous size/position. They're independent XrOverlay instances (see
+// xr_overlay.h) so both can be shown/hidden/dragged separately - deliberately
+// positioned apart by default so they don't start out overlapping.
+#define MENU_JAVA_CLASS "de.ladkau.questshock.MenuOverlay"
+#define MENU_WIDTH 512
+#define MENU_HEIGHT 256
+#define MENU_DISTANCE_METERS 1.3f
+#define MENU_WIDTH_METERS 0.35f
+#define MENU_DEFAULT_CENTER_X_METERS 0.0f
+#define MENU_DEFAULT_CENTER_Y_METERS 0.15f
+
+#define KEYBOARD_JAVA_CLASS "de.ladkau.questshock.KeyboardOverlay"
+#define KEYBOARD_WIDTH 1024
+#define KEYBOARD_HEIGHT 768
+#define KEYBOARD_DISTANCE_METERS 1.5f
+#define KEYBOARD_WIDTH_METERS 0.8f
+#define KEYBOARD_DEFAULT_CENTER_X_METERS 0.0f
+#define KEYBOARD_DEFAULT_CENTER_Y_METERS -0.45f
 
 static XrInstance g_instance = XR_NULL_HANDLE;
 static XrSystemId g_system_id = XR_NULL_SYSTEM_ID;
@@ -54,57 +71,13 @@ static XrSessionState g_session_state = XR_SESSION_STATE_UNKNOWN;
 // later state.
 static bool g_session_running = false;
 
-// One independent swapchain per quad (game, menu), each sized to exactly
-// its own content and acquired/released on its own within the same
-// xrBeginFrame/xrEndFrame pair - a completely ordinary multi-layer OpenXR
-// setup. A GL framebuffer wrapping each swapchain image is created once, up
-// front, and reused every frame, since a swapchain's images are a small,
-// fixed, runtime-owned pool (MAX_SWAPCHAIN_IMAGES above), not something
-// recreated per frame.
-typedef struct {
-    XrSwapchain swapchain;
-    GLuint fbos[MAX_SWAPCHAIN_IMAGES];
-    uint32_t image_count;
-    int width;
-    int height;
-} XrSwapchainState;
-
 static XrSwapchainState g_game_swapchain;
-static XrSwapchainState g_menu_swapchain;
+static XrOverlay *g_menu_overlay = NULL;
+static XrOverlay *g_keyboard_overlay = NULL;
 
 static XrTime g_predicted_display_time = 0;
 static bool g_frame_should_render = false;
 static bool g_have_acquired_game_image = false;
-static bool g_have_acquired_menu_image = false;
-
-// This file's GL calls otherwise resolve to gl4es (the only GL symbol
-// provider linked into this binary - see android/engine-patches/
-// 02-android-opengl-es.patch), which is fine for anything shared with the
-// engine's own gl4es-routed rendering. But the swapchain images OpenXR
-// hands us are real driver texture objects gl4es never created itself,
-// and gl4es's own glFramebufferTexture2D can't attach a texture it has no
-// tracked metadata for. So the FBO *container* is created via gl4es's own
-// glGenFramebuffers/glBindFramebuffer (so gl4es recognizes the id as its
-// own and its own per-frame glBindFramebuffer succeeds), while the
-// texture-attach step - the specifically foreign part - goes through the
-// real driver directly, via dlsym against libGLESv2.so.
-typedef void (*PFNQSFRAMEBUFFERTEXTURE2D)(GLenum, GLenum, GLenum, GLuint, GLint);
-typedef GLenum (*PFNQSCHECKFRAMEBUFFERSTATUS)(GLenum);
-
-static PFNQSFRAMEBUFFERTEXTURE2D real_glFramebufferTexture2D;
-static PFNQSCHECKFRAMEBUFFERSTATUS real_glCheckFramebufferStatus;
-
-static bool xr_load_real_gles(void) {
-    void *lib = dlopen("libGLESv2.so", RTLD_NOW | RTLD_LOCAL);
-    if (lib == NULL) {
-        LOGE("XR: dlopen(libGLESv2.so) failed: %s", dlerror());
-        return false;
-    }
-    real_glFramebufferTexture2D = (PFNQSFRAMEBUFFERTEXTURE2D)dlsym(lib, "glFramebufferTexture2D");
-    real_glCheckFramebufferStatus =
-        (PFNQSCHECKFRAMEBUFFERSTATUS)dlsym(lib, "glCheckFramebufferStatus");
-    return real_glFramebufferTexture2D && real_glCheckFramebufferStatus;
-}
 
 static bool xr_check(XrResult result, const char *what) {
     if (XR_SUCCEEDED(result))
@@ -160,111 +133,6 @@ static EGLConfig xr_get_current_egl_config(EGLDisplay display, EGLContext contex
     return config;
 }
 
-// Creates a swapchain at the given size/format and wraps each of its images
-// in its own GL framebuffer, ready to bind and render into directly -
-// shared by the game and menu swapchains below, since both need exactly the
-// same setup, just at a different size.
-static bool xr_create_swapchain_state(int64_t format, int width, int height,
-                                       XrSwapchainState *out) {
-    out->width = width;
-    out->height = height;
-
-    XrSwapchainCreateInfo swapchainInfo = {XR_TYPE_SWAPCHAIN_CREATE_INFO};
-    swapchainInfo.usageFlags =
-        XR_SWAPCHAIN_USAGE_COLOR_ATTACHMENT_BIT | XR_SWAPCHAIN_USAGE_SAMPLED_BIT;
-    swapchainInfo.format = format;
-    swapchainInfo.sampleCount = 1;
-    swapchainInfo.width = (uint32_t)width;
-    swapchainInfo.height = (uint32_t)height;
-    swapchainInfo.faceCount = 1;
-    swapchainInfo.arraySize = 1;
-    swapchainInfo.mipCount = 1;
-    if (!xr_check(xrCreateSwapchain(g_session, &swapchainInfo, &out->swapchain),
-                  "xrCreateSwapchain"))
-        return false;
-
-    uint32_t imageCount = 0;
-    xrEnumerateSwapchainImages(out->swapchain, 0, &imageCount, NULL);
-    if (imageCount > MAX_SWAPCHAIN_IMAGES) {
-        LOGE("XR: swapchain reports %u images, only room for %d", imageCount,
-             MAX_SWAPCHAIN_IMAGES);
-        return false;
-    }
-    // Zero-initialized, not just `.type` set per element - these structs
-    // also carry a `next` field the runtime may read, and an uninitialized
-    // stack array would leave it as garbage.
-    XrSwapchainImageOpenGLESKHR images[MAX_SWAPCHAIN_IMAGES] = {0};
-    for (uint32_t i = 0; i < imageCount; i++)
-        images[i].type = XR_TYPE_SWAPCHAIN_IMAGE_OPENGL_ES_KHR;
-    if (!xr_check(xrEnumerateSwapchainImages(out->swapchain, imageCount, &imageCount,
-                                              (XrSwapchainImageBaseHeader *)images),
-                  "xrEnumerateSwapchainImages"))
-        return false;
-    out->image_count = imageCount;
-
-    // Wraps each swapchain-provided texture in its own framebuffer, matching
-    // how OpenGL.cc's own CreateFrameBuffer() wraps backupBuffer - just
-    // without a depth/stencil attachment, since the final composite draw
-    // (see opengl_swap_and_restore) never needs one. The Gen/Bind calls are
-    // gl4es's own (linked, not dlsym'd) - see the comment above
-    // real_glFramebufferTexture2D for why.
-    bool all_complete = true;
-    for (uint32_t i = 0; i < imageCount; i++) {
-        glGenFramebuffers(1, &out->fbos[i]);
-        glBindFramebuffer(GL_FRAMEBUFFER, out->fbos[i]);
-        real_glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D,
-                                     images[i].image, 0);
-        GLenum status = real_glCheckFramebufferStatus(GL_FRAMEBUFFER);
-        if (status != GL_FRAMEBUFFER_COMPLETE) {
-            LOGE("XR: swapchain FBO %u incomplete: 0x%x", i, status);
-            all_complete = false;
-        }
-    }
-    glBindFramebuffer(GL_FRAMEBUFFER, 0);
-    return all_complete;
-}
-
-static void xr_destroy_swapchain_state(XrSwapchainState *state) {
-    for (uint32_t i = 0; i < state->image_count; i++) {
-        if (state->fbos[i] != 0)
-            glDeleteFramebuffers(1, &state->fbos[i]);
-    }
-    state->image_count = 0;
-    if (state->swapchain != XR_NULL_HANDLE)
-        xrDestroySwapchain(state->swapchain);
-    state->swapchain = XR_NULL_HANDLE;
-}
-
-// Acquires the next image of the given swapchain, waits for the runtime to
-// finish with it, and binds its framebuffer (sized to exactly fill it) as
-// the current render target.
-static bool xr_acquire_swapchain_image(XrSwapchainState *state) {
-    uint32_t imageIndex = 0;
-    XrSwapchainImageAcquireInfo acquireInfo = {XR_TYPE_SWAPCHAIN_IMAGE_ACQUIRE_INFO};
-    if (!xr_check(xrAcquireSwapchainImage(state->swapchain, &acquireInfo, &imageIndex),
-                  "xrAcquireSwapchainImage"))
-        return false;
-
-    XrSwapchainImageWaitInfo waitImageInfo = {XR_TYPE_SWAPCHAIN_IMAGE_WAIT_INFO};
-    waitImageInfo.timeout = XR_INFINITE_DURATION;
-    if (!xr_check(xrWaitSwapchainImage(state->swapchain, &waitImageInfo), "xrWaitSwapchainImage"))
-        return false;
-
-    // gl4es's own (linked, not dlsym'd) bind - this targets an FBO id gl4es
-    // itself created (see xr_create_swapchain_state()), so its own "current
-    // FBO" bookkeeping updates correctly and its immediate-mode draw calls
-    // (android_draw_surface_as_quad(), the laser reticle) land in the right
-    // place.
-    glBindFramebuffer(GL_FRAMEBUFFER, state->fbos[imageIndex]);
-    glViewport(0, 0, state->width, state->height);
-    return true;
-}
-
-static void xr_release_swapchain_image(XrSwapchainState *state) {
-    XrSwapchainImageReleaseInfo releaseInfo = {XR_TYPE_SWAPCHAIN_IMAGE_RELEASE_INFO};
-    xr_check(xrReleaseSwapchainImage(state->swapchain, &releaseInfo), "xrReleaseSwapchainImage");
-}
-
 static bool xr_create_instance_and_session(int game_width, int game_height) {
     JNIEnv *env = (JNIEnv *)SDL_AndroidGetJNIEnv();
     jobject activity = (jobject)SDL_AndroidGetActivity();
@@ -355,8 +223,9 @@ static bool xr_create_instance_and_session(int game_width, int game_height) {
     // GL_RGBA with no decode step anywhere in this path, matching the
     // desktop SDL_RenderCopy path this replaces - an sRGB swapchain format
     // would auto-gamma-encode on write and double-encode already-encoded
-    // data. Both swapchains below share this one choice - the compositor's
+    // data. Every swapchain below (game quad, menu/keyboard overlays)
-    // supported format set doesn't depend on swapchain size.
+    // shares this one choice - the compositor's supported format set
+    // doesn't depend on swapchain size.
     uint32_t formatCount = 0;
     xrEnumerateSwapchainFormats(g_session, 0, &formatCount, NULL);
     int64_t *formats = (int64_t *)malloc(sizeof(int64_t) * formatCount);
@@ -371,30 +240,38 @@ static bool xr_create_instance_and_session(int game_width, int game_height) {
     }
     free(formats);
 
-    if (!xr_load_real_gles()) {
+    if (!xr_swapchain_create(g_instance, g_session, chosenFormat, game_width, game_height,
-        LOGE("XR: couldn't resolve real GLES FBO functions via dlsym");
+                              &g_game_swapchain)) {
-        return false;
-    }
-
-    if (!xr_create_swapchain_state(chosenFormat, game_width, game_height, &g_game_swapchain)) {
         LOGE("XR: game swapchain setup failed");
         return false;
     }
 
-    int menu_width, menu_height;
+    // Both non-fatal - the game keeps rendering with no such panel if
-    xr_menu_get_content_size(&menu_width, &menu_height);
+    // either fails (e.g. the Java class couldn't be resolved via JNI).
-    if (!xr_create_swapchain_state(chosenFormat, menu_width, menu_height, &g_menu_swapchain)) {
+    XrOverlayConfig menuConfig = {
-        LOGE("XR: menu swapchain setup failed");
+        .java_class_name = MENU_JAVA_CLASS,
-        return false;
+        .width = MENU_WIDTH,
-    }
+        .height = MENU_HEIGHT,
+        .distance_m = MENU_DISTANCE_METERS,
+        .width_m = MENU_WIDTH_METERS,
+        .default_center_x_m = MENU_DEFAULT_CENTER_X_METERS,
+        .default_center_y_m = MENU_DEFAULT_CENTER_Y_METERS,
+    };
+    g_menu_overlay = xr_overlay_create(g_instance, g_session, chosenFormat, &menuConfig);
 
-    LOGI("XR: instance/session ready (game %dx%d, menu %dx%d)", game_width, game_height,
+    XrOverlayConfig keyboardConfig = {
-         menu_width, menu_height);
+        .java_class_name = KEYBOARD_JAVA_CLASS,
-
+        .width = KEYBOARD_WIDTH,
-    // Also non-fatal - the game keeps rendering with no menu quad if this
+        .height = KEYBOARD_HEIGHT,
-    // fails (e.g. MenuOverlay couldn't be resolved via JNI).
+        .distance_m = KEYBOARD_DISTANCE_METERS,
-    xr_menu_init(g_instance, g_session);
+        .width_m = KEYBOARD_WIDTH_METERS,
+        .default_center_x_m = KEYBOARD_DEFAULT_CENTER_X_METERS,
+        .default_center_y_m = KEYBOARD_DEFAULT_CENTER_Y_METERS,
+    };
+    g_keyboard_overlay = xr_overlay_create(g_instance, g_session, chosenFormat, &keyboardConfig);
 
+    LOGI("XR: instance/session ready (game %dx%d, menu %dx%d, keyboard %dx%d)", game_width,
+         game_height, MENU_WIDTH, MENU_HEIGHT, KEYBOARD_WIDTH, KEYBOARD_HEIGHT);
     return true;
 }
 
@@ -407,6 +284,9 @@ bool xr_init(int game_width, int game_height) {
     return true;
 }
 
+XrOverlay *xr_session_get_menu_overlay(void) { return g_menu_overlay; }
+XrOverlay *xr_session_get_keyboard_overlay(void) { return g_keyboard_overlay; }
+
 void xr_poll_events(void) {
     if (g_instance == XR_NULL_HANDLE)
         return;
@@ -461,7 +341,7 @@ bool xr_frame_begin(void) {
     if (!g_frame_should_render)
         return false;
 
-    g_have_acquired_game_image = xr_acquire_swapchain_image(&g_game_swapchain);
+    g_have_acquired_game_image = xr_swapchain_acquire(g_instance, &g_game_swapchain);
     return g_have_acquired_game_image;
 }
 
@@ -474,32 +354,25 @@ void xr_frame_end(void) {
     // this frame's game content already drew there. Syncing actions
     // happens even when there's nothing to draw (no acquired image this
     // frame), so edge detection (trigger/menu-button clicks) doesn't miss a
-    // frame. Only ever draws while the menu isn't visible (see xr_input.c) -
+    // frame. Only ever draws while neither overlay is visible (see
-    // the menu renders into its own independent swapchain image below.
+    // xr_input.c) - the menu/keyboard render into their own independent
+    // swapchain images below.
     if (xr_is_session_running())
         xr_input_sync_and_draw(g_local_space, g_predicted_display_time,
                                 g_have_acquired_game_image);
 
     if (g_have_acquired_game_image)
-        xr_release_swapchain_image(&g_game_swapchain);
+        xr_swapchain_release(g_instance, &g_game_swapchain);
-
-    // The menu gets its own acquire/render/release cycle against its own
-    // swapchain - unlike the game quad, there's no per-frame engine
-    // rendering to wrap around here, just xr_menu.c's own blit, so this
-    // can happen any time before xrEndFrame rather than needing to bracket
-    // anything.
-    g_have_acquired_menu_image = false;
-    if (g_frame_should_render && xr_menu_is_visible()) {
-        g_have_acquired_menu_image = xr_acquire_swapchain_image(&g_menu_swapchain);
-        if (g_have_acquired_menu_image) {
-            xr_menu_render_if_visible();
-            xr_release_swapchain_image(&g_menu_swapchain);
-        }
-    }
 
     if (!xr_is_session_running())
         return;
 
+    // Up to 3 layers: the game quad (if a frame was actually rendered),
+    // and, while visible, the menu launcher and keyboard overlays - each
+    // gets its own acquire/render/release cycle against its own swapchain
+    // (see xr_overlay_render_and_build_layer()) any time before xrEndFrame,
+    // unlike the game quad there's no per-frame engine rendering to wrap
+    // around here, just each overlay's own blit.
     XrCompositionLayerQuad gameQuad = {XR_TYPE_COMPOSITION_LAYER_QUAD};
     gameQuad.space = g_local_space;
     gameQuad.eyeVisibility = XR_EYE_VISIBILITY_BOTH;
@@ -512,33 +385,34 @@ void xr_frame_end(void) {
     gameQuad.size.height =
         QUAD_WIDTH_METERS * (float)g_game_swapchain.height / (float)g_game_swapchain.width;
 
-    // Up to 2 layers: the game quad (if a frame was actually rendered) and,
+    const XrCompositionLayerBaseHeader *layers[3];
-    // while toggled on and rendered this frame, the menu quad in front of
-    // it - see xr_input.c's menu_toggle handling. Each references its own
-    // independent swapchain (see XrSwapchainState above).
-    const XrCompositionLayerBaseHeader *layers[2];
     uint32_t layerCount = 0;
     if (g_have_acquired_game_image)
         layers[layerCount++] = (XrCompositionLayerBaseHeader *)&gameQuad;
 
-    XrCompositionLayerQuad menuQuad = {XR_TYPE_COMPOSITION_LAYER_QUAD};
+    XrCompositionLayerQuad menuQuad;
-    if (g_have_acquired_menu_image) {
+    if (g_frame_should_render && xr_overlay_render_and_build_layer(g_menu_overlay, &menuQuad)) {
-        float centerX, centerY, distance, halfWidth, halfHeight;
-        xr_menu_get_quad_extent(&centerX, &centerY, &distance, &halfWidth, &halfHeight);
         menuQuad.space = g_local_space;
         menuQuad.eyeVisibility = XR_EYE_VISIBILITY_BOTH;
-        menuQuad.subImage.swapchain = g_menu_swapchain.swapchain;
-        menuQuad.subImage.imageRect.extent.width = g_menu_swapchain.width;
-        menuQuad.subImage.imageRect.extent.height = g_menu_swapchain.height;
-        menuQuad.pose.orientation.w = 1.0f;
-        menuQuad.pose.position.x = centerX;
-        menuQuad.pose.position.y = centerY;
-        menuQuad.pose.position.z = -distance;
-        menuQuad.size.width = halfWidth * 2.0f;
-        menuQuad.size.height = halfHeight * 2.0f;
         layers[layerCount++] = (XrCompositionLayerBaseHeader *)&menuQuad;
     }
 
+    XrCompositionLayerQuad keyboardQuad;
+    if (g_frame_should_render &&
+        xr_overlay_render_and_build_layer(g_keyboard_overlay, &keyboardQuad)) {
+        keyboardQuad.space = g_local_space;
+        keyboardQuad.eyeVisibility = XR_EYE_VISIBILITY_BOTH;
+        // Lets the keyboard sit on screen translucently while playing
+        // rather than fully hiding whatever's behind it - KeyboardOverlay
+        // writes real sub-255 alpha into the swapchain texture (see
+        // OverlayPanel.compositeAndPublish()), already in Android's
+        // default premultiplied format, which is what this flag's absence
+        // of UNPREMULTIPLIED_ALPHA_BIT assumes. The menu launcher above
+        // deliberately doesn't get this flag - it stays fully opaque.
+        keyboardQuad.layerFlags = XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT;
+        layers[layerCount++] = (XrCompositionLayerBaseHeader *)&keyboardQuad;
+    }
+
     XrFrameEndInfo endInfo = {XR_TYPE_FRAME_END_INFO};
     endInfo.displayTime = g_predicted_display_time;
     endInfo.environmentBlendMode = XR_ENVIRONMENT_BLEND_MODE_OPAQUE;
@@ -548,8 +422,7 @@ void xr_frame_end(void) {
 }
 
 void xr_shutdown(void) {
-    xr_destroy_swapchain_state(&g_game_swapchain);
+    xr_swapchain_destroy(&g_game_swapchain);
-    xr_destroy_swapchain_state(&g_menu_swapchain);
 
     if (g_local_space != XR_NULL_HANDLE)
         xrDestroySpace(g_local_space);
@@ -557,7 +430,10 @@ void xr_shutdown(void) {
 
     // Before the session/instance they were created from.
     xr_input_shutdown();
-    xr_menu_shutdown();
+    xr_overlay_destroy(g_menu_overlay);
+    g_menu_overlay = NULL;
+    xr_overlay_destroy(g_keyboard_overlay);
+    g_keyboard_overlay = NULL;
 
     if (g_session != XR_NULL_HANDLE)
         xrDestroySession(g_session);

android/app/src/main/cpp/xr_session.h

@@ -9,6 +9,8 @@
 
 #include <stdbool.h>
 
+#include "xr_overlay.h"
+
 #ifdef __cplusplus
 extern "C" {
 #endif
@@ -17,13 +19,25 @@ extern "C" {
 // current the GL context SDL/gl4es already use - creates the OpenXR
 // instance/session sharing that same EGL display/context, plus the game
 // quad's own swapchain, sized to the game's logical resolution
-// (game_width/height, i.e. grd_cap->w/h - see Shock.c's InitSDL()). The
+// (game_width/height, i.e. grd_cap->w/h - see Shock.c's InitSDL()). Also
-// menu quad's own, independently-sized swapchain (see xr_menu.h) is created
+// creates the menu launcher and keyboard overlay quads (see xr_overlay.h) -
-// alongside it. Returns false if OpenXR bring-up failed (e.g. no runtime
+// two independent XrOverlay instances, each with their own swapchain,
-// installed) - callers should fall back to the existing window-present path
+// visibility, and position, retrievable via xr_session_get_menu_overlay()/
-// in that case.
+// xr_session_get_keyboard_overlay() below. Returns false if OpenXR
+// bring-up failed (e.g. no runtime installed) - callers should fall back
+// to the existing window-present path in that case.
 bool xr_init(int game_width, int game_height);
 
+// The menu launcher (small "Keyboard" button panel, toggled by the
+// controller's menu button) and keyboard (the on-screen key grid, shown/
+// hidden independently via MenuOverlay's "Keyboard" button and
+// KeyboardOverlay's own Close button) overlay quads - used by xr_input.c
+// for ray/quad hit-testing and touch/cursor dispatch, and by
+// questshock_native.c's JNI glue (nativeShowKeyboard()/
+// nativeRequestClose()). NULL if xr_init() failed before creating them.
+XrOverlay *xr_session_get_menu_overlay(void);
+XrOverlay *xr_session_get_keyboard_overlay(void);
+
 // Pumps XR session-state events. Call once per frame, before
 // xr_frame_begin(). Must still be called even when xr_init() returned
 // false (no-op in that case).
@@ -44,10 +58,12 @@ bool xr_is_session_running(void);
 bool xr_frame_begin(void);
 
 // Releases the game quad's swapchain image (if one was acquired this
-// frame), acquires/renders/releases the menu quad's own swapchain image if
+// frame), then does the same acquire/render/release cycle for the menu
-// it's currently visible (see xr_menu.h), submits whichever of the two
+// launcher and keyboard overlay quads via xr_overlay_render_and_build_layer()
-// quads actually rendered this frame as composition layers positioned in
+// for whichever of them is currently visible, and submits whichever of the
-// front of the local reference space's origin, then ends the XR frame.
+// three quads actually rendered this frame as composition layers
+// positioned in front of the local reference space's origin, then ends the
+// XR frame.
 void xr_frame_end(void);
 
 void xr_shutdown(void);

android/app/src/main/cpp/xr_swapchain.c

@@ -0,0 +1,160 @@
+#include "xr_swapchain.h"
+
+#include <dlfcn.h>
+
+#include <android/log.h>
+
+#define TAG "QuestShock"
+#define LOGE(...) __android_log_print(ANDROID_LOG_ERROR, TAG, __VA_ARGS__)
+
+static bool xr_check(XrInstance instance, XrResult result, const char *what) {
+    if (XR_SUCCEEDED(result))
+        return true;
+    char resultString[XR_MAX_RESULT_STRING_SIZE] = {0};
+    if (instance != XR_NULL_HANDLE)
+        xrResultToString(instance, result, resultString);
+    LOGE("XR: %s failed: %s (%d)", what, resultString[0] ? resultString : "?", (int)result);
+    return false;
+}
+
+// This file's other GL calls otherwise resolve to gl4es (the only GL
+// symbol provider linked into this binary - see android/engine-patches/
+// 02-android-opengl-es.patch), which is fine for anything shared with the
+// engine's own gl4es-routed rendering. But the swapchain images OpenXR
+// hands us are real driver texture objects gl4es never created itself,
+// and gl4es's own glFramebufferTexture2D can't attach a texture it has no
+// tracked metadata for. So the FBO *container* is created via gl4es's own
+// glGenFramebuffers/glBindFramebuffer (so gl4es recognizes the id as its
+// own and its own per-frame glBindFramebuffer succeeds), while the
+// texture-attach step - the specifically foreign part - goes through the
+// real driver directly, via dlsym against libGLESv2.so.
+typedef void (*PFNQSFRAMEBUFFERTEXTURE2D)(GLenum, GLenum, GLenum, GLuint, GLint);
+typedef GLenum (*PFNQSCHECKFRAMEBUFFERSTATUS)(GLenum);
+
+static PFNQSFRAMEBUFFERTEXTURE2D real_glFramebufferTexture2D;
+static PFNQSCHECKFRAMEBUFFERSTATUS real_glCheckFramebufferStatus;
+static bool g_real_gles_loaded = false;
+
+static bool xr_swapchain_load_real_gles(void) {
+    if (g_real_gles_loaded)
+        return true;
+    void *lib = dlopen("libGLESv2.so", RTLD_NOW | RTLD_LOCAL);
+    if (lib == NULL) {
+        LOGE("XR: swapchain dlopen(libGLESv2.so) failed: %s", dlerror());
+        return false;
+    }
+    real_glFramebufferTexture2D = (PFNQSFRAMEBUFFERTEXTURE2D)dlsym(lib, "glFramebufferTexture2D");
+    real_glCheckFramebufferStatus =
+        (PFNQSCHECKFRAMEBUFFERSTATUS)dlsym(lib, "glCheckFramebufferStatus");
+    g_real_gles_loaded = real_glFramebufferTexture2D && real_glCheckFramebufferStatus;
+    return g_real_gles_loaded;
+}
+
+bool xr_swapchain_create(XrInstance instance, XrSession session, int64_t format, int width,
+                          int height, XrSwapchainState *out) {
+    if (!xr_swapchain_load_real_gles()) {
+        LOGE("XR: swapchain couldn't resolve real GLES FBO functions via dlsym");
+        return false;
+    }
+
+    out->width = width;
+    out->height = height;
+
+    XrSwapchainCreateInfo swapchainInfo = {XR_TYPE_SWAPCHAIN_CREATE_INFO};
+    swapchainInfo.usageFlags =
+        XR_SWAPCHAIN_USAGE_COLOR_ATTACHMENT_BIT | XR_SWAPCHAIN_USAGE_SAMPLED_BIT;
+    swapchainInfo.format = format;
+    swapchainInfo.sampleCount = 1;
+    swapchainInfo.width = (uint32_t)width;
+    swapchainInfo.height = (uint32_t)height;
+    swapchainInfo.faceCount = 1;
+    swapchainInfo.arraySize = 1;
+    swapchainInfo.mipCount = 1;
+    if (!xr_check(instance, xrCreateSwapchain(session, &swapchainInfo, &out->swapchain),
+                  "xrCreateSwapchain"))
+        return false;
+
+    uint32_t imageCount = 0;
+    xrEnumerateSwapchainImages(out->swapchain, 0, &imageCount, NULL);
+    if (imageCount > XR_SWAPCHAIN_MAX_IMAGES) {
+        LOGE("XR: swapchain reports %u images, only room for %d", imageCount,
+             XR_SWAPCHAIN_MAX_IMAGES);
+        return false;
+    }
+    // Zero-initialized, not just `.type` set per element - these structs
+    // also carry a `next` field the runtime may read, and an uninitialized
+    // stack array would leave it as garbage.
+    XrSwapchainImageOpenGLESKHR images[XR_SWAPCHAIN_MAX_IMAGES] = {0};
+    for (uint32_t i = 0; i < imageCount; i++)
+        images[i].type = XR_TYPE_SWAPCHAIN_IMAGE_OPENGL_ES_KHR;
+    if (!xr_check(instance,
+                  xrEnumerateSwapchainImages(out->swapchain, imageCount, &imageCount,
+                                              (XrSwapchainImageBaseHeader *)images),
+                  "xrEnumerateSwapchainImages"))
+        return false;
+    out->image_count = imageCount;
+
+    // Wraps each swapchain-provided texture in its own framebuffer, matching
+    // how OpenGL.cc's own CreateFrameBuffer() wraps backupBuffer - just
+    // without a depth/stencil attachment, since the final composite draw
+    // (see opengl_swap_and_restore) never needs one. The Gen/Bind calls are
+    // gl4es's own (linked, not dlsym'd) - see the comment above
+    // real_glFramebufferTexture2D for why.
+    bool all_complete = true;
+    for (uint32_t i = 0; i < imageCount; i++) {
+        glGenFramebuffers(1, &out->fbos[i]);
+        glBindFramebuffer(GL_FRAMEBUFFER, out->fbos[i]);
+        real_glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D,
+                                     images[i].image, 0);
+        GLenum status = real_glCheckFramebufferStatus(GL_FRAMEBUFFER);
+        if (status != GL_FRAMEBUFFER_COMPLETE) {
+            LOGE("XR: swapchain FBO %u incomplete: 0x%x", i, status);
+            all_complete = false;
+        }
+    }
+    glBindFramebuffer(GL_FRAMEBUFFER, 0);
+    return all_complete;
+}
+
+void xr_swapchain_destroy(XrSwapchainState *state) {
+    for (uint32_t i = 0; i < state->image_count; i++) {
+        if (state->fbos[i] != 0)
+            glDeleteFramebuffers(1, &state->fbos[i]);
+    }
+    state->image_count = 0;
+    if (state->swapchain != XR_NULL_HANDLE)
+        xrDestroySwapchain(state->swapchain);
+    state->swapchain = XR_NULL_HANDLE;
+}
+
+// Acquires the next image of the given swapchain, waits for the runtime to
+// finish with it, and binds its framebuffer (sized to exactly fill it) as
+// the current render target.
+bool xr_swapchain_acquire(XrInstance instance, XrSwapchainState *state) {
+    uint32_t imageIndex = 0;
+    XrSwapchainImageAcquireInfo acquireInfo = {XR_TYPE_SWAPCHAIN_IMAGE_ACQUIRE_INFO};
+    if (!xr_check(instance, xrAcquireSwapchainImage(state->swapchain, &acquireInfo, &imageIndex),
+                  "xrAcquireSwapchainImage"))
+        return false;
+
+    XrSwapchainImageWaitInfo waitImageInfo = {XR_TYPE_SWAPCHAIN_IMAGE_WAIT_INFO};
+    waitImageInfo.timeout = XR_INFINITE_DURATION;
+    if (!xr_check(instance, xrWaitSwapchainImage(state->swapchain, &waitImageInfo),
+                  "xrWaitSwapchainImage"))
+        return false;
+
+    // gl4es's own (linked, not dlsym'd) bind - this targets an FBO id gl4es
+    // itself created (see xr_swapchain_create()), so its own "current FBO"
+    // bookkeeping updates correctly and its immediate-mode draw calls
+    // (android_draw_surface_as_quad(), the laser reticle) land in the right
+    // place.
+    glBindFramebuffer(GL_FRAMEBUFFER, state->fbos[imageIndex]);
+    glViewport(0, 0, state->width, state->height);
+    return true;
+}
+
+void xr_swapchain_release(XrInstance instance, XrSwapchainState *state) {
+    XrSwapchainImageReleaseInfo releaseInfo = {XR_TYPE_SWAPCHAIN_IMAGE_RELEASE_INFO};
+    xr_check(instance, xrReleaseSwapchainImage(state->swapchain, &releaseInfo),
+              "xrReleaseSwapchainImage");
+}

android/app/src/main/cpp/xr_swapchain.h

@@ -0,0 +1,55 @@
+// Generic OpenXR swapchain + per-image GL framebuffer helper, shared by
+// every quad this app submits - the game quad (xr_session.c) and every
+// xr_overlay.c instance (menu launcher, keyboard) - since they all need
+// exactly the same setup, just at their own size/format.
+#ifndef QUESTSHOCK_XR_SWAPCHAIN_H
+#define QUESTSHOCK_XR_SWAPCHAIN_H
+
+#include <stdbool.h>
+#include <stdint.h>
+
+#include <EGL/egl.h>
+#include <GLES3/gl3.h>
+#include <jni.h>
+
+#define XR_USE_PLATFORM_ANDROID 1
+#define XR_USE_GRAPHICS_API_OPENGL_ES 1
+#include <openxr/openxr.h>
+#include <openxr/openxr_platform.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+// Up to this many swapchain images/FBOs - real runtimes report small counts
+// (2-4); this is just a fixed upper bound for the cache arrays below.
+#define XR_SWAPCHAIN_MAX_IMAGES 8
+
+typedef struct {
+    XrSwapchain swapchain;
+    GLuint fbos[XR_SWAPCHAIN_MAX_IMAGES];
+    uint32_t image_count;
+    int width;
+    int height;
+} XrSwapchainState;
+
+// Creates the swapchain and wraps each of its images in its own GL
+// framebuffer, ready to bind and render into directly. instance is only
+// used to log a human-readable error string on failure.
+bool xr_swapchain_create(XrInstance instance, XrSession session, int64_t format, int width,
+                          int height, XrSwapchainState *out);
+
+void xr_swapchain_destroy(XrSwapchainState *state);
+
+// Acquires the next image, waits for the runtime to finish with it, and
+// binds its framebuffer (sized to exactly fill it) as the current render
+// target.
+bool xr_swapchain_acquire(XrInstance instance, XrSwapchainState *state);
+
+void xr_swapchain_release(XrInstance instance, XrSwapchainState *state);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif

android/app/src/main/java/de/ladkau/questshock/KeyboardOverlay.java

@@ -0,0 +1,474 @@
+package de.ladkau.questshock;
+
+import android.app.Activity;
+import android.util.TypedValue;
+import android.view.Gravity;
+import android.view.KeyEvent;
+import android.view.View;
+import android.view.ViewGroup;
+import android.widget.Button;
+import android.widget.LinearLayout;
+import android.widget.TextView;
+import java.util.ArrayList;
+import java.util.List;
+import org.libsdl.app.SDLActivity;
+
+/**
+ * The on-screen keyboard quad (see OverlayPanel for the shared off-screen-
+ * render/touch/cursor plumbing this builds on) - a hand-built, full
+ * US-layout key grid (there's no system IME to borrow once immersive) plus
+ * a text field echoing what's been sent. Each key press is forwarded
+ * straight to the game as real input (see handleKeyPress()/
+ * handleCharKeyPress()) - non-printable keys as an
+ * SDLActivity.onNativeKeyDown()/onNativeKeyUp() pair, the same one a
+ * physical Bluetooth keyboard's presses already drive, and printable keys
+ * via a synthesized SDL_TEXTINPUT event pushed from native code (see
+ * commitPrintableChar()/nativeSendPrintableChar() in questshock_native.c) -
+ * except while a Ctrl/Alt "armed" modifier is active, when they go through
+ * the same onNativeKeyDown()/onNativeKeyUp() path instead, so the modifier
+ * carries across (see toggleModifier()). Shift is a purely local layer
+ * toggle (uppercase/symbols vs lowercase/numbers), never itself sent to the
+ * game.
+ *
+ * This is a fully independent overlay quad from MenuOverlay's small
+ * launcher - opened via that launcher's "Keyboard" button
+ * (nativeShowKeyboard()) but otherwise self-contained: once open, the
+ * controller's menu button (which only ever affects the menu launcher, see
+ * xr_input.c) doesn't hide it, so it stays up as a standing input panel
+ * while actually playing rather than a modal you open and close. The only
+ * way to hide it is its own title bar's Close button (buildTitleBar()),
+ * which also doubles as a drag handle for repositioning it (handled
+ * entirely on the native side - see xr_input.c - since it never reaches
+ * this class's own touch dispatch). contentAlpha() renders the whole panel
+ * semi-transparent so whatever's behind it stays visible while it's up.
+ */
+public class KeyboardOverlay extends OverlayPanel {
+    // Matches xr_session.c's KEYBOARD_WIDTH/KEYBOARD_HEIGHT swapchain size -
+    // fixed, not tied to any real display metric, since this tree is never
+    // shown in a real window.
+    private static final int WIDTH = 1024;
+    private static final int HEIGHT = 768;
+
+    private static final int TYPED_TEXT_HEIGHT = 100;
+
+    // Height of the title bar (buildTitleBar()) and width of its Close
+    // button - xr_input.c's TITLE_BAR_V_FRACTION/CLOSE_BUTTON_U_FRACTION
+    // must be kept in sync with these (as fractions of WIDTH/HEIGHT above),
+    // since the drag-handle-vs-Close-button-vs-rest-of-the-keyboard
+    // decision happens entirely on the native side, before a hit ever
+    // reaches this class's own touch dispatch.
+    private static final int TITLE_BAR_HEIGHT = 90;
+    private static final int CLOSE_BUTTON_WIDTH = 160;
+
+    // ~70% opaque - readable but lets whatever's behind the panel (the game
+    // quad, or empty space) show through while it's left up during play.
+    private static final int PANEL_ALPHA = 180;
+
+    private static KeyboardOverlay instance;
+
+    private TextView typedTextView;
+    private final StringBuilder typedText = new StringBuilder();
+
+    // Shift is a local-only layer toggle (never itself sent to the game -
+    // see updateShiftVisual()); Ctrl/Alt are one-shot "armed" modifiers,
+    // sent to the game the moment they're pressed and released again as
+    // soon as the next key consumes them (see toggleModifier()/
+    // releaseArmedModifiers()) - there's no press-and-hold gesture with a
+    // laser pointer + trigger click, so armed-then-consumed stands in for
+    // holding the key down.
+    private boolean shiftActive = false;
+    private boolean ctrlArmed = false;
+    private boolean altArmed = false;
+    private Button shiftButton;
+    private Button ctrlButton;
+    private Button altButton;
+
+    // Every CharKey created by addRow(), so updateShiftVisual() can relabel
+    // all of them at once when Shift toggles. Assigned (not a field
+    // initializer) at the top of buildContent() - see OverlayPanel's
+    // constructor note on why a field a subclass's buildContent() depends
+    // on can't rely on normal field-initializer timing.
+    private List<CharKey> charKeys;
+
+    // A key whose printed character (and, while unmodified, whose game
+    // input) depends on the Shift layer - covers letters, digits, and
+    // punctuation uniformly (e.g. 'q'/'Q', '1'/'!', '-'/'_'). androidKeyCode
+    // is only used for the Ctrl/Alt-combo path (see handleCharKeyPress()) -
+    // the unmodified path goes through commitPrintableChar() instead, which
+    // doesn't need an Android keycode at all.
+    private static final class CharKey {
+        final char base;
+        final char shifted;
+        final int androidKeyCode;
+        Button button;
+
+        CharKey(char base, char shifted, int androidKeyCode) {
+            this.base = base;
+            this.shifted = shifted;
+            this.androidKeyCode = androidKeyCode;
+        }
+    }
+
+    private static CharKey charKey(char base, char shifted, int androidKeyCode) {
+        return new CharKey(base, shifted, androidKeyCode);
+    }
+
+    // Space is a CharKey like any other (see addRow()), but a literal
+    // space character makes for an unreadable, blank-looking button - keep
+    // showing the word "Space" instead, same as before it became a CharKey.
+    private static String charKeyLabel(CharKey key, boolean shiftActive) {
+        if (key.base == ' ') {
+            return "Space";
+        }
+        return String.valueOf(shiftActive ? key.shifted : key.base);
+    }
+
+    private KeyboardOverlay(Activity activity) {
+        super(activity, WIDTH, HEIGHT);
+    }
+
+    @Override
+    protected int contentAlpha() {
+        return PANEL_ALPHA;
+    }
+
+    @Override
+    protected View buildContent() {
+        charKeys = new ArrayList<>();
+
+        LinearLayout panel = new LinearLayout(activity);
+        panel.setOrientation(LinearLayout.VERTICAL);
+        panel.setBackgroundColor(0xFF202020);
+
+        panel.addView(buildTitleBar(),
+            new LinearLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT, TITLE_BAR_HEIGHT));
+
+        typedTextView = new TextView(activity);
+        typedTextView.setTextColor(0xFFFFFFFF);
+        typedTextView.setTextSize(TypedValue.COMPLEX_UNIT_SP, 28);
+        typedTextView.setBackgroundColor(0xFF303030);
+        typedTextView.setPadding(20, 20, 20, 20);
+        panel.addView(typedTextView, new LinearLayout.LayoutParams(
+            ViewGroup.LayoutParams.MATCH_PARENT, TYPED_TEXT_HEIGHT));
+
+        // Rows follow real keyboard geometry where it helps (Tab starting
+        // the QWERTY row, Enter ending the home row, Shift starting the
+        // bottom-letter row, a compact arrow cluster bottom-right) so the
+        // layout reads as a familiar keyboard rather than an arbitrary grid.
+        addRow(panel, "Esc", "F1", "F2", "F3", "F4", "F5", "F6", "F7", "F8", "F9", "F10",
+            "F11", "F12", "Back");
+        addRow(panel,
+            charKey('`', '~', KeyEvent.KEYCODE_GRAVE),
+            charKey('1', '!', KeyEvent.KEYCODE_1),
+            charKey('2', '@', KeyEvent.KEYCODE_2),
+            charKey('3', '#', KeyEvent.KEYCODE_3),
+            charKey('4', '$', KeyEvent.KEYCODE_4),
+            charKey('5', '%', KeyEvent.KEYCODE_5),
+            charKey('6', '^', KeyEvent.KEYCODE_6),
+            charKey('7', '&', KeyEvent.KEYCODE_7),
+            charKey('8', '*', KeyEvent.KEYCODE_8),
+            charKey('9', '(', KeyEvent.KEYCODE_9),
+            charKey('0', ')', KeyEvent.KEYCODE_0),
+            charKey('-', '_', KeyEvent.KEYCODE_MINUS),
+            charKey('=', '+', KeyEvent.KEYCODE_EQUALS));
+        addRow(panel, "Tab",
+            charKey('q', 'Q', KeyEvent.KEYCODE_Q), charKey('w', 'W', KeyEvent.KEYCODE_W),
+            charKey('e', 'E', KeyEvent.KEYCODE_E), charKey('r', 'R', KeyEvent.KEYCODE_R),
+            charKey('t', 'T', KeyEvent.KEYCODE_T), charKey('y', 'Y', KeyEvent.KEYCODE_Y),
+            charKey('u', 'U', KeyEvent.KEYCODE_U), charKey('i', 'I', KeyEvent.KEYCODE_I),
+            charKey('o', 'O', KeyEvent.KEYCODE_O), charKey('p', 'P', KeyEvent.KEYCODE_P),
+            charKey('[', '{', KeyEvent.KEYCODE_LEFT_BRACKET),
+            charKey(']', '}', KeyEvent.KEYCODE_RIGHT_BRACKET),
+            charKey('\\', '|', KeyEvent.KEYCODE_BACKSLASH));
+        addRow(panel, "Ctrl",
+            charKey('a', 'A', KeyEvent.KEYCODE_A), charKey('s', 'S', KeyEvent.KEYCODE_S),
+            charKey('d', 'D', KeyEvent.KEYCODE_D), charKey('f', 'F', KeyEvent.KEYCODE_F),
+            charKey('g', 'G', KeyEvent.KEYCODE_G), charKey('h', 'H', KeyEvent.KEYCODE_H),
+            charKey('j', 'J', KeyEvent.KEYCODE_J), charKey('k', 'K', KeyEvent.KEYCODE_K),
+            charKey('l', 'L', KeyEvent.KEYCODE_L),
+            charKey(';', ':', KeyEvent.KEYCODE_SEMICOLON),
+            charKey('\'', '"', KeyEvent.KEYCODE_APOSTROPHE),
+            "Enter");
+        addRow(panel, "Shift",
+            charKey('z', 'Z', KeyEvent.KEYCODE_Z), charKey('x', 'X', KeyEvent.KEYCODE_X),
+            charKey('c', 'C', KeyEvent.KEYCODE_C), charKey('v', 'V', KeyEvent.KEYCODE_V),
+            charKey('b', 'B', KeyEvent.KEYCODE_B), charKey('n', 'N', KeyEvent.KEYCODE_N),
+            charKey('m', 'M', KeyEvent.KEYCODE_M),
+            charKey(',', '<', KeyEvent.KEYCODE_COMMA),
+            charKey('.', '>', KeyEvent.KEYCODE_PERIOD),
+            charKey('/', '?', KeyEvent.KEYCODE_SLASH),
+            "Up");
+        addRow(panel, "Alt", charKey(' ', ' ', KeyEvent.KEYCODE_SPACE), "Left", "Down", "Right");
+        return panel;
+    }
+
+    // The title bar: a decorative "drag here" label (not itself
+    // interactive - see the class doc, dragging is handled entirely by
+    // xr_input.c before a hit ever reaches here) plus a real Close button,
+    // which is dispatched as an ordinary click like any other key. Its
+    // height and the Close button's width must stay in sync with
+    // xr_input.c's TITLE_BAR_V_FRACTION/CLOSE_BUTTON_U_FRACTION.
+    private LinearLayout buildTitleBar() {
+        LinearLayout bar = new LinearLayout(activity);
+        bar.setOrientation(LinearLayout.HORIZONTAL);
+        bar.setBackgroundColor(0xFF3A3A3A);
+
+        TextView label = new TextView(activity);
+        label.setText("Keyboard  -  drag here to move");
+        label.setTextColor(0xFFFFFFFF);
+        label.setTextSize(TypedValue.COMPLEX_UNIT_SP, 22);
+        label.setGravity(Gravity.CENTER_VERTICAL);
+        label.setPadding(24, 0, 0, 0);
+        bar.addView(label, new LinearLayout.LayoutParams(
+            0, ViewGroup.LayoutParams.MATCH_PARENT, 1f));
+
+        Button close = new Button(activity);
+        close.setText("Close");
+        close.setOnClickListener(v -> handleKeyPress("Close"));
+        close.setOnTouchListener(CLICK_ON_TOUCH_UP);
+        bar.addView(close,
+            new LinearLayout.LayoutParams(CLOSE_BUTTON_WIDTH, ViewGroup.LayoutParams.MATCH_PARENT));
+
+        return bar;
+    }
+
+    // Each row fills the panel's remaining height evenly (weight 1 on the
+    // row itself). Accepts a mix of String (a fixed-action control key,
+    // handled by handleKeyPress()) and CharKey (a Shift-sensitive character
+    // key, handled by handleCharKeyPress()) per item, so a row can match
+    // real keyboard geometry (e.g. "Tab" followed by a run of CharKeys).
+    // Within a row, CharKeys and most control keys share width equally
+    // except Space (wider, like a real keyboard) and Esc/Back/Enter/Tab/
+    // Shift/Ctrl/Alt (narrower, see keyWeight()).
+    private void addRow(LinearLayout panel, Object... items) {
+        LinearLayout row = new LinearLayout(activity);
+        row.setOrientation(LinearLayout.HORIZONTAL);
+        for (Object item : items) {
+            Button key = new Button(activity);
+            key.setOnTouchListener(CLICK_ON_TOUCH_UP);
+            float weight;
+            if (item instanceof CharKey) {
+                CharKey charKey = (CharKey) item;
+                charKey.button = key;
+                key.setText(charKeyLabel(charKey, shiftActive));
+                key.setOnClickListener(v -> handleCharKeyPress(charKey));
+                charKeys.add(charKey);
+                // Space is a CharKey too (base == shifted == ' '), but keeps
+                // its traditional wide key like the other control keys do.
+                weight = (charKey.base == ' ') ? 4f : 1f;
+            } else {
+                String label = (String) item;
+                key.setText(label);
+                key.setOnClickListener(v -> handleKeyPress(label));
+                weight = keyWeight(label);
+                if (label.equals("Shift")) {
+                    shiftButton = key;
+                } else if (label.equals("Ctrl")) {
+                    ctrlButton = key;
+                } else if (label.equals("Alt")) {
+                    altButton = key;
+                }
+            }
+            row.addView(key, new LinearLayout.LayoutParams(
+                0, ViewGroup.LayoutParams.MATCH_PARENT, weight));
+        }
+        panel.addView(row, new LinearLayout.LayoutParams(
+            ViewGroup.LayoutParams.MATCH_PARENT, 0, 1f));
+    }
+
+    private static float keyWeight(String label) {
+        if (label.equals("Back") || label.equals("Esc") || label.equals("Enter")
+                || label.equals("Tab") || label.equals("Shift") || label.equals("Ctrl")
+                || label.equals("Alt")) {
+            return 1.5f;
+        }
+        return 1f;
+    }
+
+    // Maps a control key's label to the Android keycode forwarded via
+    // SDLActivity.onNativeKeyDown()/onNativeKeyUp() - the same pair a
+    // physical Bluetooth keyboard's key events already drive (see
+    // SDLActivity.handleKeyEvent()). F-keys are contiguous in
+    // android.view.KeyEvent (KEYCODE_F1..KEYCODE_F12), so "F1".."F12" are
+    // computed rather than listed individually.
+    private static int controlKeyCode(String label) {
+        switch (label) {
+            case "Back": return KeyEvent.KEYCODE_DEL;
+            case "Esc": return KeyEvent.KEYCODE_ESCAPE;
+            case "Enter": return KeyEvent.KEYCODE_ENTER;
+            case "Tab": return KeyEvent.KEYCODE_TAB;
+            case "Up": return KeyEvent.KEYCODE_DPAD_UP;
+            case "Down": return KeyEvent.KEYCODE_DPAD_DOWN;
+            case "Left": return KeyEvent.KEYCODE_DPAD_LEFT;
+            case "Right": return KeyEvent.KEYCODE_DPAD_RIGHT;
+            default:
+                if (label.charAt(0) == 'F') {
+                    int n = Integer.parseInt(label.substring(1));
+                    return KeyEvent.KEYCODE_F1 + (n - 1);
+                }
+                throw new IllegalArgumentException("no keycode for " + label);
+        }
+    }
+
+    // "Close" (the title bar's button - see buildTitleBar()) hides this
+    // overlay's quad entirely (see nativeRequestClose()) and never reaches
+    // the game. Shift/Ctrl/Alt are handled separately below (Shift is a
+    // local layer toggle; Ctrl/Alt are one-shot armed modifiers) since
+    // neither sends a plain key event of its own the way every other
+    // control key here does. Everything else (Back/Esc/Enter/Tab/arrows/
+    // F1-F12) is non-printable, so it's forwarded as a real Android key
+    // event via controlKeyCode() above, then releases any armed Ctrl/Alt
+    // modifier (this key just consumed it). This all matters beyond just
+    // the visible typedTextView below: the engine's cutscene skip handler
+    // (cutsloop.c's cutscene_key_handler()) only reacts to Esc/Enter/Space,
+    // so those need to reach the game as actual game input, not just be
+    // echoed locally.
+    private void handleKeyPress(String label) {
+        if (label.equals("Close")) {
+            nativeRequestClose();
+            return;
+        }
+        if (label.equals("Shift")) {
+            shiftActive = !shiftActive;
+            updateShiftVisual();
+            return;
+        }
+        if (label.equals("Ctrl") || label.equals("Alt")) {
+            toggleModifier(label);
+            return;
+        }
+
+        int keyCode = controlKeyCode(label);
+        SDLActivity.onNativeKeyDown(keyCode);
+        SDLActivity.onNativeKeyUp(keyCode);
+        releaseArmedModifiers();
+
+        if (label.equals("Back")) {
+            if (typedText.length() > 0) {
+                typedText.setLength(typedText.length() - 1);
+            }
+        } else {
+            typedText.append('[').append(label).append(']');
+        }
+        typedTextView.setText(typedText.toString());
+    }
+
+    // A CharKey's printed character depends only on the Shift layer
+    // (updateShiftVisual() keeps its button's label in sync). Which game
+    // input it produces depends on whether a modifier is armed: unmodified,
+    // it goes through commitPrintableChar() below like a normal typed
+    // character; with Ctrl/Alt armed, it instead goes through the same
+    // onNativeKeyDown()/onNativeKeyUp() path handleKeyPress() uses for
+    // control keys, since that's the only path that carries
+    // ev.key.keysym.mod through to the engine (sdl_events.c's pump_events()
+    // reads Ctrl/Alt only off that path, never off SDL_TEXTINPUT).
+    private void handleCharKeyPress(CharKey key) {
+        char c = shiftActive ? key.shifted : key.base;
+        if (ctrlArmed || altArmed) {
+            SDLActivity.onNativeKeyDown(key.androidKeyCode);
+            SDLActivity.onNativeKeyUp(key.androidKeyCode);
+            releaseArmedModifiers();
+        } else {
+            commitPrintableChar(c);
+        }
+        typedText.append(c);
+        typedTextView.setText(typedText.toString());
+    }
+
+    // Ctrl/Alt are "armed" rather than held: pressing one immediately sends
+    // its keydown (so SDL's own modifier tracking picks it up for whatever
+    // key comes next - see handleCharKeyPress()/handleKeyPress()) and
+    // brackets its label for feedback; pressing the same key again before
+    // it's been used cancels it (sends the matching keyup, un-brackets).
+    // The normal case - actually being consumed by the next key press - is
+    // handled by releaseArmedModifiers() below, not here.
+    private void toggleModifier(String label) {
+        boolean ctrl = label.equals("Ctrl");
+        Button button = ctrl ? ctrlButton : altButton;
+        int keyCode = ctrl ? KeyEvent.KEYCODE_CTRL_LEFT : KeyEvent.KEYCODE_ALT_LEFT;
+        boolean nowArmed = ctrl ? !ctrlArmed : !altArmed;
+        if (ctrl) {
+            ctrlArmed = nowArmed;
+        } else {
+            altArmed = nowArmed;
+        }
+        if (nowArmed) {
+            SDLActivity.onNativeKeyDown(keyCode);
+        } else {
+            SDLActivity.onNativeKeyUp(keyCode);
+        }
+        button.setText(nowArmed ? "[" + label + "]" : label);
+    }
+
+    // Called after any key actually reaches the game (a control key in
+    // handleKeyPress(), or a CharKey in handleCharKeyPress()) so an armed
+    // Ctrl/Alt only ever applies to the very next key, then releases -
+    // matching a real Ctrl/Alt+key combo's keyup once the combo is done.
+    private void releaseArmedModifiers() {
+        if (ctrlArmed) {
+            SDLActivity.onNativeKeyUp(KeyEvent.KEYCODE_CTRL_LEFT);
+            ctrlArmed = false;
+            ctrlButton.setText("Ctrl");
+        }
+        if (altArmed) {
+            SDLActivity.onNativeKeyUp(KeyEvent.KEYCODE_ALT_LEFT);
+            altArmed = false;
+            altButton.setText("Alt");
+        }
+    }
+
+    // Shift never itself reaches the game (see class doc) - it only flips
+    // which character each CharKey shows/sends, including its own label.
+    // Note this means a Ctrl+Shift+key chord won't carry Shift to the
+    // engine (Ctrl/Alt are real forwarded modifier keys, Shift here isn't) -
+    // not worth solving unless it actually comes up.
+    private void updateShiftVisual() {
+        shiftButton.setText(shiftActive ? "[Shift]" : "Shift");
+        for (CharKey key : charKeys) {
+            key.button.setText(charKeyLabel(key, shiftActive));
+        }
+    }
+
+    // Synthesizes the SDL_TEXTINPUT event pump_events() (sdl_events.c)
+    // requires for printable characters (see handleCharKeyPress() above),
+    // by building and pushing it directly in native code (see
+    // nativeSendPrintableChar() in questshock_native.c) rather than through
+    // Android's IME (there's no real IME session behind this off-screen,
+    // never-attached grid for that plumbing to hook into).
+    private static void commitPrintableChar(char c) {
+        nativeSendPrintableChar(c);
+    }
+
+    private static native void nativeSendPrintableChar(char c);
+
+    // Hides this overlay's quad (xr_overlay_set_visible(..., false)) - the
+    // only way to do so, since the controller's menu button doesn't affect
+    // the keyboard (see this class's doc comment and xr_input.c).
+    private static native void nativeRequestClose();
+
+    // Called from xr_overlay.c's xr_overlay_init_jni(), on the render
+    // thread, right after it resolves this class via the activity's own
+    // ClassLoader (plain FindClass() can't see app classes from a thread
+    // that was attached to the JVM rather than spawned from Java).
+    public static void nativeInit(Activity activity) {
+        if (instance == null) {
+            instance = new KeyboardOverlay(activity);
+        }
+    }
+
+    public static byte[] nativeTakePixelsIfDirty() {
+        return instance == null ? null : instance.takePixelsIfDirty();
+    }
+
+    public static void nativeDispatchTouch(final float u, final float v, final boolean down) {
+        if (instance != null) {
+            instance.dispatchTouch(u, v, down);
+        }
+    }
+
+    public static void nativeUpdateCursor(final float u, final float v, final boolean visible) {
+        if (instance != null) {
+            instance.dispatchUpdateCursor(u, v, visible);
+        }
+    }
+}

android/app/src/main/java/de/ladkau/questshock/MenuOverlay.java

@@ -1,380 +1,76 @@
 package de.ladkau.questshock;
 
 import android.app.Activity;
-import android.graphics.Bitmap;
-import android.graphics.Canvas;
-import android.graphics.Paint;
-import android.os.SystemClock;
-import android.util.TypedValue;
 import android.view.Gravity;
-import android.view.KeyEvent;
-import android.view.MotionEvent;
 import android.view.View;
-import android.view.ViewGroup;
 import android.widget.Button;
 import android.widget.FrameLayout;
-import android.widget.LinearLayout;
-import android.widget.TextView;
-import java.nio.ByteBuffer;
-import org.libsdl.app.SDLActivity;
 
 /**
- * An off-screen, never-attached-to-a-window View tree for questshock's
+ * The small main menu launcher quad (see OverlayPanel for the shared off-
- * OpenXR menu quad (see android/app/src/main/cpp/xr_menu.c, which owns the
+ * screen-render/touch/cursor plumbing this builds on) - just a "Keyboard"
- * quad's swapchain and drives this class entirely via JNI - construction,
+ * button for now. Toggled by the controller's menu button (see
- * touch input, and pixel readback). Rendering to a Bitmap and dispatching
+ * xr_input.c); its "Keyboard" click only opens the (fully independent)
- * synthetic MotionEvents into an unattached hierarchy both work the same
+ * keyboard overlay quad via nativeShowKeyboard() - it never touches this
- * way they would for an attached View - draw(Canvas)/dispatchTouchEvent()
+ * panel's own visibility, so both can be shown together (see
- * don't require a ViewRootImpl/window, just a measured+laid-out tree.
+ * KeyboardOverlay for the keyboard itself).
- *
- * Holds two same-size panels inside root, only one of which is visible at a
- * time (see showPanel()): mainPanel (just the "Keyboard" button so far) and
- * keyboardPanel (a hand-built key grid - there's no system IME to borrow
- * once immersive - plus a text field echoing what's been sent). Each key
- * press is forwarded straight to the game as real input (see
- * handleKeyPress()) - non-printable keys as an SDLActivity.onNativeKeyDown()/
- * onNativeKeyUp() pair, the same one a physical Bluetooth keyboard's presses
- * already drive, and printable keys via a synthesized SDL_TEXTINPUT event
- * pushed from native code (see commitPrintableChar()/
- * nativeSendPrintableChar() in questshock_native.c) - this keyboard is
- * meant as a general stand-in for whatever keyboard-driven functionality
- * isn't (yet, or ever) mapped onto the controllers, not just a text-entry
- * widget.
  */
-public class MenuOverlay {
+public class MenuOverlay extends OverlayPanel {
-    // Matches xr_menu.c's MENU_WIDTH/MENU_HEIGHT swapchain size - fixed,
+    // Matches xr_session.c's MENU_WIDTH/MENU_HEIGHT swapchain size - fixed,
     // not tied to any real display metric, since this tree is never shown
     // in a real window.
-    static final int WIDTH = 1024;
+    private static final int WIDTH = 512;
-    static final int HEIGHT = 768;
+    private static final int HEIGHT = 256;
-
-    private static final float CURSOR_RADIUS = 10f;
-    private static final int TYPED_TEXT_HEIGHT = 100;
-
-    // Button's own click handling (View.onTouchEvent()'s ACTION_UP case)
-    // calls View.post(mPerformClick) rather than invoking performClick()
-    // directly; post() queues the runnable to run once the view is attached
-    // to a window and returns true immediately even when unattached, so on
-    // this permanently-unattached tree that queued click silently never
-    // fires - dispatchTouchEvent() still delivers the down/up events
-    // correctly, only the click callback is swallowed. Registering this as
-    // each button's OnTouchListener bypasses that path entirely: returning
-    // true here skips View's internal onTouchEvent() (see
-    // ViewGroup/View#dispatchTouchEvent), so performClick() (which still
-    // runs any OnClickListener set via setOnClickListener()) is called
-    // directly instead.
-    private static final View.OnTouchListener CLICK_ON_TOUCH_UP = (v, event) -> {
-        switch (event.getAction()) {
-            case MotionEvent.ACTION_DOWN:
-                v.setPressed(true);
-                break;
-            case MotionEvent.ACTION_UP:
-                v.setPressed(false);
-                v.performClick();
-                break;
-            case MotionEvent.ACTION_CANCEL:
-                v.setPressed(false);
-                break;
-        }
-        return true;
-    };
 
     private static MenuOverlay instance;
 
-    private final Activity activity;
-    private final FrameLayout root;
-    private final FrameLayout mainPanel;
-    private final LinearLayout keyboardPanel;
-    private final TextView typedTextView;
-    private final StringBuilder typedText = new StringBuilder();
-    // contentBitmap holds just the panel's own rendered content (no cursor),
-    // redrawn only when that content actually changes (showPanel()/
-    // handleTouch()/handleKeyPress()) via root.draw() - a full off-screen
-    // View-tree traversal that gets noticeably more expensive on the
-    // busier keyboardPanel. bitmap is what's actually published to native
-    // (see nativeTakePixelsIfDirty()): compositeAndPublish() cheaply blits
-    // contentBitmap plus the cursor circle into it. Splitting these two
-    // apart matters because nativeUpdateCursor() below fires every single
-    // frame (~90Hz) while aiming at the menu - if it called the expensive
-    // root.draw() path every time (as an earlier version did), the
-    // keyboardPanel's larger view tree made each redraw slow enough that
-    // runOnUiThread() posts piled up faster than the UI thread could drain
-    // them, so the cursor visibly lagged minutes behind the controller's
-    // actual aim instead of tracking it.
-    private final Bitmap contentBitmap;
-    private final Canvas contentCanvas;
-    private final Bitmap bitmap;
-    private final Canvas canvas;
-    private final Paint cursorPaint;
-
-    private final Object pixelLock = new Object();
-    private byte[] pendingPixels;
-
-    // Only ever touched on the UI thread (both nativeUpdateCursor() and
-    // compositeAndPublish() run/are posted there) - no lock needed, unlike
-    // pendingPixels above.
-    private boolean cursorVisible = false;
-    private float cursorX = 0f;
-    private float cursorY = 0f;
-
     private MenuOverlay(Activity activity) {
-        this.activity = activity;
+        super(activity, WIDTH, HEIGHT);
-
-        root = new FrameLayout(activity);
-        root.setBackgroundColor(0xFF202020);
-
-        mainPanel = buildMainPanel();
-        keyboardPanel = buildKeyboardPanel();
-        // buildKeyboardPanel() adds the text field as keyboardPanel's first
-        // child, before any key rows.
-        typedTextView = (TextView) keyboardPanel.getChildAt(0);
-        root.addView(mainPanel, new FrameLayout.LayoutParams(WIDTH, HEIGHT));
-        root.addView(keyboardPanel, new FrameLayout.LayoutParams(WIDTH, HEIGHT));
-
-        contentBitmap = Bitmap.createBitmap(WIDTH, HEIGHT, Bitmap.Config.ARGB_8888);
-        contentCanvas = new Canvas(contentBitmap);
-        bitmap = Bitmap.createBitmap(WIDTH, HEIGHT, Bitmap.Config.ARGB_8888);
-        canvas = new Canvas(bitmap);
-
-        cursorPaint = new Paint();
-        cursorPaint.setColor(0xFFFFFFFF);
-        cursorPaint.setAntiAlias(true);
-
-        showPanel(mainPanel);
-        redrawContent();
     }
 
-    private FrameLayout buildMainPanel() {
+    @Override
+    protected View buildContent() {
         FrameLayout panel = new FrameLayout(activity);
+        panel.setBackgroundColor(0xFF202020);
         Button keyboardButton = new Button(activity);
         keyboardButton.setText("Keyboard");
-        keyboardButton.setOnClickListener(v -> showPanel(keyboardPanel));
+        keyboardButton.setOnClickListener(v -> nativeShowKeyboard());
         keyboardButton.setOnTouchListener(CLICK_ON_TOUCH_UP);
-        FrameLayout.LayoutParams lp = new FrameLayout.LayoutParams(420, 140);
+        FrameLayout.LayoutParams lp = new FrameLayout.LayoutParams(300, 100);
         lp.gravity = Gravity.CENTER;
         panel.addView(keyboardButton, lp);
         return panel;
     }
 
-    private LinearLayout buildKeyboardPanel() {
+    // Opens the keyboard overlay quad (questshock_native.c ->
-        LinearLayout panel = new LinearLayout(activity);
+    // xr_overlay_set_visible(xr_session_get_keyboard_overlay(), true)) -
-        panel.setOrientation(LinearLayout.VERTICAL);
+    // see KeyboardOverlay for the panel itself and its own Close button,
+    // the only way to hide it again.
+    private static native void nativeShowKeyboard();
 
-        TextView textView = new TextView(activity);
+    // Called from xr_overlay.c's xr_overlay_init_jni(), on the render
-        textView.setTextColor(0xFFFFFFFF);
+    // thread, right after it resolves this class via the activity's own
-        textView.setTextSize(TypedValue.COMPLEX_UNIT_SP, 28);
+    // ClassLoader (plain FindClass() can't see app classes from a thread
-        textView.setBackgroundColor(0xFF303030);
+    // that was attached to the JVM rather than spawned from Java).
-        textView.setPadding(20, 20, 20, 20);
-        panel.addView(textView, new LinearLayout.LayoutParams(
-            ViewGroup.LayoutParams.MATCH_PARENT, TYPED_TEXT_HEIGHT));
-
-        addKeyRow(panel, "Q", "W", "E", "R", "T", "Y", "U", "I", "O", "P");
-        addKeyRow(panel, "A", "S", "D", "F", "G", "H", "J", "K", "L");
-        addKeyRow(panel, "Z", "X", "C", "V", "B", "N", "M");
-        addKeyRow(panel, "Esc", "Space", "Back", "Enter", "Done");
-        return panel;
-    }
-
-    // Each row fills the panel's remaining height evenly (weight 1 on the
-    // row itself); within a row, most keys share width equally except
-    // "Space" (wider, like a real keyboard) and "Esc"/"Back"/"Enter"/"Done"
-    // (narrower).
-    private void addKeyRow(LinearLayout panel, String... labels) {
-        LinearLayout row = new LinearLayout(activity);
-        row.setOrientation(LinearLayout.HORIZONTAL);
-        for (String label : labels) {
-            Button key = new Button(activity);
-            key.setText(label);
-            key.setOnClickListener(v -> handleKeyPress(label));
-            key.setOnTouchListener(CLICK_ON_TOUCH_UP);
-            row.addView(key, new LinearLayout.LayoutParams(
-                0, ViewGroup.LayoutParams.MATCH_PARENT, keyWeight(label)));
-        }
-        panel.addView(row, new LinearLayout.LayoutParams(
-            ViewGroup.LayoutParams.MATCH_PARENT, 0, 1f));
-    }
-
-    private static float keyWeight(String label) {
-        if (label.equals("Space")) {
-            return 4f;
-        }
-        if (label.equals("Back") || label.equals("Done") || label.equals("Esc")
-                || label.equals("Enter")) {
-            return 1.5f;
-        }
-        return 1f;
-    }
-
-    // "Done" is this overlay's own UI navigation (closes the key grid, no
-    // game-visible effect). Esc/Back/Enter are non-printable, so they're
-    // forwarded as real Android key events via SDLActivity.onNativeKeyDown()/
-    // onNativeKeyUp() - the same pair a physical Bluetooth keyboard's key
-    // events already drive (see SDLActivity.handleKeyEvent()). Every other
-    // key (letters, Space) instead goes through commitPrintableChar() below:
-    // the engine's own pump_events() (sdl_events.c) deliberately ignores
-    // SDL_KEYDOWN for printable ASCII and only reacts to SDL_TEXTINPUT for
-    // those, to avoid double-counting a typed character against a system
-    // IME - so that event has to be synthesized directly (see
-    // commitPrintableChar()). This all matters beyond just the visible
-    // typedTextView below: the engine's cutscene skip handler (cutsloop.c's
-    // cutscene_key_handler()) only reacts to Esc/Enter/Space, so those need
-    // to reach the game as actual game input, not just be echoed locally.
-    private void handleKeyPress(String label) {
-        if (label.equals("Done")) {
-            showPanel(mainPanel);
-            return;
-        }
-
-        if (label.equals("Back")) {
-            SDLActivity.onNativeKeyDown(KeyEvent.KEYCODE_DEL);
-            SDLActivity.onNativeKeyUp(KeyEvent.KEYCODE_DEL);
-            if (typedText.length() > 0) {
-                typedText.setLength(typedText.length() - 1);
-            }
-        } else if (label.equals("Esc") || label.equals("Enter")) {
-            int keyCode = label.equals("Esc") ? KeyEvent.KEYCODE_ESCAPE : KeyEvent.KEYCODE_ENTER;
-            SDLActivity.onNativeKeyDown(keyCode);
-            SDLActivity.onNativeKeyUp(keyCode);
-            typedText.append('[').append(label).append(']');
-        } else if (label.equals("Space")) {
-            commitPrintableChar(' ');
-            typedText.append(' ');
-        } else {
-            commitPrintableChar(Character.toLowerCase(label.charAt(0)));
-            typedText.append(label);
-        }
-        typedTextView.setText(typedText.toString());
-    }
-
-    // Synthesizes the SDL_TEXTINPUT event pump_events() (sdl_events.c)
-    // requires for printable characters (see handleKeyPress() above), by
-    // building and pushing it directly in native code (see
-    // nativeSendPrintableChar() in questshock_native.c) rather than through
-    // Android's IME (there's no real IME session behind this off-screen,
-    // never-attached grid for that plumbing to hook into).
-    private static void commitPrintableChar(char c) {
-        nativeSendPrintableChar(c);
-    }
-
-    private static native void nativeSendPrintableChar(char c);
-
-    // Only one of mainPanel/keyboardPanel is ever visible at a time. Both
-    // were already measured/laid out once in the constructor's initial
-    // showPanel() call - re-running measure()/layout() here (rather than
-    // just flipping visibility) is required every time regardless, since
-    // FrameLayout skips GONE children during measure/layout, so a panel
-    // switching from GONE to VISIBLE needs a fresh pass to get valid bounds
-    // before its content can be drawn or hit-tested.
-    private void showPanel(View panel) {
-        mainPanel.setVisibility(panel == mainPanel ? View.VISIBLE : View.GONE);
-        keyboardPanel.setVisibility(panel == keyboardPanel ? View.VISIBLE : View.GONE);
-        int widthSpec = View.MeasureSpec.makeMeasureSpec(WIDTH, View.MeasureSpec.EXACTLY);
-        int heightSpec = View.MeasureSpec.makeMeasureSpec(HEIGHT, View.MeasureSpec.EXACTLY);
-        root.measure(widthSpec, heightSpec);
-        root.layout(0, 0, WIDTH, HEIGHT);
-    }
-
-    // Called from xr_menu.c's xr_menu_init(), on the render thread, right
-    // after it resolves this class via the activity's own ClassLoader
-    // (plain FindClass() can't see app classes from a thread that was
-    // attached to the JVM rather than spawned from Java - see that file's
-    // xr_menu_find_class()).
     public static void nativeInit(Activity activity) {
         if (instance == null) {
             instance = new MenuOverlay(activity);
         }
     }
 
-    // Polled once per frame while the menu quad is visible (see
-    // xr_menu_render_if_visible()) - returns the current pixels only once
-    // per redraw (null otherwise), so the native side knows when it can
-    // skip re-uploading a texture that hasn't actually changed.
     public static byte[] nativeTakePixelsIfDirty() {
-        if (instance == null) {
+        return instance == null ? null : instance.takePixelsIfDirty();
-            return null;
-        }
-        synchronized (instance.pixelLock) {
-            byte[] pixels = instance.pendingPixels;
-            instance.pendingPixels = null;
-            return pixels;
-        }
     }
 
-    // u/v are the menu quad's own hit-test coordinates (0..1, top-left
-    // origin) - computed by xr_input.c's ray/quad intersection against
-    // xr_menu.c's reported quad geometry, forwarded here as a synthetic
-    // tap. Runs on the UI thread since the View/Bitmap/Canvas objects here
-    // are otherwise only ever touched from there.
     public static void nativeDispatchTouch(final float u, final float v, final boolean down) {
-        if (instance == null) {
+        if (instance != null) {
-            return;
+            instance.dispatchTouch(u, v, down);
         }
-        instance.activity.runOnUiThread(() -> instance.handleTouch(u, v, down));
     }
 
-    // u/v are the same menu-quad hit-test coordinates nativeDispatchTouch()
-    // uses, but polled once per frame regardless of click state (see
-    // xr_menu_update_cursor()) rather than only on click edges - lets the
-    // cursor track the aim ray continuously instead of only jumping when a
-    // trigger is pressed. visible=false (no hand's ray currently on the
-    // quad) hides it.
     public static void nativeUpdateCursor(final float u, final float v, final boolean visible) {
-        if (instance == null) {
+        if (instance != null) {
-            return;
+            instance.dispatchUpdateCursor(u, v, visible);
-        }
-        instance.activity.runOnUiThread(() -> instance.updateCursor(u, v, visible));
-    }
-
-    private void updateCursor(float u, float v, boolean visible) {
-        float x = u * WIDTH;
-        float y = v * HEIGHT;
-        if (visible == cursorVisible && x == cursorX && y == cursorY) {
-            return;
-        }
-        cursorVisible = visible;
-        cursorX = x;
-        cursorY = y;
-        compositeAndPublish();
-    }
-
-    private void handleTouch(float u, float v, boolean down) {
-        float x = u * WIDTH;
-        float y = v * HEIGHT;
-        long time = SystemClock.uptimeMillis();
-        MotionEvent event = MotionEvent.obtain(
-            time, time, down ? MotionEvent.ACTION_DOWN : MotionEvent.ACTION_UP, x, y, 0);
-        try {
-            root.dispatchTouchEvent(event);
-        } finally {
-            event.recycle();
-        }
-        redrawContent();
-    }
-
-    // Re-runs the full off-screen View-tree draw (expensive - see the
-    // contentBitmap field comment above) - call only when the panel's
-    // actual content changed, not for the cursor-only updates
-    // compositeAndPublish() below handles on its own.
-    private void redrawContent() {
-        contentCanvas.drawColor(0xFF202020);
-        root.draw(contentCanvas);
-        compositeAndPublish();
-    }
-
-    // Cheap per-frame path: blits the last-rendered contentBitmap (no View
-    // traversal) plus the cursor circle into bitmap and publishes it.
-    private void compositeAndPublish() {
-        canvas.drawBitmap(contentBitmap, 0, 0, null);
-        if (cursorVisible) {
-            canvas.drawCircle(cursorX, cursorY, CURSOR_RADIUS, cursorPaint);
-        }
-
-        byte[] pixels = new byte[WIDTH * HEIGHT * 4];
-        // ARGB_8888's actual in-memory byte order is R,G,B,A - matches
-        // GL_RGBA/GL_UNSIGNED_BYTE on the native side with no swizzling.
-        bitmap.copyPixelsToBuffer(ByteBuffer.wrap(pixels));
-        synchronized (pixelLock) {
-            pendingPixels = pixels;
         }
     }
 }

android/app/src/main/java/de/ladkau/questshock/OverlayPanel.java

@@ -0,0 +1,237 @@
+package de.ladkau.questshock;
+
+import android.app.Activity;
+import android.graphics.Bitmap;
+import android.graphics.Canvas;
+import android.graphics.Paint;
+import android.os.SystemClock;
+import android.view.MotionEvent;
+import android.view.View;
+import java.nio.ByteBuffer;
+
+/**
+ * Shared machinery for an off-screen, never-attached-to-a-window View tree
+ * rendered into its own OpenXR quad (see android/app/src/main/cpp/
+ * xr_overlay.c, which owns the quad's swapchain and drives whichever
+ * concrete subclass via JNI - construction, touch input, and pixel
+ * readback). Rendering to a Bitmap and dispatching synthetic MotionEvents
+ * into an unattached hierarchy both work the same way they would for an
+ * attached View - draw(Canvas)/dispatchTouchEvent() don't require a
+ * ViewRootImpl/window, just a measured+laid-out tree.
+ *
+ * Subclasses (MenuOverlay, KeyboardOverlay) provide their own content via
+ * buildContent() and, if they want translucency, contentAlpha() - and, since
+ * Java has no static virtual dispatch and each overlay needs its own
+ * singleton, each subclass still declares its own thin nativeInit()/
+ * nativeTakePixelsIfDirty()/nativeDispatchTouch()/nativeUpdateCursor()
+ * static methods (matching what xr_overlay.c's JNI glue resolves by class
+ * name) that just delegate into the instance methods here
+ * (takePixelsIfDirty()/dispatchTouch()/dispatchUpdateCursor()).
+ */
+abstract class OverlayPanel {
+    private static final float CURSOR_RADIUS = 10f;
+
+    // Button's own click handling (View.onTouchEvent()'s ACTION_UP case)
+    // calls View.post(mPerformClick) rather than invoking performClick()
+    // directly; post() queues the runnable to run once the view is attached
+    // to a window and returns true immediately even when unattached, so on
+    // this permanently-unattached tree that queued click silently never
+    // fires - dispatchTouchEvent() still delivers the down/up events
+    // correctly, only the click callback is swallowed. Registering this as
+    // each button's OnTouchListener bypasses that path entirely: returning
+    // true here skips View's internal onTouchEvent() (see
+    // ViewGroup/View#dispatchTouchEvent), so performClick() (which still
+    // runs any OnClickListener set via setOnClickListener()) is called
+    // directly instead.
+    protected static final View.OnTouchListener CLICK_ON_TOUCH_UP = (v, event) -> {
+        switch (event.getAction()) {
+            case MotionEvent.ACTION_DOWN:
+                v.setPressed(true);
+                break;
+            case MotionEvent.ACTION_UP:
+                v.setPressed(false);
+                v.performClick();
+                break;
+            case MotionEvent.ACTION_CANCEL:
+                v.setPressed(false);
+                break;
+        }
+        return true;
+    };
+
+    protected final Activity activity;
+    private final int width;
+    private final int height;
+    private final View root;
+
+    // contentBitmap holds just the panel's own rendered content (no cursor),
+    // redrawn only when that content actually changes (redrawContent(), via
+    // handleTouch() or a subclass's own key handling) via root.draw() - a
+    // full off-screen View-tree traversal that gets noticeably more
+    // expensive on a busier content tree. bitmap is what's actually
+    // published to native (see takePixelsIfDirty()): compositeAndPublish()
+    // cheaply blits contentBitmap plus the cursor circle into it. Splitting
+    // these two apart matters because updateCursor() below fires every
+    // single frame (~90Hz) while aiming at the panel - if it called the
+    // expensive root.draw() path every time (as an earlier version did), a
+    // busy content tree made each redraw slow enough that runOnUiThread()
+    // posts piled up faster than the UI thread could drain them, so the
+    // cursor visibly lagged behind the controller's actual aim instead of
+    // tracking it.
+    private final Bitmap contentBitmap;
+    private final Canvas contentCanvas;
+    private final Bitmap bitmap;
+    private final Canvas canvas;
+    private final Paint cursorPaint;
+    // Draws contentBitmap at contentAlpha() - see compositeAndPublish().
+    private final Paint contentAlphaPaint;
+
+    private final Object pixelLock = new Object();
+    private byte[] pendingPixels;
+
+    // Only ever touched on the UI thread (both dispatchUpdateCursor() and
+    // compositeAndPublish() run/are posted there) - no lock needed, unlike
+    // pendingPixels above.
+    private boolean cursorVisible = false;
+    private float cursorX = 0f;
+    private float cursorY = 0f;
+
+    protected OverlayPanel(Activity activity, int width, int height) {
+        this.activity = activity;
+        this.width = width;
+        this.height = height;
+
+        root = buildContent();
+        int widthSpec = View.MeasureSpec.makeMeasureSpec(width, View.MeasureSpec.EXACTLY);
+        int heightSpec = View.MeasureSpec.makeMeasureSpec(height, View.MeasureSpec.EXACTLY);
+        root.measure(widthSpec, heightSpec);
+        root.layout(0, 0, width, height);
+
+        contentBitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
+        contentCanvas = new Canvas(contentBitmap);
+        bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
+        canvas = new Canvas(bitmap);
+
+        cursorPaint = new Paint();
+        cursorPaint.setColor(0xFFFFFFFF);
+        cursorPaint.setAntiAlias(true);
+
+        // contentAlpha() must not depend on any subclass instance state -
+        // it's called here, from the superclass constructor, before the
+        // subclass's own field initializers/constructor body have run (the
+        // usual Java construction order: this class's fields/constructor
+        // first, then the subclass's). A static constant is the only thing
+        // that's safe to return.
+        contentAlphaPaint = new Paint();
+        contentAlphaPaint.setAlpha(contentAlpha());
+
+        redrawContent();
+    }
+
+    // Builds this overlay's content View tree - called once, from this
+    // class's own constructor (see the note on contentAlpha() above: don't
+    // rely on subclass instance fields being initialized yet here; if a
+    // subclass needs its own mutable state available while building its
+    // content, e.g. a list of buttons to later relabel, initialize that
+    // field's value at the top of this method instead of via a field
+    // initializer).
+    protected abstract View buildContent();
+
+    // 0-255; defaults to fully opaque. Override for translucency (see the
+    // constructor's note on what's safe to depend on here).
+    protected int contentAlpha() {
+        return 255;
+    }
+
+    // Polled once per frame while this overlay's quad is visible (see
+    // xr_overlay_render_and_build_layer()) - returns the current pixels
+    // only once per redraw (null otherwise), so the native side knows when
+    // it can skip re-uploading a texture that hasn't actually changed.
+    protected final byte[] takePixelsIfDirty() {
+        synchronized (pixelLock) {
+            byte[] pixels = pendingPixels;
+            pendingPixels = null;
+            return pixels;
+        }
+    }
+
+    // u/v are this quad's own hit-test coordinates (0..1, top-left origin)
+    // - computed by xr_input.c's ray/quad intersection against xr_overlay.c's
+    // reported quad geometry, forwarded here as a synthetic tap. Runs on the
+    // UI thread since the View/Bitmap/Canvas objects here are otherwise only
+    // ever touched from there.
+    protected final void dispatchTouch(final float u, final float v, final boolean down) {
+        activity.runOnUiThread(() -> handleTouch(u, v, down));
+    }
+
+    // u/v are the same hit-test coordinates dispatchTouch() uses, but
+    // polled once per frame regardless of click state rather than only on
+    // click edges - lets the cursor track the aim ray continuously instead
+    // of only jumping when a trigger is pressed. visible=false (no hand's
+    // ray currently on the quad) hides it.
+    protected final void dispatchUpdateCursor(final float u, final float v,
+                                               final boolean visible) {
+        activity.runOnUiThread(() -> updateCursor(u, v, visible));
+    }
+
+    private void updateCursor(float u, float v, boolean visible) {
+        float x = u * width;
+        float y = v * height;
+        if (visible == cursorVisible && x == cursorX && y == cursorY) {
+            return;
+        }
+        cursorVisible = visible;
+        cursorX = x;
+        cursorY = y;
+        compositeAndPublish();
+    }
+
+    private void handleTouch(float u, float v, boolean down) {
+        float x = u * width;
+        float y = v * height;
+        long time = SystemClock.uptimeMillis();
+        MotionEvent event = MotionEvent.obtain(
+            time, time, down ? MotionEvent.ACTION_DOWN : MotionEvent.ACTION_UP, x, y, 0);
+        try {
+            root.dispatchTouchEvent(event);
+        } finally {
+            event.recycle();
+        }
+        redrawContent();
+    }
+
+    // Re-runs the full off-screen View-tree draw (expensive - see the
+    // contentBitmap field comment above) - call only when the panel's
+    // actual content changed, not for the cursor-only updates
+    // compositeAndPublish() below handles on its own.
+    protected final void redrawContent() {
+        contentCanvas.drawColor(0xFF202020);
+        root.draw(contentCanvas);
+        compositeAndPublish();
+    }
+
+    // Cheap per-frame path: blits the last-rendered contentBitmap (no View
+    // traversal) plus the cursor circle into bitmap and publishes it.
+    // contentBitmap itself is fully opaque (redrawContent() draws an opaque
+    // background first); contentAlpha() translucency is applied right here
+    // instead, via contentAlphaPaint - which needs bitmap cleared to fully
+    // transparent first, since otherwise each frame's alpha-blended draw
+    // would blend against whatever was left over in bitmap from the
+    // previous frame rather than against nothing, compounding into a
+    // ghosting trail across frames.
+    private void compositeAndPublish() {
+        bitmap.eraseColor(0);
+        canvas.drawBitmap(contentBitmap, 0, 0, contentAlphaPaint);
+        if (cursorVisible) {
+            canvas.drawCircle(cursorX, cursorY, CURSOR_RADIUS, cursorPaint);
+        }
+
+        byte[] pixels = new byte[width * height * 4];
+        // ARGB_8888's actual in-memory byte order is R,G,B,A - matches
+        // GL_RGBA/GL_UNSIGNED_BYTE on the native side with no swizzling.
+        bitmap.copyPixelsToBuffer(ByteBuffer.wrap(pixels));
+        synchronized (pixelLock) {
+            pendingPixels = pixels;
+        }
+    }
+}