questshock/modding/README.md

# Modding

Tools and reference material for modding System Shock (1994) itself - distinct
from the rest of this repository, which is the [QuestShock](../README.md) VR
port of the game engine. Everything modding-related lives under this folder.

`hacked` and `ss-specs` come from the [InkyBlackness](https://inkyblackness.github.io)
project; `sqweek-dialog` is a separate third-party dependency of `hacked`.
All three are vendored unmodified under `src/`, each under its own upstream
license (see below) - none is relicensed by this repository's own MIT
license. `mdlp` (also under `src/`) is first-party code written for this
repository.

## 1. Layout

- `src/` - every vendored and first-party source tree, kept apart from the
  Docker/script scaffolding that builds them:
  - `hacked/` - [InkyBlackness HackEd](https://github.com/inkyblackness/hacked),
    the graphical mod editor. A vendored source snapshot (New BSD License),
    taken from the `main` branch on 2026-09-26. To update it, replace this
    directory with a fresh checkout and update that date.
  - `hacked-patches/` - numbered `patch -p1` files applied to a scratch copy
    of `hacked/` at build time (same pattern as the root project's
    `android/engine-patches/`, see the repository root's
    [AGENTS.md](../AGENTS.md) section 6) - `hacked/` itself is never edited
    in place. See section 1.1 for what each patch does.
  - `sqweek-dialog/` - `github.com/sqweek/dialog`, `hacked`'s native
    file-dialog dependency. Vendored at the exact version `hacked/go.mod`
    would otherwise pull from the module proxy (ISC License), so it can be
    patched the same way as `hacked/` itself (see section 1.1).
  - `sqweek-dialog-patches/` - patches for `sqweek-dialog/`, same pattern as
    `hacked-patches/` (section 1.1).
  - `mdlp/` - the **Mod Definition Language Processor** (section 4). First-party
    Go source, not vendored - edit it directly, no patch system needed.
  - `ss-specs/` - [InkyBlackness ss-specs](https://github.com/inkyblackness/ss-specs),
    the unofficial file-format specifications the editor (and any other
    modding tool) is written against. Reference documentation only, not
    built. A vendored snapshot (CC BY-NC-SA 4.0 - **non-commercial**, unlike
    the rest of this repository), taken from the `main` branch on
    2026-09-26.
- `hacked-build-image/` - the Dockerfile (and supporting scripts) for the
  `hacked` build environment: the Go toolchain plus every dependency
  `src/hacked/go.mod` pins, fetched once at image-build time so the actual
  build needs no network access. Mirrors the main project's own
  `build-image/` (see the repository root's [README.md](../README.md)
  section 4) one level down, for this subproject.
- `hacked-build-image.sh` / `hacked-build-run-image.sh` /
  `hacked-build-upload-image.sh` - build the image, run it to compile
  `hacked/`, and push it to a registry, respectively. Same pattern as the
  root project's own `build-image.sh`/`run-image.sh`/`upload-image.sh`.
- `mdlp-image/`, `mdlp-image.sh` / `mdlp-run-image.sh` / `mdlp-upload-image.sh` -
  the same pattern again, for `mdlp` (section 4.2).
- `mods/` - mod definitions in the Mod Definition Language (section 4).
- `Makefile` - `hacked` (default) builds `hacked/` via the image;
  `build-image` (re)builds that image; `mdlp-image` builds the `mdlp` image;
  `mods` processes every mod under `mods/` via that image; `clean` removes
  `dist/`.
- `dist/` - build output (git-ignored): `make hacked` under `linux/`/`win/`,
  `make mods` under `mods/`.

Shockolate (the engine `../engine/` vendors) already has its own modding
support (level/resource files, `res/data/`); this folder is about the
external tools that create and inspect that content, not about changing the
engine itself.

### 1.1. Patch reference

`src/hacked-patches/` and `src/sqweek-dialog-patches/` (applied to scratch
copies of `src/hacked/` and `src/sqweek-dialog/` respectively) are both
plain, numbered patch files applied in order at build time - neither
vendored source is ever modified in place. What each one does:

`src/hacked-patches/`:

- **`01-use-patched-dialog.patch`** - adds a `go.mod` `replace` directive
  pointing `github.com/sqweek/dialog` at the local, patched
  `../sqweek-dialog` instead of the upstream module, so `hacked` (and
  anything else building against this source, such as `mdlp`/`animgallery`)
  picks up the crash fix below.
- **`02-fix-hashall-race.patch`** - fixes a real data race in
  `ss1/resource/ChangeNotifier.go`'s `hashAll()`: it hashes many resources
  concurrently (one goroutine per ID) to detect what changed after a
  modification, but only guarded the resource *lookup* with a mutex, not
  the actual block-reading inside `hashResource()` - resources from the
  same underlying file share a stateful reader, so concurrent unguarded
  reads corrupted each other, non-deterministically. Found via `go build
  -race` after `animgallery` produced a different resource count on every
  run against the exact same input. The mutex now guards the whole hash
  computation per resource, not just the lookup.

`src/sqweek-dialog-patches/`:

- **`01-xinitthreads.patch`** - adds `XInitThreads()` before `gtk_init()`
  in the Linux backend. Without it, `hacked` segfaults the moment a native
  GTK3 dialog (file/folder browse) opens while a GLFW/OpenGL context is
  live: GLFW's own X11 backend installs a process-wide Xlib error handler
  that conflicts with GTK's unless Xlib has been told up front that it
  will be used from multiple threads.

## 2. Building `hacked`

Packages `dist/<version>.linux64.tgz` and `dist/<version>.win64.zip` (native
Linux build, Windows amd64 cross-compiled via mingw-w64 - both from the same
source, in one run).

**Prerequisites:** Docker - same as the root project, nothing modding-specific
to install locally.

```sh
cd modding

# 1. Build the hacked-build-image (once, or after hacked-build-image/ or
#    src/hacked/go.mod|go.sum changes)
./hacked-build-image.sh

# 2. Compile hacked/ for Linux and Windows
make hacked
```

`<version>` is `hacked-<date>`, not an InkyBlackness release version:
`hacked/` is a vendored snapshot here, not its own git checkout, so there is
no tag to derive a real version from (see
`hacked-build-image/build-hacked.sh`'s own comment).

Run the result: `dist/linux/hacked/hacked` on Linux,
`dist\win\hacked\hacked.exe` on Windows. Needs OpenGL 3.2; usage is documented
on [hacked's own wiki](https://github.com/inkyblackness/hacked/wiki).

### 2.1. Publishing the build-image

Same registry as the root project's own build-image (see the repository
root's `registry.env.example`), under a separate image name so the two don't
collide:

```sh
./hacked-build-upload-image.sh
```

Reads `REGISTRY`/`REGISTRY_USER`/`REGISTRY_PASSWORD` and (optionally)
`HACKED_REGISTRY_IMAGE` from `../registry.env`, same as the root scripts.
Don't bump `hacked-build-image/VERSION` just because the Dockerfile changed -
only once a build with the current version has actually succeeded, same
policy as the root project's own `build-image/VERSION`.

## 3. Example: Editing a message

`hacked` has no inline text editor - every text field is edited by right-click
-> **Copy to Clipboard**, editing the text in an external editor, then
right-click -> **Copy from Clipboard** back into the field. Example: changing
Rebecca Lansing's first email.

1. Run `dist/linux/hacked/hacked` (or the Windows build).
2. **Window -> Project.** Under **Static World Data**, **Add...** and browse to
   the game's whole `res/assets/ss_ee/data/` directory (see the root
   [README.md](../README.md) section 3 for where that comes from), so every
   `.res`/`.dat` file is available.
3. **Window -> Messages** (F5). **Message Type**: `Mails`. **Index**: `26`
   (Rebecca's first mail). Edit **Sender**/**Subject**/**Title**/the message
   text via the clipboard round-trip above.
4. Back in the **Project** window, under **Mod Location**, click **Save**
   (first time: **Browse...** to an empty folder, for example
   `./modding/mymod`).
5. Test it from the root project's desktop build:
   ```sh
   ./dist/run.sh "$(pwd)/modding/mymod"
   ```
   The mod-folder argument **must be an absolute path**: `run.sh` `cd`s into
   `dist/` before launching the engine, so a relative path resolves against
   the wrong directory and the mod silently fails to load (the engine's
   `ProcessModDirectory()` reports nothing on a bad path). When the path is
   right, the terminal shows `ProcessModDirectory ...` / `Found resource
   file: ...` / `Loading mod file ...` for confirmation.

## 4. Mod Definition Language (mods/)

MDL is a small, declarative JSON format for describing a mod's content -
one file per piece of content (a message override, an upscaled sprite, and
so on), naming the game resource it touches and the values to set, rather
than the resource's actual binary data. `mdlp` (the **Mod Definition
Language Processor**, `mdlp/`) reads these files and produces the same kind
of mod folder `hacked` itself would save, reproducibly and offline. Since a
plain-text description is all that's checked into git, mods stay reviewable
and reproducible from the game's own data alone (see section 4.3) - unlike
the workflow in section 3, whose result only ever lives in whatever you
clicked through in `hacked`.

### 4.1. Layout

```
mods/
  <mod-name>/
    messages/
      <label>.json   # one message's overrides
      <label>.txt    # optional: a text field's body, referenced by ...File
    bitmaps/
      <label>.json   # an AI-upscaled set of critter/object art resources
```

Each `messages/*.json` overrides one message (email, log, or fragment) by
type and index. Only fields present in the file are changed - everything
else is read from the game's own data (Static World Data), the same
inheritance the Messages window in `hacked` shows via its "(read-only)"
label:

| Field | Meaning |
|---|---|
| `type` | `Mails`, `Logs`, or `Fragments` |
| `index` | The message's index within that type |
| `language` | `Default`, `French`, or `German` - omit for `Default` |
| `title`, `sender`, `subject` | Plain strings |
| `verboseText` / `verboseTextFile` | The long body, inline or from a sibling file (avoids escaping newlines/quotes in JSON) |
| `terseText` / `terseTextFile` | The short body, inline or from a sibling file |
| `nextMessage`, `isInterrupt`, `colorIndex`, `leftDisplay`, `rightDisplay` | See `ElectronicMessage` in `src/hacked/ss1/content/text/ElectronicMessage.go` |

`mods/mymod/` is a worked example matching section 3's Rebecca's-mail edit,
expressed as MDL instead of a `hacked` session.

Each `bitmaps/*.json` names a shared upscale configuration and the
resources to apply it to - `gen_animgallery.sh`'s own checkbox-driven MDL
generator (its rendered gallery page has a selection UI that assembles this
exact JSON shape from whatever's checked) produces this format directly:

| Field | Meaning |
|---|---|
| `model` | Which baked-in ONNX model to upscale with - see the table below |
| `scale` | Output size as a multiple of the original (for example `2`) |
| `resources` | Each entry is `"0x<ID>"` (language `Any`) or `"0x<ID>_<Language>"` (for example `"0x25_German"`), matching `gen_animgallery.sh`'s own id format |
| `considerPreviousFrames` | Optional, same id format as `resources` - an id here that isn't also in `resources` is simply never applied, not an error. A listed resource with 3+ frames gets a temporal-consistency pass (section 5) that pulls a frame's upscaled pixel back toward the rest of the animation's own consensus wherever it disagrees by more than the real low-res source justifies - a real mitigation for per-frame upscale flicker. Lets one resource opt in without splitting it into its own file or changing `model`/`scale` for the rest. |

Available `model` values (baked into `mdlp-image`, see
`mdlp-image/upscale_batch.py`):

| `model` | Checkpoint | Best for |
|---|---|---|
| `realesrgan-x4plus-anime` | Real-ESRGAN's 6-block anime/illustration RRDBNet (16.7M params) | Flat-color sprite art - keeps hard edges instead of smoothing them the way a photo/video model would. Default choice. |
| `realesrgan-x4plus` | Real-ESRGAN's general-purpose photo RRDBNet (16.7M params) | A specific resource where the anime model genuinely mis-renders a region (seen on some high-contrast color boundaries) - less edge-sharp overall, much slower, and can mis-render its own different regions, so check the result before keeping it |
| `realesr-general-x4v3` | Real-ESRGAN's compact general-purpose SRVGGNet (1.2M params) | A specific resource neither RRDBNet model renders cleanly - noticeably softer than either, but the most robust against both of their failure modes. Not a blanket replacement, just a last resort for one stubborn sprite |

### 4.2. `mdlp`

`src/mdlp/` is first-party Go source (not vendored - edit it directly, no
patch system needed). It reuses `hacked`'s own resource read/write code
(`ss1/edit`, `ss1/world`, `ss1/content/text`) through a `go.mod` `replace`
pointing at the sibling `../hacked` (both under `src/`), instead of
reimplementing System Shock's file formats. It needs no GUI, GLFW, or GTK -
only the packages that model resource data.

`mdlp-image/` is its own Docker build/runtime environment, separate from
`hacked-build-image/`: mod processing is expected to grow further external
tool dependencies (image/audio conversion, etc.) of its own over time,
independent of `hacked`'s GTK/GLFW/mingw toolchain.

### 4.3. Building mods

```sh
cd modding

# 1. Build the mdlp-image (once, or after mdlp-image/ or
#    src/mdlp/go.mod|go.sum changes)
./mdlp-image.sh

# 2. Process every mod under mods/
make mods

# ...or only some of them:
MODS="mymod" make mods
```

Needs the game's own data directory at `../res/assets/ss_ee/data` (see the
root [README.md](../README.md) section 3) as read-only reference data, so
overrides that touch only some fields of a message can inherit the rest.

Output lands in `dist/mods/<mod-name>/mod/` (the same shape `hacked`'s own
Project window Save produces) for each mod, plus one combined
`dist/questshock-mods.zip` packaging all of them together. Nothing but
`mods/`'s own JSON/text files needs to be checked into git - the game's own
data (already required locally, already git-ignored) plus those files is
always enough to reproduce this output from scratch.

Test every built mod at once with `run-mods.sh` (Linux) / `run-mods.bat`
(Windows), copied into the root project's own `dist/`/`dist-win/` by
`make dist` (root [README.md](../README.md) section 4) alongside the plain
`run.sh`/`run.bat`:

```sh
../dist/run-mods.sh
```

It scans `dist/mods/*/mod` and passes each one it finds to `run.sh` as a
separate, absolute argument - the engine's `ProcessModDirectory()` only scans
one directory level deep (see `res/run-mods.sh`'s own comment), so it can't
simply be pointed at `dist/mods/` itself. To test a single mod instead, use
`run.sh` directly the same way as section 3's step 5:

```sh
../dist/run.sh "$(pwd)/dist/mods/mymod/mod"
```

### 4.4. Publishing the mdlp-image

Same as `hacked-build-image/` (section 2.1): `./mdlp-upload-image.sh` reads
`REGISTRY`/`REGISTRY_USER`/`REGISTRY_PASSWORD` and (optionally)
`MDLP_REGISTRY_IMAGE` from `../registry.env`. Don't bump
`mdlp-image/VERSION` just because the Dockerfile changed - only once a build
with the current version has actually succeeded, same policy as the root
project's own `build-image/VERSION`.

## 5. Testing

Verifying a mod's built output (particularly an AI-upscaled one, section
4.1) means comparing its rendered result against the unmodified base game,
not just eyeballing the result in isolation - a subtly wrong color or a
corrupted frame can look plausible on its own. `gen_animgallery.sh` renders
straight from a built mod's own resource data (`dist/mods/<name>/mod`), the
same bytes the engine loads, so a problem visible there is a problem in the
mod, not an artifact of some other preview path. Regenerate it with `-data
res/assets/ss_ee/data -data dist/mods/<name>/mod` layered over the base
data to see the mod's effect, or with just the base `-data` argument to
check what the unmodified resource looks like for comparison. Each card
shows the looping GIF plus a static filmstrip of every one of its frames
side by side, so one specific frame can be singled out and compared without
having to pause the animation by eye. When two or more `-data` directories
are layered this way, a card for any resource a later layer actually
overrides shows both versions side by side - the first directory's own
("base") and the winning one ("this build") - so a mod's effect is always
checked against the original it's replacing, not recalled from memory or a
separate base-only gallery run. Cards are sectioned by the `-data`
directory they come from - "base game" first, then one section per mod
(named after `dist/mods/<name>`) - so a mod's own animations, for example
all of `upscale_medbot`'s, are listed together.

`gen_animgallery.sh` is a from-scratch Go decoder, not the game engine -
it never exercises the engine's own texture upload/caching code at all.
A clean gallery render rules out a bug in the mod's resource *data*, not a
bug in how the engine *renders* that data - the two are independent, and a
problem can exist in only one of them. Actually running the built mod
(`../dist/run-mods.sh` or `run.sh` with a single mod folder, section 4.3)
is the only way to check the second. This in-engine verification has so
far only been through the desktop build, not the Quest headset - both
read the same resource data and the engine-side bugs found here
(`engine-patches/`, root [README.md](../README.md) section 4's patch
reference) were resource-data-adjacent, not OpenXR-specific, so a fix at
this level should carry over, but it isn't headset-confirmed until tested
there.

The AI-upscale pipeline (`src/mdlp/upscale.go`) and the engine's own
rendering of its output have a failure mode worth specifically checking
for, found by comparing upscaled output against the base game across many
resources rather than trusting a handful of spot-checks:

- **Wrong-hue speckling in dark or muted regions.** SS1's palette has a
  handful of fully-saturated "pure color ramp" entries (pure cyan, red,
  green, ...) that sit deceptively close, in color-distance terms, to
  common dark/muted shading colors - a plain nearest-color search, with or
  without dithering, can pick one of these for a near-black pixel the AI
  upscaler produced, since color distance near black barely penalizes an
  unrelated hue. `isPureRampColor` excludes these entries from
  re-quantization entirely rather than relying on a brightness or
  saturation threshold, which proved too easy for a genuinely
  dark-but-muted pixel to slip past.
- **Per-frame hallucination in an animation's upscaled output.**
  `upscale_batch.py` runs every frame through the ONNX model independently,
  with no shared state between frames - a well-known limitation of
  single-image super-resolution. On a low-detail/ambiguous source region
  the model can guess differently frame to frame, so a detail (an extra
  claw, a stray highlight) pops in for a few frames and back out, even
  though the real low-resolution source barely changed there. Review by eye
  (`make animgallery_mods`/`animgallery_dist`, section 4.3) before trusting
  a new upscale definition - this is invisible from a single frame or a
  handful of spot-checks and needs to be checked per-animation, ideally
  with the GIF slowed down and its base-vs-this-build comparison columns
  (both features of the gallery itself) rather than played at real speed.

  `considerPreviousFrames` (section 4.1) is a real mitigation, not a
  different model/scale gamble, for a resource found to have this problem:
  list just the resources that need it and the rest of the definition keeps
  its own `model`/`scale` unchanged. For each listed resource with 3+
  frames, after independent per-frame upscaling, each frame's pixels are
  compared against the per-pixel median across every frame of the same
  (cyclic) animation - does the upscaled difference from that consensus
  exceed what the real low-res source's own difference from its consensus
  justifies - and pulled to the median where it does, on the same model the
  rest of the definition already uses. Try it before reaching for a
  different model or scale. It needs 3+ frames for a meaningful consensus (a
  2-frame animation has no independent third frame to vote with), and never
  touches alpha or
  an already-transparent pixel.

`encodeFrames` compresses every frame against an all-zero reference
instead of diffing against the previous frame's real pixels, because
that's the only convention the game's actual RSD8 decoder implements - so
an animation's frames never depend on each other, and a size change
between poses isn't a concern to test for. This is confirmed directly
against Night Dive's officially published original Mac source
(`ShockMac/Libraries/2D/Source/RSD/{RSDCVT.C,RSDUnpack.c}`), not just the
vendored Shockolate snapshot in `engine/`: both files are logic-identical
to their vendored counterparts (same 1994 RCS revision comments, same
unconditional `LG_memset(p_dst, kSkipColor, ...)` zero-fill in every skip
branch), and `RSDUnpack.c` additionally carries the original hand-written
68K assembly decoder alongside its later PowerPC/C port - its own skip
handler also zeroes an output register rather than reading from a
reference. Two independently-authored implementations, spanning different
CPU architectures and years apart, agreeing on "skip means zero" down to
matching source comments ("skip (zero) bytes") rules this out as an
overlooked bug rather than deliberate, original design.

The RLE format's own opcode grammar does permit the alternative reading -
"skip" meaning "retain the reference image's value" rather than "write
zero" (`ss-specs` documents this as relevant to compressing "video
clips", which is what led `hacked`'s own GIF-import path, and therefore
earlier versions of `encodeFrames`, to use it). Searching the original
source for anywhere that convention is actually exercised turned up
nothing: SS1's two "video clip" features - cutscenes (`MoviePlay.c`) and
"video mail" (`VMAIL.C`, itself built directly on `MoviePlay.h`) - both
use an entirely separate movie codec, not RSD8, and neither references
the RSD8 decoder at all.

`gen_animgallery.sh` and the real engine therefore always agree on an
upscaled animation's frames; a difference between them points at a
genuine engine rendering bug (root [README.md](../README.md) section 4's
patch reference) rather than this pipeline's own output, and
`engine-debug-tools/`'s debug dump hotkey (root README.md section 4.5
"Debugging") dumps the engine's own decoded frame data independent of its
renderer to tell the two apart.