Add engine-debug-tools and fix critter animation rendering bugs
build / build (push) Successful in 2m52s
build / build (push) Successful in 2m52s
- engine-debug-tools/: a debug dump hotkey + dump2png to inspect decoded critter frames independent of the renderer (see root README.md's "Debugging" section). - engine-patches/04,05: fix critter sprite scale and an OpenGL texture cache collision, found using that tooling. - mdlp: fix encodeFrames RLE-diffing upscaled frames against a previous-frame reference the real RSD8 decoder never implements (verified against Night Dive's original Mac source down to the 68K assembly) - was rendering as scattered static in-game. - Upscale pipeline: better color quantization (dithering, pure-ramp exclusion) and per-resource model selection.
This commit is contained in:
@@ -192,6 +192,8 @@ mods/
|
||||
messages/
|
||||
<label>.json # one message's overrides
|
||||
<label>.txt # optional: a text field's body, referenced by ...File
|
||||
bitmaps/
|
||||
<label>.json # an AI-upscaled set of critter/object art resources
|
||||
```
|
||||
|
||||
Each `messages/*.json` overrides one message (email, log, or fragment) by
|
||||
@@ -213,6 +215,26 @@ label:
|
||||
`mods/mymod/` is a worked example matching section 3's Rebecca's-mail edit,
|
||||
expressed as MDL instead of a `hacked` session.
|
||||
|
||||
Each `bitmaps/*.json` names a shared upscale configuration and the
|
||||
resources to apply it to - `gen_animgallery.sh`'s own checkbox-driven MDL
|
||||
generator (its rendered gallery page has a selection UI that assembles this
|
||||
exact JSON shape from whatever's checked) produces this format directly:
|
||||
|
||||
| Field | Meaning |
|
||||
|---|---|
|
||||
| `model` | Which baked-in ONNX model to upscale with - see the table below |
|
||||
| `scale` | Output size as a multiple of the original (for example `2`) |
|
||||
| `resources` | Each entry is `"0x<ID>"` (language `Any`) or `"0x<ID>_<Language>"` (for example `"0x25_German"`), matching `gen_animgallery.sh`'s own id format |
|
||||
|
||||
Available `model` values (baked into `mdlp-image`, see
|
||||
`mdlp-image/upscale_batch.py`):
|
||||
|
||||
| `model` | Checkpoint | Best for |
|
||||
|---|---|---|
|
||||
| `realesrgan-x4plus-anime` | Real-ESRGAN's 6-block anime/illustration RRDBNet (16.7M params) | Flat-color sprite art - keeps hard edges instead of smoothing them the way a photo/video model would. Default choice. |
|
||||
| `realesrgan-x4plus` | Real-ESRGAN's general-purpose photo RRDBNet (16.7M params) | A specific resource where the anime model genuinely mis-renders a region (seen on some high-contrast color boundaries) - less edge-sharp overall, much slower, and can mis-render its own different regions, so check the result before keeping it |
|
||||
| `realesr-general-x4v3` | Real-ESRGAN's compact general-purpose SRVGGNet (1.2M params) | A specific resource neither RRDBNet model renders cleanly - noticeably softer than either, but the most robust against both of their failure modes. Not a blanket replacement, just a last resort for one stubborn sprite |
|
||||
|
||||
### 4.2. `mdlp`
|
||||
|
||||
`src/mdlp/` is first-party Go source (not vendored - edit it directly, no
|
||||
@@ -281,3 +303,83 @@ Same as `hacked-build-image/` (section 2.1): `./mdlp-upload-image.sh` reads
|
||||
`mdlp-image/VERSION` just because the Dockerfile changed - only once a build
|
||||
with the current version has actually succeeded, same policy as the root
|
||||
project's own `build-image/VERSION`.
|
||||
|
||||
## 5. Testing
|
||||
|
||||
Verifying a mod's built output (particularly an AI-upscaled one, section
|
||||
4.1) means comparing its rendered result against the unmodified base game,
|
||||
not just eyeballing the result in isolation - a subtly wrong color or a
|
||||
corrupted frame can look plausible on its own. `gen_animgallery.sh` renders
|
||||
straight from a built mod's own resource data (`dist/mods/<name>/mod`), the
|
||||
same bytes the engine loads, so a problem visible there is a problem in the
|
||||
mod, not an artifact of some other preview path. Regenerate it with `-data
|
||||
res/assets/ss_ee/data -data dist/mods/<name>/mod` layered over the base
|
||||
data to see the mod's effect, or with just the base `-data` argument to
|
||||
check what the unmodified resource looks like for comparison.
|
||||
|
||||
`gen_animgallery.sh` is a from-scratch Go decoder, not the game engine -
|
||||
it never exercises the engine's own texture upload/caching code at all.
|
||||
A clean gallery render rules out a bug in the mod's resource *data*, not a
|
||||
bug in how the engine *renders* that data - the two are independent, and a
|
||||
problem can exist in only one of them. Actually running the built mod
|
||||
(`../dist/run-mods.sh` or `run.sh` with a single mod folder, section 4.3)
|
||||
is the only way to check the second. This in-engine verification has so
|
||||
far only been through the desktop build, not the Quest headset - both
|
||||
read the same resource data and the engine-side bugs found here
|
||||
(`engine-patches/`, root [README.md](../README.md) section 4's patch
|
||||
reference) were resource-data-adjacent, not OpenXR-specific, so a fix at
|
||||
this level should carry over, but it isn't headset-confirmed until tested
|
||||
there.
|
||||
|
||||
The AI-upscale pipeline (`src/mdlp/upscale.go`) and the engine's own
|
||||
rendering of its output have a failure mode worth specifically checking
|
||||
for, found by comparing upscaled output against the base game across many
|
||||
resources rather than trusting a handful of spot-checks:
|
||||
|
||||
- **Wrong-hue speckling in dark or muted regions.** SS1's palette has a
|
||||
handful of fully-saturated "pure color ramp" entries (pure cyan, red,
|
||||
green, ...) that sit deceptively close, in color-distance terms, to
|
||||
common dark/muted shading colors - a plain nearest-color search, with or
|
||||
without dithering, can pick one of these for a near-black pixel the AI
|
||||
upscaler produced, since color distance near black barely penalizes an
|
||||
unrelated hue. `isPureRampColor` excludes these entries from
|
||||
re-quantization entirely rather than relying on a brightness or
|
||||
saturation threshold, which proved too easy for a genuinely
|
||||
dark-but-muted pixel to slip past.
|
||||
|
||||
`encodeFrames` compresses every frame against an all-zero reference
|
||||
instead of diffing against the previous frame's real pixels, because
|
||||
that's the only convention the game's actual RSD8 decoder implements - so
|
||||
an animation's frames never depend on each other, and a size change
|
||||
between poses isn't a concern to test for. This is confirmed directly
|
||||
against Night Dive's officially published original Mac source
|
||||
(`ShockMac/Libraries/2D/Source/RSD/{RSDCVT.C,RSDUnpack.c}`), not just the
|
||||
vendored Shockolate snapshot in `engine/`: both files are logic-identical
|
||||
to their vendored counterparts (same 1994 RCS revision comments, same
|
||||
unconditional `LG_memset(p_dst, kSkipColor, ...)` zero-fill in every skip
|
||||
branch), and `RSDUnpack.c` additionally carries the original hand-written
|
||||
68K assembly decoder alongside its later PowerPC/C port - its own skip
|
||||
handler also zeroes an output register rather than reading from a
|
||||
reference. Two independently-authored implementations, spanning different
|
||||
CPU architectures and years apart, agreeing on "skip means zero" down to
|
||||
matching source comments ("skip (zero) bytes") rules this out as an
|
||||
overlooked bug rather than deliberate, original design.
|
||||
|
||||
The RLE format's own opcode grammar does permit the alternative reading -
|
||||
"skip" meaning "retain the reference image's value" rather than "write
|
||||
zero" (`ss-specs` documents this as relevant to compressing "video
|
||||
clips", which is what led `hacked`'s own GIF-import path, and therefore
|
||||
earlier versions of `encodeFrames`, to use it). Searching the original
|
||||
source for anywhere that convention is actually exercised turned up
|
||||
nothing: SS1's two "video clip" features - cutscenes (`MoviePlay.c`) and
|
||||
"video mail" (`VMAIL.C`, itself built directly on `MoviePlay.h`) - both
|
||||
use an entirely separate movie codec, not RSD8, and neither references
|
||||
the RSD8 decoder at all.
|
||||
|
||||
`gen_animgallery.sh` and the real engine therefore always agree on an
|
||||
upscaled animation's frames; a difference between them points at a
|
||||
genuine engine rendering bug (root [README.md](../README.md) section 4's
|
||||
patch reference) rather than this pipeline's own output, and
|
||||
`engine-debug-tools/`'s debug dump hotkey (root README.md section 4.5
|
||||
"Debugging") dumps the engine's own decoded frame data independent of its
|
||||
renderer to tell the two apart.
|
||||
|
||||
Reference in New Issue
Block a user