Add engine-debug-tools and fix critter animation rendering bugs
build / build (push) Successful in 2m52s

- engine-debug-tools/: a debug dump hotkey + dump2png to inspect
  decoded critter frames independent of the renderer (see root
  README.md's "Debugging" section).
- engine-patches/04,05: fix critter sprite scale and an OpenGL
  texture cache collision, found using that tooling.
- mdlp: fix encodeFrames RLE-diffing upscaled frames against a
  previous-frame reference the real RSD8 decoder never implements
  (verified against Night Dive's original Mac source down to the
  68K assembly) - was rendering as scattered static in-game.
- Upscale pipeline: better color quantization (dithering, pure-ramp
  exclusion) and per-resource model selection.
This commit is contained in:
ml
2026-10-18 06:59:47 +02:00
parent ab2b778f44
commit 9d0a0d5798
21 changed files with 1185 additions and 82 deletions
+102
View File
@@ -192,6 +192,8 @@ mods/
messages/
<label>.json # one message's overrides
<label>.txt # optional: a text field's body, referenced by ...File
bitmaps/
<label>.json # an AI-upscaled set of critter/object art resources
```
Each `messages/*.json` overrides one message (email, log, or fragment) by
@@ -213,6 +215,26 @@ label:
`mods/mymod/` is a worked example matching section 3's Rebecca's-mail edit,
expressed as MDL instead of a `hacked` session.
Each `bitmaps/*.json` names a shared upscale configuration and the
resources to apply it to - `gen_animgallery.sh`'s own checkbox-driven MDL
generator (its rendered gallery page has a selection UI that assembles this
exact JSON shape from whatever's checked) produces this format directly:
| Field | Meaning |
|---|---|
| `model` | Which baked-in ONNX model to upscale with - see the table below |
| `scale` | Output size as a multiple of the original (for example `2`) |
| `resources` | Each entry is `"0x<ID>"` (language `Any`) or `"0x<ID>_<Language>"` (for example `"0x25_German"`), matching `gen_animgallery.sh`'s own id format |
Available `model` values (baked into `mdlp-image`, see
`mdlp-image/upscale_batch.py`):
| `model` | Checkpoint | Best for |
|---|---|---|
| `realesrgan-x4plus-anime` | Real-ESRGAN's 6-block anime/illustration RRDBNet (16.7M params) | Flat-color sprite art - keeps hard edges instead of smoothing them the way a photo/video model would. Default choice. |
| `realesrgan-x4plus` | Real-ESRGAN's general-purpose photo RRDBNet (16.7M params) | A specific resource where the anime model genuinely mis-renders a region (seen on some high-contrast color boundaries) - less edge-sharp overall, much slower, and can mis-render its own different regions, so check the result before keeping it |
| `realesr-general-x4v3` | Real-ESRGAN's compact general-purpose SRVGGNet (1.2M params) | A specific resource neither RRDBNet model renders cleanly - noticeably softer than either, but the most robust against both of their failure modes. Not a blanket replacement, just a last resort for one stubborn sprite |
### 4.2. `mdlp`
`src/mdlp/` is first-party Go source (not vendored - edit it directly, no
@@ -281,3 +303,83 @@ Same as `hacked-build-image/` (section 2.1): `./mdlp-upload-image.sh` reads
`mdlp-image/VERSION` just because the Dockerfile changed - only once a build
with the current version has actually succeeded, same policy as the root
project's own `build-image/VERSION`.
## 5. Testing
Verifying a mod's built output (particularly an AI-upscaled one, section
4.1) means comparing its rendered result against the unmodified base game,
not just eyeballing the result in isolation - a subtly wrong color or a
corrupted frame can look plausible on its own. `gen_animgallery.sh` renders
straight from a built mod's own resource data (`dist/mods/<name>/mod`), the
same bytes the engine loads, so a problem visible there is a problem in the
mod, not an artifact of some other preview path. Regenerate it with `-data
res/assets/ss_ee/data -data dist/mods/<name>/mod` layered over the base
data to see the mod's effect, or with just the base `-data` argument to
check what the unmodified resource looks like for comparison.
`gen_animgallery.sh` is a from-scratch Go decoder, not the game engine -
it never exercises the engine's own texture upload/caching code at all.
A clean gallery render rules out a bug in the mod's resource *data*, not a
bug in how the engine *renders* that data - the two are independent, and a
problem can exist in only one of them. Actually running the built mod
(`../dist/run-mods.sh` or `run.sh` with a single mod folder, section 4.3)
is the only way to check the second. This in-engine verification has so
far only been through the desktop build, not the Quest headset - both
read the same resource data and the engine-side bugs found here
(`engine-patches/`, root [README.md](../README.md) section 4's patch
reference) were resource-data-adjacent, not OpenXR-specific, so a fix at
this level should carry over, but it isn't headset-confirmed until tested
there.
The AI-upscale pipeline (`src/mdlp/upscale.go`) and the engine's own
rendering of its output have a failure mode worth specifically checking
for, found by comparing upscaled output against the base game across many
resources rather than trusting a handful of spot-checks:
- **Wrong-hue speckling in dark or muted regions.** SS1's palette has a
handful of fully-saturated "pure color ramp" entries (pure cyan, red,
green, ...) that sit deceptively close, in color-distance terms, to
common dark/muted shading colors - a plain nearest-color search, with or
without dithering, can pick one of these for a near-black pixel the AI
upscaler produced, since color distance near black barely penalizes an
unrelated hue. `isPureRampColor` excludes these entries from
re-quantization entirely rather than relying on a brightness or
saturation threshold, which proved too easy for a genuinely
dark-but-muted pixel to slip past.
`encodeFrames` compresses every frame against an all-zero reference
instead of diffing against the previous frame's real pixels, because
that's the only convention the game's actual RSD8 decoder implements - so
an animation's frames never depend on each other, and a size change
between poses isn't a concern to test for. This is confirmed directly
against Night Dive's officially published original Mac source
(`ShockMac/Libraries/2D/Source/RSD/{RSDCVT.C,RSDUnpack.c}`), not just the
vendored Shockolate snapshot in `engine/`: both files are logic-identical
to their vendored counterparts (same 1994 RCS revision comments, same
unconditional `LG_memset(p_dst, kSkipColor, ...)` zero-fill in every skip
branch), and `RSDUnpack.c` additionally carries the original hand-written
68K assembly decoder alongside its later PowerPC/C port - its own skip
handler also zeroes an output register rather than reading from a
reference. Two independently-authored implementations, spanning different
CPU architectures and years apart, agreeing on "skip means zero" down to
matching source comments ("skip (zero) bytes") rules this out as an
overlooked bug rather than deliberate, original design.
The RLE format's own opcode grammar does permit the alternative reading -
"skip" meaning "retain the reference image's value" rather than "write
zero" (`ss-specs` documents this as relevant to compressing "video
clips", which is what led `hacked`'s own GIF-import path, and therefore
earlier versions of `encodeFrames`, to use it). Searching the original
source for anywhere that convention is actually exercised turned up
nothing: SS1's two "video clip" features - cutscenes (`MoviePlay.c`) and
"video mail" (`VMAIL.C`, itself built directly on `MoviePlay.h`) - both
use an entirely separate movie codec, not RSD8, and neither references
the RSD8 decoder at all.
`gen_animgallery.sh` and the real engine therefore always agree on an
upscaled animation's frames; a difference between them points at a
genuine engine rendering bug (root [README.md](../README.md) section 4's
patch reference) rather than this pipeline's own output, and
`engine-debug-tools/`'s debug dump hotkey (root README.md section 4.5
"Debugging") dumps the engine's own decoded frame data independent of its
renderer to tell the two apart.
+34 -14
View File
@@ -21,23 +21,43 @@ RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
ca-certificates curl gosu patch python3 python3-pip \
&& rm -rf /var/lib/apt/lists/*
RUN pip3 install --no-cache-dir onnxruntime pillow numpy
RUN pip3 install --no-cache-dir onnxruntime pillow numpy scipy
# The AI upscaler model for mdlp's upscale mod type (see ../README.md
# section 4): RealESRGAN_x4plus_anime_6B, a 6-block RRDBNet ONNX-exported
# from xinntao/Real-ESRGAN's anime-illustration checkpoint (BSD-3-Clause).
# It's trained on flat-color line art, so it keeps hard sprite edges rather
# than smoothing them the way a photo- or video-denoising model would. A
# mod's own "model" field (upscaleDef in ../src/mdlp/upscale.go) is not
# consulted; every resource goes through this one model. upscale_batch.py
# resizes its fixed native 4x output down to whatever scale a mod actually
# asks for.
ARG UPSCALE_MODEL_URL=https://huggingface.co/skillsafe-ai/realesrgan-x4plus-anime-6b/resolve/main/model.onnx
RUN mkdir -p /opt/upscale-model \
&& curl -sSL -o /opt/upscale-model/model.onnx "$UPSCALE_MODEL_URL"
# The AI upscaler models for mdlp's upscale mod type (see ../README.md
# section 4), selected per mod by a bitmaps/*.json file's own "model" field
# (upscaleDef in ../src/mdlp/upscale.go) - upscale_batch.py resolves a name
# straight to $UPSCALE_MODELS_DIR/<name>.onnx, and picks tiled or single-pass
# inference based on each model's own declared ONNX input shape (fixed vs
# dynamic height/width), so adding another model here needs no code change.
# Each model's RGB output is re-quantized to SS1's fixed 256-color palette
# with Floyd-Steinberg dithering (imageToBitmap in ../src/mdlp/upscale.go),
# not a plain nearest-color map - without it, any model's smooth gradients
# can land on a visibly wrong, much darker palette entry in a solid patch
# rather than unnoticeable dithering noise.
# All three are BSD-3-Clause ONNX exports of xinntao/Real-ESRGAN checkpoints:
# - realesrgan-x4plus-anime: a 6-block RRDBNet trained on flat-color
# illustration, keeping hard sprite edges rather than smoothing them the
# way a photo- or video-denoising model would. Dynamic input shape. Default
# choice - sharpest result on most sprites.
# - realesrgan-x4plus: the general-purpose photo checkpoint (16.7M-param
# RRDBNet). Fixed 128x128 input, needs tiling for anything larger, and
# much slower than the other two. Worth trying when the anime model
# mis-renders a specific sprite, but can itself mis-render different ones.
# - realesr-general-x4v3: a compact 1.2M-param SRVGGNet, dynamic input
# shape, ~100x faster than realesrgan-x4plus. Noticeably softer than
# either RRDBNet model, but the most robust against both models' failure
# modes - the model to reach for on a specific sprite neither RRDBNet
# model renders cleanly, not as a blanket replacement.
ARG UPSCALE_MODEL_ANIME_URL=https://huggingface.co/skillsafe-ai/realesrgan-x4plus-anime-6b/resolve/main/model.onnx
ARG UPSCALE_MODEL_GENERAL_URL=https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/resolve/6f5749d5adff39de0127e42bc45c8ec8e6f76a66/Real-ESRGAN-x4plus.onnx
ARG UPSCALE_MODEL_COMPACT_URL=https://huggingface.co/jonathanst29/tinier-upscale-models/resolve/main/realesr-general-x4v3.onnx
RUN mkdir -p /opt/upscale-models \
&& curl -sSL -o /opt/upscale-models/realesrgan-x4plus-anime.onnx "$UPSCALE_MODEL_ANIME_URL" \
&& curl -sSL -o /opt/upscale-models/realesrgan-x4plus.onnx "$UPSCALE_MODEL_GENERAL_URL" \
&& curl -sSL -o /opt/upscale-models/realesr-general-x4v3.onnx "$UPSCALE_MODEL_COMPACT_URL"
COPY mdlp-image/upscale_batch.py /usr/local/bin/upscale_batch.py
RUN chmod 755 /usr/local/bin/upscale_batch.py
ENV UPSCALE_MODEL_PATH=/opt/upscale-model/model.onnx
ENV UPSCALE_MODELS_DIR=/opt/upscale-models
RUN curl -sSLO "https://go.dev/dl/go${GO_VERSION}.linux-amd64.tar.gz" \
&& tar -C /usr/local -xzf "go${GO_VERSION}.linux-amd64.tar.gz" \
+102 -9
View File
@@ -8,7 +8,13 @@ Runs on CPU only, not Vulkan: this Docker image has no real GPU, and
Vulkan software emulation (Mesa lavapipe) gives wrong colors for ML
inference workloads.
Usage: upscale_batch.py <model.onnx> <in-dir> <out-dir> <scale>
Resolves <model-name> to $UPSCALE_MODELS_DIR/<model-name>.onnx and picks
single-pass or tiled inference from that model's own declared ONNX input
shape - a model with a fixed height/width (e.g. 128x128) needs tiling for
any image bigger than that; one with symbolic dimensions runs the whole
image through in one pass, whatever its size.
Usage: upscale_batch.py <model-name> <in-dir> <out-dir> <scale>
"""
import sys
import os
@@ -16,31 +22,118 @@ import os
import numpy as np
import onnxruntime as ort
from PIL import Image
from scipy import ndimage
def extend_opaque_color(rgb, alpha, threshold=128):
"""Replaces every below-threshold-alpha pixel's RGB with its nearest
opaque neighbor's, so the model (RGB-only, see run_tile()) sees a
smooth color field instead of a hard edge against flat black -
otherwise its convolutions blend genuine edge colors with that black
background, leaving dark fringing on the final (correctly re-masked)
output."""
transparent = np.asarray(alpha) < threshold
if not transparent.any() or transparent.all():
return rgb
_, indices = ndimage.distance_transform_edt(transparent, return_indices=True)
arr = np.asarray(rgb)
filled = arr[indices[0], indices[1]]
return Image.fromarray(np.where(transparent[..., None], filled, arr))
def run_tile(session, input_name, tile_rgb):
"""Runs one tile (already at the model's required size) and returns its
upscaled RGB as a uint8 HxWx3 array - handles both the with-batch-dim
([1,3,H,W]) and without ([3,H,W]) output conventions different ONNX
exports use."""
arr = tile_rgb.astype(np.float32) / 255.0
arr = np.transpose(arr, (2, 0, 1))[np.newaxis, ...]
out = session.run(None, {input_name: arr})[0]
if out.ndim == 4:
out = out[0]
out = np.clip(out, 0.0, 1.0) * 255.0
return np.transpose(out, (1, 2, 0)).astype(np.uint8)
def upscale_dynamic(session, input_name, rgb_image):
return Image.fromarray(run_tile(session, input_name, np.asarray(rgb_image)))
def upscale_tiled(session, input_name, rgb_image, tile_size, native_scale):
"""Splits an image larger than the model's fixed input size into
overlapping tile_size x tile_size tiles (edge-padding any tile that
would otherwise run past the image bounds), runs each individually, and
stitches the results back into one canvas - later tiles simply
overwrite the small overlap region of earlier ones, which is invisible
at sprite scale."""
src = np.asarray(rgb_image)
h, w = src.shape[:2]
canvas = np.zeros((h * native_scale, w * native_scale, 3), dtype=np.uint8)
overlap = 16
stride = tile_size - overlap
y = 0
while True:
ty = min(y, max(0, h - tile_size))
x = 0
while True:
tx = min(x, max(0, w - tile_size))
tile = src[ty:ty + tile_size, tx:tx + tile_size]
th, tw = tile.shape[:2]
if th < tile_size or tw < tile_size:
tile = np.pad(tile, ((0, tile_size - th), (0, tile_size - tw), (0, 0)), mode="edge")
out = run_tile(session, input_name, tile)[: th * native_scale, : tw * native_scale]
oy, ox = ty * native_scale, tx * native_scale
canvas[oy:oy + th * native_scale, ox:ox + tw * native_scale] = out
if tx + tile_size >= w:
break
x += stride
if ty + tile_size >= h:
break
y += stride
return Image.fromarray(canvas)
def main():
model_path, in_dir, out_dir, scale_arg = sys.argv[1:5]
model_name, in_dir, out_dir, scale_arg = sys.argv[1:5]
scale = int(scale_arg)
models_dir = os.environ["UPSCALE_MODELS_DIR"]
model_path = os.path.join(models_dir, f"{model_name}.onnx")
if not os.path.isfile(model_path):
available = sorted(f[:-5] for f in os.listdir(models_dir) if f.endswith(".onnx"))
sys.exit(f"unknown upscale model {model_name!r} - available: {', '.join(available)}")
session = ort.InferenceSession(model_path, providers=["CPUExecutionProvider"])
input_name = session.get_inputs()[0].name
input_info = session.get_inputs()[0]
input_name = input_info.name
# A fixed (integer) height/width means this model only ever accepts
# that exact input size - a dynamic one (reported as a symbolic name
# such as "height") accepts any size in one pass. For a fixed-size
# model, its own declared output shape gives the native scale factor
# directly, with no need to run it just to find out.
fixed_size = input_info.shape[2] if isinstance(input_info.shape[2], int) else None
native_scale = session.get_outputs()[0].shape[-1] // fixed_size if fixed_size else None
names = sorted(n for n in os.listdir(in_dir) if n.lower().endswith(".png"))
for name in names:
img = Image.open(os.path.join(in_dir, name)).convert("RGBA")
rgb = img.convert("RGB")
alpha = img.split()[3]
rgb = extend_opaque_color(rgb, alpha)
# The model has no concept of transparency (3-channel RGB in/out) -
# run it on the flattened RGB only, and scale the alpha mask
# separately with a plain resize. Alpha edges don't need AI
# enhancement, just to end up the right size.
arr = np.asarray(rgb, dtype=np.float32) / 255.0
arr = np.transpose(arr, (2, 0, 1))[np.newaxis, ...]
output = session.run(None, {input_name: arr})[0]
output = np.clip(output[0], 0.0, 1.0) * 255.0
output = np.transpose(output, (1, 2, 0)).astype(np.uint8)
upscaled_rgb = Image.fromarray(output, "RGB")
if fixed_size is not None and (img.width > fixed_size or img.height > fixed_size):
upscaled_rgb = upscale_tiled(session, input_name, rgb, fixed_size, native_scale)
elif fixed_size is not None:
padded = np.pad(np.asarray(rgb), ((0, fixed_size - img.height), (0, fixed_size - img.width), (0, 0)), mode="edge")
out = run_tile(session, input_name, padded)
upscaled_rgb = Image.fromarray(out[: img.height * native_scale, : img.width * native_scale])
else:
upscaled_rgb = upscale_dynamic(session, input_name, rgb)
target_size = (img.width * scale, img.height * scale)
if upscaled_rgb.size != target_size:
+2 -2
View File
@@ -470,13 +470,13 @@ func writeIndex(path string, entries []galleryEntry) error {
`<label>Model: <select id="model">` +
`<option>realesrgan-x4plus-anime</option>` +
`<option>realesrgan-x4plus</option>` +
`<option>realesr-animevideov3</option>` +
`<option>realesr-general-x4v3</option>` +
`</select></label>` +
`<label>Scale: <input id="scale" type="number" value="2" min="1" max="4" style="width:4em"></label>` +
`<button id="clear-sel" type="button">Clear selection</button>` +
`<br><label>Filter: <input type="text" id="search" placeholder="id, language, file, transparent..." style="width:22em"></label>` +
`<span id="shown-count"></span>` +
`<textarea id="mdl-output" rows="12" readonly placeholder="Select animations below to generate an MDL upscale definition here - copy it into modding/mods/&lt;name&gt;/upscale/&lt;label&gt;.json"></textarea>` +
`<textarea id="mdl-output" rows="12" readonly placeholder="Select animations below to generate an MDL upscale definition here - copy it into modding/mods/&lt;name&gt;/bitmaps/&lt;label&gt;.json"></textarea>` +
`</div>`)
b.WriteString(`<h2>Selected (<span id="basket-count">0</span>)</h2><div id="basket"></div>`)
+133 -17
View File
@@ -14,6 +14,7 @@ import (
"image"
"image/color"
"image/png"
"math"
"os"
"os/exec"
"path/filepath"
@@ -46,8 +47,8 @@ var languageByName = map[string]resource.Language{
// upscaleDef is one modding/mods/<name>/bitmaps/<label>.json file: a shared
// set of upscale parameters and every resource to apply them to. Model
// is not consulted yet - mdlp-image bakes in a single ONNX model, applied
// regardless of this field's value.
// names a .onnx file baked into mdlp-image - see upscale_batch.py for the
// set currently available.
type upscaleDef struct {
Model string `json:"model"`
Scale int `json:"scale"`
@@ -282,11 +283,95 @@ func bitmapToImage(bmp *bitmap.Bitmap, pal bitmap.Palette) *image.RGBA {
// sprite renders at the right size but visibly offset from where it should
// sit. Writing {0,0} here (as if there were no anchor) makes the renderer
// fall back to a path that ignores that patch's size override entirely.
func clamp255(v float64) byte {
return byte(math.Max(0, math.Min(255, v)))
}
// isPureRampColor flags a palette entry from one of SS1's synthetic "pure
// color ramp" sections (used for UI/effect colors) rather than naturalistic
// sprite shading: either two of its three channels are exactly zero (a pure
// red/green/blue ramp), or one channel is exactly zero and the other two
// are exactly equal (a pure cyan/magenta/yellow ramp). A muted shading
// color, even a dark or desaturated one, never has two channels in an exact
// relationship like this - skin tones and dyed cloth have no reason to - so
// these entries are excluded from upscale re-quantization entirely rather
// than relying on a saturation/distance threshold to rule them out case by
// case: a target this dark reads as "saturated" under any max/min-based
// metric even when it's visually indistinguishable from a neutral dark
// gray, which is exactly what let these entries win a plain nearest-color
// search in the first place.
func isPureRampColor(c color.Color) bool {
r, g, b, _ := c.RGBA()
r, g, b = r>>8, g>>8, b>>8
zeros := 0
if r == 0 {
zeros++
}
if g == 0 {
zeros++
}
if b == 0 {
zeros++
}
if zeros >= 2 {
return true
}
return zeros == 1 && ((r == 0 && g == b) || (g == 0 && r == b) || (b == 0 && r == g))
}
// nearestSpriteColor finds the closest palette entry to target by plain
// squared-RGB distance, skipping Bitmapper's own reserved indices (outside
// 0x01-0x02, 0x08-0x0A, 0x20-0xFF) and the pure-ramp entries isPureRampColor
// flags. Squared-RGB rather than bitmap.Bitmapper's Lab-space search: close
// enough for re-quantizing an already-upscaled sprite, and avoids
// duplicating that package's unexported Lab-distance machinery just to add
// the ramp exclusion.
func nearestSpriteColor(palColors []color.Color, target color.RGBA) byte {
tr, tg, tb, _ := target.RGBA()
var best byte
bestDist := math.Inf(1)
for i, pc := range palColors {
isRegularColor := (i >= 0x01 && i <= 0x02) || (i >= 0x08 && i <= 0x0A) || i >= 0x20
if !isRegularColor || isPureRampColor(pc) {
continue
}
pr, pg, pb, _ := pc.RGBA()
dr := float64(tr>>8) - float64(pr>>8)
dg := float64(tg>>8) - float64(pg>>8)
db := float64(tb>>8) - float64(pb>>8)
if dist := dr*dr + dg*dg + db*db; dist < bestDist {
bestDist = dist
best = byte(i)
}
}
return best
}
func imageToBitmap(img *image.RGBA, pal bitmap.Palette, transparent bool, origAnchor bitmap.Area) *bitmap.Bitmap {
bounds := img.Bounds()
w, h := bounds.Dx(), bounds.Dy()
bitmapper := bitmap.NewBitmapper(&pal)
palColors := pal.ColorPalette(false)
pixels := make([]byte, w*h)
// Floyd-Steinberg error diffusion: a straight nearest-palette-color map
// (SS1's fixed 256-color palette, no per-sprite palette) can push an AI
// upscaler's smooth color gradients onto a visibly wrong palette entry
// wherever the true color sits in a gap between two distant entries -
// diffusing each pixel's quantization error into its unprocessed
// neighbors spreads that mismatch into unnoticeable dithering noise
// instead of a solid wrong-colored patch.
errR := make([]float64, w*h)
errG := make([]float64, w*h)
errB := make([]float64, w*h)
addErr := func(x, y int, er, eg, eb, weight float64) {
if x < 0 || x >= w || y < 0 || y >= h {
return
}
i := y*w + x
errR[i] += er * weight
errG[i] += eg * weight
errB[i] += eb * weight
}
for y := 0; y < h; y++ {
for x := 0; x < w; x++ {
c := img.RGBAAt(bounds.Min.X+x, bounds.Min.Y+y)
@@ -294,7 +379,32 @@ func imageToBitmap(img *image.RGBA, pal bitmap.Palette, transparent bool, origAn
pixels[y*w+x] = 0
continue
}
pixels[y*w+x] = bitmapper.MapColor(color.RGBA{R: c.R, G: c.G, B: c.B, A: 0xff})
i := y*w + x
r, g, b := float64(c.R)+errR[i], float64(c.G)+errG[i], float64(c.B)+errB[i]
idx := nearestSpriteColor(palColors, color.RGBA{R: clamp255(r), G: clamp255(g), B: clamp255(b), A: 0xff})
pixels[i] = idx
pr, pg, pb, _ := palColors[idx].RGBA()
er := r - float64(pr>>8)
eg := g - float64(pg>>8)
eb := b - float64(pb>>8)
// Skip diffusion into transparent neighbors so dithering noise
// never bleeds past the sprite's own silhouette.
rightOK := !(transparent && x+1 < w && img.RGBAAt(bounds.Min.X+x+1, bounds.Min.Y+y).A < 0x80)
belowOK := y+1 < h && !(transparent && img.RGBAAt(bounds.Min.X+x, bounds.Min.Y+y+1).A < 0x80)
belowLeftOK := y+1 < h && x > 0 && !(transparent && img.RGBAAt(bounds.Min.X+x-1, bounds.Min.Y+y+1).A < 0x80)
belowRightOK := y+1 < h && x+1 < w && !(transparent && img.RGBAAt(bounds.Min.X+x+1, bounds.Min.Y+y+1).A < 0x80)
if rightOK {
addErr(x+1, y, er, eg, eb, 7.0/16.0)
}
if belowLeftOK {
addErr(x-1, y+1, er, eg, eb, 3.0/16.0)
}
if belowOK {
addErr(x, y+1, er, eg, eb, 5.0/16.0)
}
if belowRightOK {
addErr(x+1, y+1, er, eg, eb, 1.0/16.0)
}
}
}
@@ -370,13 +480,10 @@ func upscaleBatch(all []resourceFrames, pal bitmap.Palette, model string, scale
fmt.Printf(" upscaling %d resource(s), %d frame(s) total (model %s, scale %d):\n",
len(all), totalFrames, model, scale)
// UPSCALE_MODEL_PATH (set by mdlp-image's Dockerfile) points at the ONNX
// model baked into the image.
modelPath := os.Getenv("UPSCALE_MODEL_PATH")
if modelPath == "" {
return nil, fmt.Errorf("UPSCALE_MODEL_PATH is not set - see mdlp-image/Dockerfile")
}
cmd := exec.Command("python3", "/usr/local/bin/upscale_batch.py", modelPath, inDir, outDir, strconv.Itoa(scale))
// upscale_batch.py resolves model (this def's own "model" field) against
// $UPSCALE_MODELS_DIR (set by mdlp-image's Dockerfile) itself, and fails
// with the available model names if it doesn't recognize this one.
cmd := exec.Command("python3", "/usr/local/bin/upscale_batch.py", model, inDir, outDir, strconv.Itoa(scale))
// Streamed live, not buffered via CombinedOutput: a single directory
// batch can run for a while, and silence until the whole thing exits
// gives no sense that it's still making progress.
@@ -426,17 +533,26 @@ func toRGBA(img image.Image) *image.RGBA {
return converted
}
// encodeFrames RLE-diffs each frame against the previous, matching the
// convention hacked's own editor/animations/View.go uses when importing a
// GIF (the first frame diffs against an implicit all-zero reference).
// encodeFrames RLE-compresses each frame against an all-zero reference (the
// "still image" convention - see ss-specs media/Bitmaps.md). The RLE format
// also grammatically allows a "skip" run to mean "copy from a reference
// image" rather than "write zero" (the "video clip" convention, which
// hacked's own GIF-import path uses) - but the game's actual RSD8 decoder
// (engine/src/Libraries/2D/Source/RSD/{rsdcvt.c,RSDUnpack.c}, confirmed
// against Night Dive's published original source down to the hand-written
// 68K assembly) only ever implements the zero-reference case, for every
// resource type checked including critter sprites: it writes literal zero
// for every skip run unconditionally, with no mechanism to carry a
// reference image's pixels forward at all. Compressing against the real
// previous frame here produced RLE streams only hacked's own reader could
// decode correctly - the game rendered scattered noise wherever a frame
// carried content unchanged from the one before it.
func encodeFrames(frames []*bitmap.Bitmap) [][]byte {
blocks := make([][]byte, 0, len(frames))
var prevPixels []byte
for _, bmp := range frames {
buf := new(bytes.Buffer)
_ = binary.Write(buf, binary.LittleEndian, &bmp.Header)
_ = rle.Compress(buf, bmp.Pixels, prevPixels)
prevPixels = bmp.Pixels
_ = rle.Compress(buf, bmp.Pixels, nil)
blocks = append(blocks, buf.Bytes())
}
return blocks