Add AI-upscaled critter/object art as a new MDL mod type
build / build (push) Successful in 3m36s

Extends the MDL system beyond text overrides: bitmaps/<label>.json
names a model/scale and resource list, and mdlp extracts each
resource's frames, runs them through an ONNX Runtime CPU upscaler
baked into mdlp-image, re-quantizes to the game's palette, and writes
the result back - same build-time pipeline as message overrides.

Adds animgallery, a standalone viewer that renders any data
directory's animated resources as an HTML/GIF gallery with a
checkbox-driven MDL JSON generator, and -only-changed to preview just
what a mod overrides against the base game.

Fixes a real data race in hacked's ChangeNotifier.hashAll() (found via
go build -race causing non-deterministic resource counts), via a new
numbered patch rather than editing the vendored source.

Adds engine-patches/, a general-fixes counterpart to
android/engine-patches/ now staged and applied to the desktop builds
too (previously unpatched): a real GL_UNPACK_ROW_LENGTH stride bug, an
audio error-logging improvement, and two critter-rendering fixes -
mod art was losing to the base game's own re-opened resource files,
and critters rendered at the wrong size/position once that was fixed,
both from how the billboard renderer derives on-screen size and
placement straight from bitmap pixel dimensions.

Updates README.md and AGENTS.md to document engine-patches/.
This commit is contained in:
ml
2026-09-27 20:52:35 +02:00
parent 389abd7de7
commit ab2b778f44
25 changed files with 2009 additions and 35 deletions
+24 -1
View File
@@ -11,10 +11,33 @@ FROM ubuntu:22.04
# needs to fetch a newer toolchain over the network at build time.
ARG GO_VERSION=1.27.1
# patch: applies src/hacked-patches/ to the scratch copy of hacked/ that
# process-mods.sh (and ../gen_animgallery.sh) stage before building against
# it - see stage-hacked.sh. python3/pip: runs the AI upscaler below via
# plain CPU inference (ONNX Runtime). Not a Vulkan-based tool: this image
# has no real GPU, and Vulkan software emulation (Mesa lavapipe) gives
# wrong colors for ML inference workloads.
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
ca-certificates curl gosu \
ca-certificates curl gosu patch python3 python3-pip \
&& rm -rf /var/lib/apt/lists/*
RUN pip3 install --no-cache-dir onnxruntime pillow numpy
# The AI upscaler model for mdlp's upscale mod type (see ../README.md
# section 4): RealESRGAN_x4plus_anime_6B, a 6-block RRDBNet ONNX-exported
# from xinntao/Real-ESRGAN's anime-illustration checkpoint (BSD-3-Clause).
# It's trained on flat-color line art, so it keeps hard sprite edges rather
# than smoothing them the way a photo- or video-denoising model would. A
# mod's own "model" field (upscaleDef in ../src/mdlp/upscale.go) is not
# consulted; every resource goes through this one model. upscale_batch.py
# resizes its fixed native 4x output down to whatever scale a mod actually
# asks for.
ARG UPSCALE_MODEL_URL=https://huggingface.co/skillsafe-ai/realesrgan-x4plus-anime-6b/resolve/main/model.onnx
RUN mkdir -p /opt/upscale-model \
&& curl -sSL -o /opt/upscale-model/model.onnx "$UPSCALE_MODEL_URL"
COPY mdlp-image/upscale_batch.py /usr/local/bin/upscale_batch.py
RUN chmod 755 /usr/local/bin/upscale_batch.py
ENV UPSCALE_MODEL_PATH=/opt/upscale-model/model.onnx
RUN curl -sSLO "https://go.dev/dl/go${GO_VERSION}.linux-amd64.tar.gz" \
&& tar -C /usr/local -xzf "go${GO_VERSION}.linux-amd64.tar.gz" \
+11 -1
View File
@@ -22,10 +22,20 @@ OUT_DIR="$MODDING_ROOT/dist/mods"
[ -d "$DATA_DIR" ] \
|| { echo "Game data not found at $DATA_DIR - see the root README.md section 3." >&2; exit 1; }
echo "== Staging a patched copy of hacked/ for mdlp to build against =="
# mdlp's own go.mod points a `replace` at the sibling ../hacked - stage a
# patched copy here (never build src/hacked/ in place) so mdlp picks up the
# same fixes hacked's own binary does (see src/hacked-patches/).
SCRATCH_DIR="$MODDING_ROOT/build/mdlp-src"
rm -rf "$SCRATCH_DIR"
mkdir -p "$SCRATCH_DIR"
bash "$MODDING_ROOT/mdlp-image/stage-hacked.sh" "$MODDING_ROOT" "$SCRATCH_DIR/hacked"
cp -a "$MODDING_ROOT/src/mdlp" "$SCRATCH_DIR/mdlp"
echo "== Building mdlp =="
BIN_DIR="$(mktemp -d)"
trap 'rm -rf "$BIN_DIR"' EXIT
(cd "$MODDING_ROOT/src/mdlp" && go build -trimpath -o "$BIN_DIR/mdlp" .)
(cd "$SCRATCH_DIR/mdlp" && go build -trimpath -o "$BIN_DIR/mdlp" .)
MDLP_ARGS=(-data "$DATA_DIR" -mods "$MODDING_ROOT/mods" -out "$OUT_DIR")
if [ -n "${MODS:-}" ]; then
+20
View File
@@ -0,0 +1,20 @@
#!/usr/bin/env bash
# Stages a scratch, patched copy of src/hacked/ at the given destination -
# shared by process-mods.sh (mdlp) and ../gen_animgallery.sh (animgallery),
# both of which build against hacked's source directly (via their own
# go.mod's `replace ../hacked`) and so both need the same patches applied
# that hacked-build-image/build-hacked.sh already applies for hacked's own
# binary. src/hacked/ is never built or patched in place.
#
# Usage: stage-hacked.sh <modding-root> <dest-dir-for-hacked>
set -euo pipefail
MODDING_ROOT="$1"
DEST="$2"
rm -rf "$DEST"
mkdir -p "$(dirname "$DEST")"
cp -a "$MODDING_ROOT/src/hacked" "$DEST"
for p in "$MODDING_ROOT"/src/hacked-patches/*.patch; do
patch -p1 -d "$DEST" < "$p"
done
+59
View File
@@ -0,0 +1,59 @@
#!/usr/bin/env python3
"""Upscales every PNG in one directory into another, via one ONNX Runtime
CPU session shared across the whole batch (same reasoning as mdlp's own
Go-side batching: the fixed per-invocation model-load cost otherwise
dominates for the many small sprite frames one upscale definition has).
Runs on CPU only, not Vulkan: this Docker image has no real GPU, and
Vulkan software emulation (Mesa lavapipe) gives wrong colors for ML
inference workloads.
Usage: upscale_batch.py <model.onnx> <in-dir> <out-dir> <scale>
"""
import sys
import os
import numpy as np
import onnxruntime as ort
from PIL import Image
def main():
model_path, in_dir, out_dir, scale_arg = sys.argv[1:5]
scale = int(scale_arg)
session = ort.InferenceSession(model_path, providers=["CPUExecutionProvider"])
input_name = session.get_inputs()[0].name
names = sorted(n for n in os.listdir(in_dir) if n.lower().endswith(".png"))
for name in names:
img = Image.open(os.path.join(in_dir, name)).convert("RGBA")
rgb = img.convert("RGB")
alpha = img.split()[3]
# The model has no concept of transparency (3-channel RGB in/out) -
# run it on the flattened RGB only, and scale the alpha mask
# separately with a plain resize. Alpha edges don't need AI
# enhancement, just to end up the right size.
arr = np.asarray(rgb, dtype=np.float32) / 255.0
arr = np.transpose(arr, (2, 0, 1))[np.newaxis, ...]
output = session.run(None, {input_name: arr})[0]
output = np.clip(output[0], 0.0, 1.0) * 255.0
output = np.transpose(output, (1, 2, 0)).astype(np.uint8)
upscaled_rgb = Image.fromarray(output, "RGB")
target_size = (img.width * scale, img.height * scale)
if upscaled_rgb.size != target_size:
# The model upscales at its own fixed native factor - resize to
# whatever scale the mod actually asked for.
upscaled_rgb = upscaled_rgb.resize(target_size, Image.LANCZOS)
upscaled_alpha = alpha.resize(target_size, Image.LANCZOS)
result = Image.merge("RGBA", (*upscaled_rgb.split(), upscaled_alpha))
result.save(os.path.join(out_dir, name))
print(f"upscaled {len(names)} frame(s)")
if __name__ == "__main__":
main()