Fix animgallery rendering bugs and mitigate AI-upscale frame flicker
build / build (push) Successful in 2m52s

- writeGIF now pads every frame to one shared canvas instead of
  writing each at its own (possibly storage-cropped) size - GIF
  disposal rules were showing a previous frame's stale pixels through
  the uncovered border of a smaller frame, bleeding content between
  frames in the gallery.
- decodeFrames/decodeViewFrames/decodeAnimationFrames now always hand
  bitmap.DecodeReferenced a zero buffer instead of the previous
  frame/block's real pixels. The game's actual RSD8 decoder treats a
  "skip" run as literal zero, with no mechanism to carry a reference
  frame's pixels forward (confirmed against Night Dive's published
  source), and hacked's own multi-block Bitmap preview decodes the
  same way. The previous behavior ghosted a previous frame's stale
  content into the current frame's skip-run regions, in both the
  gallery and the AI upscaler's source input.
- index.html now shows a labeled per-frame filmstrip next to each
  animated GIF, and a base-vs-this-build side-by-side comparison for
  every resource a mod overrides.
- Gallery GIFs play back at a third of real speed by default, for
  easier frame-by-frame review.
- modding/mods/*/bitmaps/*.json gains considerPreviousFrames: a
  per-resource list that runs a temporal-consistency pass (median
  across the animation's own frames) on independently-upscaled frames
  to reduce per-frame AI flicker. Applied to 0x7B1 in upscale_medbot.
- Drop the rebecca-test example mod.
This commit is contained in:
ml
2026-10-27 07:54:19 +01:00
parent a8d7ff76df
commit 90303894ce
7 changed files with 538 additions and 123 deletions
+37 -1
View File
@@ -225,6 +225,7 @@ exact JSON shape from whatever's checked) produces this format directly:
| `model` | Which baked-in ONNX model to upscale with - see the table below |
| `scale` | Output size as a multiple of the original (for example `2`) |
| `resources` | Each entry is `"0x<ID>"` (language `Any`) or `"0x<ID>_<Language>"` (for example `"0x25_German"`), matching `gen_animgallery.sh`'s own id format |
| `considerPreviousFrames` | Optional, same id format as `resources` - an id here that isn't also in `resources` is simply never applied, not an error. A listed resource with 3+ frames gets a temporal-consistency pass (section 5) that pulls a frame's upscaled pixel back toward the rest of the animation's own consensus wherever it disagrees by more than the real low-res source justifies - a real mitigation for per-frame upscale flicker. Lets one resource opt in without splitting it into its own file or changing `model`/`scale` for the rest. |
Available `model` values (baked into `mdlp-image`, see
`mdlp-image/upscale_batch.py`):
@@ -315,7 +316,15 @@ same bytes the engine loads, so a problem visible there is a problem in the
mod, not an artifact of some other preview path. Regenerate it with `-data
res/assets/ss_ee/data -data dist/mods/<name>/mod` layered over the base
data to see the mod's effect, or with just the base `-data` argument to
check what the unmodified resource looks like for comparison.
check what the unmodified resource looks like for comparison. Each card
shows the looping GIF plus a static filmstrip of every one of its frames
side by side, so one specific frame can be singled out and compared without
having to pause the animation by eye. When two or more `-data` directories
are layered this way, a card for any resource a later layer actually
overrides shows both versions side by side - the first directory's own
("base") and the winning one ("this build") - so a mod's effect is always
checked against the original it's replacing, not recalled from memory or a
separate base-only gallery run.
`gen_animgallery.sh` is a from-scratch Go decoder, not the game engine -
it never exercises the engine's own texture upload/caching code at all.
@@ -346,6 +355,33 @@ resources rather than trusting a handful of spot-checks:
re-quantization entirely rather than relying on a brightness or
saturation threshold, which proved too easy for a genuinely
dark-but-muted pixel to slip past.
- **Per-frame hallucination in an animation's upscaled output.**
`upscale_batch.py` runs every frame through the ONNX model independently,
with no shared state between frames - a well-known limitation of
single-image super-resolution. On a low-detail/ambiguous source region
the model can guess differently frame to frame, so a detail (an extra
claw, a stray highlight) pops in for a few frames and back out, even
though the real low-resolution source barely changed there. Review by eye
(`make animgallery_mods`/`animgallery_dist`, section 4.3) before trusting
a new upscale definition - this is invisible from a single frame or a
handful of spot-checks and needs to be checked per-animation, ideally
with the GIF slowed down and its base-vs-this-build comparison columns
(both features of the gallery itself) rather than played at real speed.
`considerPreviousFrames` (section 4.1) is a real mitigation, not a
different model/scale gamble, for a resource found to have this problem:
list just the resources that need it and the rest of the definition keeps
its own `model`/`scale` unchanged. For each listed resource with 3+
frames, after independent per-frame upscaling, each frame's pixels are
compared against the per-pixel median across every frame of the same
(cyclic) animation - does the upscaled difference from that consensus
exceed what the real low-res source's own difference from its consensus
justifies - and pulled to the median where it does, on the same model the
rest of the definition already uses. Try it before reaching for a
different model or scale. It needs 3+ frames for a meaningful consensus (a
2-frame animation has no independent third frame to vote with), and never
touches alpha or
an already-transparent pixel.
`encodeFrames` compresses every frame against an all-zero reference
instead of diffing against the previous frame's real pixels, because