Files
d2r-mobile/dm/README.md
T
shenleiandClaude Opus 5.5 3c25cd89fc Android: move the shell to dm/android and the Vulkan backend to dm/gfx/vulkan
Mirror the iOS layout instead of nesting Android inside dm/host:
- dm/android/app       GameActivity app (manifest, java, res, build_apk.sh)
- dm/android/Sources   front_app.c, touch_app.c, front_headless.c
- dm/android/Makefile  entry point (apk / lib / install / headless / tests);
                       rules in build.mk (was dm/host/Makefile.android),
                       run with dm/host as cwd; outputs in dm/android/build/
- dm/android/tests     t_vk_perf
- dm/gfx/vulkan        nvn_vk.c and friends, next to the Metal backend

Only include paths, build paths and docs change; no code changes.
The Mac / iOS source lists do not pick up gfx/vulkan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:59:36 +09:00

21 KiB

dm/ — rewriter, host, graphics layer, iOS shell and verification

中文:README.zh-CN.md

This is all of the code of D2R Mobile. This page explains how the whole thing works, what each directory is, the common commands, and what each of the main files in host/ is responsible for.

The steps for preparing the environment and building the app are not repeated here:

Below, $DM_ROOT is the workspace root directory (default ~/dm-work): the game files, rewrite build outputs, caches and run logs all live under it and are not committed to the repository.

How it works

The game program is three NSO files (the Switch executable module format): main (the game itself), sdk (the Nintendo SDK) and subsdk0. They use the same instruction set as the iPhone, so the vast majority of the code can be executed as is; what has to be handled is their dependence on the Switch operating system (Horizon).

1. Rewriting the main program (tools/patch_nso.py)

Each module is rewritten once, offline, touching only the instructions that cannot be executed as is under the Apple kernel:

Original instruction Why it cannot be executed as is Changed to
Reads and writes of the x18 register The Apple kernel modifies this register Reads and writes of a thread slot (a per-thread storage location)
Reads and writes of tpidr_el0, reads of tpidrro_el0 The game uses them to find thread-local storage and Horizon's thread-local region; the Apple kernel modifies tpidr_el0, and Horizon's thread-local region has no counterpart on Apple platforms Reads and writes of the corresponding thread slot
svc #n (system call) There is no Horizon kernel on Apple platforms A jump to the "host gate": an entry point into the host's C code
Reads of cntpct_el0 (the system counter) Horizon's timer has no counterpart on Apple platforms Also goes through the host gate; the host answers as a 19.2 MHz system counter

In addition there are two tables registered per program build number, both written in patch_nso.py:

  • Cuts (CUTS): targeted modifications of individual instructions or data. Each entry carries the original bytes, the new bytes and a reason; before rewriting, the original bytes are checked entry by entry, and the rewrite aborts on any mismatch.
  • Observation gates (OBS): a gate inserted at the entry of some function that enters a host callback, executes the original instruction and then jumps back, so that the host can read the state at that point.

The rewritten code is never modified again at run time: thread slot numbers are obtained indirectly through a read-only "key page", and gate addresses are fetched from thread slots. The reason is that iOS does not allow executing code written at run time, so the game code must be compiled into the App as a signed code segment.

The output for each module is in the build output directory: <name>.patched.bin (the rewritten image, with the key page and the patch area appended), <name>.fixups.tsv (layout), <name>.gates.tsv (gate table), <name>.sites.tsv, <name>.cuts.tsv, <name>.obs.tsv.

Build outputs have a codename, written as the single line of PRODUCT; tools/product.sh build uses the same rewriter to produce three kinds of build output in $DM_ROOT/build/cache/dm/out/<codename>-<kind>:

Kind Ruleset Where it is used
keypage DM_RULESET=native verify/regress.sh verifies it by default
rules DM_RULESET=touch-run-v4, without host-private gates (DM_PRIVATE_GATES=0) verify/rulegate.sh uses it by default
touch DM_RULESET=touch-run-v4, with host-private gates The kind that ios/run.sh compiles into the App by default

2. Loading and startup (host/loader.c, host/main.c)

The host dmhost plays the role of Horizon's loader:

  1. loader.c maps each module's rewritten image into memory, creates the thread slots and fills in the key page, registers every gate according to <name>.gates.tsv, performs cross-module relocation, and finally sets segment permissions. The code area is read-only plus executable from the moment it is loaded; after that the loader writes only to data areas.
  2. Imported symbols (functions a module requests from outside) are resolved in one of three ways; see the next section.
  3. main.c initializes the SDK runtime in the SDK's own startup order, runs each module's init_array (the table of global constructors), and then enters the game entry point nnMain.

Each game thread is a host pthread. guest_thread_init in host/thread.c fills in those thread slots for the thread: the gate entry, the stand-in for x18, the thread-local storage block, and Horizon's thread-local region.

3. Taking over system calls and system libraries (host/hle*.c)

The game's dependence on the system has three layers, each connected in its own way:

  • Kernel system calls. A rewritten svc goes through dm_gate in host/entry.S, which saves the registers, and enters gate_dispatch in host/hle.c, which looks up by the gate's return address which instruction it originally was; system calls are handed to svc_dispatch. hle_svc.c implements the memory-related calls (it reserves one whole block of guest address space and carves the heap, stack and other regions out of it), and hle_thread.c implements the thread and synchronization calls. The SDK's nn::os (threads, locks, events and so on) therefore runs as is; only the kernel beneath it is replaced.
  • C library functions. hle_table.c connects the C library functions whose calling convention matches and that do not depend on structure layout directly to Apple's libc.
  • System services. On a real device the SDK connects to system services such as files, accounts, audio and input through inter-process communication; the host has no such service processes, so it takes over at the SDK interface layer by function name and supplies host implementations directly. Takeovers are registered with two macros in host/hle.h: HLE fills in imports that no module defines; HLE_FORCE is a forced takeover — even if the SDK has a definition, all references resolve to the host implementation. Every such entry must state a reason, and an entry that is registered but referenced by no module aborts at load time.

An import with no implementation at all is connected to a named stub: when called, it prints one line UNIMPL <function name> in the log and returns 0.

Online features such as multiplayer, friends and the shop are out of scope: the related interfaces uniformly answer "unavailable", and the game takes its offline branch.

4. Graphics layer (gfx/)

The game draws with NVN (the Switch graphics interface). main imports only one entry that fetches function addresses, nvnBootstrapLoader; the host forcibly takes it over, and not a single instruction of the NVN driver in the SDK is executed. gfx/nvn/hle_nvn.c records the game's NVN calls as commands, and gfx/nvn/nvn_mtl.m replays them as Metal calls. The shaders are not hand-written: the game's original shaders are translated offline into Metal shader libraries by the translation chain under gfx/tools/ (the last few steps of ../setup.sh). See gfx/README.md for details.

5. iOS shell (ios/)

The App compiles the sources of host/ and gfx/ in directly. The front-end interface host/front.h is implemented on the Mac by host/front_mac.m and on iOS by ios/Sources/front_ios.m. The rewritten module images are turned into embedded assembly by ios/tools/gen_embed.py and compiled into the executable; the iOS branch of loader.c does not copy code, it only maps these blocks back at their original relative positions. Touch controls and the user interface are also in ios/. See ios/README.md for details.

6. Android shell (android/, Adreno devices only)

The same host/ and gfx/ sources are built with the NDK into libd2r.so, loaded by a GameActivity; the front-end interface is implemented by android/Sources/front_app.c, and the touch HUD is in android/Sources/touch_app.c. Graphics replay switches to Vulkan: gfx/vulkan/nvn_vk.c consumes the same commands recorded by hle_nvn.c. Differences in system interfaces are filled in by host/compat/linux/. See android/README.md for details.

Directories

Location Contents
PRODUCT The codename of the current rewrite build output, one line
tools/ The rewriter patch_nso.py, the build output script product.sh, dmx.py for reading the game's resource storage, casc_overlay.py for rewriting the resource storage at build time, and a batch of code-reading and analysis scripts
host/ The host: loading, system call and system library takeover, the Mac front end, in-game features; host/tests/ holds the unit tests
gfx/ The graphics layer: nvn/ (interface layer and Metal replay), vulkan/ (Vulkan replay for Android), tex/ (texture formats and deswizzling), tools/ (shader translation chain), tests/
ios/ The iOS App: Sources/, the project description project.yml, the build-and-install script run.sh, tools/ (asset generation, smoke tests, UI regression), tests/
android/ The Android App: app/ (GameActivity, res/, packaging script), Sources/ (front end and touch HUD), the build entry point Makefile, tests/
verify/ Verification: the rewrite-equivalence regression regress.sh, the full acceptance run accept.sh, and the scripts and format descriptions its items use
data/ Small data shipped with the repository: the registry of borrowed paths for the resource overlay. The startup prefetch list is not in the public repository: without it there is no prefetch, and startup skips one step of decoding ahead of time
i18n/ The Chinese-English string table ui_text.tsv; scan_literals.py checks that all Chinese text the player can see goes through this table, with exceptions registered in literal_whitelist.tsv
third_party/ Third-party code, see ../docs/DEPENDENCIES.md

Common workflows

Rewrite the three modules (already included in ../setup.sh; rerun after changing patch_nso.py or changing the codename):

dm/tools/product.sh build

To rewrite a single module: patch_nso.py <NSO file> <output directory> <module name>.

Check rewrite equivalence:

dm/verify/regress.sh [build output directory] [module…]

It does three things: a structural check (structck.py), a before/after equivalence trial of every rewritten site using Unicorn (a CPU emulation library) (equiv.py), and host gate functionality (gatefn). The default build output directory is product.sh path keypage, and the default modules are main sdk subsdk0. --fresh first produces a fresh output with the current rewriter and then verifies it; --struct-only skips the equivalence trial. If any item fails the exit code is 1.

Build and run the Mac host:

make -C dm/host
cd dm/host
DM_ROMFS=<Update/RomFS directory> DM_SAVE=<save directory> DM_TEMP=<temporary directory> DM_WINDOW=1 \
  ./dmhost "$(../tools/product.sh path touch)" main sdk

Usage is dmhost <build output directory> [module…] [-- game arguments…]. DM_SAVE must be set; without it the host aborts immediately. Common environment switches:

Switch Effect
DM_ROMFS / DM_SAVE / DM_TEMP The host directories for the three mount points: game resources, saves, temporary files
DM_WINDOW=1 Open a window and show the picture (no window if unset)
DM_AUDIO=1 Actually play sound on the Mac (by default no sound is played, but audio buffers are consumed in real time)
DM_WATCH=<seconds> Watchdog: end the process when the time is up
DM_SVC=1 / DM_IMPORTS=1 / DM_DBG=1 Log, respectively, memory-related system calls, import resolution and takeover, and the game's own debug output

Unit tests: make -C dm/host/tests run, make -C dm/gfx/tests run, make -C dm/ios/tests test.

Build the Android package: make -C dm/android -j8 (output dm/android/build/app/d2r.apk); make -C dm/android install installs it onto the phone connected over adb.

Build the iOS package: cd dm/ios && ./run.sh build; to install on a device use ./run.sh install. For signing configuration see the top-level README.

Full acceptance run: dm/verify/accept.sh [--fast|--full] runs rewrite equivalence, save-file comparison, the screenshot gate, unit tests, the iOS build, functional cases and more together. Besides the game files it also needs reference run data and standard save files, which are not provided with the repository; the emulator driver scripts used to obtain the reference run data are not shipped either, so the full acceptance run cannot be run with this repository alone.

Files in host/

Loading and execution

File Responsible for
main.c Bootstrap: load modules, initialize the SDK runtime, enter nnMain
loader.c Multi-module loading: mapping, thread slots, gate registration, cross-module relocation, takeover by name, segment permissions
entry.S The common gate entry dm_gate, the table of named stubs for unimplemented imports
hle.c Import resolution (resolve_import) and gate dispatch (gate_dispatch)
thread.c Thread slot initialization for each game thread
hle.h, dm.h Common headers: the HLE / HLE_FORCE registration macros, the module table, logging

Kernel and system libraries

File Responsible for
hle_svc.c Kernel system calls: address space and heap, queries, debug output, timing
hle_thread.c Thread and synchronization system calls (threads, locks, condition variables, events)
hle_guestthread.c Game threads started by the host that carry a full SDK thread context
hle_table.c The pass-through table of C library functions
hle_os.c Common odds and ends: the side table from game objects to host objects, clocks
hle_sdkinit.c Takeover of the initialization of each system service at SDK startup

System services

File Responsible for
hle_fs.c File system: mount names mapped to host directories
hle_tact.c, hle_granny.c Host implementations of zlib decompression of resource storage blocks and of Granny data section decoding
hle_audio.c, hle_codec.c Audio output and Opus decoding
hle_hid.c, hle_touch.c Gamepad and touch-screen input
hle_swkbd.c, hle_charname.c The system software keyboard connected to a host input box; Chinese allowed in character names
hle_oe.c, hle_time.c, hle_account.c Operating environment, clocks, local user
hle_movie.c Cutscene videos: not decoded, no picture shown, played as empty for the original duration
hle_net.c, hle_friends.c, hle_misc.c Networking and friends connected as unavailable; the remaining miscellaneous services
hle_lifecycle.c The host's exit flow

Front end and diagnostics

File Responsible for
front.h, front_mac.m The front-end interface and its Mac implementation (window, keyboard)
dm_input.h The state the front end writes for the input layer
crash.c Crash state printing, watchdog
diag.c The probe DM_PROBE, the UI message log DM_UIMSG, main-thread sampling DM_SAMPLE
memstat.m, hle_count.c, hle_fdstat.c Memory accounting, takeover call counts, file descriptor statistics
romfs_validate.c Checks the game resource directory at startup

In-game features

The dm_*.c files are features and touch adaptations added on top of the original game; the comment at the top of each file explains what it does and which addresses in the game it relies on. For example dm_cursor.c, dm_world.c, dm_actions.c (touch selection, interacting with the world, buying/selling and spending points), dm_hud.c (the numbers on the bottom interface), dm_vault.c and dm_vault_ui.c (the Chronicle vault), dm_pouch.c (the pouch), dm_rift.c (the rift), dm_cloudsync.c (iCloud save sync), dm_custom.c (the switches for these features). The addresses in the files are all offsets into the original 3.3.94012 image.

Generating assets from your game files

The repository contains no images, tables or text derived from the original game files (the widened stash, the pouch frame, the right-hand panel of character select, the minimap goal, the skill atlas, the bottom UI, the Chronicle vault table, the experience table and so on). They are made on the spot at build time from your own game files, and the outputs do not go into git.

Which files are needed

  • Required: the update package RomFS of the Switch release 1.0.34.0 (corresponding to PC version 3.3.94012) (Update/RomFS, which contains data/build_manifest.psv). If the version is wrong, the generation scripts report an error based on the registered checksums ("原件和冻结的不一样", i.e. "the original differs from the frozen one"; "原文变了", i.e. "the original text has changed").
  • Optional: an unpacked directory of the PC version of D2R (containing hd/global/ui/..., the raw .sprite files). With it, the bottom UI uses the PC version's 2x original images; without it, the bottom UI uses upscaled Switch assets and will be slightly blurry, and generation prints "没找到 PC 版文件……底栏用 Switch 素材放大,会略糊" ("PC version files not found … the bottom bar uses upscaled Switch assets and will be slightly blurry").

Where to put them

  • Switch RomFS: default $DM_ROOT/data/switch/3.3.94012/Update/RomFS, or use the environment variable DM_ROMFS_SRC=<your Update/RomFS>.
  • PC version unpacked directory: default $DM_ROOT/data/assets-src/raw, or DM_PC_DATA=<directory>.

Which command to run

  • Build the iOS package: cd dm/ios && ./run.sh build. It automatically runs tools/gen_assets.py and the host table header generation at the start; there is no need to run them separately.
  • Generate assets only: python3 dm/ios/tools/gen_assets.py [<Update/RomFS>]. Nothing is regenerated if the inputs have not changed.
  • Mac host: make -C dm/host. The three missing generated files (dm_vault_tab.h, dm_hud_exp.h, dm_bright_levels.inc) are generated automatically according to DM_ROMFS_SRC.
  • iOS unit tests: make -C dm/ios/tests test. The missing skill index is generated automatically.

When game files are missing, a one-sentence message says directly which directory to put them in. These commands call python3 and must be run inside the Python environment created by setup.sh (Pillow, numpy and the rest are installed there; for versions see ../docs/DEPENDENCIES.md): run source ~/dm-work/build/cache/dm/venv/bin/activate first (if you set DM_ROOT, use it in place of ~/dm-work).

What is produced

Output Location (not in git) Generation script (under dm/ios/tools/)
Widened stash background, Chronicle page cover dm/ios/Gen/assets/NativeHUD/widestash/ gen_widestash_art.py, gen_vault_cover.py
Pouch frame dm/ios/Gen/assets/NativeHUD/pouch/ gen_pouch_art.py
Right-hand panel of character select dm/ios/Gen/assets/NativeHUD/touch/ gen_fe_rightpanel.py
Minimap goal marker dm/ios/Gen/assets/NativeHUD/mapgoal/ gen_map_goal_art.py
Bottom UI (background and layout) dm/ios/Gen/assets/NativeHUD/v4/ gen_hud_v4.py (geometry in hud_v4_design.py; layout = original layout + touchrectv1_patch.json)
Skill atlas, skill index dm/ios/Sources/TouchArt/dm_native_*.png, dm/touch/dm_touch_native_art.c native_art/generate.py (checks the originals against dm/ios/Sources/TouchArt/native_art_manifest.json)
Chronicle vault table, experience table dm/host/dm_vault_tab.h, dm/host/dm_hud_exp.h gen_vault_table.py, gen_hud_exp.py (called by dm/host/Makefile)
Level day/night table (used by 3D picture brightening) dm/host/dm_bright_levels.inc dm/tools/gen_bright_levels.py (also called by dm/host/Makefile)

The original game text used by item annotations is read from the game's string tables at build time; the repository stores none of it. The old-translation table and the one-line-comment table are not shipped with the repository; you can supply your own, in the format described under dm/ios/tools/annot/. Without them these two steps are skipped automatically and the rest of the annotations work as usual.

dm/ios/tools/check_hud_gen.sh checks that these generated files have not been added back to the repository.