Files

T

shenleiandClaude Opus 5 2b873f190f port(2P step 3a/3b): stdlib on sys.path + the pack module

step 3a — 标准库:
  tools/py_embed/make_stdlib_zip.py packs the vendored Lib/ into
  <build>/python27.zip (397 modules, 6876547 bytes, ZIP_STORED — the zlib
  module is trimmed out of our CPython, so zipimport cannot inflate).
  The mtpython_stdlib target builds it; py_embed_test and the launcher test
  both run against it.
  platform/ScriptLib/PythonHost sets the embedding flags before
  Py_Initialize and, after it, puts the zip on sys.path and re-registers
  encodings.search_function with codecs — 2.7 builds the codec registry
  once, at startup, so replacing sys.path afterwards needs it back.

step 3b — pack 模块:
  UserInterface/PythonPackModule.cpp copied verbatim; packExist/packGet/
  initpack read root/ and uiscript/ through CEterPackManager, i.e. through
  asset_io. The CHINA_CRYPT_KEY block in that unit is CAccountConnector's
  and is kept #if 0 until the login slice (2V1). UserInterface/StdAfx.h is
  a documented partial mirror — the original is the whole client's PCH.

The launcher test now runs initpack() as RunMainScript does (its first
line), reads system.py / uiscript / locale text back out of the packs, and
confirms cp1252 decoding works off the zip. system.py still stops at
"No module named app" — the C++ modules are the 2V0 slice.

gates: port_map.py check 0 errors · key leak check 8/8 none · macOS ctest
27/27 · android + ios port_platform compile clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

2026-09-23 09:16:37 +09:00

8.9 KiB

Raw Blame History

Third-party native dependencies

The GDExtension links five native libraries that are not part of Godot or godot-cpp. Phase 1 (macOS) pulled the first three from Homebrew; that stops working the moment we cross-compile for the Android NDK or the iOS SDK (BACKLOG F1/F2), so as of 2026-08-30 they are built from source as part of our own CMake build. Crypto++ joined later (2026-09-02) with the MT_PROTOCOL=classic 40250 backend and was vendored the same way from the start.

CPython joined on 2026-09-23 with 批次 2P (the embedded 40250 script layer) and is vendored source as well.

Everything lives under extension/third_party/ and is wired by extension/third_party/CMakeLists.txt, which exposes five aliases:

Alias	Backing target	Consumed by
`mt3p::sodium`	`sodium`	`mtnet`, `mtpack`, `mtproto`
`mt3p::zstd`	`libzstd_static`	`mtpack`
`mt3p::minilzo`	`minilzo`	`mtproto`
`mt3p::cryptopp`	`cryptopp`	`mtnet` (classic backend)
`mt3p::python`	`mtpython`	the script layer (`py_embed_test`)

No source file changed: #include <sodium.h>, #include <zstd.h>, #include <lzo/lzo1x.h> and #include <cryptopp/…> all still resolve (the LZO one via a 1-line shim header, see below).

libsodium — via `robinlinden/libsodium-cmake`

Submodule: extension/third_party/libsodium-cmake @ 9b2848d
- itself carries jedisct1/libsodium @ 93a7d0d (libsodium 1.0.20 line) as a nested submodule — clones need --recursive.
Why the wrapper: upstream libsodium is autotools-only. This MIT-licensed wrapper is a pure CMakeLists over an untouched libsodium checkout; it generates version.h, sets CONFIGURED, and builds clean for macOS / iOS / Android / Windows. Used by many mobile projects.
Build options we force: SODIUM_DISABLE_TESTS=ON, SODIUM_MINIMAL=OFF (we need crypto_kx, crypto_auth, crypto_aead_xchacha20poly1305, crypto_stream_xchacha20, crypto_generichash/BLAKE2b — all outside the "minimal" set).
License: ISC (libsodium) + MIT (wrapper). Ship-safe.

libzstd — `facebook/zstd`

Submodule: extension/third_party/zstd @ tag v1.5.6 (794ea1b0).
Built through upstream build/cmake with: ZSTD_BUILD_PROGRAMS/SHARED/TESTS/CONTRIB=OFF, ZSTD_BUILD_STATIC=ON, ZSTD_LEGACY_SUPPORT=OFF, ZSTD_MULTITHREAD_SUPPORT=OFF (the eterpack path only does single-shot ZSTD_compress/ZSTD_decompress).
Target consumed: libzstd_static (carries its own public include dir).
License: BSD-3-Clause / GPLv2 dual. Ship-safe under BSD.

miniLZO — vendored source (not a submodule)

Files: extension/third_party/minilzo/{minilzo.c,minilzo.h,lzoconf.h,lzodefs.h} copied verbatim from lzo-2.10 (minilzo.c sha1 019debb3…), plus README.LZO, COPYING, AUTHORS as required by the LZO license.
Upstream LZO has no git repo (oberhumer.com tarball only), and miniLZO is a 4-file amalgamation, so it is copied in rather than submoduled.
Shim: minilzo/lzo/lzo1x.h is a 1-line #include "../minilzo.h" so extension/src/proto/proto.cpp keeps #include <lzo/lzo1x.h> unchanged. miniLZO's API (lzo_init, lzo1x_decompress_safe, LZO_E_OK, lzo_uint) is a strict subset of full LZO and covers everything the CLZO path uses.
License: GPLv2. Acceptable for this internal, non-published, non-commercial project on the same footing as libgr2/src/oodle1.c — but LZO must be removed, replaced, or commercially licensed before any public release or commercial use. (item_proto/mob_proto are the only LZO consumers; a clean-room LZO1X decompressor is the eventual fix.)

Crypto++ — vendored source (not a submodule)

Files: extension/third_party/cryptopp/ (~6.6 MB), copied from the 40250 client tree (ClientVS22/extern/include/cryptopp), CRYPTOPP_VERSION 840 = 8.4.0. The 40250 server links this exact version (Server/.../extern/cryptopp_8_4_0.tar.gz), so the version is pinned by wire compatibility, not preference.
Why: the MT_PROTOCOL=classic backend's _IMPROVED_PACKET_ENCRYPTION_ cipher — DH2 key agreement + hint-selected block ciphers (Twofish default) in CTR mode. Only consumer is extension/src/net/classic/classic_cipher.{h,cpp}. The m2dev backend does not use it (that one is libsodium/XChaCha20).
No upstream CMakeLists, so cryptopp/CMakeLists.txt is ours: it globs the library TUs and drops the cryptest / benchmark / adhoc / FIPS-selftest ones (adhoc, bench1..3, datatest, dlltest, fipsalgt, fipstest, pch, regtest1..4, test, validat0..10).
ASM disabled everywhere (CRYPTOPP_DISABLE_ASM / _SSSE3 / _AESNI) so one pure-C++ build works on macOS, the Android NDK and the iOS SDK; the classic cipher needs no AES-NI / SHA-ext. Android additionally compiles the NDK's sources/android/cpufeatures/cpu-features.c in, because upstream cpu.cpp includes cpu-features.h unconditionally under __ANDROID__.
Built with -w (upstream is not warning-clean under -Wall) and -fno-strict-aliasing.
License: public domain — the compilation carries Wei Dai's copyright, but every individual file is placed in the public domain by its authors (extension/third_party/cryptopp/License.txt). Ship-safe; unlike miniLZO it imposes no obligation before a public or commercial release.

CPython 2.7.18 — vendored source (not a submodule)

Files: extension/third_party/cpython-2.7.18/ (~25 MB), the official Python-2.7.18.tgz (sha256 b62c0e7937551d0cc02b8fd5cb0f544f9405bafc9a54d3808ed4594812edef43) trimmed of what an embedded interpreter never builds or ships: Doc Demo Tools Mac PCbuild, Modules/{_ctypes,expat,zlib,_sqlite}, the stdlib test trees and the pre-VS2010 PC/ project files. Lib/ stays: the mtpython_stdlib target packs it into <build>/python27.zip with tools/py_embed/make_stdlib_zip.py, which is what the host puts on sys.path (batch 2P step 3). Entries are stored, never deflated — the zlib module is trimmed out above, so zipimport has nothing to inflate with.
Why vendored: 2.7 is end-of-life, so there is no package to fetch on the four target SDKs, and the build must keep working offline and on the cross-build machines. It is the version the 40250 scripts are written for (ScriptLib embeds 2.7), so it is pinned by the scripts, not by preference.
Built by our own cpython-2.7.18/CMakeLists.txt, not autotools: one mtpython static library from the exact 133 objects the reference libpython2.7.a contains. The source list and the built-in module table (config/config.c, config/Setup.static, 39 entries) are shared; pyconfig.h is not — it is a probe result, so each platform keeps its own under config/<platform>/, regenerated by tools/py_embed/gen_pyconfig.sh. Off by default: -DMTGODOT_EMBED_PYTHON=ON builds it.
Three vendor patches (everything else is verbatim upstream):
1. configure + 2. configure.ac — add arm64) MACOSX_DEFAULT_ARCH="arm64" to both case \/usr/bin/arch`` blocks; upstream predates Apple silicon and dies with "Unexpected output of 'arch' on OSX".
2. Modules/posixmodule.c — after the "all other compilers" block that hard-defines HAVE_SYSTEM/FORK/EXECV/WAIT/POPEN regardless of pyconfig.h, undefine them again under TARGET_OS_IPHONE: the iphoneos SDK marks system() unavailable and the sandbox forbids the rest.
Platform notes: Android API 24 turns off HAVE_LANGINFO_H (bionic declares nl_langinfo only from API 26); iOS derives its pyconfig.h from the macOS one (2.7's configure refuses a darwin cross build) and drops process control plus <sys/random.h>/getentropy, but keeps HAVE_GETGROUPS because posixmodule.c defines MAX_GROUPS — which its setgroups path needs — only inside that #ifdef. Windows is not built yet: upstream supports MSVC + PC/pyconfig.h, while our Windows gate cross-compiles with MinGW, where PC/pyconfig.h's gnu-win32 branch leaves HAVE_UNISTD_H unset and posixmodule.c / dynload_win.c then collide with the MinGW headers. Decided in 批次 2P step 4 (PORT-PLAN).
License: PSF License Agreement (cpython-2.7.18/LICENSE). Ship-safe; the only obligation is keeping the license and a change notice, which this section and the patched files' comments serve.

Rebuilding / fetching

./build.sh auto-runs git submodule update --init --recursive if any of the three submodules (godot-cpp, zstd, libsodium-cmake+libsodium) is missing. miniLZO, Crypto++ and CPython are vendored source and need no fetch.

Homebrew libsodium / zstd / lzo are no longer referenced by the build and can be uninstalled.

8.9 KiB Raw Blame History