4v5u09Z5a4Yuc/mtgodot-poc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

port 2P step 1: vendor CPython 2.7.18 as the mtpython static library

Browse Source 
The embedded 40250 script layer needs its own interpreter on five platforms.
2.7 is end-of-life, so nothing can be fetched from the target SDKs and the
source is vendored (official 2.7.18 tarball, trimmed to 25MB; Lib/ stays for
step 3's python27.zip).

cpython-2.7.18/CMakeLists.txt builds one `mtpython` static library from exactly
the 133 objects the reference libpython2.7.a contains, off by default behind
-DMTGODOT_EMBED_PYTHON=ON. The source list and the built-in module table
(config/config.c, 39 entries) are shared; pyconfig.h is a probe result and is
not, so each platform keeps its own under config/<platform>/, regenerated by
tools/py_embed/gen_pyconfig.sh and checked against the shared table.

Three vendor patches, documented in docs/THIRD-PARTY.md: configure/configure.ac
learn arm64 on macOS, and posixmodule.c undefines the process-control calls it
hard-defines past pyconfig.h when building for iOS.

macOS (ctest python.embed: every builtin imports, codecs and pickle work off the
vendored Lib/), Android arm64 and iOS arm64 build. Linux needs a Linux host and
Windows needs a MinGW-vs-MSVC decision; both are noted for step 4.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
shenlei
2026-09-23 08:27:35 +09:00
co-authored by Claude Opus 5
parent a9b31dec67
commit 5fad769ee8
1018 changed files with 652582 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/PORT-PLAN.md
+11 -7
View File
@@ -82,8 +82,8 @@ extension/third_party/cpython-2.7.18/ # 静态库（2P 批次
| — | 定下目录结构，评估内嵌 Python；Android 独立程序验证 | 完成（9d0e50de、b02c49bb） |
| **2A** | 基础：Win32 类型层、参考公共头的最小闭包、`port_logic` CMake 目标、platform 接口骨架、头文件可编译门禁、port-map 重新基线 | **进行中**（剩 Python 库闭包头，等 2P） |
| 2R | 资源包能力盘点：EPK 类型/密钥/覆盖顺序/路径大小写/移动端交付 | 完成：只有 NONE/COMPRESS/SECURITY，不需要服务器密钥；设备上直接读 EPK |
| 2P | CPython 2.7.18 编进 libmtgodot，五个平台分别配置和验证 | 2A 之后 |
| 2D | 数据源切到 40250（msm 路径、proto、资源根、严格资源测试） | proto 部分在 2A 之后；资源根在 2R 之后 |
| **2P** | CPython 2.7.18 编进 libmtgodot，五个平台分别配置和验证 | **进行中**：step 1（静态库 + macOS/Android/iOS 配置）完成 |
| 2D | 数据源切到 40250（msm 路径、proto、资源根、严格资源测试） | 完成（5a26e93f、232d0461、240d2827、a9b31dec）；资源根只切了 proto/pack 后端，其余按第 5 节由 2V 切片逐个切换 |
| **2V0** | UI 壳：真实 `PythonLauncher/wndMgr/grp/app` + platform UI，其他玩法模块用桩，显示并操作 Logo/Popup | 需要 2A、2P 和 root/uiscript 子集 |
| **2V1** | 登录/选角：真实 `net` phase、登录/选角脚本及所需模块，走到 Loading | 2V0 之后 |
| **2V2** | GamePhase：真实角色管理、主角创建和最小渲染，进入游戏并显示角色 | 2V1、2D 之后 |
@@ -281,6 +281,10 @@ extension/third_party/cpython-2.7.18/ # 静态库（2P 批次
### 批次 2P：内嵌 Python 集成
1. CPython 2.7.18 源码放进 `extension/third_party/cpython-2.7.18/`。**共用源码清单和静态模块清单**（`Modules/Setup` 中启用的 C 模块，见 `tools/py_embed_android/build-and-run.sh`）；**每个平台各自一份 `pyconfig.h`**（由该平台的 configure 生成后提交，或 Windows 用 `PC/pyconfig.h`），因为它是对目标平台类型大小和系统 API 的探测结果，不能共用。已知平台差异：Android API 24 需关掉 `HAVE_LANGINFO_H`。
**已完成**：`mtpython` 静态库（`-DMTGODOT_EMBED_PYTHON=ON`，133 个目标文件 = 参考 `libpython2.7.a` 的全集），
`tools/py_embed/gen_pyconfig.sh <平台>` 生成各平台 `config/<平台>/pyconfig.h` 并校验内建模块表一致，
`extension/tests/py_embed_test.cpp`（ctest `python.embed`）。三处 vendor 补丁见 `docs/THIRD-PARTY.md`。
Linux/Windows 见下表。
2. 静态链接进 `libmtgodot`，照抄 `ScriptLib/PythonLauncher.cpp`。
3. `pack` 模块通过 `asset_io` 读取 40250 的 root/uiscript。标准库沿用已验证的 `python27.zip + sys.path`：
- 桌面端把 zip 放在 CPython 可读的真实文件系统路径；
@@ -291,11 +295,11 @@ extension/third_party/cpython-2.7.18/ # 静态库（2P 批次
| 平台 | 状态 |
| --- | --- |
| macOS arm64 | 系统 Python 2.7.18 跑通 spike；静态嵌入未做 |
| Android arm64 | 独立可执行文件在真机跑通（b02c49bb）；APK 进程内未做 |
| iOS arm64 | 未做 |
| Linux x86_64 | 未做 |
| Windows x64 | 未做（用 `PCbuild` 的源码清单和 `PC/pyconfig.h`，不走 configure） |
| macOS arm64 | step 1 完成：`mtpython` 静态库 + `python.embed` 测试（39 个内建模块全部 import，标准库 codec/pickle 通过）；`system.py` 在 app 进程内未做 |
| Android arm64 | step 1 完成：NDK API 24 交叉编译出 `libmtpython.a`；独立可执行文件在真机跑通（b02c49bb）；APK 进程内未做 |
| iOS arm64 | step 1 完成：iphoneos arm64 编出 133 个目标文件，无 `_system`/`_fork`/`_popen` 未定义符号；真机未跑 |
| Linux x86_64 | 未做（`gen_pyconfig.sh linux` 需要 Linux 主机，当前无） |
| Windows x64 | 未做：上游 2.7 只支持 MSVC + `PC/pyconfig.h`，而本项目 Windows 门禁是 MinGW 交叉编译，`PC/pyconfig.h` 的 gnu-win32 分支不设 `HAVE_UNISTD_H`，`posixmodule.c`/`dynload_win.c` 与 MinGW 头文件冲突，需要一套补丁——在本步骤（step 4）决定走 MSVC 还是打补丁 |
### 批次 2V0–2V3：纵向切片
docs/PYTHON-EMBED-EVAL.md
+5 -2
View File
@@ -96,8 +96,11 @@ Godot 这边只剩一个宿主 Control：把输入转发给 `CPythonWindowManage
## 下一步
1. 把 CPython 2.7.18 放进 `extension/third_party/`，共用源码/静态模块清单，但为 macOS、Linux、Windows、
Android、iOS 分别生成或维护 `pyconfig.h`，静态链接进 `libmtgodot`；
1. ~~把 CPython 2.7.18 放进 `extension/third_party/`，共用源码/静态模块清单，但为 macOS、Linux、Windows、
Android、iOS 分别生成或维护 `pyconfig.h`~~，静态链接进 `libmtgodot`；
源码和三平台配置已完成（2026-09-23，批次 2P step 1）：`mtpython` 静态库（`-DMTGODOT_EMBED_PYTHON=ON`）、
`tools/py_embed/gen_pyconfig.sh`、ctest `python.embed`；macOS/Android/iOS 编译通过，Linux 缺主机、
Windows 的 MinGW 冲突见 `docs/THIRD-PARTY.md`。静态链接进 `libmtgodot` 是 step 2；
2. `pack` 模块走 `asset_io`，继续负责 40250 的 root/uiscript。标准库沿用已验证的 `python27.zip + sys.path`
方案：桌面端放在解释器可读的真实文件系统路径，移动端首次启动从应用资源复制到应用沙盒并校验 sha256；静态启用
`zipimport`。不要依赖 `system.py` 的简单 `name + '.py'` 钩子加载带 package/dotted import 的标准库；
docs/THIRD-PARTY.md
+55 -10
View File
@@ -1,21 +1,25 @@
# Third-party native dependencies
The GDExtension links four native libraries that are **not** part of Godot or
The GDExtension links five native libraries that are **not** part of Godot or
godot-cpp. Phase 1 (macOS) pulled the first three from Homebrew; that stops
working the moment we cross-compile for the Android NDK or the iOS SDK (BACKLOG
F1/F2), so as of 2026-08-30 they are built from source as part of our own CMake
build. Crypto++ joined later (2026-09-02) with the `MT_PROTOCOL=classic` 40250
backend and was vendored the same way from the start.
Everything lives under `extension/third_party/` and is wired by
`extension/third_party/CMakeLists.txt`, which exposes four aliases:
CPython joined on 2026-09-23 with 批次 2P (the embedded 40250 script layer) and is
vendored source as well.
| Alias | Backing target | Consumed by |
|-------------------|------------------|------------------------------|
| `mt3p::sodium` | `sodium` | `mtnet`, `mtpack`, `mtproto` |
| `mt3p::zstd` | `libzstd_static` | `mtpack` |
| `mt3p::minilzo` | `minilzo` | `mtproto` |
| `mt3p::cryptopp` | `cryptopp` | `mtnet` (classic backend) |
Everything lives under `extension/third_party/` and is wired by
`extension/third_party/CMakeLists.txt`, which exposes five aliases:
| Alias | Backing target | Consumed by |
|-------------------|------------------|------------------------------------|
| `mt3p::sodium` | `sodium` | `mtnet`, `mtpack`, `mtproto` |
| `mt3p::zstd` | `libzstd_static` | `mtpack` |
| `mt3p::minilzo` | `minilzo` | `mtproto` |
| `mt3p::cryptopp` | `cryptopp` | `mtnet` (classic backend) |
| `mt3p::python` | `mtpython` | the script layer (`py_embed_test`) |
No source file changed: `#include <sodium.h>`, `#include <zstd.h>`,
`#include <lzo/lzo1x.h>` and `#include <cryptopp/…>` all still resolve (the LZO
@@ -90,11 +94,52 @@ one via a 1-line shim header, see below).
(`extension/third_party/cryptopp/License.txt`). Ship-safe; unlike miniLZO it
imposes no obligation before a public or commercial release.
## CPython 2.7.18 — vendored source (not a submodule)
* **Files:** `extension/third_party/cpython-2.7.18/` (~25 MB), the official
`Python-2.7.18.tgz` (sha256 `b62c0e7937551d0cc02b8fd5cb0f544f9405bafc9a54d3808ed4594812edef43`)
trimmed of what an embedded interpreter never builds or ships: `Doc Demo Tools
Mac PCbuild`, `Modules/{_ctypes,expat,zlib,_sqlite}`, the stdlib test trees and
the pre-VS2010 `PC/` project files. `Lib/` stays — batch 2P step 3 ships it as
`python27.zip`.
* **Why vendored:** 2.7 is end-of-life, so there is no package to fetch on the
four target SDKs, and the build must keep working offline and on the
cross-build machines. It is the version the 40250 scripts are written for
(`ScriptLib` embeds 2.7), so it is pinned by the scripts, not by preference.
* **Built by our own `cpython-2.7.18/CMakeLists.txt`**, not autotools: one
`mtpython` static library from the exact 133 objects the reference
`libpython2.7.a` contains. The source list and the built-in module table
(`config/config.c`, `config/Setup.static`, 39 entries) are **shared**;
`pyconfig.h` is **not** — it is a probe result, so each platform keeps its own
under `config/<platform>/`, regenerated by `tools/py_embed/gen_pyconfig.sh`.
Off by default: `-DMTGODOT_EMBED_PYTHON=ON` builds it.
* **Three vendor patches** (everything else is verbatim upstream):
1. `configure` + 2. `configure.ac` — add `arm64) MACOSX_DEFAULT_ARCH="arm64"`
to both `case \`/usr/bin/arch\`` blocks; upstream predates Apple silicon and
dies with "Unexpected output of 'arch' on OSX".
3. `Modules/posixmodule.c` — after the "all other compilers" block that
hard-defines `HAVE_SYSTEM`/`FORK`/`EXECV`/`WAIT`/`POPEN` regardless of
`pyconfig.h`, undefine them again under `TARGET_OS_IPHONE`: the iphoneos SDK
marks `system()` unavailable and the sandbox forbids the rest.
* **Platform notes:** Android API 24 turns off `HAVE_LANGINFO_H` (bionic declares
`nl_langinfo` only from API 26); iOS derives its `pyconfig.h` from the macOS one
(2.7's configure refuses a darwin cross build) and drops process control plus
`<sys/random.h>`/`getentropy`, but keeps `HAVE_GETGROUPS` because
`posixmodule.c` defines `MAX_GROUPS` — which its `setgroups` path needs — only
inside that `#ifdef`. **Windows is not built yet**: upstream supports MSVC +
`PC/pyconfig.h`, while our Windows gate cross-compiles with MinGW, where
`PC/pyconfig.h`'s gnu-win32 branch leaves `HAVE_UNISTD_H` unset and
`posixmodule.c` / `dynload_win.c` then collide with the MinGW headers. Decided
in 批次 2P step 4 (PORT-PLAN).
* **License:** PSF License Agreement (`cpython-2.7.18/LICENSE`). Ship-safe; the
only obligation is keeping the license and a change notice, which this section
and the patched files' comments serve.
## Rebuilding / fetching
`./build.sh` auto-runs `git submodule update --init --recursive` if any of the
three submodules (godot-cpp, zstd, libsodium-cmake+libsodium) is missing.
miniLZO and Crypto++ are vendored source and need no fetch.
miniLZO, Crypto++ and CPython are vendored source and need no fetch.
Homebrew `libsodium` / `zstd` / `lzo` are no longer referenced by the build and
can be uninstalled.