port(2P step 3c): stage python27.zip into the mobile sandbox

CPython opens the stdlib zip with its own stdio, so it needs a real path.
On the desktop res:// is a directory and globalize_path() is enough; in an
exported build res:// is inside the PCK, which nothing outside Godot reads.

  make_stdlib_zip.py now also writes <out>.sha256
  mtpython_stdlib_project copies python27.zip + .sha256 into project/
    (gitignored), and both export presets' include_filter lists them, so
    they enter the PCK — checked with --export-pack
  extension/src/python_stdlib.{h,cpp} (Metin2Python.stdlib_path in
    GDScript): reads the bytes out of the PCK, writes
    user://python27.zip.part, hashes the file as written against the
    shipped digest, then renames. A sandbox copy is reused only when it
    matches, so a first start killed mid-copy and a zip replaced by a new
    build both re-stage instead of feeding zipimport a torn file.
  PythonHost::SetDefaultStdLibPath / DefaultStdLibPath() answer with that
    path ($MT_PYTHON_STDLIB still wins). python_stdlib.cpp is the only unit
    that knows res:// / user://; port_platform stays godot-free.

Also fixes the Windows gate, which step 3b broke: without mtpython the
MinGW build compiled UserInterface/StdAfx.h (it includes ScriptLib/StdAfx.h,
as the original PCH does). port/CMakeLists.txt excludes that header and
PythonPackModule.cpp with ScriptLib, and platform/CMakeLists.txt excludes
platform/ScriptLib/ the same way.

Not done: on-device staging. The macOS test drives the mobile path with
force_stage=true, and nothing in the app boot calls it yet — the
interpreter only starts in the process with 2V0.

gates: python_stdlib_test.gd PASS (in-place path, staging, sha256, bytes
equal res://, no .part, ZIPReader finds encodings/__init__.py, no re-copy
on a second call, corrupted copy re-staged) · macOS ctest 27/27 ·
mingw + android + ios port_platform compile clean · port_map.py check 0
errors · key leak check 8/8 none.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
shenlei
2026-09-23 09:39:33 +09:00
co-authored by Claude Opus 5
parent 2b873f190f
commit ffbb5d48d6
17 changed files with 329 additions and 18 deletions
+4
View File
@@ -25,6 +25,10 @@ compile_commands.json
/project/bin/*.wasm
/project/bin/ios/
/project/bin/android/
# The embedded interpreter's stdlib, copied here from the build tree so the exporter packs it
# (extension/CMakeLists.txt, 批次 2P step 3c). Derived — never committed.
/project/python27.zip
/project/python27.zip.sha256
# Android Gradle build template (~1.2 GB of regenerable Gradle output + Godot
# .aar). Recreate per machine:
# godot --headless --path project --install-android-build-template \
+1
View File
@@ -495,3 +495,4 @@
{"time": "2026-09-23T15:30:00Z", "event": "third_party", "unit": "PORT-PLAN 批次 2P step 1: vendored CPython 2.7.18 as the mtpython static library", "source": "official Python-2.7.18.tgz sha256 b62c0e7937551d0cc02b8fd5cb0f544f9405bafc9a54d3808ed4594812edef43, trimmed to 25MB (Doc/Demo/Tools/Mac/PCbuild, Modules/{_ctypes,expat,zlib,_sqlite}, stdlib test trees, pre-VS2010 PC projects); Lib/ kept for step 3's python27.zip", "shared": "source list = the 133 objects of the reference libpython2.7.a; built-in table config/config.c = 39 entries (config/Setup.static)", "per_platform": "config/<plat>/pyconfig.h from tools/py_embed/gen_pyconfig.sh; android undefs HAVE_LANGINFO_H (nl_langinfo needs API 26); ios is derived from macos (2.7 configure refuses a darwin cross build) and drops process control + sys/random.h, keeping HAVE_GETGROUPS because posixmodule.c defines MAX_GROUPS only inside it", "patches": ["configure + configure.ac: arm64 MACOSX_DEFAULT_ARCH", "Modules/posixmodule.c: undef HAVE_SYSTEM/POPEN/FORK/EXECV/WAIT under TARGET_OS_IPHONE, which its own 'all other compilers' block hard-defines past pyconfig.h"], "not_done": ["Linux x86_64: gen_pyconfig.sh linux needs a Linux host", "Windows x64: upstream supports MSVC + PC/pyconfig.h while the Windows gate is MinGW, where PC/pyconfig.h leaves HAVE_UNISTD_H unset and posixmodule.c/dynload_win.c collide with the MinGW headers; decided in step 4"], "ported": [], "deleted": [], "divergent": [], "needs_live": [], "status_note": "no port-map entry: this is third-party source, not a ported 40250 unit. Linking into libmtgodot is step 2 (ScriptLib/PythonLauncher.cpp)", "tests": ["ctest python.embed (macOS arm64): 39 builtin modules, all import, sys.version 2.7.18, utf-8/cp1252 codecs + pickle round-trip via the vendored Lib/ PASS", "android arm64 NDK API 24: libmtpython.a builds clean", "ios arm64 iphoneos: libmtpython.a = 133 objects, no undefined _system/_fork/_popen"], "commit": "2P-step1"}
{"time": "2026-09-23T17:30:00Z", "event": "port_round", "unit": "PORT-PLAN 批次 2P step 2: ScriptLib/PythonLauncher.cpp into port_logic on the embedded interpreter", "ported": ["ScriptLib/PythonLauncher.cpp (10 of 12 fns)", "ScriptLib/PythonMarshal.cpp (4 of 11)"], "adapted": [], "deleted": [], "divergent": [], "needs_live": [], "status_note": "statuses set only for what port.python_launcher actually runs. PythonUtils.cpp is copied and compiled but nothing calls it yet (the binding modules are 2V0), so all 19 stay TODO; TraceFunc/SetTraceFunc are _DEBUG-only; in PythonMarshal only the small-.pyc path runs (r_long, _PyMarshal_ReadLongFromFile, getfilesize, _PyMarshal_ReadLastObjectFromFile) because that one delegates to CPython's own PyMarshal_ReadObjectFromString under 16KB, leaving the ported r_object reader unreached. Two // PORT: edits: `unsigned char(val)` -> a cast (MSVC extension) in PythonUtils.cpp x2, and PyObject_AsCharBuffer's Py_ssize_t* in TraceFunc. CPythonLauncher::GetError dereferences the NULL PyErr_Fetch yields when nothing is pending — kept verbatim, the test only calls it with an error set.", "shims": "extension/src/port/common/shim/sdk/Python-2.7/: 12 forwarding headers for 40250's #include <Python-2.7/*>; symtable.h forward-declares mod_ty instead of including Python-ast.h, whose one-word macros (Delete, Module, Set, Print) collide with EterLib/Pool.h — 40250 sidestepped the same clash by commenting out PySymtable_Build", "tests": ["ctest port.python_launcher (macOS arm64): Create/RunLine/RunMemoryTextFile(CRLF)/RunCompiledFile(.pyc)/RunFile PASS; RunFile(\"system.py\") reads the real pack and stops at ImportError: No module named app (the 2V0 C++ modules)", "ctest macOS: 27/27 PASS", "android arm64 NDK API 24 + ios arm64 iphoneos: port_logic (incl. ScriptLib) compiles"], "commit": "2P-step2"}
{"time": "2026-09-23T19:30:00Z", "event": "port_round", "unit": "PORT-PLAN 批次 2P step 3a/3b: python27.zip on sys.path (platform/ScriptLib/PythonHost) + the pack module", "ported": ["UserInterface/PythonPackModule.cpp (3 of 5 fns: packExist, packGet, initpack)"], "adapted": [], "deleted": [], "divergent": [], "needs_live": [], "stdlib": {"builder": "tools/py_embed/make_stdlib_zip.py <Lib> <out.zip>, CMake target mtpython_stdlib -> <build>/python27.zip", "contents": "397 modules, 6876547 bytes, ZIP_STORED (no zlib module, so zipimport cannot inflate); deterministic (sorted, fixed timestamp) so the sha256 identifies it", "sha256": "5fce8f974a89b66110823d7529be96e359b56dea05fc6875189a241d2d37a7d6", "skipped": "test/tests trees, lib-tk, idlelib, lib2to3, bsddb, ensurepip, curses, distutils, unittest, wsgiref, multiprocessing, compiler, plat-*", "host": "platform/ScriptLib/PythonHost.cpp: Configure() sets Py_NoSiteFlag/IgnoreEnvironment/Frozen/DontWriteBytecode + program name before Py_Initialize; InstallStdLib() does PySys_SetPath then re-registers encodings.search_function, because 2.7 initializes the codec registry once at startup and never retries"}, "status_note": "PythonPackModule's other two functions are CAccountConnector's login key builder, which happens to live in that unit; kept verbatim under #if 0 until the 2V1 login slice brings AccountConnector.h. port/UserInterface/StdAfx.h is a partial mirror: the original includes every library's StdAfx plus <dshow.h>/<qedit.h>, so it carries only what compiles today (ScriptLib/StdAfx.h, APP_NAME, the name length enum, initpack) and grows with the 2V slices.", "tests": ["ctest python.embed now runs against python27.zip itself (zipimport), not the loose Lib/", "ctest port.python_launcher: InstallStdLib + `import os, string, types, codecs` + cp1252 encode; initpack then pack.Exist/pack.Get for system.py (byte count equals CEterPackManager's), uiscript/selectcharacterwindow.py and locale/en/locale_game.txt; .tga and a missing file give None", "ctest macOS: 27/27 PASS", "android arm64 + ios arm64: port_platform (incl. PythonHost, PythonPackModule) compiles"], "not_done": ["3c: stage python27.zip out of the app resources into the sandbox on Android/iOS and check it against the manifest sha256; PythonHost::DefaultStdLibPath() only reads MT_PYTHON_STDLIB today"], "commit": "2P-step3ab"}
{"time": "2026-09-23T21:40:00Z", "event": "port_round", "unit": "PORT-PLAN 批次 2P step 3c: staging python27.zip into the mobile sandbox", "ported": [], "adapted": [], "deleted": [], "divergent": [], "needs_live": [], "staging": {"ships": "make_stdlib_zip.py also writes <out>.sha256; target mtpython_stdlib_project copy_if_different's python27.zip + .sha256 into project/ (gitignored), and both export presets' include_filter lists them so they enter the PCK", "runtime": "extension/src/python_stdlib.cpp (GDScript: Metin2Python.stdlib_path(force_stage)) — desktop uses ProjectSettings.globalize_path(res://python27.zip) in place; exported builds read the bytes out of the PCK, write user://python27.zip.part, verify the sha256 of the written file, then rename to user://python27.zip. An existing sandbox copy is reused only when it hashes to the shipped digest, so a torn first start or a new build re-stages.", "handoff": "PythonHost::SetDefaultStdLibPath() — DefaultStdLibPath() now answers with the staged path; $MT_PYTHON_STDLIB still wins. python_stdlib.cpp is the only unit that knows res:// / user://; port_platform stays godot-free.", "build_fix": "the Windows gate (MinGW, no mtpython) was broken by step 3b and is fixed here: port/CMakeLists.txt now also excludes UserInterface/StdAfx.h (it includes ScriptLib/StdAfx.h, as the original PCH does) and UserInterface/PythonPackModule.cpp, and platform/CMakeLists.txt excludes platform/ScriptLib/ the way port_logic already excluded port/ScriptLib/."}, "status_note": "On-device staging is not verified: the macOS test drives the mobile path with force_stage=true, and Android/iOS user:// sandboxes come with step 4. The zip's presence in an exported pack was checked with --export-pack, not with an APK. Nothing in the app's boot path calls Metin2Python.stdlib_path() yet — the interpreter only starts in the process with 2V0; today the test is its only caller.", "tests": ["project/python_stdlib_test.gd (godot --headless --path project --script python_stdlib_test.gd) -> PASS: in-place path on desktop; force_stage copies into user://, sha256 matches the shipped digest, bytes equal res://, no .part left, ZIPReader opens it and finds encodings/__init__.py and os.py; a second call does not re-copy (mtime unchanged); a deliberately corrupted sandbox copy is re-staged", "godot --headless --export-pack macOS <zip> lists python27.zip (6876547) and python27.zip.sha256 (65)"], "not_done": "真机验证(Android/iOS 进程内落盘 + system.py -> prototype.RunApp())随 2P step 4Linux/Windows 见 step 4 表格", "commit": "2P-step3c"}
+16 -2
View File
@@ -312,8 +312,22 @@ extension/third_party/cpython-2.7.18/ # 静态库(2P 批次
`locale/en/locale_game.txt`,非 `.py/.pyc/.txt` 和不存在的文件返回 `None`
`UserInterface/StdAfx.h` 只镜像了当前能编的部分(原文件 include 整个客户端和 DirectShow),
`PythonPackModule.cpp` 里属于 `CAccountConnector` 的登录密钥函数原样保留但编译时排除,随 2V1 回来。
**完成(3c**Android/iOS 从应用资源把 zip 复制进沙盒并校验 sha256;`PythonHost::DefaultStdLibPath()`
现在只认 `MT_PYTHON_STDLIB`
**完成(3c 落盘**`make_stdlib_zip.py` 额外写出 `python27.zip.sha256`;构建把两个文件
`copy_if_different``project/`(目标 `mtpython_stdlib_project`,两文件 gitignore),
导出预设的 `include_filter` 列出它们,所以它们进 PCK——`--export-pack` 验证过两个条目都在。
`extension/src/python_stdlib.{h,cpp}`GDScript 侧 `Metin2Python.stdlib_path(force_stage)`):
桌面 `res://` 就是磁盘目录,`globalize_path` 得到真实路径直接用,不复制;导出后 `res://` 在 PCK 里,
于是读出字节 → 写 `user://python27.zip.part` → 按随包 sha256 校验**落盘后的文件**(不是内存里的 buffer,
磁盘满要在这里报,而不是三个 import 之后变成 zipimport 失败)→ rename 成 `user://python27.zip`
已经在沙盒里且 sha 相符就直接复用;被杀在半路的半成品或换了一版构建的旧 zip 都会重新落盘。
路径交给 `PythonHost::SetDefaultStdLibPath()``DefaultStdLibPath()` 从此答得出(`MT_PYTHON_STDLIB` 仍优先)。
测试 `project/python_stdlib_test.gd``godot --headless --path project --script python_stdlib_test.gd`)在
macOS 上用 `force_stage=true` 跑的就是移动端那条路径:落盘、sha 一致、字节与 `res://` 相同、`.part` 不残留、
`ZIPReader` 能打开且含 `encodings/__init__.py`,重复调用不重复复制,人为弄坏副本后重新落盘。
**未完成**:真机上跑(Android/iOS 的 `user://` 沙盒路径本身没验过),随 step 4 一起;
app 的启动流程还没有调用它——解释器要到 2V0 才在进程里起来,现在只有测试调。
`port_platform` 现在在没有 `mtpython` 时也会排除 `platform/ScriptLib/`(与 `port_logic` 同一条规则),
否则 Windows 门禁会编到 `PythonHost.cpp`
4. 五个平台分别验证:编译、链接无未定义符号、`Py_Initialize`、静态 C 模块逐个 `import`、在 app 进程里跑 `system.py``prototype.RunApp()`(对照结果:74 个模块中 66 个加载成功,引导期调用 33 个 C++ 函数)。
| 平台 | 状态 |
+4 -1
View File
@@ -104,7 +104,10 @@ one via a 1-line shim header, see below).
packs it into `<build>/python27.zip` with `tools/py_embed/make_stdlib_zip.py`,
which is what the host puts on `sys.path` (batch 2P step 3). Entries are
**stored, never deflated** — the `zlib` module is trimmed out above, so
`zipimport` has nothing to inflate with.
`zipimport` has nothing to inflate with. The zip and its `.sha256` are copied
into `project/` (gitignored) so the exporter packs them; on Android/iOS
`extension/src/python_stdlib.cpp` stages the zip out of the PCK into `user://`
and checks that digest, because CPython opens it with its own stdio.
* **Why vendored:** 2.7 is end-of-life, so there is no package to fetch on the
four target SDKs, and the build must keep working offline and on the
cross-build machines. It is the version the 40250 scripts are written for
+19
View File
@@ -198,10 +198,29 @@ add_library(mtgodot ${MT_LIB_KIND}
src/net/m2_client.cpp
src/proto/proto_node.cpp
src/pack40250_node.cpp
src/python_stdlib.cpp
)
target_compile_features(mtgodot PRIVATE cxx_std_20)
target_link_libraries(mtgodot PRIVATE godot::cpp xrender::libgr2 xrender::formats mtnet mtproto port_platform)
# The embedded interpreter's standard library (批次 2P step 3c). python27.zip and its digest are
# copied next to project.godot so the exporter packs them into the APK/IPA (the export presets'
# include_filter lists them); src/python_stdlib.cpp stages them out of the PCK at runtime.
if(TARGET mtpython_stdlib)
target_compile_definitions(mtgodot PRIVATE MTGODOT_HAVE_PYTHON)
# A target rather than a custom command on the two files: the command that produces them lives in
# third_party/, and a Makefile generator only resolves a generated-file dependency inside the
# directory that declared it. copy_if_different keeps the no-op build a compare.
add_custom_target(mtpython_stdlib_project ALL
COMMAND ${CMAKE_COMMAND} -E copy_if_different
"${MT_PYTHON_STDLIB_ZIP}" "${CMAKE_SOURCE_DIR}/project/python27.zip"
COMMAND ${CMAKE_COMMAND} -E copy_if_different
"${MT_PYTHON_STDLIB_SHA}" "${CMAKE_SOURCE_DIR}/project/python27.zip.sha256"
COMMENT "Copying python27.zip into project/ (res://)")
add_dependencies(mtpython_stdlib_project mtpython_stdlib)
add_dependencies(mtgodot mtpython_stdlib_project)
endif()
# Godot's .gdextension expects, per platform:
# project/bin/libmtgodot.macos.template_{debug,release}.dylib
# project/bin/libmtgodot.ios.template_{debug,release}.a
+5
View File
@@ -4,6 +4,11 @@
# so it shares the per-library definitions.
file(GLOB_RECURSE MT_PLATFORM_SOURCES CONFIGURE_DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/*.cpp)
# Same rule as port_logic: without the embedded interpreter there is no ScriptLib to back
# (PORT-PLAN 批次 2P step 4 — Windows).
if(NOT TARGET mtpython)
list(FILTER MT_PLATFORM_SOURCES EXCLUDE REGEX "/ScriptLib/")
endif()
add_library(port_platform STATIC ${MT_PLATFORM_SOURCES})
target_link_libraries(port_platform PUBLIC port_logic)
foreach(src IN LISTS MT_PLATFORM_SOURCES)
@@ -94,13 +94,24 @@ bool InstallStdLib(const char* stdlib_path, std::string* error)
return true;
}
namespace
{
std::string g_staged_path;
}
void SetDefaultStdLibPath(const char* stdlib_path)
{
g_staged_path = stdlib_path ? stdlib_path : "";
}
std::string DefaultStdLibPath()
{
// The environment still wins, so a developer can point a build at another stdlib without rebuilding.
if (const char* env = getenv("MT_PYTHON_STDLIB"))
return env;
// Step 3c stages python27.zip out of the packed assets on Android/iOS and answers here; until then
// the caller passes the path (the build tree's copy, for the native tests).
return "";
// Whatever staged the zip for this platform (extension/src/python_stdlib.cpp inside Godot; nothing
// in the native tests, which pass the build tree's copy on the command line).
return g_staged_path;
}
}
@@ -20,8 +20,9 @@ void Configure(const char* program_name = "eter.python");
// and re-registers the codec search function. False leaves *error set.
bool InstallStdLib(const char* stdlib_path, std::string* error);
// The stdlib path this build was given: $MT_PYTHON_STDLIB when set, else the python27.zip next to
// the executable, else "". Mobile staging (copy out of the app resources, checked against the
// manifest sha256) is 批次 2P step 3c and will answer here too.
// Where the host put the stdlib: $MT_PYTHON_STDLIB when set, else whatever SetDefaultStdLibPath was
// given, else "". Inside Godot that is the staged python27.zip (extension/src/python_stdlib.h), which
// on Android/iOS is the sandbox copy of the one in the read-only bundle.
std::string DefaultStdLibPath();
void SetDefaultStdLibPath(const char* stdlib_path);
}
+5 -2
View File
@@ -11,9 +11,12 @@ if(WIN32)
endif()
# The script layer needs the embedded interpreter (mt3p::python), which has no Windows build yet
# (PORT-PLAN 批次 2P step 4). Without it, ScriptLib and its <Python-2.7/*> shims are left out.
# The units outside ScriptLib that reach it: UserInterface/StdAfx.h includes ScriptLib/StdAfx.h (as the
# original PCH does), and PythonPackModule.cpp is a Python module. Both go with it.
set(MT_PORT_NEEDS_PYTHON "/ScriptLib/|/UserInterface/StdAfx\\.h|/UserInterface/PythonPackModule\\.cpp")
if(NOT TARGET mtpython)
list(FILTER MT_PORT_SOURCES EXCLUDE REGEX "/ScriptLib/")
list(FILTER MT_PORT_HEADERS EXCLUDE REGEX "/ScriptLib/|/common/shim/sdk/Python-2\\.7/")
list(FILTER MT_PORT_SOURCES EXCLUDE REGEX "${MT_PORT_NEEDS_PYTHON}")
list(FILTER MT_PORT_HEADERS EXCLUDE REGEX "${MT_PORT_NEEDS_PYTHON}|/common/shim/sdk/Python-2\\.7/")
endif()
add_library(port_logic STATIC ${MT_PORT_SOURCES})
+127
View File
@@ -0,0 +1,127 @@
#include "python_stdlib.h"
#include <godot_cpp/classes/dir_access.hpp>
#include <godot_cpp/classes/file_access.hpp>
#include <godot_cpp/classes/project_settings.hpp>
#include <godot_cpp/core/class_db.hpp>
#include <godot_cpp/variant/utility_functions.hpp>
#include <string>
#ifdef MTGODOT_HAVE_PYTHON
#include "platform/ScriptLib/PythonHost.h"
#endif
using namespace godot;
namespace mtgodot {
namespace {
// Both ship in the PCK; the build copies them next to project.godot (extension/CMakeLists.txt).
constexpr const char *ZIP_RES = "res://python27.zip";
constexpr const char *SHA_RES = "res://python27.zip.sha256";
constexpr const char *ZIP_USER = "user://python27.zip";
constexpr const char *ZIP_USER_TMP = "user://python27.zip.part";
// std::string, not String: this lives at file scope and a godot::String's constructor would run at
// dlopen time, before the extension's bindings exist.
std::string g_last_error;
String fail(String *error, const String &text) {
g_last_error = text.utf8().get_data();
if (error) {
*error = text;
}
return String();
}
String expected_sha() {
if (!FileAccess::file_exists(SHA_RES)) {
return String();
}
return FileAccess::get_file_as_string(SHA_RES).strip_edges();
}
} // namespace
String stage_python_stdlib(String *error, bool force_stage) {
g_last_error.clear();
if (error) {
*error = String();
}
if (!FileAccess::file_exists(ZIP_RES)) {
return fail(error, String("no ") + ZIP_RES +
" (build with -DMTGODOT_EMBED_PYTHON=ON, which copies it into project/)");
}
ProjectSettings *settings = ProjectSettings::get_singleton();
// Desktop and the editor: res:// is a directory on disk, so CPython can open the file in place and
// nothing is copied. In an exported game globalize_path() returns a path next to the executable
// that does not exist, which is exactly the case the staging below is for.
const String in_place = settings->globalize_path(ZIP_RES);
if (!force_stage && !in_place.begins_with("res://") && FileAccess::file_exists(in_place)) {
return in_place;
}
const String want = expected_sha();
if (want.is_empty()) {
return fail(error, String("no ") + SHA_RES + " beside the stdlib zip");
}
// An already-staged copy is reused only when it is the zip this build ships: a new version of the
// app brings a new digest, and a first start that was killed mid-copy leaves a short file.
if (FileAccess::file_exists(ZIP_USER) && FileAccess::get_sha256(ZIP_USER) == want) {
return settings->globalize_path(ZIP_USER);
}
const PackedByteArray data = FileAccess::get_file_as_bytes(ZIP_RES);
if (data.is_empty()) {
return fail(error, String("cannot read ") + ZIP_RES);
}
{
Ref<FileAccess> out = FileAccess::open(ZIP_USER_TMP, FileAccess::WRITE);
if (out.is_null()) {
return fail(error, String("cannot write ") + ZIP_USER_TMP + ": " +
itos((int) FileAccess::get_open_error()));
}
out->store_buffer(data);
out->close();
}
// Hash the file as written, not the buffer: a full disk reports itself here and not later as a
// zipimport failure three imports into system.py.
if (FileAccess::get_sha256(ZIP_USER_TMP) != want) {
DirAccess::remove_absolute(ZIP_USER_TMP);
return fail(error, String("staged ") + ZIP_USER + " does not match " + want);
}
if (FileAccess::file_exists(ZIP_USER)) {
DirAccess::remove_absolute(ZIP_USER);
}
if (DirAccess::rename_absolute(ZIP_USER_TMP, ZIP_USER) != OK) {
return fail(error, String("cannot rename ") + ZIP_USER_TMP + " to " + ZIP_USER);
}
UtilityFunctions::print("[python27] staged ", ZIP_USER, " (", data.size(), " bytes, sha256 ", want, ")");
return settings->globalize_path(ZIP_USER);
}
String Metin2Python::stdlib_path(bool force_stage) {
const String path = stage_python_stdlib(nullptr, force_stage);
#ifdef MTGODOT_HAVE_PYTHON
if (!path.is_empty()) {
PythonHost::SetDefaultStdLibPath(path.utf8().get_data());
}
#endif
return path;
}
String Metin2Python::last_error() {
return String::utf8(g_last_error.c_str());
}
void Metin2Python::_bind_methods() {
ClassDB::bind_static_method("Metin2Python", D_METHOD("stdlib_path", "force_stage"),
&Metin2Python::stdlib_path, DEFVAL(false));
ClassDB::bind_static_method("Metin2Python", D_METHOD("last_error"), &Metin2Python::last_error);
}
} // namespace mtgodot
+37
View File
@@ -0,0 +1,37 @@
#pragma once
// python_stdlib — where the embedded interpreter's standard library is on this platform
// (docs/PORT-PLAN.md 批次 2P step 3c).
//
// CPython opens python27.zip with its own stdio, so it needs a real filesystem path. That is free on
// the desktop, where res:// is a directory on disk, but on Android and iOS res:// is inside the PCK,
// which nothing outside Godot can read. There the zip is copied once into the sandbox (user://) and
// the copy is accepted only when it hashes to the digest built beside the zip, so an interrupted
// first start or a zip replaced by a new build re-stages instead of feeding zipimport a torn file.
//
// Only this file knows about res:// / user://: PythonHost (platform/ScriptLib) stays godot-free and
// is told the path, through PythonHost::SetDefaultStdLibPath.
#include <godot_cpp/classes/object.hpp>
#include <godot_cpp/variant/string.hpp>
namespace mtgodot {
// The real filesystem path of python27.zip, staging it out of res:// when res:// is not on disk.
// Empty when the zip is missing or the staged copy does not verify; the reason goes to *error.
// force_stage runs the mobile path even where res:// is readable, which is how it is tested.
godot::String stage_python_stdlib(godot::String *error = nullptr, bool force_stage = false);
class Metin2Python : public godot::Object {
GDCLASS(Metin2Python, godot::Object)
public:
// Returns the path and, when the embedded interpreter is built in, hands it to PythonHost so
// PythonHost::DefaultStdLibPath() answers with it from then on.
static godot::String stdlib_path(bool force_stage);
static godot::String last_error();
protected:
static void _bind_methods();
};
} // namespace mtgodot
+2
View File
@@ -12,6 +12,7 @@
#include "net/m2_client.h"
#include "pack40250_node.h"
#include "proto/proto_node.h"
#include "python_stdlib.h"
#include "static_object.h"
#include "terrain_splat.h"
#include "tree_placeholder.h"
@@ -45,6 +46,7 @@ void initialize_mtgodot_module(ModuleInitializationLevel p_level) {
GDREGISTER_CLASS(mtgodot::M2Client);
GDREGISTER_CLASS(mtgodot::Metin2Proto);
GDREGISTER_CLASS(mtgodot::Metin2Pack);
GDREGISTER_CLASS(mtgodot::Metin2Python);
}
void uninitialize_mtgodot_module(ModuleInitializationLevel p_level) {
+3 -2
View File
@@ -215,10 +215,11 @@ add_library(mt3p::python ALIAS mtpython)
# is where every consumer (the native tests now, the mobile staging in step 3c) picks it up.
find_package(Python3 COMPONENTS Interpreter REQUIRED)
set(MT_PYTHON_STDLIB_ZIP "${CMAKE_BINARY_DIR}/python27.zip" CACHE INTERNAL "stdlib zip for the embedded interpreter")
set(MT_PYTHON_STDLIB_SHA "${MT_PYTHON_STDLIB_ZIP}.sha256" CACHE INTERNAL "sha256 of MT_PYTHON_STDLIB_ZIP")
file(GLOB_RECURSE MT_PYTHON_STDLIB_SOURCES CONFIGURE_DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/Lib/*.py")
add_custom_command(OUTPUT "${MT_PYTHON_STDLIB_ZIP}"
add_custom_command(OUTPUT "${MT_PYTHON_STDLIB_ZIP}" "${MT_PYTHON_STDLIB_SHA}"
COMMAND ${Python3_EXECUTABLE} "${CMAKE_SOURCE_DIR}/tools/py_embed/make_stdlib_zip.py"
"${CMAKE_CURRENT_SOURCE_DIR}/Lib" "${MT_PYTHON_STDLIB_ZIP}"
DEPENDS "${CMAKE_SOURCE_DIR}/tools/py_embed/make_stdlib_zip.py" ${MT_PYTHON_STDLIB_SOURCES}
COMMENT "Packing python27.zip from the vendored Lib/")
add_custom_target(mtpython_stdlib ALL DEPENDS "${MT_PYTHON_STDLIB_ZIP}")
add_custom_target(mtpython_stdlib ALL DEPENDS "${MT_PYTHON_STDLIB_ZIP}" "${MT_PYTHON_STDLIB_SHA}")
+2 -2
View File
@@ -7,7 +7,7 @@ advanced_options=false
dedicated_server=false
custom_features=""
export_filter="all_resources"
include_filter=""
include_filter="python27.zip,python27.zip.sha256"
exclude_filter=""
export_path="../build/export/mtgodot-poc.app"
patches=PackedStringArray()
@@ -69,7 +69,7 @@ advanced_options=false
dedicated_server=false
custom_features=""
export_filter="all_resources"
include_filter=""
include_filter="python27.zip,python27.zip.sha256"
exclude_filter=""
export_path="../build/export/mtgodot-poc.apk"
patches=PackedStringArray()
+77
View File
@@ -0,0 +1,77 @@
# python_stdlib_test —— 内嵌解释器标准库的落盘(docs/PORT-PLAN.md 批次 2P step 3c)。
# 桌面上 res:// 就是磁盘目录,CPython 直接打开;Android/iOS 的 res:// 在 PCK 里,必须先复制进
# user:// 沙箱并按随包发的 sha256 校验。force_stage=true 就是在桌面上跑那条移动端路径。
# godot --headless --path project --script python_stdlib_test.gd
extends SceneTree
const ZIP_USER := "user://python27.zip"
const ZIP_PART := "user://python27.zip.part"
var _fail := 0
func _ck(ok: bool, message: String) -> void:
if not ok:
_fail += 1
printerr("FAIL: " + message)
func _init() -> void:
if not ClassDB.class_exists("Metin2Python"):
printerr("FAIL: Metin2Python extension is not registered")
quit(1)
return
if not FileAccess.file_exists("res://python27.zip"):
print("SKIP: python_stdlib_test(没有 res://python27.zip,需要 -DMTGODOT_EMBED_PYTHON=ON 的构建)")
quit(0)
return
var want := FileAccess.get_file_as_string("res://python27.zip.sha256").strip_edges()
_ck(want.length() == 64, "res://python27.zip.sha256 是一行 sha256")
# 桌面:res:// 在磁盘上,原地用,不复制。
if DirAccess.dir_exists_absolute(ProjectSettings.globalize_path("res://")):
var in_place := Metin2Python.stdlib_path(false)
_ck(in_place == ProjectSettings.globalize_path("res://python27.zip"),
"res:// 可直接读时返回原路径,不落盘:" + Metin2Python.last_error())
# 移动端路径:复制进沙箱,校验 sha256,返回真实文件系统路径。
if FileAccess.file_exists(ZIP_USER):
DirAccess.remove_absolute(ZIP_USER)
var staged := Metin2Python.stdlib_path(true)
_ck(staged != "", "落盘成功:" + Metin2Python.last_error())
_ck(staged == ProjectSettings.globalize_path(ZIP_USER), "返回的是 user:// 的真实路径")
_ck(FileAccess.file_exists(staged), "返回的路径在文件系统上存在")
_ck(FileAccess.get_sha256(ZIP_USER) == want, "落盘的 zip 与随包的 sha256 一致")
_ck(not FileAccess.file_exists(ZIP_PART), "不留下半成品 .part")
_ck(FileAccess.get_file_as_bytes(ZIP_USER) == FileAccess.get_file_as_bytes("res://python27.zip"),
"落盘的字节与 res:// 里的一致")
# zipimport 读不了 deflate(没有 zlib 模块),所以内容必须是 STORED。
var reader := ZIPReader.new()
var opened := reader.open(staged)
_ck(opened == OK, "落盘的 zip 能打开")
if opened == OK:
_ck(reader.file_exists("encodings/__init__.py"), "zip 里有 encodings 包")
_ck(reader.file_exists("os.py"), "zip 里有 os.py")
reader.close()
# 第二次启动:已经在沙箱里且 sha 对得上,直接复用。
var modified := FileAccess.get_modified_time(ProjectSettings.globalize_path(ZIP_USER))
var again := Metin2Python.stdlib_path(true)
_ck(again == staged, "第二次返回同一路径")
_ck(FileAccess.get_modified_time(ProjectSettings.globalize_path(ZIP_USER)) == modified,
"sha 相符时不重新复制")
# 上次启动被杀在半路 / 换了一版构建:沙箱里的副本对不上就重新落盘。
var f := FileAccess.open(ZIP_USER, FileAccess.WRITE)
f.store_string("torn")
f.close()
_ck(FileAccess.get_sha256(ZIP_USER) != want, "已经把沙箱里的副本弄坏")
var restaged := Metin2Python.stdlib_path(true)
_ck(restaged == staged, "坏副本之后仍返回同一路径:" + Metin2Python.last_error())
_ck(FileAccess.get_sha256(ZIP_USER) == want, "坏副本被重新落盘覆盖")
if _fail == 0:
print("PASS: python_stdlib_test (res:// -> user:// staging + sha256)")
quit(0)
else:
printerr("%d check(s) failed" % _fail)
quit(1)
+1
View File
@@ -0,0 +1 @@
uid://cfmd68ffjkrtv
+8 -3
View File
@@ -11,12 +11,15 @@ tools/py_embed_android/build-and-run.sh (PORT-PLAN 批次 2P step 3).
Entries are STORED, never deflated: the zlib module is not in the vendored Modules/ (see
docs/THIRD-PARTY.md), so zipimport could not inflate them. Only .py goes in; zipimport reads
source as happily as bytecode, and .pyc would pin the magic number to the build host's interpreter.
Written deterministically (sorted, fixed timestamp) so the file's sha256 identifies its contents
3c checks that hash when staging the zip into a mobile sandbox.
Written deterministically (sorted, fixed timestamp) so the file's sha256 identifies its contents.
The hash is written next to the zip as <out>.sha256 and ships with it: on Android/iOS the zip lives
in the read-only bundle and has to be copied into the sandbox before CPython can open it, and that
copy is accepted only when it hashes to this value (extension/src/python_stdlib.cpp, step 3c).
"""
from __future__ import annotations
import hashlib
import sys
import zipfile
from pathlib import Path
@@ -51,7 +54,9 @@ def main() -> int:
info.external_attr = 0o644 << 16
zf.writestr(info, path.read_bytes())
tmp.replace(out)
print(f"make_stdlib_zip: {out} ({len(members)} modules, {out.stat().st_size} bytes)")
digest = hashlib.sha256(out.read_bytes()).hexdigest()
out.with_name(out.name + ".sha256").write_text(digest + "\n")
print(f"make_stdlib_zip: {out} ({len(members)} modules, {out.stat().st_size} bytes, sha256 {digest})")
return 0