port(2P step 3a/3b): stdlib on sys.path + the pack module

step 3a — 标准库:
  tools/py_embed/make_stdlib_zip.py packs the vendored Lib/ into
  <build>/python27.zip (397 modules, 6876547 bytes, ZIP_STORED — the zlib
  module is trimmed out of our CPython, so zipimport cannot inflate).
  The mtpython_stdlib target builds it; py_embed_test and the launcher test
  both run against it.
  platform/ScriptLib/PythonHost sets the embedding flags before
  Py_Initialize and, after it, puts the zip on sys.path and re-registers
  encodings.search_function with codecs — 2.7 builds the codec registry
  once, at startup, so replacing sys.path afterwards needs it back.

step 3b — pack 模块:
  UserInterface/PythonPackModule.cpp copied verbatim; packExist/packGet/
  initpack read root/ and uiscript/ through CEterPackManager, i.e. through
  asset_io. The CHINA_CRYPT_KEY block in that unit is CAccountConnector's
  and is kept #if 0 until the login slice (2V1). UserInterface/StdAfx.h is
  a documented partial mirror — the original is the whole client's PCH.

The launcher test now runs initpack() as RunMainScript does (its first
line), reads system.py / uiscript / locale text back out of the packs, and
confirms cp1252 decoding works off the zip. system.py still stops at
"No module named app" — the C++ modules are the 2V0 slice.

gates: port_map.py check 0 errors · key leak check 8/8 none · macOS ctest
27/27 · android + ios port_platform compile clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
shenlei
2026-09-23 09:16:37 +09:00
co-authored by Claude Opus 5
parent 43bb561551
commit 2b873f190f
13 changed files with 414 additions and 17 deletions
+1
View File
@@ -494,3 +494,4 @@
{"time":"2026-09-23T13:30:00Z","event":"asset_gate","unit":"PORT-PLAN §5 steps 3-4: MT_ASSETS_STRICT + provenance manifest + script/run_40250_asset_gate.sh","manifest":"audit/assets/40250-client.json (pack/Index + 214 .eix/.epk + 8 Eternexus/root/*.msm = 223 files, 1407474045 bytes, sha256)","env":{"MT_40250_CLIENT":"40250 Client dir (native tests, AssetRoot.client_path, gate)","MT_ASSETS":"m2dev loose dev override only","M2_ASSETS":"no longer used by tests"},"fixes":["remote_player_test accepted only twohand_sword/wait.msa; WAIT picks wait/wait_1 by weight, so it failed ~1/3 of runs"],"tests":["script/run_40250_asset_gate.sh macOS PASS (manifest, ctest port.eterpack|proto.*, godot netbridge/equip_model/remote_player/mob_view)","gate with MT_40250_CLIENT=/nonexistent exits 1"],"commit":"2D-gate"}
{"time": "2026-09-23T15:30:00Z", "event": "third_party", "unit": "PORT-PLAN 批次 2P step 1: vendored CPython 2.7.18 as the mtpython static library", "source": "official Python-2.7.18.tgz sha256 b62c0e7937551d0cc02b8fd5cb0f544f9405bafc9a54d3808ed4594812edef43, trimmed to 25MB (Doc/Demo/Tools/Mac/PCbuild, Modules/{_ctypes,expat,zlib,_sqlite}, stdlib test trees, pre-VS2010 PC projects); Lib/ kept for step 3's python27.zip", "shared": "source list = the 133 objects of the reference libpython2.7.a; built-in table config/config.c = 39 entries (config/Setup.static)", "per_platform": "config/<plat>/pyconfig.h from tools/py_embed/gen_pyconfig.sh; android undefs HAVE_LANGINFO_H (nl_langinfo needs API 26); ios is derived from macos (2.7 configure refuses a darwin cross build) and drops process control + sys/random.h, keeping HAVE_GETGROUPS because posixmodule.c defines MAX_GROUPS only inside it", "patches": ["configure + configure.ac: arm64 MACOSX_DEFAULT_ARCH", "Modules/posixmodule.c: undef HAVE_SYSTEM/POPEN/FORK/EXECV/WAIT under TARGET_OS_IPHONE, which its own 'all other compilers' block hard-defines past pyconfig.h"], "not_done": ["Linux x86_64: gen_pyconfig.sh linux needs a Linux host", "Windows x64: upstream supports MSVC + PC/pyconfig.h while the Windows gate is MinGW, where PC/pyconfig.h leaves HAVE_UNISTD_H unset and posixmodule.c/dynload_win.c collide with the MinGW headers; decided in step 4"], "ported": [], "deleted": [], "divergent": [], "needs_live": [], "status_note": "no port-map entry: this is third-party source, not a ported 40250 unit. Linking into libmtgodot is step 2 (ScriptLib/PythonLauncher.cpp)", "tests": ["ctest python.embed (macOS arm64): 39 builtin modules, all import, sys.version 2.7.18, utf-8/cp1252 codecs + pickle round-trip via the vendored Lib/ PASS", "android arm64 NDK API 24: libmtpython.a builds clean", "ios arm64 iphoneos: libmtpython.a = 133 objects, no undefined _system/_fork/_popen"], "commit": "2P-step1"}
{"time": "2026-09-23T17:30:00Z", "event": "port_round", "unit": "PORT-PLAN 批次 2P step 2: ScriptLib/PythonLauncher.cpp into port_logic on the embedded interpreter", "ported": ["ScriptLib/PythonLauncher.cpp (10 of 12 fns)", "ScriptLib/PythonMarshal.cpp (4 of 11)"], "adapted": [], "deleted": [], "divergent": [], "needs_live": [], "status_note": "statuses set only for what port.python_launcher actually runs. PythonUtils.cpp is copied and compiled but nothing calls it yet (the binding modules are 2V0), so all 19 stay TODO; TraceFunc/SetTraceFunc are _DEBUG-only; in PythonMarshal only the small-.pyc path runs (r_long, _PyMarshal_ReadLongFromFile, getfilesize, _PyMarshal_ReadLastObjectFromFile) because that one delegates to CPython's own PyMarshal_ReadObjectFromString under 16KB, leaving the ported r_object reader unreached. Two // PORT: edits: `unsigned char(val)` -> a cast (MSVC extension) in PythonUtils.cpp x2, and PyObject_AsCharBuffer's Py_ssize_t* in TraceFunc. CPythonLauncher::GetError dereferences the NULL PyErr_Fetch yields when nothing is pending — kept verbatim, the test only calls it with an error set.", "shims": "extension/src/port/common/shim/sdk/Python-2.7/: 12 forwarding headers for 40250's #include <Python-2.7/*>; symtable.h forward-declares mod_ty instead of including Python-ast.h, whose one-word macros (Delete, Module, Set, Print) collide with EterLib/Pool.h — 40250 sidestepped the same clash by commenting out PySymtable_Build", "tests": ["ctest port.python_launcher (macOS arm64): Create/RunLine/RunMemoryTextFile(CRLF)/RunCompiledFile(.pyc)/RunFile PASS; RunFile(\"system.py\") reads the real pack and stops at ImportError: No module named app (the 2V0 C++ modules)", "ctest macOS: 27/27 PASS", "android arm64 NDK API 24 + ios arm64 iphoneos: port_logic (incl. ScriptLib) compiles"], "commit": "2P-step2"}
{"time": "2026-09-23T19:30:00Z", "event": "port_round", "unit": "PORT-PLAN 批次 2P step 3a/3b: python27.zip on sys.path (platform/ScriptLib/PythonHost) + the pack module", "ported": ["UserInterface/PythonPackModule.cpp (3 of 5 fns: packExist, packGet, initpack)"], "adapted": [], "deleted": [], "divergent": [], "needs_live": [], "stdlib": {"builder": "tools/py_embed/make_stdlib_zip.py <Lib> <out.zip>, CMake target mtpython_stdlib -> <build>/python27.zip", "contents": "397 modules, 6876547 bytes, ZIP_STORED (no zlib module, so zipimport cannot inflate); deterministic (sorted, fixed timestamp) so the sha256 identifies it", "sha256": "5fce8f974a89b66110823d7529be96e359b56dea05fc6875189a241d2d37a7d6", "skipped": "test/tests trees, lib-tk, idlelib, lib2to3, bsddb, ensurepip, curses, distutils, unittest, wsgiref, multiprocessing, compiler, plat-*", "host": "platform/ScriptLib/PythonHost.cpp: Configure() sets Py_NoSiteFlag/IgnoreEnvironment/Frozen/DontWriteBytecode + program name before Py_Initialize; InstallStdLib() does PySys_SetPath then re-registers encodings.search_function, because 2.7 initializes the codec registry once at startup and never retries"}, "status_note": "PythonPackModule's other two functions are CAccountConnector's login key builder, which happens to live in that unit; kept verbatim under #if 0 until the 2V1 login slice brings AccountConnector.h. port/UserInterface/StdAfx.h is a partial mirror: the original includes every library's StdAfx plus <dshow.h>/<qedit.h>, so it carries only what compiles today (ScriptLib/StdAfx.h, APP_NAME, the name length enum, initpack) and grows with the 2V slices.", "tests": ["ctest python.embed now runs against python27.zip itself (zipimport), not the loose Lib/", "ctest port.python_launcher: InstallStdLib + `import os, string, types, codecs` + cp1252 encode; initpack then pack.Exist/pack.Get for system.py (byte count equals CEterPackManager's), uiscript/selectcharacterwindow.py and locale/en/locale_game.txt; .tga and a missing file give None", "ctest macOS: 27/27 PASS", "android arm64 + ios arm64: port_platform (incl. PythonHost, PythonPackModule) compiles"], "not_done": ["3c: stage python27.zip out of the app resources into the sandbox on Android/iOS and check it against the manifest sha256; PythonHost::DefaultStdLibPath() only reads MT_PYTHON_STDLIB today"], "commit": "2P-step3ab"}
@@ -0,0 +1,35 @@
{
"reference": "UserInterface/PythonPackModule.cpp",
"reference_sha256": "2c1492f167bae752c61c63a3a8eabd0bb7e8b08a3de94f138ce1e23bfc006c9f",
"priority": "P4",
"contracts": [],
"functions": {
"GetKey_20050304Myevan": {
"status": "TODO"
},
"CAccountConnector::__BuildClientKey_20050304Myevan": {
"status": "TODO"
},
"packExist": {
"impl": [
"extension/src/port/UserInterface/PythonPackModule.cpp:packExist"
],
"status": "PORTED",
"test": "extension/tests/port_python_launcher_test.cpp (port.python_launcher)"
},
"packGet": {
"impl": [
"extension/src/port/UserInterface/PythonPackModule.cpp:packGet"
],
"status": "PORTED",
"test": "extension/tests/port_python_launcher_test.cpp (port.python_launcher)"
},
"initpack": {
"impl": [
"extension/src/port/UserInterface/PythonPackModule.cpp:initpack"
],
"status": "PORTED",
"test": "extension/tests/port_python_launcher_test.cpp (port.python_launcher)"
}
}
}
+16 -1
View File
@@ -82,7 +82,7 @@ extension/third_party/cpython-2.7.18/ # 静态库(2P 批次
| — | 定下目录结构,评估内嵌 Python;Android 独立程序验证 | 完成(9d0e50de、b02c49bb |
| **2A** | 基础:Win32 类型层、参考公共头的最小闭包、`port_logic` CMake 目标、platform 接口骨架、头文件可编译门禁、port-map 重新基线 | **进行中**(剩 Python 库闭包头,等 2P |
| 2R | 资源包能力盘点:EPK 类型/密钥/覆盖顺序/路径大小写/移动端交付 | 完成:只有 NONE/COMPRESS/SECURITY,不需要服务器密钥;设备上直接读 EPK |
| **2P** | CPython 2.7.18 编进 libmtgodot,五个平台分别配置和验证 | **进行中**step 1(静态库 + macOS/Android/iOS 配置)、step 2ScriptLib 进 `port_logic`)完成 |
| **2P** | CPython 2.7.18 编进 libmtgodot,五个平台分别配置和验证 | **进行中**step 1(静态库 + macOS/Android/iOS 配置)、step 2ScriptLib 进 `port_logic`、step 3a/3bpython27.zip + `PythonHost` + `pack` 模块)完成 |
| 2D | 数据源切到 40250msm 路径、proto、资源根、严格资源测试) | 完成(5a26e93f、232d0461、240d2827、a9b31dec);资源根只切了 proto/pack 后端,其余按第 5 节由 2V 切片逐个切换 |
| **2V0** | UI 壳:真实 `PythonLauncher/wndMgr/grp/app` + platform UI,其他玩法模块用桩,显示并操作 Logo/Popup | 需要 2A、2P 和 root/uiscript 子集 |
| **2V1** | 登录/选角:真实 `net` phase、登录/选角脚本及所需模块,走到 Loading | 2V0 之后 |
@@ -299,6 +299,21 @@ extension/third_party/cpython-2.7.18/ # 静态库(2P 批次
- Android/iOS 首次启动从应用资源复制到应用沙盒,校验提交在资源清单中的 sha256,再加入 `sys.path`
- `zipimport` 静态编进解释器。`system.py` 的导入钩子只负责 pack 内的简单 root 模块,不能用来加载标准库 package、
dotted import 或 `encodings`
**已完成(3a 标准库 + 3b pack 模块)**`tools/py_embed/make_stdlib_zip.py` 把 vendored `Lib/` 打成
`<build>/python27.zip`397 个模块 / 6.9MB,全部 STORED——没有 zlib 模块,zipimport 解不了 deflate
内容确定性写入,sha256 就是 3c 的校验值),CMake 目标 `mtpython_stdlib` 跟着 `Lib/*.py` 重建。
宿主引导收进 `extension/src/platform/ScriptLib/PythonHost.{h,cpp}``Configure()` 设四个标志和程序名,
`InstallStdLib()``Py_Initialize` 之后换掉 `sys.path` 并重新 `codecs.register(encodings.search_function)`
(2.7 的 codec 注册表只在启动时初始化一次)。`python.embed` 现在直接跑 zip。
`UserInterface/PythonPackModule.cpp` 已移植(`packExist`/`packGet`/`initpack`),经 `CEterPackManager`
读 pack,即 `asset_io``pack://` 用的同一套;`port.python_launcher` 验证 `pack.Get` 取到的
`system.py``CEterPackManager` 的字节数一致,并能读 `uiscript/selectcharacterwindow.py`
`locale/en/locale_game.txt`,非 `.py/.pyc/.txt` 和不存在的文件返回 `None`
`UserInterface/StdAfx.h` 只镜像了当前能编的部分(原文件 include 整个客户端和 DirectShow),
`PythonPackModule.cpp` 里属于 `CAccountConnector` 的登录密钥函数原样保留但编译时排除,随 2V1 回来。
**未完成(3c**Android/iOS 从应用资源把 zip 复制进沙盒并校验 sha256;`PythonHost::DefaultStdLibPath()`
现在只认 `MT_PYTHON_STDLIB`
4. 五个平台分别验证:编译、链接无未定义符号、`Py_Initialize`、静态 C 模块逐个 `import`、在 app 进程里跑 `system.py``prototype.RunApp()`(对照结果:74 个模块中 66 个加载成功,引导期调用 33 个 C++ 函数)。
| 平台 | 状态 |
+5 -2
View File
@@ -100,8 +100,11 @@ one via a 1-line shim header, see below).
`Python-2.7.18.tgz` (sha256 `b62c0e7937551d0cc02b8fd5cb0f544f9405bafc9a54d3808ed4594812edef43`)
trimmed of what an embedded interpreter never builds or ships: `Doc Demo Tools
Mac PCbuild`, `Modules/{_ctypes,expat,zlib,_sqlite}`, the stdlib test trees and
the pre-VS2010 `PC/` project files. `Lib/` stays — batch 2P step 3 ships it as
`python27.zip`.
the pre-VS2010 `PC/` project files. `Lib/` stays: the `mtpython_stdlib` target
packs it into `<build>/python27.zip` with `tools/py_embed/make_stdlib_zip.py`,
which is what the host puts on `sys.path` (batch 2P step 3). Entries are
**stored, never deflated** — the `zlib` module is trimmed out above, so
`zipimport` has nothing to inflate with.
* **Why vendored:** 2.7 is end-of-life, so there is no package to fetch on the
four target SDKs, and the build must keep working offline and on the
cross-build machines. It is the version the 40250 scripts are written for
+3 -4
View File
@@ -159,10 +159,9 @@ if(BUILD_TESTING AND MT_HOST_BUILD)
if(TARGET mtpython)
add_executable(py_embed_test tests/py_embed_test.cpp)
target_link_libraries(py_embed_test PRIVATE mtpython)
# The stdlib the host puts on sys.path: here the vendored Lib/ directly (batch 2P step 3 ships
# it as python27.zip).
add_test(NAME python.embed COMMAND $<TARGET_FILE:py_embed_test>
"${CMAKE_CURRENT_SOURCE_DIR}/third_party/cpython-2.7.18/Lib")
add_dependencies(py_embed_test mtpython_stdlib)
# The stdlib exactly as the host ships it: python27.zip, read through zipimport (批次 2P step 3).
add_test(NAME python.embed COMMAND $<TARGET_FILE:py_embed_test> "${MT_PYTHON_STDLIB_ZIP}")
endif()
endif()
@@ -0,0 +1,106 @@
// See PythonHost.h: the interpreter setup 40250 got from a real python27 installation.
#include "ScriptLib/StdAfx.h"
#include "PythonHost.h"
#include <cstdio>
#include <cstdlib>
namespace
{
// The pending exception as text, so a failed bootstrap says what went wrong instead of leaving the
// error for the first unrelated script to trip over.
std::string take_error()
{
if (!PyErr_Occurred())
return "no exception";
PyObject *type = NULL, *value = NULL, *tb = NULL;
PyErr_Fetch(&type, &value, &tb);
PyErr_NormalizeException(&type, &value, &tb);
std::string text;
if (PyObject* str = value ? PyObject_Str(value) : NULL)
{
if (PyString_Check(str))
text = PyString_AS_STRING(str);
Py_DECREF(str);
}
if (type && text.empty())
text = ((PyTypeObject*) type)->tp_name;
Py_XDECREF(type);
Py_XDECREF(value);
Py_XDECREF(tb);
return text.empty() ? "unknown error" : text;
}
bool fail(std::string* error, const std::string& text)
{
if (error)
*error = text;
return false;
}
}
namespace PythonHost
{
void Configure(const char* program_name)
{
// site.py lives in a prefix this interpreter does not have, the environment must not be able to
// redirect a game's script layer (PYTHONPATH/PYTHONHOME), the "could not find platform libraries"
// probe has nothing to find, and the pack directories are read-only on the mobile targets.
Py_NoSiteFlag = 1;
Py_IgnoreEnvironmentFlag = 1;
Py_FrozenFlag = 1;
Py_DontWriteBytecodeFlag = 1;
Py_SetProgramName((char*) program_name);
}
bool InstallStdLib(const char* stdlib_path, std::string* error)
{
if (!Py_IsInitialized())
return fail(error, "interpreter is not initialized");
if (!stdlib_path || !*stdlib_path)
return fail(error, "no stdlib path (set MT_PYTHON_STDLIB or pass python27.zip)");
// zipimport and the source importer both go through the C library, so the path has to be a real
// file: a pack entry or an Android asset must be staged out first (step 3c).
if (FILE* fp = fopen(stdlib_path, "rb"))
fclose(fp);
else if (FILE* dir = fopen((std::string(stdlib_path) + "/os.py").c_str(), "rb"))
fclose(dir);
else
return fail(error, std::string("cannot open ") + stdlib_path);
PySys_SetPath((char*) stdlib_path);
// Py_Initialize ran _PyCodecRegistry_Init against the then-empty path and 2.7 never retries it
// ("no codec search functions registered" for the rest of the process), so the search function is
// registered again now that encodings is importable.
PyObject* encodings = PyImport_ImportModule("encodings");
if (!encodings)
return fail(error, "import encodings: " + take_error());
PyObject* search = PyObject_GetAttrString(encodings, (char*) "search_function");
Py_DECREF(encodings);
if (!search)
return fail(error, "encodings.search_function: " + take_error());
PyObject* codecs = PyImport_ImportModule("codecs");
PyObject* registered = codecs ? PyObject_CallMethod(codecs, (char*) "register", (char*) "O", search) : NULL;
Py_DECREF(search);
Py_XDECREF(codecs);
if (!registered)
return fail(error, "codecs.register: " + take_error());
Py_DECREF(registered);
return true;
}
std::string DefaultStdLibPath()
{
if (const char* env = getenv("MT_PYTHON_STDLIB"))
return env;
// Step 3c stages python27.zip out of the packed assets on Android/iOS and answers here; until then
// the caller passes the path (the build tree's copy, for the native tests).
return "";
}
}
@@ -0,0 +1,27 @@
#pragma once
// The interpreter setup 40250 got for free from a real python27 installation.
//
// 40250's Main() only constructs CPythonLauncher: python27.dll found its own prefix, so `import site`,
// the codec registry and every stdlib import worked before a single script ran. The embedded
// interpreter has no prefix and no filesystem to probe, so the host does that part itself —
// the flags before Py_Initialize, and the stdlib (python27.zip) on sys.path afterwards.
// See docs/PORT-PLAN.md 批次 2P step 3.
#include <string>
namespace PythonHost
{
// Before the interpreter starts, i.e. before CPythonLauncher's constructor.
// `program_name` is what CPythonLauncher::Create defaults to; it only names the interpreter.
void Configure(const char* program_name = "eter.python");
// After Py_Initialize: replaces sys.path with `stdlib_path` (a python27.zip or a stdlib directory
// on a real filesystem — zipimport and the .py importer both need a path the C library can open)
// and re-registers the codec search function. False leaves *error set.
bool InstallStdLib(const char* stdlib_path, std::string* error);
// The stdlib path this build was given: $MT_PYTHON_STDLIB when set, else the python27.zip next to
// the executable, else "". Mobile staging (copy out of the app resources, checked against the
// manifest sha256) is 批次 2P step 3c and will answer here too.
std::string DefaultStdLibPath();
}
+2 -1
View File
@@ -164,8 +164,9 @@ if(BUILD_TESTING AND CMAKE_SYSTEM_NAME STREQUAL CMAKE_HOST_SYSTEM_NAME)
if(TARGET mtpython)
add_executable(port_python_launcher_test ${CMAKE_CURRENT_SOURCE_DIR}/../../tests/port_python_launcher_test.cpp)
target_link_libraries(port_python_launcher_test PRIVATE port_platform)
add_dependencies(port_python_launcher_test mtpython_stdlib)
add_test(NAME port.python_launcher
COMMAND $<TARGET_FILE:port_python_launcher_test> ${MT_40250_CLIENT})
COMMAND $<TARGET_FILE:port_python_launcher_test> ${MT_40250_CLIENT} ${MT_PYTHON_STDLIB_ZIP})
set_tests_properties(port.python_launcher PROPERTIES SKIP_RETURN_CODE 77)
endif()
endif()
@@ -0,0 +1,92 @@
#include "StdAfx.h"
#include "../EterPack/EterPackManager.h"
#include "../EterBase/tea.h"
// CHINA_CRYPT_KEY
DWORD g_adwEncryptKey[4];
DWORD g_adwDecryptKey[4];
// PORT: the login cipher's key builder happens to live in this unit, but it is CAccountConnector's —
// AccountConnector.h and the class come with the login slice (docs/PORT-PLAN.md 批次 2V1). Kept verbatim
// and compiled out until then; the two keys above stay, they are what the network stream reads.
#if 0
#include "AccountConnector.h"
inline const BYTE* GetKey_20050304Myevan()
{
volatile static DWORD s_adwKey[1938];
volatile DWORD seed=1491971513;
for (UINT i=0; i<BYTE(seed); i++)
{
seed^=2148941891;
seed+=3592385981;
s_adwKey[i]=seed;
}
return (const BYTE*)s_adwKey;
}
//#include <eterCrypt.h>
void CAccountConnector::__BuildClientKey_20050304Myevan()
{
const BYTE * c_pszKey = GetKey_20050304Myevan();
memcpy(g_adwEncryptKey, c_pszKey+157, 16);
for (DWORD i = 0; i < 4; ++i)
g_adwEncryptKey[i] = random();
tea_encrypt((DWORD *) g_adwDecryptKey, (const DWORD *) g_adwEncryptKey, (const DWORD *) (c_pszKey+37), 16);
// TEA_Encrypt((DWORD *) g_adwDecryptKey, (const DWORD *) g_adwEncryptKey, (const DWORD *) (c_pszKey+37), 16);
}
#endif
// END_OF_CHINA_CRYPT_KEY
PyObject * packExist(PyObject * poSelf, PyObject * poArgs)
{
char * strFileName;
if (!PyTuple_GetString(poArgs, 0, &strFileName))
return Py_BuildException();
return Py_BuildValue("i", CEterPackManager::Instance().isExist(strFileName)?1:0);
}
PyObject * packGet(PyObject * poSelf, PyObject * poArgs)
{
char * strFileName;
if (!PyTuple_GetString(poArgs, 0, &strFileName))
return Py_BuildException();
// 파이썬에서 읽어드리는 패킹 파일은 python 파일과 txt 파일에 한정한다
const char* pcExt = strrchr(strFileName, '.');
if (pcExt) // 확장자가 있고
{
if ((stricmp(pcExt, ".py") == 0) ||
(stricmp(pcExt, ".pyc") == 0) ||
(stricmp(pcExt, ".txt") == 0))
{
CMappedFile file;
const void * pData = NULL;
if (CEterPackManager::Instance().Get(file,strFileName,&pData))
return Py_BuildValue("s#",pData, file.Size());
}
}
return Py_BuildException();
}
void initpack()
{
static PyMethodDef s_methods[] =
{
{ "Exist", packExist, METH_VARARGS },
{ "Get", packGet, METH_VARARGS },
{ NULL, NULL },
};
Py_InitModule("pack", s_methods);
}
+27
View File
@@ -0,0 +1,27 @@
#pragma once
// PORT: a partial mirror of 40250 UserInterface/StdAfx.h.
//
// The original is the precompiled header of the whole client: it includes eterLib, eterPythonLib,
// gameLib, scriptLib, milesLib, EffectLib, PRTerrainLib and SpeedTreeLib's StdAfx.h, plus <dshow.h>
// and <qedit.h>, Locale.h and GameType.h. Only scriptLib exists in the port tree today, so this file
// carries the part the ported units actually use and grows with the 2V slices that bring the rest
// (docs/PORT-PLAN.md 批次 2V0-2V3).
#include "../ScriptLib/StdAfx.h"
#define APP_NAME "Metin 2"
enum
{
POINT_MAX_NUM = 255,
CHARACTER_NAME_MAX_LEN = 24,
#if defined(LOCALE_SERVICE_JAPAN)
PLAYER_NAME_MAX_LEN = 16,
#else
PLAYER_NAME_MAX_LEN = 12,
#endif
};
// The Python module initializers RunMainScript calls, each defined in its own Python*Module.cpp.
// Listed here as 40250 lists them; only the ported ones are defined.
void initpack();
+28 -9
View File
@@ -14,6 +14,8 @@
#include "ScriptLib/StdAfx.h"
#include "ScriptLib/PythonLauncher.h"
#include "EterPack/EterPackManager.h"
#include "UserInterface/StdAfx.h" // initpack
#include "../src/platform/ScriptLib/PythonHost.h"
#include "../src/platform/UserInterface/UserInterface.h"
#include <algorithm>
@@ -58,6 +60,7 @@ int main(int argc, char** argv)
const char* strict = std::getenv("MT_ASSETS_STRICT");
const bool is_strict = strict && std::string(strict) == "1";
const std::string client = argc > 1 ? argv[1] : "";
const std::string stdlib = argc > 2 ? argv[2] : PythonHost::DefaultStdLibPath();
if (client.empty() || chdir(client.c_str()) != 0 || access("pack/Index", 0) != 0)
{
std::fprintf(stderr, "port_python_launcher_test: no 40250 Client/pack at '%s'\n", client.c_str());
@@ -67,21 +70,36 @@ int main(int argc, char** argv)
PackSingletons();
CHECK(PackInitialize("pack"));
// Host policy, set before the launcher's ctor runs Py_Initialize: 40250 shipped a Python with a real
// prefix on disk, the embedded one has none, so `import site` would fail at startup. The flags are the
// ones tools/py_embed_android/main.c uses; step 3 moves them, and the stdlib on sys.path, into the
// platform layer that every host shares.
Py_NoSiteFlag = 1;
Py_IgnoreEnvironmentFlag = 1;
Py_FrozenFlag = 1;
Py_DontWriteBytecodeFlag = 1;
Py_SetProgramName((char*) "eter.python"); // CPythonLauncher::Create's default program name
// What a real python27 installation did for 40250, the host does here: flags before Py_Initialize,
// stdlib on sys.path after it (platform/ScriptLib/PythonHost.cpp).
PythonHost::Configure();
{
CPythonLauncher launcher; // ctor runs Py_Initialize, as 40250 Main() does on the stack
CHECK(Py_IsInitialized());
CHECK(launcher.Create());
std::string stdlib_error;
CHECK(PythonHost::InstallStdLib(stdlib.c_str(), &stdlib_error));
if (!stdlib_error.empty())
std::fprintf(stderr, "InstallStdLib: %s\n", stdlib_error.c_str());
// The stdlib is what the script layer's imports and the CP949/cp1252 text in the packs need.
CHECK(launcher.RunLine("import os, string, types, codecs\nassert u'\\xe4'.encode('cp1252') == '\\xe4'"));
// RunMainScript's first line (UserInterface.cpp:243): the pack module, which is how system.py's
// import hook and the window system read root/ and uiscript/ out of the registered packs.
initpack();
CHECK(launcher.RunLine("import pack"));
CHECK(launcher.RunLine("assert pack.Exist('system.py') == 1"));
CHECK(launcher.RunLine("assert pack.Exist('no/such.py') == 0"));
CHECK(launcher.RunLine("__pack_len__ = len(pack.Get('system.py'))"));
CHECK(launcher.RunLine("assert len(pack.Get('uiscript/selectcharacterwindow.py')) > 0"));
CHECK(launcher.RunLine("assert len(pack.Get('locale/en/locale_game.txt')) > 0"));
// Only .py/.pyc/.txt come out of it, and a miss is None rather than an exception
// (packGet returns Py_BuildException(), which clears the error and builds None).
CHECK(launcher.RunLine("assert pack.Get('icon/item/00000.tga') is None"));
CHECK(launcher.RunLine("assert pack.Get('no/such.py') is None"));
// RegisterDebugFlag / RegisterCommandLine, verbatim from RunMainScript.
CHECK(launcher.RunLine("__DEBUG__ = 0"));
CHECK(launcher.RunLine("__COMMAND_LINE__ = \"\""));
@@ -141,6 +159,7 @@ int main(int argc, char** argv)
const VOID* data = NULL;
CHECK(CEterPackManager::Instance().Get(file, "system.py", &data));
CHECK(file.Size() > 0);
CHECK(main_long("__pack_len__") == (long) file.Size()); // pack.Get returned the same entry
// Where it stops, spelled out: RunFile reports through Traceback(), which fetches the
// exception and so leaves nothing behind, and LogBoxf is a platform stub. Running the same
+13
View File
@@ -209,3 +209,16 @@ endif()
find_package(Threads REQUIRED)
target_link_libraries(mtpython PUBLIC Threads::Threads m)
add_library(mt3p::python ALIAS mtpython)
# --- the stdlib the host puts on sys.path (批次 2P step 3) ---
# Built from the same vendored Lib/, so the interpreter and its library always match. MT_PYTHON_STDLIB_ZIP
# is where every consumer (the native tests now, the mobile staging in step 3c) picks it up.
find_package(Python3 COMPONENTS Interpreter REQUIRED)
set(MT_PYTHON_STDLIB_ZIP "${CMAKE_BINARY_DIR}/python27.zip" CACHE INTERNAL "stdlib zip for the embedded interpreter")
file(GLOB_RECURSE MT_PYTHON_STDLIB_SOURCES CONFIGURE_DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/Lib/*.py")
add_custom_command(OUTPUT "${MT_PYTHON_STDLIB_ZIP}"
COMMAND ${Python3_EXECUTABLE} "${CMAKE_SOURCE_DIR}/tools/py_embed/make_stdlib_zip.py"
"${CMAKE_CURRENT_SOURCE_DIR}/Lib" "${MT_PYTHON_STDLIB_ZIP}"
DEPENDS "${CMAKE_SOURCE_DIR}/tools/py_embed/make_stdlib_zip.py" ${MT_PYTHON_STDLIB_SOURCES}
COMMENT "Packing python27.zip from the vendored Lib/")
add_custom_target(mtpython_stdlib ALL DEPENDS "${MT_PYTHON_STDLIB_ZIP}")
+59
View File
@@ -0,0 +1,59 @@
#!/usr/bin/env python3
"""Pack the vendored CPython 2.7.18 Lib/ into the python27.zip the host puts on sys.path.
make_stdlib_zip.py <Lib dir> <out.zip>
40250 shipped a real python27 installation, so `import encodings` found Lib/ through the
interpreter's prefix. The embedded interpreter has no prefix (Py_FrozenFlag, no site), so the host
supplies the stdlib as this one file — the arrangement validated on the device by
tools/py_embed_android/build-and-run.sh (PORT-PLAN 批次 2P step 3).
Entries are STORED, never deflated: the zlib module is not in the vendored Modules/ (see
docs/THIRD-PARTY.md), so zipimport could not inflate them. Only .py goes in; zipimport reads
source as happily as bytecode, and .pyc would pin the magic number to the build host's interpreter.
Written deterministically (sorted, fixed timestamp) so the file's sha256 identifies its contents —
3c checks that hash when staging the zip into a mobile sandbox.
"""
from __future__ import annotations
import sys
import zipfile
from pathlib import Path
# Test suites, the Tk/2to3/bsddb/pip trees and the platform-specific stubs: none of the 40250 scripts
# or their imports reach them, and they are most of the size.
SKIP_DIRS = {"test", "tests", "lib-tk", "idlelib", "lib2to3", "bsddb", "ensurepip", "curses",
"distutils", "unittest", "wsgiref", "multiprocessing", "compiler"}
def main() -> int:
if len(sys.argv) != 3:
print(__doc__, file=sys.stderr)
return 2
lib, out = Path(sys.argv[1]), Path(sys.argv[2])
if not (lib / "os.py").is_file():
print(f"make_stdlib_zip: {lib} is not a CPython Lib/ directory", file=sys.stderr)
return 1
members = []
for path in sorted(lib.rglob("*.py")):
rel = path.relative_to(lib)
if rel.parts[0].startswith("plat-") or SKIP_DIRS.intersection(rel.parts[:-1]):
continue
members.append((rel, path))
out.parent.mkdir(parents=True, exist_ok=True)
tmp = out.with_suffix(".zip.tmp")
with zipfile.ZipFile(tmp, "w", zipfile.ZIP_STORED) as zf:
for rel, path in members:
info = zipfile.ZipInfo(rel.as_posix(), date_time=(1980, 1, 1, 0, 0, 0))
info.external_attr = 0o644 << 16
zf.writestr(info, path.read_bytes())
tmp.replace(out)
print(f"make_stdlib_zip: {out} ({len(members)} modules, {out.stat().st_size} bytes)")
return 0
if __name__ == "__main__":
raise SystemExit(main())