port(2P step 3a/3b): stdlib on sys.path + the pack module
step 3a — 标准库: tools/py_embed/make_stdlib_zip.py packs the vendored Lib/ into <build>/python27.zip (397 modules, 6876547 bytes, ZIP_STORED — the zlib module is trimmed out of our CPython, so zipimport cannot inflate). The mtpython_stdlib target builds it; py_embed_test and the launcher test both run against it. platform/ScriptLib/PythonHost sets the embedding flags before Py_Initialize and, after it, puts the zip on sys.path and re-registers encodings.search_function with codecs — 2.7 builds the codec registry once, at startup, so replacing sys.path afterwards needs it back. step 3b — pack 模块: UserInterface/PythonPackModule.cpp copied verbatim; packExist/packGet/ initpack read root/ and uiscript/ through CEterPackManager, i.e. through asset_io. The CHINA_CRYPT_KEY block in that unit is CAccountConnector's and is kept #if 0 until the login slice (2V1). UserInterface/StdAfx.h is a documented partial mirror — the original is the whole client's PCH. The launcher test now runs initpack() as RunMainScript does (its first line), reads system.py / uiscript / locale text back out of the packs, and confirms cp1252 decoding works off the zip. system.py still stops at "No module named app" — the C++ modules are the 2V0 slice. gates: port_map.py check 0 errors · key leak check 8/8 none · macOS ctest 27/27 · android + ios port_platform compile clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -494,3 +494,4 @@
|
||||
{"time":"2026-09-23T13:30:00Z","event":"asset_gate","unit":"PORT-PLAN §5 steps 3-4: MT_ASSETS_STRICT + provenance manifest + script/run_40250_asset_gate.sh","manifest":"audit/assets/40250-client.json (pack/Index + 214 .eix/.epk + 8 Eternexus/root/*.msm = 223 files, 1407474045 bytes, sha256)","env":{"MT_40250_CLIENT":"40250 Client dir (native tests, AssetRoot.client_path, gate)","MT_ASSETS":"m2dev loose dev override only","M2_ASSETS":"no longer used by tests"},"fixes":["remote_player_test accepted only twohand_sword/wait.msa; WAIT picks wait/wait_1 by weight, so it failed ~1/3 of runs"],"tests":["script/run_40250_asset_gate.sh macOS PASS (manifest, ctest port.eterpack|proto.*, godot netbridge/equip_model/remote_player/mob_view)","gate with MT_40250_CLIENT=/nonexistent exits 1"],"commit":"2D-gate"}
|
||||
{"time": "2026-09-23T15:30:00Z", "event": "third_party", "unit": "PORT-PLAN 批次 2P step 1: vendored CPython 2.7.18 as the mtpython static library", "source": "official Python-2.7.18.tgz sha256 b62c0e7937551d0cc02b8fd5cb0f544f9405bafc9a54d3808ed4594812edef43, trimmed to 25MB (Doc/Demo/Tools/Mac/PCbuild, Modules/{_ctypes,expat,zlib,_sqlite}, stdlib test trees, pre-VS2010 PC projects); Lib/ kept for step 3's python27.zip", "shared": "source list = the 133 objects of the reference libpython2.7.a; built-in table config/config.c = 39 entries (config/Setup.static)", "per_platform": "config/<plat>/pyconfig.h from tools/py_embed/gen_pyconfig.sh; android undefs HAVE_LANGINFO_H (nl_langinfo needs API 26); ios is derived from macos (2.7 configure refuses a darwin cross build) and drops process control + sys/random.h, keeping HAVE_GETGROUPS because posixmodule.c defines MAX_GROUPS only inside it", "patches": ["configure + configure.ac: arm64 MACOSX_DEFAULT_ARCH", "Modules/posixmodule.c: undef HAVE_SYSTEM/POPEN/FORK/EXECV/WAIT under TARGET_OS_IPHONE, which its own 'all other compilers' block hard-defines past pyconfig.h"], "not_done": ["Linux x86_64: gen_pyconfig.sh linux needs a Linux host", "Windows x64: upstream supports MSVC + PC/pyconfig.h while the Windows gate is MinGW, where PC/pyconfig.h leaves HAVE_UNISTD_H unset and posixmodule.c/dynload_win.c collide with the MinGW headers; decided in step 4"], "ported": [], "deleted": [], "divergent": [], "needs_live": [], "status_note": "no port-map entry: this is third-party source, not a ported 40250 unit. Linking into libmtgodot is step 2 (ScriptLib/PythonLauncher.cpp)", "tests": ["ctest python.embed (macOS arm64): 39 builtin modules, all import, sys.version 2.7.18, utf-8/cp1252 codecs + pickle round-trip via the vendored Lib/ PASS", "android arm64 NDK API 24: libmtpython.a builds clean", "ios arm64 iphoneos: libmtpython.a = 133 objects, no undefined _system/_fork/_popen"], "commit": "2P-step1"}
|
||||
{"time": "2026-09-23T17:30:00Z", "event": "port_round", "unit": "PORT-PLAN 批次 2P step 2: ScriptLib/PythonLauncher.cpp into port_logic on the embedded interpreter", "ported": ["ScriptLib/PythonLauncher.cpp (10 of 12 fns)", "ScriptLib/PythonMarshal.cpp (4 of 11)"], "adapted": [], "deleted": [], "divergent": [], "needs_live": [], "status_note": "statuses set only for what port.python_launcher actually runs. PythonUtils.cpp is copied and compiled but nothing calls it yet (the binding modules are 2V0), so all 19 stay TODO; TraceFunc/SetTraceFunc are _DEBUG-only; in PythonMarshal only the small-.pyc path runs (r_long, _PyMarshal_ReadLongFromFile, getfilesize, _PyMarshal_ReadLastObjectFromFile) because that one delegates to CPython's own PyMarshal_ReadObjectFromString under 16KB, leaving the ported r_object reader unreached. Two // PORT: edits: `unsigned char(val)` -> a cast (MSVC extension) in PythonUtils.cpp x2, and PyObject_AsCharBuffer's Py_ssize_t* in TraceFunc. CPythonLauncher::GetError dereferences the NULL PyErr_Fetch yields when nothing is pending — kept verbatim, the test only calls it with an error set.", "shims": "extension/src/port/common/shim/sdk/Python-2.7/: 12 forwarding headers for 40250's #include <Python-2.7/*>; symtable.h forward-declares mod_ty instead of including Python-ast.h, whose one-word macros (Delete, Module, Set, Print) collide with EterLib/Pool.h — 40250 sidestepped the same clash by commenting out PySymtable_Build", "tests": ["ctest port.python_launcher (macOS arm64): Create/RunLine/RunMemoryTextFile(CRLF)/RunCompiledFile(.pyc)/RunFile PASS; RunFile(\"system.py\") reads the real pack and stops at ImportError: No module named app (the 2V0 C++ modules)", "ctest macOS: 27/27 PASS", "android arm64 NDK API 24 + ios arm64 iphoneos: port_logic (incl. ScriptLib) compiles"], "commit": "2P-step2"}
|
||||
{"time": "2026-09-23T19:30:00Z", "event": "port_round", "unit": "PORT-PLAN 批次 2P step 3a/3b: python27.zip on sys.path (platform/ScriptLib/PythonHost) + the pack module", "ported": ["UserInterface/PythonPackModule.cpp (3 of 5 fns: packExist, packGet, initpack)"], "adapted": [], "deleted": [], "divergent": [], "needs_live": [], "stdlib": {"builder": "tools/py_embed/make_stdlib_zip.py <Lib> <out.zip>, CMake target mtpython_stdlib -> <build>/python27.zip", "contents": "397 modules, 6876547 bytes, ZIP_STORED (no zlib module, so zipimport cannot inflate); deterministic (sorted, fixed timestamp) so the sha256 identifies it", "sha256": "5fce8f974a89b66110823d7529be96e359b56dea05fc6875189a241d2d37a7d6", "skipped": "test/tests trees, lib-tk, idlelib, lib2to3, bsddb, ensurepip, curses, distutils, unittest, wsgiref, multiprocessing, compiler, plat-*", "host": "platform/ScriptLib/PythonHost.cpp: Configure() sets Py_NoSiteFlag/IgnoreEnvironment/Frozen/DontWriteBytecode + program name before Py_Initialize; InstallStdLib() does PySys_SetPath then re-registers encodings.search_function, because 2.7 initializes the codec registry once at startup and never retries"}, "status_note": "PythonPackModule's other two functions are CAccountConnector's login key builder, which happens to live in that unit; kept verbatim under #if 0 until the 2V1 login slice brings AccountConnector.h. port/UserInterface/StdAfx.h is a partial mirror: the original includes every library's StdAfx plus <dshow.h>/<qedit.h>, so it carries only what compiles today (ScriptLib/StdAfx.h, APP_NAME, the name length enum, initpack) and grows with the 2V slices.", "tests": ["ctest python.embed now runs against python27.zip itself (zipimport), not the loose Lib/", "ctest port.python_launcher: InstallStdLib + `import os, string, types, codecs` + cp1252 encode; initpack then pack.Exist/pack.Get for system.py (byte count equals CEterPackManager's), uiscript/selectcharacterwindow.py and locale/en/locale_game.txt; .tga and a missing file give None", "ctest macOS: 27/27 PASS", "android arm64 + ios arm64: port_platform (incl. PythonHost, PythonPackModule) compiles"], "not_done": ["3c: stage python27.zip out of the app resources into the sandbox on Android/iOS and check it against the manifest sha256; PythonHost::DefaultStdLibPath() only reads MT_PYTHON_STDLIB today"], "commit": "2P-step3ab"}
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"reference": "UserInterface/PythonPackModule.cpp",
|
||||
"reference_sha256": "2c1492f167bae752c61c63a3a8eabd0bb7e8b08a3de94f138ce1e23bfc006c9f",
|
||||
"priority": "P4",
|
||||
"contracts": [],
|
||||
"functions": {
|
||||
"GetKey_20050304Myevan": {
|
||||
"status": "TODO"
|
||||
},
|
||||
"CAccountConnector::__BuildClientKey_20050304Myevan": {
|
||||
"status": "TODO"
|
||||
},
|
||||
"packExist": {
|
||||
"impl": [
|
||||
"extension/src/port/UserInterface/PythonPackModule.cpp:packExist"
|
||||
],
|
||||
"status": "PORTED",
|
||||
"test": "extension/tests/port_python_launcher_test.cpp (port.python_launcher)"
|
||||
},
|
||||
"packGet": {
|
||||
"impl": [
|
||||
"extension/src/port/UserInterface/PythonPackModule.cpp:packGet"
|
||||
],
|
||||
"status": "PORTED",
|
||||
"test": "extension/tests/port_python_launcher_test.cpp (port.python_launcher)"
|
||||
},
|
||||
"initpack": {
|
||||
"impl": [
|
||||
"extension/src/port/UserInterface/PythonPackModule.cpp:initpack"
|
||||
],
|
||||
"status": "PORTED",
|
||||
"test": "extension/tests/port_python_launcher_test.cpp (port.python_launcher)"
|
||||
}
|
||||
}
|
||||
}
|
||||
+16
-1
@@ -82,7 +82,7 @@ extension/third_party/cpython-2.7.18/ # 静态库(2P 批次
|
||||
| — | 定下目录结构,评估内嵌 Python;Android 独立程序验证 | 完成(9d0e50de、b02c49bb) |
|
||||
| **2A** | 基础:Win32 类型层、参考公共头的最小闭包、`port_logic` CMake 目标、platform 接口骨架、头文件可编译门禁、port-map 重新基线 | **进行中**(剩 Python 库闭包头,等 2P) |
|
||||
| 2R | 资源包能力盘点:EPK 类型/密钥/覆盖顺序/路径大小写/移动端交付 | 完成:只有 NONE/COMPRESS/SECURITY,不需要服务器密钥;设备上直接读 EPK |
|
||||
| **2P** | CPython 2.7.18 编进 libmtgodot,五个平台分别配置和验证 | **进行中**:step 1(静态库 + macOS/Android/iOS 配置)、step 2(ScriptLib 进 `port_logic`)完成 |
|
||||
| **2P** | CPython 2.7.18 编进 libmtgodot,五个平台分别配置和验证 | **进行中**:step 1(静态库 + macOS/Android/iOS 配置)、step 2(ScriptLib 进 `port_logic`)、step 3a/3b(python27.zip + `PythonHost` + `pack` 模块)完成 |
|
||||
| 2D | 数据源切到 40250(msm 路径、proto、资源根、严格资源测试) | 完成(5a26e93f、232d0461、240d2827、a9b31dec);资源根只切了 proto/pack 后端,其余按第 5 节由 2V 切片逐个切换 |
|
||||
| **2V0** | UI 壳:真实 `PythonLauncher/wndMgr/grp/app` + platform UI,其他玩法模块用桩,显示并操作 Logo/Popup | 需要 2A、2P 和 root/uiscript 子集 |
|
||||
| **2V1** | 登录/选角:真实 `net` phase、登录/选角脚本及所需模块,走到 Loading | 2V0 之后 |
|
||||
@@ -299,6 +299,21 @@ extension/third_party/cpython-2.7.18/ # 静态库(2P 批次
|
||||
- Android/iOS 首次启动从应用资源复制到应用沙盒,校验提交在资源清单中的 sha256,再加入 `sys.path`;
|
||||
- `zipimport` 静态编进解释器。`system.py` 的导入钩子只负责 pack 内的简单 root 模块,不能用来加载标准库 package、
|
||||
dotted import 或 `encodings`。
|
||||
|
||||
**已完成(3a 标准库 + 3b pack 模块)**:`tools/py_embed/make_stdlib_zip.py` 把 vendored `Lib/` 打成
|
||||
`<build>/python27.zip`(397 个模块 / 6.9MB,全部 STORED——没有 zlib 模块,zipimport 解不了 deflate;
|
||||
内容确定性写入,sha256 就是 3c 的校验值),CMake 目标 `mtpython_stdlib` 跟着 `Lib/*.py` 重建。
|
||||
宿主引导收进 `extension/src/platform/ScriptLib/PythonHost.{h,cpp}`:`Configure()` 设四个标志和程序名,
|
||||
`InstallStdLib()` 在 `Py_Initialize` 之后换掉 `sys.path` 并重新 `codecs.register(encodings.search_function)`
|
||||
(2.7 的 codec 注册表只在启动时初始化一次)。`python.embed` 现在直接跑 zip。
|
||||
`UserInterface/PythonPackModule.cpp` 已移植(`packExist`/`packGet`/`initpack`),经 `CEterPackManager`
|
||||
读 pack,即 `asset_io` 的 `pack://` 用的同一套;`port.python_launcher` 验证 `pack.Get` 取到的
|
||||
`system.py` 与 `CEterPackManager` 的字节数一致,并能读 `uiscript/selectcharacterwindow.py`、
|
||||
`locale/en/locale_game.txt`,非 `.py/.pyc/.txt` 和不存在的文件返回 `None`。
|
||||
`UserInterface/StdAfx.h` 只镜像了当前能编的部分(原文件 include 整个客户端和 DirectShow),
|
||||
`PythonPackModule.cpp` 里属于 `CAccountConnector` 的登录密钥函数原样保留但编译时排除,随 2V1 回来。
|
||||
**未完成(3c)**:Android/iOS 从应用资源把 zip 复制进沙盒并校验 sha256;`PythonHost::DefaultStdLibPath()`
|
||||
现在只认 `MT_PYTHON_STDLIB`。
|
||||
4. 五个平台分别验证:编译、链接无未定义符号、`Py_Initialize`、静态 C 模块逐个 `import`、在 app 进程里跑 `system.py` → `prototype.RunApp()`(对照结果:74 个模块中 66 个加载成功,引导期调用 33 个 C++ 函数)。
|
||||
|
||||
| 平台 | 状态 |
|
||||
|
||||
+5
-2
@@ -100,8 +100,11 @@ one via a 1-line shim header, see below).
|
||||
`Python-2.7.18.tgz` (sha256 `b62c0e7937551d0cc02b8fd5cb0f544f9405bafc9a54d3808ed4594812edef43`)
|
||||
trimmed of what an embedded interpreter never builds or ships: `Doc Demo Tools
|
||||
Mac PCbuild`, `Modules/{_ctypes,expat,zlib,_sqlite}`, the stdlib test trees and
|
||||
the pre-VS2010 `PC/` project files. `Lib/` stays — batch 2P step 3 ships it as
|
||||
`python27.zip`.
|
||||
the pre-VS2010 `PC/` project files. `Lib/` stays: the `mtpython_stdlib` target
|
||||
packs it into `<build>/python27.zip` with `tools/py_embed/make_stdlib_zip.py`,
|
||||
which is what the host puts on `sys.path` (batch 2P step 3). Entries are
|
||||
**stored, never deflated** — the `zlib` module is trimmed out above, so
|
||||
`zipimport` has nothing to inflate with.
|
||||
* **Why vendored:** 2.7 is end-of-life, so there is no package to fetch on the
|
||||
four target SDKs, and the build must keep working offline and on the
|
||||
cross-build machines. It is the version the 40250 scripts are written for
|
||||
|
||||
@@ -159,10 +159,9 @@ if(BUILD_TESTING AND MT_HOST_BUILD)
|
||||
if(TARGET mtpython)
|
||||
add_executable(py_embed_test tests/py_embed_test.cpp)
|
||||
target_link_libraries(py_embed_test PRIVATE mtpython)
|
||||
# The stdlib the host puts on sys.path: here the vendored Lib/ directly (batch 2P step 3 ships
|
||||
# it as python27.zip).
|
||||
add_test(NAME python.embed COMMAND $<TARGET_FILE:py_embed_test>
|
||||
"${CMAKE_CURRENT_SOURCE_DIR}/third_party/cpython-2.7.18/Lib")
|
||||
add_dependencies(py_embed_test mtpython_stdlib)
|
||||
# The stdlib exactly as the host ships it: python27.zip, read through zipimport (批次 2P step 3).
|
||||
add_test(NAME python.embed COMMAND $<TARGET_FILE:py_embed_test> "${MT_PYTHON_STDLIB_ZIP}")
|
||||
endif()
|
||||
endif()
|
||||
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
// See PythonHost.h: the interpreter setup 40250 got from a real python27 installation.
|
||||
#include "ScriptLib/StdAfx.h"
|
||||
#include "PythonHost.h"
|
||||
|
||||
#include <cstdio>
|
||||
#include <cstdlib>
|
||||
|
||||
namespace
|
||||
{
|
||||
// The pending exception as text, so a failed bootstrap says what went wrong instead of leaving the
|
||||
// error for the first unrelated script to trip over.
|
||||
std::string take_error()
|
||||
{
|
||||
if (!PyErr_Occurred())
|
||||
return "no exception";
|
||||
|
||||
PyObject *type = NULL, *value = NULL, *tb = NULL;
|
||||
PyErr_Fetch(&type, &value, &tb);
|
||||
PyErr_NormalizeException(&type, &value, &tb);
|
||||
std::string text;
|
||||
if (PyObject* str = value ? PyObject_Str(value) : NULL)
|
||||
{
|
||||
if (PyString_Check(str))
|
||||
text = PyString_AS_STRING(str);
|
||||
Py_DECREF(str);
|
||||
}
|
||||
if (type && text.empty())
|
||||
text = ((PyTypeObject*) type)->tp_name;
|
||||
Py_XDECREF(type);
|
||||
Py_XDECREF(value);
|
||||
Py_XDECREF(tb);
|
||||
return text.empty() ? "unknown error" : text;
|
||||
}
|
||||
|
||||
bool fail(std::string* error, const std::string& text)
|
||||
{
|
||||
if (error)
|
||||
*error = text;
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
namespace PythonHost
|
||||
{
|
||||
|
||||
void Configure(const char* program_name)
|
||||
{
|
||||
// site.py lives in a prefix this interpreter does not have, the environment must not be able to
|
||||
// redirect a game's script layer (PYTHONPATH/PYTHONHOME), the "could not find platform libraries"
|
||||
// probe has nothing to find, and the pack directories are read-only on the mobile targets.
|
||||
Py_NoSiteFlag = 1;
|
||||
Py_IgnoreEnvironmentFlag = 1;
|
||||
Py_FrozenFlag = 1;
|
||||
Py_DontWriteBytecodeFlag = 1;
|
||||
Py_SetProgramName((char*) program_name);
|
||||
}
|
||||
|
||||
bool InstallStdLib(const char* stdlib_path, std::string* error)
|
||||
{
|
||||
if (!Py_IsInitialized())
|
||||
return fail(error, "interpreter is not initialized");
|
||||
if (!stdlib_path || !*stdlib_path)
|
||||
return fail(error, "no stdlib path (set MT_PYTHON_STDLIB or pass python27.zip)");
|
||||
|
||||
// zipimport and the source importer both go through the C library, so the path has to be a real
|
||||
// file: a pack entry or an Android asset must be staged out first (step 3c).
|
||||
if (FILE* fp = fopen(stdlib_path, "rb"))
|
||||
fclose(fp);
|
||||
else if (FILE* dir = fopen((std::string(stdlib_path) + "/os.py").c_str(), "rb"))
|
||||
fclose(dir);
|
||||
else
|
||||
return fail(error, std::string("cannot open ") + stdlib_path);
|
||||
|
||||
PySys_SetPath((char*) stdlib_path);
|
||||
|
||||
// Py_Initialize ran _PyCodecRegistry_Init against the then-empty path and 2.7 never retries it
|
||||
// ("no codec search functions registered" for the rest of the process), so the search function is
|
||||
// registered again now that encodings is importable.
|
||||
PyObject* encodings = PyImport_ImportModule("encodings");
|
||||
if (!encodings)
|
||||
return fail(error, "import encodings: " + take_error());
|
||||
PyObject* search = PyObject_GetAttrString(encodings, (char*) "search_function");
|
||||
Py_DECREF(encodings);
|
||||
if (!search)
|
||||
return fail(error, "encodings.search_function: " + take_error());
|
||||
|
||||
PyObject* codecs = PyImport_ImportModule("codecs");
|
||||
PyObject* registered = codecs ? PyObject_CallMethod(codecs, (char*) "register", (char*) "O", search) : NULL;
|
||||
Py_DECREF(search);
|
||||
Py_XDECREF(codecs);
|
||||
if (!registered)
|
||||
return fail(error, "codecs.register: " + take_error());
|
||||
Py_DECREF(registered);
|
||||
return true;
|
||||
}
|
||||
|
||||
std::string DefaultStdLibPath()
|
||||
{
|
||||
if (const char* env = getenv("MT_PYTHON_STDLIB"))
|
||||
return env;
|
||||
// Step 3c stages python27.zip out of the packed assets on Android/iOS and answers here; until then
|
||||
// the caller passes the path (the build tree's copy, for the native tests).
|
||||
return "";
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
#pragma once
|
||||
// The interpreter setup 40250 got for free from a real python27 installation.
|
||||
//
|
||||
// 40250's Main() only constructs CPythonLauncher: python27.dll found its own prefix, so `import site`,
|
||||
// the codec registry and every stdlib import worked before a single script ran. The embedded
|
||||
// interpreter has no prefix and no filesystem to probe, so the host does that part itself —
|
||||
// the flags before Py_Initialize, and the stdlib (python27.zip) on sys.path afterwards.
|
||||
// See docs/PORT-PLAN.md 批次 2P step 3.
|
||||
|
||||
#include <string>
|
||||
|
||||
namespace PythonHost
|
||||
{
|
||||
// Before the interpreter starts, i.e. before CPythonLauncher's constructor.
|
||||
// `program_name` is what CPythonLauncher::Create defaults to; it only names the interpreter.
|
||||
void Configure(const char* program_name = "eter.python");
|
||||
|
||||
// After Py_Initialize: replaces sys.path with `stdlib_path` (a python27.zip or a stdlib directory
|
||||
// on a real filesystem — zipimport and the .py importer both need a path the C library can open)
|
||||
// and re-registers the codec search function. False leaves *error set.
|
||||
bool InstallStdLib(const char* stdlib_path, std::string* error);
|
||||
|
||||
// The stdlib path this build was given: $MT_PYTHON_STDLIB when set, else the python27.zip next to
|
||||
// the executable, else "". Mobile staging (copy out of the app resources, checked against the
|
||||
// manifest sha256) is 批次 2P step 3c and will answer here too.
|
||||
std::string DefaultStdLibPath();
|
||||
}
|
||||
@@ -164,8 +164,9 @@ if(BUILD_TESTING AND CMAKE_SYSTEM_NAME STREQUAL CMAKE_HOST_SYSTEM_NAME)
|
||||
if(TARGET mtpython)
|
||||
add_executable(port_python_launcher_test ${CMAKE_CURRENT_SOURCE_DIR}/../../tests/port_python_launcher_test.cpp)
|
||||
target_link_libraries(port_python_launcher_test PRIVATE port_platform)
|
||||
add_dependencies(port_python_launcher_test mtpython_stdlib)
|
||||
add_test(NAME port.python_launcher
|
||||
COMMAND $<TARGET_FILE:port_python_launcher_test> ${MT_40250_CLIENT})
|
||||
COMMAND $<TARGET_FILE:port_python_launcher_test> ${MT_40250_CLIENT} ${MT_PYTHON_STDLIB_ZIP})
|
||||
set_tests_properties(port.python_launcher PROPERTIES SKIP_RETURN_CODE 77)
|
||||
endif()
|
||||
endif()
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
#include "StdAfx.h"
|
||||
#include "../EterPack/EterPackManager.h"
|
||||
#include "../EterBase/tea.h"
|
||||
|
||||
// CHINA_CRYPT_KEY
|
||||
DWORD g_adwEncryptKey[4];
|
||||
DWORD g_adwDecryptKey[4];
|
||||
|
||||
// PORT: the login cipher's key builder happens to live in this unit, but it is CAccountConnector's —
|
||||
// AccountConnector.h and the class come with the login slice (docs/PORT-PLAN.md 批次 2V1). Kept verbatim
|
||||
// and compiled out until then; the two keys above stay, they are what the network stream reads.
|
||||
#if 0
|
||||
#include "AccountConnector.h"
|
||||
|
||||
inline const BYTE* GetKey_20050304Myevan()
|
||||
{
|
||||
volatile static DWORD s_adwKey[1938];
|
||||
|
||||
volatile DWORD seed=1491971513;
|
||||
for (UINT i=0; i<BYTE(seed); i++)
|
||||
{
|
||||
seed^=2148941891;
|
||||
seed+=3592385981;
|
||||
s_adwKey[i]=seed;
|
||||
}
|
||||
|
||||
return (const BYTE*)s_adwKey;
|
||||
}
|
||||
|
||||
//#include <eterCrypt.h>
|
||||
|
||||
void CAccountConnector::__BuildClientKey_20050304Myevan()
|
||||
{
|
||||
const BYTE * c_pszKey = GetKey_20050304Myevan();
|
||||
memcpy(g_adwEncryptKey, c_pszKey+157, 16);
|
||||
|
||||
for (DWORD i = 0; i < 4; ++i)
|
||||
g_adwEncryptKey[i] = random();
|
||||
|
||||
tea_encrypt((DWORD *) g_adwDecryptKey, (const DWORD *) g_adwEncryptKey, (const DWORD *) (c_pszKey+37), 16);
|
||||
// TEA_Encrypt((DWORD *) g_adwDecryptKey, (const DWORD *) g_adwEncryptKey, (const DWORD *) (c_pszKey+37), 16);
|
||||
}
|
||||
#endif
|
||||
// END_OF_CHINA_CRYPT_KEY
|
||||
|
||||
PyObject * packExist(PyObject * poSelf, PyObject * poArgs)
|
||||
{
|
||||
char * strFileName;
|
||||
|
||||
if (!PyTuple_GetString(poArgs, 0, &strFileName))
|
||||
return Py_BuildException();
|
||||
|
||||
return Py_BuildValue("i", CEterPackManager::Instance().isExist(strFileName)?1:0);
|
||||
}
|
||||
|
||||
PyObject * packGet(PyObject * poSelf, PyObject * poArgs)
|
||||
{
|
||||
char * strFileName;
|
||||
|
||||
if (!PyTuple_GetString(poArgs, 0, &strFileName))
|
||||
return Py_BuildException();
|
||||
|
||||
// 파이썬에서 읽어드리는 패킹 파일은 python 파일과 txt 파일에 한정한다
|
||||
const char* pcExt = strrchr(strFileName, '.');
|
||||
if (pcExt) // 확장자가 있고
|
||||
{
|
||||
if ((stricmp(pcExt, ".py") == 0) ||
|
||||
(stricmp(pcExt, ".pyc") == 0) ||
|
||||
(stricmp(pcExt, ".txt") == 0))
|
||||
{
|
||||
CMappedFile file;
|
||||
const void * pData = NULL;
|
||||
|
||||
if (CEterPackManager::Instance().Get(file,strFileName,&pData))
|
||||
return Py_BuildValue("s#",pData, file.Size());
|
||||
}
|
||||
}
|
||||
|
||||
return Py_BuildException();
|
||||
}
|
||||
|
||||
void initpack()
|
||||
{
|
||||
static PyMethodDef s_methods[] =
|
||||
{
|
||||
{ "Exist", packExist, METH_VARARGS },
|
||||
{ "Get", packGet, METH_VARARGS },
|
||||
{ NULL, NULL },
|
||||
};
|
||||
|
||||
Py_InitModule("pack", s_methods);
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
#pragma once
|
||||
// PORT: a partial mirror of 40250 UserInterface/StdAfx.h.
|
||||
//
|
||||
// The original is the precompiled header of the whole client: it includes eterLib, eterPythonLib,
|
||||
// gameLib, scriptLib, milesLib, EffectLib, PRTerrainLib and SpeedTreeLib's StdAfx.h, plus <dshow.h>
|
||||
// and <qedit.h>, Locale.h and GameType.h. Only scriptLib exists in the port tree today, so this file
|
||||
// carries the part the ported units actually use and grows with the 2V slices that bring the rest
|
||||
// (docs/PORT-PLAN.md 批次 2V0-2V3).
|
||||
|
||||
#include "../ScriptLib/StdAfx.h"
|
||||
|
||||
#define APP_NAME "Metin 2"
|
||||
|
||||
enum
|
||||
{
|
||||
POINT_MAX_NUM = 255,
|
||||
CHARACTER_NAME_MAX_LEN = 24,
|
||||
#if defined(LOCALE_SERVICE_JAPAN)
|
||||
PLAYER_NAME_MAX_LEN = 16,
|
||||
#else
|
||||
PLAYER_NAME_MAX_LEN = 12,
|
||||
#endif
|
||||
};
|
||||
|
||||
// The Python module initializers RunMainScript calls, each defined in its own Python*Module.cpp.
|
||||
// Listed here as 40250 lists them; only the ported ones are defined.
|
||||
void initpack();
|
||||
@@ -14,6 +14,8 @@
|
||||
#include "ScriptLib/StdAfx.h"
|
||||
#include "ScriptLib/PythonLauncher.h"
|
||||
#include "EterPack/EterPackManager.h"
|
||||
#include "UserInterface/StdAfx.h" // initpack
|
||||
#include "../src/platform/ScriptLib/PythonHost.h"
|
||||
#include "../src/platform/UserInterface/UserInterface.h"
|
||||
|
||||
#include <algorithm>
|
||||
@@ -58,6 +60,7 @@ int main(int argc, char** argv)
|
||||
const char* strict = std::getenv("MT_ASSETS_STRICT");
|
||||
const bool is_strict = strict && std::string(strict) == "1";
|
||||
const std::string client = argc > 1 ? argv[1] : "";
|
||||
const std::string stdlib = argc > 2 ? argv[2] : PythonHost::DefaultStdLibPath();
|
||||
if (client.empty() || chdir(client.c_str()) != 0 || access("pack/Index", 0) != 0)
|
||||
{
|
||||
std::fprintf(stderr, "port_python_launcher_test: no 40250 Client/pack at '%s'\n", client.c_str());
|
||||
@@ -67,21 +70,36 @@ int main(int argc, char** argv)
|
||||
PackSingletons();
|
||||
CHECK(PackInitialize("pack"));
|
||||
|
||||
// Host policy, set before the launcher's ctor runs Py_Initialize: 40250 shipped a Python with a real
|
||||
// prefix on disk, the embedded one has none, so `import site` would fail at startup. The flags are the
|
||||
// ones tools/py_embed_android/main.c uses; step 3 moves them, and the stdlib on sys.path, into the
|
||||
// platform layer that every host shares.
|
||||
Py_NoSiteFlag = 1;
|
||||
Py_IgnoreEnvironmentFlag = 1;
|
||||
Py_FrozenFlag = 1;
|
||||
Py_DontWriteBytecodeFlag = 1;
|
||||
Py_SetProgramName((char*) "eter.python"); // CPythonLauncher::Create's default program name
|
||||
// What a real python27 installation did for 40250, the host does here: flags before Py_Initialize,
|
||||
// stdlib on sys.path after it (platform/ScriptLib/PythonHost.cpp).
|
||||
PythonHost::Configure();
|
||||
|
||||
{
|
||||
CPythonLauncher launcher; // ctor runs Py_Initialize, as 40250 Main() does on the stack
|
||||
CHECK(Py_IsInitialized());
|
||||
CHECK(launcher.Create());
|
||||
|
||||
std::string stdlib_error;
|
||||
CHECK(PythonHost::InstallStdLib(stdlib.c_str(), &stdlib_error));
|
||||
if (!stdlib_error.empty())
|
||||
std::fprintf(stderr, "InstallStdLib: %s\n", stdlib_error.c_str());
|
||||
// The stdlib is what the script layer's imports and the CP949/cp1252 text in the packs need.
|
||||
CHECK(launcher.RunLine("import os, string, types, codecs\nassert u'\\xe4'.encode('cp1252') == '\\xe4'"));
|
||||
|
||||
// RunMainScript's first line (UserInterface.cpp:243): the pack module, which is how system.py's
|
||||
// import hook and the window system read root/ and uiscript/ out of the registered packs.
|
||||
initpack();
|
||||
CHECK(launcher.RunLine("import pack"));
|
||||
CHECK(launcher.RunLine("assert pack.Exist('system.py') == 1"));
|
||||
CHECK(launcher.RunLine("assert pack.Exist('no/such.py') == 0"));
|
||||
CHECK(launcher.RunLine("__pack_len__ = len(pack.Get('system.py'))"));
|
||||
CHECK(launcher.RunLine("assert len(pack.Get('uiscript/selectcharacterwindow.py')) > 0"));
|
||||
CHECK(launcher.RunLine("assert len(pack.Get('locale/en/locale_game.txt')) > 0"));
|
||||
// Only .py/.pyc/.txt come out of it, and a miss is None rather than an exception
|
||||
// (packGet returns Py_BuildException(), which clears the error and builds None).
|
||||
CHECK(launcher.RunLine("assert pack.Get('icon/item/00000.tga') is None"));
|
||||
CHECK(launcher.RunLine("assert pack.Get('no/such.py') is None"));
|
||||
|
||||
// RegisterDebugFlag / RegisterCommandLine, verbatim from RunMainScript.
|
||||
CHECK(launcher.RunLine("__DEBUG__ = 0"));
|
||||
CHECK(launcher.RunLine("__COMMAND_LINE__ = \"\""));
|
||||
@@ -141,6 +159,7 @@ int main(int argc, char** argv)
|
||||
const VOID* data = NULL;
|
||||
CHECK(CEterPackManager::Instance().Get(file, "system.py", &data));
|
||||
CHECK(file.Size() > 0);
|
||||
CHECK(main_long("__pack_len__") == (long) file.Size()); // pack.Get returned the same entry
|
||||
|
||||
// Where it stops, spelled out: RunFile reports through Traceback(), which fetches the
|
||||
// exception and so leaves nothing behind, and LogBoxf is a platform stub. Running the same
|
||||
|
||||
@@ -209,3 +209,16 @@ endif()
|
||||
find_package(Threads REQUIRED)
|
||||
target_link_libraries(mtpython PUBLIC Threads::Threads m)
|
||||
add_library(mt3p::python ALIAS mtpython)
|
||||
|
||||
# --- the stdlib the host puts on sys.path (批次 2P step 3) ---
|
||||
# Built from the same vendored Lib/, so the interpreter and its library always match. MT_PYTHON_STDLIB_ZIP
|
||||
# is where every consumer (the native tests now, the mobile staging in step 3c) picks it up.
|
||||
find_package(Python3 COMPONENTS Interpreter REQUIRED)
|
||||
set(MT_PYTHON_STDLIB_ZIP "${CMAKE_BINARY_DIR}/python27.zip" CACHE INTERNAL "stdlib zip for the embedded interpreter")
|
||||
file(GLOB_RECURSE MT_PYTHON_STDLIB_SOURCES CONFIGURE_DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/Lib/*.py")
|
||||
add_custom_command(OUTPUT "${MT_PYTHON_STDLIB_ZIP}"
|
||||
COMMAND ${Python3_EXECUTABLE} "${CMAKE_SOURCE_DIR}/tools/py_embed/make_stdlib_zip.py"
|
||||
"${CMAKE_CURRENT_SOURCE_DIR}/Lib" "${MT_PYTHON_STDLIB_ZIP}"
|
||||
DEPENDS "${CMAKE_SOURCE_DIR}/tools/py_embed/make_stdlib_zip.py" ${MT_PYTHON_STDLIB_SOURCES}
|
||||
COMMENT "Packing python27.zip from the vendored Lib/")
|
||||
add_custom_target(mtpython_stdlib ALL DEPENDS "${MT_PYTHON_STDLIB_ZIP}")
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Pack the vendored CPython 2.7.18 Lib/ into the python27.zip the host puts on sys.path.
|
||||
|
||||
make_stdlib_zip.py <Lib dir> <out.zip>
|
||||
|
||||
40250 shipped a real python27 installation, so `import encodings` found Lib/ through the
|
||||
interpreter's prefix. The embedded interpreter has no prefix (Py_FrozenFlag, no site), so the host
|
||||
supplies the stdlib as this one file — the arrangement validated on the device by
|
||||
tools/py_embed_android/build-and-run.sh (PORT-PLAN 批次 2P step 3).
|
||||
|
||||
Entries are STORED, never deflated: the zlib module is not in the vendored Modules/ (see
|
||||
docs/THIRD-PARTY.md), so zipimport could not inflate them. Only .py goes in; zipimport reads
|
||||
source as happily as bytecode, and .pyc would pin the magic number to the build host's interpreter.
|
||||
Written deterministically (sorted, fixed timestamp) so the file's sha256 identifies its contents —
|
||||
3c checks that hash when staging the zip into a mobile sandbox.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
# Test suites, the Tk/2to3/bsddb/pip trees and the platform-specific stubs: none of the 40250 scripts
|
||||
# or their imports reach them, and they are most of the size.
|
||||
SKIP_DIRS = {"test", "tests", "lib-tk", "idlelib", "lib2to3", "bsddb", "ensurepip", "curses",
|
||||
"distutils", "unittest", "wsgiref", "multiprocessing", "compiler"}
|
||||
|
||||
|
||||
def main() -> int:
|
||||
if len(sys.argv) != 3:
|
||||
print(__doc__, file=sys.stderr)
|
||||
return 2
|
||||
lib, out = Path(sys.argv[1]), Path(sys.argv[2])
|
||||
if not (lib / "os.py").is_file():
|
||||
print(f"make_stdlib_zip: {lib} is not a CPython Lib/ directory", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
members = []
|
||||
for path in sorted(lib.rglob("*.py")):
|
||||
rel = path.relative_to(lib)
|
||||
if rel.parts[0].startswith("plat-") or SKIP_DIRS.intersection(rel.parts[:-1]):
|
||||
continue
|
||||
members.append((rel, path))
|
||||
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = out.with_suffix(".zip.tmp")
|
||||
with zipfile.ZipFile(tmp, "w", zipfile.ZIP_STORED) as zf:
|
||||
for rel, path in members:
|
||||
info = zipfile.ZipInfo(rel.as_posix(), date_time=(1980, 1, 1, 0, 0, 0))
|
||||
info.external_attr = 0o644 << 16
|
||||
zf.writestr(info, path.read_bytes())
|
||||
tmp.replace(out)
|
||||
print(f"make_stdlib_zip: {out} ({len(members)} modules, {out.stat().st_size} bytes)")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Reference in New Issue
Block a user