From 389b64386384f4a1de7d96c5a3ba3e2175921254 Mon Sep 17 00:00:00 2001 From: shenlei Date: Wed, 23 Sep 2026 09:57:27 +0900 Subject: [PATCH] =?UTF-8?q?docs:=20=E8=AE=B0=E5=BD=95=20Android=20GBK=20?= =?UTF-8?q?=E7=BC=96=E8=A7=A3=E7=A0=81=E7=9A=84=E5=B7=B2=E7=9F=A5=E9=97=AE?= =?UTF-8?q?=E9=A2=98=EF=BC=88=E6=8C=82=E8=B5=B7=EF=BC=89?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 用户决定：先保证 macOS 可用，Android 后期再想办法。本次只记录根因， 不动 text_codec.cpp。 症状：build-android.sh 默认 API 24 下 extension/src/net/text_codec.cpp 报 use of undeclared identifier 'iconv_open'。 根因：text_codec.cpp:10 用 __has_include() 探测；NDK 里该头在任何 API 等级都存在，但声明被 __BIONIC_AVAILABILITY_GUARD(28) 挡住。用 NDK 28.2 的 clang++ -fsyntax-only 实测：android24 报 3 处 undeclared， android28 通过。 但升 API 不是解法：NDK iconv.h 写明 bionic 只支持 utf8/ascii/usascii/ utf16{be,le}/utf32{be,le}/wchart，没有 GB2312/GBK。升到 28 后 iconv_open("GB2312","UTF-8") 运行期返回 -1，encode_one()/decode_wire() 对所有输入都返回 false——连纯 ASCII 也被拒，比现有 #else 的 ASCII 回退 分支（text_codec.cpp:104-127）更糟。且 APK min_sdk 要跟着 >=28， config/android/pyconfig.h 是 API 24 下 configure 的，需返工。 将来的修法写进了 PORT-PLAN 的新小节。 Co-Authored-By: Claude Opus 5 --- audit/history.jsonl | 1 + docs/PORT-PLAN.md | 21 +++++++++++++++++++++ 2 files changed, 22 insertions(+) diff --git a/audit/history.jsonl b/audit/history.jsonl index 2ea9a888..1dabc816 100644 --- a/audit/history.jsonl +++ b/audit/history.jsonl @@ -497,3 +497,4 @@ {"time": "2026-09-23T19:30:00Z", "event": "port_round", "unit": "PORT-PLAN 批次 2P step 3a/3b: python27.zip on sys.path (platform/ScriptLib/PythonHost) + the pack module", "ported": ["UserInterface/PythonPackModule.cpp (3 of 5 fns: packExist, packGet, initpack)"], "adapted": [], "deleted": [], "divergent": [], "needs_live": [], "stdlib": {"builder": "tools/py_embed/make_stdlib_zip.py , CMake target mtpython_stdlib -> /python27.zip", "contents": "397 modules, 6876547 bytes, ZIP_STORED (no zlib module, so zipimport cannot inflate); deterministic (sorted, fixed timestamp) so the sha256 identifies it", "sha256": "5fce8f974a89b66110823d7529be96e359b56dea05fc6875189a241d2d37a7d6", "skipped": "test/tests trees, lib-tk, idlelib, lib2to3, bsddb, ensurepip, curses, distutils, unittest, wsgiref, multiprocessing, compiler, plat-*", "host": "platform/ScriptLib/PythonHost.cpp: Configure() sets Py_NoSiteFlag/IgnoreEnvironment/Frozen/DontWriteBytecode + program name before Py_Initialize; InstallStdLib() does PySys_SetPath then re-registers encodings.search_function, because 2.7 initializes the codec registry once at startup and never retries"}, "status_note": "PythonPackModule's other two functions are CAccountConnector's login key builder, which happens to live in that unit; kept verbatim under #if 0 until the 2V1 login slice brings AccountConnector.h. port/UserInterface/StdAfx.h is a partial mirror: the original includes every library's StdAfx plus /, so it carries only what compiles today (ScriptLib/StdAfx.h, APP_NAME, the name length enum, initpack) and grows with the 2V slices.", "tests": ["ctest python.embed now runs against python27.zip itself (zipimport), not the loose Lib/", "ctest port.python_launcher: InstallStdLib + `import os, string, types, codecs` + cp1252 encode; initpack then pack.Exist/pack.Get for system.py (byte count equals CEterPackManager's), uiscript/selectcharacterwindow.py and locale/en/locale_game.txt; .tga and a missing file give None", "ctest macOS: 27/27 PASS", "android arm64 + ios arm64: port_platform (incl. PythonHost, PythonPackModule) compiles"], "not_done": ["3c: stage python27.zip out of the app resources into the sandbox on Android/iOS and check it against the manifest sha256; PythonHost::DefaultStdLibPath() only reads MT_PYTHON_STDLIB today"], "commit": "2P-step3ab"} {"time": "2026-09-23T21:40:00Z", "event": "port_round", "unit": "PORT-PLAN 批次 2P step 3c: staging python27.zip into the mobile sandbox", "ported": [], "adapted": [], "deleted": [], "divergent": [], "needs_live": [], "staging": {"ships": "make_stdlib_zip.py also writes .sha256; target mtpython_stdlib_project copy_if_different's python27.zip + .sha256 into project/ (gitignored), and both export presets' include_filter lists them so they enter the PCK", "runtime": "extension/src/python_stdlib.cpp (GDScript: Metin2Python.stdlib_path(force_stage)) — desktop uses ProjectSettings.globalize_path(res://python27.zip) in place; exported builds read the bytes out of the PCK, write user://python27.zip.part, verify the sha256 of the written file, then rename to user://python27.zip. An existing sandbox copy is reused only when it hashes to the shipped digest, so a torn first start or a new build re-stages.", "handoff": "PythonHost::SetDefaultStdLibPath() — DefaultStdLibPath() now answers with the staged path; $MT_PYTHON_STDLIB still wins. python_stdlib.cpp is the only unit that knows res:// / user://; port_platform stays godot-free.", "build_fix": "the Windows gate (MinGW, no mtpython) was broken by step 3b and is fixed here: port/CMakeLists.txt now also excludes UserInterface/StdAfx.h (it includes ScriptLib/StdAfx.h, as the original PCH does) and UserInterface/PythonPackModule.cpp, and platform/CMakeLists.txt excludes platform/ScriptLib/ the way port_logic already excluded port/ScriptLib/."}, "status_note": "On-device staging is not verified: the macOS test drives the mobile path with force_stage=true, and Android/iOS user:// sandboxes come with step 4. The zip's presence in an exported pack was checked with --export-pack, not with an APK. Nothing in the app's boot path calls Metin2Python.stdlib_path() yet — the interpreter only starts in the process with 2V0; today the test is its only caller.", "tests": ["project/python_stdlib_test.gd (godot --headless --path project --script python_stdlib_test.gd) -> PASS: in-place path on desktop; force_stage copies into user://, sha256 matches the shipped digest, bytes equal res://, no .part left, ZIPReader opens it and finds encodings/__init__.py and os.py; a second call does not re-copy (mtime unchanged); a deliberately corrupted sandbox copy is re-staged", "godot --headless --export-pack macOS lists python27.zip (6876547) and python27.zip.sha256 (65)"], "not_done": "真机验证（Android/iOS 进程内落盘 + system.py -> prototype.RunApp()）随 2P step 4；Linux/Windows 见 step 4 表格", "commit": "2P-step3c"} {"time": "2026-09-23T22:20:00Z", "event": "scope_decision", "unit": "目标平台收窄为 macOS arm64 + Android arm64（Linux / Windows 不做）", "ported": [], "adapted": [], "deleted": [], "divergent": [], "needs_live": [], "decision": {"by": "user", "statement": "Linux 和 Windows 不考虑，目前主要支持 macOS arm64 和 Android arm64", "ios": "iOS arm64 保留为交叉编译干净的次要目标，不作为验收目标（我的判断，待用户确认）", "mingw": "仓库里的 mingw-w64 交叉编译只留作可移植性门禁——它这次抓到了 UserInterface/StdAfx.h 的真问题——不代表支持 Windows，不为它投入额外工作，也不阻塞任何步骤"}, "docs": ["docs/PORT-PLAN.md: 第 1 节新增范围段；批次 2A step 3 平台列表；批次 2P step 1/2 措辞；step 4 由「五个平台」改为「两个目标平台」，Linux/Windows 两行划掉并保留原因备查；总表 2P 行", "docs/PLATFORMS.md: 新增 Scope 段", "docs/THIRD-PARTY.md: CPython 的 Windows 段由「not built yet」改为 out of scope + 原因备查", "docs/PYTHON-EMBED-EVAL.md: 下一步 1/3 两条"], "tests": "仅文档改动，无代码改动；port_map.py check 0 error / 1 tag；参考 EterPack.cpp 的 8 个 pack key 值逐个 git grep --cached -w，均未入库", "not_done": "2P step 4 剩余项仍在：Android 真机 APK 进程内的 3c 沙盒落地，以及 app 进程里跑 system.py → prototype.RunApp()"} +{"time": "2026-09-23T22:35:00Z", "event": "known_issue", "unit": "Android GBK 编解码挂起；优先保证 macOS 可用", "ported": [], "adapted": [], "deleted": [], "divergent": [], "needs_live": [], "decision": {"by": "user", "statement": "先不管这个问题了，先保证 macOS 能正常使用，Android 后期再想办法"}, "finding": {"symptom": "build-android.sh 默认 API 24 下 extension/src/net/text_codec.cpp 编译失败：use of undeclared identifier 'iconv_open' / 'iconv' / 'iconv_close'", "root_cause": "text_codec.cpp:10 用 __has_include() 探测。NDK 里该头在任何 API 等级都存在，但声明被 __BIONIC_AVAILABILITY_GUARD(28) 挡住，探测方式与 bionic 的可用性模型不匹配", "measured": "NDK 28.2.13676358 clang++ -fsyntax-only：--target=aarch64-linux-android24 报 3 处 undeclared；android28 通过", "why_api_bump_is_wrong": "NDK iconv.h 注释写明 bionic 只支持 utf8/ascii/usascii/utf16be/utf16le/utf32be/utf32le/wchart，没有 GB2312/GBK。升到 28 后 iconv_open(\"GB2312\",\"UTF-8\") 运行期返回 -1，encode_one()/decode_wire() 对所有输入都返回 false，连纯 ASCII 也被拒，比现有 #else 的 ASCII 回退分支（text_codec.cpp:104-127）更糟。附带成本：APK min_sdk 需 >=28；config/android/pyconfig.h 是 API 24 下 configure 的（HAVE_LANGINFO_H 因 nl_langinfo 需 API 26 被关掉），要重跑 gen_pyconfig.sh android 并重验内建模块表", "future_fix": "1) 探测改为 #if __has_include() && !defined(__ANDROID__)，不要写 __ANDROID_API__ >= 28；2) Android 自带 GBK<->Unicode 表（约 22k 码位）。40250 原客户端走 Windows MultiByteToWideChar(CP949/CP936)，跨平台无对应物；macOS 能用只是系统 libiconv 恰好带 GBK"}, "docs": ["docs/PORT-PLAN.md 新增「已知问题：Android 的 GBK 编解码（挂起，2026-09-23）」小节"], "tests": "仅文档改动；诊断由 NDK clang++ -fsyntax-only 双 API 等级实测得出", "not_done": "text_codec.cpp 未改动；Android 扩展构建目前仍是红的"} diff --git a/docs/PORT-PLAN.md b/docs/PORT-PLAN.md index af2a6baf..cb530d4d 100644 --- a/docs/PORT-PLAN.md +++ b/docs/PORT-PLAN.md @@ -341,6 +341,27 @@ extension/third_party/cpython-2.7.18/ # 静态库（2P 批次 | ~~Linux x86_64~~ | 不做（2026-09-23 定，见第 1 节） | | ~~Windows x64~~ | 不做（2026-09-23 定，见第 1 节）。记录原因备查：上游 2.7 只支持 MSVC + `PC/pyconfig.h`，而本仓库的 Windows 门禁是 mingw-w64 交叉编译，`PC/pyconfig.h` 的 gnu-win32 分支不设 `HAVE_UNISTD_H`，`posixmodule.c`/`dynload_win.c` 与 MinGW 头文件冲突。因此 mingw 门禁下不建 `mtpython`，`port_logic`/`port_platform` 自动排除 ScriptLib | +### 已知问题：Android 的 GBK 编解码（挂起，2026-09-23） + +**优先级：macOS 先跑通，Android 后面再说**（用户 2026-09-23 定）。 + +`extension/src/net/text_codec.cpp:10` 用 `__has_include()` 探测 iconv。NDK 里这个头文件在 +任何 API 等级都存在，但函数声明被 `__BIONIC_AVAILABILITY_GUARD(28)` 挡住，所以 `build-android.sh` +的默认 API 24（`build-android.sh:14`）下编译报 `use of undeclared identifier 'iconv_open'`。 + +**升到 API 28 不是解法**：NDK `iconv.h` 原文写明 bionic 只支持 +`utf8/ascii/usascii/utf16be/utf16le/utf32be/utf32le/wchart`，**没有 GB2312/GBK**。升级后 +`iconv_open("GB2312","UTF-8")` 运行期返回 `-1`，`encode_one()`/`decode_wire()` 对所有输入都 +返回 false——连纯 ASCII 也被拒，比现在 `#else` 的 ASCII 回退分支（`text_codec.cpp:104-127`）更糟。 +附带成本：APK `min_sdk` 必须跟着 ≥28，且 `config/android/pyconfig.h` 是 API 24 下 configure 出来的 +（`HAVE_LANGINFO_H` 因 `nl_langinfo` 需 API 26 被关掉），要重跑 `gen_pyconfig.sh android` 并重验内建模块表。 + +将来动手时分两步： +1. 探测条件从「头文件在不在」改成「这个平台的 iconv 能不能用」——`#if __has_include() && !defined(__ANDROID__)`， + Android 一律走回退分支，先把交叉编译修绿。**不要**写成 `__ANDROID_API__ >= 28`。 +2. 补真正的转换：Android 需要自带 GBK↔Unicode 表（约 22k 码位）。40250 原客户端走 Windows + `MultiByteToWideChar(CP949/CP936)`，跨平台没有对应物；macOS 能用只是因为系统 libiconv 恰好带 GBK。 + ### 批次 2V0–2V3：纵向切片 2A 先根据真实 `#include` 和 Python import 生成每个切片的机器可读单元清单；清单提交到 `audit/`，进入切片前不得仍有