docs: 记录 Android GBK 编解码的已知问题(挂起)

用户决定:先保证 macOS 可用,Android 后期再想办法。本次只记录根因,
不动 text_codec.cpp。

症状:build-android.sh 默认 API 24 下 extension/src/net/text_codec.cpp 报
use of undeclared identifier 'iconv_open'。

根因:text_codec.cpp:10 用 __has_include(<iconv.h>) 探测;NDK 里该头在任何
API 等级都存在,但声明被 __BIONIC_AVAILABILITY_GUARD(28) 挡住。用
NDK 28.2 的 clang++ -fsyntax-only 实测:android24 报 3 处 undeclared,
android28 通过。

但升 API 不是解法:NDK iconv.h 写明 bionic 只支持 utf8/ascii/usascii/
utf16{be,le}/utf32{be,le}/wchart,没有 GB2312/GBK。升到 28 后
iconv_open("GB2312","UTF-8") 运行期返回 -1,encode_one()/decode_wire()
对所有输入都返回 false——连纯 ASCII 也被拒,比现有 #else 的 ASCII 回退
分支(text_codec.cpp:104-127)更糟。且 APK min_sdk 要跟着 >=28,
config/android/pyconfig.h 是 API 24 下 configure 的,需返工。

将来的修法写进了 PORT-PLAN 的新小节。

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
shenlei
2026-09-23 09:57:27 +09:00
co-authored by Claude Opus 5
parent 19a9b18554
commit 389b643863
2 changed files with 22 additions and 0 deletions
+1
View File
@@ -497,3 +497,4 @@
{"time": "2026-09-23T19:30:00Z", "event": "port_round", "unit": "PORT-PLAN 批次 2P step 3a/3b: python27.zip on sys.path (platform/ScriptLib/PythonHost) + the pack module", "ported": ["UserInterface/PythonPackModule.cpp (3 of 5 fns: packExist, packGet, initpack)"], "adapted": [], "deleted": [], "divergent": [], "needs_live": [], "stdlib": {"builder": "tools/py_embed/make_stdlib_zip.py <Lib> <out.zip>, CMake target mtpython_stdlib -> <build>/python27.zip", "contents": "397 modules, 6876547 bytes, ZIP_STORED (no zlib module, so zipimport cannot inflate); deterministic (sorted, fixed timestamp) so the sha256 identifies it", "sha256": "5fce8f974a89b66110823d7529be96e359b56dea05fc6875189a241d2d37a7d6", "skipped": "test/tests trees, lib-tk, idlelib, lib2to3, bsddb, ensurepip, curses, distutils, unittest, wsgiref, multiprocessing, compiler, plat-*", "host": "platform/ScriptLib/PythonHost.cpp: Configure() sets Py_NoSiteFlag/IgnoreEnvironment/Frozen/DontWriteBytecode + program name before Py_Initialize; InstallStdLib() does PySys_SetPath then re-registers encodings.search_function, because 2.7 initializes the codec registry once at startup and never retries"}, "status_note": "PythonPackModule's other two functions are CAccountConnector's login key builder, which happens to live in that unit; kept verbatim under #if 0 until the 2V1 login slice brings AccountConnector.h. port/UserInterface/StdAfx.h is a partial mirror: the original includes every library's StdAfx plus <dshow.h>/<qedit.h>, so it carries only what compiles today (ScriptLib/StdAfx.h, APP_NAME, the name length enum, initpack) and grows with the 2V slices.", "tests": ["ctest python.embed now runs against python27.zip itself (zipimport), not the loose Lib/", "ctest port.python_launcher: InstallStdLib + `import os, string, types, codecs` + cp1252 encode; initpack then pack.Exist/pack.Get for system.py (byte count equals CEterPackManager's), uiscript/selectcharacterwindow.py and locale/en/locale_game.txt; .tga and a missing file give None", "ctest macOS: 27/27 PASS", "android arm64 + ios arm64: port_platform (incl. PythonHost, PythonPackModule) compiles"], "not_done": ["3c: stage python27.zip out of the app resources into the sandbox on Android/iOS and check it against the manifest sha256; PythonHost::DefaultStdLibPath() only reads MT_PYTHON_STDLIB today"], "commit": "2P-step3ab"}
{"time": "2026-09-23T21:40:00Z", "event": "port_round", "unit": "PORT-PLAN 批次 2P step 3c: staging python27.zip into the mobile sandbox", "ported": [], "adapted": [], "deleted": [], "divergent": [], "needs_live": [], "staging": {"ships": "make_stdlib_zip.py also writes <out>.sha256; target mtpython_stdlib_project copy_if_different's python27.zip + .sha256 into project/ (gitignored), and both export presets' include_filter lists them so they enter the PCK", "runtime": "extension/src/python_stdlib.cpp (GDScript: Metin2Python.stdlib_path(force_stage)) — desktop uses ProjectSettings.globalize_path(res://python27.zip) in place; exported builds read the bytes out of the PCK, write user://python27.zip.part, verify the sha256 of the written file, then rename to user://python27.zip. An existing sandbox copy is reused only when it hashes to the shipped digest, so a torn first start or a new build re-stages.", "handoff": "PythonHost::SetDefaultStdLibPath() — DefaultStdLibPath() now answers with the staged path; $MT_PYTHON_STDLIB still wins. python_stdlib.cpp is the only unit that knows res:// / user://; port_platform stays godot-free.", "build_fix": "the Windows gate (MinGW, no mtpython) was broken by step 3b and is fixed here: port/CMakeLists.txt now also excludes UserInterface/StdAfx.h (it includes ScriptLib/StdAfx.h, as the original PCH does) and UserInterface/PythonPackModule.cpp, and platform/CMakeLists.txt excludes platform/ScriptLib/ the way port_logic already excluded port/ScriptLib/."}, "status_note": "On-device staging is not verified: the macOS test drives the mobile path with force_stage=true, and Android/iOS user:// sandboxes come with step 4. The zip's presence in an exported pack was checked with --export-pack, not with an APK. Nothing in the app's boot path calls Metin2Python.stdlib_path() yet — the interpreter only starts in the process with 2V0; today the test is its only caller.", "tests": ["project/python_stdlib_test.gd (godot --headless --path project --script python_stdlib_test.gd) -> PASS: in-place path on desktop; force_stage copies into user://, sha256 matches the shipped digest, bytes equal res://, no .part left, ZIPReader opens it and finds encodings/__init__.py and os.py; a second call does not re-copy (mtime unchanged); a deliberately corrupted sandbox copy is re-staged", "godot --headless --export-pack macOS <zip> lists python27.zip (6876547) and python27.zip.sha256 (65)"], "not_done": "真机验证(Android/iOS 进程内落盘 + system.py -> prototype.RunApp())随 2P step 4Linux/Windows 见 step 4 表格", "commit": "2P-step3c"}
{"time": "2026-09-23T22:20:00Z", "event": "scope_decision", "unit": "目标平台收窄为 macOS arm64 + Android arm64Linux / Windows 不做)", "ported": [], "adapted": [], "deleted": [], "divergent": [], "needs_live": [], "decision": {"by": "user", "statement": "Linux 和 Windows 不考虑,目前主要支持 macOS arm64 和 Android arm64", "ios": "iOS arm64 保留为交叉编译干净的次要目标,不作为验收目标(我的判断,待用户确认)", "mingw": "仓库里的 mingw-w64 交叉编译只留作可移植性门禁——它这次抓到了 UserInterface/StdAfx.h 的真问题——不代表支持 Windows,不为它投入额外工作,也不阻塞任何步骤"}, "docs": ["docs/PORT-PLAN.md: 第 1 节新增范围段;批次 2A step 3 平台列表;批次 2P step 1/2 措辞;step 4 由「五个平台」改为「两个目标平台」,Linux/Windows 两行划掉并保留原因备查;总表 2P 行", "docs/PLATFORMS.md: 新增 Scope 段", "docs/THIRD-PARTY.md: CPython 的 Windows 段由「not built yet」改为 out of scope + 原因备查", "docs/PYTHON-EMBED-EVAL.md: 下一步 1/3 两条"], "tests": "仅文档改动,无代码改动;port_map.py check 0 error / 1 tag;参考 EterPack.cpp 的 8 个 pack key 值逐个 git grep --cached -w,均未入库", "not_done": "2P step 4 剩余项仍在:Android 真机 APK 进程内的 3c 沙盒落地,以及 app 进程里跑 system.py → prototype.RunApp()"}
{"time": "2026-09-23T22:35:00Z", "event": "known_issue", "unit": "Android GBK 编解码挂起;优先保证 macOS 可用", "ported": [], "adapted": [], "deleted": [], "divergent": [], "needs_live": [], "decision": {"by": "user", "statement": "先不管这个问题了,先保证 macOS 能正常使用,Android 后期再想办法"}, "finding": {"symptom": "build-android.sh 默认 API 24 下 extension/src/net/text_codec.cpp 编译失败:use of undeclared identifier 'iconv_open' / 'iconv' / 'iconv_close'", "root_cause": "text_codec.cpp:10 用 __has_include(<iconv.h>) 探测。NDK 里该头在任何 API 等级都存在,但声明被 __BIONIC_AVAILABILITY_GUARD(28) 挡住,探测方式与 bionic 的可用性模型不匹配", "measured": "NDK 28.2.13676358 clang++ -fsyntax-only--target=aarch64-linux-android24 报 3 处 undeclaredandroid28 通过", "why_api_bump_is_wrong": "NDK iconv.h 注释写明 bionic 只支持 utf8/ascii/usascii/utf16be/utf16le/utf32be/utf32le/wchart,没有 GB2312/GBK。升到 28 后 iconv_open(\"GB2312\",\"UTF-8\") 运行期返回 -1encode_one()/decode_wire() 对所有输入都返回 false,连纯 ASCII 也被拒,比现有 #else 的 ASCII 回退分支(text_codec.cpp:104-127)更糟。附带成本:APK min_sdk 需 >=28config/android/pyconfig.h 是 API 24 下 configure 的(HAVE_LANGINFO_H 因 nl_langinfo 需 API 26 被关掉),要重跑 gen_pyconfig.sh android 并重验内建模块表", "future_fix": "1) 探测改为 #if __has_include(<iconv.h>) && !defined(__ANDROID__),不要写 __ANDROID_API__ >= 282) Android 自带 GBK<->Unicode 表(约 22k 码位)。40250 原客户端走 Windows MultiByteToWideChar(CP949/CP936),跨平台无对应物;macOS 能用只是系统 libiconv 恰好带 GBK"}, "docs": ["docs/PORT-PLAN.md 新增「已知问题:Android 的 GBK 编解码(挂起,2026-09-23)」小节"], "tests": "仅文档改动;诊断由 NDK clang++ -fsyntax-only 双 API 等级实测得出", "not_done": "text_codec.cpp 未改动;Android 扩展构建目前仍是红的"}
+21
View File
@@ -341,6 +341,27 @@ extension/third_party/cpython-2.7.18/ # 静态库(2P 批次
| ~~Linux x86_64~~ | 不做(2026-09-23 定,见第 1 节) |
| ~~Windows x64~~ | 不做(2026-09-23 定,见第 1 节)。记录原因备查:上游 2.7 只支持 MSVC + `PC/pyconfig.h`,而本仓库的 Windows 门禁是 mingw-w64 交叉编译,`PC/pyconfig.h` 的 gnu-win32 分支不设 `HAVE_UNISTD_H``posixmodule.c`/`dynload_win.c` 与 MinGW 头文件冲突。因此 mingw 门禁下不建 `mtpython``port_logic`/`port_platform` 自动排除 ScriptLib |
### 已知问题:Android 的 GBK 编解码(挂起,2026-09-23
**优先级:macOS 先跑通,Android 后面再说**(用户 2026-09-23 定)。
`extension/src/net/text_codec.cpp:10``__has_include(<iconv.h>)` 探测 iconv。NDK 里这个头文件在
任何 API 等级都存在,但函数声明被 `__BIONIC_AVAILABILITY_GUARD(28)` 挡住,所以 `build-android.sh`
的默认 API 24`build-android.sh:14`)下编译报 `use of undeclared identifier 'iconv_open'`
**升到 API 28 不是解法**NDK `iconv.h` 原文写明 bionic 只支持
`utf8/ascii/usascii/utf16be/utf16le/utf32be/utf32le/wchart`**没有 GB2312/GBK**。升级后
`iconv_open("GB2312","UTF-8")` 运行期返回 `-1``encode_one()`/`decode_wire()` 对所有输入都
返回 false——连纯 ASCII 也被拒,比现在 `#else` 的 ASCII 回退分支(`text_codec.cpp:104-127`)更糟。
附带成本:APK `min_sdk` 必须跟着 ≥28,且 `config/android/pyconfig.h` 是 API 24 下 configure 出来的
`HAVE_LANGINFO_H``nl_langinfo` 需 API 26 被关掉),要重跑 `gen_pyconfig.sh android` 并重验内建模块表。
将来动手时分两步:
1. 探测条件从「头文件在不在」改成「这个平台的 iconv 能不能用」——`#if __has_include(<iconv.h>) && !defined(__ANDROID__)`
Android 一律走回退分支,先把交叉编译修绿。**不要**写成 `__ANDROID_API__ >= 28`
2. 补真正的转换:Android 需要自带 GBK↔Unicode 表(约 22k 码位)。40250 原客户端走 Windows
`MultiByteToWideChar(CP949/CP936)`,跨平台没有对应物;macOS 能用只是因为系统 libiconv 恰好带 GBK。
### 批次 2V02V3:纵向切片
2A 先根据真实 `#include` 和 Python import 生成每个切片的机器可读单元清单;清单提交到 `audit/`,进入切片前不得仍有