port(2P step 3c): stage python27.zip into the mobile sandbox

CPython opens the stdlib zip with its own stdio, so it needs a real path.
On the desktop res:// is a directory and globalize_path() is enough; in an
exported build res:// is inside the PCK, which nothing outside Godot reads.

  make_stdlib_zip.py now also writes <out>.sha256
  mtpython_stdlib_project copies python27.zip + .sha256 into project/
    (gitignored), and both export presets' include_filter lists them, so
    they enter the PCK — checked with --export-pack
  extension/src/python_stdlib.{h,cpp} (Metin2Python.stdlib_path in
    GDScript): reads the bytes out of the PCK, writes
    user://python27.zip.part, hashes the file as written against the
    shipped digest, then renames. A sandbox copy is reused only when it
    matches, so a first start killed mid-copy and a zip replaced by a new
    build both re-stage instead of feeding zipimport a torn file.
  PythonHost::SetDefaultStdLibPath / DefaultStdLibPath() answer with that
    path ($MT_PYTHON_STDLIB still wins). python_stdlib.cpp is the only unit
    that knows res:// / user://; port_platform stays godot-free.

Also fixes the Windows gate, which step 3b broke: without mtpython the
MinGW build compiled UserInterface/StdAfx.h (it includes ScriptLib/StdAfx.h,
as the original PCH does). port/CMakeLists.txt excludes that header and
PythonPackModule.cpp with ScriptLib, and platform/CMakeLists.txt excludes
platform/ScriptLib/ the same way.

Not done: on-device staging. The macOS test drives the mobile path with
force_stage=true, and nothing in the app boot calls it yet — the
interpreter only starts in the process with 2V0.

gates: python_stdlib_test.gd PASS (in-place path, staging, sha256, bytes
equal res://, no .part, ZIPReader finds encodings/__init__.py, no re-copy
on a second call, corrupted copy re-staged) · macOS ctest 27/27 ·
mingw + android + ios port_platform compile clean · port_map.py check 0
errors · key leak check 8/8 none.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
shenlei
2026-09-23 09:39:33 +09:00
co-authored by Claude Opus 5
parent 2b873f190f
commit ffbb5d48d6
17 changed files with 329 additions and 18 deletions
+5
View File
@@ -4,6 +4,11 @@
# so it shares the per-library definitions.
file(GLOB_RECURSE MT_PLATFORM_SOURCES CONFIGURE_DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/*.cpp)
# Same rule as port_logic: without the embedded interpreter there is no ScriptLib to back
# (PORT-PLAN 批次 2P step 4 — Windows).
if(NOT TARGET mtpython)
list(FILTER MT_PLATFORM_SOURCES EXCLUDE REGEX "/ScriptLib/")
endif()
add_library(port_platform STATIC ${MT_PLATFORM_SOURCES})
target_link_libraries(port_platform PUBLIC port_logic)
foreach(src IN LISTS MT_PLATFORM_SOURCES)
@@ -94,13 +94,24 @@ bool InstallStdLib(const char* stdlib_path, std::string* error)
return true;
}
namespace
{
std::string g_staged_path;
}
void SetDefaultStdLibPath(const char* stdlib_path)
{
g_staged_path = stdlib_path ? stdlib_path : "";
}
std::string DefaultStdLibPath()
{
// The environment still wins, so a developer can point a build at another stdlib without rebuilding.
if (const char* env = getenv("MT_PYTHON_STDLIB"))
return env;
// Step 3c stages python27.zip out of the packed assets on Android/iOS and answers here; until then
// the caller passes the path (the build tree's copy, for the native tests).
return "";
// Whatever staged the zip for this platform (extension/src/python_stdlib.cpp inside Godot; nothing
// in the native tests, which pass the build tree's copy on the command line).
return g_staged_path;
}
}
@@ -20,8 +20,9 @@ void Configure(const char* program_name = "eter.python");
// and re-registers the codec search function. False leaves *error set.
bool InstallStdLib(const char* stdlib_path, std::string* error);
// The stdlib path this build was given: $MT_PYTHON_STDLIB when set, else the python27.zip next to
// the executable, else "". Mobile staging (copy out of the app resources, checked against the
// manifest sha256) is 批次 2P step 3c and will answer here too.
// Where the host put the stdlib: $MT_PYTHON_STDLIB when set, else whatever SetDefaultStdLibPath was
// given, else "". Inside Godot that is the staged python27.zip (extension/src/python_stdlib.h), which
// on Android/iOS is the sandbox copy of the one in the read-only bundle.
std::string DefaultStdLibPath();
void SetDefaultStdLibPath(const char* stdlib_path);
}
+5 -2
View File
@@ -11,9 +11,12 @@ if(WIN32)
endif()
# The script layer needs the embedded interpreter (mt3p::python), which has no Windows build yet
# (PORT-PLAN 批次 2P step 4). Without it, ScriptLib and its <Python-2.7/*> shims are left out.
# The units outside ScriptLib that reach it: UserInterface/StdAfx.h includes ScriptLib/StdAfx.h (as the
# original PCH does), and PythonPackModule.cpp is a Python module. Both go with it.
set(MT_PORT_NEEDS_PYTHON "/ScriptLib/|/UserInterface/StdAfx\\.h|/UserInterface/PythonPackModule\\.cpp")
if(NOT TARGET mtpython)
list(FILTER MT_PORT_SOURCES EXCLUDE REGEX "/ScriptLib/")
list(FILTER MT_PORT_HEADERS EXCLUDE REGEX "/ScriptLib/|/common/shim/sdk/Python-2\\.7/")
list(FILTER MT_PORT_SOURCES EXCLUDE REGEX "${MT_PORT_NEEDS_PYTHON}")
list(FILTER MT_PORT_HEADERS EXCLUDE REGEX "${MT_PORT_NEEDS_PYTHON}|/common/shim/sdk/Python-2\\.7/")
endif()
add_library(port_logic STATIC ${MT_PORT_SOURCES})
+127
View File
@@ -0,0 +1,127 @@
#include "python_stdlib.h"
#include <godot_cpp/classes/dir_access.hpp>
#include <godot_cpp/classes/file_access.hpp>
#include <godot_cpp/classes/project_settings.hpp>
#include <godot_cpp/core/class_db.hpp>
#include <godot_cpp/variant/utility_functions.hpp>
#include <string>
#ifdef MTGODOT_HAVE_PYTHON
#include "platform/ScriptLib/PythonHost.h"
#endif
using namespace godot;
namespace mtgodot {
namespace {
// Both ship in the PCK; the build copies them next to project.godot (extension/CMakeLists.txt).
constexpr const char *ZIP_RES = "res://python27.zip";
constexpr const char *SHA_RES = "res://python27.zip.sha256";
constexpr const char *ZIP_USER = "user://python27.zip";
constexpr const char *ZIP_USER_TMP = "user://python27.zip.part";
// std::string, not String: this lives at file scope and a godot::String's constructor would run at
// dlopen time, before the extension's bindings exist.
std::string g_last_error;
String fail(String *error, const String &text) {
g_last_error = text.utf8().get_data();
if (error) {
*error = text;
}
return String();
}
String expected_sha() {
if (!FileAccess::file_exists(SHA_RES)) {
return String();
}
return FileAccess::get_file_as_string(SHA_RES).strip_edges();
}
} // namespace
String stage_python_stdlib(String *error, bool force_stage) {
g_last_error.clear();
if (error) {
*error = String();
}
if (!FileAccess::file_exists(ZIP_RES)) {
return fail(error, String("no ") + ZIP_RES +
" (build with -DMTGODOT_EMBED_PYTHON=ON, which copies it into project/)");
}
ProjectSettings *settings = ProjectSettings::get_singleton();
// Desktop and the editor: res:// is a directory on disk, so CPython can open the file in place and
// nothing is copied. In an exported game globalize_path() returns a path next to the executable
// that does not exist, which is exactly the case the staging below is for.
const String in_place = settings->globalize_path(ZIP_RES);
if (!force_stage && !in_place.begins_with("res://") && FileAccess::file_exists(in_place)) {
return in_place;
}
const String want = expected_sha();
if (want.is_empty()) {
return fail(error, String("no ") + SHA_RES + " beside the stdlib zip");
}
// An already-staged copy is reused only when it is the zip this build ships: a new version of the
// app brings a new digest, and a first start that was killed mid-copy leaves a short file.
if (FileAccess::file_exists(ZIP_USER) && FileAccess::get_sha256(ZIP_USER) == want) {
return settings->globalize_path(ZIP_USER);
}
const PackedByteArray data = FileAccess::get_file_as_bytes(ZIP_RES);
if (data.is_empty()) {
return fail(error, String("cannot read ") + ZIP_RES);
}
{
Ref<FileAccess> out = FileAccess::open(ZIP_USER_TMP, FileAccess::WRITE);
if (out.is_null()) {
return fail(error, String("cannot write ") + ZIP_USER_TMP + ": " +
itos((int) FileAccess::get_open_error()));
}
out->store_buffer(data);
out->close();
}
// Hash the file as written, not the buffer: a full disk reports itself here and not later as a
// zipimport failure three imports into system.py.
if (FileAccess::get_sha256(ZIP_USER_TMP) != want) {
DirAccess::remove_absolute(ZIP_USER_TMP);
return fail(error, String("staged ") + ZIP_USER + " does not match " + want);
}
if (FileAccess::file_exists(ZIP_USER)) {
DirAccess::remove_absolute(ZIP_USER);
}
if (DirAccess::rename_absolute(ZIP_USER_TMP, ZIP_USER) != OK) {
return fail(error, String("cannot rename ") + ZIP_USER_TMP + " to " + ZIP_USER);
}
UtilityFunctions::print("[python27] staged ", ZIP_USER, " (", data.size(), " bytes, sha256 ", want, ")");
return settings->globalize_path(ZIP_USER);
}
String Metin2Python::stdlib_path(bool force_stage) {
const String path = stage_python_stdlib(nullptr, force_stage);
#ifdef MTGODOT_HAVE_PYTHON
if (!path.is_empty()) {
PythonHost::SetDefaultStdLibPath(path.utf8().get_data());
}
#endif
return path;
}
String Metin2Python::last_error() {
return String::utf8(g_last_error.c_str());
}
void Metin2Python::_bind_methods() {
ClassDB::bind_static_method("Metin2Python", D_METHOD("stdlib_path", "force_stage"),
&Metin2Python::stdlib_path, DEFVAL(false));
ClassDB::bind_static_method("Metin2Python", D_METHOD("last_error"), &Metin2Python::last_error);
}
} // namespace mtgodot
+37
View File
@@ -0,0 +1,37 @@
#pragma once
// python_stdlib — where the embedded interpreter's standard library is on this platform
// (docs/PORT-PLAN.md 批次 2P step 3c).
//
// CPython opens python27.zip with its own stdio, so it needs a real filesystem path. That is free on
// the desktop, where res:// is a directory on disk, but on Android and iOS res:// is inside the PCK,
// which nothing outside Godot can read. There the zip is copied once into the sandbox (user://) and
// the copy is accepted only when it hashes to the digest built beside the zip, so an interrupted
// first start or a zip replaced by a new build re-stages instead of feeding zipimport a torn file.
//
// Only this file knows about res:// / user://: PythonHost (platform/ScriptLib) stays godot-free and
// is told the path, through PythonHost::SetDefaultStdLibPath.
#include <godot_cpp/classes/object.hpp>
#include <godot_cpp/variant/string.hpp>
namespace mtgodot {
// The real filesystem path of python27.zip, staging it out of res:// when res:// is not on disk.
// Empty when the zip is missing or the staged copy does not verify; the reason goes to *error.
// force_stage runs the mobile path even where res:// is readable, which is how it is tested.
godot::String stage_python_stdlib(godot::String *error = nullptr, bool force_stage = false);
class Metin2Python : public godot::Object {
GDCLASS(Metin2Python, godot::Object)
public:
// Returns the path and, when the embedded interpreter is built in, hands it to PythonHost so
// PythonHost::DefaultStdLibPath() answers with it from then on.
static godot::String stdlib_path(bool force_stage);
static godot::String last_error();
protected:
static void _bind_methods();
};
} // namespace mtgodot
+2
View File
@@ -12,6 +12,7 @@
#include "net/m2_client.h"
#include "pack40250_node.h"
#include "proto/proto_node.h"
#include "python_stdlib.h"
#include "static_object.h"
#include "terrain_splat.h"
#include "tree_placeholder.h"
@@ -45,6 +46,7 @@ void initialize_mtgodot_module(ModuleInitializationLevel p_level) {
GDREGISTER_CLASS(mtgodot::M2Client);
GDREGISTER_CLASS(mtgodot::Metin2Proto);
GDREGISTER_CLASS(mtgodot::Metin2Pack);
GDREGISTER_CLASS(mtgodot::Metin2Python);
}
void uninitialize_mtgodot_module(ModuleInitializationLevel p_level) {