feat(web): add source-first Python autopilot and optimize provider waits
Some checks failed
M6 deployable RC / quick (push) Has been cancelled
M6 deployable RC / chromium (push) Has been cancelled
M6 deployable RC / release (push) Has been cancelled

This commit is contained in:
mes123456
2026-09-08 10:59:49 +08:00
parent a69e41457a
commit edcf0fef9a
29 changed files with 10149 additions and 412 deletions

168
my-tools/AUTOPILOT.md Normal file
View File

@@ -0,0 +1,168 @@
# Python Blender Web Autopilot
This supervisor advances the authoritative `docs/EXECUTION_QUEUE.md` task toward
source-backed Blender Web behavior. Product backlog entries are bounded work
items inside that task, not a replacement queue. Blender source/Main remains the
specification. A task, route assignment, successful build or agent-written
`ready` result is never counted as completed migration.
## Commands
Run from the repository root with Python 3.11 or newer on Linux:
```bash
python3 my-tools/run_web_product_migration.py doctor
python3 my-tools/run_web_product_migration.py start
python3 my-tools/run_web_product_migration.py status
python3 my-tools/run_web_product_migration.py timings
python3 my-tools/run_web_product_migration.py pause
python3 my-tools/run_web_product_migration.py reconcile
python3 my-tools/run_web_product_migration.py resume
```
`start` detaches from the terminal and transfers an exclusive OS lock to its
child without an ownership gap. `run --once` performs at most one implementation
attempt in the foreground. `--max-cycles N` bounds a run; zero means unlimited
cycles within the token and failure budgets. The existing
`bash my-tools/start_unattended.sh` product entrypoint uses this supervisor.
`stop` aliases `pause`. It terminates the active process group and records the
outcome. Interrupted implementation/commit/push work enters `needs_reconcile`.
`reconcile` is read-only. After inspecting the referenced edits, process receipts
and remote HEAD where relevant, `resume --retry-blocked` explicitly acknowledges
that state and resets the affected failure budget. An unreadable checkpoint or
unknown write outcome is never silently replayed.
For login/boot service management:
```bash
python3 my-tools/run_web_product_migration.py service
systemctl --user daemon-reload
systemctl --user enable --now blender-web-autopilot
```
The generated service captures the executable search path and restarts unexpected
process failures. Exit codes 2/3/4 prevent restart loops for known blockers,
unknown outcomes and budget exhaustion. Unattended operation without a logged-in
user also requires the host's user-service lingering configuration.
## Execution Contract
1. Read and validate the current queue, parent manifest and compact context.
2. Select a bounded current-task workflow, prioritizing audited production gaps.
3. Run one noninteractive Codex CLI turn to implement source/protocol/UI code and
prepare focused tests. The configured model is inherited; no model override,
API credential or SSH password is embedded in the program.
4. Require actual process exit 0, `turn.completed` and schema-valid output.
Finding a result file alone never ends or accepts a worker.
5. Independently run source build, typecheck, lint, Chromium, positive/negative,
undo/redo, save/reopen and determinism checks. Duplicate argv/cwd pairs run
once. Build parallelism is capped at eight jobs.
6. Require fresh runtime case receipts, native Main state observations and
SHA-256-bound compiled/runtime artifacts. Bind the validation logs and input
files in a supervisor-generated receipt.
7. Reuse receipts only while every input and artifact hash still matches.
Complete untouched workflows before revisiting shared-file invalidations.
8. After all current workflows pass, run a separate handoff turn restricted to
repository generators and evidence files, then independently run governance
gates and verify the resulting manifest. Advance only to its `nextTask`.
9. Commit exact declared handoff paths using `git commit --only`, retaining
unrelated staged edits. Pre-existing dirty paths cannot be silently included.
Push without force and require remote `main` HEAD to equal the local commit.
A successor is generated through repository tools, never inferred from a task
number. With unverified source units and no valid successor, the supervisor
reports `planning_required`; it does not announce full desktop parity.
## Policy And Evidence
Configuration: `my-tools/wbv2_autopilot_policy.json`. Defaults are 30 minutes per
worker/check, three attempts per failing workflow, 500,000 reported input/output
tokens per run, 32 MiB per process log, eight build jobs, and six push retries.
The token limit is checked between turns, so the last turn can exceed it.
`autoCommit` and `autoPush` control repository publication. All mandatory runtime
check categories remain required even when policy is edited.
The supervisor uses the existing authenticated `codex exec` CLI with
`workspace-write` sandbox and no interactive approvals. Worker stdout is JSONL;
structured output is requested with `--output-schema` and
`--output-last-message`. Consult the installed `codex exec --help` for the
supported CLI flags in this environment.
State is under `tests/golden/WBV2/autopilot/`:
- `state.json`: atomic checkpoint, job states, PIDs and reasons.
- `owner.lock`: OS-backed exclusive ownership with process identity.
- `baseline.json`: initial HEAD and pre-existing dirty paths for publication.
- `events.jsonl`, `daemon.log`: concise lifecycle events and detached output.
- `runs/<attempt>/`: worker output, check logs, process exit receipts, fresh
behavioral receipts and the final verified receipt.
`node tools/web/print-wbv2-live-status.mjs --json` includes Python supervisor
state. The existing Python status display no longer infers completion from
34/34 implemented backlog entries.
Production implementation workers do not SSH, deploy or mutate the live website.
They verify new code against disposable local runtime fixtures. The production
origin is available as `WBV2_DEPLOYED_ORIGIN`; release deployment remains a
separate operation after tested artifacts and a rollback target exist. Configure
deployment credentials through the existing host credential facilities, never
inside this policy or logs.
## Verification
```bash
python3 -m unittest discover -s my-tools -p test_wbv2_autopilot.py -v
python3 -m py_compile my-tools/wbv2_autopilot.py my-tools/run_web_product_migration.py
node tools/web/run-wbv2-scheduler.mjs --preflight-only
git diff --check
```
The tests exercise supervisor behavior with synthetic fixtures and real child
processes. They are not Blender migration coverage. Actual coverage is earned
only by source-backed native/Chromium module validation and repository binding.
## Source-First Workflow Rule
Business modules and function routes drive implementation: module -> user
workflow -> function route -> native source/operator cases. Search the supplied
`blender-5.2.0` source and task-scoped references before editing Web behavior.
Use `search-wbv2-batch-source.mjs --unit <id> --pattern <symbol>` for a bounded
source search. Worker results must name `workflow`, exact catalog `routeIds`
and native `sourceReferences`; the independent receipt binds those files.
## Timing And Provider Backoff
Displayed time labels use Beijing time, `YYYY-MM-DD HH:MM`. Machine checkpoints
retain precise timestamps; reports express elapsed time in minutes.
```bash
python3 my-tools/run_web_product_migration.py timings
python3 my-tools/run_web_product_migration.py timings --json --write
python3 my-tools/benchmark_wbv2_autopilot.py
```
`timings` is read-only unless `--write` is given. It reports each completed step,
phase totals/shares, maximum durations and the current process. JSONL tool spans
separate provider wait, model response, reads, edits and command execution.
Nested spans are not added to process totals again. Old logs cannot reconstruct
per-item timestamps or unrecorded gaps.
Short commands wait for process completion directly. Heartbeats are written
every five seconds rather than on every poll. Provider events are read
incrementally with a default observation interval of 0.25 seconds. Shared
evidence paths are hashed once per fingerprint operation.
Before any agent action, overload/concurrency/rate-limit errors end the waiting
process promptly. No first action within 60 seconds also ends the request.
Automatic retry requires a known JSONL handshake, no actions or unknown output,
and unchanged source/protected fingerprints. Retry delays are 30, 60, 120 and
at most 240 seconds; four provider failures stop with `provider_blocked`.
Provider failures do not consume implementation attempts. Silence after actions
has a 180-second limit unless a tool is active; unknown outcomes require
reconciliation. Build/test limits and behavioral gates remain in force.
After a reviewed maintenance commit, `resume --retry-blocked --refresh-baseline`
can refresh the Git baseline only when no workflow edits or receipts are recorded.
It retains remaining dirty paths as protected pre-existing work, and cannot
absorb partially implemented or already verified workflow changes.

View File

@@ -0,0 +1,56 @@
#!/usr/bin/env python3
"""Compare recorded scheduling timings with real read-only context checks."""
import contextlib
import io
import json
from pathlib import Path
import statistics
import tempfile
from wbv2_autopilot import Autopilot, CONTROL, POLICY, ROOT, read_json, write_json
from wbv2_timing import timing_report
def main():
policy = read_json(ROOT / POLICY)
before = timing_report(ROOT / CONTROL)
samples = []
with tempfile.TemporaryDirectory(prefix="wbv2-timing-") as temporary:
root = Path(temporary)
runner = Autopilot(root, policy)
with contextlib.redirect_stdout(io.StringIO()):
for script in ["check-task-context.mjs", "check-context-governance.mjs"]:
for iteration in range(3):
result = runner.run_process(["node", str(ROOT / "tools/web" / script)], root, root / f"{script}-{iteration}.log", 30, phase="context_check")
if result["exitCode"] != 0:
raise RuntimeError(f"context benchmark failed: {script}")
samples.append({"script": script, "seconds": result["durationSeconds"]})
old = [row["seconds"] for row in before["steps"] if row["phase"] == "context_check"]
median_old = statistics.median(old)
median_new = statistics.median(sample["seconds"] for sample in samples)
report = {
"schemaVersion": 1,
"operation": "AUTOPILOT_PERFORMANCE_OPTIMIZATION",
"observedAt": timing_report(ROOT / CONTROL)["observedAt"],
"timezone": "Asia/Shanghai",
"labelPrecision": "minute",
"baseline": before,
"benchmark": {
"scope": "Same read-only context entrypoints: historical observed baseline versus three current samples per entrypoint; not whole-project throughput.",
"samples": samples,
"baselineMedianMinutes": round(median_old / 60, 6),
"optimizedMedianMinutes": round(median_new / 60, 6),
"observedSpeedup": round(median_old / median_new, 2),
"observedReductionPercent": round((1 - median_new / median_old) * 100, 2),
},
"providerPolicy": {key: policy[key] for key in ["modelStartTimeoutSeconds", "modelIdleTimeoutSeconds", "providerBackoffSeconds", "providerMaxBackoffSeconds", "maxProviderFailures"]},
"sourcePolicy": {"driver": policy["strategy"], "authority": policy["sourceAuthority"], "sourceFirst": policy["sourceFirst"]},
"limitations": ["Provider availability/account concurrency is external; these changes do not remove upstream limits.", "No source capability completion or inventory increment is claimed.", "Legacy JSONL lacks per-item timestamps; new runs record observed per-item intervals separately."],
}
destination = ROOT / "tests/golden/WBV2-P1424-001/performance-optimization-report.json"
write_json(destination, report)
print(json.dumps({"report": str(destination.relative_to(ROOT)), "observedAt": report["observedAt"], **report["benchmark"]}, indent=2))
if __name__ == "__main__":
main()

View File

@@ -1,373 +1,7 @@
#!/usr/bin/env python3
"""Implement the Web product backlog through bounded non-interactive Codex turns."""
"""Compatibility entrypoint for the independently verified Python supervisor."""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import signal
import subprocess
import time
from datetime import datetime, timezone
from pathlib import Path
from wbv2_runtime import ROOT
CONTROL = ROOT / "tests/golden/WBV2"
BACKLOG = CONTROL / "web-product-backlog.json"
STATUS = CONTROL / "web-product-migration-status.json"
LOCK = CONTROL / "web-product-migration.lock"
PID = CONTROL / "web-product-migration.pid"
LOG_ROOT = CONTROL / "product-migration"
ACTIVE_PROCESS: subprocess.Popen[str] | None = None
def now() -> str:
return datetime.now(timezone.utc).isoformat()
def read_json(path: Path) -> dict:
return json.loads(path.read_text(encoding="utf-8"))
def write_json(path: Path, value: object) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
temporary = path.with_suffix(path.suffix + f".tmp-{os.getpid()}")
temporary.write_text(json.dumps(value, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
temporary.replace(path)
def alive(pid: object) -> bool:
if not isinstance(pid, int) or pid <= 0:
return False
try:
os.kill(pid, 0)
except ProcessLookupError:
return False
except PermissionError:
return True
return True
def acquire_lock() -> None:
try:
handle = os.open(LOCK, os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0o644)
except FileExistsError:
try:
owner = read_json(LOCK).get("pid")
except (OSError, json.JSONDecodeError):
owner = None
if not alive(owner):
LOCK.unlink(missing_ok=True)
return acquire_lock()
raise SystemExit(f"web product migration already running: pid={owner}")
os.write(handle, (json.dumps({"pid": os.getpid(), "startedAt": now()}) + "\n").encode())
os.close(handle)
def file_hash(path: str) -> str | None:
absolute = ROOT / path
if not absolute.is_file():
return None
return hashlib.sha256(absolute.read_bytes()).hexdigest()
def refresh_modules(backlog: dict) -> None:
states = {task["id"]: task["status"] for task in backlog.get("tasks", [])}
for module in backlog.get("workflowModules", []):
task_ids = module.get("taskIds", [])
if not task_ids:
continue
task_states = [states.get(task_id, "pending") for task_id in task_ids]
if all(state == "implemented" for state in task_states):
module["status"] = "implemented"
elif any(state == "active" for state in task_states):
module["status"] = "active"
elif any(state == "blocked" for state in task_states):
module["status"] = "blocked"
else:
module["status"] = "partial"
def finalize_product_task(backlog: dict) -> None:
task_id = str(backlog.get("queueTask", "WBV2-P1420-001"))
parent_id = str(backlog.get("rootTask", "WBV2-P1420-000"))
index = read_json(CONTROL / "task-index.json")
plan_path = ROOT / index["source"]["path"]
plan = read_json(plan_path)
for record in plan.get("tasks", []):
if record.get("id") == task_id:
record["status"] = "done"
record["ready"] = False
plan["status"] = "PRODUCT_IMPLEMENTATION_COMPLETE"
write_json(plan_path, plan)
parent_manifest_path = ROOT / "tests/golden" / parent_id / "manifest.json"
parent_manifest = read_json(parent_manifest_path)
if isinstance(parent_manifest.get("artifacts", {}).get("plan"), dict):
parent_manifest["artifacts"]["plan"]["sha256"] = file_hash(str(plan_path.relative_to(ROOT)))
write_json(parent_manifest_path, parent_manifest)
output = ROOT / "tests/golden" / task_id
report_path = output / "product-implementation-report.json"
status_path = ROOT / "docs/status" / f"{task_id}.md"
task_context_path = output / "task-context.json"
changed_files = []
for task in backlog.get("tasks", []):
result_path = ROOT / task["resultPath"]
result = read_json(result_path) if result_path.is_file() else {}
changed_files.extend(result.get("changedFiles", []))
changed_files = list(dict.fromkeys(path for path in changed_files if isinstance(path, str)))
write_json(report_path, {
"schemaVersion": 1,
"task": task_id,
"operation": "WEB_PRODUCT_IMPLEMENTATION_COMPLETE",
"status": "implemented",
"strategy": backlog.get("strategy"),
"verificationPolicy": backlog.get("verificationPolicy"),
"workflowModules": backlog.get("workflowModules", []),
"implementationTasks": backlog.get("tasks", []),
"changedFiles": changed_files,
"sourceInventory": backlog.get("sourceInventory"),
})
status_path.write_text(
f"# {task_id} status\n\n- status: `implemented`\n- workflow modules: `{len(backlog.get('workflowModules', []))}/{len(backlog.get('workflowModules', []))}`\n- implementation tasks: `{len(backlog.get('tasks', []))}/{len(backlog.get('tasks', []))}`\n- verification: `deferred_until_user_reports`\n- nextTask: `NONE`\n",
encoding="utf-8",
)
artifacts = {
"parentManifest": {"path": f"tests/golden/{parent_id}/manifest.json", "sha256": file_hash(f"tests/golden/{parent_id}/manifest.json")},
"backlog": {"path": str(BACKLOG.relative_to(ROOT)), "sha256": file_hash(str(BACKLOG.relative_to(ROOT)))},
"strategy": {"path": "docs/WEB_PRODUCT_MIGRATION_STRATEGY.md", "sha256": file_hash("docs/WEB_PRODUCT_MIGRATION_STRATEGY.md")},
"report": {"path": str(report_path.relative_to(ROOT)), "sha256": file_hash(str(report_path.relative_to(ROOT)))},
"status": {"path": str(status_path.relative_to(ROOT)), "sha256": file_hash(str(status_path.relative_to(ROOT)))},
"plan": {"path": str(plan_path.relative_to(ROOT)), "sha256": file_hash(str(plan_path.relative_to(ROOT)))},
}
if task_context_path.is_file():
artifacts["taskContext"] = {"path": str(task_context_path.relative_to(ROOT)), "sha256": file_hash(str(task_context_path.relative_to(ROOT)))}
write_json(output / "manifest.json", {
"schemaVersion": 1,
"task": task_id,
"parentTask": parent_id,
"operation": "WEB_PRODUCT_IMPLEMENTATION_COMPLETE",
"status": "done",
"implementationComplete": True,
"verificationDeferred": True,
"artifacts": artifacts,
"nextTask": None,
})
subprocess.run(["node", "tools/web/generate-v2-task-index.mjs", "--plan-path", str(plan_path.relative_to(ROOT))], cwd=ROOT, check=True)
def write_status(backlog: dict, status: str, **extra: object) -> None:
tasks = backlog.get("tasks", [])
modules = backlog.get("workflowModules", [])
write_json(STATUS, {
"schemaVersion": 1,
"pid": os.getpid(),
"status": status,
"updatedAt": now(),
"tasks": {
"total": len(tasks),
"implemented": sum(task.get("status") == "implemented" for task in tasks),
"active": [task["id"] for task in tasks if task.get("status") == "active"],
"pending": sum(task.get("status") == "pending" for task in tasks),
"blocked": sum(task.get("status") == "blocked" for task in tasks),
},
"workflowModules": {
"total": len(modules),
"implemented": sum(module.get("status") == "implemented" for module in modules),
"active": [module["id"] for module in modules if module.get("status") == "active"],
"blocked": [module["id"] for module in modules if module.get("status") == "blocked"],
},
**extra,
})
def prompt(task: dict) -> str:
backlog = read_json(BACKLOG)
queue_task = str(backlog.get("queueTask", "WBV2-P1420-001"))
attempt = int(task.get("attempts", 1))
retry_rules = ""
if attempt > 1:
retry_rules = f"""
Retry continuation (attempt {attempt}):
- A previous attempt may already have changed owned files. Treat those edits as the starting point; do not recreate or discard them.
- After the mandatory queue/print/check commands, use at most 2 additional read/search commands.
- Inspect only the exact unfinished edited regions, fix the remaining integration issue, and write the result JSON immediately.
- Do not repeat broad discovery, reread unchanged context, or polish unrelated code.
"""
return f"""Implement exactly one Web Blender product workflow task in {ROOT}.
Task: {task['id']} - {task['title']}
Commands to expose: {', '.join(task['commands'])}
Owned paths: {', '.join(task['ownedPaths'])}
Implementation requirement: {task['implementation']}
Rules:
- Follow the repository AGENTS.md minimal-context instructions first.
- The authoritative queue task is {queue_task}; this product task is one bounded subtask inside it. Run the default task-context check once, and never pass the product subtask id to check-task-context.
- Preserve all existing dirty changes. Do not revert or rewrite unrelated work.
- Read only the owned paths and their direct imported types/helpers.
- For scoped searches, pass each owned path explicitly through the batch search runtime option; never pass a bare --runtime flag.
- Hard budget: at most 16 tool commands, including at most 8 read/search commands. After the read budget, implement immediately.
- Do not print or reread the complete diff; inspect only the edited regions once before writing the result.
- Implement production behavior through the existing Blender Main/WASM command routes; do not simulate Blender behavior in JavaScript.
- Do not run tests, builds, typecheck, lint, Chromium, full validators, or coverage binding. The user explicitly deferred verification to real usage.
- Do not modify docs/EXECUTION_QUEUE.md, nextTask.md, source coverage ledgers, manifests, or this backlog.
- Do not commit or push. The outer unattended scheduler owns rotation and repository handoff.
- Finish the implementation in this turn. Do not stop at a plan.
- At the end write {task['resultPath']} as JSON with schemaVersion=1, task={task['id']!r}, status='implemented', changedFiles, and a concise summary.
{retry_rules}
"""
def run_task(task: dict, heartbeat_seconds: float) -> tuple[int, bool]:
global ACTIVE_PROCESS
before = {path: file_hash(path) for path in task["ownedPaths"]}
result_path = ROOT / task["resultPath"]
result_path.unlink(missing_ok=True)
log_path = LOG_ROOT / f"{task['id']}.codex.jsonl"
LOG_ROOT.mkdir(parents=True, exist_ok=True)
command = [
"codex", "exec", "--dangerously-bypass-approvals-and-sandbox",
"--ephemeral", "--cd", str(ROOT), "--json", "-",
]
completed_turn = False
result_seen_at: float | None = None
task_started_at = time.monotonic()
with log_path.open("w", encoding="utf-8") as log:
process = subprocess.Popen(
command,
cwd=ROOT,
stdin=subprocess.PIPE,
stdout=log,
stderr=subprocess.STDOUT,
text=True,
start_new_session=True,
)
ACTIVE_PROCESS = process
assert process.stdin is not None
process.stdin.write(prompt(task))
process.stdin.close()
while process.poll() is None:
write_status(read_json(BACKLOG), "implementing", activeTask=task["id"], childPid=process.pid, log=str(log_path.relative_to(ROOT)))
log.flush()
if result_path.is_file():
if result_seen_at is None:
result_seen_at = time.monotonic()
try:
with log_path.open("rb") as current_log:
end = current_log.seek(0, os.SEEK_END)
current_log.seek(max(0, end - 8192))
completed_turn = b'"type":"turn.completed"' in current_log.read()
except OSError:
completed_turn = False
if time.monotonic() - result_seen_at >= 5:
completed_turn = True
if completed_turn:
try:
os.killpg(process.pid, signal.SIGTERM)
process.wait(timeout=5)
except ProcessLookupError:
pass
except subprocess.TimeoutExpired:
os.killpg(process.pid, signal.SIGKILL)
process.wait()
break
if time.monotonic() - task_started_at >= 900:
try:
os.killpg(process.pid, signal.SIGTERM)
process.wait(timeout=5)
except ProcessLookupError:
pass
except subprocess.TimeoutExpired:
os.killpg(process.pid, signal.SIGKILL)
process.wait()
break
time.sleep(heartbeat_seconds)
code = 0 if completed_turn else (process.returncode or 0)
ACTIVE_PROCESS = None
after = {path: file_hash(path) for path in task["ownedPaths"]}
changed = any(before[path] != after[path] for path in before)
result_ok = False
if result_path.is_file():
try:
result = read_json(result_path)
result_ok = result.get("task") == task["id"] and result.get("status") == "implemented"
except json.JSONDecodeError:
pass
return code, result_ok and (changed or completed_turn)
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--once", action="store_true", help="run at most one task")
parser.add_argument("--max-attempts", type=int, default=3)
parser.add_argument("--heartbeat-seconds", type=float, default=5.0)
args = parser.parse_args()
if args.max_attempts < 1 or args.heartbeat_seconds <= 0:
parser.error("limits must be positive")
acquire_lock()
PID.write_text(f"{os.getpid()}\n", encoding="ascii")
stopped = False
def stop(signum: int, _frame: object) -> None:
nonlocal stopped
stopped = True
if ACTIVE_PROCESS and ACTIVE_PROCESS.poll() is None:
try:
os.killpg(ACTIVE_PROCESS.pid, signum)
except ProcessLookupError:
pass
signal.signal(signal.SIGINT, stop)
signal.signal(signal.SIGTERM, stop)
try:
ran = 0
while not stopped:
backlog = read_json(BACKLOG)
for task in backlog.get("tasks", []):
if task.get("status") == "active":
task["status"] = "pending"
refresh_modules(backlog)
pending = next((task for task in backlog.get("tasks", []) if task.get("status") == "pending"), None)
if pending is None:
refresh_modules(backlog)
write_json(BACKLOG, backlog)
blocked = [task for task in backlog.get("tasks", []) if task.get("status") == "blocked"]
if not blocked:
finalize_product_task(backlog)
write_status(backlog, "blocked" if blocked else "implementation_complete")
return 2 if blocked else 0
pending["status"] = "active"
pending["attempts"] = int(pending.get("attempts", 0)) + 1
pending["startedAt"] = now()
refresh_modules(backlog)
write_json(BACKLOG, backlog)
write_status(backlog, "implementing", activeTask=pending["id"])
code, implemented = run_task(pending, args.heartbeat_seconds)
backlog = read_json(BACKLOG)
task = next(item for item in backlog["tasks"] if item["id"] == pending["id"])
if code == 0 and implemented:
task["status"] = "implemented"
task["completedAt"] = now()
task.pop("lastError", None)
elif int(task.get("attempts", 0)) >= args.max_attempts:
task["status"] = "blocked"
task["lastError"] = f"codex exit={code}; implementation result or owned-path change missing"
else:
task["status"] = "pending"
task["lastError"] = f"codex exit={code}; retry scheduled"
refresh_modules(backlog)
write_json(BACKLOG, backlog)
ran += 1
if args.once and ran >= 1:
write_status(backlog, "idle", lastTask=pending["id"], exitCode=code)
return 0 if implemented else 2
finally:
PID.unlink(missing_ok=True)
LOCK.unlink(missing_ok=True)
return 0
from wbv2_autopilot import main
if __name__ == "__main__":

View File

@@ -49,16 +49,13 @@ def main() -> int:
product = status["progress"].get("productMigration", {})
product_pid = product.get("pid")
product_alive = isinstance(product_pid, int) and (Path("/proc") / str(product_pid)).exists()
product_complete = (
product.get("workflowTotal", 0) > 0
and product.get("workflowImplemented") == product.get("workflowTotal")
and product.get("taskImplemented") == product.get("taskTotal")
)
product_complete = status.get("autopilot", {}).get("state") == "complete"
status["myTools"]["productMigrationAlive"] = product_alive
status["unattendedState"] = (
"IMPLEMENTING" if product_alive and product.get("status") == "implementing"
"WAITING_PROVIDER" if product_alive and product.get("status") == "waiting_provider"
else "IMPLEMENTING" if product_alive and product.get("status") in {"implementing", "validating", "finalizing", "preflight", "retry_pending"}
else "COMPLETE" if product_complete
else "BLOCKED" if product.get("status") == "blocked"
else "BLOCKED" if product.get("status") in {"blocked", "provider_blocked", "needs_reconcile", "planning_required", "budget_exhausted", "ready_to_publish"}
else "PLANNING" if status.get("state") == "IDLE" and watchdog_alive and watchdog.get("status") == "waiting_for_source_batch"
else status.get("state", "UNKNOWN")
)
@@ -112,9 +109,9 @@ def main() -> int:
for line in format_progress(status["progress"], status["fastPath"]):
print(line)
if status.get("unattendedState") == "IMPLEMENTING":
print(" message: IMPLEMENTING: one workflow task is being edited; verification is deferred to user operation.")
print(" message: IMPLEMENTING: bounded source work and independent runtime acceptance are in progress.")
elif status.get("unattendedState") == "COMPLETE":
print(" message: COMPLETE: the Web workflow implementation backlog is closed; user-reported defects become focused follow-up tasks.")
print(" message: COMPLETE: the supervisor accepted terminal runtime evidence and found no unverified source units.")
elif status.get("unattendedState") == "PLANNING":
print(" message: PLANNING: watchdog heartbeat is active; exact source gaps are queued without false coverage.")
else:

View File

@@ -79,19 +79,8 @@ start_watchdog() {
}
start_product_migration() {
local pid_file="$CONTROL_DIR/web-product-migration.pid"
if test -f "$pid_file"; then
local product_pid
product_pid="$(tr -d '[:space:]' < "$pid_file")"
if test -n "$product_pid" && kill -0 "$product_pid" 2>/dev/null; then
printf 'product migration already running: %s\n' "$product_pid"
return
fi
mv "$pid_file" "$pid_file.stale-$(date +%s)"
fi
nohup setsid python3 "$ROOT_DIR/my-tools/run_web_product_migration.py" \
> "$CONTROL_DIR/web-product-migration.stdout.log" 2>&1 < /dev/null &
printf 'product migration launch pid: %s\n' "$!"
python3 "$ROOT_DIR/my-tools/run_web_product_migration.py" start \
--max-cycles "$SCHEDULER_MAX_ITERATIONS"
}
if test "$MIGRATION_MODE" = "source-verification"; then

View File

@@ -0,0 +1,375 @@
"""Supervisor regression tests; synthetic receipts are not Blender coverage."""
import contextlib
import copy
import io
import json
import os
from pathlib import Path
import subprocess
import sys
import tempfile
import time
import unittest
from unittest.mock import patch
from wbv2_autopilot import Autopilot, CONTROL, Lease, POLICY, REQUIRED, ProviderUnavailable, inventory_proof, manifest_proof, queue_context, sha, validate_command, write_json
from wbv2_timing import WorkerTrace, display_times, minute_label, timing_report
class SupervisorTests(unittest.TestCase):
def setUp(self):
self.temporary = tempfile.TemporaryDirectory()
self.addCleanup(self.temporary.cleanup)
self.root = Path(self.temporary.name)
self.policy = json.loads((Path(__file__).with_name("wbv2_autopilot_policy.json")).read_text())
self.policy.update(heartbeatSeconds=0.01, retryDelaySeconds=0, autoCommit=False, autoPush=False)
self.task = "WBV2-P42-001"
self.parent = "tests/golden/WBV2-P42-000/manifest.json"
self.file("docs/EXECUTION_QUEUE.md", f"| 当前任务 | `{self.task}` |\n| parent manifest | `{self.parent}` |\n")
self.file(f"docs/tasks/{self.task}.md", "one bounded source workflow\n")
self.file("source.cpp", "int real_source;\n")
self.native_source = "blender-5.2.0/source/blender/web_engine/test_fixture.cpp"
self.file(self.native_source, "int supervisor_test_fixture;\n")
self.file("engine.wasm", "test fixture, not production WASM\n")
write_json(self.root / POLICY, self.policy)
write_json(self.root / self.parent, {"task": "WBV2-P42-000", "status": "done", "nextTask": self.task})
write_json(self.root / "tests/golden/WBV2/dynamic-inventory.json", {"units": [{"id": "operator:TEST"}], "coverage": {"unverifiedUnits": 1}})
write_json(self.root / "web/app/src/capabilities/blender-feature-route-catalog.json", {"entries": [{"id": "operator:TEST", "route": "WEB_MAIN"}]})
self.runner = Autopilot(self.root, self.policy)
self.context = queue_context(self.root)
self.job = {"id": "SOURCE-JOB", "ownedPaths": ["source.cpp"], "title": "test source", "queueTask": self.task}
def file(self, path, content):
p = self.root / path
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(content)
return p
def ready_result(self, run_dir):
kinds = ["positive", "negative", "undo_redo", "save_reopen", "determinism"]
cases = [{"id": kind, "kind": kind, "checkId": "native", "receipt": str((run_dir / "cases" / f"{kind}.json").relative_to(self.root)), "unitIds": ["operator:TEST"]} for kind in kinds]
result = {"status": "ready", "summary": "synthetic supervisor fixture", "changedFiles": ["source.cpp", "check.py"], "sourceUnits": ["operator:TEST"], "dependencies": ["source.cpp", "check.py"], "artifacts": ["engine.wasm"], "checks": [{"id": "native", "kinds": sorted(REQUIRED), "argv": ["python3", "check.py"], "cwd": "."}], "cases": cases}
result.update(workflow="Synthetic supervisor workflow", routeIds=["operator:TEST"], sourceReferences=[self.native_source])
self.file("check.py", """import hashlib, json, os
from pathlib import Path
root = Path.cwd()
counter = root / 'executions.txt'
counter.write_text(str(int(counter.read_text()) + 1) if counter.exists() else '1')
out = Path(os.environ['WBV2_RECEIPT_DIR'])
out.mkdir(parents=True, exist_ok=True)
a, b = 'a' * 64, 'b' * 64
observations = {'positive': {'before': a, 'after': b}, 'negative': {'before': a, 'after': a}, 'undo_redo': {'before': a, 'after': b, 'undo': a, 'redo': b}, 'save_reopen': {'saved': b, 'reopened': b}, 'determinism': {'first': b, 'second': b}}
for kind, values in observations.items():
receipt = {'status': 'PASS', 'runtime': 'CHROMIUM_WEBENGINE', 'caseId': kind, 'kind': kind, 'unitIds': ['operator:TEST'], 'observations': values, 'artifacts': [{'path': 'engine.wasm', 'sha256': hashlib.sha256((root / 'engine.wasm').read_bytes()).hexdigest()}]}
(out / (kind + '.json')).write_text(json.dumps(receipt))
""")
return result
def worker_fixture(self, prompt, run_dir):
return self.ready_result(run_dir)
def execute_fixture(self):
with patch.object(self.runner, "worker", side_effect=self.worker_fixture), contextlib.redirect_stdout(io.StringIO()):
return self.runner.attempt(self.context, self.job)
def test_parent_pointer_mismatch_is_rejected(self):
write_json(self.root / self.parent, {"status": "done", "nextTask": "WBV2-P99-001"})
with self.assertRaisesRegex(ValueError, "does not authorize"):
queue_context(self.root)
def test_route_complete_or_deferred_is_not_runtime_acceptance(self):
p = self.root / f"tests/golden/{self.task}/manifest.json"
write_json(p, {"task": self.task, "status": "done", "nextTask": None, "implementationComplete": True, "verificationDeferred": True})
self.assertFalse(manifest_proof(self.root, self.task)["ok"])
def test_terminal_manifest_checks_artifact_bytes(self):
p = self.root / f"tests/golden/{self.task}/manifest.json"
write_json(p, {"task": self.task, "status": "done", "acceptanceComplete": True, "nextTask": None, "artifacts": {"source": {"path": "source.cpp", "sha256": sha(self.root / "source.cpp")}}})
self.assertTrue(manifest_proof(self.root, self.task)["ok"])
self.file("source.cpp", "changed")
self.assertFalse(manifest_proof(self.root, self.task)["ok"])
def test_policy_cannot_disable_required_checks(self):
self.policy["requiredChecks"] = ["typecheck"]
with self.assertRaisesRegex(ValueError, "mandatory"):
Autopilot(self.root, self.policy)
def test_zero_summary_does_not_hide_unverified_units_or_drift(self):
p = self.root / "tests/golden/WBV2/dynamic-inventory.json"
write_json(p, {"units": [{"id": "operator:TEST", "coverageStatus": "UNVERIFIED"}], "coverage": {"unverifiedUnits": 0}})
self.assertFalse(inventory_proof(self.root)["ok"])
write_json(p, {"units": [{"id": "operator:TEST", "coverageStatus": "VERIFIED", "coverage": {}}], "coverage": {"unverifiedUnits": 0}})
self.assertEqual(1, inventory_proof(self.root)["unitsWithInvalidReferences"])
def test_lock_excludes_concurrent_owner_and_reuses_inode(self):
p = self.root / "owner.lock"
with Lease(p):
inode = p.stat().st_ino
with self.assertRaisesRegex(RuntimeError, "owns"):
with Lease(p):
pass
with Lease(p):
self.assertEqual(inode, p.stat().st_ino)
def test_command_rejects_shell_inline_code_other_browsers_and_escape(self):
for argv, cwd in [(["bash", "-c", "true"], "."), (["node", "-e", "process.exit(0)"], "."), (["npm", "run", "test", "--", "--project=webkit"], "."), (["git", "reset", "--hard"], "."), (["python3", "test.py"], "../outside")]:
with self.subTest(argv=argv), self.assertRaises(ValueError):
validate_command(self.root, argv, cwd)
def test_symlink_escape_is_rejected(self):
(self.root / "external").symlink_to("/tmp", target_is_directory=True)
with self.assertRaises(ValueError):
validate_command(self.root, ["npm", "run", "test"], "external")
def test_agent_ready_without_behavior_matrix_is_not_success(self):
with patch.object(self.runner, "worker", return_value={"status": "ready", "summary": "done", "sourceUnits": ["operator:TEST"], "checks": []}), contextlib.redirect_stdout(io.StringIO()):
self.assertFalse(self.runner.attempt(self.context, self.job))
self.assertEqual("retry_pending", self.runner.state["jobs"]["SOURCE-JOB"]["state"])
def test_fresh_runtime_receipts_and_hashes_are_required(self):
run_dir = self.runner.directory / "runs/test"
result = self.ready_result(run_dir)
self.file(result["cases"][0]["receipt"], "{}")
with self.assertRaisesRegex(ValueError, "fresh"):
self.runner.validate_result(result, run_dir)
def test_real_process_checks_create_receipts_and_cache_invalidates(self):
self.assertTrue(self.execute_fixture())
record = self.runner.state["jobs"]["SOURCE-JOB"]
self.assertTrue(self.runner.receipt_valid(record))
self.assertEqual("1", (self.root / "executions.txt").read_text())
self.file("source.cpp", "changed native source")
self.assertFalse(self.runner.receipt_valid(record))
def test_identical_checks_are_run_once(self):
def worker(prompt, run_dir):
result = self.ready_result(run_dir)
duplicate = copy.deepcopy(result["checks"][0])
duplicate["id"] = "same-command"
result["checks"].append(duplicate)
return result
with patch.object(self.runner, "worker", side_effect=worker), contextlib.redirect_stdout(io.StringIO()):
self.assertTrue(self.runner.attempt(self.context, self.job))
self.assertEqual("1", (self.root / "executions.txt").read_text())
def test_source_mutation_during_validation_is_not_accepted(self):
def worker(prompt, run_dir):
result = self.ready_result(run_dir)
with (self.root / "check.py").open("a") as stream:
stream.write("\n(root / 'source.cpp').write_text('drift')\n")
return result
with patch.object(self.runner, "worker", side_effect=worker), contextlib.redirect_stdout(io.StringIO()):
self.assertFalse(self.runner.attempt(self.context, self.job))
self.assertEqual("needs_reconcile", self.runner.state["state"])
def test_protected_queue_edit_requires_reconciliation(self):
def worker(prompt, run_dir):
self.file("docs/EXECUTION_QUEUE.md", "changed")
return self.ready_result(run_dir)
with patch.object(self.runner, "worker", side_effect=worker), contextlib.redirect_stdout(io.StringIO()):
self.assertFalse(self.runner.attempt(self.context, self.job))
self.assertEqual("needs_reconcile", self.runner.state["state"])
def test_failed_check_never_marks_job_verified_and_retries_are_bounded(self):
def worker(prompt, run_dir):
result = self.ready_result(run_dir)
self.file("check.py", "raise SystemExit(7)\n")
return result
with patch.object(self.runner, "worker", side_effect=worker), contextlib.redirect_stdout(io.StringIO()):
for _ in range(3):
self.assertFalse(self.runner.attempt(self.context, self.job))
self.assertEqual("blocked", self.runner.state["state"])
self.assertEqual(3, self.runner.state["jobs"]["SOURCE-JOB"]["attempts"])
def test_negative_case_with_mutation_fails(self):
self.assertTrue(self.execute_fixture())
record = self.runner.state["jobs"]["SOURCE-JOB"]
run_dir = self.root / record["runDirectory"]
result = self.ready_result(run_dir)
p = run_dir / "cases/negative.json"
receipt = json.loads(p.read_text())
receipt["observations"]["after"] = "b" * 64
write_json(p, receipt)
with self.assertRaisesRegex(ValueError, "behavior assertion"):
self.runner.verify_cases(result)
def test_worker_result_does_not_override_nonzero_process_exit(self):
run_dir = self.runner.directory / "runs/failed-worker"
run_dir.mkdir(parents=True)
def process(*args, **kwargs):
self.file(str((run_dir / "worker.jsonl").relative_to(self.root)), '{"type":"turn.completed","usage":{"input_tokens":10,"output_tokens":2}}\n')
write_json(run_dir / "worker-result.json", {"status": "ready"})
return {"exitCode": 9, "interrupted": False, "timedOut": False, "logLimit": False}
with patch.object(self.runner, "run_process", side_effect=process), self.assertRaisesRegex(ValueError, "exit=9"):
self.runner.worker("test", run_dir)
self.assertEqual(12, self.runner.run_tokens)
def test_unknown_previous_writer_is_never_replayed(self):
self.runner.state["state"] = "implementing"
with patch.object(self.runner, "worker") as worker:
self.assertEqual(3, self.runner.run(max_cycles=1))
worker.assert_not_called()
self.assertEqual("needs_reconcile", self.runner.state["state"])
def test_timeout_kills_descendants_before_late_publication(self):
child = self.file("late.py", "import time\nfrom pathlib import Path\ntime.sleep(1)\nPath('late-result').write_text('wrong')\n")
parent = self.file("hang.py", f"import subprocess, sys, time\nsubprocess.Popen([sys.executable, {str(child)!r}])\ntime.sleep(30)\n")
self.runner.checkpoint("validating")
with contextlib.redirect_stdout(io.StringIO()):
result = self.runner.run_process([sys.executable, str(parent)], self.root, self.runner.directory / "timeout.log", 0.1)
self.assertTrue(result["timedOut"])
time.sleep(1.1)
self.assertFalse((self.root / "late-result").exists())
def test_dirty_user_file_cannot_be_included_in_automatic_commit(self):
self.policy["autoCommit"] = True
write_json(self.runner.directory / "baseline.json", {"head": "none", "dirtyPaths": ["source.cpp"]})
with self.assertRaisesRegex(ValueError, "pre-existing edits"):
self.runner.publish(self.context, ["source.cpp"])
def test_commit_excludes_unrelated_staged_changes(self):
def git(*args):
return subprocess.check_output(["git", *args], cwd=self.root, stderr=subprocess.DEVNULL, text=True).strip()
git("init", "-b", "main")
git("config", "user.name", "Supervisor Test")
git("config", "user.email", "supervisor@example.invalid")
self.file("user.txt", "original")
git("add", ".")
git("commit", "-m", "fixture")
self.file("user.txt", "user staged edit")
git("add", "user.txt")
self.runner.remember_baseline()
self.file("source.cpp", "verified implementation")
self.policy["autoCommit"] = True
self.assertTrue(self.runner.publish(self.context, ["source.cpp"]))
self.assertEqual("source.cpp", git("diff-tree", "--no-commit-id", "--name-only", "-r", "HEAD"))
self.assertEqual("user.txt", git("diff", "--cached", "--name-only"))
self.assertEqual("user staged edit", (self.root / "user.txt").read_text())
def test_daemon_lease_handoff_keeps_single_owner(self):
lock = self.root / "daemon.lock"
module_dir = str(Path(__file__).parent)
with Lease(lock) as lease:
fd = lease.stream.fileno()
code = f"from pathlib import Path\nimport time\nfrom wbv2_autopilot import Lease\nwith Lease(Path({str(lock)!r}), {fd}):\n print('owned', flush=True)\n time.sleep(0.4)\n"
child = subprocess.Popen([sys.executable, "-c", code], stdout=subprocess.PIPE, text=True, pass_fds=(fd,), env={**os.environ, "PYTHONPATH": module_dir})
lease.transferred = True
try:
self.assertEqual("owned", child.stdout.readline().strip())
with self.assertRaises(RuntimeError):
with Lease(lock):
pass
self.assertEqual(0, child.wait(timeout=3))
finally:
if child.poll() is None:
child.kill()
child.wait()
child.stdout.close()
with Lease(lock):
pass
def test_legacy_preflight_is_read_only_and_rejects_incomplete_terminal(self):
repo = Path(__file__).resolve().parents[1]
for name in ("run-wbv2-scheduler.mjs", "wbv2-failure-ledger.mjs"):
self.file("tools/web/" + name, (repo / "tools/web" / name).read_text())
self.file("tools/web/check-task-context.mjs", "process.exit(0);\n")
write_json(self.root / "tests/golden/WBV2/task-index.json", {"activeTask": None, "entries": {self.task: {}}})
write_json(self.root / f"tests/golden/{self.task}/manifest.json", {"task": self.task, "status": "in_progress", "nextTask": None})
checkpoint = self.file("tests/golden/WBV2/continuous-scheduler.json", '{"state":"sentinel"}\n')
before = checkpoint.read_bytes()
result = subprocess.run(["node", "tools/web/run-wbv2-scheduler.mjs", "--preflight-only"], cwd=self.root, capture_output=True, text=True, timeout=10)
self.assertEqual(2, result.returncode, result.stderr)
self.assertIn("implementation_entrypoint_missing", result.stdout)
self.assertEqual(before, checkpoint.read_bytes())
self.assertFalse((self.root / "tests/golden/WBV2/continuous-scheduler.lock").exists())
def test_time_labels_are_beijing_minutes(self):
self.assertEqual("2026-09-08 10:10", minute_label("2026-09-08T02:10:59.999Z"))
self.assertEqual({"updatedAt": "2026-09-08 10:10", "durationSeconds": 0.04}, display_times({"updatedAt": "2026-09-08T02:10:59+00:00", "durationSeconds": 0.04}))
def test_quick_command_has_no_one_second_polling_floor(self):
script = self.file("quick.py", "import time\ntime.sleep(0.03)\n")
self.policy.update(heartbeatSeconds=5, monitorPollSeconds=0.25)
with patch.object(self.runner, "checkpoint", wraps=self.runner.checkpoint) as checkpoints, contextlib.redirect_stdout(io.StringIO()):
result = self.runner.run_process([sys.executable, str(script)], self.root, self.runner.directory / "quick.log", 5)
self.assertLess(result["durationSeconds"], 0.9)
self.assertEqual(2, checkpoints.call_count)
def model_script(self, source):
script = self.file("model_fixture.py", source)
self.policy.update(monitorPollSeconds=0.02, modelStartTimeoutSeconds=0.15, modelIdleTimeoutSeconds=0.15)
with contextlib.redirect_stdout(io.StringIO()):
return self.runner.run_process([sys.executable, str(script)], self.root, self.runner.directory / "model.log", 5, monitor_worker=True, phase="model_request")
def test_provider_error_before_actions_releases_process_early(self):
result = self.model_script("import time\nprint('{\"type\":\"turn.started\"}', flush=True)\nprint('{\"type\":\"error\",\"message\":\"Concurrency limit exceeded\"}', flush=True)\ntime.sleep(10)\n")
self.assertEqual("provider_unavailable", result["providerAbort"])
self.assertTrue(result["workerTrace"]["safeToRetry"])
self.assertLess(result["durationSeconds"], 1)
def test_model_start_without_progress_is_bounded(self):
result = self.model_script("import time\nprint('{\"type\":\"turn.started\"}', flush=True)\ntime.sleep(10)\n")
self.assertEqual("model_start_timeout", result["providerAbort"])
self.assertTrue(result["workerTrace"]["safeToRetry"])
def test_stalled_model_after_actions_requires_reconciliation(self):
result = self.model_script("import time\nprint('{\"type\":\"turn.started\"}', flush=True)\nprint('{\"type\":\"item.completed\",\"item\":{\"id\":\"edit\",\"type\":\"file_change\"}}', flush=True)\ntime.sleep(10)\n")
self.assertEqual("model_idle_timeout", result["providerAbort"])
self.assertFalse(result["workerTrace"]["safeToRetry"])
def test_active_tool_is_not_mistaken_for_model_idle(self):
result = self.model_script("import time\nprint('{\"type\":\"turn.started\"}', flush=True)\nprint('{\"type\":\"item.started\",\"item\":{\"id\":\"build\",\"type\":\"command_execution\",\"command\":\"cmake --build build\"}}', flush=True)\ntime.sleep(0.25)\nprint('{\"type\":\"item.completed\",\"item\":{\"id\":\"build\",\"type\":\"command_execution\",\"exit_code\":0}}', flush=True)\n")
self.assertIsNone(result["providerAbort"])
self.assertEqual(0, result["exitCode"])
self.assertEqual("build", result["workerTrace"]["items"][0]["phase"])
def test_provider_failures_have_separate_bounded_budget(self):
with patch.object(self.runner, "worker", side_effect=ProviderUnavailable("rate limit")), contextlib.redirect_stdout(io.StringIO()):
for failure in range(1, 5):
self.assertFalse(self.runner.attempt(self.context, self.job))
record = self.runner.state["jobs"][self.job["id"]]
self.assertEqual(0, record["attempts"])
self.assertEqual(failure, record["providerFailures"])
self.assertEqual("provider_blocked", self.runner.state["state"])
def test_provider_failure_with_source_changes_is_not_retried(self):
def changed(*args):
self.file("source.cpp", "unexpected change")
raise ProviderUnavailable("overload")
with patch.object(self.runner, "worker", side_effect=changed), contextlib.redirect_stdout(io.StringIO()):
self.assertFalse(self.runner.attempt(self.context, self.job))
self.assertEqual("needs_reconcile", self.runner.state["state"])
def test_jsonl_partial_reads_and_nested_tool_timing(self):
trace = WorkerTrace()
p = self.file("events.jsonl", '{"type":"turn.star')
trace.read(p, 0.1)
self.assertFalse(trace.safe_to_retry)
with p.open("a") as stream:
stream.write('ted"}\n')
trace.read(p, 0.2)
self.assertTrue(trace.safe_to_retry)
trace.feed({"type": "item.started", "item": {"id": "read", "type": "command_execution", "command": "rg pattern source.cpp"}}, 1)
trace.feed({"type": "item.completed", "item": {"id": "read", "type": "command_execution", "exit_code": 0}}, 1.4)
self.assertAlmostEqual(0.4, trace.items[0]["durationSeconds"])
self.assertEqual("reading", trace.items[0]["phase"])
event = {"event": "process.finished", "argv": ["codex", "exec"], "log": "model.log", "startedAt": "2026-09-08T02:10:00+00:00", "finishedAt": "2026-09-08T02:10:02+00:00", "durationSeconds": 2, "workerTrace": trace.snapshot(2)}
self.file(CONTROL + "/events.jsonl", json.dumps(event) + "\n")
report = timing_report(self.runner.directory)
self.assertAlmostEqual(2 / 60, report["totalMeasuredMinutes"], places=4)
self.assertEqual("2026-09-08 10:10", report["steps"][0]["start"])
def test_business_routes_and_native_source_are_mandatory(self):
run_dir = self.runner.directory / "runs/contract"
result = self.ready_result(run_dir)
result["routeIds"] = []
with self.assertRaisesRegex(ValueError, "business workflow"):
self.runner.validate_result(result, run_dir)
result["routeIds"] = ["operator:TEST"]
result["sourceReferences"] = ["source.cpp"]
with self.assertRaisesRegex(ValueError, "native Blender source"):
self.runner.validate_result(result, run_dir)
if __name__ == "__main__":
unittest.main()

946
my-tools/wbv2_autopilot.py Normal file
View File

@@ -0,0 +1,946 @@
#!/usr/bin/env python3
"""Persistent, independently verified Blender Web migration supervisor."""
from __future__ import annotations
import argparse
from contextlib import contextmanager
import fcntl
import hashlib
import json
import os
import re
import shutil
import signal
import subprocess
import sys
import time
import uuid
from datetime import datetime, timezone
from pathlib import Path
from wbv2_timing import WorkerTrace, command_phase, display_times, format_report, minute_label, timing_report
ROOT = Path(__file__).resolve().parents[1]
CONTROL = "tests/golden/WBV2/autopilot"
POLICY = "my-tools/wbv2_autopilot_policy.json"
REQUIRED = {"source_build", "typecheck", "lint", "chromium", "positive", "negative", "undo_redo", "save_reopen", "determinism"}
ACTIVE = {"implementing", "validating", "finalizing", "committing", "committed", "pushing", "pushed"}
class ProviderUnavailable(ValueError):
"""No agent action started before a bounded provider failure."""
def now():
return datetime.now(timezone.utc).isoformat()
def read_json(path, default=None):
try:
return json.loads(path.read_text(encoding="utf-8"))
except FileNotFoundError:
if default is not None:
return default
raise
def write_json(path, value):
path.parent.mkdir(parents=True, exist_ok=True)
temporary = path.with_name(path.name + f".tmp-{os.getpid()}")
with temporary.open("w", encoding="utf-8") as stream:
json.dump(value, stream, ensure_ascii=True, indent=2)
stream.write("\n")
stream.flush()
os.fsync(stream.fileno())
temporary.replace(path)
def sha(path):
if not path.is_file():
return None
with path.open("rb") as stream:
return hashlib.file_digest(stream, "sha256").hexdigest()
def contained(root, relative):
if not isinstance(relative, str) or not relative or Path(relative).is_absolute() or ".." in Path(relative).parts:
raise ValueError(f"invalid relative path: {relative!r}")
result = (root / relative).resolve()
if not result.is_relative_to(root.resolve()):
raise ValueError(f"path escapes repository: {relative}")
return result
def alive(pid):
if not isinstance(pid, int) or pid <= 0:
return False
try:
os.kill(pid, 0)
return True
except ProcessLookupError:
return False
except PermissionError:
return True
def identity(pid):
try:
return Path(f"/proc/{pid}/stat").read_text().rsplit(")", 1)[1].split()[19]
except (OSError, IndexError):
return None
class Lease:
def __init__(self, path, inherited=None):
self.path = path
self.inherited = inherited
self.transferred = False
def __enter__(self):
self.path.parent.mkdir(parents=True, exist_ok=True)
self.stream = os.fdopen(self.inherited, "a+") if self.inherited is not None else self.path.open("a+")
if os.fstat(self.stream.fileno()).st_ino != self.path.stat().st_ino:
self.stream.close()
raise RuntimeError("inherited lease does not match the owner lock")
try:
fcntl.flock(self.stream, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
self.stream.close()
raise RuntimeError("another Python autopilot owns the project") from None
self.stream.seek(0)
self.stream.truncate()
json.dump({"pid": os.getpid(), "identity": identity(os.getpid()), "startedAt": now()}, self.stream)
self.stream.flush()
return self
def __exit__(self, *_):
# Never unlink a flock inode while another process can hold it open.
if not self.transferred:
fcntl.flock(self.stream, fcntl.LOCK_UN)
self.stream.close()
class ProjectFence:
"""Share the existing scheduler's exclusive marker with Python runs."""
def __init__(self, root):
self.directory = root / "tests/golden/WBV2"
self.path = self.directory / "continuous-scheduler.lock"
def __enter__(self):
self.directory.mkdir(parents=True, exist_ok=True)
for name in ("continuous-scheduler.lock", "continuous-guardian.lock", "continuous-supervisor.lock", "web-product-migration.lock"):
owner = read_json(self.directory / name, {})
if alive(owner.get("pid")):
raise RuntimeError(f"another project runner is active: {name}")
try:
with self.path.open("x", encoding="utf-8") as stream:
json.dump({"pid": os.getpid(), "identity": identity(os.getpid()), "runtime": "python-autopilot", "startedAt": now()}, stream)
except FileExistsError:
raise RuntimeError("stale scheduler lock requires reconciliation before Python can claim it") from None
return self
def __exit__(self, *_):
owner = read_json(self.path, {})
if owner.get("pid") == os.getpid() and owner.get("runtime") == "python-autopilot":
self.path.unlink()
def queue_context(root):
queue = (root / "docs/EXECUTION_QUEUE.md").read_text(encoding="utf-8")
task = re.search(r"\| 当前任务 \| `([^`]+)` \|", queue)
parent = re.search(r"\| parent manifest \| `([^`]+)` \|", queue)
if not task or not parent or not re.fullmatch(r"WBV2-P\d+-\d+", task[1]):
raise ValueError("authoritative queue pointer is malformed")
manifest = read_json(contained(root, parent[1]))
if manifest.get("status") != "done" or manifest.get("nextTask") != task[1]:
raise ValueError("parent manifest does not authorize the current task")
card = f"docs/tasks/{task[1]}.md"
if not (root / card).is_file():
raise ValueError("current task card is missing")
return {"task": task[1], "parentManifest": parent[1], "card": card, "parentSha256": sha(root / parent[1])}
def manifest_proof(root, task):
manifest = read_json(contained(root, f"tests/golden/{task}/manifest.json"), {})
reasons = []
if manifest.get("task") != task or manifest.get("status") != "done":
reasons.append("task_not_done")
if manifest.get("verificationDeferred") or manifest.get("acceptanceComplete") is not True:
reasons.append("runtime_acceptance_not_complete")
artifacts = manifest.get("artifacts", {})
if not isinstance(artifacts, dict) or not artifacts:
reasons.append("artifacts_missing")
else:
for name, item in artifacts.items():
try:
if not item.get("sha256") or sha(contained(root, item["path"])) != item["sha256"]:
reasons.append(f"artifact_drift:{name}")
except (KeyError, ValueError, AttributeError):
reasons.append(f"artifact_invalid:{name}")
return {"ok": not reasons, "reasons": reasons, "nextTask": manifest.get("nextTask")}
def inventory_proof(root):
inventory = read_json(root / "tests/golden/WBV2/dynamic-inventory.json")
units = inventory.get("units", [])
unverified = sum(unit.get("coverageStatus") != "VERIFIED" for unit in units)
invalid, checked = 0, {}
for unit in units:
if unit.get("coverageStatus") != "VERIFIED":
continue
matches = True
for key in ("evidenceManifest", "productionEvidence"):
item = unit.get("coverage", {}).get(key, {})
try:
signature = (item["path"], item["sha256"])
if signature not in checked:
checked[signature] = bool(item["sha256"]) and sha(contained(root, item["path"])) == item["sha256"]
matches = matches and checked[signature]
except (KeyError, ValueError):
matches = False
invalid += not matches
return {"ok": bool(units) and unverified == 0 and invalid == 0, "unitCount": len(units), "recordedVerified": len(units) - unverified, "unverifiedUnits": unverified, "unitsWithInvalidReferences": invalid}
def result_schema():
strings = {"type": "array", "items": {"type": "string"}}
def obj(properties):
return {"type": "object", "additionalProperties": False, "required": list(properties), "properties": properties}
check = obj({"id": {"type": "string"}, "kinds": strings, "argv": strings, "cwd": {"type": "string"}})
case = obj({"id": {"type": "string"}, "kind": {"type": "string"}, "checkId": {"type": "string"}, "receipt": {"type": "string"}, "unitIds": strings})
return obj({"status": {"type": "string", "enum": ["ready", "blocked"]}, "summary": {"type": "string"}, "workflow": {"type": "string"}, "routeIds": strings, "sourceReferences": strings, "changedFiles": strings, "sourceUnits": strings, "dependencies": strings, "artifacts": strings, "checks": {"type": "array", "items": check}, "cases": {"type": "array", "items": case}})
def validate_command(root, argv, cwd):
if not argv or not all(isinstance(v, str) and v and "\0" not in v for v in argv):
raise ValueError("command must be an argv array")
directory = root if cwd == "." else contained(root, cwd)
if not directory.is_dir():
raise ValueError("command cwd does not exist")
if argv[0] not in {"node", "python3", "npm", "cmake", "ninja", "ctest", "git"}:
raise ValueError("unsupported validation executable")
if any(arg in {"-e", "--eval", "-c", "--command", "--input-type=module"} for arg in argv[1:]):
raise ValueError("inline code is not a validation entrypoint")
if any(re.search(r"firefox|webkit", arg, re.I) for arg in argv):
raise ValueError("only Chromium is permitted")
if argv[0] == "git" and argv != ["git", "diff", "--check"]:
raise ValueError("validation cannot change Git state")
if argv[0] in {"node", "python3"}:
scripts = [arg for arg in argv[1:] if arg.endswith((".mjs", ".js", ".py"))]
if not scripts or any(not (directory / p).resolve().is_relative_to(root) or not (directory / p).is_file() for p in scripts):
raise ValueError("a concrete repository validation script is required")
if argv[0] == "npm" and "run" not in argv:
raise ValueError("only repository npm run scripts are permitted")
return argv, directory
class Autopilot:
def __init__(self, root=ROOT, policy=None):
self.root = root.resolve()
self.directory = self.root / CONTROL
self.policy = policy or read_json(self.root / POLICY)
if self.policy.get("sourceFirst") is not True:
raise ValueError("source-first business workflow policy is mandatory")
if not REQUIRED.issubset(self.policy.get("requiredChecks", [])):
raise ValueError("policy cannot omit mandatory behavioral gates")
for key in ("workerTimeoutSeconds", "checkTimeoutSeconds", "heartbeatSeconds", "maxAttempts", "maxTokensPerRun", "maxLogBytes", "maxBuildJobs", "monitorPollSeconds", "modelStartTimeoutSeconds", "modelIdleTimeoutSeconds", "providerBackoffSeconds", "providerMaxBackoffSeconds", "maxProviderFailures"):
if not isinstance(self.policy.get(key), (float, int)) or self.policy[key] <= 0:
raise ValueError(f"invalid positive policy limit: {key}")
self.state_path = self.directory / "state.json"
self.state = read_json(self.state_path, {"schemaVersion": 1, "state": "idle", "jobs": {}, "tokensUsed": 0})
self.stop_requested = False
self.child = None
self.run_tokens = 0
def checkpoint(self, state, **values):
self.state.update(state=state, updatedAt=now(), pid=os.getpid(), identity=identity(os.getpid()), **values)
write_json(self.state_path, self.state)
write_json(self.root / "tests/golden/WBV2/web-product-migration-status.json", {"schemaVersion": 2, "pid": os.getpid(), "status": state, "updatedAt": now(), "activeTask": self.state.get("activeJob"), "checkpoint": str(self.state_path.relative_to(self.root)), "verifiedJobs": sum(j.get("state") == "verified" for j in self.state["jobs"].values()), "reason": self.state.get("reason")})
def event(self, kind, **values):
self.directory.mkdir(parents=True, exist_ok=True)
record = {"time": now(), "event": kind, **values}
line = json.dumps(record, ensure_ascii=True)
with (self.directory / "events.jsonl").open("a", encoding="utf-8") as stream:
stream.write(line + "\n")
print(json.dumps(display_times({**record, "time": minute_label(record["time"])}), ensure_ascii=True), flush=True)
@contextmanager
def timed(self, phase):
started, wall = time.monotonic(), now()
try:
yield
finally:
self.event("timing.span", phase=phase, startedAt=wall, finishedAt=now(), durationSeconds=round(time.monotonic() - started, 6))
def request_stop(self, *_):
self.stop_requested = True
if self.child and self.child.poll() is None:
try:
os.killpg(self.child.pid, signal.SIGTERM)
except ProcessLookupError:
pass
def run_process(self, argv, directory, log, timeout, stdin=None, env=None, monitor_worker=False, phase=None):
log.parent.mkdir(parents=True, exist_ok=True)
started = time.monotonic()
phase = phase or command_phase(argv)
result = {"argv": argv, "cwd": str(directory.relative_to(self.root)), "phase": phase, "startedAt": now(), "log": str(log.relative_to(self.root))}
trace = WorkerTrace() if monitor_worker else None
provider_abort = None
next_heartbeat = started + self.policy["heartbeatSeconds"]
last_provider_error = None
interrupted = timed_out = log_limit = False
self.event("process.started", **result)
with log.open("wb") as output:
self.child = subprocess.Popen(argv, cwd=directory, env={**os.environ, **(env or {})}, stdin=subprocess.PIPE if stdin is not None else subprocess.DEVNULL, stdout=output, stderr=subprocess.STDOUT, start_new_session=True)
self.checkpoint(self.state["state"], childPid=self.child.pid, childIdentity=identity(self.child.pid), currentProcess={**result, "status": "running", "elapsedSeconds": 0, "phase": "provider_wait" if trace else phase})
if stdin is not None:
try:
self.child.stdin.write(stdin.encode())
self.child.stdin.close()
except BrokenPipeError:
pass
while self.child.poll() is None:
elapsed = time.monotonic() - started
if trace:
trace.read(log, elapsed)
if trace.provider_error and not trace.actions_seen:
provider_abort = "provider_unavailable"
elif not trace.actions_seen and elapsed >= self.policy.get("modelStartTimeoutSeconds", 60):
provider_abort = "model_start_timeout"
elif trace.actions_seen and not trace.active_items and elapsed - trace.last_progress >= self.policy.get("modelIdleTimeoutSeconds", 180):
provider_abort = "model_idle_timeout"
interrupted = self.stop_requested or (self.directory / "pause.request").exists()
timed_out = elapsed > timeout
log_limit = log.stat().st_size > self.policy["maxLogBytes"]
if interrupted or timed_out or log_limit or provider_abort:
try:
os.killpg(self.child.pid, signal.SIGTERM)
self.child.wait(timeout=5)
except ProcessLookupError:
pass
except subprocess.TimeoutExpired:
os.killpg(self.child.pid, signal.SIGKILL)
self.child.wait()
break
changed_error = trace and trace.provider_error != last_provider_error
if time.monotonic() >= next_heartbeat or changed_error:
current = {**result, "status": "running", "elapsedSeconds": round(elapsed, 3)}
if trace:
current.update(phase="provider_wait" if not trace.actions_seen else "agent_work", providerError=trace.provider_error, firstActionSeconds=trace.first_action, idleSeconds=round(elapsed - trace.last_progress, 3), activeItems=list(trace.active_items.values()))
last_provider_error = trace.provider_error
self.checkpoint(self.state["state"], heartbeatAt=now(), currentProcess=current)
next_heartbeat = time.monotonic() + self.policy["heartbeatSeconds"]
wait_for = max(0.001, min(self.policy.get("monitorPollSeconds", 0.25), next_heartbeat - time.monotonic(), timeout - elapsed))
try:
self.child.wait(timeout=wait_for)
except subprocess.TimeoutExpired:
pass
code = self.child.wait()
interrupted = interrupted or self.stop_requested or (self.directory / "pause.request").exists()
if trace:
trace.read(log, time.monotonic() - started)
try:
os.killpg(self.child.pid, signal.SIGTERM)
time.sleep(0.1)
os.killpg(self.child.pid, signal.SIGKILL)
except ProcessLookupError:
pass
self.child = None
elapsed = time.monotonic() - started
result.update(exitCode=code, durationSeconds=round(elapsed, 6), interrupted=interrupted, timedOut=timed_out, logLimit=log_limit, providerAbort=provider_abort, logSha256=sha(log), finishedAt=now())
if trace:
result["workerTrace"] = trace.snapshot(elapsed)
self.state.pop("childPid", None)
self.state.pop("childIdentity", None)
write_json(log.with_suffix(log.suffix + ".process.json"), result)
self.event("process.finished", **result)
self.checkpoint(self.state["state"], currentProcess={"phase": phase, "startedAt": result["startedAt"], "finishedAt": result["finishedAt"], "status": "finished", "durationSeconds": result["durationSeconds"], "providerAbort": provider_abort})
return result
def protected_hashes(self, context):
files = ["docs/EXECUTION_QUEUE.md", "nextTask.md", context["parentManifest"], "tests/golden/WBV2/web-product-backlog.json", "tests/golden/WBV2/dynamic-inventory.json", "tests/golden/WBV2/task-index.json", "tests/golden/WBV2-P0-002/parity-map.json", "my-tools/wbv2_autopilot.py", POLICY, f"tests/golden/{context['task']}/manifest.json"]
inventory = read_json(self.root / "tests/golden/WBV2/dynamic-inventory.json", {})
for unit in inventory.get("units", []):
for key in ("evidenceManifest", "productionEvidence"):
artifact = unit.get("coverage", {}).get(key)
if artifact:
files.append(artifact["path"])
files.append("my-tools/wbv2_timing.py")
with self.timed("evidence_hashing"):
return {p: sha(self.root / p) for p in sorted(set(files))}
def jobs(self, context):
backlog = read_json(self.root / "tests/golden/WBV2/web-product-backlog.json", {})
jobs = [dict(j) for j in backlog.get("tasks", [])] if backlog.get("queueTask") == context["task"] else [{"id": context["task"], "title": "Implement the current source-backed module", "ownedPaths": [], "implementation": "Follow the current module card and its source/case contract."}]
priority = self.policy.get("priority", [])
jobs.sort(key=lambda j: priority.index(j["id"]) if j["id"] in priority else len(priority))
for job in jobs:
if not re.fullmatch(r"[A-Z0-9_-]+", job["id"]):
raise ValueError("unsafe job identifier")
job["queueTask"] = context["task"]
return jobs
def receipt_valid(self, record):
if record.get("state") != "verified" or not record.get("receipt"):
return False
try:
receipt = read_json(contained(self.root, record["receipt"]))
if sha(self.root / record["receipt"]) != record.get("receiptSha256"):
return False
return all(sha(contained(self.root, p)) == digest for p, digest in receipt["inputs"].items()) and all(sha(contained(self.root, a["path"])) == a["sha256"] for a in receipt["artifacts"])
except (OSError, KeyError, ValueError):
return False
def choose_job(self, jobs):
# Complete untouched workflows before revisiting shared-file changes.
for job in jobs:
if self.state["jobs"].get(job["id"], {}).get("state") != "verified":
return job
return next((job for job in jobs if not self.receipt_valid(self.state["jobs"][job["id"]])), None)
def worker_prompt(self, context, job, run_dir, previous):
receipt_dir = str((run_dir / "cases").relative_to(self.root))
return f"""Implement one bounded Blender Web workflow in {self.root}.
Queue: {json.dumps(context, ensure_ascii=True)}
Workflow inside this task: {json.dumps(job, ensure_ascii=True)}
Previous attempt: {json.dumps(previous, ensure_ascii=True)[:4000]}
Known findings: {json.dumps(self.policy.get('knownFindings', []), ensure_ascii=True)}
Follow AGENTS.md: read the queue, print-task-context, check-task-context, then only named owned paths and bounded source searches.
Drive this task by its business module, user workflow and concrete function routes, then the exact source/operator case matrix.
FIRST search the supplied Blender 5.2 sources and task-scoped related references to resolve behavior before editing the Web implementation.
Use node tools/web/search-wbv2-batch-source.mjs --unit <source-unit-id> --pattern <symbol> to read bounded native source locations.
Blender 5.2 source/Main is authoritative. Never simulate Blender in JS or count route classification, timing, builds or bookkeeping as completed behavior.
Preserve dirty files. Reconcile previous edits before changing code. Never replay an unknown generator or external write.
Implement production code and executable focused tests, adding only directly related helpers/tests.
Do not modify queue, parent manifests, nextTask.md, backlog, historical evidence, source inventory or autopilot control files.
Do not commit, push, SSH, deploy, install services, start agents or change the live website.
Do not run expensive validation yourself: the independent supervisor executes your checks after your process exits successfully.
Return ready only when implementation and runnable tests are prepared; ready is NOT completion.
Checks use argv arrays and repo-relative cwd, with no shell or inline code and only Chromium.
Required check kinds: {', '.join(sorted(REQUIRED))}. Reuse existing build/test entrypoints; one command may cover multiple kinds.
Tests must prove positive Main mutation or exact Main reader, negative no-mutation, undo/redo, save/reopen and determinism.
Declare cases for positive, negative, undo_redo, save_reopen and determinism, bound to checkId and exact source inventory unitIds.
Tests must write fresh case receipts under {receipt_dir}/ at validation time using WBV2_RECEIPT_DIR. Do NOT write these receipts now.
Case JSON: status=PASS, runtime=CHROMIUM_WEBENGINE or BLENDER_NATIVE, caseId, kind, unitIds, observations, artifacts=[{{path,sha256}}].
observations use SHA-256 of actual canonical native Main snapshots: positive before/after differ, or reader expected/actual equal with reader=true; negative before/after equal; undo_redo before/after/undo/redo prove reversal; save_reopen saved/reopened equal; determinism first/second equal. At least one case must run in Chromium.
Use actual native/Chromium receipts, screenshots, .blend or WASM artifacts. Test assertions must verify behavior, not static markers.
List source/protocol/test/fixture/build dependencies for hash invalidation and compiled WASM/native paths in artifacts.
Return workflow (the concrete user workflow), routeIds (exact current route catalog IDs) and sourceReferences (native Blender files actually used).
sourceUnits must be existing dynamic inventory IDs. Do not claim verified coverage; the supervisor checks and hashes results.
Return blocked with a concrete reason if prerequisites cannot be established.
"""
def worker(self, prompt, run_dir):
schema_path, output_path = run_dir / "result.schema.json", run_dir / "worker-result.json"
write_json(schema_path, result_schema())
command = ["codex", "exec", "--sandbox", self.policy["sandbox"], "-c", 'approval_policy="never"', "--ephemeral", "--cd", str(self.root), "--json", "--output-schema", str(schema_path), "--output-last-message", str(output_path), "-"]
proc = self.run_process(command, self.root, run_dir / "worker.jsonl", self.policy["workerTimeoutSeconds"], stdin=prompt, monitor_worker=True, phase="handoff_model" if self.state["state"] == "finalizing" else "model_request")
completed = False
for line in (run_dir / "worker.jsonl").read_text(encoding="utf-8", errors="replace").splitlines():
try:
event = json.loads(line)
except json.JSONDecodeError:
continue
if event.get("type") == "turn.completed":
completed = True
usage = event.get("usage", {})
tokens = int(usage.get("input_tokens", 0)) + int(usage.get("output_tokens", 0))
self.run_tokens += tokens
self.state["tokensUsed"] = self.state.get("tokensUsed", 0) + tokens
if proc["interrupted"] or proc["timedOut"] or proc["logLimit"]:
self.checkpoint("needs_reconcile", reason="worker interrupted; inspect edits and process receipt before resuming")
raise InterruptedError(self.state["reason"])
trace = proc.get("workerTrace", {})
if proc.get("providerAbort") or trace.get("providerError"):
if trace.get("safeToRetry"):
raise ProviderUnavailable(trace.get("providerError") or proc["providerAbort"])
if proc.get("providerAbort"):
raise InterruptedError("model stalled after actions or unrecognized output; inspect the bounded trace before retry")
if proc["exitCode"] != 0 or not completed:
raise ValueError(f"worker failed: exit={proc['exitCode']}, turnCompleted={completed}")
return read_json(output_path)
def validate_result(self, result, run_dir):
if result.get("status") != "ready":
raise ValueError(result.get("summary", "worker reported blocked"))
known = {u["id"] for u in read_json(self.root / "tests/golden/WBV2/dynamic-inventory.json")["units"]}
units = set(result.get("sourceUnits", []))
if not units or not units.issubset(known):
raise ValueError("missing or unknown source unit bindings")
catalog = read_json(self.root / "web/app/src/capabilities/blender-feature-route-catalog.json")
route_ids = set(result.get("routeIds", []))
if not result.get("workflow", "").strip() or not route_ids or not route_ids.issubset({entry["id"] for entry in catalog["entries"]}) or units != route_ids:
raise ValueError("a concrete business workflow and source-bound function routes are required")
references = result.get("sourceReferences", [])
if not references or any(not p.startswith("blender-5.2.0/source/blender/") or not contained(self.root, p).is_file() for p in references):
raise ValueError("native Blender source references are required before Web implementation")
kinds, ids = set(), {}
for check in result.get("checks", []):
if not re.fullmatch(r"[A-Za-z0-9_-]+", check["id"]) or check["id"] in ids:
raise ValueError("check IDs must be unique and path-safe")
ids[check["id"]] = check
kinds.update(check["kinds"])
validate_command(self.root, check["argv"], check["cwd"])
if not REQUIRED.issubset(kinds):
raise ValueError(f"missing validation kinds: {sorted(REQUIRED - kinds)}")
case_kinds, case_ids, paths, covered = set(), set(), set(), set()
positive_units = set()
for case in result.get("cases", []):
destination = contained(self.root, case["receipt"])
if not destination.is_relative_to(run_dir / "cases") or destination.exists():
raise ValueError("behavioral receipts must be fresh validation output in this attempt")
if case["checkId"] not in ids or case["kind"] not in ids[case["checkId"]]["kinds"] or case["id"] in case_ids or str(destination) in paths:
raise ValueError("invalid or duplicate case/check binding")
case_ids.add(case["id"])
paths.add(str(destination))
case_kinds.add(case["kind"])
if not case["unitIds"] or not set(case["unitIds"]).issubset(units):
raise ValueError("invalid source-unit case binding")
covered.update(case["unitIds"])
if case["kind"] == "positive":
positive_units.update(case["unitIds"])
if not {"positive", "negative", "undo_redo", "save_reopen", "determinism"}.issubset(case_kinds) or covered != units or positive_units != units:
raise ValueError("incomplete behavioral case matrix")
for p in result.get("changedFiles", []) + result.get("dependencies", []) + result.get("artifacts", []):
contained(self.root, p)
if not any(p.endswith((".wasm", "/blender", "/web_engine")) for p in result.get("artifacts", [])):
raise ValueError("a compiled Blender/WASM artifact is required")
def verify_cases(self, result):
artifacts = []
chromium = False
for case in result["cases"]:
p = contained(self.root, case["receipt"])
receipt = read_json(p)
if receipt.get("status") != "PASS" or receipt.get("runtime") not in {"CHROMIUM_WEBENGINE", "BLENDER_NATIVE"}:
raise ValueError(f"runtime case failed: {case['id']}")
if receipt.get("caseId") != case["id"] or receipt.get("kind") != case["kind"] or set(receipt.get("unitIds", [])) != set(case["unitIds"]):
raise ValueError("runtime receipt does not match the requested case")
chromium = chromium or receipt["runtime"] == "CHROMIUM_WEBENGINE"
observations = receipt.get("observations", {})
def hashes(*keys):
values = [observations.get(key, "") for key in keys]
if not all(isinstance(v, str) and re.fullmatch(r"[a-f0-9]{64}", v) for v in values):
raise ValueError(f"missing native state hashes: {case['id']}")
return values
kind = case["kind"]
if kind == "positive" and observations.get("reader"):
left, right = hashes("expected", "actual")
valid = left == right
elif kind in {"positive", "negative"}:
left, right = hashes("before", "after")
valid = (left != right) if kind == "positive" else (left == right)
elif kind == "undo_redo":
before, after, undo, redo = hashes("before", "after", "undo", "redo")
valid = before != after and before == undo and after == redo
elif kind == "save_reopen":
saved, reopened = hashes("saved", "reopened")
valid = saved == reopened
elif kind == "determinism":
first, second = hashes("first", "second")
valid = first == second
else:
valid = False
if not valid:
raise ValueError(f"native behavior assertion failed: {case['id']}")
if not receipt.get("artifacts"):
raise ValueError("runtime case has no production artifacts")
for artifact in receipt["artifacts"]:
if not artifact.get("sha256") or sha(contained(self.root, artifact["path"])) != artifact["sha256"]:
raise ValueError(f"runtime artifact hash drift: {case['id']}")
artifacts.append(artifact)
artifacts.append({"path": case["receipt"], "sha256": sha(p)})
if not chromium:
raise ValueError("no Chromium runtime case receipt")
return artifacts
def attempt(self, context, job):
previous = self.state["jobs"].get(job["id"], {})
count = int(previous.get("attempts", 0)) + 1
run_dir = self.directory / "runs" / f"{job['id']}-{uuid.uuid4().hex[:12]}"
run_dir.mkdir(parents=True)
record = {"state": "implementing", "attempts": count, "providerFailures": previous.get("providerFailures", 0), "runDirectory": str(run_dir.relative_to(self.root))}
self.state["jobs"][job["id"]] = record
protected = self.protected_hashes(context)
with self.timed("source_fingerprint"):
owned_before = {p: sha(contained(self.root, p)) for p in job.get("ownedPaths", [])}
write_json(run_dir / "before-worker.json", {"protected": protected, "owned": owned_before})
self.checkpoint("implementing", activeJob=job["id"], queue=context, reason=None)
try:
result = self.worker(self.worker_prompt(context, job, run_dir, previous), run_dir)
if self.protected_hashes(context) != protected:
raise InterruptedError("worker changed protected queue/evidence files")
self.validate_result(result, run_dir)
inputs = sorted(set(job.get("ownedPaths", []) + result["changedFiles"] + result["dependencies"] + result["sourceReferences"] + [context["card"], context["parentManifest"], POLICY]))
before = {p: sha(contained(self.root, p)) for p in inputs}
if any(value is None for value in before.values()):
raise ValueError("declared dependency does not exist")
record["state"] = "validating"
self.checkpoint("validating")
checks, cache = [], {}
env = {"WBV2_RECEIPT_DIR": str(run_dir / "cases"), "WBV2_DEPLOYED_ORIGIN": self.policy["origin"], "WEB_BUILD_JOBS": str(min(self.policy["maxBuildJobs"], 8)), "CMAKE_BUILD_PARALLEL_LEVEL": str(min(self.policy["maxBuildJobs"], 8))}
for check in result["checks"]:
argv, cwd = validate_command(self.root, check["argv"], check["cwd"])
key = json.dumps([argv, str(cwd)])
if key not in cache:
cache[key] = self.run_process(argv, cwd, run_dir / f"check-{check['id']}.log", self.policy["checkTimeoutSeconds"], env=env, phase="validation:" + "+".join(sorted(check["kinds"])))
else:
self.event("check.reused", checkId=check["id"], originalLog=cache[key]["log"], avoidedSeconds=cache[key]["durationSeconds"])
receipt = {**cache[key], "id": check["id"], "kinds": check["kinds"]}
checks.append(receipt)
if receipt["exitCode"] != 0 or receipt["interrupted"] or receipt["timedOut"] or receipt["logLimit"]:
raise ValueError(f"check {check['id']} failed: exit={receipt['exitCode']}, timeout={receipt['timedOut']}")
if self.protected_hashes(context) != protected or any(sha(self.root / p) != digest for p, digest in before.items()):
raise InterruptedError("source/protected evidence changed during validation")
artifacts = self.verify_cases(result)
for p in result["artifacts"]:
digest = sha(contained(self.root, p))
if digest is None:
raise ValueError(f"compiled artifact missing: {p}")
artifacts.append({"path": p, "sha256": digest})
for check in checks:
artifacts.append({"path": check["log"], "sha256": check["logSha256"]})
receipt_path = run_dir / "verified-receipt.json"
write_json(receipt_path, {"schemaVersion": 1, "queueTask": context["task"], "job": job["id"], "workflow": result["workflow"], "routeIds": result["routeIds"], "sourceReferences": result["sourceReferences"], "status": "verified", "verifiedAt": now(), "sourceUnits": result["sourceUnits"], "inputs": before, "checks": checks, "artifacts": artifacts, "coverageIncrementClaimed": 0, "summary": result["summary"]})
record.update(state="verified", receipt=str(receipt_path.relative_to(self.root)), receiptSha256=sha(receipt_path), changedFiles=result["changedFiles"], attempts=0, providerFailures=0)
self.checkpoint("ready", activeJob=None)
self.event("job.verified", job=job["id"], receipt=record["receipt"])
return True
except ProviderUnavailable as error:
if self.protected_hashes(context) != protected or any(sha(self.root / p) != value for p, value in owned_before.items()):
record.update(state="needs_reconcile", reason="provider failure coincided with source/evidence changes")
self.checkpoint("needs_reconcile", reason=record["reason"])
return False
failures = int(previous.get("providerFailures", 0)) + 1
delay = min(self.policy["providerMaxBackoffSeconds"], self.policy["providerBackoffSeconds"] * 2 ** min(failures - 1, 20))
retry_at = datetime.fromtimestamp(time.time() + delay, timezone.utc).isoformat()
state = "provider_blocked" if failures >= self.policy["maxProviderFailures"] else "waiting_provider"
record.update(state=state, attempts=int(previous.get("attempts", 0)), providerFailures=failures, reason=str(error), nextRetryAt=retry_at)
self.checkpoint(state, reason=str(error), nextRetryAt=retry_at, providerFailures=failures)
self.event("provider.unavailable", job=job["id"], failures=failures, nextRetryAt=retry_at, reason=str(error), noAgentActions=True)
return False
except InterruptedError as error:
record.update(state="needs_reconcile", reason=str(error))
self.checkpoint("needs_reconcile", reason=str(error))
return False
except (ValueError, OSError, KeyError, TypeError) as error:
record.update(state="blocked" if count >= self.policy["maxAttempts"] else "retry_pending", reason=str(error))
self.checkpoint("paused" if self.stop_requested else record["state"], reason=str(error))
self.event("job.failed", job=job["id"], attempts=count, reason=str(error))
return False
def git(self, args):
log = self.directory / "git" / f"{uuid.uuid4().hex}.log"
result = self.run_process(["git", *args], self.root, log, 120)
if result["interrupted"] or result["timedOut"] or result["logLimit"]:
raise InterruptedError("Git command interrupted; reconcile its outcome before continuing")
output = log.read_text(encoding="utf-8", errors="surrogateescape")
if result["exitCode"] != 0:
raise subprocess.CalledProcessError(result["exitCode"], ["git", *args], output=output)
return output
def remember_baseline(self, refresh=False):
location = self.directory / "baseline.json"
if location.exists() and not refresh:
return
dirty = set(self.git(["diff", "--name-only", "-z", "HEAD"]).split("\0"))
dirty.update(self.git(["ls-files", "--others", "--exclude-standard", "-z"]).split("\0"))
write_json(location, {"head": self.git(["rev-parse", "HEAD"]).strip(), "dirtyPaths": sorted(dirty - {""}), "recordedAt": now()})
def publish(self, context, changed_files):
if not self.policy.get("autoCommit"):
self.checkpoint("ready_to_publish", reason="automatic commit disabled")
return False
baseline = read_json(self.directory / "baseline.json")
paths = sorted(set(changed_files))
if not paths:
raise ValueError("handoff has no declared files to commit")
for p in paths:
contained(self.root, p)
if p == ".git" or p.startswith(".git/"):
raise ValueError("cannot publish Git internals")
dirty_overlap = set(paths) & set(baseline["dirtyPaths"])
if dirty_overlap:
raise ValueError(f"publication would include pre-existing edits: {sorted(dirty_overlap)[:12]}")
if self.git(["branch", "--show-current"]).strip() != "main":
raise ValueError("automatic publication requires branch main")
if self.git(["rev-parse", "HEAD"]).strip() != baseline["head"]:
raise ValueError("HEAD changed since the supervisor baseline")
self.checkpoint("committing", publishPaths=paths)
self.git(["add", "-f", "--", *paths])
# --only excludes unrelated staged changes from this commit.
self.git(["commit", "--only", "-m", f"feat(web): verified workflow {context['task']}", "--", *paths])
head = self.git(["rev-parse", "HEAD"]).strip()
self.checkpoint("committed", commit=head)
if self.policy.get("autoPush"):
self.checkpoint("pushing")
pushed = False
for attempt in range(self.policy.get("pushRetries", 6) + 1):
result = self.run_process(["git", "push", "origin", "HEAD:refs/heads/main"], self.root, self.directory / f"push-{attempt}.log", 120)
try:
remote = self.git(["ls-remote", "--refs", "origin", "refs/heads/main"]).split()[0]
except (subprocess.CalledProcessError, IndexError):
remote = None
if remote == head:
pushed = True
self.checkpoint("pushed", remoteHead=remote)
break
if result["interrupted"] or self.stop_requested:
break
deadline = time.monotonic() + min(30, 2 ** attempt)
while time.monotonic() < deadline and not self.stop_requested:
time.sleep(0.2)
if not pushed:
raise InterruptedError("push outcome not confirmed; reconcile remote HEAD before continuing")
baseline["head"] = head
write_json(self.directory / "baseline.json", baseline)
return True
def finalize(self, context):
run_dir = self.directory / "runs" / f"handoff-{uuid.uuid4().hex[:12]}"
run_dir.mkdir(parents=True)
records = self.state["jobs"]
if not records or not all(self.receipt_valid(r) for r in records.values()):
raise ValueError("cannot finalize stale workflow receipts")
references = [{"path": r["receipt"], "sha256": r["receiptSha256"]} for r in records.values()]
self.checkpoint("finalizing")
prompt = f"""Finalize only the already verified Blender Web task {context['task']}.
Read the minimal queue/context/card first. Current pointer: {json.dumps(context)}.
Independent runtime receipts: {json.dumps(references)}.
Use repository generators to write aggregate report, status, manifest, task-context and publish the successor pointer.
Bind these verified receipts and their source/unit/build/test hashes. Set acceptanceComplete=true only if the module contract is fully met.
No source changes, tests, historical inventory/provenance rewrites, blind hash refresh, commit, push, SSH or production deployment.
Do not rewrite the parent manifest. Preserve dirty files. Never use deferred verification or route counts as completion.
If source inventory remains unverified and the current module is complete, use the repository generator to select the next source-backed module; do not invent task IDs or declare full parity.
Return structured status=ready and exact changedFiles; other array fields may be empty for this handoff-only turn.
If a required generator/evidence is unavailable, return blocked with the exact cause. Do not advance the queue on failure.
"""
result = self.worker(prompt, run_dir)
if result.get("status") != "ready":
raise ValueError(result.get("summary", "handoff worker blocked"))
if sha(self.root / context["parentManifest"]) != context["parentSha256"]:
raise InterruptedError("handoff changed the parent manifest")
for record in records.values():
receipt = read_json(self.root / record["receipt"])
for p, digest in receipt["inputs"].items():
if p == context["card"]:
continue
if sha(self.root / p) != digest:
raise InterruptedError("handoff changed a verified implementation dependency")
for index, argv in enumerate(self.policy["handoffChecks"]):
process = self.run_process(argv, self.root, run_dir / f"gate-{index}.log", self.policy["checkTimeoutSeconds"])
if process["exitCode"] != 0 or process["interrupted"] or process["timedOut"] or process["logLimit"]:
raise ValueError(f"handoff gate failed: {argv}")
proof = manifest_proof(self.root, context["task"])
if not proof["ok"]:
raise ValueError(f"aggregate acceptance missing: {proof['reasons']}")
after = queue_context(self.root)
if proof["nextTask"] is not None and after["task"] != proof["nextTask"]:
raise ValueError("generator did not publish the authorized successor")
if proof["nextTask"] is None and after["task"] != context["task"]:
raise ValueError("terminal handoff changed the queue unexpectedly")
changed = list(result["changedFiles"])
for record in records.values():
changed.extend(record.get("changedFiles", []))
receipt = read_json(self.root / record["receipt"])
changed.append(record["receipt"])
changed.extend(a["path"] for a in receipt["artifacts"])
if not self.publish(context, changed):
return False
self.checkpoint("handed_off", before=context, after=after)
return True
def doctor(self):
context = queue_context(self.root)
jobs = self.jobs(context)
return {"schemaVersion": 1, "queue": context, "taskCompletion": manifest_proof(self.root, context["task"]), "inventory": inventory_proof(self.root), "jobCount": len(jobs), "nextJob": (self.choose_job(jobs) or {}).get("id"), "tools": {n: bool(shutil.which(n)) for n in ("python3", "node", "git", "codex", "cmake")}, "policy": POLICY, "deployment": "implementation workers do not alter the live server", "state": self.state.get("state")}
def wait_for_provider(self, record):
retry_at = record.get("nextRetryAt")
if not retry_at:
return
remaining = max(0, datetime.fromisoformat(retry_at).timestamp() - time.time())
if remaining == 0:
return
self.checkpoint("waiting_provider", nextRetryAt=retry_at, reason=record.get("reason"))
deadline, heartbeat = time.monotonic() + remaining, time.monotonic()
with self.timed("provider_backoff"):
while time.monotonic() < deadline and not self.stop_requested and not (self.directory / "pause.request").exists():
if time.monotonic() >= heartbeat:
self.checkpoint("waiting_provider", heartbeatAt=now())
heartbeat = time.monotonic() + self.policy["heartbeatSeconds"]
time.sleep(min(0.25, max(0, deadline - time.monotonic())))
def run(self, max_cycles=0):
inherited = os.environ.pop("WBV2_AUTOPILOT_LEASE_FD", None)
with Lease(self.directory / "owner.lock", int(inherited) if inherited else None), ProjectFence(self.root):
if self.state.get("state") in ACTIVE | {"needs_reconcile"}:
self.checkpoint("needs_reconcile", reason="previous writer has no completed handoff; run read-only reconcile")
return 3
if self.state.get("state") in {"blocked", "provider_blocked"}:
return 2
if (self.directory / "pause.request").exists():
self.checkpoint("paused")
return 0
signal.signal(signal.SIGINT, self.request_stop)
signal.signal(signal.SIGTERM, self.request_stop)
self.remember_baseline()
cycles = 0
last_success = True
self.checkpoint("preflight", reason=None)
try:
while not self.stop_requested and (max_cycles == 0 or cycles < max_cycles):
if self.run_tokens >= self.policy["maxTokensPerRun"]:
self.checkpoint("budget_exhausted", reason="per-run token budget reached")
return 4
if (self.directory / "pause.request").exists():
self.checkpoint("paused")
return 0
context = queue_context(self.root)
prior = self.state.get("queue")
if prior and prior["task"] != context["task"]:
proof = manifest_proof(self.root, prior["task"])
if not proof["ok"] or proof["nextTask"] != context["task"]:
raise ValueError("queue advanced without an accepted parent handoff")
self.state["jobs"] = {}
self.checkpoint("preflight", queue=context)
for index, argv in enumerate(self.policy["preflight"]):
proc = self.run_process(argv, self.root, self.directory / f"preflight-{index}.log", 120)
if proc["exitCode"] != 0 or proc["interrupted"] or proc["timedOut"] or proc["logLimit"]:
raise ValueError(f"context preflight failed: {argv}")
job = self.choose_job(self.jobs(context))
if job is None:
if not self.finalize(context):
return 2
proof = manifest_proof(self.root, context["task"])
if proof["nextTask"] is None:
inventory = inventory_proof(self.root)
if not inventory["ok"]:
self.checkpoint("planning_required", reason="no successor but source inventory remains unverified; parity not complete")
return 2
self.checkpoint("complete", reason="accepted terminal manifest and no unverified source units")
return 0
continue
record = self.state["jobs"].get(job["id"], {})
if record.get("state") in {"blocked", "provider_blocked"}:
self.checkpoint(record["state"], activeJob=job["id"], reason=record.get("reason"))
return 2
if record.get("state") == "waiting_provider":
self.wait_for_provider(record)
if self.stop_requested or (self.directory / "pause.request").exists():
self.checkpoint("paused")
return 2
success = self.attempt(context, job)
last_success = success
cycles += 1
if self.state["state"] in {"needs_reconcile", "blocked", "provider_blocked", "paused"}:
return 3 if self.state["state"] == "needs_reconcile" else 2
if not success and self.state["state"] != "waiting_provider":
with self.timed("retry_backoff"):
deadline = time.monotonic() + self.policy.get("retryDelaySeconds", 10)
while time.monotonic() < deadline and not self.stop_requested:
time.sleep(0.2)
final_state = "paused" if self.stop_requested else "idle" if last_success else self.state["state"]
self.checkpoint(final_state, reason="stop requested" if self.stop_requested else self.state.get("reason") or "cycle limit reached")
return 0 if last_success and not self.stop_requested else 2
except InterruptedError as error:
self.checkpoint("needs_reconcile", reason=str(error))
return 3
except (ValueError, OSError, KeyError, RuntimeError, subprocess.CalledProcessError) as error:
self.checkpoint("blocked", reason=str(error))
self.event("supervisor.blocked", reason=str(error))
return 2
def main(argv=None):
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("action", choices=["run", "start", "status", "timings", "doctor", "pause", "stop", "resume", "reconcile", "service"], nargs="?", default="run")
parser.add_argument("--once", action="store_true")
parser.add_argument("--max-cycles", type=int, default=0)
parser.add_argument("--retry-blocked", action="store_true")
parser.add_argument("--refresh-baseline", action="store_true", help="adopt a reviewed maintenance commit before any workflow edits/receipts")
parser.add_argument("--json", action="store_true", help="JSON timing report")
parser.add_argument("--write", action="store_true", help="write timing report snapshots")
args = parser.parse_args(argv)
if args.max_cycles < 0:
parser.error("--max-cycles must be non-negative")
runner = Autopilot()
directory = runner.directory
if args.action == "doctor":
print(json.dumps(runner.doctor(), ensure_ascii=False, indent=2))
return 0
if args.action == "status":
state = dict(runner.state)
state["alive"] = alive(state.get("pid")) and identity(state["pid"]) == state.get("identity")
state["timezone"] = "Asia/Shanghai"
print(json.dumps(display_times(state), ensure_ascii=False, indent=2))
return 0
if args.action == "timings":
report = timing_report(directory, runner.state)
if args.write:
write_json(directory / "timing-report.json", report)
(directory / "timing-report.md").write_text(format_report(report) + "\n", encoding="utf-8")
print(json.dumps(report, ensure_ascii=False, indent=2) if args.json else format_report(report))
return 0
if args.action == "service":
def quote(value):
if "\n" in value or "\r" in value:
raise ValueError("newline in systemd value")
return '"' + value.replace("\\", "\\\\").replace('"', '\\"').replace("%", "%%") + '"'
unit = Path.home() / ".config/systemd/user/blender-web-autopilot.service"
unit.parent.mkdir(parents=True, exist_ok=True)
executable = Path(__file__).with_name("run_web_product_migration.py")
unit.write_text("[Unit]\nDescription=Blender Web source migration supervisor\nAfter=network-online.target\n\n[Service]\nType=simple\n" + f"WorkingDirectory={quote(str(ROOT))}\nExecStart={quote(sys.executable)} {quote(str(executable))} run\nEnvironment={quote('PATH=' + os.environ.get('PATH', '/usr/bin:/bin'))}\n" + "Restart=on-failure\nRestartPreventExitStatus=2 3 4\nRestartSec=10\nKillMode=control-group\nTimeoutStopSec=15\n\n[Install]\nWantedBy=default.target\n", encoding="utf-8")
print(f"Created {unit}; enable with systemctl --user daemon-reload and systemctl --user enable --now blender-web-autopilot")
return 0
if args.action in {"pause", "stop"}:
directory.mkdir(parents=True, exist_ok=True)
write_json(directory / "pause.request", {"requestedAt": now()})
owner = read_json(directory / "owner.lock", {})
if alive(owner.get("pid")) and identity(owner["pid"]) == owner.get("identity"):
os.kill(owner["pid"], signal.SIGTERM)
deadline = time.monotonic() + 10
while alive(owner["pid"]) and identity(owner["pid"]) == owner.get("identity") and time.monotonic() < deadline:
time.sleep(0.1)
print("Pause requested; interrupted work requires reconciliation before retry.")
return 0
if args.action == "reconcile":
print(json.dumps({"queue": queue_context(ROOT), "state": runner.state, "childAlive": alive(runner.state.get("childPid")), "receipts": {key: runner.receipt_valid(record) for key, record in runner.state["jobs"].items()}, "decision": "read-only; inspect interrupted edits before resume --retry-blocked"}, ensure_ascii=False, indent=2))
return 0
if args.action == "resume":
with Lease(directory / "owner.lock"):
if alive(runner.state.get("childPid")) and identity(runner.state["childPid"]) == runner.state.get("childIdentity"):
parser.error("previous child is alive; stop and reconcile first")
if runner.state["state"] in ACTIVE | {"needs_reconcile", "blocked", "provider_blocked"} and not args.retry_blocked:
parser.error("run reconcile; --retry-blocked acknowledges the inspected repository state")
for record in runner.state["jobs"].values():
if record.get("state") in ACTIVE | {"needs_reconcile", "blocked", "provider_blocked"} or (args.retry_blocked and record.get("state") != "verified"):
record.update(state="retry_pending", attempts=0, providerFailures=0)
record.pop("nextRetryAt", None)
if args.refresh_baseline and any(record.get("receipt") or record.get("changedFiles") for record in runner.state["jobs"].values()):
parser.error("cannot refresh a baseline containing workflow edits or verified receipts")
(directory / "pause.request").unlink(missing_ok=True)
if args.refresh_baseline:
runner.remember_baseline(refresh=True)
runner.checkpoint("idle", reason="explicit resume")
if args.action in {"start", "resume"}:
with Lease(directory / "owner.lock") as lease:
if (directory / "pause.request").exists():
parser.error("paused; use resume")
if runner.state["state"] in ACTIVE | {"needs_reconcile", "blocked", "provider_blocked"}:
parser.error("run reconcile before restarting interrupted or blocked work")
command = [sys.executable, str(Path(__file__).with_name("run_web_product_migration.py")), "run", "--max-cycles", str(1 if args.once else args.max_cycles)]
fd = lease.stream.fileno()
with (directory / "daemon.log").open("ab") as log:
child = subprocess.Popen(command, cwd=ROOT, stdin=subprocess.DEVNULL, stdout=log, stderr=subprocess.STDOUT, start_new_session=True, pass_fds=(fd,), env={**os.environ, "WBV2_AUTOPILOT_LEASE_FD": str(fd)})
lease.transferred = True
print(json.dumps({"launchedPid": child.pid, "statusCommand": "python3 my-tools/run_web_product_migration.py status", "log": str(directory / "daemon.log")}))
return 0
return runner.run(1 if args.once else args.max_cycles)
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -0,0 +1,42 @@
{
"schemaVersion": 1,
"strategy": "business_module_and_function_route",
"sourceAuthority": "blender-5.2.0/source/blender",
"sourceFirst": true,
"origin": "https://49.232.250.35:58014",
"sandbox": "workspace-write",
"workerTimeoutSeconds": 1800,
"checkTimeoutSeconds": 1800,
"heartbeatSeconds": 5,
"monitorPollSeconds": 0.25,
"modelStartTimeoutSeconds": 60,
"modelIdleTimeoutSeconds": 180,
"providerBackoffSeconds": 30,
"providerMaxBackoffSeconds": 240,
"maxProviderFailures": 4,
"maxAttempts": 3,
"maxTokensPerRun": 500000,
"maxLogBytes": 33554432,
"maxBuildJobs": 8,
"retryDelaySeconds": 10,
"autoCommit": true,
"autoPush": true,
"pushRetries": 6,
"priority": ["SERVER-OPERATOR-HTTP-PROVIDER", "SERVER-OPERATOR-SCHEMA-PALETTE", "BROWSER-EQUIVALENT-RUNTIME"],
"requiredChecks": ["source_build", "typecheck", "lint", "chromium", "positive", "negative", "undo_redo", "save_reopen", "determinism"],
"preflight": [["node", "tools/web/check-task-context.mjs"], ["node", "tools/web/check-context-governance.mjs"]],
"handoffChecks": [
["node", "tools/web/check-task-context.mjs"],
["node", "tools/web/check-context-governance.mjs"],
["node", "tools/web/check-source-dynamic-inventory.mjs"],
["node", "tools/web/check-execution-control-plane.mjs"],
["node", "tools/web/check-task-index.mjs"],
["node", "tools/web/check-blender-next-task-plan.mjs"],
["git", "diff", "--check"]
],
"knownFindings": [
"The live /api/server-operator-job POST describe returns HTTP 405. Implement deployment integration and test a disposable local deployment; do not alter the live server during implementation.",
"SCREEN_OT_area_split switches Modeling to Layout without adding an editor. Implement the actual source-equivalent behavior, not a generic action-group alias.",
"Of 1025 recorded VERIFIED units, 902 have drifted direct evidence references. Reverify behavior and create current receipts; never rewrite historical hashes to silence a gate."
]
}

211
my-tools/wbv2_timing.py Normal file
View File

@@ -0,0 +1,211 @@
"""Incremental process telemetry and minute-labelled execution reports."""
from __future__ import annotations
import json
import re
import shlex
from collections import defaultdict
from datetime import datetime, timezone
from zoneinfo import ZoneInfo
LOCAL_ZONE = ZoneInfo("Asia/Shanghai")
def minute_label(value):
if not value:
return None
try:
return datetime.fromisoformat(value.replace("Z", "+00:00")).astimezone(LOCAL_ZONE).strftime("%Y-%m-%d %H:%M")
except (ValueError, TypeError):
return value
def display_times(value):
if isinstance(value, list):
return [display_times(item) for item in value]
if not isinstance(value, dict):
return value
return {key: minute_label(item) if key.endswith("At") and isinstance(item, str) else display_times(item) for key, item in value.items()}
def command_phase(argv):
if not argv:
return "other"
if argv[0] == "codex":
return "model_request"
if argv[0] == "git":
return "git_" + (argv[1] if len(argv) > 1 else "other")
if any("task-context" in arg or "context-governance" in arg for arg in argv):
return "context_check"
if argv[0] in {"cmake", "ninja"}:
return "build"
return "command"
def item_phase(item):
kind = item.get("type", "unknown")
if kind == "file_change":
return "editing"
if kind != "command_execution":
return "tool" if kind != "agent_message" else "model_response"
try:
argv = shlex.split(item.get("command", ""))
if len(argv) >= 3 and argv[0].endswith(("bash", "sh")) and argv[1] in {"-lc", "-c"}:
argv = shlex.split(argv[2])
except ValueError:
return "command"
if not argv:
return "command"
if argv[0] in {"cat", "sed", "rg", "ls", "head", "tail", "wc", "find", "stat"} or any("search-wbv2" in arg or "print-task-context" in arg for arg in argv):
return "reading"
if "apply_patch" in argv[0]:
return "editing"
if any("test" in arg or "lint" in arg or "typecheck" in arg for arg in argv):
return "testing"
return command_phase(argv)
class WorkerTrace:
"""Times observed JSONL events; never infers model CPU time from silence."""
def __init__(self):
self.offset = 0
self.pending = b""
self.turn_started = False
self.actions_seen = False
self.unknown_output = False
self.first_event = None
self.first_action = None
self.last_progress = 0.0
self.last_output = 0.0
self.provider_error = None
self.errors = 0
self.active_items = {}
self.items = []
self.counts = defaultdict(int)
def feed(self, event, elapsed):
kind = event.get("type", "unknown")
if not isinstance(kind, str):
self.unknown_output = True
return
self.counts[kind] += 1
self.first_event = elapsed if self.first_event is None else self.first_event
if kind == "turn.started":
self.turn_started = True
if kind in {"error", "turn.failed"}:
self.errors += 1
error = event.get("error")
message = event.get("message") or (error.get("message", "") if isinstance(error, dict) else error) or ""
if not isinstance(message, str):
message = str(message)
if re.search(r"concurrency limit|overload|rate.?limit|too many requests|\b429\b|\b503\b|service unavailable", message, re.I):
self.provider_error = re.sub(r"sk-[A-Za-z0-9_-]+", "[redacted]", message)[:600]
if kind.startswith("item."):
self.actions_seen = True
self.first_action = elapsed if self.first_action is None else self.first_action
self.last_progress = elapsed
item = event.get("item") or {}
if not isinstance(item, dict):
self.unknown_output = True
return
key = item.get("id")
if kind == "item.started" and key:
self.active_items[key] = {"id": key, "phase": item_phase(item), "startedSeconds": elapsed}
elif kind == "item.completed":
record = self.active_items.pop(key, {"id": key, "phase": item_phase(item), "startedSeconds": elapsed})
record.update(finishedSeconds=elapsed, durationSeconds=max(0, elapsed - record["startedSeconds"]), status=item.get("status"), exitCode=item.get("exit_code"))
self.items.append(record)
if kind == "turn.completed":
self.last_progress = elapsed
def read(self, path, elapsed):
with path.open("rb") as stream:
stream.seek(self.offset)
chunk = stream.read()
self.offset = stream.tell()
if not chunk:
return
self.last_output = elapsed
lines = (self.pending + chunk).split(b"\n")
self.pending = lines.pop()
for line in lines:
if not line.strip():
continue
try:
event = json.loads(line)
if not isinstance(event, dict):
raise ValueError("non-object event")
self.feed(event, elapsed)
except (ValueError, UnicodeDecodeError):
self.unknown_output = True
@property
def safe_to_retry(self):
return self.turn_started and not self.actions_seen and not self.unknown_output and not self.pending.strip()
def snapshot(self, elapsed):
return {
"firstEventSeconds": self.first_event,
"firstActionSeconds": self.first_action,
"providerWaitSeconds": self.first_action if self.first_action is not None else elapsed,
"idleSeconds": max(0, elapsed - self.last_progress),
"actionsSeen": self.actions_seen,
"safeToRetry": self.safe_to_retry,
"providerError": self.provider_error,
"errorCount": self.errors,
"eventCounts": dict(self.counts),
"activeItems": list(self.active_items.values()),
"items": self.items,
}
def timing_report(directory, state=None):
rows, worker_details, seen, intervals = [], [], set(), []
path = directory / "events.jsonl"
if path.exists():
with path.open(encoding="utf-8") as stream:
for line in stream:
try:
event = json.loads(line)
except ValueError:
continue
if event.get("event") not in {"process.finished", "timing.span"}:
continue
key = (event.get("log"), event.get("startedAt"), event.get("phase"))
if key in seen:
continue
seen.add(key)
phase = event.get("phase") or command_phase(event.get("argv", []))
seconds = max(0, event.get("durationSeconds", 0))
try:
intervals.append((datetime.fromisoformat(event["startedAt"]), datetime.fromisoformat(event.get("finishedAt") or event["time"])))
except (KeyError, ValueError):
pass
rows.append({"phase": phase, "start": minute_label(event.get("startedAt")), "finish": minute_label(event.get("finishedAt") or event.get("time")), "seconds": seconds, "minutes": round(seconds / 60, 4), "exitCode": event.get("exitCode"), "log": event.get("log"), "providerAbort": event.get("providerAbort")})
if event.get("workerTrace"):
worker_details.append({"log": event.get("log"), **event["workerTrace"]})
totals = defaultdict(lambda: {"count": 0, "seconds": 0.0, "maxSeconds": 0.0})
for row in rows:
total = totals[row["phase"]]
total["count"] += 1
total["seconds"] += row["seconds"]
total["maxSeconds"] = max(total["maxSeconds"], row["seconds"])
measured = sum(row["seconds"] for row in rows)
wall = (max(end for _, end in intervals) - min(start for start, _ in intervals)).total_seconds() if intervals else 0
summary = [{"phase": phase, **{k: round(v, 6) if isinstance(v, float) else v for k, v in total.items()}, "minutes": round(total["seconds"] / 60, 4), "sharePercent": round(total["seconds"] / measured * 100, 2) if measured else 0} for phase, total in totals.items()]
summary.sort(key=lambda row: row["seconds"], reverse=True)
current = dict((state or {}).get("currentProcess") or {})
if current.get("status") == "running" and current.get("startedAt"):
current["elapsedMinutes"] = round((datetime.now(timezone.utc) - datetime.fromisoformat(current["startedAt"])).total_seconds() / 60, 4)
return {"schemaVersion": 1, "observedAt": datetime.now(LOCAL_ZONE).strftime("%Y-%m-%d %H:%M"), "timezone": "Asia/Shanghai", "labelPrecision": "minute", "totalMeasuredMinutes": round(measured / 60, 4), "wallClockMinutes": round(wall / 60, 4), "outsideMeasuredStepsMinutes": round(max(0, wall - measured) / 60, 4), "summary": summary, "steps": rows, "workers": worker_details, "current": display_times(current), "accounting": "Top-level measured spans only; worker item durations are nested and must not be added again. Outside measured steps includes unrecorded retry, scheduling or manual pause gaps; legacy logs cannot attribute them further."}
def format_report(report):
lines = [f"Observed: {report['observedAt']} (Asia/Shanghai)", "Phase | Count | Minutes | Share", "--- | ---: | ---: | ---:"]
lines += [f"{r['phase']} | {r['count']} | {r['minutes']:.4f} | {r['sharePercent']:.2f}%" for r in report["summary"]]
lines += [f"Measured: {report['totalMeasuredMinutes']:.4f} minutes; outside measured steps: {report['outsideMeasuredStepsMinutes']:.4f} minutes."]
lines += ["", "Start | Finish | Phase | Minutes | Exit", "--- | --- | --- | ---: | ---:"]
lines += [f"{r['start']} | {r['finish']} | {r['phase']} | {r['minutes']:.4f} | {r['exitCode']}" for r in report["steps"]]
if report["current"]:
lines += ["", "Current: " + json.dumps(report["current"], ensure_ascii=False)]
return "\n".join(lines)