Advance M7 workflows and release operations
This commit is contained in:
@@ -13,6 +13,173 @@ Production uses HTTPS. Loopback development may use `http://127.0.0.1`; `file://
|
||||
Runtime assets are same-origin. A reverse proxy must not strip the isolation, range, cache, MIME or
|
||||
ETag headers.
|
||||
|
||||
## Empty-directory install runbook
|
||||
|
||||
The release delivery contains `blender-web-offline.tar.gz`,
|
||||
`blender-web-corresponding-source.tar.gz` and `SHA256SUMS.txt`. Install as an unprivileged deployment
|
||||
user on a Linux host with GNU `tar`, `sha256sum`, `awk`, `mktemp`, `ln` and `mv`. Start with an empty
|
||||
install root; do not extract directly over a running release.
|
||||
|
||||
Set explicit paths. `BLENDER_WEB_INSTALL_ROOT` must be a new or dedicated directory, never `/`, a
|
||||
home directory or a shared workspace.
|
||||
|
||||
```bash
|
||||
BLENDER_WEB_DELIVERY=/absolute/path/to/delivery
|
||||
BLENDER_WEB_INSTALL_ROOT=/srv/blender-web
|
||||
BLENDER_WEB_ARCHIVE="$BLENDER_WEB_DELIVERY/blender-web-offline.tar.gz"
|
||||
BLENDER_WEB_SUMS="$BLENDER_WEB_DELIVERY/SHA256SUMS.txt"
|
||||
|
||||
test -f "$BLENDER_WEB_ARCHIVE"
|
||||
test -f "$BLENDER_WEB_SUMS"
|
||||
mkdir -p "$BLENDER_WEB_INSTALL_ROOT/releases"
|
||||
|
||||
BLENDER_WEB_EXPECTED_SHA256="$(awk '$2 == "blender-web-offline.tar.gz" { print $1 }' "$BLENDER_WEB_SUMS")"
|
||||
BLENDER_WEB_ACTUAL_SHA256="$(sha256sum "$BLENDER_WEB_ARCHIVE" | awk '{ print $1 }')"
|
||||
test -n "$BLENDER_WEB_EXPECTED_SHA256"
|
||||
test "$BLENDER_WEB_ACTUAL_SHA256" = "$BLENDER_WEB_EXPECTED_SHA256"
|
||||
|
||||
tar -tzf "$BLENDER_WEB_ARCHIVE"
|
||||
BLENDER_WEB_STAGE="$(mktemp -d "$BLENDER_WEB_INSTALL_ROOT/.install.XXXXXX")"
|
||||
tar --no-same-owner --no-same-permissions -xzf "$BLENDER_WEB_ARCHIVE" -C "$BLENDER_WEB_STAGE"
|
||||
test -f "$BLENDER_WEB_STAGE/blender-web-offline/app/index.html"
|
||||
test -f "$BLENDER_WEB_STAGE/blender-web-offline/deployment-contract.json"
|
||||
|
||||
BLENDER_WEB_RELEASE_DIR="$BLENDER_WEB_INSTALL_ROOT/releases/$BLENDER_WEB_ACTUAL_SHA256"
|
||||
test ! -e "$BLENDER_WEB_RELEASE_DIR"
|
||||
mv "$BLENDER_WEB_STAGE/blender-web-offline" "$BLENDER_WEB_RELEASE_DIR"
|
||||
rmdir "$BLENDER_WEB_STAGE"
|
||||
ln -s "releases/$BLENDER_WEB_ACTUAL_SHA256" "$BLENDER_WEB_INSTALL_ROOT/.current.next"
|
||||
mv -Tf "$BLENDER_WEB_INSTALL_ROOT/.current.next" "$BLENDER_WEB_INSTALL_ROOT/current"
|
||||
test -f "$BLENDER_WEB_INSTALL_ROOT/current/app/index.html"
|
||||
```
|
||||
|
||||
Before extraction, operators must inspect the `tar -tzf` output and reject absolute paths, `..`
|
||||
components, links and entries outside `blender-web-offline/`. The automated M6 gate performs these
|
||||
checks before it extracts anything. The content SHA-256 is the immutable release directory name;
|
||||
`current` is the only mutable pointer. The server document root is
|
||||
`$BLENDER_WEB_INSTALL_ROOT/current/app`, while `deployment-contract.json` remains beside `app/`.
|
||||
|
||||
## Transport preflight
|
||||
|
||||
Choose and record one origin before starting the server:
|
||||
|
||||
- Production: `https://<public-host>[:port]`. TLS terminates at the compatible server or reverse
|
||||
proxy. Plain HTTP on a public DNS name, LAN address, `0.0.0.0` or container hostname is rejected.
|
||||
- Local verification only: `http://127.0.0.1:<port>`. Do not publish the loopback-only server by
|
||||
changing its bind address.
|
||||
- `file://`, origins with embedded credentials and deployments below a URL path prefix are
|
||||
unsupported. The application and every runtime asset must share one origin.
|
||||
|
||||
An HTTPS reverse proxy must forward requests to the document root without rewriting asset paths and
|
||||
must preserve every header in `responseHeaders.allResponses`. A loopback check is not evidence that a
|
||||
public plain-HTTP deployment is supported.
|
||||
|
||||
After the server starts, replace `BLENDER_WEB_ORIGIN` with the recorded HTTPS or loopback origin and
|
||||
run:
|
||||
|
||||
```bash
|
||||
curl --fail --silent --show-error --head "$BLENDER_WEB_ORIGIN/"
|
||||
curl --fail --silent --show-error --head "$BLENDER_WEB_ORIGIN/engine-manifest.json"
|
||||
curl --fail --silent --show-error \
|
||||
--header 'Range: bytes=0-15' \
|
||||
"$BLENDER_WEB_ORIGIN/vendor/blender/single/web_engine.wasm" >/dev/null
|
||||
```
|
||||
|
||||
The first two responses must contain the three isolation headers, the contract cache policy, a
|
||||
strong ETag and the exact MIME type. The range request must return `206` with `Accept-Ranges`,
|
||||
`Content-Range`, `Content-Length` and the same strong ETag validator. Open the recorded origin in a
|
||||
new Chromium profile and require `window.isSecureContext === true`,
|
||||
`window.crossOriginIsolated === true`, the manifest to be verified and the selected engine to be
|
||||
ready before directing users to the release.
|
||||
|
||||
## Upgrade runbook
|
||||
|
||||
Keep the public origin unchanged. IndexedDB and OPFS are origin-bound; changing scheme, host or port
|
||||
creates a different storage boundary and is not an upgrade. Do not clear site data, browser profiles,
|
||||
OPFS, IndexedDB or the old release directory during an upgrade.
|
||||
|
||||
1. Announce a maintenance window. Require active edits and saves to finish, then have users close or
|
||||
reload existing tabs. A document and all of its content-hashed Workers must come from one release.
|
||||
2. Install the new archive into a new content SHA-256 release directory by repeating the validation
|
||||
and staging steps above, but do not change `current` yet.
|
||||
3. Compare `current/release-metadata.json` with the new `release-metadata.json`. Record the old and new
|
||||
product versions, engine release IDs, IndexedDB schema versions, OPFS project manifest versions,
|
||||
entry assets and Worker assets. Reject a lower storage schema during upgrade. An OPFS schema change
|
||||
requires an explicitly documented migration and recovery test; V1 currently declares schema `1`.
|
||||
4. Start the new directory on a separate loopback-only validation port. Run the header, MIME, range,
|
||||
manifest hash and browser cold-start gates before changing public traffic.
|
||||
5. Export or download a backup of each critical project before the first forward-only storage
|
||||
migration. The current IndexedDB schema is `6`; `onupgradeneeded` applies migrations atomically when
|
||||
the new StorageWorker starts. A failed transaction must leave the old database version intact.
|
||||
6. Stop admitting new sessions, then create `.current.next` pointing to the complete new release and
|
||||
atomically rename it over `current`. Never switch `app/`, `engine-manifest.json`, stable engine files
|
||||
or Worker files separately.
|
||||
7. Revalidate `/`, `/engine-manifest.json`, one engine range, `window.isSecureContext`,
|
||||
`window.crossOriginIsolated`, the selected engine and Storage schema. Reopen a saved OPFS project,
|
||||
compare its revision and blend SHA-256, perform one edit/save, and only then end maintenance.
|
||||
|
||||
HTML, `engine-manifest.json` and stable engine URLs remain `no-cache`; a release switch therefore
|
||||
revalidates them. Content-hashed entry and Worker URLs remain immutable. A manifest `releaseId`
|
||||
mismatch requires a full refresh and must not initialize an engine or open a project. If any preflight
|
||||
or post-switch check fails, leave or restore `current` to the old complete release; do not delete the
|
||||
new directory until diagnostics are captured, and do not delete the old directory until the rollback
|
||||
window ends.
|
||||
|
||||
## Rollback runbook
|
||||
|
||||
Rollback changes only the `current` app/engine release pointer. It never deletes or rewrites browser
|
||||
site data. Keep the public origin unchanged and retain both release directories plus the pre-upgrade
|
||||
project exports throughout the rollback window.
|
||||
|
||||
1. Stop new sessions and require active saves to finish. Record the current project revision and blend
|
||||
SHA-256, the active `current` target, and both releases' `release-metadata.json` files.
|
||||
2. Compare the browser's already-opened IndexedDB schema and OPFS project manifest schema with the
|
||||
target old release. A target with the same IndexedDB schema and the same OPFS schema is directly
|
||||
readable. Re-run its loopback cold-start and project-reopen checks before switching.
|
||||
3. If the browser database has migrated above the old release's IndexedDB schema, in-place rollback is
|
||||
`BLOCKED`: opening the older database version would fail. Do not delete or recreate the database.
|
||||
Deploy a forward-compatible repair release that retains the current storage schema, or restore the
|
||||
pre-upgrade project export under a separate recovery origin/profile supported by the old release.
|
||||
4. If the OPFS project manifest schema differs, in-place rollback is also `BLOCKED` unless that exact
|
||||
reverse reader has its own tested migration. V1 does not claim reverse OPFS migrations.
|
||||
5. For a compatible target, create `.current.next` pointing to the complete old content-hash directory
|
||||
and atomically rename it over `current`. Never copy individual HTML, manifest, engine or Worker files.
|
||||
6. Revalidate transport, cache, manifest and engine identity. Reopen the same project and require its
|
||||
revision and blend SHA-256 to match the pre-rollback values before allowing one edit/save.
|
||||
|
||||
If post-switch validation fails, atomically restore `current` to the newer release. Capture diagnostics
|
||||
before removing a failed target. Browser data, the newer release and project exports remain untouched.
|
||||
|
||||
## Failure diagnostics
|
||||
|
||||
The machine-readable source is `operations-diagnostics.json`. Capture the active release metadata,
|
||||
origin, failing URL, response headers, stable error code and project revision before recovery.
|
||||
|
||||
| Domain | Primary signal | Confirm | Recovery and data boundary |
|
||||
| --- | --- | --- | --- |
|
||||
| MIME | `DEPLOYMENT_MIME_MISMATCH` or streaming/module refusal | HEAD the URL; compare `Content-Type` with the contract | Fix the MIME map and repeat HEAD. Do not clear project storage. |
|
||||
| Range | `DEPLOYMENT_RANGE_INVALID`, `416`, failed resume | Request `bytes=0-15`; inspect `206`, range headers, ETag and If-Range | Fix byte ranges; discard only partial bytes whose ETag changed. Keep verified OPFS bytes. |
|
||||
| Isolation | `PLATFORM_CAPABILITY_UNAVAILABLE`, no SharedArrayBuffer | Verify HTTPS or `127.0.0.1`, all three isolation headers and both browser flags | Restore headers/same-origin assets or use the gated single variant. Do not open through a blocked pthread path. |
|
||||
| Hash | `ENGINE_VARIANT_INTEGRITY_FAILED` | Re-fetch no-cache manifest; hash JS/WASM/Worker and compare one `releaseId` | Restore one complete release and reload. No fallback or project open is allowed on mismatch. |
|
||||
| Quota | `STORAGE_QUOTA` or `QuotaExceededError` | Record storage estimate and compare committed revision/SHA-256 | Export the project and remove only selected disposable caches. Never clear all site data. |
|
||||
| Worker | `WORKER_TERMINATED` or Worker load failure | HEAD the hashed Worker; compare MIME/cache/path with release metadata | Reload one release, restart Worker and reopen the verified revision. Reject late results. |
|
||||
| GPU | `GPU_DEVICE_LOST` or a GPU budget code | Record loss reason, limits/resident bytes and unchanged project revision | Release/recreate viewport resources or reduce budget. Never rewrite Main/OPFS for GPU recovery. |
|
||||
|
||||
Run `npm --prefix web run test:operations-diagnostics` to validate all seven entries against the
|
||||
deployment contract and the packaged documentation.
|
||||
|
||||
## Fresh-directory rehearsal
|
||||
|
||||
Before freezing an RC, run `npm --prefix web run test:operations-rehearsal`. It creates one new
|
||||
temporary root and executes delivery checksum/path validation, initial install, loopback HTTP
|
||||
preflight, project seeding, complete-release upgrade, HTTP revalidation, compatible rollback and a
|
||||
final HTTP/project-integrity check in that order. It never uses an existing install directory.
|
||||
|
||||
The machine report is `release/operations-reports/rehearsal.json`. Every record contains the runbook
|
||||
command, exit code, duration and observed output. `READY` requires all records to pass, the same
|
||||
project revision and SHA-256 before/after both switches, both release directories to remain present,
|
||||
and cleanup of the temporary rehearsal root after the report is assembled.
|
||||
|
||||
## Cache policy
|
||||
|
||||
- `/` and `/index.html`: `Cache-Control: no-cache` so a deployment is revalidated.
|
||||
@@ -21,6 +188,11 @@ ETag headers.
|
||||
- `/vendor/blender/*`: `Cache-Control: no-cache`; these stable URLs are verified by the engine manifest.
|
||||
- Other paths: `Cache-Control: no-cache` until a more specific content-addressed rule exists.
|
||||
|
||||
Responses carry a strong content SHA-256 ETag. A matching `If-None-Match` returns `304` with no body;
|
||||
after HTML, manifest or a stable engine asset changes, the old validator returns the new complete response
|
||||
with `200` and a different ETag. Immutable `/assets/*` names must include the Vite content hash and are
|
||||
never reused for different bytes.
|
||||
|
||||
The server must use the MIME map in the JSON contract. In particular, WebAssembly is
|
||||
`application/wasm`, JavaScript is `text/javascript; charset=utf-8`, and NanoVDB is
|
||||
`application/x-nanovdb`.
|
||||
@@ -32,5 +204,9 @@ The server must use the MIME map in the JSON contract. In particular, WebAssembl
|
||||
unsatisfied range returns `416`. An `If-Range` mismatch ignores the range and returns the complete
|
||||
resource with status `200`; partial bytes from a different revision must never be combined.
|
||||
|
||||
Run `node tools/web/check-deployment-contract.mjs` for the static contract gate. Production HTTP
|
||||
behavior is a separate M6 acceptance gate and must exercise an actual server before release.
|
||||
Run `node tools/web/check-deployment-contract.mjs` for the static contract gate and
|
||||
`npm --prefix web run test:deployment-runbook` for the fresh-directory install and transport
|
||||
preflight gate. Run `npm --prefix web run test:upgrade-runbook` for the two-release manifest, cache,
|
||||
storage schema, Worker and atomic-switch gate, and `npm --prefix web run test:rollback-runbook` for
|
||||
compatible and blocked-old-reader rollback paths. Production HTTP behavior is a separate M6
|
||||
acceptance gate and must exercise an actual server before release.
|
||||
|
||||
Reference in New Issue
Block a user