public/platform

Fork 0

fix: adopt atomic STEVE metadata publication #486

Manually merged

anton merged 1 commit from fix/adopt-atomic-steve-metadata into main

2026-10-19 06:07:02 +00:00

anton commented

2026-10-19 00:27:18 +00:00

Owner

Summary

Adopt STEVE's atomic metadata publication fix in normal platform builds.

This PR changes only DEFAULT_STEVE_COMMIT, from 6431612db09899314c7c30bc7ca0b1c54f7bc3c0 to landed STEVE main 694b5afbd6cf6f9ec743e27266097921d4102b09. The implementation and regression tests live in the linked STEVE PR; this is independent of the E2E-speed optimization.

The old writer exposed /metadata.json before serialization finished. Normal asynchronous enclave startup lets bootproofd encounter the incomplete file and exit. The fixed writer stages complete JSON on the same filesystem, then publishes it without clobbering an existing destination. No consumer retries, timing workarounds or supervisor changes are included.

The new revision is based on current upstream STEVE main, so this default-pin update also includes upstream commits since the previous default. Existing operator STEVE_COMMIT or prebuilt component selections remain explicit overrides; updating this constant does not silently change already-running deployments or already-prepared component images. Deployments with overrides must select and prepare the fixed revision themselves.

Landed-pin refresh

The linked fix is now on STEVE main. The default pin has been refreshed to 694b5afbd6cf6f9ec743e27266097921d4102b09 rather than the pre-merge d070349.

Fresh python3 -B tests/test_prepare_components.py -v: 18 passed; git diff --check passed.
Canonical commit-addressed archive compared to the subsequent qualified da6d1ca85ddf15fec9864c9a588a046d77c32a85: all 92 tracked paths and file bytes match, no differences.
The existing build/live receipts below retain their original exact revisions; they are historical validation, not a fresh build or live run of the refreshed commit pair.
No running services or installed component selections were changed. Normal preparation must qualify the new platform/STEVE commit pair before merging/activating this adoption. Source-byte equality alone does not relabel an old accepted component set.

Historical validation before the pin refresh

python3 -B tests/test_prepare_components.py: 18 passed.
make -j1 build-api-e2e API_IMAGE=caution-api-metadata-e2e: passed.
Referenced STEVE fix: 145 workspace tests passed, 0 failed, 2 opt-in integration tests ignored; deterministic regression failed on the old publication order and passed after the fix. Independent review found no blocker.
30 offline real-binary startup-order checks passed with unchanged bootproofd. This tests metadata parsing/liveness, not NSM cryptographic attestation.
Published Git revision and public source archive contents read back and verified against local committed files.

Historical completed live dev validation

The unchanged make -j1 test-e2e-platform-ports passed: 8 steps, 0 failures, exit 0, on the normal dev host/account. Platform efc9562f9987b8fe0f282dc67818f4c61c370ec5; STEVE d0703497e5c7b04871d43fed8407d1d1a09ea892.

Normal make prepare-components built STEVE for x86_64-unknown-linux-musl, published the matching dev prebuilt set, and passed the API component preflight. The live Make invocation selected that exact set: 139a69cc01ced50ba04b70da515ab4c2676d9c3c576589d75599758b6e91e081.
All suite steps passed: gateway health; login/account setup; SSH key; E2E fixture; deployment running; AWS/host routing; direct STEVE E2P endpoint; health, attestation response, fail-closed routing and Caddy E2P parity.
The attestation assertion specifically requires HTTP 200 and a nonempty attestation-document string. This is not an independent certificate/COSE-signature validation or full successful encrypted application round trip.
The app was destroyed and test containers/database volume were removed. Independent post-run checks confirmed all five original service images running with active units, API/gateway/email HTTP 200, unchanged configuration/unit hashes, and no new live instances, VPCs, security groups or internet gateways in either checked dev region (us-west-2, ap-south-1). Prepared component artifacts are intentionally retained.
An initial attempt stopped during component preparation before suite assertions: the local validation checkout borrowed Git objects from an unmounted alternate directory. Its original services were restored. Making the owned checkout self-contained fixed the container-visible source read without changing tracked source. A retry-only external-runner procfs disappearance guard was also corrected and tested before launch (40 offline runner tests passed). No platform assertions, timeouts, startup ordering or concurrency settings were altered.

The other component pins were the existing dev selections. This is one successful live correctness run, not a production failure-rate estimate, exhaustive default-pin matrix, full E2E suite, or speed benchmark. The successful Make command took 629.433 seconds, including its setup and test teardown, but excluding separate component qualification and external host restoration; no timing comparison is claimed.

Evidence is preserved locally under /home/tuna/git/metadata-publication-fix-evidence/: component-preparation.log, qualified-components.lock.json, retry-02/live-ports/ports/{console.log,result.json}, retry-02/restoration-independent.json, and the before/after AWS inventories with retry-02/aws-resource-additions.json. Public source archives were independently downloaded and matched to every tracked committed file.

The linked STEVE fix has landed and the pin above is refreshed. Requalify the selected prebuilt set for the new commit pair before merging this adoption PR. Already-running deployments and explicit operator component overrides do not change automatically.

## Summary Adopt [STEVE's atomic metadata publication fix](https://git.caution.co/public/steve/pulls/18) in normal platform builds. This PR changes **only `DEFAULT_STEVE_COMMIT`**, from `6431612db09899314c7c30bc7ca0b1c54f7bc3c0` to landed STEVE main `694b5afbd6cf6f9ec743e27266097921d4102b09`. The implementation and regression tests live in the linked STEVE PR; this is independent of the E2E-speed optimization. The old writer exposed `/metadata.json` before serialization finished. Normal asynchronous enclave startup lets bootproofd encounter the incomplete file and exit. The fixed writer stages complete JSON on the same filesystem, then publishes it without clobbering an existing destination. No consumer retries, timing workarounds or supervisor changes are included. The new revision is based on current upstream STEVE main, so this default-pin update also includes upstream commits since the previous default. Existing operator `STEVE_COMMIT` or prebuilt component selections remain explicit overrides; updating this constant does not silently change already-running deployments or already-prepared component images. Deployments with overrides must select and prepare the fixed revision themselves. ## Landed-pin refresh The linked fix is now on STEVE main. The default pin has been refreshed to `694b5afbd6cf6f9ec743e27266097921d4102b09` rather than the pre-merge `d070349`. - Fresh `python3 -B tests/test_prepare_components.py -v`: **18 passed**; `git diff --check` passed. - Canonical commit-addressed archive compared to the subsequent qualified `da6d1ca85ddf15fec9864c9a588a046d77c32a85`: **all 92 tracked paths and file bytes match**, no differences. - The existing build/live receipts below retain their original exact revisions; they are **historical validation**, not a fresh build or live run of the refreshed commit pair. - No running services or installed component selections were changed. Normal preparation must qualify the new platform/STEVE commit pair before merging/activating this adoption. Source-byte equality alone does not relabel an old accepted component set. ## Historical validation before the pin refresh - `python3 -B tests/test_prepare_components.py`: **18 passed**. - `make -j1 build-api-e2e API_IMAGE=caution-api-metadata-e2e`: **passed**. - Referenced STEVE fix: **145 workspace tests passed, 0 failed, 2 opt-in integration tests ignored**; deterministic regression failed on the old publication order and passed after the fix. Independent review found no blocker. - **30 offline real-binary startup-order checks** passed with unchanged bootproofd. This tests metadata parsing/liveness, not NSM cryptographic attestation. - Published Git revision and public source archive contents read back and verified against local committed files. ## Historical completed live dev validation The unchanged `make -j1 test-e2e-platform-ports` **passed: 8 steps, 0 failures, exit 0**, on the normal dev host/account. Platform `efc9562f9987b8fe0f282dc67818f4c61c370ec5`; STEVE `d0703497e5c7b04871d43fed8407d1d1a09ea892`. - Normal `make prepare-components` built STEVE for `x86_64-unknown-linux-musl`, published the matching dev prebuilt set, and passed the API component preflight. The live Make invocation selected that exact set: `139a69cc01ced50ba04b70da515ab4c2676d9c3c576589d75599758b6e91e081`. - All suite steps passed: gateway health; login/account setup; SSH key; E2E fixture; deployment running; AWS/host routing; direct STEVE E2P endpoint; health, attestation response, fail-closed routing and Caddy E2P parity. - The attestation assertion specifically requires HTTP 200 and a nonempty attestation-document string. This is **not** an independent certificate/COSE-signature validation or full successful encrypted application round trip. - The app was destroyed and test containers/database volume were removed. Independent post-run checks confirmed all five original service images running with active units, API/gateway/email HTTP 200, unchanged configuration/unit hashes, and no new live instances, VPCs, security groups or internet gateways in either checked dev region (`us-west-2`, `ap-south-1`). Prepared component artifacts are intentionally retained. - An initial attempt stopped during component preparation before suite assertions: the local validation checkout borrowed Git objects from an unmounted alternate directory. Its original services were restored. Making the owned checkout self-contained fixed the container-visible source read without changing tracked source. A retry-only external-runner procfs disappearance guard was also corrected and tested before launch (**40 offline runner tests passed**). No platform assertions, timeouts, startup ordering or concurrency settings were altered. The other component pins were the existing dev selections. This is one successful live correctness run, not a production failure-rate estimate, exhaustive default-pin matrix, full E2E suite, or speed benchmark. The successful Make command took **629.433 seconds**, including its setup and test teardown, but excluding separate component qualification and external host restoration; no timing comparison is claimed. Evidence is preserved locally under `/home/tuna/git/metadata-publication-fix-evidence/`: `component-preparation.log`, `qualified-components.lock.json`, `retry-02/live-ports/ports/{console.log,result.json}`, `retry-02/restoration-independent.json`, and the before/after AWS inventories with `retry-02/aws-resource-additions.json`. Public source archives were independently downloaded and matched to every tracked committed file. The linked STEVE fix has landed and the pin above is refreshed. Requalify the selected prebuilt set for the new commit pair before merging this adoption PR. Already-running deployments and explicit operator component overrides do not change automatically.

anton added 1 commit

2026-10-19 00:27:18 +00:00

fix: adopt atomic STEVE metadata publication efc9562f99

anton referenced this pull request from public/steve

2026-10-19 00:52:06 +00:00

fix: publish enclave metadata atomically without clobbering #18

anton added 1 commit

2026-10-19 05:22:29 +00:00

fix: pin landed STEVE metadata publication c28ab8cd84

anton referenced this pull request

2026-10-19 05:22:31 +00:00

perf(tests): default live E2E to six parallel suites #487

anton commented

2026-10-19 06:01:06 +00:00

Author

Owner

Six-worker live E2E recheck — PASS

One complete batch ran on the existing dev host/account, with no overlapping live experiment and no follow-on batch. Both PRs remain unmerged.

Exact inputs and scope

Platform: d9a99e00be0a42f49c2c711615cb90817fe2b7fb (published PR #487).
STEVE: 694b5afbd6cf6f9ec743e27266097921d4102b09 (landed revision, selected through the supported Make override).
Normal source Makefile, make -j1 test-e2e E2E_JOBS=6, including build, component preparation, all suites and test teardown. No experimental Makefile and no separate prebuild.
This qualifies that exact platform/STEVE pair. It is not execution of PR #486's framework head or a hypothetical merged tree.
Exact public platform and STEVE archives matched their Git trees. Forty offline runner checks and 27 scheduler checks passed before launch; these are separate from the live results below.

Live result

Full Make: 1151.300 seconds (19m11.3s), exit 0.
All six suites started within 0.004 seconds of one another.
6/6 suites passed, 56 reported checks passed, 0 failed.

Suite	Passed	Failed
Environment parity	9	0
Happy path	10	0
PGP units	11	0
PGP audit constraints	9	0
SSH units	9	0
Platform ports	8	0

The previous six-worker batch took 841.090 seconds (14m01.1s). This recheck took 310.210 seconds longer. Build/setup before suite launch grew from 51.618 to 372.368 seconds; suites plus teardown took 778.932 seconds. Different framework/component revisions and retained build caches make this a correctness recheck, not a controlled performance comparison. Full Make timing excludes preflight and external original-service restoration.

Component verification

Normal Make component preparation and startup preflight passed. Post-run, read-only verification confirmed the selected canonical S3 manifest matched its accepted local lock and all five source pins. The downloaded STEVE ELF matched its declared size and digest:

Component set: 03327d5f458efc2e045d46c09cc177bcec42fc3c71ec4e4af171c047dacbc93f.
STEVE SHA-256: 4f6aa867e3a7068904acc0b1a86a8266f4d854e82cbf7f5f1b8595c84919f8bf.
STEVE bytes: 6458584.

Cleanup and restoration

Launcher, independent restoration check, and AWS inventory check: all exit 0.
Original postgres, API, gateway, email and metering services active, with original images.
Direct-IP API/gateway/email health: HTTP 200.
Original PostgreSQL persistent mounts retained; config and unit files unchanged.
No test containers or test volume remain; Make process group is quiescent.
Before/after nonterminated EC2 instances, VPCs, security groups and internet gateways have identical IDs in us-west-2 and ap-south-1. This inventory is not a claim about every AWS resource class. Immutable component objects remain intentionally retained.
Source, runner, frozen run inputs and predecessor evidence hashes unchanged.

Evidence

FINAL-VERIFICATION.json, component-provenance.json, live-ports/ports/result.json, live-ports/result.json, live-ports/restored.json, restoration-independent.json, six source/tests/e2e/logs/*.log files, AWS before/after inventories, and completion-sha256.json.

The historical live-ports/ports folder name is retained by the reused runner; its receipt records the full six-suite Make command, not a standalone platform-ports run.

Evidence SHA-256 receipts:

FINAL-VERIFICATION.json: 355646dbb62564098d5dbe84af2d3f471a347c5fc132c10db880120dd647444a
component-provenance.json: 25cf0e66c458e66e97435c8348b7da8733d274ca4f83924fb8f2ae12a46e3ccd
RESULTS.md: c362f4a1406106b2f98afd44806a7c2680c5de581403c935d48ebb750f3020a0

# Six-worker live E2E recheck — PASS One complete batch ran on the existing dev host/account, with no overlapping live experiment and no follow-on batch. Both PRs remain unmerged. ## Exact inputs and scope - Platform: `d9a99e00be0a42f49c2c711615cb90817fe2b7fb` (published PR #487). - STEVE: `694b5afbd6cf6f9ec743e27266097921d4102b09` (landed revision, selected through the supported Make override). - Normal source Makefile, `make -j1 test-e2e E2E_JOBS=6`, including build, component preparation, all suites and test teardown. No experimental Makefile and no separate prebuild. - This qualifies that exact platform/STEVE pair. It is not execution of PR #486's framework head or a hypothetical merged tree. - Exact public platform and STEVE archives matched their Git trees. Forty offline runner checks and 27 scheduler checks passed before launch; these are separate from the live results below. ## Live result - Full Make: **1151.300 seconds (19m11.3s), exit 0**. - All six suites started within 0.004 seconds of one another. - **6/6 suites passed, 56 reported checks passed, 0 failed.** | Suite | Passed | Failed | | --- | ---: | ---: | | Environment parity | 9 | 0 | | Happy path | 10 | 0 | | PGP units | 11 | 0 | | PGP audit constraints | 9 | 0 | | SSH units | 9 | 0 | | Platform ports | 8 | 0 | The previous six-worker batch took 841.090 seconds (14m01.1s). This recheck took 310.210 seconds longer. Build/setup before suite launch grew from 51.618 to 372.368 seconds; suites plus teardown took 778.932 seconds. Different framework/component revisions and retained build caches make this a correctness recheck, **not a controlled performance comparison**. Full Make timing excludes preflight and external original-service restoration. ## Component verification Normal Make component preparation and startup preflight passed. Post-run, read-only verification confirmed the selected canonical S3 manifest matched its accepted local lock and all five source pins. The downloaded STEVE ELF matched its declared size and digest: - Component set: `03327d5f458efc2e045d46c09cc177bcec42fc3c71ec4e4af171c047dacbc93f`. - STEVE SHA-256: `4f6aa867e3a7068904acc0b1a86a8266f4d854e82cbf7f5f1b8595c84919f8bf`. - STEVE bytes: `6458584`. ## Cleanup and restoration - Launcher, independent restoration check, and AWS inventory check: all exit 0. - Original postgres, API, gateway, email and metering services active, with original images. - Direct-IP API/gateway/email health: HTTP 200. - Original PostgreSQL persistent mounts retained; config and unit files unchanged. - No test containers or test volume remain; Make process group is quiescent. - Before/after nonterminated EC2 instances, VPCs, security groups and internet gateways have identical IDs in `us-west-2` and `ap-south-1`. This inventory is not a claim about every AWS resource class. Immutable component objects remain intentionally retained. - Source, runner, frozen run inputs and predecessor evidence hashes unchanged. ## Evidence `FINAL-VERIFICATION.json`, `component-provenance.json`, `live-ports/ports/result.json`, `live-ports/result.json`, `live-ports/restored.json`, `restoration-independent.json`, six `source/tests/e2e/logs/*.log` files, AWS before/after inventories, and `completion-sha256.json`. The historical `live-ports/ports` folder name is retained by the reused runner; its receipt records the full six-suite Make command, not a standalone platform-ports run. Evidence SHA-256 receipts: - `FINAL-VERIFICATION.json`: `355646dbb62564098d5dbe84af2d3f471a347c5fc132c10db880120dd647444a` - `component-provenance.json`: `25cf0e66c458e66e97435c8348b7da8733d274ca4f83924fb8f2ae12a46e3ccd` - `RESULTS.md`: `c362f4a1406106b2f98afd44806a7c2680c5de581403c935d48ebb750f3020a0`

anton referenced this pull request

2026-10-19 06:01:08 +00:00

perf(tests): default live E2E to six parallel suites #487

anton force-pushed fix/adopt-atomic-steve-metadata from c28ab8cd84 to f1cef7a621

2026-10-19 06:06:49 +00:00

Compare

anton manually merged commit f1cef7a621 into main

2026-10-19 06:07:02 +00:00