WIP: feat(aws): add staged development platform bootstrap #5
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feat/aws-platform-bootstrap"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Automatic DNS default —
eedf46f80e4c9a0ebbed2e2337e8a7cb804b54c0environment, select the establishedaposdw.spaceparent references in memory, and publish parent NS delegation during approved install/update. The example no longer repeatsdns_mode.parent_dnssettings.Verification
Exact commit
eedf46f80e4c9a0ebbed2e2337e8a7cb804b54c0: 14 offline behavioral checks passed, plus Python syntax/example JSON, CLI help, OpenTofu formatting, Make dry-runs, DNS protection and synthetic encrypted-schema/filtering checks. Scratch probes remain outside the repository; no test framework was added.Independent final review passed with no security/logic findings, including supplemental comparisons against the immediate pre-change HEAD and receipt-resume/publication-order probes. The committed diff matches the frozen reviewed source manifest. Follow-up size: four files, +65/−33 (net +32).
Deferred/live prerequisites: no cloud plan/apply, live IAM/DNS, installation or E2E acceptance was performed. Verify that the DNS user's deployed permissions cover the default parent zone; the checked-in policy does not establish that. Normal SOPS access is required. This publication does not merge or deploy anything.
Previous implementation notes
Cleanup follow-up
Commit:
a30e9ba7d9b0f33a3b243a3d990c386ebf29ad8c.dev02-dns-replacement.patch, an unreferenced one-off patch disabling DNSprevent_destroysafeguards. The actual modules retain all three protections.aws/environments/example.jsonwas already converted to environment-based naming in985688a: no storeddns_zone_name,dashboard_hostnameorapps_dns_zone_name.Exact-commit offline verification
Existing tools image
sha256:16cfcebf27906476bcf393ae3c999879e7fe946c7182dcc9c77766afefb464eb, no network, read-only repository/scratch mounts; no decryption/cloud calls:python3 -B /checks/check_dns.py -v: all 8 behavioral checks passed.python3 -B /checks/check_cleanup.py: patch absence, retained DNS protections, migrated example, encrypted credential schema and synthetic legacy-domain filtering checks passed.python3 -B scripts/bootstrap-platform.py --help, all 3 Python script ASTs and example JSON parsing: passed.tofu fmt -check -recursive aws: passed.make -n PROVIDER=aws ENVIRONMENT=dev03 DOCKER=0 bootstrap-preflight bootstrap-plan-state bootstrap-apply-state bootstrap-plan bootstrap-apply bootstrap-install bootstrap-update-config bootstrap-verify bootstrap-platform: passed (dry-run only).git diff HEAD^ HEAD --checkand complete previous/current tree comparison: only the requested patch was deleted; all other tracked blobs/modes unchanged.Publication only; no merge, infrastructure apply, DNS mutation or deployment. Offline checks are not live deployment acceptance.
Latest update: derive DNS names from the environment
Published correction:
985688ac2265d3893e2d0a6e667577a797b80e00.environment, not redundantdns_zone_name,dashboard_hostname, orapps_dns_zone_namevalues. The shared config loader derives those names in memory before planning, installing, or verifying; it does not write them back to the source file.dev03producesdev03.aposdw.space,dashboard.dev03.aposdw.space, andapps.dev03.aposdw.space. Delegated mode uses the same names. Optionaldns_base_domainpreserves other parent domains..local.jsonfiles were present in the inspected infrastructure worktrees; no claim is made that external dev01/dev02 files were edited. Preserve environment/state identity and existing effective addresses when converting; nonstandard addresses require a reviewed migration.Verification of this correction
At
985688ac2265d3893e2d0a6e667577a797b80e00, in the existing tools imagesha256:16cfcebf27906476bcf393ae3c999879e7fe946c7182dcc9c77766afefb464eb, with networking disabled, a read-only source mount, synthetic inputs and no credentials:python3 /checks/check_dns.py -v: 8 temporary behavioral checks passed. Covers generated OpenTofu variables, unchanged source-file bytes, normalization idempotence, all config stages, manual/delegated/parent DNS, custom domains, invalid/ambiguous inputs, host-only/deferred behavior and saved-plan invalidation. Migration fixtures match the exact previous revision's validator. The new-behavior probes fail against the previous PR head and pass against this correction. The probes remain local-only; no test suite is added to the PR.Python AST parsing for all three scripts and example JSON parsing: passed.
python3 scripts/bootstrap-platform.py --help: passed.tofu fmt -check -recursive aws: passed.make -n PROVIDER=aws ENVIRONMENT=dev03 DOCKER=0 bootstrap-preflight bootstrap-plan-state bootstrap-apply-state bootstrap-plan bootstrap-apply bootstrap-install bootstrap-update-config bootstrap-verify bootstrap-platform: passed (dry-runs only).git diff --check, changed-file allowlist and added-lines credential/injection scan: passed. IaC resources, Make targets and installer are unchanged.Two independent read-only reviews found no blocking defects; the final reviewer also ran all eight offline checks. Review evidence covers the focused correction, not live deployment acceptance.
Publication only: no merge, cloud calls, infrastructure plan/apply, DNS changes, installation or deployment. Offline normalization checks do not establish live deployment acceptance.
Previous PR description
Latest update: component S3 IAM parity
Published correction:
05a8cf75caa5.components/v1/*ListBucket prefixes for the platform deployment principal and builder.s3:if-none-match = "*"; remove the overlapping unrestricted component write grant.builds/*/eifs/*; other runtime resources and IAM statements are unchanged.Previous update: existing DNS IAM user
Published correction:
34abcf30d3b3cd8b3abd2ea5d99829e2c905c0eaonfeat/aws-platform-bootstrap.arn:aws:iam::902883118030:user/dnsdeclared byaws_iam_user.dns, not an assumed role. Configuration now takeszone_id,user_arn, and an explicit repository-relativecredentials_filepointing at an existing SOPS YAML Secret. No role, trust policy, new IAM key or credential store is created.Previous update: remove added tests
Published infrastructure revision:
41236e3aae9af7da7790512b6e0201e2a21dd9b5.make setup/make up; optional bundle packaging and installation remain removed.Verification of this update
tofu init -backend=false -input=false -lockfile=readonlyandtofu validatein disposable copies with a local provider mirror. No credentials, remote backend initialization or cloud calls.git diff --checkand the added-lines security scan passed; no dangling deleted-test references remain.The current head adds no test files or test suite. Historical test counts and commands below describe earlier revisions.
Historical updates (superseded where noted above)
Source-on-EC2 bootstrap update
Latest infrastructure commit:
b0a8e68c27cbf8a9022fac36689d5759181041b8.make setup/make upworkflow. A local application-image bundle is no longer required; explicit legacy bundles remain supported.platform_revisioninaws/environments/dev01.local.jsontoc332167adbea46ea8755c6decccc268531001bef. That PR need not be merged: the published commit is fetched directly.make testremains blocked by a toolchain/dependency MSRV mismatch. Live fresh-EC2 build/start, DNS/TLS, reboot/retry and terminal/server deployment acceptance remain outstanding.Earlier implementation history
The source-on-EC2 workflow above supersedes earlier instructions that required a local platform bundle.
Latest Debian bootstrap correction
Published commit:
899fffc564979ce837597e7c45cef21d5857957b.adminSSH provisioning; checksum-pinned SSM Agent installation with a host-readiness marker.Earlier updates (historical; superseded where noted above)
Latest developer SSH access update
0.0.0.0/0is explicit, including host-only and DNS-deferred modes. Existing application-port behavior remains unchanged.ssh_public_keytoaws/environments/<environment>.local.json: the complete single-line OpenSSH Ed25519 or RSA public key, not a private key, path or authorized_keys options. Empty/omitted skips provisioning./home/ubuntu/.ssh/authorized_keys, with Ubuntu ownership and modes 0700/0600. Existing keys/restrictions are preserved, repeated keys are not duplicated, and clearing/changing the field does not revoke old keys.480651012ae3ef65e0412ca3b5e2a7661b40dc48: 177 tests run, 14 conditional skips, including enabled offline OpenTofu mocks/transitions; formatting, diff and credential-pattern checks passed. The generated shell also ran in an isolated root filesystem fixture with actual root-to-UID-1000 ownership changes, correct modes, byte preservation and idempotence. The two opt-in SOPS/GPG integration tests, all ten tools-container smoke tests, and the real pinned-source contract test also passed separately with isolated fixtures: 176 tests exercised successfully. The only unrun check is the optional real PostgreSQL integration. Independent review completed with no blocking findings.Latest development-host sizing update
m7i.2xlarge(8 vCPUs, 32 GiB RAM) for platform services plus co-located agent/development workloads. The 100 GiB data-volume default is unchanged; explicit instance-type overrides remain supported.d95dbbe6c25f26b565e79adb9ce0a6e9fed0de27: tools-container suite 158 tests run, 14 conditional skips, including enabled offline OpenTofu mock/transition checks; OpenTofu formatting, staged-byte verification and diff checks passed.Latest dev-environment and manual-DNS updates
dns_mode: manualfor a dashboard A record pointing to the platform Elastic IP. Installation rejects missing/wrong/extra A answers, stale AAAA answers and lookup failures. Omitted mode retains delegated DNS.public_ipoutput and update the example/runbook. Existing Route53/runtime resources remain; application domains still need separate DNS records.Summary
Bootstrap a development Caution platform inside an existing, manually created AWS account without changing the existing DigitalOcean workflow or adopting a live environment.
m7i.2xlargeEC2 host (8 vCPUs, 32 GiB RAM), with SSM administration and persistent encrypted EBS.make PROVIDER=aws ENVIRONMENT=dev01 DOCKER=0 secrets, thenmake PROVIDER=aws ENVIRONMENT=dev01 shell. Credentials live insecrets/aws/dev01.enc.env, not plaintext configuration or manual exports. Inner Make retains the selected provider/environment.Latest tools-container verification
Canonical
BUILDX_BUILDER=default make PROVIDER=aws PROGRESS=plain out/image/tools.digest load-tools: successful OCI build/load.BOOTSTRAP_TOOLS_IMAGE=git.internal.caution.co/public/tools:latest python3 -m unittest discover -s tests -p test_tools_container.py -v: 9 passed. Uses disposable GPG keys, synthetic secrets, no network except a loopback DNS fixture, no published ports and no daemon socket. Covers missing/wrong-key failures, new-secret creation, forwarded-agent cleanup, upstream license/notice retention, real DNS-client parsing, real local encrypted OpenTofu state and 100 consecutive Docker/Buildx starts. The repeated-start test failed before the static-client correction.Preservation coverage includes existing Unix sockets, live symlinks and dangling symlinks. A disposable mutant entrypoint with destructive cleanup was rejected in all three cases; the real entrypoint passed. The license test failed on the previous image, then passed after adding upstream Docker CLI LICENSE/NOTICE and Buildx LICENSE under
/usr/share/licenses/.Actual
make secrets/make shellexercised through a PTY with disposable home/key/repository fixtures: ciphertext creation, decryption and inherited AWS environment selection passed.Canonical tools-container Docker/Buildx clients completed a real
FROM scratchbuild through the existing host socket and exported byte-identical fixture content. No daemon was installed or started; no application service or cloud resource was changed.make test-bootstrapinside the canonical tools container: 82 tests, successful suite, 11 conditional skips. Nine skipped container tests ran separately above; the other skips are the optional pinned-source fixture and opt-in real PostgreSQL integration. Includes real local OpenTofu encrypted saved-plan/apply/state verification. Executable ephemeral test scratch is required for generated command fixtures (--tmpfs /tmp:exec,mode=1777).Containerized OpenTofu mock-provider tests against isolated temporary copies: platform 20 passed, state foundation 7 passed. No AWS credentials/API calls.
Containerized
tofu fmt -check -recursive aws,git diff --check, syntax checks and added-diff credential-pattern scan passed. No disposable tools-test containers remained.Account-target relocation: initialized legacy roots and provider/module caches were relocated and reinitialized offline. A real encrypted local-state fixture preserved encrypted bytes and decrypted lineage/serial. Regression coverage includes all five rejected symlink locations and resuming interruption between stage renames. Existing test-account workspaces were not modified.
Earlier bootstrap verification
Before the tools-container correction:
t3.xlarge, zero DNS resources, zero configured inbound rules and no application runtime. The plan was inspected but not applied; the EC2 inventory was empty at that check.Explicit limits
Review and follow-ups
The account-target layout changes passed two independent reviews with no blocking correctness or security findings. Both nonblocking suggestions were addressed: symlink/interruption tests and the runbook verification warning. The final tested tree is unchanged by the history squash.
The independent Docker/SOPS correction review found no blockers. Its two non-blocking suggestions are addressed: static-client upstream license/notice files are retained, and cleanup preservation now has socket/live-link/dangling-link regression coverage. Required current-image smoke and 100-start gates passed.
The earlier staged-bootstrap and domain-free fresh-host reviews found no blockers. The host-only ingress omission is superseded by this SSH change: it declares explicit public TCP/22 and tests the exact host-only rule. The remaining non-blocking follow-up is:
Target in AWS should be the account that you want to deploy to
667600c64ctocffd2531a7feat(aws): add staged development platform bootstrapto WIP: feat(aws): add staged development platform bootstrapcffd2531a7to7174832c6b@ -24,0 +33,4 @@Like other operations, run this in the Docker tools shell with SOPS-managed secrets:```shmake PROVIDER=aws ENVIRONMENT=dev01 DOCKER=0 secretsreplace dev01 with <your_env_name(e.g dev01)>
@ -24,0 +35,4 @@```shmake PROVIDER=aws ENVIRONMENT=dev01 DOCKER=0 secretsmake PROVIDER=aws ENVIRONMENT=dev01 shell# Inside the container, after creating aws/environments/dev01.local.json:link to example.json to make it easier to figure out what the file looks like
@ -0,0 +1,37 @@# Development-only runtime compatibility boundaryi don't think we need this file aws/src/platform_runtime/README.md
@ -0,0 +1,634 @@# Bootstrap an isolated AWS development platformthis file is way too long, make it more succinct
9ce4ce1bebto43a3c4470b9f091f25e6to05a8cf75caView command line instructions
Checkout
From your project repository, check out a new branch and test the changes.