From 6472890fa2c42e8e5927a6cffb0c28de5be7c292 Mon Sep 17 00:00:00 2001 From: Klagarge Date: Sat, 10 Oct 2026 18:56:05 +0200 Subject: [PATCH] fix: self-bootstrapping shiroa, help on bare just, AGENT.md - Justfile: pinned shiroa (v0.4.0-rc2, arch-aware) bootstrapped into tools/ by the 'tools' recipe; build-docs/preview-docs use it, so a fresh clone needs only Go and just. Bare 'just' now shows the recipe overview instead of starting a build. - AGENT.md: operating rules for agents (commands, layout, security invariants, shiroa version-pairing trap, testing expectations). - README: prerequisites updated (no global shiroa), bare-just behavior, tools/ in the layout. gitignore tools/ and docs/dist/. --- .gitignore | 6 +++++ AGENT.md | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++ Justfile | 31 +++++++++++++++------ README.md | 12 ++++++--- dist/.gitkeep | 0 5 files changed, 112 insertions(+), 11 deletions(-) create mode 100644 AGENT.md delete mode 100644 dist/.gitkeep diff --git a/.gitignore b/.gitignore index 1b08e70..39837ca 100644 --- a/.gitignore +++ b/.gitignore @@ -13,3 +13,9 @@ yggdrasil-server # delta workspace /.delta/ + +# pinned toolchain downloads (shiroa), managed by the Justfile +tools/ + +# shiroa serve default output +docs/dist/ diff --git a/AGENT.md b/AGENT.md new file mode 100644 index 0000000..9dc0a9d --- /dev/null +++ b/AGENT.md @@ -0,0 +1,74 @@ +# AGENT.md — operating rules for agents working on Yggdrasil + +Personal knowledge base: Typst sources (`docs/`, a shiroa book) compiled to +a static site, embedded into a Go binary that enforces per-page OIDC group +permissions (Authentik). README.md is the human-facing documentation; this +file is the working rulebook. Follow it; when it conflicts with the user, +the user wins. + +## Commands + +- `just` — shows the command overview (bare `just` never builds anything). +- `just build` — parse ACL, compile the book, build `./server`. +- `just run` — build everything, then serve on :8080. +- `just check` — go vet, go test, gofmt. Run it before every commit. +- `just clean` — removes `dist/` and `server` (keeps `tools/`). +- The shiroa toolchain is pinned and self-bootstrapped into `tools/` by the + `tools` recipe. Never assume a global shiroa/typst install exists; do not + install shiroa globally; do not bump the pin casually (see Traps). + +## Layout + +- `docs/` — shiroa book. `book.typ` lists every built page; files not listed + there are not part of the site. Categories are `=` part headers in the + summary, pages are `# chapter("path.typ")[Title]` entries. Final URL = + source path with `.typ` replaced by `.html`. +- `cmd/parser` — build step: scans `docs/` for `#let access-groups = (..)` + tags and emits `acl.json`. Untagged pages are public. +- `cmd/server` + `embed.go` (repo root) — `//go:embed` of `dist/` and + `acl.json` into a single binary. +- `internal/acl` — ACL loading and page-key normalization (security critical). +- `internal/oidc` — OIDC authorization-code client. +- `internal/middleware` — sessions, login/callback/logout, access gate. +- `legacy_docs/` — frozen previous site. Never modify or migrate it unless + the user asks explicitly. + +## Security invariants — do not break + +- Fail closed: a malformed `acl.json` aborts startup; protected pages with + no configured IdP answer 503, never the content. +- Page keys: shiroa pages keep their rendered content in sibling + `..multi.sir.in` files fetched by client-side JS. + `acl.PageKey` reduces any URL to a page key so those files inherit the + page's protection. If the shiroa output format changes, re-verify this + mapping: content in any new sidecar file must be covered or blocked. +- `/searchindex.json` contains the text of protected pages: blocked at the + router, always. +- Session cookies: HttpOnly, SameSite=Lax, Secure unless COOKIE_SECURE=false. + OAuth2 states are random and single use; post-login redirect targets are + sanitized (open-redirect guard). + +## Traps + +- Shiroa version pairing: the CLI and the `@preview/shiroa:*` typst package + must match (CLI v0.4.0-rc2 pairs with package 0.4.0; the v0.3.0 release + CLI rejects 0.4.0 books). Bump both together and rebuild. +- Typst call syntax: no space between `)` and the trailing content block — + `#chapter("x.typ")[Title]`, not `#chapter("x.typ") [Title]`. +- `dist/` and `acl.json` are embedded at Go build time; there is no runtime + reload. After changing docs or ACL, rebuild the binary. +- `go build` needs `dist/` and `acl.json` to exist: fresh clones carry + `dist/.gitkeep` and a committed `acl.json` for exactly this reason. + Never gitignore those two. + +## Testing expectations + +- Test what the user runs, not a proxy for it: from a clean state + (`rm -rf tools dist server`), `just build` and `just run` must succeed + with no global shiroa on PATH. Test the bare `just` output too. +- Server behavior: curl a public page (200), a protected page (503 without + OIDC env; it must never serve protected content), and `/searchindex.json` + (404). Extend the unit tests in `internal/acl` and `internal/middleware` + whenever touching those paths. +- A live OIDC round trip needs a real IdP and is not testable here. Say + that explicitly when reporting what was verified. diff --git a/Justfile b/Justfile index 01c95d7..0b8fe16 100644 --- a/Justfile +++ b/Justfile @@ -1,17 +1,32 @@ # Yggdrasil: Typst documentation behind an authenticated Go server. -# Run `just` alone to build everything. + +# Pinned shiroa toolchain, self-managed under tools/ (no global install). +# The CLI and the @preview/shiroa package version must match (see AGENT.md). +shiroa-version := "v0.4.0-rc2" +shiroa-target := if arch() == "x86_64" { "x86_64-unknown-linux-musl" } else if arch() == "aarch64" { "aarch64-unknown-linux-musl" } else { error("unsupported architecture " + arch()) } +shiroa-dir := "tools/shiroa" +shiroa-bin := shiroa-dir + "/shiroa-" + shiroa-target + "/shiroa" +shiroa-url := "https://github.com/Myriad-Dreamin/shiroa/releases/download/" + shiroa-version + "/shiroa-" + shiroa-target + ".tar.gz" + +# Show the available commands. +default: + @just --list # Build everything: acl.json, the static site, and the server binary. -default: build build: parse build-docs build-server # Scan docs/ for access-groups tags and regenerate acl.json. parse: go run ./cmd/parser -in docs -out acl.json -# Compile the Typst book into dist/ (requires shiroa on PATH). -build-docs: - shiroa build docs --dest-dir ../dist +# Download the pinned shiroa into tools/ if missing. +tools: + @mkdir -p {{shiroa-dir}} + @test -x {{shiroa-bin}} || { echo "Downloading shiroa {{shiroa-version}} ({{shiroa-target}})..."; curl -sSL -o {{shiroa-dir}}/shiroa.tar.gz {{shiroa-url}}; tar -xzf {{shiroa-dir}}/shiroa.tar.gz -C {{shiroa-dir}}; rm {{shiroa-dir}}/shiroa.tar.gz; {{shiroa-bin}} --version; } + +# Compile the Typst book into dist/. +build-docs: tools + {{shiroa-bin}} build docs --dest-dir ../dist # Build the server binary (embeds dist/ and acl.json). build-server: @@ -22,8 +37,8 @@ run: build ./server # Live-preview the book without authentication (shiroa dev server). -preview-docs: - shiroa serve docs +preview-docs: tools + {{shiroa-bin}} serve docs # Run static checks and tests. check: @@ -31,6 +46,6 @@ check: go test ./... @test -z "$(gofmt -l cmd internal embed.go)" || { echo 'gofmt needed:'; gofmt -l cmd internal embed.go; exit 1; } -# Remove build artifacts (keeps the committed acl.json and dist/.gitkeep). +# Remove build artifacts (keeps tools/, the committed acl.json and dist/.gitkeep). clean: rm -rf dist server diff --git a/README.md b/README.md index ca54210..14bf598 100644 --- a/README.md +++ b/README.md @@ -49,6 +49,7 @@ internal/acl/ ACL loading, page-key normalization, group checks internal/oidc/ OIDC client (authorization code flow) internal/middleware/ sessions, login/logout/callback, access gate dist/ compiled site (generated, embedded at build time) +tools/ pinned shiroa binary (downloaded by `just tools`) acl.json generated ACL (embedded at build time) legacy_docs/ the old documentation base (kept for manual migration) .gitea/workflows/ Gitea Actions CI @@ -56,10 +57,13 @@ legacy_docs/ the old documentation base (kept for manual migration) ## Quick start -Prerequisites: Go 1.22+, [just](https://github.com/casey/just), and -[shiroa](https://github.com/Myriad-Dreamin/shiroa/releases) on your `PATH`. +Prerequisites: Go 1.22+ and [just](https://github.com/casey/just). The +pinned shiroa toolchain is downloaded automatically into `tools/` on the +first build (needs `curl` + `tar` and network access); nothing is +installed globally. ```sh +just # show the available commands just run # parse ACL, compile the book, build and start the server ``` @@ -155,7 +159,9 @@ requested. Logout is available at `/logout`. ## Build and CI Local commands (see `Justfile`): `just build`, `just run`, `just parse`, -`just check`, `just clean`, `just preview-docs`. +`just check`, `just clean`, `just preview-docs`. Bare `just` prints the +command overview. The pinned shiroa lives in `tools/` (gitignored) and is +downloaded on demand. `.gitea/workflows/build.yml` (Gitea Actions, not GitLab CI) runs on push to `main` and on pull requests: installs shiroa, regenerates `acl.json`, diff --git a/dist/.gitkeep b/dist/.gitkeep deleted file mode 100644 index e69de29..0000000