Test Matrix
This matrix is a normative proof map, not a product-behavior source of truth. The Requirement column summarizes canonical contracts from spec.md and companion docs so each gate has a testable obligation; if a row conflicts with a canonical contract, the canonical contract wins and this matrix must be patched in the same PR.
Required phase column. v0.1 CI means the gate is enforced from the first v0.1 commit. Release means the gate is enforced at release tagging. Same PR means the proof artifact MUST land in the same PR as the change that triggered the requirement — no deferral to a follow-up PR is permitted. vN.M CI means the gate is enforced starting with the named version's CI; PRs targeting that version must include the proof.
| Requirement | Proof artifact | Required phase |
|---|---|---|
| Tier A has exactly 9 meta-tools | TestMetaToolNamesReturnsExactly9 |
v0.1 CI |
| Tier A has at most 18 convenience tools | TestTierAConvenienceToolCount |
v0.1 CI |
Tier A cold-start schema budget <= 8,000 cl100k_base tokens |
TestTierATokenBudget |
v0.1 CI and release |
Tier A output schemas preserve $defs |
TestTierARegistrationOutputSchemasValid / TestRegisteredResultSchemasMatchSpecDefs |
v0.1 CI |
Tier A representative structuredContent validates against ToonResult, SingleObjectResult, RawJsonResult, GainResult, and CacheStatsResult schemas; diff-only 304 responses ({"unchanged": true, "etag": "..."}) are exempt and the test MUST include at least one 304 fixture and verify validation is skipped (not failed) for that shape |
TestTierAResponseShapeConformance / TestGainOutputSchema / TestCacheStatsOutputSchema |
v0.1 CI |
All Tier A tool registrations include outputSchema |
TestTierARegistrationScan |
v0.1 CI |
Every registered tool that advertises an outputSchema returns a structuredContent that validates against that exact registered schema, and every tool that advertises none returns no structuredContent; the check runs over the live tools/list roster, so a new tool with no entry in the call table fails rather than skips |
TestEveryRegisteredToolPairsOutputSchemaWithStructuredContent |
v0.1 CI |
gum.describe_op output validates against DescribeOpResult for every op in the embedded catalog |
TestDescribeOpResultValidatesForEveryCatalogOp |
v0.1 CI |
No production code path imports any package under a testdata/ path segment; test-only registration helpers in testdata/ cannot contribute to the Tier A budget |
TestTestdataNoProductionImport |
v0.1 CI |
Tool annotations serialize according to go-sdk v1.7.0 wire semantics: readOnlyHint present on every Tier A tool carrying that tool's own boolean value (v1.7.0 dropped the omitempty that hid false under v1.6.x), destructiveHint present with explicit true/false on every Tier A tool, and optional idempotentHint / openWorldHint never serialize as null |
TestToolAnnotations / TestToolAnnotationsWireForm |
v0.1 CI |
Confirmation-required paths are bound to exact pending requests: gum.destructive, gum.write variants with confirmation_policy="high_stakes_write", and gum.code emit short-lived confirmation tokens, reject missing/expired/replayed/mismatched tokens with CONFIRMATION_TOKEN_INVALID, and make no upstream call on rejection; gum.code tokens bind language, source_sha256, allow_write, allow_destructive, destructive_budget, and canonical destructive_scope; fixture set MUST include (a) replay attempt within TTL, (b) replay attempt after TTL, (c) cross-process replay (token minted by one gum process and presented to another), (d) cross-profile replay (token minted under profile A presented under profile B), (e) confirmed gum.code re-invocation that increases destructive_budget, broadens destructive_scope, or adds allow_write after approval, and (f) a gum.write high-stakes variant whose token cannot be reused for another write or destructive variant — all reject with CONFIRMATION_TOKEN_INVALID. The cross-profile case carries reason: "mismatch", because profile_name is inside the §6.1.2 binding tuple. The cross-process case carries reason: "replayed": the signing key is per user and persistent ($XDG_DATA_HOME/gum/confirmation-signing.key), so an unspent token deliberately verifies in a second gum process, which is what makes the two-step CLI confirm work; the durable replay marker, not a key mismatch, is what refuses the reuse |
TestConfirmationTokenBinding / TestPolicyHighStakesWriteRequiresConfirmationToken / TestPolicyHighStakesWriteTokenDoesNotSatisfyDestructive |
v0.1 CI |
gum.code confirmation token re-hashes submitted source at re-invocation: presenting a valid token with a different source body (same source_sha256 in token, different actual source) is rejected with CONFIRMATION_TOKEN_INVALID (reason: mismatch); no script execution occurs |
TestConfirmationTokenSourceRehash |
v0.1 CI |
Tier A convenience-tool ABI is generated from the normative §4.1 table: each row binds to the listed backing op_id, fixed/default variant rule, required/optional args, wire formats, and confirmation-passthrough policy; the row names no output profile, because a convenience tool resolves one from its backing catalog variant; rows that wrap confirmation_policy="high_stakes_write" variants without confirmation_passthrough=yes fail catalog generation; every convenience tool registers an outputSchema covering its confirmation branch when applicable |
TestTierAConvenienceABI / TestConvenienceHighStakesConfirmation |
v0.1 CI |
MCP completions are available for op IDs, variant IDs, resource-template params, plugin names, help topics, operation handles, and the closed enums gum.code.language (risor only in v0.1.0) and gum.*.format (toon, csv, json, markdown) without upstream calls; CLI invoke completions cover op IDs (per risk class), --variant-id, --profile (the profile directories under <config home>/gum, plus the implicit default), plugin names (every inventory row, on plugin remove|reload|unquarantine|setup|run, first positional only), help topics, and gum call boolean output flags (--json, --toon, --csv, --markdown), while gum call --format and gum code --lang are absent in v0.1 (gum code exposes --language); reporting subcommands may expose their own --format flag |
TestMCPCompletions / TestCompleteHasNoToolReferenceType / TestCLICompletionsAllShells / TestCLICompletionInvokeSurfaceParity |
v0.1 CI |
Tier A registrations complete before MCP transport accepts connections; no Server.AddTool calls occur after Server.Run; no spurious tools/list_changed notifications fire during handshake |
TestMCPStartupOrdering |
v0.1 CI |
Loading active plugins before Server.Run does not grow tools/list: the wire roster stays the 27 names in docs/tier-a-roster.v1.json (9 meta-tools + 18 convenience tools) plus the two out-of-roster skill helpers skills_list and skills_get, 29 entries, and no plugin name reaches a tool. Active plugins stay visible through gum://plugins. The §5 line 405 clause that their variants also reach the session catalog snapshot, and so gum://op/{id} and search, is not implemented and is tracked under gum-26nz |
TestTierARosterManifest / TestTierAToolCountWithPlugins |
v0.1 CI |
logging server capability is absent from the v0.1.0 initialize response; the §13.2 capability matrix declares exactly which server-owned capabilities are advertised (tools, resources, prompts, completions — notifications/cancelled and notifications/progress honoured; roots and elicitation consumed only when the client-owned capability is declared; sampling/logging/list_changed absent from the server capabilities object) |
TestMCPInitializeCapabilities |
v0.1 CI |
MCP 2025-11-25 task capability is absent from the v0.1.0 initialize response; tasks/get, tasks/result, tasks/list, tasks/cancel, notifications/tasks/status, and notifications/elicitation/complete are not advertised or emitted in v0.1.0; emitted tools/list, resources/list, resources/templates/list, and prompts/list payloads contain no icons fields |
TestMCPInitializeCapabilities / TestNoTaskCapabilityV01 / TestNoIconMetadataV01 |
v0.1 CI |
gum.poll sends progress only when _meta.progressToken is present, uses MCP progress wire shape, and preserves the token's JSON type (string or integer) end-to-end without stringifying integers or coercing to float |
TestPollProgressTokenContract |
v0.1 CI |
gum.poll timeout and context cancellation do not leak goroutines |
TestPollTimeoutAndCancellation |
v0.1 CI |
MCP roots drive project-local expression-profile lookup; fixtures cover single file:// root, multiple roots with _meta.gumRoot, multiple roots without _meta.gumRoot failing with PROJECT_ROOT_REQUIRED, non-file roots rejected for project-local lookup, roots-unavailable MCP sessions disabling project-local lookup by default, and --allow-implicit-project-root as the only path that may use GUM_PROJECT_ROOT / $PWD with _profile_resolution_warning: "implicit_project_root" |
TestProfileResolutionFromMCPRoots |
v0.1 CI |
Prompts are registered and retrievable through prompts/list and prompts/get; v0.1 prompts are exactly the zero-argument static prompts in §13, prompts/list reports empty arguments, prompts/get rejects supplied arguments with INVALID_ARGS, embedded templates stay under 6 KiB, and returned payloads contain one user-message text block |
TestPromptRegistration / TestPromptZeroArgumentContract |
v0.1 CI |
No unsolicited server notifications are emitted before notifications/initialized |
TestMCPInitializedWaitRule |
v0.1 CI |
MCP resource templates appear in resources/templates/list (not resources/list) |
TestResourceTemplateRegistration |
v0.1 CI |
resources/list and resources/templates/list accept only MCP cursor pagination: requests use optional cursor, responses use nextCursor, the server page cap is 100 entries, and no client page-size parameter is accepted or required |
TestMCPResourceCursorPagination |
v0.1 CI |
gum://schema/{ref} returns JSON Schema documents for schema refs exposed by active gum://op/{id} / gum://variant/{id} resources using exactly one MCP text resource content item with requested uri, mimeType="application/schema+json", and JCS-canonical JSON payload, enabling exact TOON reconstruction; gum://op/{id} and gum://variant/{id} success reads use the same one-item JSON shape with mimeType="application/json"; refs must satisfy the safe served-ref grammar; unknown refs, invalid ref grammar, inactive-plugin refs (installed_pending_restart or needs_configuration), and quarantined-plugin refs return canonical RESOURCE_NOT_FOUND as a JSON-RPC resource error; divergent full-profile-inventory ref collisions fail build/install with SCHEMA_REF_COLLISION while identical-body reuse is allowed; a ref generated by gen-catalog -emit-schemas resolves against the real embedded catalog, not only an injected one |
TestSchemaResourceLookup / TestOpVariantResourceWireShape / TestSchemaResourceGrammarRejection / TestSchemaRefCollision / TestSchemaResourceServesAGeneratedRequestSchema |
v0.1 CI |
Static resources (gum://catalog, gum://status/canaries, gum://plugins, gum://help/topics) appear in resources/list |
TestStaticResourceRegistration |
v0.1 CI |
Recovery links for recovery="resource_link" appear as both _expression.full_result_resource and exactly one MCP resource_link content block with the same gum://results/<hash> URI; local_artifact never emits a resource-link block |
TestRecoveryResourceLinkContentBlock |
v0.1 CI |
tee.secret lifecycle and principal scoping: (a) absent secret is generated lazily on first lossy write, exactly 32 bytes, mode 600; (b) corrupt or malformed secret (not 64 lowercase hex chars) causes TEE_SECRET_CORRUPT error, no silent regeneration; (c) gum://results/{hash} resolution uses directory scan, success reads return exactly one decompressed application/json resource item, and absent hashes return RESULT_ARTIFACT_EXPIRED; (d) same op/variant/args under two credential subjects in one profile yields different recovery URIs, tee paths, HTTP cache keys, semantic cache keys, and gain-ledger subject fingerprints |
TestTeeSecretStability / TestResourceReadResultsHit / TestPrincipalScopedRecoveryAndCache |
v0.1 CI |
installed_pending_restart and needs_configuration plugins are inventory-only: not searchable, operation-completable, describable as active, invokable via tools, or reachable from gum.code; gum://plugin/{name} validates against the fixed per-status resource shape, including safe credential descriptors for needs_configuration and no raw env var names; gum://op/{id} and gum://variant/{id} may consult inventory only for status-only inactive-plugin responses with execution_support: "schema_only" and status, never full invocable schemas; quarantined gum://op/{id} and gum://variant/{id} return VARIANT_QUARANTINED JSON-RPC resource errors instead of status-only responses; invoking inactive plugins via risk invoke tools returns UNSUPPORTED_CAPABILITY with the same status; inactive-plugin UNSUPPORTED_CAPABILITY envelopes MUST contain status and MUST NOT contain unsupported_capabilities (the two-variant mutual exclusion of §5.8); the capability-class path MUST NOT contain status; fixtures prove that live inventory registry and active session catalog snapshot can diverge without leaking inactive variants into search/describe/completions/invoke, and that quarantined wins when a plugin also has pending-restart or needs-configuration state |
TestPluginInactiveInventoryOnly / TestPluginResourceFullSurface / TestLoadPluginResourceRecordInstalledPendingRestart / TestLoadPluginResourceRecordNeedsConfiguration / TestLoadPluginResourceRecordQuarantined / TestPendingRestartExcludedFromCompletions / TestVariantResourceInactivePluginPath / TestOpResourceInactivePluginPath / TestPluginStatusPrecedence |
v0.1 CI |
gum plugin info <name> exists under gum plugin with --format text|json (§12 line 2495 roster, line 2520 JSON root); --format=json emits the JCS-canonical gum://plugin/{name} payload from the same loader the MCP resource handler uses, so the field set and byte order match; the text form reports status and prints a remediation line for quarantined, needs_configuration, and installed_pending_restart; an unknown plugin fails with RESOURCE_NOT_FOUND and a non-zero exit; an unsupported --format value is rejected |
TestPluginInfoJSONCarriesSpecFields / TestPluginInfoJSONIsCanonical / TestPluginInfoTextReportsStatus / TestPluginInfoUnknownPluginErrors / TestPluginInfoRejectsUnknownFormat / TestPluginInfoCmdRegistered |
v0.1 CI |
JSON-valued resource reads (gum://op/{id}, gum://variant/{id}, gum://results/{hash}, gum://plugin/{name}, and deprecated gum://help/{topic} redirects) return exactly one MCP text resource content item with requested uri, mimeType="application/json", and a byte-exact JCS-canonical JSON payload, so &, < and > in a value stay literal instead of being HTML-escaped; markdown help topics return exactly one text/markdown item |
TestHandlePluginReadHappyPath / TestPluginResourceFullSurface / TestHelpTopicsSeedSet / TestDeprecatedHelpRedirectBodyIsJCSCanonical / TestInactivePluginResourceBodyIsJCSCanonical / TestResultArtifactBodyIsJCSCanonical |
v0.1 CI |
| Sampling remains optional | capability-negotiation test once sampling lands | v0.3 CI |
Runtime third-party plugin install resolves one manifest schema_ref and materializes the served request/response refs: unsafe ref grammar, traversal, path separators, percent-encoded separators and over-long refs are rejected with PLUGIN_SCHEMA_REF_INVALID before any path is built; accepted refs write plugin-schemas/<schema_ref>.request.<sha256>.json and plugin-schemas/<schema_ref>.response.<sha256>.json and the MCP schema resource serves them back; a divergent-body ref already in the profile's plugin inventory fails with SCHEMA_REF_COLLISION whether the incumbent is active, pending-restart, needs-configuration or quarantined, while identical-body reuse is allowed. The same collision rule covers the embedded first-party store, which gen-catalog -emit-schemas fills with one request document per op that declares request_fields; it carries no response schemas, because the repository has no offline source for them |
TestPluginSchemaRefThirdPartyInstall / TestPluginSchemaRefCrossPluginCollision / TestSchemaRefCollisionSpansEveryPluginState / TestSchemaResourceServesInstalledPluginSchema / TestFirstPartySchemaRefsCoverTheEmbeddedStore / TestPluginCannotClaimAFirstPartyRef / TestPluginMayMirrorAFirstPartyRefByteForByte |
v0.1 CI |
| Unknown capabilities fail closed | generator + plugin install tests for UNKNOWN_CAPABILITY |
v0.1 CI |
Unknown backend_kind values fail closed at catalog build with UNKNOWN_BACKEND_KIND: Catalog.Validate rejects any value outside the closed enum, and cmd/gen-catalog is the only caller, so an unknown kind can never reach an embedded catalog. §5.8 defines no runtime loader arm for this: the main catalog is //go:embed-ed with no override path, so a binary never loads a catalog it did not build |
TestOpValidateRejectsUnknownBackendKind |
v0.1 CI |
Unknown interface_kind values fail closed at catalog build with UNKNOWN_INTERFACE_KIND, on the same Catalog.Validate path and with the same build-only reach as backend_kind. §5.8 defines no runtime loader arm for this either, for the same reason |
TestOpValidateRejectsUnknownInterfaceKind |
v0.1 CI |
| Variant lifecycle does not silently fall back | tests for deprecated, superseded, removed, and quarantined variants | v0.1 CI |
| Default variant never selects removed/quarantined/deprecated variants when an active executable alternative exists | TestDefaultVariantLifecycleSelection |
v0.1 CI |
Plugin registry writes are atomic across the canonical registry ABI (plugin-catalog.json, plugins.lock, and plugin-state.json schemas in spec.md §8.7 / docs/catalog-abi.md): install/remove/startup-activation stages temp files with one install_generation/install_txid, publishes all three under plugins.install.lock, keeps the previous complete generation authoritative on failure, and startup recovers from mixed generations by selecting the last complete shared generation and quarantining orphan staged artifacts; the stamp is on all three files, so a publish that renamed only plugin-catalog.json is detected, while a catalog written before the field pair adopts the lock/state generation once and is stamped by the next transaction |
TestPluginRegistryABISchemas / TestPluginInstallTransactionAtomicity / TestPluginInstallCrashRecovery / TestCatalogOnlyTearIsDetected / TestUnstampedCatalogIsAcceptedOnce / TestWriteTransactionStampsTheCatalog |
v0.1 CI |
A failed §8.7 publish rolls back and leaves no staging files: a directory-fsync failure deletes all three temps, a rename that fails partway restores every file already renamed so the previous complete generation stays byte-identical, a failed first-ever publish removes the file it had already renamed, and a rollback that cannot restore reports rollback incomplete alongside the original error; on a filesystem that cannot fsync, the install completes and exactly one fsync_not_supported warning per profile per process reaches both sinks §8.7 names, the host log and the profile audit log, carrying path, syscall_errno and the remediation suggestion |
TestWriteTransactionRemovesTempsWhenDirSyncFails / TestWriteTransactionRollsBackTornPublish / TestWriteTransactionRollbackRemovesFirstEverPublish / TestWriteTransactionReportsIncompleteRollback / TestWriteTransactionSurvivesUnsupportedFsync / TestUnsupportedFsyncWritesAuditRow / TestRegistryAuditSinkWritesProfileAuditRow |
v0.1 CI |
The §8.7 publish sorts every array before it writes: plugins.lock and plugin-state.json by plugin name, and plugin-catalog.json variants by variant_id (§8.7 line 1884), so two profiles that installed the same plugins in a different order produce byte-identical files |
TestWriteTransactionSortsCatalogVariants |
v0.1 CI |
gum plugin remove drops the plugin's rows from all three registry files under the same transaction protocol install uses (§8.7 line 1893): its plugin-catalog.json variants, its plugins.lock row with the namespace lease it holds, and its plugin-state.json quarantine row; after remove a different namespace_owner may claim the same prefix; the gum plugin remove cobra command resolves the profile directory and takes that registry path, not the profile-less dispatch path |
TestRemoveWithRegistryClearsAllThreeFiles / TestRemoveThenInstallDifferentOwnerSucceeds / TestPluginRemoveUsesRegistryWhenProfileExists / TestPluginRemoveCmdClearsRegistry |
v0.1 CI |
A reinstall or upgrade merges on plugin_id rather than appending (§8.7 step 2): plugins.lock, plugin-state.json and plugin-catalog.json keep one row per plugin per advertised tool, the surviving lock row carries the new version and the new executable_sha256, and that digest matches the executable actually installed |
TestReinstallMergesRowsAndKeepsNewDigest |
v0.1 CI |
Install writes the §8.7 step 2 initial state: installed_at is an RFC 3339 stamp, activated_at is null, quarantined is false, and status is installed_pending_restart; a manifest declaring needs_user_creds parks in needs_configuration with the flag set, so gum plugin setup is what activates it |
TestInstallWritesSpecInitialState / TestInstallArmsNeedsConfiguration |
v0.1 CI |
Startup activation promotes installed_pending_restart to active with an activated_at stamp, skips quarantined and needs_configuration rows, and writes nothing (no new install_generation, no registry files created) when no row is pending; the promotion runs from the CLI root so gum mcp --stdio and every standalone command get it |
TestPromotePendingRestartActivatesInstalledRow / TestPromotePendingRestartSkipsNeedsConfiguration / TestPromotePendingRestartWithoutPendingRowsSkipsWrite / TestCLIStartupPromotesPendingPlugin / TestCLIStartupLeavesQuarantinedPluginPending / TestCLIStartupDoesNotCreateRegistryFiles |
v0.1 CI |
gum canary --plugin=<id> applies the §8.6 quarantine gate before it spawns: a permanently quarantined plugin and a quarantined plugin with no scheduled next_retry_at are both refused with error_code: "VARIANT_QUARANTINED" and source_error_code: "ErrPluginQuarantined", while a non-quarantined row still reaches the spawn and keeps its own failure code; the canary records no crash, so a diagnostic run does not advance the backoff ladder, and gum plugin reload <id> stays the re-test path that clears a quarantine |
TestCanaryRefusesPermanentlyQuarantinedPlugin / TestCanaryRefusesQuarantinedPluginWithoutRetry / TestCanaryUnquarantinedPluginStillReachesSpawn |
v0.1 CI |
A cached plugin handle whose subprocess session ended is evicted and respawned instead of reused: Plugin.Alive flips when the plugin side closes the session and when Stop tears it down, and plugin.mcp drops the dead handle from its cache before starting a replacement through the supervisor starter, so a failed restart leaves no corpse behind and the §8.6 restart ladder still gates the new spawn |
TestPluginAliveFlipsWhenSessionEnds / TestPluginAliveFalseAfterStop / TestEnsureRunningRestartsDeadPlugin / TestEnsureRunningRestartFailureDropsCacheEntry |
v0.1 CI |
gum://plugins inventory is deterministic and includes a status field per plugin; fixture set MUST include at least one plugin in each status value (active, installed_pending_restart, needs_configuration, quarantined) |
resource read test sorted by plugin name | v0.1 CI |
Gain ledger begins with {"record_type":"header","schema_version":1,"tokenizer":"cl100k_base"} and every dispatch row has record_type:"entry" plus the required v0.1 fields (session, op_id, variant_id, output_profile, args_hash, auth_subject_fingerprint, token counts, cache/field-mask status, served_from_cache, is_retry, op_family, baseline_method); gum_parallel outer rows use the exact sentinel values from §12.3 (output_profile=null, auth_subject_fingerprint="batch", raw_tokens=0, cache_status/field_mask_status="not_applicable", op_family="gum_parallel"); cancelled parallel rows include cancelled: true |
TestGainLedgerHeader / TestGainLedgerEntrySchema / TestGainParallelOuterEntrySchema |
v0.1 CI |
gum.gain and gum gain both read the selected profile's server-local gain-ledger.jsonl; usage.jsonl is not read in v0.1.0; remote/containerized MCP behavior depends on server-side ledger availability, not client filesystem access; disabled or unavailable ledger returns stable isError=true tool errors with GAIN_DISABLED or GAIN_LEDGER_UNAVAILABLE respectively and no GainResult structuredContent |
TestGumGainReportsRealDispatch / TestDefaultDispatcherWiresGainLedger / TestGainLedgerHonorsProfileDataDir / TestGainDisabledSkipsLedger / TestGainDisabledReturnsTerminalError / TestGainLedgerUnavailableReturnsTerminalError / TestGainConfigOptOutReturnsGainDisabled |
v0.1 CI |
Release-gated >=80% savings uses end-to-end ledger totals including gum_parallel outer entries; the test MUST verify batch_id linkage (outer entry batch_id equals all inner entries' batch_id), element_count equals the number of inner entries, end_to_end_savings includes outer envelope overhead, batch_envelope_overhead matches outer-entry token contribution, per_op_shaping_savings is diagnostic only, GainResult validates its mode discriminator and exactly-one-array summary/session/history schema branches, and outer entry variant_id is null (per §12.3 intentional design — a parallel dispatch has no single variant) |
TestGainEndToEndSavingsIncludesBatchEnvelope / TestGainEndToEndSavingsExcludesEstimatedEntries / TestGainOutputSchema / TestGainParallelOuterEntrySchema / TestGainSummarySessionsAggregatePerSession / TestGumParallelInnerInvocationsCarryBatchLinkage / TestRecordParallelBatchWritesOuterSentinel |
Release gate |
The release fixture set clears two savings floors measured on the same replay: a like-for-like registration comparison where both sides carry only name, description and inputSchema (>=0.84), and the full-wire comparison that charges GUM every byte its tools/list reply sends including output schemas (>=0.80, the spec §2 normative claim). The two differ because the §2 baseline returns the raw upstream body unchanged and so declares no output schema, and the catalog carries no response_ref to synthesise one from; splitting the gates keeps output-schema growth from reading as a shaping regression |
TestGainReleaseFixtureSavingsFloor |
Release gate |
gum gain --session <ID> filters by the local ledger session field and emits GainResult.operations[] grouped by op_id x cache_status x field_mask_status; gum gain --since <RFC3339> applies a UTC lower-bound filter and relabels window; gum gain --history groups JSON output by session and op_family into GainResult.history[] in ledger order; gum gain --exclude-retries drops is_retry=true entries from the reported aggregate and leaves the ledger file unchanged; each mode emits its own array and omits the other two; no mode flag is summary mode; --format is one enum text|json|csv|toon that defaults to text and renders every mode, csv emits one row per detail row, and toon is rejected with CLI_ARG_INVALID outside --fixture-replay, where --format selects the shaping format measured and defaults to toon |
TestGainSessionHistoryAndRetryFilters / TestGainDefaultsToTheTextRenderer / TestGainSummaryModeEmitsSessions / TestGainTextRendersEveryMode / TestGainCSVEmitsOneRowPerDetailRow / TestGainByOpCSVCarriesEveryOp / TestGainFormatEnumIsEnforced / TestGainFixtureReplayKeepsToonAsItsDefault |
v0.1 CI |
Diff-only mode 304 response short-circuits the expression pipeline; ledger records cache_status: "etag_304" and served_from_cache: true with response_tokens: 0; _expression is absent in the 304 response; a second call with the same op_id and args_canonical but a different resolved variant_id or different auth_subject_fingerprint MUST NOT produce a 304 (different cache key) |
TestDiffOnlyModeEtagReplay |
v0.1 CI |
gum.code enforces cumulative output byte budget across prints and final return |
TestCodeOutputBudget |
v0.1 CI |
gum.code destructive fan-out is bounded across MCP and CLI: confirmed destructive scripts require destructive_budget / --destructive-budget in 1..20, consume one unit before each destructive call, reject over-budget calls with DESTRUCTIVE_BUDGET_EXCEEDED, reject calls outside destructive_scope / --destructive-scope with DESTRUCTIVE_SCOPE_MISMATCH, consume each gum_confirm_destructive(op_id, resource_key?) on the immediately following destructive call only, reject v0.1 script-header pragmas as unsupported rather than parsing them silently, and make no upstream request on any rejection path |
TestCodeDestructiveBudgetAbortsAtNplus1 / TestCodeDestructiveScopeEnforced / TestCodeDestructiveBudgetNoUpstreamOnExceeded / TestCodeRejectsScriptHeaderPragma |
v0.1 CI |
Expression profiles validate; recovery="resource_link" with tee_mode!="always" fails with PROFILE_TEE_MODE_CONFLICT before dispatch |
gum profile validate; JSON Schema fixture tests; TestProfileTeeModeConflict |
v0.1 CI |
| Expression-profile fixtures meet token budgets | gum profile test with cl100k_base |
Release gate |
The semantic profile validator runs alongside the structural parser: on_empty over 500 codepoints after NFC normalization fails with ON_EMPTY_TOO_LONG, recovery="resource_link" with tee_mode="off"/"failures" fails with PROFILE_TEE_MODE_CONFLICT, both codes are reported together, gum profile test runs the same gate, the §9.2 resolver rejects a bad project-local or user-global file before it shapes anything, Parse stores on_empty in NFC form, and gum profile validate --variant binds a catalog variant so PROFILE_STRIP_NULLS_UNSAFE can run (an unbound run says it skipped, an unknown variant fails with VARIANT_NOT_FOUND) |
TestValidateSemanticsRejectsLongOnEmpty / TestValidateSemanticsAcceptsExactly500 / TestOnEmptyLengthCountsAfterNFC / TestParseNormalizesOnEmpty / TestValidateSemanticsRejectsResourceLinkWithoutAlwaysTee / TestValidateSemanticsAcceptsResourceLinkWithAlwaysOrDefault / TestValidateSemanticsReportsBothFailures / TestResolveProfileRunsTheSemanticValidator / TestResolveProfileUserGlobalRunsTheSemanticValidator / TestBuiltinProfilesPassSemanticValidation / TestBuiltinProfilesPassStripNullsSafetyForTheirVariant / TestProfileValidateRejectsLongOnEmpty / TestProfileValidateAcceptsResourceLinkWithAlwaysTee / TestProfileValidateStripNullsNeedsAVariant / TestProfileValidateSkipsStripNullsWithoutAVariant / TestProfileValidateUnknownVariant / TestProfileTestRunsTheSemanticValidator |
v0.1 CI |
Step 9 actually appends: the default CLI dispatcher opens the profile's gain-ledger.jsonl and passes it to the kernel, so a real gum dispatch leaves one header line plus one entry line carrying all §12.3 keys; a dispatcher built for a profile whose data dir cannot be opened still dispatches |
TestDefaultDispatcherWiresGainLedger / TestGumGainReportsRealDispatch |
v0.1 CI |
Ledger location and permissions: the default path is <XDG_DATA_HOME|~/.local/share>/gum/<profile>/gain-ledger.jsonl, one file per profile, created mode 600 inside a mode 700 directory; an existing looser file is tightened to 600 on open |
TestNewLedgerDefaultPathHonorsXDGDataHome / TestDefaultPathIsPerProfile / TestGainLedgerHonorsProfileDataDir / TestLedgerFileIsMode600 / TestExistingLedgerIsTightenedOnOpen |
v0.1 CI |
Step 9 entry mapping from a live dispatch: token counts come from the canonical args and from the raw and shaped bodies, cache_status distinguishes hit / miss / not_applicable (an uncacheable variant is not a miss), field_mask_status distinguishes applied / skipped / not_applicable, is_retry is set for the same session + op_family + args_hash inside 5 minutes, and a ledger write error is logged without failing the dispatch |
TestRecordAndReturnLedgerEntryPopulated / TestRecordAndReturnCacheHitFlagged / TestRecordAndReturnCacheIneligibleIsNotAMiss / TestRecordAndReturnFieldMaskStatus / TestRecordAndReturnMarksRetry / TestRecordAndReturnLedgerErrorDoesNotFailDispatch / TestRecordAndReturnNilLedgerNoop |
v0.1 CI |
Gain opt-out is per profile and covers both the writer and the reader: gum config set gain.enabled=false stops the step-9 append and makes gum.gain return GAIN_DISABLED; GUM_GAIN_DISABLED=1 is the process-scoped override; a missing key, an unparseable value, or an unreadable config leaves accounting on, and disabling one profile does not disable another |
TestGainConfigOptOutReturnsGainDisabled / TestGainConfigTrueKeepsReporting / TestGainConfigGarbageValueKeepsReporting / TestGainDisabledSkipsLedger / TestEnabledDefaultsOnWithNoConfig / TestEnabledHonorsConfigKey / TestEnabledEnvOverrideWins / TestEnabledIsPerProfile / TestEnabledUnreadableConfigKeepsAccounting |
v0.1 CI |
| Catalog ABI versions reject unsupported future artifacts | loader tests for CATALOG_SCHEMA_UNSUPPORTED, PLUGIN_MANIFEST_SCHEMA_UNSUPPORTED, PLUGIN_CATALOG_SCHEMA_UNSUPPORTED, PLUGIN_LOCK_SCHEMA_UNSUPPORTED, and PLUGIN_STATE_SCHEMA_UNSUPPORTED |
v0.1 CI |
Third-party plugin manifests use top-level manifest_schema_version only: a missing version, a version nested under plugin, and a nested version sitting beside a valid top-level one all fail install with PLUGIN_MANIFEST_SCHEMA_UNSUPPORTED, and so do explicit 0 and 9999. The v0.1.0 compatibility default for bundled development manifests has no implementation and no inputs, because gum ships no bundled plugin and none is planned (gum-wzmb) |
TestPluginManifestSchemaVersionPlacement / TestPluginManifestVersionAccepted / TestPluginManifestVersionRejectedTooOld / TestPluginManifestVersionRejectedTooNew |
v0.1 CI |
service_root_template is rejected before v0.4.0 with SERVICE_ROOT_TEMPLATE_DEFERRED; v0.1-v0.3 dispatch uses discovery-derived root metadata only and never advertises sovereign/government/private-service-connect endpoint variants as executable |
TestServiceRootTemplateDeferred |
v0.1 CI |
Embedded catalog.json integrity is verified against a committed SHA256 digest in catalog.json.sha256; cmd/gen-catalog writes both files in lockstep so silent disk corruption, accidental edits, or unauthorized modification fail the build |
TestCatalogIntegrity / TestCatalogChecksumFileFormat |
v0.1 CI |
| JCS canonicalization is stable | RFC 8785 test vectors in internal/cache/canonical_test.go |
v0.1 CI |
| Audit log rotation and recovery are deterministic | rotation, ENOENT retry, audit.broken sentinel tests |
v0.1 CI |
| CI runs on the Go floor and current stable Go toolchains | CI matrix: go1.25.x and current stable |
v0.1 CI |
Easy auth contract: gum init runs auth readiness and launches/prints the default gum auth login path when no profile credential exists; GUM-managed browser OAuth uses built-in client ID + PKCE + loopback + CSRF state, never commits OAuth client secrets to source, and treats release-injected Desktop client material as public client material rather than a confidential secret; refresh tokens and plugin secrets are stored only in OS keychain; profile config stores only non-secret metadata; interactive desktop profiles do not silently consume ambient GOOGLE_APPLICATION_CREDENTIALS unless gum auth use-adc was run; AUTH_REQUIRED and SCOPE_MISSING user messages point gum_oauth users to gum auth login ..., but non-gum_oauth strategies point to gum auth setup <op_id>; CLI/MCP/code mode reuse the same selected-profile credential on the GUM host |
TestAuthHappyPathNoUserClientSecret / TestAuthKeychainStorageOnly / TestAuthNoAmbientADCWithoutOptIn / TestAuthErrorNextAction / TestAuthLoopbackStateRequired |
v0.1 CI |
Bundled OAuth scope allowlist is enforced for variants that use auth_strategy="gum_oauth": apps/gum/internal/embedded/data/auth-managed-scopes.v1.json is the only source of eligible scopes; only status="active" + verification_state="verified" + project_evidence_state="ready" + live_canary_state="passing" scopes count; planned scopes are not requested; generator rejects a gum_oauth variant requiring any out-of-manifest or unverified scope with GUM_OAUTH_SCOPE_NOT_MANAGED; release gate verifies every active restricted/sensitive scope has an evidence pointer and every active scope has project, token-exchange, and refresh-canary evidence, otherwise GUM_OAUTH_MANAGED_CLIENT_NOT_READY |
TestManagedOAuthScopeManifest / TestGumOAuthScopeNotManaged / TestManagedScopeVerificationEvidence / TestManagedOAuthProjectReadiness / TestManagedOAuthLiveCanaryRequired |
v0.1 CI and release |
Testing-window opt-in gate (§7 transitional): a scope is eligible for gum_oauth before full promotion only when managed_project.publishing_status == "testing" and the scope sets testing_allowed: true; the opt-in is per-scope (flipping the project to testing never auto-exposes non-opted-in scopes) and inert under any other publishing status (production stays strict by default); embedded_client_secret=true still fails GUM_OAUTH_MANIFEST_INVALID at the gate even with eligible scopes |
TestCanStartGumOAuthTestingModeAllowsOptedInScope / TestCanStartGumOAuthTestingModeRequiresOptIn / TestCanStartGumOAuthTestingFlagInertOutsideTestingStatus / TestCanStartGumOAuthEmbeddedSecretRejected |
v0.1 CI |
| Managed OAuth scope expansion is full re-consent, not implicit incremental append: requesting additional scopes builds the complete desired managed scope set, verifies granted scopes and subject fingerprint before replacing the keychain credential, rejects subject mismatch without storing tokens, and returns BYO/compound setup when any requested scope is not managed-ready | TestAuthScopeUpgradeFullReconsent / TestManagedLoginRefusesASecondAccountForTheSameScopes / TestManagedLoginSwitchesAccountsWhenAsked / TestManagedOAuthLiveCanaryRequired / TestCanStartGumOAuthTestingFlagInertOutsideTestingStatus |
v0.1 CI |
Auth requirement taxonomy is enforced: every executable variant/plugin declares one auth_strategy; unknown non-x-* auth components fail build/install with AUTH_COMPONENT_UNKNOWN; non-gum_oauth variants emit errors with auth_strategy, missing_components, and setup_command; fixture includes a Google Ads Keyword Planner-like compound operation requiring developer token, OAuth client/client secret or refresh token, customer ID, optional login customer ID, billing/account prerequisites, account-permission checks, and approved access/permissible-use/allowlist hints, and proves plain gum auth login is not suggested as sufficient |
TestAuthStrategyRequired / TestAuthComponentUnknown / TestCompoundAuthErrorEnvelope / TestCompoundAuthReportsRealMissingComponents / TestCompoundComponentsDeduplicateAndKeepOrder / TestCompoundComponentsCarryNoSecretValues |
v0.1 CI |
BYO OAuth grant storage (spec §7 "BYO grant storage"): refresh tokens are keyed per sha256(client_id) with the granted-scope set stored alongside; a stored grant is reused when it is a superset of the requested scopes (broad gum login satisfies narrow per-op resolves), an uncovered scope routes to NO_REFRESH_TOKEN carrying the op's full scopes, separate per-op authorizations union into one grant (no clobber), and distinct client_ids stay isolated |
TestByoOAuthBroadGrantSatisfiesNarrowResolve / TestByoOAuthMissingScopeForcesReauth / TestByoOAuthGrantUnionAccumulates / TestByoOAuthPerClientIsolation |
v0.1 CI |
The byo_oauth auth_subject_fingerprint names the account, not the token (spec §10.0.1): the login flow adds openid to the requested scopes, reads the account principal from the id_token (lower-case email, else the sub claim), and stores it on the grant, so re-login for the same account keeps one fingerprint even though the refresh token rotated; the silent refresh path derives the same value from the stored principal; two accounts under one OAuth client never share a fingerprint; email case is normalized; a grant with no stored principal falls back to the refresh token rather than collapsing every account into one partition |
TestByoOAuthFingerprintSurvivesReLogin / TestByoOAuthFingerprintDiffersPerAccount / TestByoOAuthFingerprintNormalizesEmailCase / TestByoOAuthLegacyGrantFallsBackToRefreshToken |
v0.1 CI |
A profile records the auth_subject_fingerprint of the account each auth strategy signed in with, under auth.expected_subject.<auth_strategy>; once one is recorded, an interactive login whose consent returns a different account is refused and stores nothing, dispatch step 5 refuses a credential resolved under that strategy with a different fingerprint using AUTH_SUBJECT_MISMATCH and emits the §10.0.1 credential_subject_changed audit event before any upstream request, a strategy with no recorded subject and a credential carrying no subject are both unchecked, and gum login --switch-account is the deliberate rebind |
TestByoLoginRefusesASubjectTheProfileDidNotExpect / TestDispatchRefusesACredentialFromAnotherAccount / TestDispatchKeepsSubjectExpectationsPerStrategy / TestDispatchIgnoresAnEmptyFingerprint / TestLoginSwitchAccountRebindsTheProfile |
v0.1 CI |
The shared CompositeResolver never mutates itself on the resolve path: eight goroutines resolving a gum_oauth variant against one unwired resolver leave GumOAuth nil and trip no race detector, so the dispatcher may hold a single resolver across concurrent invocations (gum_parallel, the MCP server) |
TestResolveAuthGumOAuthConcurrentIsRaceFree |
v0.1 CI |
v0.1.0 login surface + just-in-time auth (spec §7 "v0.1.0 login surface", "Just-in-time authorization"): gum auth login / gum login run the BYO loopback flow against the registered client, pre-authorizing the lean core Workspace scope set when no --scope is given, with --service and --all as the opt-in widenings; the first gum call on an unauthorized byo_oauth variant prompts a TTY operator Authorize <scope>? [Y/n] then retries once on assent, while agents/pipes fall through to structured AUTH_REQUIRED (no gcloud dependency) |
TestRunLoginWithConfiguredClientRunsFlow / TestResolveLoginScopesDefaultIsCore / TestResolveLoginScopesServiceSelects / TestResolveLoginScopesAllUnion / TestResolveLoginScopesEmbeddedCatalogNonEmpty / TestTopLevelLoginAliasRegistered / TestMaybeJITLoginAccepted / TestMaybeJITLoginNotInteractive / TestCallRetriesAfterJITLogin |
v0.1 CI |
BYO-only public auth posture: gum login and the JIT byo_oauth resolver require an operator-registered OAuth client. Injected bundled-client values do not satisfy byo_oauth, and the public auth CLI does not register managed-status. |
TestResolveAuthIgnoresInjectedManagedClient / TestResolveAuthRequiresBYOForAllScopes / TestResolveAuthNoManagedClientStillNotConfigured / TestRunLoginIgnoresInjectedManagedClient / TestAuthManagedStatusNotRegisteredForV1 / TestManagedSupportedScopesIncludesSearchConsoleReadonly |
v1 CI |
Workspace and account-policy prerequisites are actionable pre-flight: a compound-auth variant declaring workspace_admin_trust or org_policy_exception is refused before the upstream call with AUTH_REQUIRED, those kinds in missing_components, retryable=false and an op-scoped setup_command, never a retry-login loop. Optional components are reported too, because nothing resolves them per account. An upstream refusal maps to the same components: a 403 whose Google reason is USER_BLOCKED_BY_ADMIN, Drive's legacy domainPolicy, or one of the three org-policy reasons carries the matching kind out of UpstreamError.AsStructuredError with retryable=false and an op-scoped setup_command, while an unmapped reason, a non-403 status and a 429 all keep their existing shape. admin_policy_enforced is not in that table: it is an OAuth authorization-endpoint error, not a google.api.ErrorReason, so it cannot appear in a response body. No shipped catalog variant pre-declares either kind, because whether an administrator trusts the client varies per domain and a static declaration would report a gap to administrators who have none. The §6.1.2 confirmation token stays profile-scoped rather than principal-scoped, because confirmation runs in step 2 and credentials resolve in step 4; the wrong-account defence is the step-4 AUTH_SUBJECT_MISMATCH refusal, which fires when the profile recorded a subject for that auth strategy |
TestWorkspaceAdminPolicyAuthEnvelope / TestUpstreamAdminBlockCarriesWorkspaceAdminTrust / TestUpstreamUnmappedReasonKeepsGenericShape / TestDispatchKeepsUpstreamPolicyRefusalEnvelope / TestConfirmationTokenIsProfileScopedNotPrincipalScoped |
v0.1 CI |
Plugin credential setup is centralized: manifests with credential descriptors and auth components can be configured through gum plugin setup <name>; setup stores secret components in the OS keychain, exposes only descriptor aliases/display names in resources and errors, displays external prerequisites as checklist items, runs the live canary after credentials are supplied, and clears needs_configuration only on canary success; a passing canary also clears the supervisor quarantine bookkeeping (quarantined, quarantined_at, last_error_code, next_retry_at, retry_count, backoff_step, permanent_quarantine) that Supervisor.Start gates on, so the next spawn is not refused after setup reports the plugin activated; raw env var names and secret values are not emitted in MCP resources |
TestPluginSetupCredentialFlow / TestPluginCredentialNoRawEnvLeak / TestPluginExternalPrerequisiteChecklist / TestPluginSetupChecklistWithoutSecrets / TestManifestRejectsUnknownAuthComponent / TestSetupSuccessClearsQuarantine |
v0.1 CI |
The credentials gum plugin setup stored reach the plugin subprocess: Host.Start resolves every §8.2 credential descriptor from the profile keychain under PluginCredentialKey(profile, plugin, alias) and injects it under the descriptor's raw env name; a requirements.needs_user_creds name with no stored secret falls back to the ambient value; a stored secret wins over a stale ambient one; keychain entries are profile-scoped, so a work credential never reaches a default spawn; the §8.1 denylist applies to keychain-sourced values too |
TestPluginSpawnInjectsStoredCredential / TestPluginSpawnPassesAmbientCredential / TestPluginSpawnStoredCredentialBeatsAmbient / TestPluginSpawnKeyringIsProfileScoped / TestBuildSubprocessEnvDeniedCredentialDropped / TestBuildSubprocessEnvCredentialBeatsEnvAllow |
v0.1 CI |
CLI gum call argument grammar parses typed JSON, repeated arrays, @file, stdin, and dotted-key escaping deterministically; every call requires --risk=read|write|destructive; host-control flags (--fields, --page-size, --page-token, boolean output flags) are not aliases for positional operation args and cannot be overridden by them; v0.1 rejects gum call --format while allowing reporting subcommands such as gum gain --format; duplicate output-format flags fail with CLI_ARG_DUPLICATE; --variant-id selects a non-default active variant and unknown/removed/quarantined/installed_pending_restart/needs_configuration variants fail before upstream dispatch; mismatched resolved variant risk returns RISK_TOOL_MISMATCH; destructive plus confirmation_policy="high_stakes_write" calls require --yes or an interactive TTY confirmation before dispatch |
TestCLIArgGrammar / TestCLIRiskGate / TestCLIVariantSelection |
v0.1 CI |
CLI gum code confirmation grammar is exact: --allow-write or --allow-destructive requires interactive y or non-interactive --yes; read-only code requires neither; repeated --destructive-scope op_id[:resource_key] is the only v0.1 scope grammar; --no-confirm, script-header pragmas, and --lang fail parsing or validation rather than being silently accepted |
TestCLICodeConfirmationAndScopeGrammar |
v0.1 CI |
Automation-safe read-only/reporting CLI commands support stable --format=json roots from §12 (search, describe, plugin list, plugin info, catalog list-overrides, cache stats, profile validate, profile test, gain) and golden tests reject drift in JSON field names |
TestCacheStatsFormatJSONHasRequiredKeys / TestSearchJSONEnvelopeMatchesSpec / TestCatalogListOverridesJSONOutputIsValidNDJSON / TestPluginListJSONRowsMirrorTheResourceColumns / TestPluginListJSONEmptyProfileIsValidJSON / TestPluginListRejectsUnknownFormat |
v0.1 CI |
strip_nulls=true is rejected unless null_elision_safe_fields covers elided fields |
TestProfileStripNullsSafety |
v0.1 CI |
cmd/gen-catalog is the spec §7 Build firing point for ON_EMPTY_TOO_LONG, PROFILE_TEE_MODE_CONFLICT and PROFILE_STRIP_NULLS_UNSAFE: every generator path runs one gate that resolves each variant's output_profile against the built-in profile set, fails a name resolving to nothing, and runs the bound-profile checks with that variant's null_elision_safe_fields; the shipped catalog passes the gate |
TestEmbeddedCatalogPassesTheBuildProfileGate / TestValidateVariantProfilesRejectsDanglingName / TestValidateVariantProfilesRunsStripNullsSafety / TestValidateVariantProfilesSkipsUnsetNames |
v0.1 CI |
field_mask_mode="dual_fetch" is rejected for every write/destructive variant and every non-idempotent read variant |
TestDualFetchReadOnlyIdempotentGate / TestDualFetchGateRejectsNonIdempotentVariant |
v0.1 CI |
field_mask_mode="dual_fetch" on an eligible variant makes two upstream requests: the first carries the profile mask, the second carries no projection, and §9.0 stage 9 writes the unmasked body rather than the shaped one |
TestDualFetchIssuesSecondUnmaskedRequest / TestDualFetchArtifactHoldsUnmaskedBody / TestDualFetchLeavesInvocationArgsIntact |
v0.1 CI |
Both dual_fetch requests count against the §11 audit log, in request order, and only the unmasked one carries dual_fetch: true; the two entries differ by args_hash, and a dispatch that never selected the mode omits the key entirely |
TestDualFetchAuditRecordsSecondRequestOnly / TestDualFetchKeyAbsentWithoutMode |
v0.1 CI |
The dual_fetch second request is skipped when it would buy nothing: no mask reached the wire (including --no-field-mask), or no stage-9 artifact would be written under tee_mode off or failures |
TestDualFetchSkippedWithoutMaskOnWire / TestDualFetchSkippedWhenNoArtifactWouldBeWritten |
v0.1 CI |
A failed dual_fetch second request does not fail the invocation and does not fall back to teeing the masked body: the shaped response is returned, no artifact is written, a validation warning names the failure, and no dual_fetch audit entry is emitted. A §10.3 cache hit still issues one unmasked request, because the cache key carries the mask |
TestDualFetchFailureKeepsPrimaryResponse / TestDualFetchOnCacheHitStillRecovers |
v0.1 CI |
Stage 8 encodes the format it reports: format="csv" produces CSV rows and format="markdown" produces a markdown table, and an unrecognised format name encodes TOON and is reported as toon rather than echoed back (§9.1 stage 8) |
TestApplyEncodesCSV / TestApplyEncodesMarkdown / TestApplyUnknownFormatStillEncodesTOON / TestApplyDefaultFormatFallback |
v0.1 CI |
The MCP seam returns format="csv" and format="markdown" as SingleObjectResult whose data is the stage-8 text, not the shaped tree, and a csv result carries no toon key (§13) |
TestCSVResultCarriesCSVText / TestMarkdownResultCarriesMarkdownText |
v0.1 CI |
Stage 8 encodes, it does not filter (§9.1 stage 8): the CSV encoder wrote only the primary record table, so a top-level nextPageToken and every secondary array were deleted with nothing recorded in lossy, dropped_paths or the notice; they are now appended as envelope columns repeated on each row, emitted alone when the page holds no records, and disambiguated with a _ prefix when a name collides with a record column. The markdown encoder cut every cell at 60 terminal columns, a second truncation below the limit stage 6 truncate_strings had already applied; only the ASCII table, which is a terminal layout, still caps cell width |
TestRenderCSVKeepsTopLevelScalars / TestRenderCSVKeepsSecondaryArrayCounts / TestRenderCSVEnvelopeSurvivesZeroRows / TestRenderCSVEnvelopeNameCollision / TestRenderCSVUnchangedWithoutAnEnvelope / TestRenderMarkdownDoesNotTruncate / TestRenderCSVDoesNotTruncate / TestRenderTableStillTruncates |
v0.1 CI |
A TOON result names the variant that produced it: the §13 ToonResult envelope carries the resolved op and variant as top-level keys, surfaces neither count nor fields there, and the variant id it reports is an exact key into the catalog snapshot, resolving to exactly one variant while a near-miss id resolves to none. The toon body is a §9.0 document: DecodeTOONDocument parses it, its op: and variant: headers repeat the resolved ids, format_version is 1, count equals the number of rows, fields is the sorted union of the record keys, and the first body line is a data row rather than a repeated CSV header |
TestToonVariantHeader |
v0.1 CI |
TOON Decode splits a document into logical lines, so a raw newline inside a CSV-quoted value is data and not a line break: a map value containing "\n" round-trips byte-for-byte whether or not its tail contains = or ,, a quoted key holding = still splits at the delimiter after its closing quote, a doubled "" escape survives the split, and a bare scalar string containing "\n" decodes back to that string instead of reaching the CSV-table branch (§9 lossless-shaping requirement) |
TestDecodeKeyValueKeepsNewlineInValue / TestDecodeScalarKeepsNewline |
v0.1 CI |
TOON in-tree parser round-trips representative fixtures (list result with nulls, quoted CSV fields, zero omitted_count, empty body) losslessly |
TestToonRoundTrip |
v0.1 CI |
The TOON {} sentinel belongs to a map with no fields only: a map that has fields keeps every key even when all values are empty, so {"error":"","status":null} round-trips instead of collapsing and losing both names with no lossy flag (spec "Null representation (normative)") |
TestEncodeAllEmptyMapKeepsKeys / TestEncodeGenuinelyEmptyMapIsSentinel / TestEncodeEmptyObjectSentinel |
v0.1 CI |
RESULT_ARTIFACT_EXPIRED envelope returned on resources/read of a deleted gum://results/<hash> artifact as a JSON-RPC application error with error.code=-32010 and exact §7 error.data fields: error_code, hash, uri, expires_at, user_message, and suggestion |
TestResultArtifactExpiredError |
v0.1 CI |
Generated REST dispatch stubs propagate context (.Context(ctx) before .Do()); cancelling context aborts in-flight HTTP within 100ms |
TestExecutorContextPropagation |
v0.1 CI |
Long-tail raw REST unknown-argument handling is fail-closed by default for every risk class; read-only allowlist pass-through applies only to explicitly configured discovery-rest/raw-http variants, emits _validation_warnings, and is ignored for write/destructive, typed SDK, gRPC, and plugin backends |
TestLongTailUnknownArgHandling |
v0.1 CI |
Any PR adding a new backend_kind value includes a fixture-backed executor contract test |
TestBackendKind<Name> |
Same PR |
google-ads-sdk executes Google Ads Keyword Planner POST custom-methods (customers/{id}:generate*), injecting the secret developer-token header server-side (never an invocation arg) alongside the byo_oauth Bearer and optional login-customer-id |
TestBackendKindGoogleAdsSDK |
v0.1 CI |
Data Manager catalog entries declare the separate BYO OAuth scope, ingestion uses POST with write risk, and request-status retrieval uses GET with a required requestId query argument |
TestBuildDataManagerOps / TestInjectDataManagerOffline / TestDataManagerWireAndScope |
v1.3 CI |
Conflicting or malformed boolean body fields fail locally before ingestion; an explicit body.validateOnly cannot silently override contradictory flat validation intent |
TestDataManagerValidateOnlyConflict / TestAssembleRequestBodyBooleanSafety |
v1.3 CI |
| Typed REST POST/PATCH transport, response-read, and size-limit failures do not replay writes, including after a 429 retry; eligible GET failures retain their retry behavior | TestTypedRestUncertainWriteIsNotReplayed / TestDataManagerReadFailureRetry |
v1.3 CI |
| Data Manager errors retain request IDs and field violations; ingestion warnings and per-destination processing counts survive default, JSON, TOON, and raw output | TestDataManagerValidationDetails / TestDataManagerResponseShaping |
v1.3 CI |
Any PR adding a new interface_kind value includes a fixture-backed interface contract test |
TestInterfaceKind<Name> |
Same PR |
Any PR adding a new grpc-sdk or sdk-native adapter_key includes a binding-schema fixture and adapter-registry contract test proving the binding resolves without ad hoc code in the catalog loader |
TestBackendBinding<Name> |
Same PR |
Plugin variants materialize explicit backend binding objects: mcp-plugin requires tool_name, and bundled grpc-plugin ABI fixtures require rpc_service plus rpc_method; missing or malformed selector fields fail with PLUGIN_BINDING_INVALID before subprocess start unless the third-party Shape 2 install gate applies first |
TestPluginBindingSchema |
v0.1 CI |
Third-party Shape 2 manifests are rejected before v0.4.0: [plugin].shape="grpc-subprocess" or any backend_kind="grpc-plugin" in a third-party install fails with PLUGIN_SHAPE_UNSUPPORTED before binding selector validation, schema copy, executable staging, canary, or registry writes; malformed third-party Shape 2 selectors still return PLUGIN_SHAPE_UNSUPPORTED, while bundled ABI fixtures continue to use PLUGIN_BINDING_INVALID for selector failures |
TestThirdPartyShape2InstallRejected |
v0.1 CI |
Third-party plugin namespace ownership is stable and profile-scoped: manifests require namespace_owner, install records prefix ownership in the selected profile's plugins.lock, matching owners may upgrade/reinstall within that profile, mismatched owners fail with PLUGIN_NAMESPACE_CONFLICT, cross-profile locks never merge, and --dev-allow-namespace-conflict is rejected outside dev profiles |
TestPluginNamespaceOwnership |
v0.1 CI |
Any PR promoting a capability class from schema_only to executable (or adding a new executable atom) includes TestCapabilityClass<Name> and updates this matrix |
TestCapabilityClass<Name> |
Same PR |
code_execution (§5.8) is the one atom the generic dispatcher never runs and a typed executor does: the shipped gum.code.v1.risor variant declares that atom alone, carries no binding.http, binds the code.risor adapter, stays execution_support: "full" because the atom executes, and runs a script through CodeRunner.Execute |
TestCapabilityClassCodeExecution |
v2.2 CI |
The §927 capability gate refuses before the upstream request and the §932 warning rides the envelope: a variant whose execution_support is typed_executor_required or schema_only fails with UNSUPPORTED_CAPABILITY carrying the declared unsupported_capabilities and the remediation suggestion, with no status key (§5.8 two-variant mutual exclusion) and no adapter call; full, partial, and an absent value all dispatch; a partial variant succeeds and its envelope carries _expression._unsupported_capabilities, which a full variant omits; gum.describe_op renders the same atoms as capability_class_warnings prose and omits the key when nothing is blocked |
TestCapabilityGateRefusesBeforeTheUpstreamRequest / TestCapabilityGateLetsFullAndPartialThrough / TestPartialVariantWarnsInTheExpressionEnvelope / TestFullVariantCarriesNoCapabilityWarning / TestDescribeOpResultWarnsOnBlockedCapabilityClasses / TestDescribeOpResultOmitsWarningsWhenNothingIsBlocked |
v2.2 CI |
Every shipped variant declares capabilities[] from the §5.8 closed enum: gen-catalog -apply-capabilities derives the six generic atoms offline from each op's request record and HTTP binding, carries through atoms it does not derive (so the Discovery lro_return stamp survives a rerun), and applies the curated table for the variants whose declared atoms contradict what the adapters run. Exactly two shipped variants are non-full: drive.v3.rest.files.get and drive.v3.rest.files.export, both partial on media_download, because the typed-rest-sdk adapter stamps Format: "json" on every response and cannot return exported or downloaded bytes |
TestCatalogDeclaresCapabilities / TestCatalogCuratedExecutionSupport |
v2.2 CI |
Any PR adding or removing a language closed-enum value updates §4.3, §6.2, §12 CLI help, §13 completions, dependency floors as needed, and the TestMCPCompletions fixture in the same PR; any PR adding or removing a format value also updates docs/expression-profile-dsl.md, docs/expression-profile-dsl.json, and format fixtures |
TestMCPCompletions (extended, via assertSpecClosedEnumCompletions) |
Same PR |
Managed-scope re-consent is elicitation-gated and binding-checked (§13): a SCOPE_MISSING refusal on a byo_oauth operation returns exactly one approval form, whose form id, op_id, sorted and deduplicated required scopes, profile, expected subject fingerprint, and request hash all bind the refused call; the form is sent only to a client that declared the elicitation capability, and only when a consent login is wired; an approval whose bindings differ from the pending request, a reply under another form id, a decline, a cancel, and approve=false each store nothing and leave the original SCOPE_MISSING envelope in place; a login that returns a partial grant or a different auth_subject_fingerprint stores nothing and keeps the prior grant byte-identical; every outcome writes exactly one managed_scope_reconsent audit event carrying the requested and granted scopes, the profile, op_id, request_hash, and the resulting fingerprint; a granted upgrade returns SCOPE_GRANTED and does not re-run the operation |
TestElicitationScopeUpgradeBinding / TestScopeUpgradeApprovalObject / TestScopeUpgradeApplyGrants / TestScopeUpgradeRefusals / TestScopeUpgradeConcurrentAllowlist / TestByoOAuthRequiredScopes |
v2.2 CI |
The MCP risk tool is gated on the variant that will execute, not on the op's default variant: gum.read with a pinned write or destructive variant_id and gum.write with a pinned destructive variant_id both fail with RISK_TOOL_MISMATCH before any upstream call, and a risk field supplied in the caller's args never lowers the tier the gate applies |
TestGumReadRejectsPinnedHigherRiskVariant / TestGumWriteRejectsPinnedDestructiveVariant / TestRiskTierIgnoresCallerRiskFlags |
v0.1 CI |
tee_mode="failures" writes an artifact when the upstream call fails, covering both an HTTP 5xx carrying a body and a transport error carrying none; tee_mode="always" also writes on failure; tee_mode="off" writes nothing; a failure raised before the executor writes nothing. The upstream error keeps the raw response body, so the artifact holds what the server actually returned rather than a re-rendered summary |
TestTeeFiresOnUpstream5xxWhenModeFailures / TestTeeFiresOnTransportErrorWithoutBody / TestTeeFiresOnFailureWhenModeAlways / TestTeeSkippedOnFailureWhenModeOff / TestTeeSkippedOnPreUpstreamError / TestUpstreamErrorCarriesRawBody |
v0.1 CI |
The _expression envelope reaches the caller on every shaped path, CLI and MCP alike: it reports the shaped result_count and the omitted count, is present on an empty result, carries intentional_zero_max_items when max_items=0 produced the empty set, and is the documented sentinel on a raw pass-through. MCP structuredContent carries the shaped value, not the unshaped tree, and validates against the §13 $defs that each tool registers |
TestExpressionEnvelopeReportsShapedCounts / TestExpressionEnvelopeCarriesOnEmptyMessage / TestExpressionEnvelopeFlagsIntentionalZero / TestExpressionEnvelopeOnRawIsSentinel / TestStructuredContentCarriesShapedValue / TestDispatchAndShapeStructuredContentValidates / TestRegisteredResultSchemasMatchSpecDefs / TestSearchJSONEnvelopeMatchesSpec |
v0.1 CI |
gum code CLI destructive flags satisfy the same gate the adapter enforces: --destructive-budget and --destructive-scope are rejected without --allow-destructive, a malformed scope entry is rejected before dispatch, the parsed budget and canonical scope reach the invocation unchanged, and the scope list is capped at its documented size |
TestCodeCmdRejectsDestructiveFlagsWithoutAllow / TestCodeCmdRejectsMalformedScopeEntry / TestCodeCmdDestructiveArgsSatisfyAdapterGate / TestCodeCmdDestructiveBudgetAndScopeReachInvocation / TestCodeDestructiveScopeCap |
v0.1 CI |
Every argument key the gum code CLI stamps is a parameter the embedded gum.code op declares, so no flag can inject an undeclared argument that step 1 validation rejects with INVALID_ARGS before the sandbox starts |
TestCodeInvocationArgsAreDeclared |
v0.1 CI |
Shaping stages lose only what the profile asked them to lose: collapse_arrays truncation counts omissions exactly, truncate_strings applies per-field and dot-path overrides with the dot path winning over a bare name, clamps to the declared limit counting the appended ellipsis, and marks every value it clamped with a <field>_truncated sibling while leaving a value that fit unmarked, keep_fields allowlisting runs before drop_fields denylisting, strip_nulls removes nulls and empties without collapsing an all-zero map, dedupe distinguishes a JSON null from an absent string and skips non-map elements, flatten unwraps only a genuine single-key envelope, sort_by falls back deterministically on mixed or missing keys, and the on_empty sentinel fires for an empty array, an empty map, and a null |
TestApplyCollapseArraysTruncatesAndCountsOmissions / TestApplyTruncateStringsDotPathBeatsBareName / TestTruncateStringsWritesSiblingMetadata / TestTruncateStringsCapIncludesEllipsis / TestApplyPipelineOrderKeepBeforeDrop / TestApplyPipelineOrderCollapseBeforeFormat / TestApplyStripNullsRemovesNullsAndEmpties / TestApplyDedupeDistinguishesNullFromNilString / TestApplyDedupeSkipsNonMapElements / TestApplyFlattenUnwrapsItemsEnvelope / TestApplyFlattenEnvelopeMisses / TestApplySortByNilFallback / TestApplySortByNonMapElement / TestApplyOnEmptySentinelArray / TestApplyOnEmptySentinelMap / TestApplyOnEmptySentinelNull |
v0.1 CI |
[override_bindings] in project-local and user-global profile files attaches a profile to listed op_id or variant_id keys; override-bindings-only files are valid when referenced profiles resolve elsewhere; variant_id wins over op_id for the same resolved variant; rejects undefined-profile, unknown-op_id, unknown-variant_id, and structural errors with OVERRIDE_BINDING_INVALID; project-local wins over user-global. The table had no parser case and no runtime consumer before gum-b3sm, so OVERRIDE_BINDING_INVALID had no producer and a binding could not change what shaped a response; a binding now also attaches a profile to a variant whose catalog row names none, while an explicit caller-supplied profile still wins over both |
TestValidateOverrideBindings / TestParseFileAcceptsBindingsOnlyFile / TestParseFileRejectsStructuralBindingErrors / TestLoadOverrideBindingsProjectBeatsUserGlobal / TestOverrideBindingByOpIDWins / TestOverrideBindingByVariantIDBeatsOpID / TestOverrideBindingIgnoresOtherTargets / TestOverrideBindingAttachesProfileToUnprofiledVariant / TestPresetOutputProfileBeatsBinding / TestProfileValidateRejectsDanglingBindingProfile / TestProfileValidateRejectsUnknownBindingTarget |
v0.1 CI |
gum.describe_op registers #/$defs/DescribeOpResult as its outputSchema, validating responses including the deterministic variants[] truncation form controlled by meta_tools.describe_op.max_variants (default 5; ops with 6+ variants truncate to 5 by default, with variants_total and variants_omitted_count), an override-positive fixture that includes risk_override=true and risk_override_reason, and explicit exclusion of inactive-plugin-only status / reason fields, which appear only on gum://op/{id} and gum://variant/{id} resource responses |
TestDescribeOpTruncatesVariantsToFive / TestDescribeOpCarriesRiskOverride / TestDescribeOpExcludesPluginStatusFields |
v0.1 CI |
The #/$defs/DescribeOpResult enums carry all four §918 execution_support values at both levels, and the oneOf discriminator requires unsupported_capabilities on every non-full branch: a variant declaring execution_support: "partial" is legal per §918 and MUST validate against the registered outputSchema, and a partial payload without the list MUST be rejected |
TestDescribeOpResultAcceptsPartialExecutionSupport / TestDescribeOpResultPartialRequiresUnsupportedCapabilities |
v0.1 CI |
gum.describe_op MUST surface the variant's declared unsupported_capabilities, not its whole capabilities[]: a partial variant declaring three atoms and blocking one reports that one atom |
TestDescribeOpResultUsesDeclaredUnsupportedCapabilities |
v0.1 CI |
Op.Validate MUST enforce the §925 binding between execution_support and unsupported_capabilities: unknown enum value, a list on full, an atom outside capabilities[], a partial with no list or with every atom blocked, and a typed_executor_required/schema_only that omits a declared atom all fail; the three legal non-full shapes load |
TestOpValidateExecutionSupportBinding / TestOpValidateAcceptsDeclaredExecutionSupport |
v0.1 CI |
gum://help/{topic} returns the canonical §7 RESOURCE_NOT_FOUND envelope as a JSON-RPC application error with error.code=-32004 for topics absent from gum://help/topics; active topics return the §7 text/markdown MCP resource-content shape; deprecated topics return the §7 application/json resource-content shape containing only {"status":"deprecated","redirect":"<new-topic>"} |
TestHelpResourceNotFound / TestHelpTopicDeprecatedReturnsRedirect |
v0.1 CI |
Plugin-local failure codes map deterministically to stable GUM error codes (RATE_LIMIT→RATE_LIMITED, AUTH_EXPIRED→AUTH_REQUIRED, PARSE_FAILURE→SERVICE_DOWN, SERVICE_DOWN→SERVICE_DOWN, INVALID_INPUT→INVALID_ARGS) while preserving retry fields and sanitized source metadata as specified |
TestPluginErrorCodeMapping |
v0.1 CI |
Plugin schema bundles declare one manifest schema_ref whose JSON Schema document contains object-valued $defs.request and $defs.response; install derives request_ref=<schema_ref>.request and response_ref=<schema_ref>.response onto the catalog binding, stores each half as its JCS canonicalization under the SHA-256 of those exact bytes, records both in the variant's schema_hashes, rejects a missing or non-object def with PLUGIN_SCHEMA_REF_INVALID, and rejects divergent full-inventory collisions with SCHEMA_REF_COLLISION while letting a plugin change its own schema on reinstall |
TestPluginSchemaBundleMaterialization / TestPluginSchemaBundleMissingDefs / TestPluginSchemaRefReinstallIsNotSelfCollision / TestPluginSchemaRefCollision |
v0.1 CI |
Plugin manifests listing GUM_-prefixed env vars, exact denylist entries (GOOGLE_APPLICATION_CREDENTIALS, OPENAI_API_KEY, ANTHROPIC_API_KEY), or _GUM* variables in needs_user_creds fail install with PLUGIN_ENV_PROHIBITED; a single curated in-binary denylist source of truth is shared by catalog build, plugin install, and runtime env scrubbing; dispatch scrubs prohibited vars from the subprocess environment regardless of manifest declarations |
TestPluginEnvProhibited / TestPluginEnvDenylistMembers / TestPluginEnvDenylistSingleSource / TestPluginEnvDenylistApplied |
v0.1 CI |
Plugin manifests with non-empty needs_user_creds must declare one safe credential descriptor per env var; missing, duplicate, or extra descriptors fail with PLUGIN_CREDENTIAL_DESCRIPTOR_INVALID; install with missing required credentials records needs_configuration, skips live canary without quarantine, exposes only descriptor aliases in resources/errors, and a later successful credentialed gum canary --live is required before activation |
TestValidateCredentialDescriptorsHappyPath / TestValidateCredentialDescriptorsMissingDescriptor / TestValidateCredentialDescriptorsExtraDescriptor / TestValidateCredentialDescriptorsDuplicateAlias / TestValidateCredentialDescriptorsInvalidKind / TestValidateCredentialDescriptorsAllKinds / TestInstallArmsNeedsConfiguration / TestInstallNeedsConfigurationSkipsLiveCanary / TestPromotePendingRestartSkipsNeedsConfiguration / TestPluginSetupCredentialFlow / TestPluginCredentialNoRawEnvLeak |
v0.1 CI |
Plugin executable binding: non-dev plugin sources launch only an absolute executable inside the host-managed verified install root; the selected profile's plugins.lock records executable path, executable SHA-256, normalized argv, and install root; runtime spawn re-hashes the executable and quarantines/refuses execution on mismatch; PATH-only, shell-wrapper, and runtime uvx resolution paths fail with PLUGIN_EXECUTABLE_UNTRUSTED; the author-supplied command is an install-time selector, not the runtime argv, so install resolves command[0] against the declared executable and records [executable_path] + command[1:], refusing a PATH-only, interpreter, absolute, traversal or wrapper selector before the file copy, while a dev profile keeps the §8.7 escape hatch; the spawned subprocess receives exactly the recorded residual argv. Source-specific resolution is live for all four source classes: pypi binds command to the console script inside the host-built virtualenv (a leading uvx/pipx selector token is dropped and never spawned, pip install --no-index --no-deps installs only the checksum-verified artifact), github_release artifacts verify SHA-256 before a mode-pinned unpack, git sources verify the 40-hex pin against HEAD (unpinned refs are dev-only), and local plus unpinned-git installs carry risk="dev-untrusted" in their plugins.lock row |
TestPluginExecutableBinding / TestPluginCommandNormalization / TestNormalizedArgvReachesSubprocess / TestInstallPyPIPackage / TestInstallGitHubReleasePackage / TestInstallGitPackage |
v0.1 CI |
gum_parallel cancellation: cancelling the outer context propagates to in-flight inner calls within 200ms; after scheduling starts, result envelopes contain completed elements plus unfinished elements with error.error_code="CANCELLED" and error.cancelled=true, no success payload fields on cancelled elements, ledger records per-element cancelled: true and marks the outer entry cancelled: true; no goroutine leak |
TestGumParallelCancellationProducesCANCELLEDWhitebox / TestGumParallelCancelBeforeAnyDispatchAllCancelled / TestGumParallelCancellationPropagatesWithin200ms / TestGumParallelCancellationMarksBatchAndElements / TestRecordParallelBatchWritesCancelledInnerEntry / TestGumParallelLeaksNoGoroutines |
v0.1 CI |
gum_parallel 429 per-service-family isolation: a 429 from a workspace worker pauses only workers sharing service_family = "workspace"; workers with a different service_family continue uninterrupted; the pausing family resumes after retry_after_ms (or 60s fallback), staggered 50ms × worker_index; the gain ledger records the outer entry with the correct total element_count |
TestGumParallel429ServiceFamilyIsolation |
v0.1 CI |
grpc-sdk routing_headers rules 1-4 of docs/catalog-abi.md hold at catalog build: an entry outside the closed alphabet, a duplicate path, and an empty array fail with the documented GRPC_ROUTING_HEADER_* code, a path that does not resolve in the op's request schema fails with GRPC_ROUTING_HEADER_NOT_FOUND, and array indexing does not resolve. Every generator path enforces them, because validateGeneratedCatalog calls Catalog.Validate. At run time the adapter emits the resolved values as one x-goog-request-params metadata entry in declaration order, and omits a parameter whose field is absent or empty rather than sending an empty value. Fixtures cover both the present and the omitted form plus one per error code |
TestGrpcRoutingHeaderInvariant / TestRoutingHeaderFailsOnTheFirstBadEntry / TestRoutingHeaderRulesSkipNonGRPCVariants / TestRoutingHeaderNeedsARequestSchema / TestGeneratedCatalogGateRejectsRoutingHeaders / TestRoutingHeaderReachesTheWire / TestRoutingHeaderValueRendersScalars / TestRoutingHeaderValueHandlesMissingBinding |
v0.1 CI |
gum://help/topics is generated from apps/gum/internal/embedded/data/help-topics.v1.json; every listed active or deprecated topic resolves; deprecated rows return only a redirect object; no topic handler exists unless it is listed by the manifest; v0.1.0 manifest contains the thirteen active topics of the §13 seed set and no deprecated rows |
TestHelpTopicsSeedSet / TestHandleHelpTopicReadUnknownTopicReturnsResourceNotFound / TestHelpTopicDeprecatedReturnsRedirect |
v0.1 CI |
gum_parallel result envelope compression (§9.0.1): when ≥2 inner results share identical ExpressionMeta field values, those fields are hoisted into the outer envelope's shared_expression_fields and omitted from per-result _expression delta objects (#/$defs/ExpressionMetaDelta); the receiver reconstructs effective per-result ExpressionMeta as shared_expression_fields ∪ per_result._expression with per-result values winning on conflict; each reconstructed effective object MUST validate against full ExpressionMeta, while each emitted delta validates against ExpressionMetaDelta; round-trip MUST yield byte-identical unhoisted forms; identity is determined under canonical-JSON byte-equality so null and absent are NOT identical (§9.0.1 rule 1); fixture set MUST include (a) at least one heterogeneous batch (N≥2, no field identical across all results) verifying shared_expression_fields is absent and behavior matches the un-hoisted form, (b) one all-null fixture verifying explicit null hoists with value null, (c) one mixed null/absent fixture verifying no hoisting occurs, (d) one fixture asserting the outer _expression.variant_id is null per §12.3, and (e) one fixture where every inner result carries both intentional_zero_max_items: true and a non-null on_empty_message whose string value is byte-identical across all N results (required for §9.0.1 rule 1 hoisting), verifying that both fields hoist into shared_expression_fields and that the {intentional_zero_max_items: true, on_empty_message != null} invariant (§13 ExpressionMeta) holds on the effective ExpressionMeta of every result after reconstruction |
TestGumParallelResultEnvelopeCompression / TestExpressionMetaFieldsCoversEveryTag / TestExpressionMetaFieldsMatchesJSONMarshal |
v0.1 CI |
Per-tool inputSchema budget verification (§4.1): each Tier A risk tool's registered inputSchema stays within its declared per-tool budget after parameter additions; refreshed every parameter-adding PR; also verifies that the gum.write tool description string contains the normative irreversibility sentence (§13 gum.write irreversibility notice) |
TestTierAPerToolInputSchemaBudget |
v0.1 CI |
Tool argument validation (§4.1): every tools/call argument object is validated against the tool's registered inputSchema before the handler runs. Covers a closed enum violation (gum.read format="yaml"), a missing required property, an extra property under additionalProperties: false, a wrong declared type, a value below minimum and one above maximum, the gum.code language enum, and the destructive_budget cap; each rejection is an INVALID_ARGS §7 envelope naming the tool, and valid arguments still reach the handler. Every registered schema across the meta-tool roster, the convenience roster, and the two embedded-skill helpers MUST compile, and a schema that cannot be resolved fails closed for that tool alone with a SERVICE_DOWN envelope instead of running the handler or ending the session |
TestMetaToolInputSchemaIsEnforced / TestValidInputStillReachesHandler / TestEveryRegisteredInputSchemaResolves / TestUnresolvableSchemaFailsClosed |
v0.1 CI |
Convenience schema/op agreement (§4.1): every property a convenience tool advertises is an argument its backing catalog op accepts, a confirmation control the handler strips, the §9 format? control, or a declared body mapping; a row carries a body mapping if and only if its op has body request fields, and every mapped name is a real body field advertised by the tool; each required PATH param of the op is advertised and required; format? is present exactly when the row lists more than one format and the op does not declare its own format field. The roster runs behind the same validating wrapper as the meta-tools, proved end to end through the real kernel with a capture adapter over the embedded catalog's request fields: a read tool's args reach the executor under the op's names with the wire-format control stripped, a body-argument write tool lands its object under the reserved body key, a body-field write tool lands one field there while its query args stay top level, and an unadvertised or missing required argument is rejected with INVALID_ARGS before the executor runs |
TestConvenienceSchemaPropertiesReachTheKernel / TestConvenienceBodyArgsMapToRealBodyFields / TestConvenienceRequiredPathParamsAreAdvertised / TestConvenienceFormatControlMatchesFormats / TestConvenienceReadArgsReachExecutor / TestConvenienceWriteBodyArgReachesExecutor / TestConvenienceBodyFieldArgReachesExecutor / TestConvenienceHandlerRejectsUnadvertisedArg / TestConvenienceHandlerRejectsMissingRequiredArg |
v0.1 CI |
gum.code destructive gate schema (§1083/§1089): destructive_budget declares integer with minimum: 0 and maximum: 20; destructive_scope declares an array with default: [], maxItems: 20, and object items carrying op_id (required) and resource_key. The item type MUST be object, matching what addDestructiveArgs builds and what extractScope reads; a string item is silently discarded by the executor and would run destructive code with no scope |
TestGumCodeSchemaDestructiveScopeDefaultsToEmptyArray |
v0.1 CI |
Per-tool token delta gate (spec §2 line 129, bead gum-coo): per-tool cl100k_base description token counts are stored in testdata/tier-a-token-baseline.json; any tool whose measured count exceeds its stored baseline fails the test with TOKEN_DELTA_REGRESSION; a tool absent from the baseline fails with MISSING_BASELINE; an orphan baseline entry (in the file but not in tools/list) fails with ORPHAN_BASELINE. Increases require the token-budget-increase PR label and a paired baseline bump in the same change; decreases pass and emit a RATCHET_OPPORTUNITY Logf hint. |
TestTierAPerToolTokenDelta / TestTierABaselineJSONLoads |
v0.1 CI |
Tier A output schemas cover every successful non-error branch (spec §13 branch-specific output schemas): a shaped gum.read/gum.write/gum.destructive result and a gum.code result validate against the registered schema, gum.code carries no ParallelResults branch because a printed batch envelope lands inside the data string of a shaped result, and every branch-selecting schema keeps root type: object so go-sdk registration accepts it. Confirmation-required responses are isError: true §7 envelopes, not schema branches, so no branch describes them |
TestTierABranchOutputSchemasCoverEveryShapedBranch, TestTierABranchOutputSchemasKeepObjectRoot, TestShapedBranchesUseAnyOfNotOneOf, TestGumCodeSchemaHasNoParallelResultsBranch, TestConfirmationEnvelopeIsNotASchemaBranch |
v0.1 CI |
JSON-RPC batching is closed, not capped: GUM advertises no batching capability, and §13.1's protocol floor refuses an initialize below 2025-06-18 with -32602 / UNSUPPORTED_CAPABILITY, so no session can negotiate a version whose transport still decodes a batch frame. On a contract-version session the pinned transport rejects the frame before dispatch and no tool, resource, or prompt handler runs. The earlier size caps (32 requests, 1 MiB decoded body, 256 KiB decoded params per item) are unreachable by construction and are not implemented |
TestMCPBatchCaps |
v0.1 CI |
Audit log hard ceiling cannot be exceeded under cross-process lock contention: with audit.jsonl one entry short of the cap and audit.unbounded=false, an append that must rotate blocks while a second descriptor holds audit.jsonl.lock, then rotates and lands once the lock is released; when the lock outlasts the wait the append writes nothing, creates no archive, and records the failure in audit.broken |
TestAuditHardCeilingContention |
v0.1 CI |
Structured logging (§14.1 rule 1): build-time AST lint rejects log.Printf, fmt.Fprintln(os.Stderr, ...), and third-party logger imports in internal/dispatch, internal/adapters/*, internal/mcp, internal/cli, internal/cache, internal/auth, internal/auditlog, internal/plugins/registry, internal/profile, internal/sandbox, and internal/output. The scan set is walked recursively, so internal/output covers output/tee, output/profile, and output/gain. It MUST include every package in the §14 constructor-convention table plus stateless internal/output, and every path in it MUST exist, so a rename fails the lint instead of silently shrinking the scan |
TestStdLogProhibition |
v0.1 CI |
Logger injection (§14.1 rule 2): every §14 constructor-convention package that logs (internal/dispatch, internal/mcp, internal/auditlog, internal/output/gain, internal/plugins/registry) routes every emission through one injectable *slog.Logger reached by that package's own configuration surface. Static half: no package-level slog.Debug/Info/Warn/Error emission survives in a non-test file, each package declares the logger field and the log() accessor, each exposes an exported identifier to inject with, and the gate fails when it scans zero packages. Behavioural half: an injected logger receives the emission, an unset logger falls back to slog.Default(), and slog.New(slog.DiscardHandler) silences the package without panicking |
TestLoggerInjectionContract, TestDispatcherLoggerInjection, TestNewDispatcherWithConfigCarriesTheLogger, TestServerLoggerInjection, TestRegistryLoggerInjection |
v0.1 CI |
v0.1.0 release binaries import neither go.opentelemetry.io/* nor net/http/pprof (§14.1.5–6). TestNoOTelImportV01 verifies via the import graph (go list -deps ./cmd/gum/... MUST produce no go.opentelemetry.io/* matches); the module-graph scan (go list -m all) is run as an advisory warning, not a CI failure, so transitive SDK module presence does not produce a false positive. TestNoPprofImportV01 uses the same import-graph methodology for net/http/pprof. |
TestNoOTelImportV01, TestNoPprofImportV01 |
Release gate |
Audit-log entries always emit schema version field v as the first key; absent v is parsed as v:0 |
TestAuditEntryShape |
v0.1 CI |
internal/output statelessness (§14): AST scan of the encoder packages (internal/output, fieldmask, jcs, profile, render, toon) confirms no exported constructors (no func New* / func (T) New*), no package-level var the package writes to outside one-time sync.Once initialization, and no assignment to an unexported field of an exported type; a future PR adding a stateful encoder cache fails CI before merge. internal/output/tee and internal/output/gain are out of scope: §14 line 3506 gives tee the artifact filesystem state and the gain ledger is an append-only rotating file |
TestOutputStatelessness |
v0.1 CI |
intentional_zero_max_items invariant (§13 ExpressionMeta, §9.1): when the runtime emits _expression.intentional_zero_max_items: true, it MUST also emit a non-null _expression.on_empty_message; the combination {intentional_zero_max_items: true, on_empty_message: null} is never emitted by a v0.1.0 dispatch path; the test runs the expression pipeline on a fixture profile with collapse_arrays.max_items=0 + on_empty="..." and asserts the emitted envelope satisfies the invariant. The invariant MUST also be asserted on the effective per-result ExpressionMeta after gum_parallel envelope compression and reconstruction (covered jointly by TestGumParallelResultEnvelopeCompression fixture (e)) so that hoisting cannot smuggle in a violating combination via shared_expression_fields ∪ per_result._expression |
TestExpressionEnvelopeFlagsIntentionalZero / TestZeroMaxItemsKeepsItsMessageOnAnEmptyUpstream |
v0.1 CI |
Per-call max_items override (§9.1): the CLI --max-items flag on gum read, gum write, gum destructive, and gum call and the MCP max_items argument on gum.read, gum.write, and gum.destructive replace the active profile's collapse_arrays.max_items for that invocation only; "all" skips stage 5 so no element is dropped; a value that is not a positive integer or "all" fails with INVALID_ARGS before any upstream request; the override never enters the cache key or the args hash; --format raw stays a byte-identical passthrough; when rows are omitted the shaping notice leads with the omitted row count and names the <key>_omitted_count field |
TestShapeResponseMaxItemsOverridesBuiltinCap |
v0.1 CI |
_expression count and lossy accuracy (§9.1 rules 1 and 3): lossy reports the cap in force for the call, not the profile fields alone, so a caller max_items that caps an otherwise lossless profile reports lossy: true and max_items=all over a capped profile reports lossy: false; omitted_count counts every record the pipeline dropped, including rows removed by limit, dedupe, keep_fields, drop_fields and strip_nulls, which write no <key>_omitted_count sibling, and keeps the larger of the sibling sum and upstream_count - result_count so a collapsed record array is not double counted. The §9.1 rule 3 discriminator therefore holds: a shaped-away result reports (0, N>0) and a genuinely empty upstream reports (0, 0). on_empty_message is not the discriminator, because §9.4 requires gum.search_apis to answer a zero-hit query with it and the §13 intentional_zero_max_items invariant pairs it with an empty upstream |
TestLossyFollowsThePerCallMaxItems / TestLossyDropsWhenTheCallRemovesTheOnlyCap / TestOmittedCountsRowsLimitRemoved / TestOmittedCountsRowsDedupeRemoved / TestOmittedCountKeepsTheLargerOfTheTwoSources / TestEmptyUpstreamAndShapedAwayDifferByCounts / TestOnEmptyFiresWhenShapingEmptiesTheBody / TestZeroMaxItemsKeepsItsMessageOnAnEmptyUpstream / TestIsLossyBranches |
v0.1 CI |
Row stages that remove whole records report them (§9.1 stage 7, shaping notice): dedupe writes occurrence_count on the surviving row of a collapsed group, only when the group held more than one row, and never over an upstream occurrence_count; ApplyOutput.DedupedRows and ApplyOutput.LimitedRows carry the rows dedupe and limit removed through ShapedResponse to the CLI and MCP shaping notice, which states both counts. Neither stage writes a count into the body, so a silent notice reports a shortened result as a complete one |
TestDedupeCountsOccurrences / TestDedupeLeavesUniqueRowsUnmarked / TestDedupeKeepsAnUpstreamOccurrenceCount / TestApplyReportsRowsDedupeRemoved / TestApplyReportsRowsLimitRemoved / TestNoticeNamesRowsDedupeAndLimitRemoved / TestNoticeStaysEmptyWhenNoRowsRemoved / TestShapingNoticeNamesDedupedAndLimitedRows |
v0.1 CI |
Record-array selection (§9.1 rule 3, expression-profile-dsl.md Record array): a shaped top-level object resolves its record array to the first present key among items, data, messages and results, falling back to a single array-valued field, and _expression.result_count and the row stages (dedupe, sort_by, limit) read the same key. The shipped googleads.keyword_historical.v1 body carries unmatchedInputs beside results, which the fallback alone rejected as ambiguous: a 243-row response capped at 100 reported result_count: 0, omitted_count: 143 with 100 rows in the body, and the row stages skipped it. An object with two or more arrays and no named key still has no record array |
TestResultCountReadsResultsBesideASecondArray / TestKeywordHistoryCountsMatchTheBody / TestRowStagesRunOnResultsBesideASecondArray / TestTwoUnnamedArraysStillHaveNoRecordArray |
v0.1 CI |
Adapter annotation costs are named in the shaping notice (§9.1 Adapter annotation, gum-9l5c): ResponseAnnotator.AnnotateResponse returns the dot-paths it added beside the body; the kernel subtracts the paths the profile then dropped and carries the rest on ShapedResponse.AnnotationPaths; --format raw skips the annotator and reports none. When the profile dropped a field, the CLI and MCP notice offers raw and names those fields, so a caller who switches to recover results.closeVariants learns it loses results.matchedInputs. A response with no annotation keeps the prior wording |
TestAnnotateResponseReportsTheFieldsItAdded / TestAnnotateResponseReportsUnmatchedInputsAlone / TestAnnotateResponseReportsNoPathsWhenUnchanged / TestShapedResponseCarriesSurvivingAnnotationPaths / TestShapedResponseKeepsEveryAnnotationPath / TestShapedResponseReportsNoAnnotationPathsWhenAllDropped / TestRawFormatReportsNoAnnotationPaths / TestRawHintNamesTheAnnotationFieldsRawDrops / TestRawHintAgreesTheAnnotationNoun / TestRawHintUnchangedWithoutAnnotations / TestAnnotationPathsAloneAddNoHint / TestCLINoticeNamesTheAnnotationFieldsRawDrops / TestCLINoticeKeepsThePlainRawHint |
v0.1 CI |
Profile file shape (docs/expression-profile-dsl.md File Shape, §9.2): a file may use the documented envelope ([output_profiles."<name>"] tables, an [override_bindings] table, top-level [[tests]] whose profile key names its target) or the catalog-embedded bare-key form, and MUST NOT mix them. Parse read bare keys only and took the profile name from the filename, so every example printed in the docs failed gum profile validate on line 1 and no file could express [override_bindings]. The resolver reads <name>.toml first and then every other *.toml in name order, because one envelope file defines several names; a malformed file in the search path fails the load instead of resolving to no-profile. gum profile test needs --name <profile> when the file defines more than one, and the flag is named --name because the root command owns a persistent --profile |
TestParseFileReadsDocumentedEnvelope / TestParseFileReadsBareKeys / TestParseFileRejectsMixedShapes / TestParseFileRejectsDuplicates / TestParseFileRejectsEmptyFile / TestParseFileRejectsUnquotedProfileName / TestParseFileAttachesTestsToNamedProfile / TestMergeOverrideBindingsFirstLayerWins / TestResolveProfileReadsEnvelopeFile / TestResolveProfileFailsClosedOnMalformedFile / TestProfileValidateAcceptsDocumentedEnvelope / TestProfileTestSelectsNamedProfile |
v0.1 CI |
Profile format key (§9.1 stage 8, expression-profile-dsl.json #/$defs/profile): the DSL key is format, not default_format, and its enum is toon, csv, json, markdown. The parser read default_format and allowed toon|json|raw, so every profile printed in docs/expression-profile-dsl.md and docs/profile-dsl-reference.md failed to load and the CSV and markdown encoders Apply already implements were unreachable from a profile. raw is rejected with an error naming --format raw, because it is a caller choice that skips shaping rather than an encoder. A [[tests]] fixture's expect_format accepts the profile enum plus raw, which is the full set ApplyOutput.Format can report |
TestParseReadsTheDocumentedFormatKey / TestParseAcceptsCSVAndMarkdown / TestParseRejectsDefaultFormatKey / TestParseRejectsRawAsAProfileFormat / TestFixtureExpectFormatCoversEveryReportedFormat / TestParseFormatInvalidValueSurfacesError |
v0.1 CI |
Profile DSL key parity (§9.1 stage list, expression-profile-dsl.json #/$defs/profile): every key the parser accepts is documented and schema-accepted, and every key the schema declares parses. Five keys parsed only in Go (projection, flatten_singletons, sort_by, limit, omit_zero_counts), so a profile that used one parsed and then failed ValidateRawProfileFile against additionalProperties: false. field_mask ran the other way: schema and spec declared it, the parser had no case, and no dispatch step put it on the wire, so §9.1 stage 1 never ran. The mask is injected as the universal Google fields argument before the §10.3 cache lookup; an explicit caller fields, field_mask_mode="none" and --no-field-mask each suppress it |
TestParserAcceptsEveryDSLKey / TestSchemaAcceptsEveryDSLKey / TestParseReadsFieldMask / TestSerializeRoundTripsFieldMask / TestMergeProfilesInheritsFieldMask / TestProfileFieldMaskReachesUpstream / TestCallerFieldsBeatsTheProfileMask / TestFieldMaskModeNoneSkipsInjection / TestSuppressFieldMaskSkipsInjection / TestNoProfileMaskLeavesArgsAlone |
v0.1 CI |
The §9.1 DSL field table defaults field_mask to the variant's default_fields, so a profile that omits the key still projects upstream; an explicit field_mask overrides it, and the three stage-1 vetoes (caller fields, field_mask_mode="none", --no-field-mask) suppress it the same way |
TestVariantDefaultFieldsIsTheProfileMaskDefault / TestProfileFieldMaskBeatsVariantDefaultFields / TestVariantDefaultFieldsHonorsTheStageOneVetoes |
v0.1 CI |
Shipped catalog.json satisfies §772: at least one variant carries a non-empty default_fields. Every shipped mask parses under the §9.1 field-mask grammar, names no wildcard, and sits only on a read-class variant, because a write response carries the new resource's id and etag and a mask would drop them. Each mask is checked field by field against cmd/gen-catalog/testdata/default-fields-schema.json, a projection of the live Discovery response schemas: Google's partial-response filter drops an unknown selector instead of rejecting it, so a typo returns an empty body and no error, and this check is the only place it can be caught |
TestCatalogCarriesDefaultFields / TestDefaultFieldsAreWellFormedReadMasks / TestDefaultFieldsMatchDiscoverySchema / TestDefaultFieldsValidatorRejectsBadMasks / TestApplyDefaultFieldsWritesEveryVariant |
v0.1 CI |
gum://catalog resources/list entry carries size annotation equal to len(catalogBin) (non-zero, runtime-computed from the embedded artifact) AND "x-gum-do-not-auto-inject": true annotation (§13). TestStaticResourceRegistration is extended with both assertions |
TestStaticResourceRegistration (extended) |
v0.1 CI |
gum://status/canaries returns status="stale" for every installed plugin canary on server startup before any passive cron run completes; the resource covers plugin canaries only and never first-party Google API health (§13) |
TestCanaryStaleOnStartup |
v0.1 CI |
gum://plugins MCP view omits rows whose status is installed_pending_restart; the same plugin appears in gum plugin list --format=json CLI output for the same profile (§13 R28B-M-2 filter) |
TestPluginsResourceFiltersPendingRestart / TestHandlePluginsReadFiltersInstalledPendingRestart |
v0.1 CI |
§10.3 per-op TTL table is keyed by catalog op_id: every key names a live op, and each spec tier reaches the op the dispatcher passes to Set. Seven of eleven keys were the spec's prose tier labels (gmail.profiles.get) or invented google.-prefixed names, so they matched nothing and their ops silently ran on the 60s default; the two 24h google.youtube.i18n*.list keys named ops the catalog has never carried |
TestPerOpTTLKeysAreRealCatalogOps / TestPerOpTTLCoversEverySpecTier / TestPerOpTTLTableShape |
v0.1 CI |
§10.3 cache eviction accounting: EvictExpired returns the number of entries whose delete actually committed, not the number the scan matched, and it reports the transaction error, so gum cache clear --expired cannot print an expired_removed count for entries still on disk; the hot tier is purged only after that commit succeeds. An over-size sweep that cannot commit fails the Set that triggered it instead of letting the store grow past max_size_bytes in silence. The hot tier's lazy last_access refresh copies the stamp under the read lock before taking the write lock, so concurrent Get calls on one key are clean under -race |
TestEvictExpiredReportsDeleteFailure / TestEvictExpiredCountsOnlyDeletedEntries / TestEvictOverSizeReportsDeleteFailure / TestEvictOverSizeUnderCapIsNoError / TestBBoltHotLastAccessNoRace |
v0.1 CI |
§10.2 migration fidelity: the BoltDB copy runs inside one SQLite transaction whose final statement is the sentinel, so a second connection on the same file sees the whole migration or none of it and a rolled-back import leaves neither rows nor sentinel; an entry from the gum-cache bucket keeps its bare key while other buckets and sub-buckets keep their bucket-path prefix; each entry's own expiry is carried into the new row, so an already-expired entry stays a miss and a live one keeps its deadline. gum cache migrate --force deletes an http-wal.db that will not open as SQLite and re-derives from http.db, which is the recovery path ErrSQLiteCorrupt documents; without the flag the corruption is reported and nothing changes |
TestImportIsInvisibleUntilCommit / TestImportRollbackLeavesNothing / TestMigrateKeepsCacheBucketKeysBare / TestMigratePreservesEntryExpiry / TestMigrateForceRecoversCorruptWAL / TestMigrateCorruptWALWithoutForceStillFails |
v0.1 CI |
gum://status/health returns rows for the closed v0.1 subsystem enum [audit_log, cache_sqlite, tee_filesystem, keychain, gain_ledger, canary_runner]; values status∈{healthy, degraded, unavailable}; health probes are local-only and emit no network calls; per-subsystem sample TTL bounds resource-read cost; detail is clamped to 80 characters at a character boundary on every row, and a healthy row may carry a short reason so a deferred probe does not read as a real pass (§13) |
TestStatusHealthSubsystemEnum / TestStatusHealthNoNetwork / TestHealthDetailIsBoundedForEverySubsystem / TestHealthDetailIsBoundedOnAnOSError / TestHealthyRowsMayExplainThemselves |
v0.1 CI |
Stdio transport uses newline-delimited JSON-RPC framing only; the silent-stdout startup invariant holds: between process start and notifications/initialized, no non-JSON bytes appear on stdout (§13.1) |
TestStdioFramingClean |
v0.1 CI |
Unsolicited logging/setLevel (sent by a permissive client despite logging being unadvertised) returns a successful empty-result response and does not log to stdout; the requested level is recorded in an in-memory per-session field for forward compatibility (§13.1) |
TestLoggingSetLevelTolerant |
v0.1 CI |
MCP completion/complete returns within 100 ms (P95) and 250 ms (P99) for a worst-case Tier A argument (variant_id completion for a 50-variant op) and for every completable resource-template argument with a 100-plugin inventory; resource-template completions meet the same budget; failing the P95 budget on linux/amd64 CI hardware is a release-blocker (§13.1) |
TestMCPCompletionLatency |
Release gate |
BM25 index (gen/index/bm25.bin), dense embedding index (gen/index/embeddings.bin), and the active session catalog snapshot are built from the same catalog.json in a single go generate run; every snapshot op_id appears in bm25.bin; every snapshot op_id with embedding-enabled variants appears in embeddings.bin; every embedding entry corresponds to a BM25 entry; no completion-eligible op_id is absent from any required index (§5.3 single-source invariant) |
TestCatalogIndexSnapshotInvariant |
v0.1 CI |
internal/dispatch is the leaf of the internal import graph: no package in internal/dispatch's transitive import set re-imports internal/dispatch; internal/usage, internal/catalog, internal/profile, internal/auth, internal/cache, internal/retry, internal/sanitize, internal/output (including output/profile, output/tee, and output/gain), and internal/pluginenv do not import internal/dispatch; internal/auth may import kernel types to implement dispatch.AuthResolver because the kernel does not import it, and MUST NOT call the dispatcher entrypoint; cmd/gen-catalog imports no internal package other than internal/catalog schema types and internal/output/profile (the §5.4 profile validator); the only non-test files that call the dispatcher entrypoint are under cmd/, internal/cli/, internal/mcp/, and the two code-mode host files internal/adapters/code_risor.go and internal/adapters/code_risor_parallel.go (§14 import-graph contract) |
TestNoCyclicImports |
v0.1 CI |
A cache hit runs step 8 and therefore step 7c: a warm call whose profile sets recovery != "none" writes the same tee artifact and carries the same full_result_path, full_result_size, and (for recovery="resource_link") the same gum://results/<hash> URI as the cold call that filled the cache (§9.0 dual-path recovery) |
TestCacheHitStillWritesRecoveryArtifact / TestCacheHitEmitsResourceLink |
v0.1 CI |
Every error leaving the dispatcher entrypoint carries a §7 envelope: an adapter error that is not a *StructuredError becomes SERVICE_DOWN with retryable=false, op_id, and the original message, and keeps the cause in the chain for errors.Is; an already-structured error passes through with its own code and retryable flag; the MCP tool seam renders both as parseable JSON, never free text (§7 envelope completeness) |
TestUnstructuredAdapterErrorGetsEnvelope / TestStructuredAdapterErrorPassesThrough / TestDispatchAndShapeAlwaysEmitsErrorCode / TestDispatchAndShapeKeepsStructuredCode |
v0.1 CI |
Shipped linux/amd64 binary (including embedded gen/catalog.bin, gen/index/embeddings.bin, gen/index/bm25.bin) does not exceed 120 MB at release tagging (§15 binary-size cap) |
TestBinarySize (in internal/securityscan, cap = MaxBinarySizeBytes) + binary-size job in .github/workflows/build-matrix.yml |
Release gate (every PR) |
cmd/gum and its transitive dependency closure contain no CGo imports (§15 CGO_ENABLED=0 scoping); native keychain code lives behind the internal/auth keychain-backend abstraction |
TestReleaseBinaryNoCGo (in internal/securityscan) + CI build-matrix workflow with CGO_ENABLED=0 across linux/amd64, linux/arm64, darwin/amd64, darwin/arm64 |
Release gate + PR gate |
Fixtures under testdata/ contain no real Google API keys (AIza...), bearer tokens, OAuth refresh tokens (1//0...), AWS access keys (AKIA...), PEM private keys, or non-example.com email addresses; allowlists cover RFC 2606 reserved domains (*.example.{com,org,net}, *.test.local, localhost) and synthetic Google Calendar iCalUIDs (evNNN@google.com) |
TestFixturesNoSecrets (in internal/securityscan) |
v0.1 CI |
Release binary contains no shared-library linkage (linux ldd: "not a dynamic executable" or "statically linked"; darwin otool -L: only /usr/lib/* and /System/* entries); built with CGO_ENABLED=0 -trimpath -ldflags='-s -w' |
TestNoSharedLibDependencies (in internal/securityscan) |
Release gate |
Build-time -X main.version=<tag> propagates to gum --version and gum version output; release tags drive the stamped value via goreleaser ldflags |
TestVersionStamp (in internal/securityscan) |
Release gate |
Opt-in update notifier (gum-afcv.5): notify.enabled=true per-profile gates an async GitHub releases-API check on gum version; check honours a 2s timeout; results cache 24h at <XDG_CACHE_HOME>/gum/<profile>/notify.json; the check NEVER blocks version output and warnings emit to stderr only (not stdout — pipelines stay clean); dev builds and unparseable semvers are no-ops; fetch errors are silently swallowed |
internal/notify package tests + TestVersionNotifierDisabledByDefault / TestVersionNotifierOptInPrintsToStderrNotStdout in cmd/gum |
v0.1 CI |
Homebrew tap install path: ehmo/homebrew-tap carries Formula/gum.rb; users install with brew tap ehmo/tap https://github.com/ehmo/homebrew-tap then brew install ehmo/tap/gum. The formula name must stay qualified because Homebrew core already has Charmbracelet's unrelated gum. Binary casks stay disabled until macOS signing/notarization is active; unsigned cask binaries can be quarantined or blocked on first launch. |
ruby -c Formula/gum.rb in the tap + remote Homebrew install smoke where host policy and disk allow it |
Release gate |
Release workflow rejects tags that do not match ^v[0-9]+\.[0-9]+\.[0-9]+$ (no pre-release suffix on the stable channel) via the validate-tag job before any goreleaser step runs |
validate-tag job in .github/workflows/release.yml |
Release gate |
Two sequential builds of ./cmd/gum from the same source tree with CGO_ENABLED=0 -trimpath -ldflags='-s -w -X main.version=<stamp>' produce byte-identical SHA-256 outputs |
TestReproducibleBuild (in internal/securityscan) + reproducible job in .github/workflows/build-matrix.yml |
Release gate |
Release pipeline runs go test -race ./... on Go 1.26.x and blocks on reachable vulnerabilities from govulncheck ./...; the separate fuzz workflow runs the six targets in .github/workflows/fuzz.yml at 60s each: internal/output/toon.FuzzToonDecode, internal/output/jcs.FuzzJCSCanonical, internal/output/jcs.FuzzJCSIdempotent, internal/plugins.FuzzPluginManifest, internal/cli/callargs.FuzzParseArgs, and internal/output/profile.FuzzParse |
TestRaceModeReleaseGate / TestGovulncheckPipeline / TestFuzzWorkflowTargetsExist / FuzzToonDecode / FuzzJCSCanonical / FuzzJCSIdempotent / FuzzPluginManifest / FuzzParseArgs / FuzzParse |
Release gate (race and govulncheck); fuzz workflow |
SLSA Level 1 provenance attestations are generated by slsa-framework/slsa-github-generator/.github/workflows/generator_generic_slsa3.yml@v2.0.0 against the SHA-256 hashes of all goreleaser-produced *.tar.gz artifacts; the .intoto.jsonl is uploaded to the GitHub release; a verify-provenance job downloads the just-published artifacts + provenance and runs slsa-verifier verify-artifact --source-tag <tag> against every artifact before the workflow exits (§15 release pipeline) |
provenance + verify-provenance jobs in .github/workflows/release.yml; slsa-framework/slsa-verifier/actions/installer@v2.6.0 |
Release gate |
Release fixture set composition gate: internal/bench/fixtures/release/ contains at least 200 fixture calls with composition (50% Workspace read, 20% gum_parallel 2-4 batches, 15% non-Workspace read, 15% write/destructive across ≥3 services), validated at release within ±5% tolerance per category (§12.3) |
TestGainFixtureCompositionGate |
Release gate |
gum gain --fixture-replay runs the release fixture set against the TOON-default catalog profiles, computes end-to-end savings against the fixture-backed baseline (baseline_method = "fixture_replay"), and emits the same GainResult schema as live ledger reads; failure modes (missing fixture set, schema drift, baseline mismatch) return stable error envelopes (§12.3 R28B-M-3) |
TestGainFixtureReplay |
Release gate |
The fixture-replay token estimator measures the bytes the response path emits, not a TOON form gum never sends: for format=toon it encodes each fixture through toon.EncodeDocument (the §9.0 two-section document, op header from request.json), and falls back to canonical JSON for a body §9.0 cannot carry, which is what profile.Apply does when it relabels the format. expected-toon.txt and the out_toon count in expected-tokens-cl100k.json follow. internal/bench/release_savings.go keeps toon.Encode as its separate compact-TOON baseline for the published savings claim and says so in its doc comment (bead gum-5ij4) |
TestReplayMeasuresTheNineZeroDocument / TestReplayFallsBackToJSONWhenNotRepresentable / TestDocumentFromRepresentable / TestDocumentFromFallsBack / TestEncodeDocument |
v0.1 CI |
gum gain --fixture-replay second pass with output.default_format=json set on the profile config replays the same release fixture set, computes JSON-default end-to-end savings against the fixture-backed baseline, and is the release-gate proof artifact required by §2.1 "Savings claim measurement". The two passes share fixture data; only the default-format setting differs. Failure (savings drift beyond the documented JSON-default band, fixture-replay error envelope) blocks the release tag. |
TestGainFixtureReplayJSONDefault |
Release gate |
Managed-scope manifest (apps/gum/internal/embedded/data/auth-managed-scopes.v1.json) validates against apps/gum/internal/embedded/data/auth-managed-scopes.v1.schema.json (JSON Schema 2020-12); manifest invalidity fails the build before cmd/gen-catalog evaluates gum_oauth variants; for every scope with status="active", the schema enforces verification_state="verified", project_evidence_state="ready", live_canary_state="passing", and non-empty evidence (§7) |
TestManagedScopeManifestSchema |
v0.1 CI |
Overrides manifest schema gate: cmd/gen-catalog embeds cmd/gen-catalog/overrides.toml and cmd/gen-catalog/overrides.schema.json (JSON Schema 2020-12 applied to the TOML decoded as JSON) and validates the manifest immediately after flag parsing, before any mode runs; every violation — TOML syntax error, unknown top-level table, [apis.<name>] entry missing discovery_url, non-https URL, sdk_only entry missing a required field, risk class outside read/write/destructive — fails with OVERRIDES_SCHEMA_INVALID; the [apis.<name>] table is the live source of per-service Discovery URLs for request-field enrichment and default-field derivation (§5.2) |
TestOverridesManifestSchema |
v0.1 CI |
Cobra-forbidden patterns: AST lint rejects cobra.OnInitialize and cobra.OnFinalize anywhere in internal/cli/ or cmd/gum (plain, aliased and dot imports of spf13/cobra alike); required initialization order is constructor-driven not init-hook-driven (§12.2) |
TestForbiddenPatterns / TestForbiddenPatternsDetectorCatchesEachHookForm |
v0.1 CI |
Atomic settings.json patch: gum init holds the per-target advisory lock beside the host config (.claude/settings.lock, claude_desktop_config.lock, mcp.lock) across the whole read-merge-write; the merge is recomputed under the lock, so a key written after the preview survives; concurrent gum init runs serialize and every entry survives; a lock held by another process fails the patch with a timeout error naming the lock path and leaves the file byte-identical (§12.2 R28C-M-12) |
TestSettingsAtomicPatch |
v0.1 CI |
Profile config schema versioning: every profile config.toml declares config_schema_version = 1; a pre-normative file with no field migrates to 1 on the next write; a future version fails the load with CONFIG_SCHEMA_UNSUPPORTED; an unrecognized key is non-fatal, keeps its value across a load, and yields a structured unknown_config_key warning carrying error_code = UNKNOWN_CONFIG_KEY that gum doctor surfaces with the offending key (§12.2) |
TestProfileConfigSchemaVersion |
v0.1 CI |
Gain-ledger rotation keeps history readable: per-profile gain-ledger.jsonl rotates on a fixed 100 MB size threshold, not on the §11 audit-log schedule; each rotation archives the live file under its own gain-ledger-<unix>.jsonl name, and gum gain and gum.gain aggregate the archived segments with the live one, so no rotated window leaves the §12.3 savings evidence |
TestGainStatsIncludesRotatedSegments |
v0.1 CI |
interface_kind closed-enum membership is enforced at catalog build: unknown non-x-* values fail with UNKNOWN_INTERFACE_KIND; promotion from x-<name> to <name> requires the multi-step procedure in docs/catalog-abi.md (§Interface Kind extension procedure), and a rebuild during the promotion window keeps x-<name> and <name> as distinct values |
TestInterfaceKindValid / TestOpValidateRejectsUnknownInterfaceKind / TestInterfaceKindPromotionFixture |
v0.1 CI |
binding_schema_version is an integer; non-integer values (decimal point, string suffix, semver-triple) fail with BINDING_SCHEMA_UNSUPPORTED at load (docs/catalog-abi.md binding-version patch prohibition) |
TestBindingSchemaVersionInteger |
v0.1 CI |
prompts/get argument-rejection transport returns JSON-RPC error.code = -32602 (InvalidParams) with error.data.error_code = "INVALID_ARGS" for argument maps supplied to zero-argument v0.1 prompts (§7) |
TestPromptsGetInvalidArgs |
v0.1 CI |
gum.code with a reserved language value (starlark, yaegi, js, python) and with any other unknown value is rejected by the §4.1 tool-argument validation seam: the tools/call frame carries isError: true and the INVALID_ARGS envelope naming gum.code and the rejected value, the JSON-RPC error member is absent, and the handler never runs so no script executes (§4.3) |
TestCodeReservedLanguageRejection / TestCodeAcceptsRisor |
v0.1 CI |
MCP roots binding: the session root set is resolved once, on the first tools/call that needs project-local profile resolution, through the SEP-2322 InputRequests/InputResponses round trip (§9.2 step 1); the cache is sticky, so a client-side roots change after that first call does not change what later calls in the same session resolve (§13.2) |
TestProfileResolutionFromMCPRoots / TestRootsCacheStickyAcrossRootsChange |
v0.1 CI |
gum://results/{hash} lifecycle polling pattern: the shipped binary wires the §9.0 tee stage, so real calls emit full_result_path, full_result_resource and artifact_expires_at; the advertised expiry follows output.tee_retention_hours and the read scan window follows the same value, so an artifact inside the advertised window is still readable; clients copy artifacts before artifact_expires_at; GUM emits no notifications/resources/updated for results in v0.1.0 and refuses resources/subscribe; reads after expiry return the stable RESULT_ARTIFACT_EXPIRED envelope (§13 R28B-S1-7 reframe) |
TestResultArtifactExpiredError / TestFindArtifactRespectsMaxDays / TestFindArtifactReturnsFalseForMissing / TestReadAbsentArtifactReturnsErrNotExist / TestMCPInitializeCapabilities / TestDispatcherConfigWiresTheTeeStage / TestTeeConfigForProfileReadsGumConfig / TestArtifactExpiresAtFollowsConfiguredRetention / TestArtifactHandlesTravelTogether / TestNoArtifactHandlesWithoutRecovery / TestResultsScanWindowFollowsConfiguredRetention / TestResultsScanWindowStopsAtTheDefault / TestResultsReadsEmitNoResourceUpdatedNotifications / TestResultsSubscribeIsRefused |
v0.1 CI |
ParallelResultItem schema (§13 $defs/ParallelResultItem) is the validating schema for ParallelResults.results[] items; a real toon-carrying item is rejected by the ToonResult schema; each item requires _idx and _expression (an ExpressionMetaDelta), including when hoisting empties the delta; the optional sibling _code_output_truncated: boolean is accepted on the item, typed boolean, and never emitted inside _expression or the shared pool (R28B-min-5). The outer ParallelResults._code_output_truncated is set whenever at least one element carries it, and stays absent when no element lost bytes |
TestParallelResultItemSchema / TestGumParallelCodeOutputCeiling / TestCodeOutputBudget |
v0.1 CI |
Goroutine-leak verification: TestPollTimeoutAndCancellation and TestGumParallelResultEnvelopeCompression end with defer goleak.VerifyNone(t); the pinned uber-go/goleak floor is v1.3.0+ (Appendix A) |
(extended assertions on existing tests) | v0.1 CI |
testscript-based CLI contract tests for gum call, gum code, and gum auth flows: testdata/script/*.txtar files drive deterministic stdin/stdout/stderr assertions; the pinned rogpeppe/go-internal floor is v0.13.0+ (Appendix A) |
TestCLIScripts |
v0.1 CI |
Filesystem fsync fallback (§8.7 line 1938): on a filesystem where fsync returns EINVAL or ENOTSUP, the plugin install transaction in internal/plugins/registry does NOT fail. It publishes a complete generation, logs one structured {"event":"fsync_not_supported","path":...,"syscall_errno":...,"suggestion":...} warning to the host log AND one row to the profile audit log, names the errno rather than "unknown", and dedupes once per profile per process rather than per transaction. A real (non-ENOTSUP) fsync failure still aborts and rolls back |
TestWriteTransactionSurvivesUnsupportedFsync / TestUnsupportedFsyncWritesAuditRow / TestErrnoNameNamesEveryErrno / TestFsyncUnsupportedErrorStaysTransparent / TestWriteTransactionAbortsOnARealStagingFsyncFailure |
v0.1 CI |
TOON forward-compatibility: plugins emitting TOON with an unsupported version stamp fail with TOON_VERSION_UNSUPPORTED structured error rather than silent best-effort decoding (§9.0) |
TestToonVersionUnsupported |
v0.1 CI |
tee.secret lifecycle independence (§9 lifecycle point 5): the secret is keyed to profile identity alone. A second LoadOrCreateSecret returns the same key and does not rewrite the file; writing and rewriting every artifact an index rebuild would touch in the profile dir (embeddings.model, embeddings.bin, bm25.bin, catalog.json) changes neither the key nor the derived artifact handle; two profiles never share a secret or a handle; and a corrupt secret is refused rather than silently rotated. v0.1.0 ships no embedding-model identity to rotate: §5.7 reserves embeddings.bin as a placeholder, no embeddings.model exists in the tree, and internal/embed is bm25-only-v1 |
TestTeeSecretEmbeddingIndependence |
v0.1 CI |
§9.0.1 aggregate batch output ceilings: buildParallelFn reads the enclosing script's live §6.1 remainder and the configured concurrency. A batch whose declared input (the {"elements":[...]} object §12.3 hashes) exceeds code.output_limit_bytes × 8 is refused before the first dispatch with CODE_OUTPUT_LIMIT_EXCEEDED carrying limit_bytes and requested_bytes; raising code.output_limit_bytes raises the ceiling proportionally. A batch that overruns the remainder mid-encode cuts later elements at a UTF-8 boundary, sets _code_output_truncated: true on each element that lost bytes, and sets the outer envelope flag; a cut failure element keeps its error_code, and an element with nothing cuttable is left whole and unflagged |
TestGumParallelCodeOutputCeiling / TestParallelItemTruncationArms / TestCodeOutputBudget / TestBudgetReportsTheLiveRemainder |
v0.1 CI |
Meta-tool admin tuning: meta_tools.search_apis.truncate_strings.default_chars and meta_tools.search_apis.collapse_arrays.max_items are configurable from the admin tuning surface (§9.4); defaults and overrides are validated at config load |
TestMetaToolAdminTuningTruncateChars / TestMetaToolAdminTuningClampsTruncateChars / TestMetaToolAdminTuningCollapseMaxItems / TestMetaToolAdminTuningDefaultsWhenAbsent / TestMetaToolAdminTuningClampsCollapseMaxItems / TestMetaToolAdminTuningUnparseableTruncateChars / TestMetaToolAdminTuningUnparseableCollapseMaxItems |
v0.1 CI |
Audit log graceful shutdown: SIGTERM triggers a 2-second drain (configurable via audit.drain_timeout_seconds, 0-30, default 2) of pending audit entries; audit.broken is NOT set on clean exit; structured audit_drain_complete event is emitted (§11) |
TestAuditGracefulShutdown |
v0.1 CI |
Keychain unavailable policy: when neither native keychain nor encrypted-file vault is available (e.g., read-only filesystem), credential reads fail with AUTH_KEYCHAIN_UNAVAILABLE and user_message suggests gum auth use-adc; the error never silently falls back to an in-memory cache (§7) |
TestAuthKeychainUnavailable |
v0.1 CI |
HELP_TOPIC_TOO_LARGE: any apps/gum/internal/help/topics/<topic>.md whose rendered body exceeds 8 KiB fails the build with HELP_TOPIC_TOO_LARGE (§7 error code table; §13 gum://help/{topic} cap) |
TestValidateSizesRejectsOversizedTopic / TestErrTopicTooLargeError / TestRegisterHelpResourcesPanicsOnOversizedTopic |
v0.1 CI |
Every backticked SNAKE_CASE identifier quoted by an apps/gum/internal/help/topics/<topic>.md file appears somewhere in the repository outside that directory, so a help topic cannot teach a model client to branch on an error code or environment variable the runtime never emits and no contract names |
TestHelpTopicIdentifiersExist |
v0.1 CI |
gum.poll.operation_name description is the canonical name for the LRO handle argument; the description is reachable through gum.describe_op("gum.poll") and matches the v0.1 contract |
(extended assertion on TestTierAPerToolInputSchemaBudget) |
v0.1 CI |
Compound auth token forwarding (§7 R28A-M-5): a manifest declaring the reserved google_access_token gets GOOGLE_ACCESS_TOKEN in the subprocess env from HostConfig.TokenResolver, and the host appends exactly one plugin_token_forwarded audit entry carrying plugin, subject_fingerprint, and a copied scopes slice. The forwarded token overrides any stored gum plugin setup value under the same name |
TestCompoundAuthTokenForwarded, TestCompoundAuthTokenBeatsStoredCredential, TestCompoundAuditEntryShape, TestCompoundAuditEntryScopesAreACopy, TestGoogleTokenForwarderCarriesCredential |
v0.1 CI |
Compound token forwarding refuses every substitute for the host's live token: with no resolver, or a resolver reporting no session, GOOGLE_ACCESS_TOKEN is left unset rather than filled from a stored secret; an ambient GOOGLE_ACCESS_TOKEN in the parent environment is not passed through even when env_allow lists it; a resolver error fails the spawn instead of starting the plugin tokenless (§7) |
TestCompoundAuthTokenAbsentWithoutResolver, TestCompoundAuthTokenEmptyResultForwardsNothing, TestCompoundAuthTokenIgnoresAmbientValue, TestCompoundAuthTokenResolverErrorFailsSpawn, TestCompoundAuthTokenNotForwardedWhenUndeclared, TestGoogleTokenForwarderRequiresLogin |
v0.1 CI |
Compound strategy gate at install: because one env block serves the whole subprocess, a manifest declaring google_access_token must advertise at least one tool and must declare auth_strategy: "compound" on every advertised tool, else PLUGIN_ENV_PROHIBITED; an unrecognised auth_strategy fails as an invalid manifest; exactly one production file emits the plugin_token_forwarded literal (§7) |
TestGoogleAccessTokenRequiresCompoundTool, TestLoadManifestRefusesNonCompoundTokenDeclaration, TestLoadManifestAcceptsCompoundTokenDeclaration, TestLoadManifestRejectsUnknownAuthStrategy, TestUndeclaredReservedNameIgnoresStrategy, TestExactlyOneProductionSiteEmitsPluginTokenForwarded |
v0.1 CI |
Code-mode LRO refusal (§6.1): gum_call and gum_parallel refuse an op whose default variant declares the lro_return capability atom (§5.8) before dispatch, with LRO_UNSUPPORTED_IN_CODE, an op_id detail, and the documented message; the per-element gum_parallel envelope carries the same code; an ordinary op still dispatches, and a dispatcher that does not implement LROClassifier leaves the gate open rather than failing closed. cmd/gen-catalog stamps the atom from the Discovery response schema, matching on the longrunning shape (done plus error) rather than the $ref name, so ListOperationsResponse is not classified. No shipped op qualifies: all 228 cataloged ops return a concrete resource, so the gate is proven on synthetic catalogs and a synthetic Discovery document |
TestLROUnsupportedInCode, TestLROUnsupportedInParallel, TestLROGateLetsAnOrdinaryOpThrough, TestLROGateTolerantOfAClassifierlessDispatcher, TestDispatcherReturnsLRO, TestDefaultVariantReturnsLROWithADanglingDefault, TestMethodReturnsLRO, TestMethodReturnsLROWithoutSchemas, TestStampLROReturnCoversRESTVariantsOnly, TestStampLROReturnIsIdempotent |
v0.1 CI |
gum_print UTF-8 truncation: when the per-call byte limit is reached, the output is truncated at the last valid UTF-8 boundary before the limit; partial multi-byte sequences are excluded; the truncated suffix never contains a half-encoded character (§6.1) |
TestGumPrintUtf8Boundary |
v0.1 CI |
gum_print value encoding: a string prints verbatim; every other value, including nil, prints as JSON with HTML escaping off, so gum_print(gum_parallel([...])) yields a parseable envelope rather than a Go map[...] dump (§6.3) |
TestGumPrintEncodesStructuredValuesAsJSON |
v0.1 CI |
Registry remains structurally mutable post-Server.Run (§4.2 forward-compat invariant): Server.AddTool is callable at the object level on the v0.1.0 internal/mcp server after Run returns. The v0.1.0 behavioral policy that forbids mid-session registration is enforced by a session-scoped gate in internal/dispatch, NOT by structural prohibition; this enables the v0.2.0 tools/list_changed enablement to land without re-architecting internal/mcp |
TestMCPRegistryStructurallyMutable |
v0.1 CI |
confirmation_purpose="high_stakes_write" reserved-value rejection (§6.1.2): a confirmation_token whose embedded purpose is "high_stakes_write" (the v0.3.0 reserved value, NOT emitted by a v0.1.0 dispatcher) presented to a v0.1.0 dispatcher returns CONFIRMATION_TOKEN_INVALID with data.reason="unknown_purpose"; the closed-enum check happens before HMAC verification so that a forged token does NOT reveal whether the HMAC would have matched |
TestConfirmationTokenUnknownPurposeCheckedFirst |
v0.1 CI |
tee_mode="failures" step-6 vs step-7 boundary (§9.1, §expression-profile-dsl.md tee_mode): a RATE_LIMITED error raised by the in-process token bucket at step 6 of the dispatch lifecycle (§3.1) does NOT write a tee artifact (failure occurred before the executor); a RATE_LIMITED error raised by an upstream HTTP 429 at step 7 DOES write a tee artifact (failure occurred during the executor). The fixture pair moves the failure across the boundary and changes nothing else: one dispatch fails in a stubbed token bucket, the other in a stubbed executor returning an upstream 429, and an executor-entry counter proves the step-6 run never reached the upstream leg. The step decides whether an artifact is owed, not the error code and not the mode: a step-6 refusal writes nothing even under tee_mode="always", and a step-7 429 writes nothing under tee_mode="off" |
TestTeeFailuresStep6VsStep7 |
v0.1 CI |
cmd/gum release-binary CGo-freeness across the full release matrix (§15 release build flags): go list -deps ./cmd/gum/... returns an empty intersection with the set of CGo-flagged packages for each GOOS/GOARCH pair in (linux/amd64, linux/arm64, darwin/amd64, darwin/arm64); the assertion is broader than TestNoCGoInDispatch (which only covers internal/dispatch) and catches keychain-backend regressions before tagging |
TestReleaseBinaryNoCGo |
Release gate |
internal/pluginenv/denylist.txt single-source invariant (§14 pluginenv row): the SHA-256 of the go:embed-ed denylist material in internal/pluginenv and the SHA-256 of the go:embed-ed material in cmd/gen-catalog (which embeds the same file via a relative path or build-time copy) MUST be identical; the test fails if either embedding site drifts |
TestPluginEnvDenylistSingleSource |
v0.1 CI |
auth_strategy enum extension completeness (§7 extension procedure): the gate enforces the (a)/(b)/(c) check against the closed enum in internal/auth/strategy.go minus the v0.1.0 baseline set (gum_oauth, byo_oauth, adc, service_account, api_key, compound, plugin_managed, none) which is already covered by pre-existing tests (TestAuthStrategyRequired, TestCompoundAuthErrorEnvelope, etc.). For every residual value: (a) it appears in docs/catalog-abi.md's auth_strategy cross-reference, (b) it has an internal/auth/strategy_<name>.go file, and (c) this matrix contains a row whose test name is exactly TestAuthStrategy<Name> (CamelCase). The baseline set is enumerated in the test source so removing a value from it re-arms the residual check, and adding a constant to internal/auth/strategy.go without adding it to the baseline arms it immediately; the baseline is the ten Strategy.String() values v0.1.0 shipped, so the residual set is empty today and the (a)/(b)/(c) checks are additionally proved against fixtures |
TestAuthStrategyEnumExtensionComplete / TestResidualStrategyChecksReportEveryMissingStep / TestResidualStrategyChecksPassOnACompletePR / TestAuthStrategyCrossReferenceIsScopedToItsBullet |
v0.1 CI |
Per-package line-coverage retention gate over the full tracked surface — ./cmd/gum/... and ./internal/... (internal/coverage.GatedPackages), excluding build-time tooling (cmd/gen-*, cmd/measure-tier-a, cmd/test-matrix, cmd/coverage-floor) and the generated, gitignored gen/dispatch tree (beads gum-b22o.5, gum-5wkg, gum-8ilq, gum-ql6c, gum-syvp). internal/coverage.FloorPercent (85%) is the absolute minimum for any un-listed package; internal/coverage.Ratchets pins every tracked package at a retention baseline of floor(current − ~1% jitter headroom) (a baseline Min may sit above or below 85%, recording the level actually held). make coverage-floor (CI job coverage-floor, single Go toolchain) measures per-package coverage via go test -coverprofile, parses the raw profile, and fails when any package falls below its effective threshold (ratchet Min, else 85%). Baselines are measured on linux (internal/coverage.BaselineGOOS), so on any other GOOS it prints violations as warnings and exits 0; the Linux CI job is the gate. It also prints a non-failing RATCHET_OPPORTUNITY hint when a package exceeds its baseline by RatchetOpportunityMargin (2%). Lowering a ratchet Min requires a matching test-matrix note plus an owning Bead; gum-ql6c is the explicit v0.1 release-candidate recalibration after the gum.code/trust-boundary hardening and catalog-depth work shifted package denominators; gum-syvp is the 2026-09-19 recalibration after the repo-wide review sweep, which raised every stale Min to floor(measured on linux − 1) and gave internal/agents, internal/output/render, and internal/skills their first entries. internal/pluginenv and internal/config carry extra headroom instead: pluginenv because its two Landlock tests skip on a kernel without Landlock, config because its Save error-branch tests skip when euid==0. Undocumented lowering or removing a Bead reference is itself a regression and breaks TestRatchetEntriesHaveBeadReferences |
TestFloorIs85 / TestRatchetEntriesHaveBeadReferences / TestRatchetEntriesAreUnique / TestGatedPackages / TestCheckRespectsRatchet / TestOpportunitiesFlagsImprovedPackages / TestParseProfileAggregatesPerPackage / TestViolationFailsOnlyOnBaseline |
v0.1 CI |
Release claims such as ">=80% savings" may cite only fixture-backed local gain-ledger entries. Privacy-minimized telemetry is product telemetry, not release-gating evidence.