Policy in text, not weights:
skills as the audit bridge

When a decision procedure is invisible, it is ungovernable. AWS just shipped 38 open-source agent skills for healthcare and life sciences (HCLS) that encode domain decision procedures as plain markdown — SKILL.md files with YAML triggers, evaluated head-to-head at 70–86% win rates. Our judgment: policy-as-text is the audit bridge for regulated AI — when the procedure lives in a text file, compliance review becomes diff review, and policy changes become deployments instead of retraining. This is the same architecture OOMeta runs internally. This piece explains what skills are, what the numbers actually show, and what it changes for regulated buyers.

1. The failure mode: the model knows facts but not the procedure

Ask an agent to classify a TP53 missense variant using ACMG/AMP criteria: it will cite the correct framework yet misapply evidence categories, skip population-frequency thresholds, or hallucinate computational predictor scores. The model knows facts but lacks the structured reasoning procedures practitioners internalize over years of training. The gap produces silent failures across variant interpretation, claims adjudication, clinical trial design, and imaging analysis — outputs that look correct but apply the wrong criteria, with regulatory and patient-safety consequences. Source: AWS ML blog.

2. What a skill is: SKILL.md, progressive disclosure, not RAG or fine-tuning

Skills come in two flavors. Reasoning skills encode methodology and decision frameworks — the genomic-variant-interpretation skill carries the full ACMG/AMP classification, evidence categories, population thresholds, and predictor cutoffs. Pipeline skills carry tool-specific commands and validated parameters — the variant-calling skill provides GATK4 HaplotypeCaller commands, VQSR tranche sensitivity targets, and Mutect2 tumor-normal configurations. Unlike RAG, which retrieves passages to augment generation, a skill encodes the decision procedure and its error conditions. Unlike fine-tuning, it is a structured prompt activated contextually by trigger patterns in the query.

Three properties make the difference. Auditable — every decision criterion is human-readable markdown, not hidden in model weights. Portable — the same skill runs across 20+ services without per-service customization (Bedrock AgentCore, Strands, Claude Code, Codex, and more). Maintainable — an annual medical policy change is an edit to a text file, not a retraining. For regulated buyers, the first two are exactly what an audit asks for.

3. The evidence: 70–86% win rates, strongest on critical thinking

On a 410-prompt paired evaluation, skill-equipped agents won 70–86% of head-to-head matchups against the same agents without skills, varying by harness setup. The strongest effect was on critical thinking — 78–85% win rate, Cohen’s d = 0.65–1.03 — and the biggest lift came on the hardest queries, exactly where unguided agents fail most. Methodology caveat: this is a vendor-published evaluation and win rates vary by harness; but the evaluation framework is open, so you can rerun it on your own prompts.

4. Context engineering: specialists beat loading everything

Loading all 38 skills into a single agent context consumes ~80K tokens — workable on large-context models, but the agent must select a subset on every query while irrelevant content competes for attention. The multi-agent answer: a lightweight coordinator (no skills loaded) routes queries to eight domain specialists, each loading only its relevant skills (about 15K tokens per specialist). The coordinator handles intent classification; the specialists handle domain reasoning.

This is progressive disclosure at fleet scale: keep the routing layer lean, load domain procedure lazily. For any team building multi-agent systems, this is directly copyable — ask “how much context does each agent actually need” before asking “which model”.

5. Our judgment: policy-as-text is the audit bridge

The decision procedure is the compliance artifact. When it lives in a text file, auditors read diffs, not weights; reviewers can read the exact criteria the agent applied; and a policy update is a deployment (file edit plus re-evaluation) rather than a retraining. “Auditable, portable, maintainable” are not engineering nice-to-haves — they are the compliance properties regulators care about.

We are not commenting on someone else’s architecture from the outside — OOMeta runs the same stack internally: SKILL.md, trigger routing, progressive disclosure, with skills as the surface of our own product. This is a pattern we dogfood. The conclusion for regulated buyers: the question is not “which model” but “where does the decision procedure live” — if it lives in the weights, you cannot review it.

6. Action and the question left for buyers

Action: pick one high-frequency decision workflow; clone a relevant skill (or write your own SKILL.md); set your thresholds to your protocols; run the included pairwise-evaluation framework on prompts from your own team; ship only when the win rate clears your bar. The question left for buyers: can a compliance reviewer read the exact procedure your agent followed — in text? If not, you do not have an audit bridge yet.

References: AWS ML Blog, Improving HCLS AI reasoning with open-source agent skills (vendor-published evaluation) https://aws.amazon.com/blogs/machine-learning/improving-hcls-ai-reasoning-with-open-source-agent-skills/ ｜ GitHub, awslabs/hcls-agent-skills (MIT-0) https://github.com/awslabs/hcls-agent-skills ｜ Agent Skills open standard https://agentskills.io