Prompting Isn't the Skill. Evaluation Is.

From Prompt Craft to Evaluation Discipline

Prompt engineering gets demos on Twitter. Evaluation engineering gets products through compliance reviews. At Band9AI, every scoring surface is tested against representative learner responses with expected band ranges and criterion-level expectations.

When we change prompts or update models, we don't ask 'does it feel better?' — we measure against defined quality criteria and block release if results regress.

Quality Checklist (Conceptual)

{
  "quality_checks": [
    {"check": "band_estimate_in_range", "question": "Does the score match the response quality?"},
    {"check": "criterion_flags_accurate", "question": "Are penalty patterns correctly identified?"},
    {"check": "no_false_official_claims", "question": "Does output avoid impersonating official examiners?"},
    {"check": "actionable_feedback", "question": "Can the learner act on this immediately?"}
  ],
  "principle": "Measure before you ship — every time"
}

Drift Monitoring in Production

Model providers silently update weights; a prompt that worked in February may regress in July. Ongoing sampling against known-good cases catches drift before learners feel it.

Evaluation is not a phase. It is the product.

Evaluation Rituals That Survive Model Churn

Providers change models; prompts rot. Teams that only store 'the prompt that worked last month' rediscover failures in production. Persist test cases, expected band windows, and forbidden phrases. Run them on every meaningful change.

When evaluation is the skill, prompting becomes an implementation detail you can rewrite safely — because the tests tell you whether the product still behaves.