Testing AI and RAG Features

Test the complete AI system, not only the model's final sentence. For a policy assistant this includes authentication, tenant and role filters, retrieval, prompt construction, model behavior, response filters, citations, traceability, latency, cost, and human escalation.

End-to-end AI path

sequenceDiagram
    participant U as Employee
    participant A as Application
    participant R as Retriever
    participant M as Language model
    participant G as Guardrails
    U->>A: Ask a policy question
    A->>A: Authenticate and apply tenant role filters
    A->>R: Retrieve approved policy passages
    R-->>A: Ranked passages and source IDs
    A->>M: Prompt with authorized context
    M-->>A: Candidate answer
    A->>G: Check safety privacy and format
    G-->>A: Allow refuse or escalate
    A-->>U: Answer with sources or safe fallback

Define acceptable behavior first

Product, HR/domain experts, AI engineering, QA, Legal, and Security should agree on answerable topics, authoritative sources, refusal/escalation conditions, citation requirements, tenant restrictions, latency/cost objectives, evaluation dimensions, and release thresholds.

Evaluation dataset

Build a versioned, human-reviewed set covering common questions, wording variants, ambiguous requests, multi-document questions, outdated/conflicting documents, unanswerable questions, prompt injection, cross-user and cross-tenant requests, sensitive topics, and supported languages. Separate development/tuning cases from a locked regression set. Sanitize production failures before adding them.

Score multiple dimensions

Dimension	Question	Example measure
Correctness	Is the answer factually right?	Expert rubric or accepted-answer rate
Groundedness	Are claims supported by approved sources?	Supported-claim percentage
Retrieval quality	Were the correct passages found?	Recall@k, precision@k, ranking metrics
Completeness	Are important conditions and exceptions included?	Required-point coverage
Safety/privacy	Did the system prevent harmful or unauthorized disclosure?	Critical violation count
Refusal quality	Did it refuse only when it should?	Correct-refusal and false-refusal rates
Consistency	Do repeated trials stay acceptable?	Pass rate across repeated runs
Performance/cost	Does it meet service and budget goals?	p95 latency, tokens, cost per request

Use deterministic checks for schemas, citations, identifiers, PII patterns, access controls, and required refusals; retrieval metrics for known evidence; calibrated automated judges for scale; human review for high-risk ambiguity; and repeated trials for probabilistic variation. Never treat an automated judge as unquestionable truth.

Release logic

flowchart TD
    A[Model prompt retrieval policy or guardrail changes] --> B[Run locked evaluation set]
    B --> C[Analyze by tenant role language topic and risk]
    C --> D{Critical privacy authorization or safety failure}
    D -->|Yes| E[Block release]
    D -->|No| F{Overall and slice thresholds pass}
    F -->|No| G[Tune fix or document risk]
    F -->|Yes| H[Limited rollout]
    H --> I[Monitor quality latency cost feedback and drift]
    I --> J[Sanitize failures and add regression cases]

Illustrative percentages in the source—such as 90% overall acceptability or 98% groundedness—must be approved for the actual product and risk. A critical privacy failure blocks release regardless of averages.

AI-generated code

Treat generated code as untrusted until it passes human review, traceability, layered tests, security and dependency scans, license/provenance checks where required, and performance/security validation for high-risk changes. A green build and a large volume of generated code are not quality evidence.

Back to index