ai-quality

📚 A curated list of papers & technical articles on AI Quality & Safety

Proven 2026 Multi-Agent AI Review System – Verdict-Driven Quality Control

Open-source AI model evaluation and benchmarking framework for LLMs (OpenAI, Ollama, Claude, Gemini)

Ship evals before you ship features.

Eval framework. Define correct, test against it, get results.

Find what your AI agent gets wrong — before you have a rubric. Qualitative eval for PMs.

Diagnose your AI agents in production. Extract policies from prompts, evaluate traces, generate diagnostic reports.

A framework-agnostic metric for measuring AI code generation quality. Sealed-envelope testing protocol + reference validators.

AI Content Guard 2026: Adversarial Fact-Check Gate for Error-Free Code & Copy

Open-source AI agent security testing framework. Test for prompt injection, data leakage, and privilege escalation before production.

Provider-agnostic LLM evaluation harness: golden dataset, deterministic + LLM-as-judge scoring, RAG failure attribution, red-team suite, severity-weighted CI gate.

Python SDK for IvyCheck

Evaluate your LLM apps with one function call. Hallucination detection, RAG scoring, and agent evals for OpenAI, Anthropic, and more. 14 evaluators, pytest plugin, composite trust scores.

AI Agent Ops framework for Claude Code — independent evaluator, adversarial review, and pre-commit quality gate for AI-generated code.

AI quality method for validating LLM output against predefined criteria. Includes a builder-critic workflow, evidence checks, cost limits and stopping conditions.

朱雀 Suzaku — AI 生成品質模組。諂媚抑制、建設性挑戰、輸出適配、上下文錨定、一致性守護。基於 LDRIT 設計。

Universal skill enhancement layer for Claude Code. Sees what your skill was trying to do, grades the gap, drives the rewrite.

LLM agent harness with versioned prompts, runtime guardrails, automated evals and a CI quality gate. Runs offline with no API key.

Self-improving AI quality system - auto-score, auto-regenerate, auto-learn.

AI evaluation platform for measuring reliability, evidence grounding, and benchmark performance of LLM systems.