AI evaluation & data quality
We author and audit the tasks that frontier models are measured against. Current work includes auditing LLM text-to-SQL output for AfterQuery — judging whether generated SQL is correct against a real schema and the question's intent, and saying precisely where it fails. On Mercor's SWE-Agentic programme we authored agentic coding problems and graded model output against rubrics. For Turing we built Python services containerised with Docker against structured acceptance criteria, deterministic tests and reproducibility checks.
The discipline behind it is older than the AI industry. We designed the data quality assessment frameworks applied to national disease surveillance data used by WHO headquarters and government decision-makers, and trained over 500 staff in structured evaluation methods. Defining what "correct" means, then defending that definition to people who will challenge it, is the job.