Citizen-Bench
Can AI answer citizens' legal questions? The same 77 real questions, asked in Algeria, France and the US. Every model gives its weakest answers in Algeria.
Average score, same 77 questions
Our benchmarks show where frontier models fail. Our datasets help them improve. Every dataset starts as a sample of 5 to 100 items, with a datasheet and model results where the task can be scored.
Can AI answer citizens' legal questions? The same 77 real questions, asked in Algeria, France and the US. Every model gives its weakest answers in Algeria.
Average score, same 77 questions
New benchmarks are on the way. Each one tests a skill that current models get wrong.
Is there a task AI keeps failing for you? Tell us and we may build the benchmark.
Contact usA professional solves a design brief in open-source CAD tools. Screen video, action log and the finished file.
Request samplesOriginal parametric parts with their brief and step history, plus how frontier models do on the same brief.
Request samplesFirst-person video of skilled manual work in real workshops. Anonymised and described step by step.
Request samplesAnonymised codebases and project archives from real companies, each with a qualification report.
Request samplesTasks in LibreOffice, GIMP, Inkscape and QGIS, each with an automatic checker and agent pass rates.
Request samplesBriefs, source files, expert deliverables and scoring rubrics in architecture, engineering and accounting.
Request samplesFull working sessions where a professional reads several sources, then does the work.
Request samplesWe build to specification. Describe the skill and the format, and we'll scope a sample.
Describe your needExclusive and non-exclusive licences available.
Source, collection method, schema, licence and known limits.
Pass rates for named frontier models, where the task can be scored.
How many items we can produce, and how fast.