Testing JudgeAI's decisions.

Our internal studies examine how empirical research changes normative decisions and reveals preferable alternatives. Explore the computational findings, complete proposals and comparisons with language models, judicial decisions and legislation.

How deeper research improves decisions

JudgeAI makes normative choice computable and traceable. This allows autonomous research to investigate the grounds of a decision, identify consequential gaps and evaluate additional interventions. Our latest internal study tests these mechanisms through a UAE regulatory case.

Consequences change the choice. Removing relationships between affected participants changed the preferred option in 176 of 700 scenarios.

The calculation identifies what needs investigating. Three linked inputs resolved the choice across all 700 tested scenarios; knowledge of the direct effect alone did not.

Additional alternatives can improve the result. Expanding the intervention set produced a strictly preferred option in 128 of 3,456 scenarios under the same normative criterion.

Research can begin before a regulatory task is fixed. A separate experiment explored 16 threat directions, investigated local care and cooling mechanisms, and recalculated the shared model after adding human contexts. Its 285 scenario records demonstrate the research cycle; they do not establish an optimal ranking of threats or measured intervention effects.

These findings support the hypothesis that JudgeAI can produce better-grounded normative decisions than human analysis. Traceability enables systematic investigation of consequential relationships and alternatives that a less complete inquiry leaves unexplored, carrying each finding into subsequent comparisons.

Study and methodology

JudgeAI and GPT-6: Same-prompt evaluation, with an autonomous legislative-development study

JudgeAI and GPT-6: Same-prompt evaluation

Given the same open-ended task for Amsterdam, GPT-6 reproduced the existing policy direction of reducing tourism. JudgeAI investigated the circumstances and compared five approaches, including continuation of that policy. Its calculation ranked regulation of local impacts and service capacity first. Follow how this choice changes the permissions, financing and implementation in the two complete proposals.

A second case follows automated legislative development in England and Wales. The AI orchestrator independently investigates the dependencies exposed by its decisions, gathers further evidence and runs new comparisons. Six calculations develop a bill for energy improvements in rented homes, connecting actual funding, tenant savings, safe staged work and effective repairs. The report maps this research to the resulting clauses.

Read the two case studies
AI Lawmaking empirical and GPT baseline comparison PDF preview

Legislation and its observed consequences

Twenty generated bills are compared with historical legislation and evidence about its effects. The central question is whether JudgeAI avoids mechanisms associated with poor outcomes: abrupt cash withdrawal, excessive incarceration, reduced housing supply or school incentives that narrow the curriculum. Cases with documented benefits provide a further comparison of the mechanisms the system retains or changes.

Each case brings together the regulatory problem, the historical regime, empirical sources, the complete generated bill and a comparison of their mechanisms. A separate comparison with GPT-5.5 Extra high examines the regulatory choices made in response to the same requests. The historical evidence and proposed provisions can be inspected together.

Compare the bills and evidence
AI Lawmaking Test Results PDF preview

Avoiding reactive restrictions

This study compares JudgeAI proposals with existing laws across 22 regulatory tasks. The system received abstract problems; the proposals were then compared with legislation addressing those problems. The focus is how the system handles broad prohibitions, status-based restrictions and measures adopted in response to particular political events.

The cases include religious symbols, foreign funding, platform access, gaming restrictions and digital taxation. The report places existing provisions beside the generated alternatives, showing differences in scope, safeguards, consequences and implementation. Readers can examine whether each proposal addresses the underlying problem and how it treats the people affected.

Compare the regulatory choices
JudgeAI as AI Judge Test Results PDF preview

JudgeAI and judicial decisions

The 2024 internal tests compare JudgeAI decisions with court judgments and arbitral awards across 92 cases. They examine where the decisions follow a similar direction and where they differ in their treatment of facts, liability, remedies and procedure.

A human judgment is a comparison point, not a ground-truth answer for measuring model accuracy. Each entry presents the dispute, the judicial outcome and the JudgeAI decision, with sources and a concise comparison. The material allows readers to examine the reasoning and outcomes of both decisions, case by case.

Compare the decisions

An expanding research programme

We continue to invest in JudgeAI's normative engine and experimental programme. Published internal studies establish how empirical dependencies and alternative discovery affect normative choice. Ongoing development extends these capabilities to further problems and supports work towards external validation.

Research and self-improvement