Safeguarded AI

We sought to test and accelerate the hypothesis that AI-enabled formal methods can make high-assurance cyber defence practical at scale. This call is structured as two tracks: Track 1 (Blue Teams) for building and verifying security-critical components, and Track 2 (Red Team) for adversarial evaluation. Applicants will be evaluated for their credible interest and ability to carry developed capabilities towards large-scale deployment, whether themselves or through spinouts, partnerships, upstreaming, or other routes.

We will announce the successful Creators in the coming months, sign up to get the latest programme updates.

This solicitation focused on TA1.1 Theory, where we sought R&D Creators – individuals and teams that ARIA will fund and support – to research and construct computationally practicable mathematical representations and formal semantics to support world-models, specifications about state-trajectories, neural systems, proofs that neural outputs validate specifications, and “version control” (incremental updates or “patches”) thereof.

For TA 1.2, we sought Creators to develop the computational implementation of the theoretical frameworks being developed as part of TA 1.1 (the ‘Theory’).

For TA 1.3, we selected Creators to work on the ‘Human-Computer Interfaces’ that facilitate interaction between diverse human users and the systems being built in TA 1.2 and TA 2 (‘Machine Learning’).

This solicitation sought teams from the economic, social, legal and political sciences to consider the sound socio-technical integration of Safeguarded AI systems. 

These teams will work on problems that are plausibly critical to ensuring that the technologies developed as part of the programme will be used in the best interest of humanity at large, and that they are designed in a way that enables their governability through representative processes of collective deliberation and decision making.

This Technical Area has been cancelled as part of a strategic pivot.

ARIA initially launched a multi-phased solicitation for Technical Area 2 (TA2) to support the development of a general-purpose Safeguarded AI workflow. The programme aims to demonstrate that frontier AI techniques can be harnessed to create AI systems with verifiable safety guarantees. In TA2, we will award £18m to a non-profit entity to develop critical machine learning capabilities, requiring strong organisational governance and security standards.

Phase 1 funded up to 5 teams to spend 3.5 months to develop full Phase 2 proposals.

The scope of this Technical Area has been repurposed into an updated TA2 focussing specifically on Applications.

The second funding call sought potential individuals or organisations interested in using our gatekeeper AI to build safeguarded products for domain-specific applications, such as optimising energy networks, clinical trials and telecommunication networks.

Safeguarded AI's success will depend on showing that our gatekeeper AI actually works in a safety-critical domain. The research teams selected for TA3 work with our other programme teams, global AI experts, academics and entrepreneurs, in setting the groundwork to deploy Safeguarded AI in one or more areas.