Agents are automating the ‘white box’ world of decision-making

Data defines the opportunity (and the minefield)

Imagine you are designing a first in-human clinical trial.

You need to decide how many sites to open, and in which countries? Having done so, you will also want to have a good understanding of how long recruitment will take.

If you get this wrong in one direction, you will burn millions keeping empty sites open. Get it wrong in the other direction and you stall as a trial, taking too long to recruit and risk both technical and operational success.

This is one of the highest-value decisions in all of drug development, and it’s your responsibility.

So what do you do? If you reach for AI (as so many people do these days) you almost immediately hit a problem: there’s no clean history to learn from. This molecule has never been used in humans. The indication itself might be rare. The countries might have different regulators and patient pools. 

It gets worse. The handful of past trials that do look vaguely similar were run with different regulators, protocols and their records have fields which are either missing or just plain wrong. 

This is the uncomfortable truth that circles around most decisions that really matter: the data is often irrelevant, incomplete, scarce or wrong. Note this is not because anyone was careless; but it’s because most decisions that really matter are almost by definition rare, novel and high-stakes!

Let’s try to be a little more positive: when you do have tons of high-quality data, and a prediction problem which aligns well with it, black-box AI is genuinely awesome.

What do I mean by ‘black-box AI’? It’s a model which will predict an outcome (often way better than any other scientific modelling methods) but which can’t be interrogated for why it has made a particular prediction. Note that the ‘why’ here is a very human-specific thing and often requires tailoring to specialist users or expert judgement.

This sounds negative. So let me be clear once again: I’ve worked as an academic in both Physics and Epidemiology and have seen the outdated opinions of many technically excellent researchers get burned by the sheer automatic power of AI in the right data situations. But we’re often not in those situations in the messy real world of important decision making.

How can we resolve this? Stop treating data as the only source of information.

Here’s the idea: make up the difference in data with more informed model structure. The trick to doing this is that this model structure can encode facts you (or other experts) already know about the system the data was collected from.

You know a regulator won’t approve faster than its own minimum review period. You know a site can’t enrol patients before it has been activated, and that there are plain physical limits to the rate of enrollment that you can reason about without ever having run the same trial before.

This idea is already talked about heavily in all sorts of AI circles. It’s more commonly known as inductive bias. Black-box AI often deliberately has the thinnest, least opinionated version of this. Models of the world (or ‘world models’) are typically more rich in opinion; and simulations are the clearest example in this category.

Simulations are ‘white box’ AI

Simulations keep a picture of what the world looks like and how the world works separately from the data. This is what makes them world models, and what distinguishes them from black-box AI. By contrast, the black-box retains no clear, human-interpretable picture of how the world looks and works.

Let’s start designing that first in-human trial again. Only this time, we’re going to rely on a simulation to help us do it.

The simulation has machinery that actually represents the world, so it’s obvious that: regulators will approve on a clock; patients will be enrolled at each site with a feasible rate; dropouts will occur to reduce the sample size; and a target enrollment figure must be hit before we assess the timeline. It’s all in there.

So what are the big advantages?

First, you can pour in more expert knowledge that never existed as data. An expert who has run 40+ oncology trials just knows that this site is slow to start but recruits fast once it does. 

Second, you can ask the simulation why and get a human explanation in return. Why will my recruitment take 14 months? The simulation tells you 11 of those months are one slow regulator in one country. Black boxes just give you the number 14 and ask you to trust them.

This is so important because a decision you cannot defend is a decision you cannot make.

Agents make the white box cheap to develop (the original benefit of black box)

How come this white box approach isn’t used for everything?

As I’ve implied above; there are still many situations where the data quality is good enough, it is complete, it is directly relevant to the problem and it is plentiful. It’s also important that verifying the results of decisions made using black-box methods should be cheap and not dangerous. 

Under these conditions (especially that last one), it’s often much better to go with big black boxes that optimise for predictive performance.

Think of AlphaFold. A truly revolutionary technology that has earned (very rightly) a Nobel prize. 

AlphaFold is less of a simulator/world model and more focused on predictive accuracy. Quite rightly so! Verifying the outcomes of decisions with AlphaFold is extremely cheap and fast, with no reputational/legal risk to the decision-maker. Why not make the predictions as good as possible? Explainability is much less important here.

Contrast this with the outcomes of a clinical trial and you’re in for a world of problems.

There was, until very recently, another reason to avoid the white box approach; they were expensive and time-consuming to develop. A good simulation of a business process is a serious piece of software engineering, requiring time, expertise and effort to create. 

Coding agents have changed this picture substantially

• Acceleration #1:

The cost of building world models has fallen through the floor. We’re now in a world where the domain expert can stay at the level of expertise (sites activate on this schedule, patients arrive at this rate, here’s the regulatory clock) while watching their agents create the software in minutes.

• Acceleration #2: 

This one is pretty cool. What are predictive systems like AlphaFold or WeatherNext really? 

You might think of them as gigantic simulation emulators. They are trained to emulate what a simulation (protein folding, atmospheric physics) cannot predict at the right scale or fidelity. The black box learns to mimic the structured model and then runs in a fraction of the time, and corrects for predictive biases where data is available to do so. 

But the simulation could remain to provide a reasonable (if not perfect) human explanation nonetheless. Or could even orchestrate the results of black boxes at a different scale of modeling.

Simulations as a tool for agents

Let’s put all these pieces together.

We now have agents that build simulations cheaply, humans can interrogate them to understand why and black box models can emulate and run them quickly.

I think the natural next step is agents which run and interrogate the simulations themselves.

This is where things get interesting. Most agents today operate over information and tools: retrieving data, calling APIs, writing code or executing workflows. Give an agent access to a simulation of the system it is operating within, and it can do something different. It can explore the consequences of a decision before that decision is made.

It can ask: what happens if we do this? Why does the simulation expect that outcome? Which assumptions or constraints are driving it? What would need to change to produce a better result?

Back to our first-in-human clinical trial. An agent can now rapidly configure study design scenarios and simulate their impact on the metrics that matter most - from recruitment and timelines to cost and operational risk. Development teams can explore more possibilities, understand the trade-offs, and design trials with greater confidence, learning from each decision over time. Ultimately, that means better decisions, faster trials, and better medicines reaching patients sooner.

I’m excited to say that this is already happening at Faculty Frontier™ with some of the world’s largest pharmaceutical companies. 

And clinical trials are just one example. Wherever decisions are complex, high-stakes and difficult to learn from historical data alone, agents and simulations open up a new possibility: automating not just the work around a decision, but the process of understanding what to do next - and why.