This next week in Milan, the home of real-world evidence (RWE), the annual meeting of the International Society of Pharmacoepidemiology will take place. I am fortunate to be part of several posters, podium presentations, symposiums and a pre-conference course. Like many conferences these days, much of the program is focused on AI, with an increasing focus on agentic systems to perform various pharmacoepi tasks. This, and several conversations I have had this year, tells me there is a lingering concern about AI in the real-world evidence community. I mean, one of the sessions I am a part of is “Will AI steal my job?” But I want to alleviate the concern, fear and dread around AI. It is part of our world and will shape how we do science, errrr, pharmacoepidemiology.
There is a growing gap in pharmacoepidemiology between those who are still largely AI-naive and those who are becoming AI-fluent.
The gap can look intimidating. New models appear constantly. The language changes quickly. We hear about agents, retrieval, MCP servers, reasoning models, coding copilots, orchestration, and workflows. This is no longer speculative. FDA reviewers now work with Elsa, a generative AI assistant deployed agency-wide for reading, summarizing, and protocol review. The data and analytics vendors many of us depend on have moved just as fast: IQVIA launched IQVIA.ai in March, an agentic platform spanning clinical, commercial, and real-world workflows, with 19 of the top 20 pharmaceutical companies already running its agents somewhere in their operations.
It is easy to assume that becoming AI-fluent requires mastering all of this.
I do not think it does.
I think the delta is crossed much more simply: through trial and error. We need to look into our tool kit.
The breakthrough may not come from an AI team
Organizations are investing heavily in AI adoption. That is appropriate. Data scientists, engineers, informaticians, and dedicated AI teams will be essential to building secure, scalable, and reliable systems.
The scale of that investment is not subtle. Lilly has opened up eighteen of its internal drug discovery models to biotech partners through TuneLab, a federated platform built on datasets the company values at more than a billion dollars, so that partners can run the models against their own data without either side handing over anything proprietary. At AstraZeneca, Pascal Soriot has described an agent that combines clinical and laboratory data to estimate the probability that a Phase 3 trial will succeed, aimed squarely at the most expensive failure in the business.
Those are enormous, well-capitalized programs. They are not the only place progress comes from.
But I suspect many of the most useful applications of AI in pharmacoepidemiology will originate somewhere else.
They will come from subject-matter experts.
The epidemiologist who has written dozens of protocols knows which parts are unnecessarily painful.
The researcher who has conducted repeated literature reviews knows where the process becomes inefficient.
The scientist who has spent years working with claims or EHR data understands where data definitions break down.
The safety epidemiologist knows which questions recur again and again during signal evaluation.
These are not primarily AI problems. They are pharmacoepidemiology problems.
And that matters.
A data scientist may know far more about machine learning than an epidemiologist. An engineer may be able to build a sophisticated agentic system. But neither automatically knows which parts of an epidemiologic workflow are worth changing, which shortcuts are dangerous, or which seemingly small methodological choices determine whether a study is credible.
The domain expert has something enormously valuable in this transition: context.
Start with what annoys you
One of the easiest mistakes in AI adoption is to begin with the technology.
A new model is released, and we ask: What can I use this for?
I think a better starting point is much less glamorous:
What part of my work annoys me?
What do I do repeatedly?
What takes longer than it should?
Where do I routinely search through the same documents?
Which tasks require copying information from one place to another?
Where are errors common?
Which parts of a workflow exist mainly because that is how we have always done them?
Those questions are often where useful AI applications begin.
Maybe the answer is extracting study characteristics from a set of papers.
Maybe it is checking whether eligibility criteria are consistent across a protocol.
Maybe it is reviewing code lists.
Maybe it is comparing a draft study design against a methodological framework.
Maybe it is interrogating several hundred pages of regulatory guidance.
Or perhaps it is something no AI strategy document has identified yet.
That is precisely the point.
You know your own pain points better than anyone else. Let them guide the technology.
Trial and error is the curriculum
In your pocket sits access to some of the most advanced general-purpose technology we have ever developed.
That is a remarkable thing.
And yet many scientists still interact with AI only cautiously, waiting for formal training, organizational approval, or a clearly defined use case before exploring what it can do.
There are certainly situations where governance matters. Patient data, confidential information, regulated systems, and production workflows require appropriate safeguards.
But learning how these systems behave does not require beginning there.
Take something you already know well.
Give the model a problem.
See what it does.
Change the instructions.
Give it more context.
Provide an example.
Ask it to explain its assumptions.
Push it until it fails.
Then figure out why.
Try again.
Over time, something important happens.
You develop intuition.
You begin to know when a model is likely to be helpful and when it is likely to struggle. You learn how much context is enough. You start noticing when a polished answer is hiding a weak methodological decision. You recognize which tasks belong with a language model and which still belong with code, structured rules, or human judgment.
That is AI fluency.
And it is difficult to acquire passively.
You can attend webinars. You can read papers. You can follow every major model release. But at some point, you have to use the technology.
Your expertise becomes more valuable, not less
There is sometimes an assumption that as AI improves, domain expertise becomes less important.
For pharmacoepidemiology, I suspect the opposite may be true.
The better these systems become at generating text, code, analyses, and scientific artifacts, the more important it becomes to recognize when those outputs are wrong.
An experienced epidemiologist knows that a beautifully written protocol can still contain a poor comparator.
They know that a plausible analysis plan may mishandle time-varying exposure.
They know that an outcome definition can look reasonable while introducing substantial misclassification.
They know that the answer to a causal question depends on assumptions the model may never mention.
Those instincts are difficult to automate because they were developed through years of experience.
AI gives those experts leverage. It allows someone who deeply understands a problem to experiment with solving it in ways that previously required a larger technical team.
That may be one of the most important opportunities in front of us.
You do not need to build the future of AI
None of this means that every epidemiologist needs to become an AI developer.
You do not need to publish an AI paper in Nature.
You do not need to launch a startup.
You do not need to give the keynote at the next AI conference.
You may simply build something that helps you.
Perhaps it saves you 30 minutes.
Perhaps it catches an inconsistency in a protocol.
Perhaps it helps your team find evidence faster.
Perhaps it turns into a small internal tool that other researchers begin using.
That is enough.
AI adoption does not have to begin with transformation at the level of an entire organization. It can begin with one scientist solving one problem differently.
Crossing the delta
The transition from AI-naive to AI-fluent is not primarily a technical certification.
It is a change in comfort, intuition, and imagination. And the shortest path across that delta may simply be immersion.
Use the tools.
Break them.
Question them.
Find the places where they help.
Find the places where they fail.
Let your experience in pharmacoepidemiology tell you what is worth building.
This is not a hypothetical for me. Every tool in the AI Toolkit on this site started as something that annoyed me. I was tired of drawing causal diagrams that nobody checked for d-separation, so I built DAG Studio. I was tired of scoring the same reporting and bias checklists by hand, so the TARGET Checklist and ROBINS-I tools exist. None of them required an AI team. They required knowing which part of the job was worth fixing.
The next meaningful application of AI in our field may not come from someone who knows more about artificial intelligence than you do. It may come from someone who knows more about the problem.
If you want a rough sense of where you currently sit, there is an AI Fluency quiz on this site. It takes a few minutes and is meant as a starting point.
John D. Diaz-Decaro is the founder and principal of Black Swan Causal Labs, an independent research lab and consultancy working at the intersection of causal inference, real-world evidence, pharmacoepidemiology, and agentic AI.