Risk of Bias in Non-Randomized Studies
An MCP server implementing ROBINS-I V2 for follow-up cohort studies. The agent reads and cites. The algorithm decides.
Illustrative example · synthetic data, not a real publication
In Brief
Each ROBINS-I domain has a published decision flowchart: the signalling-question answers pick the path, and the path gives the judgement. Here the agent only answers, and every answer carries a quote. Walking the flowchart is code, so two assessors with the same answers reach the same verdict.
// how an assessment runs
Answers questions, cites text
Reads the paper, answers the signalling questions the algorithm actually reaches, and supplies the evidence for each one.
Deterministic. Holds the seven domain algorithms as explicit edge graphs and walks them.
The unit of assessment is one numerical result, not one paper: a study reporting three outcomes across two analyses is six assessments.
parse_document builds a source-tagged map and returns a survey of fifteen evidence cues: where each domain’s evidence tends to live, and which absences already look real. The supplement matters: the target-trial specification often lives only there.
set_prespecified_confounders supplies the review’s list of important confounders. Domain 1 asks whether all of them were controlled, and “important” is the reviewer’s judgement, not the paper’s covariate table. So the server refuses to score domain 1 without it rather than silently substituting one for the other. Then specify_result settles C4, which selects which of domain 1’s two question sets even exists.
assess_result returns a scaffold per domain, never one flat rubric; most of the forty signalling questions are unreachable on any given path. On the reference assessment, twenty-four were never reached, and the algorithm’s branching decided that, not the assessor.
Every quote is resolved to a character offset in the ingested text; an unfound quote is rejected, with the nearest actual text returned so it can be corrected. A claim that the paper is silent must name a search cue, and the server runs the search: prose asserting that you looked is refused, because it cannot be audited.
No tool accepts a domain judgement as an input. The six domain algorithms and the overall algorithm are explicit edge graphs, traced arrow by arrow from the published flowcharts, and the verdict is the traversal. A reviewer may override it, with a recorded justification and both values shown.
The report carries a provenance stamp bound to the ingested text and to a fingerprint of the traced algorithms, so a later correction is detectable in work produced under the earlier version. Anything resting on reviewer judgement rather than the paper stays in a ratification queue: the assessment is explicitly not final until a human signs it off.