Skip to Content

Case study · Medical data annotation & labeling

Head CT ground truth for a multi-indication AI model.

 



 

A global clinical AI company needed an independent reference standard for a new head CT model that detects more than a dozen critical findings in a single pass — the same three senior neuroradiologists on every case, 3,465 scans, inside a ten-week window that opened in August.

10,000+

Independent
expert reads

3,465

CT scans each;
read three times

1,000+

Hour of senior
subspecialist time

3

Neuroradiologists
on every case

More than a dozen findings on every scan


Before a multi-indication head CT model goes to the FDA, its performance has to be proved against an independent, expert-established reference standard. For this model that meant reviewing every scan for more than a dozen critical pathologies — stroke, venous sinus thrombosis, skull and facial fractures, herniation, cerebral edema, hydrocephalus, space-occupying lesions, midline shift and orbital findings among them.

The client's requirements were specific. Senior, practising neuroradiologists with at least five years of post-board-certification experience. Thousands of scans released in batches over the study. Roughly 30 hours a week from each reviewer, sustained. A regulatory deadline of about ten weeks, starting in August, when a large share of clinicians are on leave.

The hard part was not clinical. It was finding those readers, holding them for a quarter, and protecting quality and timeline at the same time.





Three readers on every case, by design



The central decision was how to staff the study. Splitting the work across twelve to eighteen readers would have made the weekly load comfortable, but it introduces reader variability — and reader variability weakens a regulatory submission.

Using real availability data from its network, Medcase confirmed it could deliver either model, which let the client choose the lower-risk design: the same three expert readers on every case in the dataset. Medcase also recommended planning against conservative time estimates, so that reviewers would never be trading accuracy for pace.





Recruiting for reliability, in August


Medcase sat between the reviewers and the client for the length of the study, resolving protocol questions as they came up so that no reviewer was left guessing and no question waited on a scheduled call.

 



A pilot before any production read


Sample cases confirmed real reading times and settled questions about pathology definitions before production began. Every reviewer signed the client's regulatory-approved protocol before their first case.





A dual workflow kept in sync


Findings had to be annotated in the client's proprietary labelling platform and documented in a structured tracking sheet, with the two kept exactly aligned. Medcase managed the study against data rather than assumptions: reviewer-level metrics and early milestones kept readers ahead of each new batch, and weekly reporting gave the client visibility throughout. A fatigue-aware daily limit protected the integrity of individual reads. When a barrier appeared — at one point the need for better equipment — it was removed quickly.

Medcase owned every correction from feedback through to client verification, so the delivered dataset was complete and internally consistent.




Study volume and pace



Work package

Count

Head CT scans

3,465

Readers per scan

3

Independent expert reads

10,000+

Critical pathologies assessed per scan

12+

Expert hours delivered

1,000+

Average time per case

~6 minutes

Weekly commitment per reviewer

~30 hours

Batches

6

Study window

~10 weeks


The real-world pace of about six minutes per head CT matched the conservative estimate Medcase recommended at the design stage, which kept the plan credible for reviewers and client alike.


Results: 3,465 scans, 10,000+ expert reads




  • 3,465 head CT scans, each read by all three neuroradiologists — more than 10,000 expert reads.

  • More than 1,000 hours of senior subspecialist time, at about six minutes per case.

  • More than a dozen critical pathologies assessed on every scan.

  • Six batches delivered inside the ten-week window, with one reviewer finishing four days ahead of the deadline.

  • 100% completion: every case read and quality-checked, and the final round of corrections closed on time.

  • A regulatory-grade dataset supporting the client's new head CT product model and its FDA submission.


The same three readers,
3,465 times.



Tell us what you're building — we'll tell you exactly how we can help. 
Start with a scoped pilot.



Get a Quote