Outcome validation is the cornerstone of real‑world evidence research. The Brighton Collaboration has greatly aided the validation process by developing standardised case definitions. However, these definitions are challenging to apply to retrospective real-world data sources such as those used in the VAC4EU network (and other multinational real-world evidence networks). These data were not collected with secondary use for research in mind; thus they often lack clinical details needed to use the definitions for validation. Now, VAC4EU has adapted the Brighton Collaboration levels of certainty criteria so that these definitions can more easily be applied in retrospective data sources, developing a scalable validation pipeline and setting a standard for future RWD studies.
In 2023, the VAC4EU network was in the middle of several multi-database post-authorisation safety studies on various COVID-19 vaccines and identified a need for a harmonised approach to validation. Thus, VAC4EU established the Validation Taskforce. Led by Amirreza Deghdan Tarazjani, the team adapted case definitions for real‑world use, creating 78 dummy cases for training, and built 15 REDCap electronic data collection forms with standardised logic. More than 30 abstractors across eight data partners were trained and evaluated through an intensive programme, resulting in a harmonised, reproducible approach that can be deployed across regulatory post‑authorisation safety studies.
This pipeline ensures consistent adjudication across sites, which is essential for multi‑country studies underpinning regulatory decision‑making. The results also give the network deep insight into where variability arises in applying complex clinical definitions, allowing for targeted improvements and higher‑quality safety monitoring. By establishing a replicable model for case validation, VAC4EU is shaping future international standards for real‑world vaccine safety research.
Applying clinical case definitions retrospectively in real-world data can be challenging when key clinical details are inconsistently recorded. As VAC4EU’s case validation coordinator and trainer, I’m pleased to share our new open-access paper, which presents a structured, scalable validation pipeline. This work shows how a multinational network can harmonise case validation across heterogeneous data sources.
— Amirreza Deghdan Tarazjani
VAC4EU would like to express our congratulations and gratitude to all authors for their excellent work on this paper, including the members and DEAPs who contributed adjudicators and specialised medical expertise essential to this work. We also thank the members of the Validation Taskforce and VAC4EU members who played a crucial role in the creation, review, and refinement of the validation materials and processes. Finally, we acknowledge the support of the Vall d’Hebron REDCap Management Center in enabling and coordinating this activity.