Big data

Massive amounts of data are generated on a daily basis that could potentially be harnessed to support medicines regulation. The European Medicines Agency (EMA) and Heads of Medicines Agencies (HMA) set up a joint task force to describe the big data landscape from a regulatory perspective and identify practical steps for the European medicines regulatory network to make best use of big data in support of innovation and public health in the European Union (EU).

'Big data' is a widely-used term without a commonly-accepted definition. The HMA-EMA Big Data Task Force defined big data as ‘extremely large datasets which may be complex, multi-dimensional, unstructured and heterogeneous, which are accumulating rapidly and which may be analysed computationally to reveal patterns, trends, and associations. In general, big data sets require advanced or specialised methods to provide an answer within reliable constraints’.

A single dataset may not strictly meet the definition of big data but, when pooled or linked with other datasets, they become sufficiently large or complex to analyse to assume the characteristics of big data. Sources include real-world data (such as electronic health records, insurance claims data and data from patient registries), genomics, clinical trials, spontaneous adverse drug reaction reports, social media and wearable devices.

Medicines regulators will increasingly use insights derived from big data to assess the benefit-risk of medicines across their lifecycle.

Also on this topic

• Data Analysis and Real World Interrogation Network (DARWIN EU)

HMA-EMA Big Data Steering Group

The joint HMA-EMA Big Data Steering Group advises the EMA Management Board and HMA on prioritisation and planning of actions to implement the ten recommendations in the Big Data Task Force final report.

Priority recommendations of the HMA-EMA joint Big Data Task Force

HMA-EMA Joint Big Data Taskforce Phase II report: Evolving data-driven regulation

Reference Number: EMA/520966/2019

The Steering Group began its work in May 2020. It is co-chaired by Jeppe Larsen, Director of Medical Devices Division at the  Danish Medicines Agency and Peter Arlett, Head of Data Analytics and Methods at EMA.

The Steering Group reviews the workplan annually to cover any new emerging topics. It last updated the workplan in July 2023.

The workplan aims to increase the utility of big data in regulation, from data quality through study methods to assessment and decision-making. It is patient-focused and guided by advances in science and technology. Implementation of the workplan will be flexible and certain actions may be re-scheduled.

 

 

For more information, see:

Big Data Steering Group (BDSG): 2023 report

Reference Number: EMA/569310/2023

Workplan 2023-2025: HMA-EMA joint Big Data Steering Group

Change management activities delivery plan

Clusters of Excellence Discussion Paper

Workplan 2022-2025: HMA-EMA joint Big Data Steering Group

Big Data Steering Group (BDSG): 2022 report

Reference Number: EMA/826814/2022

Big Data Steering Group (BDSG): 2021 report

Reference Number: EMA/40400/2022

Workplan 2021-2023: HMA-EMA joint Big Data Steering Group

Workplan 2020: HMA-EMA joint Big Data Steering Group

Big Data Steering Group (BDSG): 2020 report

Reference Number: EMA/48625/2021

Mandate: HMA-EMA joint Big Data Steering Group

Reference Number: EMA/96120/2023

Membership list: HMA-EMA joint Big Data Steering Group

Big Data Steering Group meeting minutes

Final Minutes – HMA-EMA joint Big Data Steering Group teleconference - 22 April 2024

Reference Number: EMA/51677/2024

Final Minutes – HMA-EMA joint Big Data Steering Group teleconference - 22 March 2024

Reference Number: EMA/51677/2024

Final Minutes – HMA-EMA joint Big Data Steering Group teleconference - 26 February 2024

Reference Number: EMA/51677/2024

Final Minutes – HMA-EMA joint Big Data Steering Group teleconference - 30 January 2024

Reference Number: EMA/51677/2024

Final Minutes – HMA-EMA joint Big Data Steering Group teleconference - 5 December 2023

Reference Number: EMA/130876/2023

Final Minutes – HMA-EMA joint Big Data Steering Group teleconference - 30 October 2023

Reference Number: EMA/130876/2023

Final Minutes – HMA-EMA joint Big Data Steering Group teleconference - 29 September 2023

Reference Number: EMA/130876/2023

Big data training curriculum

The Big data training curriculum is a collection of training modules aimed to help European medicines regulatory network staff develop expertise in integrating big data analysis into decision-making processes related to medicine regulation. 

The first two modules, on real-world evidence and real-world data, are available for eligible users via the EU Network Training Centre platform. The Big data training curriculum also covers biostatistics, data science and other training topics which will be added over time.

EMA's Methodology Working Party oversees its development.

The Big data curriculum is part of the Big data workplan for 2023-2025 and the European medicines agencies network strategy to 2025. For more information, see: 

Workplan 2023-2025: HMA-EMA joint Big Data Steering Group

HMA-EMA catalogues of real-world data sources and studies

Two online catalogues are available from HMA and EMA, one for real-world data sources and one for real-world data studies:

• HMA-EMA catalogues of real-world data sources and studies

Real-world data are observational data stored for instance in electronic health records and disease registries.

Making greater use of these can improve the evidence base for benefit-risk decisions and bringing medicines to patients.

The catalogues serve to:

• help regulators, researchers and pharmaceutical companies identify the most suitable data sources to address specific research questions;
• support the assessment of study protocols and results;
• promote transparency;
• encourage the use of good practices;
• build trust in research based on real-world data. 

They enhance and replace two databases previously maintained by EMA:

Catalogue	Discontinued database
Real-word data sources	European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP) catalogue
Real-world data studies	European Union electronic register of post-authorisation studies (EU PAS Register)

The catalogues use an agreed list of metadata to describe and connect data sources to studies, using ‘FAIR’ (Findable, Accessible, Interoperable and Reusable) data principles. 

The HMA-EMA Big Data Steering Group reviews the list of metadata annually, based on feedback from catalogue users. 

A draft good practice guide is also available to help users of the catalogues.

By end of June 2024, EMA intends to publish a revised version addressing feedback from a public consultation. Separate sections will cover using the catalogues to identify and assess the suitability of data sources, and definitions and descriptions of the metadata list.

All European data holders, marketing authorisation holders, networks, researchers and institutions interested in having their data used for medicines regulation, or mandated by policy on non-interventional post-authorisation safety studies (PASS), are encouraged to use these catalogues.

List of metadata for Real World Data catalogues

Reference Number: EMA/563896/2022

Good practice guide for the use of the Metadata Catalogue of Real-World Data Sources

Consultation dates: 27/09/2022 to 16/11/2022Draft: consultation closedReference Number: EMA/787647/2022

Improving data discoverability is a priority of the HMA-EMA joint Big Data Task Force final report (phase two), reflected in the European medicines agencies network strategy to 2025 and implemented through the joint HMA-EMA Big Data Steering Group workplan. 

Data discoverability refers to the identification and use of real-word data sources and studies.

For more information: 

• HMA-EMA Big Data Steering Group
• European Network of Centres for Pharmacoepidemiology and Pharmacovigilance

Data quality framework for EU medicine regulation

The final version of the data quality framework for EU medicines regulation is available below, following a public consultation.

This guideline sets out the criteria for a more consistent and standardised approach to the quality of data used in medicine regulation to support benefit-risk decisions.

Data Quality Framework for EU medicines regulation

Consultation dates: 10/10/2022 to 18/11/2022

It is meant to:

• help identify, define and further develop data quality assessment procedures and recommendations for current and novel data types;
• support pharmaceutical companies and other stakeholders in selecting data sources for their studies;
• ensure the trust of patients and healthcare professionals in data-driven regulatory decision-making.

The data quality framework was co-produced by EMA, the Heads of Medicines Agencies (HMA) and the Joint Action Towards the European Health Data Space (TEHDAS).

Establishing this framework is a key element in the HMA/EMA joint Big Data Steering Group workplan. EMA will work with stakeholders to use the framework’s concepts and develop practical guidelines for assessing the quality level of data. These will initially focus on the domains of real-world data and adverse drug reactions.

Data standardisation strategy

The European medicines regulatory network's data standardisation strategy sets out principles to guide the definition, adoption and implementation of international data standards by the network.

It aims to:

• enable quicker uptake of international data standards across the EU;
• improve data quality;
• enable data linkage and data analysis to support medicine regulation.

The strategy is a key deliverable of the Big Data Steering Group workplan.

EMA and HMA published the strategy in December 2021 and will maintain it over time to reflect any changing priorities or new requirements.

European Medicines Regulatory Network Data Standardisation Strategy

Darwin EU

EMA is establishing a coordination centre to provide timely and reliable evidence on the use, safety and effectiveness of medicines for human use, including vaccines, from real world healthcare databases across the EU. 

This capability is called the Data Analysis and Real World Interrogation Network (DARWIN EU®). For more information, see:

• Data Analysis and Real World Interrogation Network (DARWIN EU)

Pilot on using raw data in medicine evaluation

Through a proof-of-concept pilot, selected applicants can submit 'raw data' to EMA as part of their initial and post-authorisation marketing authorisation applications. 

Information about the raw data proof-of-concept pilot for industry

Reference Number: EMA/174598/2022

Application of EMA’s transparency principles to the raw data proof-of-concept pilot

Reference Number: EMA/949891/2022

Raw data refers to individual patient data from clinical trials. These include:

• clinical laboratory results;
• imaging data;
• patient medical charts.

Currently, applicants are submitting data in an aggregated format as clinical summaries or as individual patient data in PDF listings. This can hinder data analysis and slow down the evaluation process.

In contrast, raw data are stored in electronic structured format. This enables regulators to more easily visualise and analyse the data if needed.

The pilot aims to assess whether using raw data can help speed up and improve the medicine-evaluation process. The goal of this is to allow patients faster and better informed access to innovative medicines.

EMA launched the pilot in July 2022.

The pilot currently accepts expressions of interest for procedures with a submission date up to Q1 2024.

It will run for up to two years and include approximately ten regulatory procedures submitted to EMA from September 2022.

For any queries and to apply to take part in the pilot, write to [e-mail]. 

The pilot is a key activity under the priority recommendations of the HMA-EMA Big Data Task Force. It refers to the priority of building network capability to analyse data.

For more information, see: 

Presentation - 5.2 Pilot on raw data analysis - E Psarelli (EMA) and A Bradshaw (AE)

Support for pharmaceutical companies

Further information is available to support pharmaceutical companies with their participation in EMA's raw data pilot. 

The documents include:

• a questions and answers document on the raw data pilot;
• a participation letter to confirm pilot participation for a specific regulatory procedure;
• a cover letter for pilot participants to attach to their data packages.

Questions and Answers about the raw data proof-of-concept pilot for industry

Reference Number: EMA/658116/2022

Pilot participation letter

Reference Number: EMA/659352/2022

Raw data submission cover letter template

Reference Number: EMA/658203/2022

Data protection

For information on data protection in the raw data proof-of-concept pilot, see:

European Medicines Agency’s data protection notice for the Raw data proof-of-concept pilot

Reference Number: EMA/716436/2022

Records of data processing activity for the raw data proof-of-concept pilot

Reference Number: EMA/716437/2022

Big Data Highlights

Big Data Highlights provides a quarterly update on the implementation of the HMA-EMA Big Data Steering Group workplan.

EMA publishes Big Data Highlights every three months. 

Issues from June 2023 (Issue 6) onwards are available at the link below: 

• Big Data Highlights: Issue 6 onwards

Previous issues from 2022 are available on EMA's website in PDF format: 

• Big Data Highlights: Issues 1 - 5

Use the link below to receive Big Data Highlights by email: 

• Subscribe to Big Data Highlights

Work of the former HMA-EMA Big Data Task Force

The HMA-EMA Big Data Task Force operated from 2017 until December 2019 to report on the challenges and opportunities posed by big data in medicines regulation. It carried out its work in two phases. 

In phase one, the task force:

• reviewed the landscape of big data from a regulatory perspective and identified opportunities for improvements in the operation of medicines regulation;
• performed online surveys of national regulatory agencies and the pharmaceutical industry on perspectives, expertise and challenges. This helped develop an understanding of the challenges and the current state of expertise in the regulatory network.

In phase two, the task force made practical recommendations to inform strategic decision-making and planning by the HMA and EMA and to contribute to the European medicines regulatory network's work on developing a five-year EU Network Strategy to 2025.

Final report (phase two)

The HMA/EMA Task Force on Big Data published its final report in January 2020. It contained practical recommendations on how the European medicines regulatory network could make best use of big data by evolving its approach to data use and evidence generation in support of innovation and public health  .

It identified ten priority recommendations and practical steps to implement them:

HMA-EMA Joint Big Data Taskforce Phase II report: Evolving data-driven regulation

Reference Number: EMA/520966/2019

Priority recommendations of the HMA-EMA joint Big Data Task Force

Interim report (phase one)

The task force published an interim report in February 2019. It provided a comprehensive summary of various data sources and set out recommendations for understanding the acceptability of evidence derived from big data in support of the evaluation and supervision of medicines by regulators:

HMA-EMA Joint Task Force on big data: Summary report

Reference Number: EMA/105321/2019

 

This took into account the analyses of the task force's subgroups:

Clinical Trial and Imaging - Subgroup report

Observational data (Real World Data) - Subgroup report

Genomics, Genetics, Transcriptomics and Epigenetics - Subgroup report

Spontaneous Adverse Drug Reactions - Subgroup report

Data Analytics - Subgroup report

Bioanalytical Omics - Subgroup report

Social Media and M-Health Data - Subgroup report

HMA/EMA Joint Task Force on big data – Surveys results

The task force was composed of experienced medicines regulators and data experts appointed by the national competent authorities, EMA and the European Commission (EC). For more information, see HMA-EMA Big Data Task Force. 

Meetings and workshops

For information on related meetings and workshops, see:

• Biannual Big Data Steering Group and industry stakeholders meeting 2024 (12/06/2024)
• Multi-stakeholder workshop on Data Quality Framework for Adverse Drug Reaction reporting (01/03/2024)
• Second biannual Big Data Steering Group and industry stakeholders meeting 2023 (27/11/2023)
• Joint HMA/EMA AI workshop – Smart regulation in a rapidly evolving world (20-21/11/2023)
• HMA/EMA Big Data Stakeholder Forum 2023 (04/12/2023)
• Biannual Big Data Steering Group and industry stakeholders meeting 2023 (26/05/2023)
• Multi-stakeholder workshop on Real World Data (RWD) quality and Real World Evidence (RWE) use (26-27/06/2023)
• Second bi-annual Big Data Steering Group and industry stakeholders meeting 2022 (03/11/2022)
• EU Big Data Stakeholder Forum (01/12/2022)
• Big Data Steering Group and industry stakeholders meeting 2022 (30/05/2022)
• Data quality framework for medicines regulation (07/04/2022)
• EU Big Data Stakeholder Forum (07/12/2021)
• Learnings initiative webinar for optimal use of big data for regulatory purpose (30/11/2021)
• Veterinary Big Data stakeholder forum (01-02/06/2021)
• Data Standardisation Strategy stakeholder workshop (18/05/2021)
• Joint HMA/EMA workshop on artificial intelligence in medicines regulation (19-20/04/2021)
• Technical workshop on real-world metadata for regulatory purposes (12/04/2021)
• EU big data stakeholder virtual forum (15/12/2020)
• Heads of Medicines Agencies (HMA) / European Medicines Agency (EMA) Joint Big Data Task Force meeting: identifying solutions for big data challenges (04/05/2018)
• Workshop on identifying opportunities for 'big data' in medicines development and regulatory science (14-15/11/2016)

Veterinary big data

EMA and HMA established the veterinary big data initiative to explore the use of new digital technologies in key veterinary regulatory activities.

It takes account of the increasing amount of data generated via new digital systems put in place to implement the Veterinary Medicinal Products Regulation.

A European veterinary big data strategy sets out how the European medicines regulatory network intends to implement this initiative:

European veterinary big data strategy 2022-2027

Reference Number: EMA/648865/2021

For more information:

• Veterinary big data

Data protection

EMA is preparing dedicated guidance on the impact of EU data protection legislation on the secondary use of health data in support of the development, evaluation and supervision of medicines.

The aim is to help medicine developers, data providers and research bodies comply with EU data protection rules, and to help patients and consumers understand their rights and the existing safeguards to protect personal data.

Secondary use of data refers to the use of data for a different purpose than the one for which it was originally collected. It typically involves the use of electronic health records, health insurance claims data, registry data or drug consumption data for medicines research and public health purposes.

The guidance will cover various operational scenarios, including the development of medicines, the evaluation of marketing authorisation applications and post-authorisation safety monitoring.

By July 2020 EMA had gathered input from patients and consumers as data contributors as well as from medicines developers, research-performing and research-supporting infrastructures and other data providers (e.g. payers of healthcare).

In September 2020, stakeholders discussed with EMA the key questions concerning the application of the General Data Protection Regulation (GDPR) in the health sector and the secondary use of health data for medicines and public health purposes:

• Workshop on the application of the General Data Protection Regulation (GDPR) in the area of health and Secondary Use of Data for Medicines and Public Health Purposes (23/09/2020)
• Workshop on the General Data Protection Regulation (GDPR) and secondary use of data for medicines and public health purposes (29/09/2020)

EMA aims to finalise the guidance in consultation with the European Commission and the European Data Protection Supervisor (EDPS) in the last quarter of 2021. It will take into account stakeholder input and guidance from the EDPS on the processing of health data for research.

Ensuring that personal data are managed and analysed within a secure and ethical governance framework in compliance with EU data protection legislation is one of the recommended priorities of the HMA/EMA Big Data Task Force.

EU data protection legislation includes:

• Regulation (EU) 2016/679, known as the General Data Protection Regulation (GDPR), which applies to private and public entities in the Member States;
• Regulation (EU) 2018/1725, known as the EU Data Protection Regulation (EUDPR), which applies to all EU institutions and bodies.

Artificial Intelligence (AI)

An EMA draft reflection paper on the use of artificial intelligence (AI) in the medicinal product lifecycle is available to support the regulation of human and veterinary medicines:

• The use of Artificial Intelligence (AI) in the medicinal product lifecycle

This is in a context where pharmaceutical companies increasingly use AI-powered tools in the research, development and monitoring of medicines.

The reflection paper outlines the scientific principles relevant for regulatory evaluation of medicines developed or monitored with the help of AI.

An AI workplan for the European medicines regulatory network is also available, to guide the use of AI in medicines regulation in Europe to 2028.

Multi-annual artificial intelligence workplan 2023-2028: HMA-EMA joint Big Data Steering Group

It identifies actions for regulators in four key areas:

• Guidance, policy and product support - delivering guidance on the use of AI throughout the medicine lifecycle
• AI tools and technology - providing frameworks for the use of AI tools
• Collaboration and training - developing capacity for the use of AI technology
• Experimentation - ensuring a structured and coordinated approach

The Big Data Steering Group developed the plan endorsed by HMA in November 2023, and subsequently by EMA's Management Board in December 2023, and maintains it over time.

AI-enabled knowledge mining tool

In March 2024, EMA introduced an AI-enabled knowledge mining tool called Scientific Explorer for EU regulators.

The tool enables easy, focused and precise search of regulatory scientific information from network's sources to support decision-making and simplify processes.

The first release focuses on scientific advice procedures for human medicines.

For more information, see the document below with answers to frequently asked questions about the tool:

Scientific Explorer - Frequently Asked Questions

Reference Number: EMA/126785/2024

Use of real-world evidence

A reflection paper on non-interventional studies that use real-world data in order to generate real-world evidence for regulatory purposes is available for public consultation.

It is aimed at all stakeholders involved in the planning, conduct and analysis of this type of non-interventional studies, including marketing authorisation holders and applicants.

A non-interventional study is a clinical study that does not meet any of the conditions defining a clinical trial in Article 2.2(2) of Regulation (EU) No 536/2014 on clinical trials on medicinal products for human use.

The public consultation runs from 3 May to 31 August 2024.

You can contribute by completing the survey below:

• EU survey: Submission of comments on 'Reflection paper on use of real-world data in non-interventional studies to generate real-world evidence'

For more information on this draft reflection paper, see: 

• Reflection paper on use of real-world data in non-interventional studies to generate real-world evidence - Scientific guideline

Guidance on real-world evidence

Guidance is available on how EMA can help generate real-word evidence by analysing real-word data, with the aim of supporting regulatory decision-making.

It is meant for EU regulators and decision-makers, including EMA's scientific committees, working parties and groups, as well as national competent authorities, healthcare technology assessment bodies and payers.

It covers:

• how the mentioned stakeholders can request real-world data studies from EMA;
• what types of studies can be performed;
• how EMA can help identify resources to address research questions.

EMA cannot consider requests from other bodies and institutions, including academia, pharmaceutical companies and contract research organisations.

For queries, you can use our webform:

• Send a question to the European Medicines Agency

Guidance on real-world evidence provided by EMA: support for regulatory decision-making

Reference Number: EMA/152628/2024

The guidance builds on EMA's experience in using real-world evidence to support regulatory decision-making, described in the report below.

Real-world evidence report

The report is based on studies EMA conducted between September 2021 and February 2023, which focused on medicine safety, medicine use, disease epidemiology, design and feasibility of clinical trials and clinical management. 

It is overseen by the EMA-HMA Big Data Steering Group. 

It is also part of the European medicines regulatory network's efforts to enable the use of real-world evidence and establish its value across regulatory areas by 2025.

A list of research topics and a portfolio of use cases are also available.

Read EMA report on real-world evidence

Findings include:

• Real-world evidence can support decision-making in various regulatory contexts
• Regulators need access to additional data sources such as secondary care databases, claims databases and registries from across Europe
• Research needs should be identified as early as possible so that studies can be conducted in time for regulatory decisions
• More information on data source characteristics is needed to facilitate the interpretation of real-world evidence derived from these sources
• There is a need to build more capability and capacity for real-world evidence generation

This report informs the development of the Data Analysis and Real World Interrogation Network (DARWIN EU).

For more information, see below the report document (also accessible by clicking on the thumbnail image in this section) and the list of research topics and use cases:

Real-world evidence framework to support EU regulatory decision-making: Report on the experience gained with regulator-led studies from September 2021 to February 2023

Reference Number: EMA/289699/2023

Annex 1: List of EMA RWD study requests

Reference Number: EMA/286356/2023

Related information materials

Infosheet - Review: real-world data studies

Experience gained in conducting studies and providing real-world evidence (RWE) to support EMA regulatory decision making.

International collaboration on real-world evidence

Within the International Coalition of Medicines Regulatory Authorities (ICMRA), EMA works to help integrate real-world evidence into regulatory decision-making across the world. 

ICMRA held a workshop in June 2022 for regulators to share experience in obtaining and using real-world evidence for the assessment of medicines, and issued a pledge in July 2022 to foster global efforts in this area. 

For more information:

• International Coalition of Medicines Regulatory Authorities (ICMRA): Cooperation on other topics

Reflection paper

A reflection paper was available for public consultation aiming to harmonise real-world evidence terminology and enable the convergence of general principles for planning and reporting studies using real-world data to support regulatory decision-making. 

The document builds on the 2022 statement from the International Coalition of Medicines Regulatory Authorities (ICMRA), offering a strategic approach for future ICH guidelines on the assessment of real-world data and real-world evidence.  

EMA encouraged medicine regulators, academic researchers, the pharmaceutical industry and other stakeholders to comment. 

The public consultation was open until 30 September 2023.

The reflection paper was released by the Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH), and co-sponsored by EMA, the United States of America's (USA) Food and Drug Administration (FDA), and Health Canada.

More information:

• Global regulators call for international collaboration to integrate real-world evidence into regulatory decision-making (22/07/2022)

ICH reflection paper on proposed international harmonisation of real-world evidence terminology and convergence of general principles regarding planning and reporting of studies using real-world data, with a focus on effectiveness of medicines

Consultation dates: 30/06/2023 to 30/09/2023Draft: consultation closedReference Number: EMA/CHMP/ICH/295401/2023Summary:

The objectives of this reflection paper are to engage ICH on convergence of terminology for real-world data and real-world evidence, on the format for protocols and reports of study results submitted to regulatory agencies throughout the lifecycle of medicinal products, and on promoting registration of study protocols and reports.

This paper builds on the 2022 ICMRA statement on international collaboration to enable RWE for regulatory decision-making, and identifies areas for harmonisation in future ICH guidelines to ultimately inform the assessment of real-world data and real-world evidence for regulatory purposes.  

For submission of comments, please use the standard ICH public consultation form