Observational (OBS), Pragmatic (PRA), and Indirect (IND) Methodologies for Comparative / Relative Effectiveness (RE) and 
Benefit-Risk (BR) Analyses
Dr Florian Eichmann, Kathrin Kreyenberg – Late Stage Business Unit, inVentiv Health Clinical, Munich, Germany
Presented at ISPOR Milan, November 2015
Contact: florian.eichmann@inventivhealth.com
OBJECTIVES: Regulators and Health Technology Assessment (HTA)/Payer stakeholders are increasingly looking into Real Word Evidence (RWE) for RE and BR. Demonstrating quality, efficacy and safety plus cost effectiveness (‘4thhurdle’) is no longer sufficient to ensure market authorization and reimbursement. While some RWE stakeholders require randomized study approaches, e.g. to avoid allocation bias, typically in the form of ‘pragmatic’ trials, others consider observational studies or accept indirect comparisons in the absence of head-to-head comparison results. However, it is unknown how frequently respective comparative approaches are actually used.
METHODS: To assess relative importance of observational, pragmatic and indirect methodologies for comparative RE and BR, we surveyed Medline using purpose related terms ‘relative/comparative effectiveness’ or ‘benefit/risk’ in combination with methodology terms ‘observational/non-interventional’, ‘pragmatic study/trial’, and ‘indirect comparison/network analysis’. Resulting publication numbers were analyzed, and selected abstracts were assessed qualitatively.
RESULTS: We focused on publication title analysis as all field searches yielded heterogenous results. Publication title hits were by far highest for OBS methodologies, followed by PRA and lowest for IND (13,247; 201; 141), and higher for BR than for CE (3,017; 1,152). RE publications continuously increased over time with not more than 50 annual hits before 2005, but consistently above 250 hits after 2013. Similarly, annual CE hits were below 20 before 2006, increased to 250 as of 2010, but tended to decrease after a peak in 2013. For RE, OBS methods were used most frequently (61), other combinations (RE or CE with IND or PRA, and BR with OBS, were rare (0 – 2).
CONCLUSIONS: Although counts for pragmatic and indirect methodologies are growing, publication numbers are low compared to observational methodologies, and their explicit title relevance for RE or BR – which are both increasingly covered- still seem relatively minor. Qualitative assessments based on individual abstracts will be presented.
Please do contact Florian (florian.eichmann@inventvhealth.com) if you want to know more or discuss further…