CAN KNOWLEDGEABLE EXPERTS ASSESS COSTS and OUTCOMES AS if THEY WERE IGNORANT? AN EXPERIMENT WITHIN PRECISION MEDICINE EVALUATION

Abstract

Background: The purpose of this study is to evaluate the validity of the standard approach in expert judgement for evaluating precision medicines, in which experts are required to estimate outcomes as if they did not have access to diagnostic information, whereas in fact they do. Methods: Fourteen clinicians participated in an expert judgement task to estimate cost and medical outcomes from use of exome sequencing in pediatric patients with intractable epilepsy in Thailand. Experts were randomly assigned to either an "unblind" or "blind" group; the former were provided with the exome sequencing results for each patient case prior to the judgement task, whereas the latter were not provided with the exome sequencing results. Both groups were asked to estimate outcomes for the counterfactual scenario, in which patients had not been tested by exome sequencing. Results: Our study did not show significant results, possibly due to the small sample size of both participants and case studies. Conclusions: Comparison of the unblind and blind approach did not show conclusive evidence that there is a difference in outcomes. However, until further evidence suggests otherwise, we recommend the blind approach as preferable for when using expert judgement to evaluate precision medicines, because this approach is more representative of the counterfactual scenario than the unblind approach.