Validation of a Diagnostic Model to Differentiate Multiple Myeloma from Bone Metastasis

Abstract

Purpose: A diagnostic model to differentiate multiple myeloma (MM) from bone metastasis (BM) in patients with destructive bone lesions (MM-BM DDx) was developed to promote timely and appropriate referral of patients with MM to hematologists. External validation has never been conducted. This study aims to externally validate the performance of the MM-BM DDx model. Patients and Methods: This multi-center external validation study was conducted using retrospective data of patients over 45 years old diagnosed with MM or BM at six university-affiliated hospitals in Thailand from 2016 to 2022. The MM-BM DDx development dataset, including patients from 2012 to 2015, was utilized during external validation. Diagnostic indicators for MM included in the MM-BM DDx model are serum creatinine, serum globulin, and serum alkaline phosphatase (ALP). MM and BM diagnosis was based on the documented International Classification of Diseases 10th Revision codes. Model performance was evaluated in terms of discrimination, calibration, and accuracy. Results: A total of 3018 patients were included in the validation dataset (586 with MM and 2432 with BM). Clinical characteristics were similar between the validation and development datasets. The MM-BM DDx model’s predictions showed an AUC of 0.89 (95% CI, 0.87, 0.90). The predicted probabilities of MM from the model increased concordantly with the observed proportion of MM within the validation dataset. The estimated sensitivity, specificity, and LR for each odds class in the validation dataset were similar to those of the development dataset. Conclusion: The discriminative ability and calibration of the MM-BM DDx model were found to be preserved during external validation. These findings provide support for the practical use of the MM-BM DDx model to assist clinicians in identifying patients with destructive bone lesions who are likely to have MM and enable them to arrange timely referrals for further evaluation by hematologists.