NeuroVoices: Carrie Hersh, DO, MSc, on Applying 2-Stage Models to Improve Disease-Modifying Therapy Selection in MS

Carrie Hersh, DO, MSc

Since the turn of the century, there has been an influx of new therapeutics approved to treat relapsing forms of multiple sclerosis (MS). Navigating through the various options has been a challenge for some within the field, as each medication has a different safety and efficacy profile. At the 2022 European Committee for Research and Treatment in Multiple Sclerosis (ECTRIMS) Congress, held October 26-28, in Amsterdam, the Netherlands, Carrie Hersh, DO, MSc, presented a proof-of-concept idea for 2-stage models that may help with these issues.

The analyses included patients with MS on disease-modifying therapies (DMTs) ranging from high to low efficacy who were split into a training (70%) and test (30%) set, stratified by treatment group. In the first stage, baseline relapse risk scores were derived by logistic LASSO regression with baseline covariates as inputs and performance assessed using area under the receiver operator curve (AUROC). In the second stage, propensity score weighted models using multinomial logistic regression, with baseline relapse risk score as one of the covariates, was performed.

All told, using a real-world population of 1600 patients with MS, the risk model achieved an AUROC of 0.75. In the average treatment effect model, moderate- and high-efficacy groups had better relapse outcomes compared with low-efficacy, with the high-efficacy group approaching statistical significance (P = .058). Following the conference, Hersh sat down with NeurologyLive® to discuss the research in detail, including how it can be applied in the future. As part of a new iteration of NeuroVoices, Hersh, an associate professor of neurology at Cleveland Clinic’s Lerner College of Medicine, provided perspective on the ways to improve treatment optimization for patients with MS, and why this topic remains a significant conversation within the field today.

NeurologyLive®: Do the barriers to improving MS treatment optimization mainly stem from the technological capabilities or our understanding of these therapies?

Carrie Hersh, DO, MSc: I think it's a combination of the two. First, we need to be able to identify appropriate risk models that actually work using real world data, which is, kind of the building blocks of this particular study. And it's still very much in its infancy, but I think that we're off to a really good start. It’s also because of the layers and layers of complexity that comes with DMT decision making at the bedside. Right now, there is a little bit of mismatch of what we're able to do in clinical practice and what current studies are able to see at the more population level. What if we were able to take some of that information, derive this into a well-built predictive model, and be able to essentially funnel this at the individualized level? And make really good, accurate, DMT decision-making at the bedside at the individual level versus at the population level.

Are there aspects of MS clinical trials that need improvement/change?

I think that this goes a little bit beyond our work because we're really trying to focus this in at the real world data level. Right now, there's some need for restructuring how we look at certain MS phenotypes. For example, how we measure progression, what is the appropriate endpoint? How do we measure those endpoints that will be able to provide us with more additional information, and maybe have a better time of evaluating certain endpoints and outcomes. This is where we lack a little bit in terms of the current clinical trial regimen.

A study a study like this, that's using more of that predictive modeling, what we really want to do is try transition away from RCT (randomized controlled trial) populations, because it’s so restrictive and regimented. We really want to make sure that we're able to apply this to real world patients that we're seeing in clinical practice. I'm hoping that we'll be able to take this to the next steps, where we'll be able to continue this good work and hopefully, maybe be able to create some sort of machine learning algorithm that might help better identify individual treatments for the individual patient.

With several other methodologies being introduced to improve treatment optimization, how do you navigate which ones are appropriate to use?

Well, it's difficult. Again, it's all based on study design and the optimal performance of your model. With this particular study, the first step that we took was to evaluate the area under the operator receiver curve, essentially to make sure that this particular model makes sense. If that value is not up to par, then that means that you're probably not going to be able to fit these curves very well. The ability to predict a particular MS outcome stratified by different disease modifying therapies, you're not really going to be able to make these accurate conclusions. At the foundation is whether or not the study was designed appropriately and whether or not you have an appropriate model to essentially predict what your outcome of interest is.

The idea behind this is that you don't want to rush it. We certainly appreciate the fact that we have not yet quite met statistical significance using relapse as our endpoint. But, we are currently working on ways to try to better refine these curves a little bit better so that we can make more firm conclusions on treatment prediction.

Transcript edited for clarity. Click here for more NeuroVoices.