In a collaborative project with the VA, a team at the US Department of Energy's (DOE) Oak Ridge National Laboratory (ORNL) has taken the model and engineered the expanded version of it to run 300 times faster, gaining an unprecedented speed-up that might have a profound effect on the VAs ability to reach susceptible veterans quickly.
"Veterans should be following the medication regimens prescribed by their doctors", stated Edmon Begoli, principal investigator on the project and director of the Scalable Protected Data Facilities (SPDF) at the National Center for Computational Sciences at ORNL. "If they are attempting to increase certain medications, they could be at risk of substance dependency, and if they stop taking medications like antidepressants or opioids, it could put them at risk of withdrawal or other adverse effects that could, in the worst case, lead to suicide."
Until now, the medication possession ratio calculations have been limited in scope. The model has typically included only active psychotropic medications, such as narcotics or mental health medications, and covered a narrow class of the total veteran population in the Veterans Health Administration (VHA) database. This simplified version of the algorithm took 4 hours to run on the VA's resources.
The ORNL team sped up the algorithm and also increased its coverage to all current medications, as well as recent past prescriptions and all 9 million veterans in the database. Without the speedup, the expanded version of the model would have taken 75 hours to run. With the speed-up, it runs in only 15 minutes, a 300-fold improvement.
"Now we can observe and reach a much larger population that's potentially at risk - and look at even more risk factors", Edmon Begoli stated. "The potential to provide far greater predictive services is there."
In 2007, the Joshua Omvig Veterans Suicide Prevention Act was passed, requiring the VA to establish a comprehensive suicide prevention program. As part of this programme, the VA implemented a Veterans Crisis Line that connects veterans who call in to a hotline with a qualified crisis responder.
To provide brief and targeted overviews of veterans' medication histories, the VHA worked with the Veterans Crisis Line to establish the medication possession ratio algorithm for use in the crisis centre.
"When you're on the phone with a distressed veteran, it's very difficult to do a detailed chart review with a medical record", stated Jodie Trafton, the director of the VA Programme Evaluation and Resource Center in the VHA Office of Mental Health and Suicide Prevention. "We worked with the crisis line staff to build a report that would summarize all the key elements they would look for in a medical record, so they could enter a patient's information, and it would provide them the relevant items in a nice, simplified format."
Soon, the VHA began running the medication possession ratio algorithm once a day to guide clinicians working with potentially high-risk patients in providing treatments tailored to their personal risk factors. The crisis line responders at the VHA could access the results via clinical dashboards, identify the veterans who needed special assistance, and quickly determine whether they were on the phone with a veteran at risk.
"Veterans who are calling the crisis line are often very desperate", Edmon Begoli stated. "Veterans may be calling because they're running out of a diabetes medication, experiencing a life crisis, or having mental health issues. The opportunity here is to find out, during calls, whether veterans are eligible for benefits they might not be aware of, so we can, in some cases, recover a person from an almost certain path to suicide."
Initially, running the model took about 4 hours, and the VHA was limited in the information it could include. One option was to run the algorithm for only 1 of the 18 geographic regions in the system, significantly reducing the number of veterans in the model. The other option was to limit the number of medications analyzed.
"The VHA was faced with the dilemma of not having the budget, the staff, or the expertise to really dig into and properly analyze the massive amount of health data in its system", stated Jeremy Cohen, ORNL SPDF programme manager and program manager for the VA project.
ORNL already had a small accredited environment that could house protected health information, so the VA leveraged the opportunity to partner with the lab.
The ORNL team scaled the medication possession ratio's existing algorithms and improved its performance by applying advanced computing techniques for text mining and predictive algorithms. Joshua Arnold, a data scientist working on the project with Edmon Begoli and Jeremy Cohen, played a pivotal role in the project.
Joshua Arnold used a high-performance computing (HPC) platform at ORNL to understand the current algorithm and create a faster version of it that would run in 15 minutes - 300 times faster than the 75 hours it would have taken the VA to run the model. Jodie Trafton believes the speed-up in the model will have a wide impact.
"This work is crucial because the better our models can be, the better we'll get at calling the right people at the right time. And the better we are at calling the right people at the right time, the more lives we save", Jodie Trafton stated.
Since the VA came to ORNL, the collaborative team has taken an exceedingly cautious approach to handling veteran data. This careful treatment of the data, along with the team's success with the medication possession ratio algorithm, has spawned other research activities centered around the model.
Edmon Begoli and Jeremy Cohen's team is currently working with Jodie Trafton to improve operational aspects of a VA program called REACH VET, or Recovery Engagement and Coordination for HealthVeterans Enhanced Treatment. In this programme, health care providers at 141 VA facilities reach out to veterans who may be at risk of suicide. One of the VA's goals is to implement the revised medication possession ratio algorithm into REACH VET, a task that will require a secure environment such as the one at ORNL.
"The medication possession ratio is one of our first examples of something that we know is a predictor of a variety of important outcomes", Jodie Trafton stated. "If we can use it in our models like REACH VET, we will be able to better identify patients who might try to commit suicide."
Additionally, the VA hopes to use ORNL's algorithm for analysis of the VA's 3,5 billion clinical notes. Diagnostic codes and dates have been straightforward to input because they are structured, Edmon Begoli said, but clinical notes have proved more difficult.
"There's no diagnostic code that says, 'increased suicide risk' or 'job loss'. If a patient reports a death in the family, a transient living situation, or social disconnectedness, we need to be able to identify those factors", Edmon Begoli stated. "There's a massive repository of these risk factors that hasn't been used for predictive risk models."
ORNL's current accredited environment provides the resources needed for such a task.
"All of this work is detached from the internet and requires a substantial amount of training and background checks before anyone can even look in and touch the data", Jeremy Cohen stated. "The security of the data and the protection of the privacy of the veterans are of the utmost importance."
It's one reason why the VA came to ORNL.
Today, ORNL runs the largest mega-biobank in the world for the VA. And although it currently takes months for systems such as the VA's HPC system at ORNL to mine the VHA's entire clinical repository, the IBM AC922 Summit supercomputer at the Oak Ridge Leadership Computing Facility (OLCF) could cut these processing times to days or cut the medication possession ratio algorithm's runtime down to seconds. The OLCF is a DOE Office of Science User Facility located at ORNL.
In a separate but related project, Edmon Begoli and Jeremy Cohen are specifically working toward gaining accreditation to run analyses of protected health information on Summit to expand the predictive modelling capabilities at ORNL. US government agencies are required to be compliant with the Federal Information Security Modernization Act, or FISMA, a law that implements and supports the IT security standards defined by the National Institute of Standards and Technology's cybersecurity framework.
FISMA intrinsically requires a certain amount of security training, testing, network defenses, and data protections for every system that hosts protected health information in the United States. However, in addition to these stringent requirements, a system must also be fully accredited by a third party with an Agency Authority to Operate. Although no supercomputer of Summit's size has ever been accredited, the team is hopeful it will be able to call Summit the first, making this the most powerful protected data operation in the world.
"Our ultimate goal would be to enable our operation to be a real-time alert system so that while physicians are interacting with a veteran, they can get live predictive risk alerts and intervene", Edmon Begoli stated. "We don't want veterans to walk into a clinic and get missed because someone hasn't been specifically trained to recognize these symptoms. We never want it to be too late to reach someone."
Contributors to this research include Joshua Arnold, Edmon Begoli, Michael Bowie, Kris Brown, Jeremy Cohen, Franciel Linares, Ben Mayer, and Jodie Trafton.
B. Mayer, J. Arnold, E. Begoli, E. Rush, M. Drewry, K. Brown, E. Ponce, and S. Srinivas are the authors of the paper titled "Evaluating Text Analytic Frameworks for Mental Health Surveillance". It is published in 2018 IEEE 34th International Conference on Data Engineering Workshops (2018) - doi:10.1109/ICDEW.2018.00014.