Health Data

Artificial Intelligence: What We Learned While Implementing CDI, CAC, and Scanning Software Solutions

Everywhere you look in healthcare, there are software solutions aiming to increase productivity, improve safety or efficiency, or replace the number of “touches” required for processing. Whether you realized it or not, these solutions represent use of artificial intelligence (AI). Although this is not a foreign concept for most of us, some may still feel uneasy about AI.

In an era that bombards us with ads and recommendations for “whole,” “real,” “authentic,” and “natural,” the word “artificial” can give us pause. “Artificial” connotes words like false, fake, synthetic, insincere, contrived, and hollow. As health information (HI) professionals, we don’t want to be associated with those words; we’re trusted, detail-oriented, and accurate, and our work has a very real impact. After all, health information is human information, and that is as real as it gets.

But artificial can also mean man-made or simulated, and HI professionals love a good simulation or Plan-Do-Study-Act cycle. Basically, AI is a set of algorithms, or electronic decision trees, that allow computers to “think” or “act” more like people—a series of “if this, then that” statements written by humans who build devices and systems that are intended to make our lives easier. There’s another clarifier in there: “intended.” As you might imagine, how systems and tools are intended to work is not always consistent with reality. This is why it is so critical to be at the table when decisions are being made. HIM professionals know the requirements like the backs of our hands, so it’s crucial that the people creating the theoretical rules know what the real-world rules are.

As part of our Lean journey, our organization has adapted the Toyota Production System to healthcare. One of the concepts at the core of our management method is Jidoka, a Japanese term that embodies the concept of allowing machines to do what machines do best and allowing humans to focus on what humans do best. In order for AI to work properly, knowing what parts of a process can be automated and what parts rely on human discernment is key.

We have implemented a few different solutions commonly used in health information, and our learnings while implementing computer assisted coding (CAC), clinical documentation integrity (CDI), and scanning software solutions are described below.

Computer-Assisted Coding (CAC)

Our first lesson learned was with CAC. The logic behind CAC has made the transition to ICD-10-CM/PCS much more bearable. Natural language processors (NLP) electronically review the text of the document (or what we call “walk the note”) within the electronic health record (EHR), and based on the presence of diagnostic words and/or phrases, the software applies system logic and standard coding rules in order to propose codes to skilled coding professionals who reference the same documentation and either accept or decline the proposed codes.

The change from ICD-9-CM/PCS to ICD-10-CM/PCS and the increase in available diagnostic codes from around 14,000 to over 70,000 was anticipated to adversely impact historically productive coding teams. Because of the automation of proposed codes within the CAC, when ICD-10-CM/PCS went live, we were able to maintain productivity levels without temporary staff, additional remote coding professionals, or excessive overtime, even with the retirement of our lead coding professional coinciding with the go-live date.

Early on during implementation, we noticed that the CAC has trouble with templated checklists and distinguishing between diagnoses that are options as a part of a list. It cannot tell if the diagnosis is part of a list but NOT checked, if it is listed AND checked, or if “No” is selected as part of a drop-down prompt next to a diagnosis (Figure 1). So, the CAC will suggest all the diagnoses, which requires the coding professional to review them and deselect them for final billing.

Our CAC vendor worked with us to identify if there were templates or auto text that would always be within a certain portion of our note within the EHR. We could have turned off the CAC from that portion of the note to force direct coding, but our templates are used in every portion of the note, so that was not an option. We also determined that deselecting codes would be faster than having our coding professional direct-code only those that were warranted.

Ultimately, the setup is a decision your organization will have to make based on your unique risk-benefit analysis.

Clinical Documentation Integrity (CDI)

The next lesson learned was with our CDI software implementation. With the increase in reliance on risk-adjusted coding and publicly reported quality data, we sought out and implemented a real-time CDI software that prompts providers, within their EHR workflow, to be more specific in their documentation. For example, if a provider types or speaks the phrase “kidney disease,” a window prompt appears asking the provider to specify what stage of disease the patient is in, if known. The software, in conjunction with a dedicated coding team, has done wonders for our ability to impact risk-adjustment scores and more accurately capture chronic conditions.

That said, we were concerned about notification (or alert) fatigue and were excited to find out that we have some ability to customize the software, so we didn’t have to turn on all the diagnoses at once, or even all providers. That excitement, though, was tempered when we realized that our EHR setup could pose a barrier.

Within our EHR, we have a tool that allows providers to pull in detail from elsewhere in the record, all while keeping the original intact and attributing the “copy/paste” back to the source document and author. That functionality is a HI professional’s dream, right? Well, the challenge we ran into is that documentation relies upon separate and distinct sections within the template to hold all of that attribution information, and each of those sections appears to the CDI software as a different and unique note that can (and does) prompt the provider multiple times if that nonspecific diagnosis is mentioned in more than one section.

We worked closely with our CDI vendor on creating many-to-one relationships within documents so the prompt would only appear once per note. This remains a work in process.

We also have more traditional CDI software that allows nurses and coding professionals to review documentation while the patient is in house and after discharge, respectively. With that software, we need to keep in mind how sensitive we want our thresholds to be. This is one of the things we couldn’t predict; we just had to bite the bullet and pick one to trial. If we found we weren’t getting very many markers to review for a diagnosis, we might adjust the sensitivity from low to medium or high. We tried not to set the sensitivity too high as a first step; we were concerned that it would drain our resources. The initial setup can be labor intensive in calibrating queries against the markers that are tee’d up by the software.

And not all markers are good markers. It’s also helpful to keep in mind what you’ve spent a lot of time and energy on with your providers. What disease processes have you specifically provided education to your providers about in regards to their documentation? Know what an appropriate query would be based on the markers that are triggering for your CDI team.

Encephalopathy versus Delirium and Sepsis II versus Sepsis III are real-life examples of where we, as an organization, have chosen to set the sensitivity low. We know that we have drilled into our hospitalists the importance of noting encephalopathy versus the symptom of delirium when the patient does, in fact, have encephalopathy. We don’t query very often for that diagnosis if only delirium is noted because it typically is reflecting that the patient just has confusion or some other sort of psychiatric condition unrelated to a systemic cause. But you may want that threshold set higher if that is a known issue for your organization.

We also maintained our sensitivity thresholds when there was a transition between Sepsis II and Sepsis III criteria. Our organization embarked on a lot of work around identifying sepsis and diagnosing it accurately early in the patient encounter, so if the system markers were not keeping up, we had to adjust the sensitivity to allow the CDI markers to more closely match the clinical practice and not overburden either team with unnecessary markers and queries.

Some of these ideas may seem counterintuitive to a desire to get every possible code and/or comorbidity, but it is up to us to ensure that we are only gleaning appropriate codes from the documentation. We are chasing accuracy, not a major complication or comorbidity (MCC). We only want that MCC if it is appropriate.

Record Scanning

Our journey with the use of AI to improve our scanning processes started in 2011 and brought us many lessons. At that time, we implemented a basic scanning application, driven by bar codes, stickers, and coversheets. We had to catalog and revise hundreds of paper forms from throughout the organization because each one had to have its own bar code for the software to place it in the EHR accurately. It was a lot of work up front, but the goal was to be able to have the software perform at least some of the manual labor by sorting documents into their appropriate places in the medical record. In addition, we were able to leverage the bar coding on our existing patient labels to ensure the document went to the correct patient and encounter.

This basic “if x, then y” level of AI did provide benefits, the most significant being that it allowed us to start putting our paper-based documentation into the EHR. However, there were some limitations:

  • Our team spent a significant amount of time physically prepping the records before scanning, ensuring that the correct patient and document type labels were not only on every page but in the correct location on every page.
  • Document subject lines were hard-coded. For example, everything we scanned in as “outside records: lab” had the same subject line and no way to distinguish them from all the other records with the same name.
  • Everything had to be scanned to an encounter, even documents that should be available across encounters, such as durable power of attorney (DPOA) or physician orders for life-sustaining treatment (POLST) forms.

It was a good foundation and, within the first year, we were able to stop creating paper charts and refocused our files team to prepping and scanning into the EHR. This reduced costs and transportation of medical records (health information services is located off-site from our hospital and nine regional medical centers) and increased availability of information for the clinical teams.

Over time, we identified one of the major limitations of bar coding is that each document needs to have a bar code on it for the scanner to read. On average, our team prepped, scanned, and indexed 12,000 pages per day, and while we were able to leverage pre-printed bar codes on our internal forms, approximately 60 percent of our volume is outside records. This translated to approximately 7,200 pages that needed to have a bar code sticker placed on them every day. We were asking our team to spend hours a day putting stickers on paper. As we started looking at the data, it didn’t seem like the best use of time and resources in our department, and we began to search for a new application. One of the main requirements we had in this search was a way to eliminate this manual work, freeing up our team to concentrate on more value-added tasks like reviewing and indexing the documents.

We were able to implement our new application starting in August 2019. In contrast to our old system, our new system uses optical character reader technology to scan the entire document and pick out relevant information that identifies the document type and patient information to be able to auto-index documents without bar codes. This increased level of AI allowed us to minimize the manual prepping of documents. For example, if a document has a header of “history and physical,” the system will be able to recognize those words and match them to the history and physical work type code set in our EHR so that it gets filed correctly. In addition, the ability for the system to read the entire page for clues to the patient’s identity and document type solved our sticker problem; we were able to decrease time spent in prepping records for scanning significantly, as well as the costs associated with the stickers themselves.

In addition, the AI supports a visual user interface that makes it easy for the indexer to see what the system has and has not been able to auto-index. If the software reads the document and has a high confidence that the document matches the elements of a particular document type, it is indicated by a green bar on the left-hand side. In addition, if it can find and match at least two patient identifiers, the document will be outlined in green. If the system is not able to match any of these elements with high confidence, the side bar and/or document outline will display as red (Figure 2). This is a very clear visual cue to the indexer that no more than a quick check is required for auto-indexed documents and draws attention to documents that need to be reviewed more thoroughly. The intent is to separate the work that the machine can easily do from the work that requires critical thinking skills to accomplish.

One of the biggest learnings we have had from implementing new scanning software is that the volume of records we receive and process from outside organizations makes it more difficult for the software to propose document types. The software is reliant on the maintenance of certain rules to build its model for document type suggestions, and when the documents are not within the control of your organization, it’s more difficult to build rules that will easily apply to most documents. It is not an exact science, but we are definitely better at setting the rules than when we started.

Constant care and feeding of the model is required. Images that the system doesn’t recognize are always flowing into the module for an analyst to assess and create rules for recognition. The algorithms are only as good as the people who create them, and it is important that they are monitored on a regular basis. It is also important that this task is appropriately situated in your department. At first, we had placed this task with our analyst team, but we soon realized that they were not close enough to the daily work. At that point, we shifted the work to a registered health information technician who also works in the system daily and has a good working knowledge of the documents we receive.

Guiding Principles and Conclusions

As you’ve seen through our experience, AI is not a panacea that will fix broken processes or replace humans. It requires thoughtful design, build, and maintenance to provide long-term benefits for our patients and teams.

In all of the lessons we learned, a few things stick out as high-level guiding principles to embed in any project.

  • Know what your goal is, based on your organizational philosophy. Don’t let the software move you from your true north, no matter how shiny it seems.
  • Understand what pieces of the software will allow for customization and which parts will not. Ask to see a client who is using that feature in real life, if possible.
  • A solid working relationship with your vendor is key. There is no such thing as over-communication.
  • When it comes to vendors and their algorithms, there is quite a bit that is proprietary information that you can’t necessarily see before you go live. Be flexible and willing to change throughout implementation.
  • Share your organization’s goals with your vendor ahead of time to gain alignment.
  • Know your own data and workflows before embarking on any AI project. This seems like common sense, but it cannot be emphasized enough. This is critical to being able to identify if the advertised juice is worth your squeeze. You’re also far more likely to know if available functionality within the software will give you the benefit you’re looking for.
  • You don’t know what you don’t know. Do the research and gather the data before you decide that some sort of technology is right for your team. And be prepared to find out that even when you thought you knew it all, you likely didn’t.
  • Don’t let perfection be the enemy of good. If you try to plan and build the perfect system, you will never get started. Shoot for building something you can work with and improving it over time. With all of the examples we shared, we knew that there were limitations and problems that would arise, but we knew our end game was worth it.
Resources

 

Amy Bolin-Deon (Amy.Bolin-Deon@VirginiaMason.org) has worked at Virginia Mason Medical Center for more than eight years and is currently the director of Health Information Services and Privacy Office.

Lisa M. M. Woodley (Lisa.Woodley@VirginiaMason.org) has worked at Virginia Mason Medical Center for more than 13 years and is currently the privacy programs manager and manager of record integrity in Health Information Services.

Tags: technology , clinical documentation integrity , computer-assisted coding , artificial intelligence , health IT