Generative AI (GenAI) has been getting a lot of traction in healthcare even though its integration faces numerous challenges including skepticism over its reliability, data privacy concerns, and complex regulatory and integration hurdles. According to McKinsey, 85% of healthcare leaders in the US are exploring or have already adopted GenAI. In India, EY’s findings show 25% of healthcare CIOs have already started using GenAI, while 60% plan to in the next 6-12 months.
For GenAI to work in healthcare, general language models won’t be as effective due to the specific and contextual language of clinical diagnostics. Also the models cannot function as a black box and their decision making has to be explainable and transparent, insists John Edwards, senior vice president, Citius Healthcare Consulting, a unit of CitiusTech Inc, a Mumbai-Princeton based healthtech company. In an interaction with CXOtoday, Edwards talks about the lack of trust in such tools among clinicians, as well as the issues of data fragmentation and technical debt, which are making GenAI implementation costly and slow.
Edited excerpts:
Q. How and where is GenAI being used in healthcare?
The new wave of interest in AI that started first with GenAI and now with agentic AI has caused healthcare to finally wake up and realize the potential that AI could bring to their enterprises. Traditionally, we see healthcare lagging behind other industries. Within healthcare, usually pharma leads the path because they have more money to spend than payers (insurance) and then providers (hospitals) do a little. Interestingly, providers have actually invested more in AI than payers and pharma, which is a striking trend.
The majority of that investment over the past year has gone into ambient listening. Doctors are getting accustomed to having notes being dictated and then a machine interpreting the dictation, and then somebody else cleaning it up before they have to sign off to ensure the notes are accurate for their medical records. Around 30 to 40% of large health systems in the US have physicians that are bringing their phones into the meetings with their patients and asking permission every time.
Q. Have the practical results of using GenAI lived up to the hype?
There really hasn’t been a significant increase in productivity for doctors because the data is still not being utilized. This points to the real obstacle that we see in healthcare with data use. Doctors have been trained to interpret information and make optimal decisions about their subsequent steps in patient care. Trusting a machine to make the same decisions that they would make is difficult, and the reality is that even doctors might not make the same choices from the same set of data. There is clinical variation that occurs in practice because of their experience and training.
Getting over the obstacle of trust is probably going to be the next hurdle that we see people tackling as they try to take this investment in new digitized information and find ways to derive productivity and value from it.
Before most enterprises scale those solutions, they will want to go through data quality, trust, and training exercises and achieve a level where they can better understand how to trust this set of information.
Q. What is the reason for this gap in intent and expectations from GenAI?
Too much focus has been placed on the hallucinations generated by AI, and novice users who have tried to use AI on their own have experienced some of those hallucinations because they have not asked their questions (prompts) well. They haven’t trained the datasets how to interpret and have context. General language models are being used for interpretation, but in healthcare, language is highly specific and contextual. Same terms when put together in different ways can have very different meanings.
I believe the next horizon for providers will be converting ambient listening into useful information that inspires agentic AI workflow. However, we are seeing investment in both provider-facing tools and the revenue cycle.
Q. Where do you see agentic AI make an immediate impact?
The revenue cycle constitutes a significant BPO business in India today, employing large workforces for tasks such as coding checks, eligibility checks, timely claim submission, and appropriate payment collection. A lot of that work is highly predictable and repetitive. In the past, some companies have used machine learning (ML) based solutions for it. I expect a shift towards greater use of agentic AI to automate some of those BPO activities. It will lead to a revision in contracts and terms, where rewards will be negotiated based on outcomes rather than hours spent or resources invested.
There is also a significant threat to companies that made premature claims about AI proficiency, but have not been able to translate their enthusiasm into measurable outcomes. We are already hearing from some of our health tech customers that their clients are renegotiating prices, warranties, and guarantees. Consequently, the market will hold both health tech and service companies accountable for generating significant value from AI in healthcare.
Q. How important is transparency and reliability of AI solutions?
In healthcare, people will retrospectively question decisions made and the process by which they were reached. AI cannot be a black box. Its process must be documented step-by-step. The process has to be transparent, so it is clear where human oversight was involved, both in training and application of these decisions. So, even as we seek productivity gains and better care outcomes, a clear audit trail is essential to help companies demonstrate responsible AI use.
Responsible AI and governance will be another critical area of investment. Governance concern is one of the factors that is holding back many payers from investing in AI.
Medical decisioning solutions are easy to build, but difficult to prove reliable because you have to go through extensive testing and training. This testing and training doesn’t stop during the build phase. Therefore, you must continue to create new content that can be used to retrain the model. You also have to continuously test and implement specific operational measures and new techniques to guard against hallucinations and unexpected results.
Q. What else is holding back healthcare organizations from adopting AI?
The other factor that is holding back payers and providers from fully using AI is the technical debt. Organizations that haven’t digitized their solutions or created effective API (application programming interface) and interface strategies will face challenges. There are known solutions for building bridges that will allow data to be integrated appropriately, but many payers are still relying on mainframes for processing claims. AI is compute hungry. It uses more data than traditional rule-based transactions. The current architecture of some of these legacy solutions means that AI integration will break the bank.
The reality is that the promise of performance gains of AI requires process re-engineering, people upskilling and technology updates. The lifeblood of AI is data. What is needed is investments in unlocking and integrating data in a way that allows it to be useful for asking questions. Since we haven’t always trained our people to be critical thinkers, we will likely also need to do training focused on information as a service and insights.
Q. What sort of challenge does data fragmentation present for AI in healthcare?
Before we can understand the patient, we first have to understand the clinical dialogue doctors are having and ensure that ambient listening can be interpreted appropriately. Until recently, there were significant obstacles to pulling data from EMR (electronic medical records) and integrating the provider’s view of that data with that of the payers. Now, with FHIR (Fast Healthcare Interoperability Resources) integration and the ability to extract that data, those obstacles have been removed. The fragmentation is real. The estimate is that only 30 to 40% of the necessary data is digitized and integrated today. While it is becoming more realistic to pull out a clinical data set and organize it, creating that infrastructure still requires a significant investment.
Q. How can clinicians use these GenAI tools effectively to assist them and plan treatment for diagnosis?
The simplest way to use AI effectively is to allow it to summarize information that doctors can ask questions from before preparing to visit a patient. This will be the first step as doctors become accustomed to using AI as another lever for gaining the information needed to make their decisions. You can then show how the data was sourced and its lineage, demonstrating that it was quality data. It is the same data a human would have used to make that decision and that it’s using the same logic and trees that it has been instructed to use. Also, the doctor should be able to ask follow-up questions that might cause them to think differently.