AI and the Future of Health
The emergence of increasingly capable large-scale AI models, such as the recently released GPT-4, is one of the most significant advances in computing in decades. These innovations are rapidly transforming every aspect of the value we get from technology, as demonstrated through Microsoft’s integration of GPT-4 into Bing, Edge, Microsoft 365, Power Platform, GitHub, and other offerings. More recently, Nuance has announced DAX Express, which uses a unique combination of conversational, ambient, and generative AI to automatically draft clinical notes after patient visits – helping to reduce care providers’ cognitive burdens and increase the joy of practicing medicine (whilst releasing time for care).
We are at an inflection point for the use of AI in healthcare – one of society’s most critical sectors. The significance of this moment is reflected in Peter Lee’s recent article in the New England Journal of Medicine on the potential future clinical applications of GPT-4. At Microsoft Research’s Health Futures organization, the multidisciplinary group dedicated to discovery in this space, we see this as the continuation of a journey, and a major milestone in the long process of innovating to help address the greatest challenges in healthcare.
In this blog, we will share some of our research team’s work to make healthcare more data-driven, predictive, and precise – ultimately, empowering every person on the planet to live a healthier future.
Enabling precision medicine and connected care
We are today at a unique moment in history where medicine, biology, and technology are converging on a large scale. This presents immense possibilities to revolutionize healthcare and the practice of medicine with the aid of trustworthy AI. While we embrace the potential of AI, we understand that the practice of medicine is an intricate balance of “art” and “science.” We recognize and honor the enduring physician-patient relationship, which is fundamental and timeless. Our diverse team comprises researchers, scientists, engineers, biotechnologists, designers, social scientists, strategists, healthcare experts, and medical professionals who collaborate globally and inclusively to reimagine and transform the lives of the patients and public we serve.
As we consider how technologies have shaped the practice of medicine over the centuries, from the individual to the ecosystem level, we are reminded that no technology exists in a vacuum. Our core understanding of biological systems is rapidly evolving, and with it, our understanding of what technologies are relevant and useful. Simultaneously, the use of technology across the health and life science industries, and the way healthcare is delivered, are also rapidly changing – reshaping our traditional healthcare delivery model from one of diagnosis and treatment, to one that prioritizes prevention and precise individualized care.
SPOTLIGHT: ON-DEMAND VIDEO
AI Explainer: Foundation models and the next era of AI
Explore how the transformer architecture, larger models and more data, and in-context learning have helped advance AI from perception to creation.
Recent advancements in machine learning and AI have fueled computational technologies that allow us to aggregate complex inputs from multiple data sources, with the potential to derive rich insights that rapidly expand our knowledge base and drive deeper discovery and faster innovation. At the same time, it remains an open question how to best use and regulate these technologies in real-world settings and at scale across healthcare and the life sciences. Nonetheless, we believe that we are on a path to delivering on the goal of precision medicine – a change in clinical practice which will be enabled by precision diagnostics, precision therapeutics, and connected care technologies.
To achieve this goal, we seek to collaborate with health and life sciences organizations with a similar appetite for transformation, complementary expertise, and a commitment to propel the change required. We are also engaged with the broader community in pursuing responsible and ethical use of AI in healthcare. Our diverse team has been successful in bridging the gap between the fields of medicine, biology and chemistry on one hand, and computing on the other. We act as “translators” between these fields, and through a process of ongoing collaboration and feedback, we have discovered new challenges and innovative solutions.
Below are some examples of our collaborative research approach:
Exploring diagnostic tools from new modalities
Multimodal foundation models for medicine: an example from radiology
The field of biomedicine involves a great deal of multimodal data, such as radiology images and text-based reports. Interpreting this data at scale is essential for improving care and accelerating research. Radiology reports often compare current and prior images to track changes in findings over time. This is crucial for decision making, but most AI models do not take into account this temporal structure. We are exploring a novel self-supervised framework that pre-trains vision-language models using pairs of reports and sequences of images. This includes handling missing or misaligned images and exploiting temporal information to learn more efficiently. Our approach, called BioViL-T, achieves state-of-the-art results on several downstream tasks, such as report generation, and interpreting disease progression by focusing on relevant image regions across time. BioViL-T is part of ongoing collaboration with our colleagues at Nuance to develop scalable and flexible AI solutions for radiology that can empower care providers and augment existing workflows.
Project InnerEye: Democratizing Medical Imaging AI
Project InnerEye is a research project that is exploring ways in which machine learning has the potential to assist clinicians in planning radiotherapy treatments so that they can spend more time with their patients. Project InnerEye has been working closely with the University of Cambridge and Cambridge University Hospitals NHS Foundation Trust to make progress on this problem through a deep research collaboration. To make our research as accessible as possible, we released the InnerEye Deep Learning Toolkit as open-source software. Cambridge University Hospitals NHS Foundation Trust and University Hospitals Birmingham NHS Trust led an NHS AI in Health and Care Award to evaluate how this technology could potentially save clinicians’ time, reduce the time between the scan and commencing treatment, and scale this to more NHS Trusts. Any clinical use of the InnerEye machine learning models remains subject to regulatory approval.
- PROJECTProject InnerEye
- NEWSUHB and Cambridge University Hospitals win government funding for ground-breaking AI in radiotherapy
Immunomics: Decoding the Immune System to Diagnose Disease
The human immune system is an astonishing diagnostic engine, continuously adapting itself to detect any signal of disease in the body. Essentially, the state of the immune system tells a story about virtually everything affecting a person’s health. What if we could “read” this story? Our scientific understanding of human health would be fundamentally advanced. More importantly, this would provide a platform for a new generation of precise medical diagnostics and treatment options. We are partnering with Adaptive Biotechnologies to develop the machine learning and biotechnology tools that will allow us to realize this dream.
- PROJECTMicrosoft Immunomics
Fundamental advances towards new medicines and therapeutics
Protein Engineering
Several research groups are delving into the potential of machine learning to enhance our comprehension of proteins and their pivotal role in various biological processes. We are also using AI to design new proteins for therapeutics and industry. By applying machine learning to extract patterns from databases of sequences, structures, and properties, Microsoft hopes to train models that can make protein engineering by directed evolution more efficient, and directly generate proteins that will perform desired functions. The ability to generate computationally distinct yet viable protein structures holds tremendous promise for uncovering novel biological insights and developing targeted therapies for previously untreatable illnesses.
Investigating the Cancer Microenvironment through Ex Vivo Research
Microsoft is working on ways to identify specific characteristics of cancer cells and their surrounding microenvironments that might be targeted for treatment. By studying how cancer cells and their surroundings interact with each other, the team aims to create a more precise approach to cancer treatment that takes into account both genetic and non-genetic factors.
- PROJECTProtein Folding
- VIDEOEx Vivo project lightning talk at Research Summit 2022 (starts at 7m40s)
Accelerating biomedical research
Microsoft and the Broad Institute – combining their expertise in genomics, disease research, cloud computing and data analytics – are developing an open-source platform to accelerate biomedical research using scalable analytical tools. The platform is built on top of the Broad Institute’s Terra platform, providing a user-friendly interface for accessing and analyzing genomic data. Leveraging Microsoft’s Azure cloud computing services, the platform will enable secure storage and analysis of large datasets. Additionally, the platform will incorporate machine learning and other advanced analytical tools to help researchers gain insights into complex diseases and develop new treatments.
Advancing clinical interpretation and exploration through multimodal language models
In the quest for precision medicine and accelerating biomedical discovery, Microsoft is committed to advancing the state of the art in biomedical natural language processing (NLP). A crucial factor in future-facing, data-driven health systems is the accessibility and interpretability of multimodal health information. To meet this need, Microsoft has laid a solid foundation across multiple modalities in biomedical NLP building on our deep research assets in deep learning and biomedical machine reading.
One significant achievement is our development and application of large language models (LLMs) in biomedicine. Microsoft was among the first to create and assess the applicability of LLMs, such as PubMedBERT and BioGPT, which are highly effective in structuring biomedical data. However, to address the inherent limitations of LLMs, Microsoft is developing methods to teach them to fact-check themselves and provide fine-grained provenance. Additionally, Microsoft is exploring ways to facilitate efficient verification with humans in the loop.
Besides text, other modalities such as radiology images, digital pathology slides, and genomics contain valuable health information. Microsoft is developing multimodal learning and fusion methods that incorporate these modalities. These methods include predicting disease progression and drug response, with the ultimate goal of delivering safe and high-quality healthcare.
Observational data in biomedicine is often plagued by confounders, making it challenging to draw causal relationships. To overcome this obstacle, Microsoft is developing advanced causal methods that correct implicit biases and scale biomedical discovery. These methods will allow Microsoft to leverage real-world evidence and contribute to the creation of more effective healthcare delivery systems. For our end-to-end biomedical applications, we have made exciting progress in deep collaborations with Microsoft partners such as The Jackson Laboratory and Providence St. Joseph Health.