Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
"A learning health care system is one in which science, informatics, incentives, and culture are aligned for continuous improvement and innovation, with best practices seamlessly embedded in the care process, patients and families active participants in all elements, and new knowledge captured as an integral by-product of the care experience." [1]
The concept of a learning health system has been around for almost two decades [2]. It is envisioned as a cyclical process involving stages for transformation of Data to Knowledge (D2K), implementation of Knowledge into Practice or Performance (K2P), and assessment of Practice or Performance through Data (P2D) [3]. These cycles bring together "Discovery" and "Implementation" to address a Health Problem of Interest, guided by a multi-stakeholder learning community [4].
Electronic health record (EHR) systems capture a wealth of Health Data that can be used for clinical, quality, and research purposes. These data can be analyzed to validate existing knowledge or generate new knowledge about disease diagnosis, treatment, and prevention (D2K). This knowledge can inform inform creation of new protocols, guidelines, and educational materials, which may be put into practice as decision support tools in EHR systems (K2P). The performance of these tools for improving diagnosis, treatment, and prevention can then be assessed through new EHR and other data (P2K).
Committee on the Learning Health Care System in America; Institute of Medicine. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Smith M, Saunders R, Stuckhardt L, McGinnis JM, editors. Washington (DC): National Academies Press (US); 2013 May 10. PMID: 24901184.
Institute of Medicine (US). Digital Infrastructure for the Learning Health System: The Foundation for Continuous Improvement in Health and Health Care: Workshop Series Summary. Grossmann C, Powers B, McGinnis JM, editors. Washington (DC): National Academies Press (US); 2011. PMID: 22379651.
Flynn AJ, Friedman CP, Boisvert P, Landis-Lewis Z, Lagoze C. The Knowledge Object Reference Ontology (KORO): A formalism to support management and sharing of computable biomedical knowledge for learning health systems. Learn Health Syst. 2018 Apr 16;2(2):e10054. doi: 10.1002/lrh2.10054. PMID: 31245583; PMCID: PMC6508779.
Friedman CP. What is unique about learning health systems? Learn Health Syst. 2022 Jul 15;6(3):e10328. doi: 10.1002/lrh2.10328. PMID: 35860320; PMCID: PMC9284922.
National Academy of Medicine: The Learning Health System Series
Institute of Medicine (US) Roundtable on Evidence-Based Medicine. The Learning Healthcare System: Workshop Summary. Olsen L, Aisner D, McGinnis JM, editors. Washington (DC): National Academies Press (US); 2007. PMID: 21452449.
Rosenthal GE, McClain DA, High KP, Easterling D, Sharkey A, Wagenknecht LE, O'Byrne C, Woodside R, Houston TK. The Academic Learning Health System: A Framework for Integrating the Multiple Missions of Academic Medical Centers. Acad Med. 2023 Sep 1;98(9):1002-1007. doi: 10.1097/ACM.0000000000005259. Epub 2023 Apr 25. PMID: 37099650; PMCID: PMC10453356.
Collard HR, Grumbach K. A Call to Improve Health by Achieving the Learning Health Care System. Acad Med. 2023 Jan 1;98(1):29-35. doi: 10.1097/ACM.0000000000004949. Epub 2022 Aug 23. PMID: 36006840.
Forrest CB, Chesley FD Jr, Tregear ML, Mistry KB. Development of the Learning Health System Researcher Core Competencies. Health Serv Res. 2018 Aug;53(4):2615-2632. doi: 10.1111/1475-6773.12751. Epub 2017 Aug 4. PMID: 28777456; PMCID: PMC6051975.
Journal: Learning Health Systems (LHS)
This chapter provides foundational knowledge on biomedical informatics (and its sub-discipline of health informatics), implementation science, and related topics. Each topic page offers a brief overview as an introduction with references to other chapters in CODIAC for Health for more details as well as to external references and resources for learning more.
Visit other chapters in CODIAC for Health using the Table of Contents or menu in the upper left corner.
Implementation science can be defined as “the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice, and, hence, to improve the quality and effectiveness of health services.” [1]
It takes 17-20 years for research innovations to be used in routine clinical practice [2]. This gap, called the research-practice gap, spurred along an effort to develop methods that put evidence-based practices into the hands of the people who most need them. Though its roots can be traced back to the early 1900s, Implementation Science has existed by name since the early 2000s [3]. Over the last 25 years, there has been an explosion of resources for doing implementation science, including over 150 theories, models, and frameworks and dozens of implementation methods [4,5]. These methods are derived from myriad disciplines, including human-computer interaction, improvement science, public health, clinical psychology, and anthropology. Implementation science methods are being applied across the globe in hospitals, clinics, schools, and beyond.
A recent article (Curran, 2020) provides some helpful terminology for understanding the complementary nature of implementation research to clinical research [6] .
The thing is the innovation that one is trying to implement. The innovation can be anything: a pill, a procedure, a policy, a “nudge,” an algorithm.
Traditional efficacy and effectiveness research seeks to understand whether that thing achieves its intended clinical outcome. Did patients’ health or mental health outcomes improve resulting from use of this thing?
Implementation research seeks to understand how to help places and people do the thing in the best way possible. For example, did clinicians use the thing? Did patients like the thing? Can a healthcare system sustain the thing?
To put it practically, if cognitive behavioral therapy (CBT) delivered as a telephone app is our thing, efficacy and effectiveness research seeks to understand whether the delivery of CBT via the app decreases anxiety symptoms and diagnoses. Implementation research focuses on exploring the best way to help clinicians, clinics, and patients adopt and sustain their use of the CBT app.
Implementation scientists strive to improve health equity and close the research-practice gap by partnering with communities to explore and prepare the implementation context (e.g., a clinic or hospital), systematically develop implementation strategies to overcome barriers to evidence-based practice implementation, and evaluate implementation efforts. Implementation science methods can be used across several stages of treatment development and testing: designing innovations for sustainable implementation, adapting and implementing established evidence-based practices, and optimizing evidence-based practices that are already in use.
Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An introduction to implementation science for the non-specialist. BMC Psychol. 2015 Sep 16;3(1):32. doi: 10.1186/s40359-015-0089-9. PMID: 26376626; PMCID: PMC4573926.
Balas EA, Boren SA. Managing Clinical Knowledge for Health Care Improvement. Yearb Med Inform. 2000;(1):65-70. PMID: 27699347.
Eccles MP, Mittman BS. Welcome to Implementation Science. Implementation Sci. 2006;1(1). https://doi.org/10.1186/1748-5908-1-1
Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015 Apr 21;10:53. doi: 10.1186/s13012-015-0242-0. PMID: 25895742; PMCID: PMC4406164.
Beidas RS, Dorsey S, Lewis CC, Lyon AR, Powell BJ, Purtle J, Saldana L, Shelton RC, Stirman SW, Lane-Fall MB. Promises and pitfalls in implementation science from the perspective of US-based researchers: learning from a pre-mortem. Implement Sci. 2022 Aug 13;17(1):55. doi: 10.1186/s13012-022-01226-3. PMID: 35964095; PMCID: PMC9375077.
Curran GM. Implementation science made too simple: a teaching tool. Implement Sci Commun. 2020 Feb 25;1:27. doi: 10.1186/s43058-020-00001-z. PMID: 32885186; PMCID: PMC7427844.
Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: models for dissemination and implementation research. Am J Prev Med. 2012 Sep;43(3):337-50. doi: 10.1016/j.amepre.2012.05.024. PMID: 22898128; PMCID: PMC3592983.
"The capability of handling big data is becoming an enabler to carry out unprecedented research studies and to implement new models of healthcare delivery." [1]
The term "big data" has been used since the early 1990s [2]. Big data are characterized by the "3 Vs": Volume (size), Velocity (speed of generation), and Variety (different types) [1]. This has expanded to additional Vs (5 Vs, 10 Vs, 14 Vs, etc.) such as: Veracity, Value, Validity, Variability, and Vocabulary.
There are many sources of big data in biomedicine and health care [3]. These include Electronic Health Records (EHR) [4], Health Information Exchanges (HIE) [5], All-Payer Claims Databases (APCD) [6], biological and biomedical databases [7], and public health surveys [8].
Health data can be broadly categorized as "structured" (e.g., demographics, diagnoses, procedures, and medications) or "unstructured" (e.g., clinical reports and notes) [9]. Use of established Health Data Standards is critical for sharing and exchange of health data within and across organizations to support Artificial Intelligence in Health and Observational Health Research.
See CODIAC for Health chapter on Health Data and Data Standards (forthcoming) for more information.
Bellazzi R. Big data and biomedical informatics: a challenging opportunity. Yearb Med Inform. 2014 May 22;9(1):8-13. doi: 10.15265/IY-2014-0024. PMID: 24853034; PMCID: PMC4287065.
Lohr S. The Origins of ‘Big Data': An Etymological Detective Story. The New York Times. 2013 Feb 1. [ Link ]
Healthcare Big Data and the Promise of Value-Based Care. Catalyst Carryover. 2018 Jan 1. [ Link ]
Ehrenstein V, Kharrazi H, Lehmann H, et al. Obtaining Data From Electronic Health Records. In: Gliklich RE, Leavy MB, Dreyer NA, editors. Tools and Technologies for Registry Interoperability, Registries for Evaluating Patient Outcomes: A User’s Guide, 3rd Edition, Addendum 2 [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2019 Oct. Available from: https://www.ncbi.nlm.nih.gov/books/NBK551878/
Sarkar IN. Health Information Exchange as a Global Utility. Chest. 2023 May;163(5):1023-1025. doi: 10.1016/j.chest.2022.12.001. PMID: 37164575.
Love D, Custer W, Miller P. All-payer claims databases: state initiatives to improve health care transparency. Issue Brief (Commonw Fund). 2010 Sep;99:1-14. PMID: 20830868.
Sayers EW, Beck J, Bolton EE, et al. Database resources of the National Center for Biotechnology Information. Nucleic Acids Res. 2024 Jan 5;52(D1):D33-D43. doi: 10.1093/nar/gkad1044. PMID: 37994677; PMCID: PMC10767890.
Blewett LA, Call KT, Turner J, Hest R. Data Resources for Conducting Health Services and Policy Research. Annu Rev Public Health. 2018 Apr 1;39:437-452. doi: 10.1146/annurev-publhealth-040617-013544. Epub 2017 Dec 22. PMID: 29272166; PMCID: PMC5880724.
Weber GM, Mandl KD, Kohane IS. Finding the missing link for big biomedical data. JAMA. 2014 Jun 25;311(24):2479-80. doi: 10.1001/jama.2014.4228. PMID: 24854141.
Ehrenstein V, Kharrazi H, Lehmann H, et al. Obtaining Data From Electronic Health Records. In: Gliklich RE, Leavy MB, Dreyer NA, editors. Tools and Technologies for Registry Interoperability, Registries for Evaluating Patient Outcomes: A User’s Guide, 3rd Edition, Addendum 2 [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2019 Oct. Available from: https://www.ncbi.nlm.nih.gov/books/NBK551878/
NIH Pragmatic Trials Collaboratory Rethinking Clinical Trials
Secondary Analysis of Electronic Health Records [Internet]. Cham (CH): Springer; 2016. Available from: https://www.ncbi.nlm.nih.gov/books/NBK543630/ doi: 10.1007/978-3-319-43742-2.
Sarkar IN. Transforming Health Data to Actionable Information: Recent Progress and Future Opportunities in Health Information Exchange. Yearb Med Inform. 2022 Aug;31(1):203-214. doi: 10.1055/s-0042-1742519. Epub 2022 Dec 4. PMID: 36463879; PMCID: PMC9719753.
Sarkar IN. Health Information Exchange as a Global Utility. Chest. 2023 May;163(5):1023-1025. doi: 10.1016/j.chest.2022.12.001. PMID: 37164575.
"Health data standards are key to the U.S. quest to create an aggregated, patient-centric electronic health record; to build regional health information networks; to interchange data among independent sites involved in a person’s care; to create a population database for health surveillance and for bioterrorism defense; and to create a personal health record." [1]
Health data standards and interoperability are essential for the seamless exchange and interpretation of data within and across systems and organizations to address the issue of having "too many ways to say the same thing" (e.g., Health Data sources such as electronic health record [EHR] systems across different hospitals and health systems).
There are different levels of standards and interoperability including "syntactic" (structure or format) and "semantic" (content or meaning). Common data models (CDM) such as the Observational Medical Outcomes Partnership (OMOP) CDM are an example of syntactic standards [2]. Terminologies, vocabularies, or coding systems defined in the United States Core Data for Interoperability (USCDI) are examples of semantic standards [3]. These include:
ICD-10-CM (International Classification of Diseases, Tenth Revision, Clinical Modification) for diagnoses
CPT (Current Procedural Terminology) for procedures
LOINC (Logical Observation Identifiers Names and Codes) for laboratory tests, clinical observations, etc.
RxNorm for medications
SNOMED CT (Systematized Nomenclature of Medicine Clinical Terms) for clinical data in EHR systems
Syntactic and semantic standards are developed and maintained by numerous Standards Development Organizations (SDOs) [1]. Founded in 1987, HL7 International is a major SDO that provides a framework and standards for exchange, integration, sharing, and retrieval of electronic health information (e.g., in EHR systems). HL7 primary standards for integration and interoperability include Version 2.x (or V2), Version 3.x (or V3), CDA (Clinical Document Architecture), and Fast Healthcare Interoperability Resources (FHIR) [4].
See CODIAC for Health chapter on Observational Research with EHR Data for more information.
Hammond WE. The making and adoption of health data standards. Health Aff (Millwood). 2005 Sep-Oct;24(5):1205-13. doi: 10.1377/hlthaff.24.5.1205. PMID: 16162564.
Weeks J, Pardee R. Learning to Share Health Care Data: A Brief Timeline of Influential Common Data Models and Distributed Health Data Networks in U.S. Health Care Research. EGEMS (Wash DC). 2019 Mar 25;7(1):4. doi: 10.5334/egems.279. PMID: 30937326; PMCID: PMC6437693.
Bodenreider O, Cornet R, Vreeman DJ. Recent Developments in Clinical Terminologies - SNOMED CT, LOINC, and RxNorm. Yearb Med Inform. 2018 Aug;27(1):129-139. doi: 10.1055/s-0038-1667077. Epub 2018 Aug 29. PMID: 30157516; PMCID: PMC6115234.
Braunstein ML. Healthcare in the Age of Interoperability: The Promise of Fast Healthcare Interoperability Resources. IEEE Pulse. 2018 Nov-Dec;9(6):24-27. doi: 10.1109/MPUL.2018.2869317. PMID: 30452344.
Bodenreider O. The Unified Medical Language System (UMLS): integrating biomedical terminology. Nucleic Acids Res. 2004 Jan 1;32(Database issue):D267-70. doi: 10.1093/nar/gkh061. PMID: 14681409; PMCID: PMC308795.
Reich C, Ostropolets A, Ryan P, Rijnbeek P, Schuemie M, Davydov A, Dymshyts D, Hripcsak G. OHDSI Standardized Vocabularies-a large-scale centralized reference ontology for international data harmonization. J Am Med Inform Assoc. 2024 Feb 16;31(3):583-590. doi: 10.1093/jamia/ocad247. PMID: 38175665; PMCID: PMC10873827.
Wei WQ, Bastarache LA, Carroll RJ, Marlo JE, Osterman TJ, Gamazon ER, Cox NJ, Roden DM, Denny JC. Evaluating phecodes, clinical classification software, and ICD-9-CM codes for phenome-wide association studies in the electronic health record. PLoS One. 2017 Jul 7;12(7):e0175508. doi: 10.1371/journal.pone.0175508. PMID: 28686612; PMCID: PMC5501393.
Biomedical informatics is a trans-disciplinary field that “studies and pursues the effective uses of biomedical data, information, and knowledge for scientific inquiry, problem solving and decision making, motivated by efforts to improve human health.” [1]
The origins of biomedical informatics date back to the 1950s. While the name and definition of the field have evolved with advancements in data, technology, and knowledge, the motivations have remained the same: to advance biomedical discovery and healthcare delivery. This discipline broadly involves the development, application, and evaluation of approaches for generating, organizing, managing, analyzing, and sharing data to support clinical care, patient engagement, biomedical research, quality and safety, education, and public health. These approaches are often adapted and integrated from disciplines such as applied mathematics, biostatistics, computer science, cognitive science, data science, implementation science, library and information science, and management science.
There are many sub-disciplines of biomedical informatics such as health informatics that encompasses [2,3]:
Clinical Research Informatics: Development of approaches for enabling the discovery, management, and evaluation of new health knowledge;
Clinical Informatics: Development and application of techniques to improve health care delivery services (clinical informatics is a subspecialty of the American Board of Medical Specialties);
Consumer Health Informatics: Development of information structures and approaches for supporting patient-centric health care needs; and,
Public Health Informatics: Development of methodologies for supporting public health needs, including surveillance, prevention, preparedness, and health promotion.
The Data-Information-Knowledge-Wisdom (DIKW) model, established in 1989, serves as a fundamental framework in biomedical informatics and its sub-discipline of health informatics, illustrating the progression from raw data to meaningful knowledge and actionable wisdom within a healthcare context [4-6]. Guided by the DIKW model, health informaticians use transdisciplinary approaches and collaborations to advance Learning Health Systems and Artificial Intelligence in Health.
Kulikowski CA, Shortliffe EH, Currie LM, Elkin PL, Hunter LE, Johnson TR, Kalet IJ, Lenert LA, Musen MA, Ozbolt JG, Smith JW, Tarczy-Hornoch PZ, Williamson JJ. AMIA Board white paper: definition of biomedical informatics and specification of core competencies for graduate education in the discipline. J Am Med Inform Assoc. 2012 Nov-Dec;19(6):931-8. doi: 10.1136/amiajnl-2012-001053. Epub 2012 Jun 8. PMID: 22683918; PMCID: PMC3534470.
Chen ES, Sarkar IN. *informatics: Identifying and Tracking Informatics Sub-Discipline Terms in the Literature. Methods Inf Med. 2015;54(6):530-9. doi: 10.3414/ME14-01-0088. Epub 2015 May 22. PMID: 25998007.
American Medical Informatics Association (AMIA) - What is Informatics?
Ackoff RL. From data to wisdom. Journal of applied systems analysis. 1989;16(1):3-9.
Fayyad U, Piatetsky-Shapiro G, Smyth P. From Data Mining to Knowledge Discovery in Databases. AIMag [Internet]. 1996Mar.15 [cited 2024Nov.5];17(3):37. Available from: https://ojs.aaai.org/aimagazine/index.php/aimagazine/article/view/1230
Chen ES, Sarkar IN. Mining the electronic health record for disease knowledge. Methods Mol Biol. 2014;1159:269-86. doi: 10.1007/978-1-4939-0709-0_15. PMID: 24788272.
Shortliffe EH, Cimino JJ, Chiang M, editors. Biomedical informatics: computer applications in health care and biomedicine. 5th ed. Springer; 2021. [ Link ]
Sarkar IN, editor. Methods in Biomedical Informatics. Academic Press; 2014. ISBN: 9780124016781. [ Link ]
Hersh W, editor. Health Informatics: Practical Guide. 8th ed. 2022. [ Link ]
Collen M, Ball M, editors. The History of Medical Informatics in the United States. London: Springer; 2015. [ Link ]
Sarkar IN. Biomedical informatics and translational medicine. J Transl Med. 2010 Feb 26;8:22. doi: 10.1186/1479-5876-8-22. PMID: 20187952; PMCID: PMC2837642.
Bernstam EV, Smith JW, Johnson TR. What is biomedical informatics? J Biomed Inform. 2010 Feb;43(1):104-10. doi: 10.1016/j.jbi.2009.08.006. Epub 2009 Aug 13. PMID: 19683067; PMCID: PMC2814957.
Have you completed training modules for human subjects protection, submitted an IRB application, or engaged in a Data Use Agreement? Perhaps you are familiar with the NIH Data Management and Sharing Policy or the General Data Protection Regulation (GDPR). Then you have been a partner in health research data governance.
Not so long ago the Health Insurance Portability and Accountability Act (HIPAA) of 1996 was brand new. It was not uncommon to store unencrypted data on our desktop computers, and we transferred data via “sneaker-net.” Thanks to our biomedical Informatics and information technology pioneers, we began to set boundaries and establish best practices to handle sensitive data responsibly and foster trust among patients, healthcare providers, and researchers. Ultimately, this led to more rigorous standards for ensuring that health data were made available to support research, while ensuring that patient privacy and confidentiality principles were adopted by the biomedical and health research community.
The purpose of health research data governance is to ensure the legal and ethical stewardship of protected health information (). Governance plans include the policies, processes, and standards that ensure data are collected, stored, and used responsibly while maintaining data integrity, privacy, and security. From electronic health records (EHRs) to the fitness data on our wrists, both individuals and institutions face complex challenges in balancing data access for innovation with stringent privacy protections. As data sharing across institutions and even borders increases, governance models must evolve to handle large volumes of real-world health data and adapt to rapidly changing regulations.
Abraham R, Schneider J, Vom Brocke J. Data governance: A conceptual framework, structured review, and research agenda. International journal of information management. 2019 Dec 1;49:424-38. [ ]
Solomonides A. Research Data Governance, Roles, and Infrastructure. In: Richesson RL, Andrews JE, Fultz Hollis K (editors). Clinical Research Informatics. Health Informatics. 2023. Springer, Cham. [ ]
Hallinan CM, Ward R, Hart GK, Sullivan C, Pratt N, Ng AP, Capurro D, Van Der Vegt A, Liaw ST, Daly O, Luxan BG, Bunker D, Boyle D. Seamless EMR data access: Integrated governance, digital health and the OMOP-CDM. BMJ Health Care Inform. 2024 Feb 21;31(1):e100953. doi: 10.1136/bmjhci-2023-100953. PMID: ; PMCID: PMC10882353.
Micheli M, Ponti M, Craglia M, Berti Suman A. Emerging models of data governance in the age of datafication. Big Data & Society. 2020 Aug;7(2):2053951720948087. [ ]
Mayo KR, Basford MA, Carroll RJ, Dillon M, Fullen H, Leung J, Master H, Rura S, Sulieman L, Kennedy N, Banks E, Bernick D, Gauchan A, Lichtenstein L, Mapes BM, Marginean K, Nyemba SL, Ramirez A, Rotundo C, Wolfe K, Xia W, Azuine RE, Cronin RM, Denny JC, Kho A, Lunt C, Malin B, Natarajan K, Wilkins CH, Xu H, Hripcsak G, Roden DM, Philippakis AA, Glazer D, Harris PA. The All of Us Data and Research Center: Creating a Secure, Scalable, and Sustainable Ecosystem for Biomedical Research. Annu Rev Biomed Data Sci. 2023 Aug 10;6:443-464. doi: 10.1146/annurev-biodatasci-122120-104825. PMID: ; PMCID: PMC11157478.
Suver C, Harper J, Loomba J, Saltz M, Solway J, Anzalone AJ, Walters K, Pfaff E, Walden A, McMurry J, Chute CG, Haendel M. The N3C governance ecosystem: A model socio-technical partnership for the future of collaborative analytics at scale. J Clin Transl Sci. 2023 Nov 14;7(1):e252. doi: 10.1017/cts.2023.681. PMID: ; PMCID: PMC10789985.
"The future of Artificial Intelligence in Medicine (AIM) is bright, building on the remarkable transformation in technology, computing, medicine, and biology over the past half-century." [1]
With the wealth of big data and advancements in technology has come the rapid growth of artificial intelligence (AI) in health. From its earliest days, in the 1960s and 1970s, AI promised to positively impact the many facets of health and health care [2,3]. Through the 1980s and 1990s the phenomenon of “AI Winter” was experienced as the potential for AI was seen as limited, largely due to computational capacity. Since the early 2000s, we have seen an “AI Summer” emerge, with major advances demonstrating the potential for AI to efficiently translate language, win at complex games such as chess, and engage in conversations with efficient ability to retrieve and synthesize volumes of information (e.g., OpenAI’s ChatGPT).
AI methods such as machine learning and natural language processing can be used to discover new insights for disease diagnosis, treatment, and prevention from large amounts of disparate data such as those from electronic health record (EHR) systems [4,5]. These insights can then be implemented as AI-based solutions such as clinical decision support tools in EHR systems. However, there are a range of challenges for ensuring rigorous, reproducible, and responsible development, implementation, maintenance, and use of AI in healthcare settings [6,7].
In considering the ways that AI can be used to improve health care, key stakeholders (e.g., patients and their caregivers, clinicians, care coordination managers, clinical business leadership, and researchers) need to be engaged throughout the development process, from design to implementation to evaluation [8].
See CODIAC for Health chapter on Artificial Intelligence in Health (forthcoming) for more information.
Shortliffe EH. Artificial Intelligence in Medicine: Weighing the Accomplishments, Hype, and Promise. Yearb Med Inform. 2019 Aug;28(1):257-262. doi: 10.1055/s-0039-1677891. Epub 2019 Apr 25. PMID: 31022745; PMCID: PMC6697517.
Patel VL, Shortliffe EH, Stefanelli M, Szolovits P, Berthold MR, Bellazzi R, Abu-Hanna A. The coming of age of artificial intelligence in medicine. Artif Intell Med. 2009 May;46(1):5-17. doi: 10.1016/j.artmed.2008.07.017. Epub 2008 Sep 13. PMID: 18790621; PMCID: PMC2752210.
Shortliffe EH. The adolescence of AI in medicine: will the field come of age in the '90s? Artif Intell Med. 1993 Apr;5(2):93-106. doi: 10.1016/0933-3657(93)90011-q. PMID: 8358494.
Yu KH, Beam AL, Kohane IS. Artificial intelligence in healthcare. Nat Biomed Eng. 2018 Oct;2(10):719-731. doi: 10.1038/s41551-018-0305-z. Epub 2018 Oct 10. PMID: 31015651.
Rajkomar A, Dean J, Kohane I. Machine Learning in Medicine. N Engl J Med. 2019 Apr 4;380(14):1347-1358. doi: 10.1056/NEJMra1814259. PMID: 30943338.
Shortliffe EH, Sepúlveda MJ. Clinical Decision Support in the Era of Artificial Intelligence. JAMA. 2018 Dec 4;320(21):2199-2200. doi: 10.1001/jama.2018.17163. PMID: 30398550.
Shortliffe EH. Role of evaluation throughout the life cycle of biomedical and health AI applications. BMJ Health Care Inform. 2023 Dec 11;30(1):e100925. doi: 10.1136/bmjhci-2023-100925. PMID: 38081766; PMCID: PMC10729087.
Li RC, Asch SM, Shah NH. Developing a delivery science for artificial intelligence in healthcare. NPJ Digit Med. 2020 Aug 21;3:107. doi: 10.1038/s41746-020-00318-y. PMID: 32885053; PMCID: PMC7443141.
National Academy of Medicine:Artificial Intelligence in Health Care: The Hope, the Hype, the Promise, the Peril
Yearbook of Medical Informatics: Artificial Intelligence in Health: New Opportunities, Challenges, and Practical Applications
Cohen T, Patel V, Shortliffe E, editors. Intelligence Systems in Medicine and Health: The Role of AI. Springer. 2022. [Link]
Hulin Wu et al (Ed.). Statistics and machine learning methods for EHR data: from data extraction to data analytics. CRC Press (2021); ISBN 978-0-367-44239-2. [Link]
Goldberg CB, Adams L, Blumenthal D, et al. To do no harm - and the most good - with AI in health care. Nat Med. 2024 Mar;30(3):623-627. doi: 10.1038/s41591-024-02853-7. PMID: 38388841.
Observational research with electronic health record (EHR) data is conducted to study associations, trends, and outcomes in healthcare settings. Large-scale, real-world insights may be obtained when the research is conducted though a federated network of healthcare institutions and data sources or a centralized repository containing harmonized, multi-institutional data. Compared to clinical trials, observational studies are generally less costly in terms of time, personnel, and financial resources. When rigorous research practices are applied, the potential effects of non-randomization and systemic bias can be mitigated.
See the CODIAC for Health chapter on Observational Research with EHR Data for more information.