Knowledge Discovery
One of the true powers behind the collection of EHR data is that large health datasets may contain patterns that can improve our understanding of diseases, treatments and healthcare delivery. However, collecting data itself does not automatically produce useful knowledge. Knowledge discovery in databases (KDD) is the broader process of finding insights from patterns that can be observed in data [1].
Knowledge discovery is sometimes used as a synonym for data mining, but the terms have different meanings. Data mining is described as the application of specific algorithms or computational methods for extracting patterns from data and is a specific step in the knowledge discovery process. KDD encompasses the overall process which includes selection and preparing of data, applying data mining methods, interpreting the results, and determining whether the discovered patters reepsent useful knowledge [1].
Overview of Disease Knowledge Discovery Process
ADT = Admit Discharge Transfer, CDR = Clinical Data Repository, EHR = Electronic Health Record, EDW = Enterprise Data Warehouse.
Overview of knowledge discovery using EHR data. Data from clinical and external sources are selected, preprocessed, transformed, and analyzed to identify potentially meaningful patterns. These patterns must then be interpreated and evaluated before they can contribute to knowledge. Data standards, privacy, security, information exchange, legislation all affect this process [2].
The Knowledge Discovery Process [2]
The following section is adapted from "Mining the electronic health record for disease knowledge"
Understand the Problem
The process begins by identifying a potential area of interest to explore. Researchers must determine what they hope to learn, what prior knowledge already exists, and how the resulting information might be used.
Data Selection
Researchers identify a target dataset by creating a cohort definition which includes exclusion/inclusion criteria and then extracting that data. Criteria that can be used for cohort identification are patient demographics, encounters, diagnoses, procedures, medications, and laboratory results.
EHR-related data can come from:
Electronic health records
Clinical data repositories
Enterprise data warehouses
Laboratory and pharmacy systems
Billing and claims data
Patient registries
Biobanks
Biomedical literature
The selected data may contain structured, unstructured and semi structured data types.
Data Extraction
Once the cohort has been identified, researchers determine which data are needed, what information is available, and how the extracted dataset will be obtained and securely stored.
The formal data request should specify the type of dataset requested (aggregate, deidentified, or identified) as well as the required data types, elements, and restrictions. Restrictions may include limiting records to a defined period, including only active problem-list entries, or retrieving only the most recent clinical note. The sensitivity and identification level of the resulting dataset determine the computing environment in which it must be stored and analyzed.
These sources provide different perspectives on patient health. Billing diagnoses may associate multiple clinical codes with an encounter, while problem-list entries may include a diagnosis name, code, status, and onset date. Clinical notes contain unstructured information accompanied by metadata such as the note type and author. Structured and unstructured data may be analyzed separately or combined to provide a more complete representation of the study cohort.
After extraction, researchers should verify that the delivered data elements, date ranges, restrictions, and cohort records match the approved request [2].
Preprocess the Data
Once the dataset is obtained, preprocessing prepares it for analysis. Major tasks may include deidentification, data cleaning, standardization, integration, and enrichment.
Deidentification: Remove protected health information when required for the approved research use and computing environment.
Data cleaning: Identify and address missing, invalid, inconsistent, duplicate, or implausible values. Decisions to retain, correct, or remove these values should depend on the research question and protocol.
Standardization and integration: Convert data into consistent formats and reconcile differences among sources. This may include standardizing dates, separating concatenated values, and mapping local or source codes to standardized terminologies.
Data enrichment: Add useful information through external resources or computational methods. For example, natural language processing may extract and encode clinical concepts from notes, while resources such as the UMLS may help connect equivalent concepts across different vocabularies.
These steps improve the consistency and usability of the dataset while helping researchers combine structured and unstructured information from multiple clinical sources [2].
Transform the Data
Apply Data-Mining Methods
Interpret and Evaluate the Patterns
Knowledge to Wisdom
References
Fayyad U, Piatetsky-Shapiro G, Smyth P. From data mining to knowledge discovery in databases. AI Magazine. 1996;17(3):37–54.
Chen ES, Sarkar IN. Mining the electronic health record for disease knowledge. In: Kumar V, Tipney H, editors. Biomedical Literature Mining. Methods in Molecular Biology. Vol. 1159. Humana Press; 2014:269–286.
Last updated