> For the complete documentation index, see [llms.txt](https://docs.bcbi.brown.edu/codiac-for-health/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.bcbi.brown.edu/codiac-for-health/observational-research/research-pipeline-1.md). # Research Pipeline Below is an example process for conducting research with [**Health Data**](https://docs.bcbi.brown.edu/codiac-for-health/foundations/health-data), such as electronic health record (EHR) data. This list is not exhaustive, and the exact process will vary depending on the research question, data source, institution, and regulatory requirements. ### Conduct a literature review A literature review summarizes and synthesizes research on a topic of interest. Reviewing existing research can help refine the research question and identify gaps in the literature \[1]. Search relevant databases, journals, and books using keywords related to the topic. A literature review should address questions such as: * What research has already been conducted on the topic? * Who are the key researchers and experts in the field? * What theories and methodologies are commonly used? * What challenges, controversies, or contradictions exist? * Are there gaps in the literature that the proposed research could address? * Which datasets, variables, clinical definitions, and analytical methods have previous researchers used? ### Explicitly describe the research question A clearly defined research question guides the study design, data selection, cohort definition, and analysis. Intervention-focused research questions can often be structured using the PICO or PICOT framework: * Population: Who or what is being studied, such as individuals, patient populations, or communities? * Intervention: What treatment, exposure, program, or other factor is being studied? * Comparison: What alternative is the intervention being compared with, such as standard care, another intervention, or no intervention? * Outcome: What effect, event, or result will be measured? * Time: Over what period will the outcome be studied or measured? \[2] PICO and PICOT are especially useful for intervention-focused research questions. Other frameworks may be more appropriate for descriptive, qualitative, diagnostic, prognostic, or predictive health-data studies. ### Form an interdisciplinary team An interdisciplinary team brings together individuals with complementary expertise to guide and perform each stage of the study. Working with an interdisciplinary team: * Incorporates different perspectives into the research process. * Provides access to mentorship when challenges or roadblocks arise. * Increases efficiency by distributing responsibilities according to expertise. * Brings together the diverse skills required to address complex research problems. * Helps ensure that the research question, data, methods, and results are interpreted appropriately. ### Fully specify the research protocol *in advance* of executing the study A research protocol describes why and how a study will be conducted. Developing the protocol before executing the study helps the research team follow a consistent methodology and improves the transparency and reproducibility of the analysis. For a health-data study, the protocol may specify: * Background, rationale, and research objectives * Study design and setting * Data sources and study period * Study population and eligibility criteria * Exposure, intervention, and comparison groups, when applicable * Primary and secondary outcomes * Variables, clinical concepts, and code sets * Data extraction and management procedures * Planned data-quality assessments * Methods for handling missing or implausible values * Statistical analysis plan * Methods for addressing bias and confounding * Data privacy and ethical considerations * Plans for reporting and disseminating results \[3] The protocol should provide enough methodological detail for another researcher to understand and, where possible, reproduce the study. Any deviations from the original protocol should be documented and explained. ### Apply for IRB approval of the study Research using identifiable patient data may require Institutional Review Board (IRB) review before the study begins. Studies using synthetic, deidentified, or anonymized data may not require IRB approval, but requirements depend on the data, how they were obtained, the study activities, and institutional policies. Researchers should consult their institution’s Human Research Protection Program or IRB office rather than making this determination themselves \[4] ### Apply for an Institutional Reliance Agreement, if necessary Multisite, nonexempt human-subjects research may be required to rely on a single IRB to provide regulatory review for participating institutions. Collaborative research often has a lead site that receives the primary grant or contract and establishes subawards or subcontracts with participating sites. When institutions rely on a single IRB, an IRB reliance agreement documents which institution will serve as the reviewing IRB and which institutions will rely on its oversight. Researchers should consult their institution’s Human Research Protection Program to determine whether a reliance agreement is required \[5] ### Execute a Data (Transfer and) Use Agreement (DUA, DTUA), as required A Data Use Agreement (DUA), sometimes called a Data Transfer and Use Agreement (DTUA), is a contract between two or more parties that specifies how data may be used and protected. A DUA legally binds the researchers and institutions involved to ensure the appropriate use and protection of the data. Brown classifies its information assets into four risk-based categories: None, Level 1, Level 2, and Level 3. DUAs provide information about: * Limitations on the use of the data * Obligations to safeguard the data * Liability for harm resulting from the use of the data * Intellectual-property and publication expectations * Privacy rights associated with the transfer of confidential or protected data \[6] ### Comply with any application and approval procedures set forth by the data provider Data providers may require researchers to complete project applications, training, data-use agreements, security reviews, or other approval procedures before granting access. Researchers must understand and comply with the specific requirements established by each provider. ### Request access to or set up computing infrastructure, as necessary Use an approved computing environment with sufficient storage, processing capacity, access controls, and software for the proposed analysis. ### Assess the suitability (strengths and weaknesses) of the dataset(s) to be used in the study Determine whether the dataset is appropriate for answering the research question. Consider whether it contains the necessary population, variables, clinical detail, sample size, time period, and length of follow-up. Researchers should also identify the dataset’s strengths and weaknesses and determine whether its findings are likely to apply to the population and setting of interest \[7]. ### Assess the quality of the dataset(s) Evaluate the completeness and accuracy of the variables needed for the study. Consider missing data, measurement error, misclassification, incorrect dates, and differences in how information was recorded across sources or over time. Data-quality limitations should be documented and considered when designing the analysis and interpreting the results \[7]. ### Define the study cohort (and matching cases, if applicable) “The first step in transforming a clinical question into research is to carefully define the **study sample** or **patient cohort**, the **exposure** of interest, and the **outcome** of interest.” \[8] A **cohort** is a group of people who satisfy one or more inclusion criteria for a defined period. An EHR phenotype provides the computable criteria used to identify patients with the relevant characteristics. A cohort definition: * Uses code sets containing relevant clinical codes, such as ICD-9-CM, ICD-10-CM, NDC, and HCPCS codes. * Specifies the logic for applying those codes, such as whether cohort entry requires the first recorded occurrence, any occurrence, or multiple occurrences of a code. * Defines how and when a patient enters and exits the cohort. \[9] ### Create standard code sets for each clinical concept in the cohort definition and every independent and dependent variable. 1. Compose a computable data request / data extraction specification. 2. Clean and stage extracted data for analysis; handle missing values according to protocol. 3. Characterize the study cohort (and matching cases, if applicable). 4. Adjust for any bias or confounders in the data. 5. Analyze the data according to protocol. 6. Produce research products. 7. Comply with any review procedures required by the data provider. 8. Publish your work! ## References ## Resources ### Books * [**Secondary Analysis of Electronic Health Records**](https://www.ncbi.nlm.nih.gov/books/NBK543630/) \[Internet]. Cham (CH): Springer; 2016. doi: 10.1007/978-3-319-43742-2 * Ed. Hulin Wu et al. **Statistics and machine learning methods for EHR data: from data extraction to data analytics**. CRC Press 2021; ISBN 978-0-367-44239-2 * O’Neil ST, Beasley W, Loomba J, Patrick S, Wilkins KJ, Crowley KM., Anzalone, AJ (Eds.) (2023). [**The Researcher’s Guide to N3C: A National Resource for Analyzing Real-World Health Data**](https://zenodo.org/record/7749367#.ZEgPD3bMJmM). DOI: 10.5281/zenodo.7749367 * [**The Book of OHDSI**](https://ohdsi.github.io/TheBookOfOhdsi/) ### Articles * Blewett LA, Call KT, Turner J, Hest R. [**Data Resources for Conducting Health Services and Policy Research**](https://pmc.ncbi.nlm.nih.gov/articles/PMC5880724/). Annu Rev Public Health. 2018 Apr 1;39:437-452. doi: 10.1146/annurev-publhealth-040617-013544. Epub 2017 Dec 22. PMID: 29272166; PMCID: PMC5880724. * Sayers EW, Beck J, Bolton EE, et al. [**Database resources of the National Center for Biotechnology Information**](https://pmc.ncbi.nlm.nih.gov/articles/PMC7778943/). Nucleic Acids Res. 2021 Jan 8;49(D1):D10-D17. doi: 10.1093/nar/gkaa892. PMID: 33095870; PMCID: PMC7778943. * Shang N, Weng C, Hripcsak G. [**A conceptual framework for evaluating data suitability for observational studies**](https://pubmed.ncbi.nlm.nih.gov/29024976/)**.** J Am Med Inform Assoc. 2018 Mar 1;25(3):248-258. doi: 10.1093/jamia/ocx095. PMID: 29024976; PMCID: PMC7378879. * Weber GM, Mandl KD, Kohane IS. [**Finding the Missing Link for Big Biomedical Data**](https://jamanetwork.com/journals/jama/article-abstract/1883026). *JAMA.* 2014;311(24):2479–2480. doi:10.1001/jama.2014.4228 --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.bcbi.brown.edu/codiac-for-health/observational-research/research-pipeline-1.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.