1 of 40

Computing Skills

Introduction

The chapter provides instructions and examples of using computing skills for health data and technology research.

Visit other chapters in CODIAC for Health using the or menu in the upper left corner.

File Directory Structures

All major operating systems organize files into hierarchical directories. Understanding these file directory structures is vital when interacting with data files using Unix commands or a programming language.

This page describes file directory structures generally as well as some of the differences between file directory structures within different operating systems.

Hierarchical Structure

Directories allow users to group files into an organized structure. They are typically visualized like root systems of trees, the highest level of which is called the "root directory". Subdirectories branch down from the root directory, containing files as well as additional subdirectories.

Directories and files are typically described using the path used to reach them through the directory structure, starting with the root directory. In Linux and Mac operating systems, the root directory is indicated as "/" (In Windows OS, the root directory is indicated as "\"). An additional "/" (or "\" for Windows OS) is placed between each object in the path.

For example, looking at Figure 1, File_B1a2 could be described with:

/Directory_B/Directory_B1/Directory_B1a/File_B1a2

GUI

All major operating systems also provide users with a graphical user interface, or GUI (often pronounced "gooey"), which allows interaction with software and files through visual icons. If you are not already familiar with accessing files and directories through the command line, you are likely familiar with using a GUI file system. While not the recommended method for interacting with files while programming, the GUI file system can be a useful tool for visualizing a directory structure.

Figure 2 displays the GUI file system for a computer running MacOS. Though the GUI directory structure is visualized horizontally, the "root system" is still clearly visible. Using its complete path, the file "medication_data" should be described as:

/Users/<username>/Documents/project_a/data_files/medication_data

Resources

Text Editors

Programming languages are written using text editor applications. These applications allow users to create and edit free text, which can then be run as programs. Text editors differ in complexity, some including extra functionality for easier, more efficient programming. Text editors with auto-complete suggest common functions or existing variables as the programmer begins to type, which the programmer can then select without needing to finish typing. Some text editors offer options to run individual lines of code or entire programs while editing files.

Popular Text Editors

Microsoft Visual Studio Code (recommended)

Available for Mac, Windows, and Linux operating systems
Includes support for debugging, syntax highlighting, auto-complete, and additional user-friendly functionality
Download Microsoft Visual Studio Code

Jupyter Notebooks

Web application text editor, no download necessary
Includes options for interactive output (HTML, images, videos, LaTeX, and custom MIME types), support for big data tools, such as Apache Spark, and options for sharing notebooks with others
Run individual lines of code or entire programs at once
Get started with Jupyter Notebooks

VI/VIM

Highly configurable
Included in most UNIX operating systems (e.g., Linux, or MacOS), no download necessary
Write files from the Terminal
Interactive VIM Tutorial

Emacs

Highly configurable
Included in most UNIX operating systems (e.g., Linux, or MacOS), no download necessary, also available for Windows
Wide range of built-in features for text editing, such as syntax highlighting, automatic indentation, and search and replace
Learn more about Emacs

Pico

Included in most UNIX operating systems (e.g., Linux, or MacOS), no download necessary
Most of the editing commands are displayed at the bottom of the editing screen for easy reference
Learn more about Pico

GitHub

GitHub is a code hosting platform that allows developers to create, store, manage, and share their code. It uses Git software, providing the distributed version control of Git plus access control, bug tracking, software feature requests, task management, continuous integration, and wikis for every project. Refer to GitHub Docs for additional GitHub documentation and tutorials.

Saving and Retrieving Code Changes

Like other cloud platforms (e.g., Google Docs), GitHub allows users to work on projects together. Please note, code changes must be manually saved. GitHub does not automatically save your work. To save changes, open the Terminal application, navigate to the cloned repository, and run the following commands, replacing "INSERT PROGRESS NOTE" with brief description of changes.

git add -A
git commit -"INSERT PROGRESS NOTE"
git push

git add -A : adds all your code changes to the GitHub repository
git commit -m"INSERT PROGRESS NOTE" : adds a note to the commit which you and your team can reference later. This note should be brief and informative, describing the purpose of your code changes.
git push : saves your code changes to the GitHub repository.

If multiple users are pushing code changes to your GitHub repository, make sure to retrieve or "pull" these edits before you begin making code changes. To do so, open the Terminal application, navigate to the cloned repository, and run the following command. If you have made any code changes, you will need to save them first for the pull to work.

git pull

When your are making code changes, you should git pull before making any edits. This will keep your team from encountering "merge conflicts", which can become difficult to troubleshoot. To mitigate merge conflicts, make sure to communicate with your team. Inform your team whenever you push new code changes so that everyone is always working one the most updated version of the code.

Resolving Merge Conflicts

Merge conflicts happen when you attempt to merge code branches that have competing commits. They are often caused by users making code changes without pulling first. To resolve a merge conflict, work through the following steps:

Identify the location of the merge conflict.
Manually edit the conflicted file from a single machine, selecting the changes you want to keep in the final merge.
Push the selected changes to GitHub.

All team members should pull the corrected changes from GitHub before continuing to make code changes.

Resources

Unix

Unix is a family of operating systems officially trademarked as UNIX®. These operating systems are computing environments that are optimized for multi-tasking across multiple users. The original system was developed by AT&T in 1969 as a text only system. There are many Unix variants or Unix-like systems (e.g. GNU/Linux, Sun Solaris, IBM AIX, and Mac OS X). On Windows, Cygwin is a program that provides a Unix-like environment.

The main components of a Unix operating system include:

Kernel – bridge between hardware (i.e. silicon) and application (i.e. software)
Shell – command line interface to enable user interaction with the system
File System – the organization structure for how files are stored

Unix File System

The Unix file system organizes files and directories into a hierarchical structure like the root system of a tree.

The "root" directory (e.g. "/") is the top of the hierarchy.
- Standard directories within the root directory:
  - /bin and /usr contain commands needed by system administrators and users
  - /etc contains system-wide configuration files and system databases
  - /home contains the home directory (~) for each user (In some systems, the home directories may be in a different location such as /users or /Users)
When traversing directories
- working directory (.) is the directory that a user currently is in
- parent directory (..) is the directory above the working directory
- path or pathname specifies where a user is in the file system
- full path or absolute path points to the same location regardless of the working directory (i.e., it is written in reference to the root directory)
- relative path is the path relative to the working directory

If the working directory is the home directory for bcbi, the full path for the course directory is /home/bcbi/course while the relative path is just course. A schematic of this is below:

If code then becomes the working directory, the full path for the data directory from there is /home/bcbi/course/data while the relative path is ../data. A schematic of this is below:

Understanding Directory/File Ownership and Permissions

Unix Shell

The Unix shell provides a command line interface for interacting with the operating system and is where commands are entered. An example below is a Mac OS X Terminal Shell logged into a RedHat Linux Server as user_name.

The prompt may look different depending on your shell (e.g., Bourne shell [sh], C shell [csh], or Bourne-Again shell [bash])
Default prompts include $ and %
The prompt # typically appears when logged in as the superuser or root user who can do anything on the system, so should be restricted to trusted users, used only when necessary and with caution. While you may be able to do this on a system you control, you are unlikely to ever have root priviledges on a shared computing resource (e.g. Oscar or Stronghold at Brown University)
The prompt can be configured to include additional information such as hostname, username, and pathname (e.g., computer:/home/bcbi/course bcbi $).

Unix Commands

There are many Unix commands. Some commands will display output and then return to the shell prompt while others will just return to the shell prompt to indicate that it has executed the last command.

Unix command syntax:

Case-sensitive (pwd ≠ PWD)
May involve one or more arguments
Argument may be an option (or flag or switch) for that command
Argument may be a file or directory

To get to a Unix shell on your computer:

For Mac, launch the Terminal application (under Applications → Utilities → Terminal)
For Linux, launch the Terminal application
For Windows, launch the PowerShell application

Get help from manual (man) pages on commands: (Use spacebar or up and down arrows to scroll through pages and then press q to quit)

Determine what directory you are currently in with pwd (present working directory):

Get a listing of current directory contents using ls:

Create course directory using mkdir: (Replace course with class name - e.g., methods2020 or biol6535)

Get a listing of current directory contents with details using ls:

Change into course directory using cd: (Replace course with class name - e.g., methods2020 or biol6535)

Working with Data

System Information

Analyzing and Searching Files

Common Shortcuts

File Content Browsing

File Permissions

Exercises

Analyze the MIMIC-IV Demo Files Using Unix Commands - Forthcoming!
Analyze the SyntheticRI Demo Files Using Unix - Forthcoming!

Resources

Julia

Julia is an open source dynamic programming language for high-level, high-performance numerical computing [1]. Julia provides ease and expressiveness (similar to R, MATLAB, and Python), but also supports general programming [2].

Development of Julia began in 2009, and the first version was released in February 2012. The current version of Julia is 1.11 (as of November 2024).

References

Resources

Julia Performance Metrics
ThinkJulia: How to Think Like a Computer Scientist
Julia Data Science
Learn X in Y Minutes: X=Julia
Julia Cheat Sheet
Introducing Julia Wikibook
JuliaHealth

Installation

Instructions for installing Julia on macOS and Windows operating systems can be found .

Package managers such as (macOS and Linux) and (Windows) can be used to facilitate installation.

For most users, it is recommended to download the current stable release from .

Some developers might wish to use a different version, or to switch between versions. For this, the can be useful.

Julia is also available for use in Brown's :

Oscar (for high-performance computing)
Stronghold (for secure computing)

Resources

REPL

Julia comes with a full-featured interactive command-line REPL (read-eval-print loop) built into the julia executable. In addition to allowing quick and easy evaluation of Julia statements, it has a searchable history, tab-completion, many helpful keybindings, and dedicated help ? and shell modes ;. [1]

This page provides examples of using REPL on the command line.

Julia REPL Example (local)

Type julia in terminal to launch REPL

Julia REPL Help Pages (local)

Type "?" to enter help pages within REPL
Type a function from Julia to read help pages (ex: println)

References

Julia Contributors. (n.d.). REPL - Standard Library - Julia Language. Retrieved May 1, 2024, from https://docs.julialang.org/en/v1/stdlib/REPL/

Resources

Basic Syntax

"Hello, World!" Program

This is the typical first program for those new to a general purpose programming language like Julia. It can be used to test that the of Julia is working and also introduce Julia's basic syntax using the environment or running code written using a at the command line.

Input:

Output:

Here are variations of the "Hello, World!" programming using variables and different print statements.

Input:

Output:

Variable Assignment

In order to assign variables in Julia, you write the desired name for your variable, an = sign, and what the value of the variable should be.

Input:

Output:

Comments

We can write comments on our code, which do not run, to describe what certain lines of code or section of code do
- These comments are just for the programmer, they will not appear anywhere in the output and just are there to explain what the code is doing or to provide helpful notes
- To make a comment in Julia, you can use the “#” symbol and then type your comment
Sometimes you might want to write longer comments that span multiple lines – to do this you can surround these comments with #= above the start as well as =# below the end

Input:

Output:

Print Statements

Without using a print statement, Julia will only print out the most recent item that has an output. In order to print multiple things, we can use the print() or println() functions.

Input:

Output:

Exercises

Use Julia in Brown Oscar Computing Environment - Forthcoming!
Use Julia in Brown Stronghold Computing Environment - Forthcoming!

Resources

Numbers and Math

This page provides syntax for using numbers and mathematic operations in Julia. Each section includes an example to demonstrate the described syntax and operations.

Types of Numbers

Integer (positive and negative counting number) - e.g., -3, -2, -1, 0, 1, 2, and 3
- Signed: Int8, Int16, Int32, Int64, and Int128
- Unsigned: UInt8, UInt16, UInt32, UInt64, and UInt128
- Boolean: Bool (0 = False and 1 = True)
Float (real or floating point numbers) - e.g., -2.14, 0.0, and 3.777
- Float16, Float32, Float64

Use typeof() function to determine type

Input:

Output:

Arithmetic Operators

Input:

Output:

Comparison Operators and Functions

Input:

Output:

Exercises

Create a Health Calculator Using Julia - Forthcoming!

Resources

Strings and Characters

This page provides syntax for strings and characters in Julia as well as some of their associated functions. Each section includes an example to demonstrate the described syntax or function.

Characters and Strings

Char is a single character
String is a sequence of one or more characters (index values start at 1)

Some functions that can be performed on strings

Action

Function

Use typeof() function to determine type

Input:

Output:

Resources

Regular Expressions

Regular expressions (regex) are powerful tools for pattern matching and text processing. They are represented as a pattern that consists of a special set of characters to search for in a string str.

This page provides syntax for regular expressions in Julia . Each section includes an example to demonstrate the described methods.

Functions

Action

Function

Character Class

Character class specifies a list of characters to match ([...] where ... represents the list) or not match ([^...])

Anchors

Anchors are special characters that can be used to match a pattern at a specified position

Repetition and Quantifier Characters

Repetition or quantifier characters specify the number of times to match a particular character or set of characters

Input:

Output:

Resources

Control Flow

In computer science, control flow (or flow of control) is the order in which individual statements, instructions or function calls of an imperative program are executed or evaluated.

This page provides syntax for some of the common control flow methods in Julia . Each section includes an example to demonstrate the described methods.

Use Cases and Syntax

Test if a specified expression is true or false
Short-circuit evaluation
- Test if all of the conditions are true x && y
- Test if any of the conditions are true x || y
- Test if a condition is not true !z
Conditional evaluation
- if statement
- if-else
- if-elseif-else
- ?: (ternary operator)

Conditional Statements

Input:

Output:

Loops

Repeat a block of code a specified number of times or until some condition is met.
while loop
for loop
Use break to terminate loop

Input:

Output:

Comparison Operators and Functions

Input:

Output:

References

Resources

Collections and Data Structures

In computer programming, a collection is a grouping of some variable number of data items (possibly zero) that have some shared significance to the problem being solved and need to be operated upon together in some controlled fashion. [1]

This page provides syntax for different types of collections and data structures in Julia (arrays, sets, dictionaries, etc.). Each section includes an example to demonstrate the described methods.

Arrays

Arrays are ordered collection of elements. In Julia they are automatically indexed (consecutively numbered) by an integer starting with 1.

Creating arrays

Action

Syntax

Creating array from string

Action

Syntax

Accessing elements

Action

Syntax

Adding and removing elements

Sort and unique

Compare arrays

Convert array to string

Input:

# arrays.jl

day_array = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday"]
day = "Thursday"

array_length = length(day_array)
array_first_day = day_array[1]
array_last_day = day_array[end]

println("Length of array: $array_length")
println("First day of week: $array_first_day")
println("Third day of week: $(day_array[3])")
println("Last day of week: $array_last_day")

println("$day is in $day_array: $(in(day, day_array))")

# add Sunday to beginning and Saturday to end
pushfirst!(day_array, "Sunday")
push!(day_array, "Saturday")

# print each element of array
println("Day of week: ")
for i in 1:length(day_array)
    println("  $(day_array[i])")
end

println("Day of the week: $(join(collect(day_array), ";"))")

# sort the array and print again
sort!(day_array)
println("Day of the week (sorted): $(join(collect(day_array), ";"))")

Output:

Length of array: 5
First day of week: Monday
Third day of week: Wednesday
Last day of week: Friday
Thursday is in ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday"]: true
Day of week: 
  Sunday
  Monday
  Tuesday
  Wednesday
  Thursday
  Friday
  Saturday
Day of the week: Sunday;Monday;Tuesday;Wednesday;Thursday;Friday;Saturday
Day of the week (sorted): Friday;Monday;Saturday;Sunday;Thursday;Tuesday;Wednesday

Sets

Sets are an unordered collection of unique elements.

Creating sets

Interacting with sets

Comparing sets

Input:

# sets.jl

color_set = Set(["red", "yellow", "blue"])
color_set2 = Set(["red", "orange", "yellow"])

println("Length	of set:	$(length(color_set))")

println("Color Set 1")
for color in color_set
    println("  $(color)")
end

println("Color Set 2: $(join(collect(color_set2), "---"))")

println("Intersection: $(intersect(color_set, color_set2))")
println("Union: $(union(color_set, color_set2))")
println("Difference: $(setdiff(color_set, color_set2))")
println("Difference: $(setdiff(color_set2, color_set))")

Output:

Length	of set:	3
Color Set 1
  yellow
  blue
  red
Color Set 2: yellow---orange---red
Intersection: Set(["yellow", "red"])
Union: Set(["yellow", "orange", "blue", "red"])
Difference: Set(["blue"])
Difference: Set(["orange"])

Dictionaries

Dictionaries are unordered collection of key-value pairs where the key serves as the index (“associative collection”). Similar to elements of a set, keys are always unique.

Creating dictionaries

Accessing dictionaries

Converting dictionaries

Sorting dictionaries

Input:

# dicts.jl

day_dict = Dict()
day_length_dict = Dict()

day_dict["Mon"] = "Monday"
day_dict["Tue"] = "Tuesday"
day_dict["Wed"] = "Wednesday"
day_dict["Thu"] = "Thursday"
day_dict["Fri"] = "Friday"

if haskey(day_dict, "Wed")
   println("$(day_dict["Wed"])")
end

if !haskey(day_dict, "Sat")
   println("no key \"Sat\"")
end

println("print key-value pairs")
for day in keys(day_dict)
    println("  $day = $(day_dict[day])")
end

println("print values (sorted)")
for day_value in sort(collect(values(day_dict)))
    println("  $day_value")
end

# get length of each value and keep track of lengths
for day_value in values(day_dict)
    day_length = length(day_value)
    day_length_dict[day_value] = day_length
end

println("print lengths")
for day in keys(day_length_dict)
    println("  $day = $(day_length_dict[day])")
end

println("print lengths in descending order")
for (day, length) in sort(collect(zip(values(day_length_dict), keys(day_length_dict))), rev=true)
    println("  $day = $length")
end

println("print lengths in ascending order")
for (day, length) in sort(collect(zip(values(day_length_dict), keys(day_length_dict))), rev=false)
    println("  $day = $length")
end

Output:

Wednesday
no key "Sat"
print key-value pairs
  Wed = Wednesday
  Tue = Tuesday
  Thu = Thursday
  Mon = Monday
  Fri = Friday
print values (sorted)
  Friday
  Monday
  Thursday
  Tuesday
  Wednesday
print lengths
  Friday = 6
  Tuesday = 7
  Thursday = 8
  Wednesday = 9
  Monday = 6
print lengths in descending order
  9 = Wednesday
  8 = Thursday
  7 = Tuesday
  6 = Monday
  6 = Friday
print lengths in ascending order
  6 = Friday
  6 = Monday
  7 = Tuesday
  8 = Thursday
  9 = Wednesday

References

Wikipedia contributors (n.d.). Collection. In Wikipedia. Retrieved May 1, 2024, from https://en.wikipedia.org/wiki/Collection_(abstract_data_type)

Resources

Julia Documentation: Base - Collections and Data Structures
Think Julia: Chapter 10 - Arrays
Think Julia: Chapter 11 - Dictionaries
Think Julia: Chapter 12 - Tuples

File Input/Output

Many Julia programs involve the input and output of files. When analyzing a dataset, that dataset file will need to be pulled into your program (input). If you want to see the results of your analysis, your program will need an output.

This section provides the syntax for inputing files (reading) and outputting results (writing) use base Julia (i.e., no packages such as CSV.jl).

UC Irvine Machine Learning Repository: Adult Data Set

Tabulate and report counts for sex in Adult Data Set from the UC Irvine Machine Learning Repository.

Dataset (example lines from adult.data)

39, State-gov, 77516, Bachelors, 13, Never-married, Adm-clerical, Not-in-family, White, Male, 2174, 0, 40, United-States, <=50K
50, Self-emp-not-inc, 83311, Bachelors, 13, Married-civ-spouse, Exec-managerial, Husband, White, Male, 0, 0, 13, United-States, <=50K
38, Private, 215646, HS-grad, 9, Divorced, Handlers-cleaners, Not-in-family, White, Male, 0, 0, 40, United-States, <=50K
53, Private, 234721, 11th, 7, Married-civ-spouse, Handlers-cleaners, Husband, Black, Male, 0, 0, 40, United-States, <=50K
28, Private, 338409, Bachelors, 13, Married-civ-spouse, Prof-specialty, Wife, Black, Female, 0, 0, 40, Cuba, <=50K

Input (process_file.jl)

# process_file.jl
# Tabulate and report counts for sex in Adult Data Set
# https://archive.ics.uci.edu/ml/datasets/adult

# relative path of file
data_file = open("_data/adult/adult.data", "r")

# absolute path of file
# data_file = open("/Users/user/data/adult/adult.data", "r")

# initialize collection (dictionary for tabulating counts)
gender_dict = Dict()

# read each line, extract sex, and keep track of counts
for line in readlines(data_file)

  # skip empty lines
  if isempty(line)
      continue
   end

  # split line into array, based on delimiter (comma and space)
  line_array = split(line, ", ")

  # tabulate the counts for gender
  gender = line_array[10]
  if haskey(gender_dict, gender)
    gender_dict[gender] += 1
  else
    gender_dict[gender] = 1
  end
end

# report total counts
println("Sort by key (alphabetical):")
for gender in keys(gender_dict)
  println("  $gender = $(gender_dict[gender])")
end

# report total counts by key, in reverse order
println("Sort by key (reverse alphabetical):")
for gender in sort(collect(keys(gender_dict)), rev=true)
  println("  $gender = $(gender_dict[gender])")
end

# report total counts by value, in reverse order (send output to file)
output_file = open("process_file_output.txt", "w")
println("Sort by value (reverse numerical):")
for (count, gender) in sort(collect(zip(values(gender_dict),keys(gender_dict))), rev=true)
  println("  $gender = $(gender_dict[gender])")
  write(output_file, "$gender = $count\n")
end

Output

Sort by key (alphabetical):
  Female = 10771
  Male = 21790
Sort by key (reverse alphabetical):
  Male = 21790
  Female = 10771
Sort by value (reverse numerical):
  Male = 21790
  Female = 10771

Terminal

$ julia process_file.jl
Sort by key (alphabetical):
  Female = 10771
  Male = 21790
Sort by key (reverse alphabetical):
  Male = 21790
  Female = 10771
Sort by value (reverse numerical):
  Male = 21790
  Female = 10771

$ ls -1
process_file.jl
process_file_output.txt

$ more process_file_output.txt
Male = 21790
Female = 10771

Exercises

Analyze the MIMIC-IV Demo Files Using Julia - Forthcoming!
Analyze the SyntheticRI Demo Files Using Julia - Forthcoming!

Resources

Julia Documentation: Base - I/O and Network
Think Julia: Chapter 14 - Files

Packages

In computer programming, a package is a collection of modules or programs that are often published as tools for a range of common use cases, such as text processing and doing math. Programmers can install these packages and take advantage of their functionality within their own code.

This page provides instructions for installing, using, and troubleshooting packages in Julia.

Installing Packages

Start Julia REPL by typing the following in Terminal or PowerShell (Note: do not need to type $ - this is to indicate the shell prompt)

$ julia

Go into REPL mode for Pkg, Julia’s built in package manager, by pressing ]

$ julia ]

$ (@v1.4) pkg>

Update package repository in Pkg REPL

$ (@v1.4) pkg> update

Add packages in Pkg REPL

$ (@v1.4) pkg> add CSV

$ (@v1.4) pkg> add DataFrames

Check installation

(@v1.4) pkg> status
            Status `~/.julia/environments/v1.0/Project.toml`
                [336ed68f] CSV v0.4.3
                [a93c6f00] DataFrames v0.17.1
                ...

Get back to the Julia REPL and exit by pressing backspace or ^C.

(@v1.4) pkg>

julia>

To see REPL history

$ more ~/.julia/logs/repl_history.jl

Using Packages

julia> using CSV
julia> using DataFrames

julia> exit()

Troubleshooting

If you get an error like: ERROR: SystemError: opening file "C:\\Users\\User\\.julia\\registries\\General\\Registry.toml": No such file or directory
- Delete C:\\Users\\User\\.julia\\registries where User is your computer’s username and try again
- https://discourse.julialang.org/t/registry-toml-missing/24152

Resources

Julia Pkg
Julia Package Registries
JuliaHealth and BioJulia organizations (focused on Julia packages for health and life sciences)
Julia Package: CSV.jl
Julia Package: DataFrames.jl

DataFrames

DataFrames.jl is a Julia package that provides a set of tools for working with tabular data in Julia. Its design and functionality are similar to those of pandas (in Python) and data.frame, data.table and dplyr (in R), making it a great general purpose data science tool. [1]

This page provides examples of using DataFrames.jl, demonstrating the syntax and common functions within the package.

Example

Install and Load DataFrames.jl Package

using Pkg

# Add DataFrames package
Pkg.add("DataFrames")

# Load paackages
using DataFrames

Create Dataframe

# Create dataframe
df = DataFrame(id = 1:5, gender = ["F", "M", "F", "M", "F"], age = [68, 54, 49, 28, 36])

Display Dataframe

Input:

# display dataframe
println(df)

Output:

5×3 DataFrame
 Row │ id     gender  age
     │ Int64  String  Int64
─────┼──────────────────────
   1 │     1  F          68
   2 │     2  M          54
   3 │     3  F          49
   4 │     4  M          28
   5 │     5  F          36

First two lines of dataframe:

Input:

println(first(df, 2))

Output:

2×3 DataFrame
 Row │ id     gender  age
     │ Int64  String  Int64
─────┼──────────────────────
   1 │     1  F          68
   2 │     2  M          54

Last two lines of dataframe:

Input:

println(last(df, 2))

Output:

2×3 DataFrame
 Row │ id     gender  age
     │ Int64  String  Int64
─────┼──────────────────────
   1 │     4  M          28
   2 │     5  F          36

Describe Dataframe

Dataframe size:

Input:

# dataframe size
println(size(df))

Output:

(5, 3)

Dataframe column names:

Input:

# dataframe column names
println(names(df))

Output:

["id", "gender", "age"]

Dataframe description:

Input:

# describe dataframe
println(describe(df))

Output:

3×7 DataFrame
 Row │ variable  mean    min  median  max  nmissing  eltype
     │ Symbol    Union…  Any  Union…  Any  Int64     DataType
─────┼────────────────────────────────────────────────────────
   1 │ id        3.0     1    3.0     5           0  Int64
   2 │ gender            F            M           0  String
   3 │ age       47.0    28   49.0    68          0  Int64

Accessing DataFrames

Get "age" column (different ways to call the column)

Input:

# call by column name
println(df[!, :age])

# get column by column number
println(df[!, 3])

# alternate syntax
println(df.age)

Output:

[68, 54, 49, 28, 36]
[68, 54, 49, 28, 36]
[68, 54, 49, 28, 36]

Get row

Input:

# print row 2
println(df[2, :])

Output:

DataFrameRow
 Row │ id     gender  age
     │ Int64  String  Int64
─────┼──────────────────────
   2 │     2  M          54

Get element

Input:

# get element in row 2, column 3
println(df[2,3])

Output:

Get subset (specific rows and all columns)

Input:

# print out rows 1, 3, & 5
println(df[[1,3,5], :])

Output:

3×3 DataFrame
 Row │ id     gender  age
     │ Int64  String  Int64
─────┼──────────────────────
   1 │     1  F          68
   2 │     3  F          49
   3 │     5  F          36

Get subset (all rows and specific columns)

Input:

# print out all rows and only columns 1 (id) and 3 (age)
println("Using column names:\n")
println(df[:, [:id, :age]])
println()

println("Using column numbers:\n")
println(df[:, [1,3]])

Output:

Using column names:

5×2 DataFrame
 Row │ id     age
     │ Int64  Int64
─────┼──────────────
   1 │     1     68
   2 │     2     54
   3 │     3     49
   4 │     4     28
   5 │     5     36

Using column numbers:

5×2 DataFrame
 Row │ id     age
     │ Int64  Int64
─────┼──────────────
   1 │     1     68
   2 │     2     54
   3 │     3     49
   4 │     4     28
   5 │     5     36

Get subset (all rows meeting specified criteria - numbers)

Input:

# print out all rows where age is greater than 50
println(df[df.age .> 50, :])

Output:

2×3 DataFrame
 Row │ id     gender  age
     │ Int64  String  Int64
─────┼──────────────────────
   1 │     1  F          68
   2 │     2  M          54

Get subset (all rows meeting specified criteria - strings)

Input:

# print out all rows where gender is female ("F")
println(df[df.gender .== "F", :])

Output:

3×3 DataFrame
 Row │ id     gender  age
     │ Int64  String  Int64
─────┼──────────────────────
   1 │     1  F          68
   2 │     3  F          49
   3 │     5  F          36

Get subset (all rows meeting specified criteria)

Input:

# print out all rows where gender is female ("F") and age is between 25-50
println(df[(df.gender .== "F") .& (25 .< df.age .< 50), :])

Output:

2×3 DataFrame
 Row │ id     gender  age
     │ Int64  String  Int64
─────┼──────────────────────
   1 │     3  F          49
   2 │     5  F          36

Add Column

New columns with specified values

Input:

# add a column for weight
df.weight = [100, 120, 150, 175, 300]

# add a column for height
df.height = [62, 60, 61, 63, 64]

println(df)
println()

println("Describe dataframe to see column names and summary:\n")
println(describe(df))

Output:

5×5 DataFrame
 Row │ id     gender  age    weight  height
     │ Int64  String  Int64  Int64   Int64
─────┼──────────────────────────────────────
   1 │     1  F          68     100      62
   2 │     2  M          54     120      60
   3 │     3  F          49     150      61
   4 │     4  M          28     175      63
   5 │     5  F          36     300      64

Describe dataframe to see column names and summary:

5×7 DataFrame
 Row │ variable  mean    min  median  max  nmissing  eltype
     │ Symbol    Union…  Any  Union…  Any  Int64     DataType
─────┼────────────────────────────────────────────────────────
   1 │ id        3.0     1    3.0     5           0  Int64
   2 │ gender            F            M           0  String
   3 │ age       47.0    28   49.0    68          0  Int64
   4 │ weight    169.0   100  150.0   300         0  Int64
   5 │ height    62.0    60   62.0    64          0  Int64

New column with calculated value

Input:

# add a column with calculated BMI
df.bmi = map((x,y) -> (x/y^2)*703, df.weight, df.height)

println(df)
println()

println("Describe dataframe to see new bmi column and summary:\n")
println(describe(df))

Output:

5×6 DataFrame
 Row │ id     gender  age    weight  height  bmi
     │ Int64  String  Int64  Int64   Int64   Float64
─────┼───────────────────────────────────────────────
   1 │     1  F          68     100      62  18.2882
   2 │     2  M          54     120      60  23.4333
   3 │     3  F          49     150      61  28.3392
   4 │     4  M          28     175      63  30.9965
   5 │     5  F          36     300      64  51.4893

Describe dataframe to see new bmi column and summary:

6×7 DataFrame
 Row │ variable  mean     min      median   max      nmissing  eltype
     │ Symbol    Union…   Any      Union…   Any      Int64     DataType
─────┼──────────────────────────────────────────────────────────────────
   1 │ id        3.0      1        3.0      5               0  Int64
   2 │ gender             F                 M               0  String
   3 │ age       47.0     28       49.0     68              0  Int64
   4 │ weight    169.0    100      150.0    300             0  Int64
   5 │ height    62.0     60       62.0     64              0  Int64
   6 │ bmi       30.5093  18.2882  28.3392  51.4893         0  Float64

Get counts/frequency

Input:

# get counts of males and females in the dataframe
println(combine(groupby(df, :gender), nrow => :N))

Output:

2×2 DataFrame
 Row │ gender  N     
     │ String  Int64 
─────┼───────────────
   1 │ F           3
   2 │ M           2

Transform DataFrame

sort

Input:

# sort the dataframe by gender and then age in reverse order for age (oldest to youngest)
println(sort(df, [:gender, :age], rev=(false, true)))

Output:

TypeError: in keyword argument rev, expected Union{Bool, AbstractArray{Bool,1}}, got Tuple{Bool,Bool}

stack (reshape from wide to long format)

Input:

# Reshape from wide to long format (disclude id to see which column and value matches which patient id)
long_df = stack(df, Not(:id))
println(long_df)

Output:

25×3 DataFrame
 Row │ id     variable  value
     │ Int64  String    Any
─────┼──────────────────────────
   1 │     1  gender    F
   2 │     2  gender    M
   3 │     3  gender    F
   4 │     4  gender    M
   5 │     5  gender    F
   6 │     1  age       68
   7 │     2  age       54
   8 │     3  age       49
   9 │     4  age       28
  10 │     5  age       36
  11 │     1  weight    100
  12 │     2  weight    120
  13 │     3  weight    150
  14 │     4  weight    175
  15 │     5  weight    300
  16 │     1  height    62
  17 │     2  height    60
  18 │     3  height    61
  19 │     4  height    63
  20 │     5  height    64
  21 │     1  bmi       18.2882
  22 │     2  bmi       23.4333
  23 │     3  bmi       28.3392
  24 │     4  bmi       30.9965
  25 │     5  bmi       51.4893

unstack (reshape from long to wide format)

Input:

#unstack dataframe to get back to wide format based off "id" (unstack(df, :id, :variable, :value))
wide_df = unstack(long_df, :id, :variable, :value)
println(wide_df)

Output:

5×6 DataFrame
 Row │ id     gender  age  weight  height  bmi
     │ Int64  Any     Any  Any     Any     Any
─────┼─────────────────────────────────────────────
   1 │     1  F       68   100     62      18.2882
   2 │     2  M       54   120     60      23.4333
   3 │     3  F       49   150     61      28.3392
   4 │     4  M       28   175     63      30.9965
   5 │     5  F       36   300     64      51.4893

Traversing DataFrame (for loops)

sort

Input:

# size of dataframe = size(df)
# set number of rows to nrows and number of columns to ncols
println("(nrows, ncols) = $(size(df))")
nrows, ncols = size(df)

# use nested for loop to get information from dataframe by row and column
for row in 1:nrows
  for col in 1:ncols
	println("value for row $row and col $col is $(df[row,col])")
  end
end

Output:

(nrows, ncols) = (5, 6)
value for row 1 and col 1 is 1
value for row 1 and col 2 is F
value for row 1 and col 3 is 68
value for row 1 and col 4 is 100
value for row 1 and col 5 is 62
value for row 1 and col 6 is 18.28824141519251
value for row 2 and col 1 is 2
value for row 2 and col 2 is M
value for row 2 and col 3 is 54
value for row 2 and col 4 is 120
value for row 2 and col 5 is 60
value for row 2 and col 6 is 23.433333333333334
value for row 3 and col 1 is 3
value for row 3 and col 2 is F
value for row 3 and col 3 is 49
value for row 3 and col 4 is 150
value for row 3 and col 5 is 61
value for row 3 and col 6 is 28.339156140822357
value for row 4 and col 1 is 4
value for row 4 and col 2 is M
value for row 4 and col 3 is 28
value for row 4 and col 4 is 175
value for row 4 and col 5 is 63
value for row 4 and col 6 is 30.99647266313933
value for row 5 and col 1 is 5
value for row 5 and col 2 is F
value for row 5 and col 3 is 36
value for row 5 and col 4 is 300
value for row 5 and col 5 is 64
value for row 5 and col 6 is 51.4892578125

Exercises

Analyzing Health Datasets with DataFrames in Julia - Forthcoming!

References

JuliaData Contributors. (n.d.). DataFrames.jl - JuliaData. Retrieved May 1, 2024, from https://dataframes.juliadata.org/stable/

Resources

Julia Package: DataFrames.jl
Julia Package: CSV.jl
Julia Data Science: DataFrames.jl
Introducing Julia Wikibook: DataFrames
Julia DataFrames Cheat Sheets

Exercises

List of exercises found across the different Julia pages.

Use Julia in Brown Oscar Computing Environment - Forthcoming!
Use Julia in Brown Stronghold Computing Environment - Forthcoming!
Create a Health Calculator Using Julia - Forthcoming!
- Create a Pediatric Dosage Calculator Using Julia
- Create a BMI Calculator Using Julia
Analyze Health Datasets Using Unix Commands - Forthcoming!
- Analyze MIMIC-IV Demo Files Using Unix Commands
- Analyze SyntheticRI Demo Files Using Unix
Analyze Health Datasets Using Julia - Forthcoming!
- Analyze MIMIC-IV Demo Files Using Julia
- Analyze SyntheticRI Demo Files Using Julia

Python

Python is one of the many languages used by the data science community to perform data manipulation, statistical modeling and machine learning. Its design philosophy emphasizes code readability. The python community is huge, offering an enormous library of technical support documentation. If you don't know how to do something in Python, chances are, someone else asked a similar question online and received a comprehensive answer.

Resources

Installation

Instructions for installing Python on macOS and Windows operating systems can be found .

For most users, it is recommended to download the current stable release from .

Some developers might wish to use a different version, or to switch between versions. For this, the can be useful.

Python is also available for use in Brown's :

Oscar (for high-performance computing)
Stronghold (for secure computing)

macOS 16.X Ventura

The following instructions have been tested on computers running macOS 16 Big Ventura. In order to check the macOS version running on your computer, click on the "apple" icon in the top left hand corner of your screen and select "About This Mac." A window will pop up that includes a version number. Confirm you are running at least Version 16.X (where 'X' is any number). These instructions will likely work with earlier versions of macOS as well. If you are not running macOS 11.X Big Sur, you can upgrade for free following the instructions provided on .

Download Python

Navigate to and download the most recent version of Python for macOS.

Install Python

Open the downloaded file (e.g., python-3.12.3-macos11.pkg). A window will pop up with installation instructions. Progress through the prompts until Python has been installed in your Applications folder. Next, double click on the Python folder shortcut in your Applications folder to open it.

Run Python

Open, Terminal, type python3, and hit return. Python should open. To quit Python, type quit() and hit return.

Troubleshooting

If you get a Permission denied error, rerun the command prepended with sudo. You will be prompted to enter your computer password.

WindowsOS

The following instructions have been tested on computers running Windows 10. Confirm that you are running at least Windows 10. These instructions will likely work with earlier versions of Windows, however they have not been tested.

Download Python

Install Python

Open the downloaded file (e.g., python-3.10.10-amd64.exe). A window will pop up with installation instructions. Progress through the prompts until Python has been installed on your device. When prompted with Advanced Options, make sure to check "Add Python to environment variables".

Run Python

Open Command Prompt, type py, and hit enter. Python should open to quit Python, type quit() and hit return.

REPL

Python comes with a full-featured interactive command-line REPL (read-eval-print loop) built into the pythonexecutable. In addition to allowing quick and easy evaluation of Python statements, it has a searchable history, tab-completion, many helpful keybindings, and dedicated help ? and shell modes ;.

This page provides examples of using REPL on the command line

Python REPL Example (local)

Type python in terminal to launch REPL

Python REPL Help Pages (local)

Type "help" to enter help pages within REPL
Type a function from Python to read help pages (ex:print)
Press q to quit

Resources

Basic Syntax

"Hello, World!" Program

This is the typical first program for those new to a general purpose programming language like Python. It can be used to test that the of Python is working and also introduce Python's basic syntax using the environment or running code written using a at the command line.

Input:

Output:

Here are variations of the "Hello, World!" programming using variables and different print statements.

Input:

Output:

Variable Assignment

In order to assign variables in Python, you write the desired name for your variable, an “=” sign, and what the value of the variable should be.

Input:

Output:

Comments

We can write comments on our code, which do not run, to describe what certain lines of code or section of code do
- These comments are just for the programmer, they will not appear anywhere in the output and just are there to explain what the code is doing or to provide helpful notes
- To make a comment in Python, you can use the “#” symbol and then type your comment
Sometimes you might want to write longer comments that span multiple lines – to do this you can surround these comments with three tick marks above the start as well as three tick marks below the end

Input:

Output:

Print Statements

Without using a print statement, Python will only print out the most recent item that has an output. In order to print multiple things, we can use the print() function

Input:

Output:

Indentation

Python is very sensitive with its indentation notation. Indentation should only be used in hierarchical structures, such as a class, function, or loop. Indents in improper locations will cause an error

Input:

Output:

Exercises

Use Python in Brown Oscar Computing Environment - Forthcoming!
Use Python in Brown Stronghold Computing Environment - Forthcoming!

Resources

Numbers and Math

This page provides syntax for using numbers and mathematic operations in Python. Each section includes an example to demonstrate the described syntax and operations.

Types of Numbers

Integer (positive and negative counting number) - e.g., -3, -2, -1, 0, 1, 2, and 3:
- int - holds signed integers of non-limited length
- long - holds long integers (exists in Python 2.X, depreciated in Python 3.X)
Float (real or floating point numbers) - e.g., -2.14, 0.0, and 3.777
- float
Boolean: (0 = False and 1 = True)
- bool

Use type() function to determine type

Input:

Output:

Arithmetic Operators

Input:

Output:

Comparison Operators and Functions

Input:

Output:

Exercises

Create a Health Calculator Using Python - Forthcoming!

Resources

Strings and Characters

This page provides syntax for different data types in Python as well as some of their associated functions. Each section includes an example to demonstrate the described syntax or function.

Strings

A string is a sequence of one or more characters (index values start at 0)

Some functions and index methods that can be performed on strings

Action

Function

Input:

Output:

Resources

Regular Expressions

Regular expressions are powerful tools for pattern matching and text processing. They are represented as a pattern that consists of a special set of characters to search for in a string str. The regex module needs to be imported before use.

This page provides syntax for regular expressions in Python . Each section includes an example to demonstrate the described methods.

Functions

Action

Function

Character Class

Character class specifies a list of characters to match ([...] where ... represents the list) or not match ([^...])

Anchors

Anchors are special characters that can be used to match a pattern at a specified position

Repetition and Quantifier Characters

Repetition or quantifier characters specify the number of times to match a particular character or set of characters

Input:

Output:

Resources

DataFrames

DataFrames.jl is a Julia package that provides a set of tools for working with tabular data in Julia. Its design and functionality are similar to those of pandas (in Python) and data.frame, data.table and dplyr (in R), making it a great general purpose data science tool. [1]

This page provides examples of using DataFrames.jl, demonstrating the syntax and common functions within the package.

Example

Install and Load DataFrames.jl Package

using Pkg

# Add DataFrames package
Pkg.add("DataFrames")

# Load paackages
using DataFrames

Create Dataframe

# Create dataframe
df = DataFrame(id = 1:5, gender = ["F", "M", "F", "M", "F"], age = [68, 54, 49, 28, 36])

Display Dataframe

Input:

# display dataframe
println(df)

Output:

5×3 DataFrame
 Row │ id     gender  age
     │ Int64  String  Int64
─────┼──────────────────────
   1 │     1  F          68
   2 │     2  M          54
   3 │     3  F          49
   4 │     4  M          28
   5 │     5  F          36

First two lines of dataframe:

Input:

println(first(df, 2))

Output:

2×3 DataFrame
 Row │ id     gender  age
     │ Int64  String  Int64
─────┼──────────────────────
   1 │     1  F          68
   2 │     2  M          54

Last two lines of dataframe:

Input:

println(last(df, 2))

Output:

2×3 DataFrame
 Row │ id     gender  age
     │ Int64  String  Int64
─────┼──────────────────────
   1 │     4  M          28
   2 │     5  F          36

Describe Dataframe

Dataframe size:

Input:

# dataframe size
println(size(df))

Output:

(5, 3)

Dataframe column names:

Input:

# dataframe column names
println(names(df))

Output:

["id", "gender", "age"]

Dataframe description:

Input:

# describe dataframe
println(describe(df))

Output:

3×7 DataFrame
 Row │ variable  mean    min  median  max  nmissing  eltype
     │ Symbol    Union…  Any  Union…  Any  Int64     DataType
─────┼────────────────────────────────────────────────────────
   1 │ id        3.0     1    3.0     5           0  Int64
   2 │ gender            F            M           0  String
   3 │ age       47.0    28   49.0    68          0  Int64

Accessing DataFrames

Get "age" column (different ways to call the column)

Input:

# call by column name
println(df[!, :age])

# get column by column number
println(df[!, 3])

# alternate syntax
println(df.age)

Output:

[68, 54, 49, 28, 36]
[68, 54, 49, 28, 36]
[68, 54, 49, 28, 36]

Get row

Input:

# print row 2
println(df[2, :])

Output:

DataFrameRow
 Row │ id     gender  age
     │ Int64  String  Int64
─────┼──────────────────────
   2 │     2  M          54

Get element

Input:

# get element in row 2, column 3
println(df[2,3])

Output:

Get subset (specific rows and all columns)

Input:

# print out rows 1, 3, & 5
println(df[[1,3,5], :])

Output:

3×3 DataFrame
 Row │ id     gender  age
     │ Int64  String  Int64
─────┼──────────────────────
   1 │     1  F          68
   2 │     3  F          49
   3 │     5  F          36

Get subset (all rows and specific columns)

Input:

# print out all rows and only columns 1 (id) and 3 (age)
println("Using column names:\n")
println(df[:, [:id, :age]])
println()

println("Using column numbers:\n")
println(df[:, [1,3]])

Output:

Using column names:

5×2 DataFrame
 Row │ id     age
     │ Int64  Int64
─────┼──────────────
   1 │     1     68
   2 │     2     54
   3 │     3     49
   4 │     4     28
   5 │     5     36

Using column numbers:

5×2 DataFrame
 Row │ id     age
     │ Int64  Int64
─────┼──────────────
   1 │     1     68
   2 │     2     54
   3 │     3     49
   4 │     4     28
   5 │     5     36

Get subset (all rows meeting specified criteria - numbers)

Input:

# print out all rows where age is greater than 50
println(df[df.age .> 50, :])

Output:

2×3 DataFrame
 Row │ id     gender  age
     │ Int64  String  Int64
─────┼──────────────────────
   1 │     1  F          68
   2 │     2  M          54

Get subset (all rows meeting specified criteria - strings)

Input:

# print out all rows where gender is female ("F")
println(df[df.gender .== "F", :])

Output:

3×3 DataFrame
 Row │ id     gender  age
     │ Int64  String  Int64
─────┼──────────────────────
   1 │     1  F          68
   2 │     3  F          49
   3 │     5  F          36

Get subset (all rows meeting specified criteria)

Input:

# print out all rows where gender is female ("F") and age is between 25-50
println(df[(df.gender .== "F") .& (25 .< df.age .< 50), :])

Output:

2×3 DataFrame
 Row │ id     gender  age
     │ Int64  String  Int64
─────┼──────────────────────
   1 │     3  F          49
   2 │     5  F          36

Add Column

New columns with specified values

Input:

# add a column for weight
df.weight = [100, 120, 150, 175, 300]

# add a column for height
df.height = [62, 60, 61, 63, 64]

println(df)
println()

println("Describe dataframe to see column names and summary:\n")
println(describe(df))

Output:

5×5 DataFrame
 Row │ id     gender  age    weight  height
     │ Int64  String  Int64  Int64   Int64
─────┼──────────────────────────────────────
   1 │     1  F          68     100      62
   2 │     2  M          54     120      60
   3 │     3  F          49     150      61
   4 │     4  M          28     175      63
   5 │     5  F          36     300      64

Describe dataframe to see column names and summary:

5×7 DataFrame
 Row │ variable  mean    min  median  max  nmissing  eltype
     │ Symbol    Union…  Any  Union…  Any  Int64     DataType
─────┼────────────────────────────────────────────────────────
   1 │ id        3.0     1    3.0     5           0  Int64
   2 │ gender            F            M           0  String
   3 │ age       47.0    28   49.0    68          0  Int64
   4 │ weight    169.0   100  150.0   300         0  Int64
   5 │ height    62.0    60   62.0    64          0  Int64

New column with calculated value

Input:

# add a column with calculated BMI
df.bmi = map((x,y) -> (x/y^2)*703, df.weight, df.height)

println(df)
println()

println("Describe dataframe to see new bmi column and summary:\n")
println(describe(df))

Output:

5×6 DataFrame
 Row │ id     gender  age    weight  height  bmi
     │ Int64  String  Int64  Int64   Int64   Float64
─────┼───────────────────────────────────────────────
   1 │     1  F          68     100      62  18.2882
   2 │     2  M          54     120      60  23.4333
   3 │     3  F          49     150      61  28.3392
   4 │     4  M          28     175      63  30.9965
   5 │     5  F          36     300      64  51.4893

Describe dataframe to see new bmi column and summary:

6×7 DataFrame
 Row │ variable  mean     min      median   max      nmissing  eltype
     │ Symbol    Union…   Any      Union…   Any      Int64     DataType
─────┼──────────────────────────────────────────────────────────────────
   1 │ id        3.0      1        3.0      5               0  Int64
   2 │ gender             F                 M               0  String
   3 │ age       47.0     28       49.0     68              0  Int64
   4 │ weight    169.0    100      150.0    300             0  Int64
   5 │ height    62.0     60       62.0     64              0  Int64
   6 │ bmi       30.5093  18.2882  28.3392  51.4893         0  Float64

Get counts/frequency

Input:

# get counts of males and females in the dataframe
println(combine(groupby(df, :gender), nrow => :N))

Output:

2×2 DataFrame
 Row │ gender  N     
     │ String  Int64 
─────┼───────────────
   1 │ F           3
   2 │ M           2

Transform DataFrame

sort

Input:

# sort the dataframe by gender and then age in reverse order for age (oldest to youngest)
println(sort(df, [:gender, :age], rev=(false, true)))

Output:

TypeError: in keyword argument rev, expected Union{Bool, AbstractArray{Bool,1}}, got Tuple{Bool,Bool}

stack (reshape from wide to long format)

Input:

# Reshape from wide to long format (disclude id to see which column and value matches which patient id)
long_df = stack(df, Not(:id))
println(long_df)

Output:

25×3 DataFrame
 Row │ id     variable  value
     │ Int64  String    Any
─────┼──────────────────────────
   1 │     1  gender    F
   2 │     2  gender    M
   3 │     3  gender    F
   4 │     4  gender    M
   5 │     5  gender    F
   6 │     1  age       68
   7 │     2  age       54
   8 │     3  age       49
   9 │     4  age       28
  10 │     5  age       36
  11 │     1  weight    100
  12 │     2  weight    120
  13 │     3  weight    150
  14 │     4  weight    175
  15 │     5  weight    300
  16 │     1  height    62
  17 │     2  height    60
  18 │     3  height    61
  19 │     4  height    63
  20 │     5  height    64
  21 │     1  bmi       18.2882
  22 │     2  bmi       23.4333
  23 │     3  bmi       28.3392
  24 │     4  bmi       30.9965
  25 │     5  bmi       51.4893

unstack (reshape from long to wide format)

Input:

#unstack dataframe to get back to wide format based off "id" (unstack(df, :id, :variable, :value))
wide_df = unstack(long_df, :id, :variable, :value)
println(wide_df)

Output:

5×6 DataFrame
 Row │ id     gender  age  weight  height  bmi
     │ Int64  Any     Any  Any     Any     Any
─────┼─────────────────────────────────────────────
   1 │     1  F       68   100     62      18.2882
   2 │     2  M       54   120     60      23.4333
   3 │     3  F       49   150     61      28.3392
   4 │     4  M       28   175     63      30.9965
   5 │     5  F       36   300     64      51.4893

Traversing DataFrame (for loops)

sort

Input:

# size of dataframe = size(df)
# set number of rows to nrows and number of columns to ncols
println("(nrows, ncols) = $(size(df))")
nrows, ncols = size(df)

# use nested for loop to get information from dataframe by row and column
for row in 1:nrows
  for col in 1:ncols
	println("value for row $row and col $col is $(df[row,col])")
  end
end

Output:

(nrows, ncols) = (5, 6)
value for row 1 and col 1 is 1
value for row 1 and col 2 is F
value for row 1 and col 3 is 68
value for row 1 and col 4 is 100
value for row 1 and col 5 is 62
value for row 1 and col 6 is 18.28824141519251
value for row 2 and col 1 is 2
value for row 2 and col 2 is M
value for row 2 and col 3 is 54
value for row 2 and col 4 is 120
value for row 2 and col 5 is 60
value for row 2 and col 6 is 23.433333333333334
value for row 3 and col 1 is 3
value for row 3 and col 2 is F
value for row 3 and col 3 is 49
value for row 3 and col 4 is 150
value for row 3 and col 5 is 61
value for row 3 and col 6 is 28.339156140822357
value for row 4 and col 1 is 4
value for row 4 and col 2 is M
value for row 4 and col 3 is 28
value for row 4 and col 4 is 175
value for row 4 and col 5 is 63
value for row 4 and col 6 is 30.99647266313933
value for row 5 and col 1 is 5
value for row 5 and col 2 is F
value for row 5 and col 3 is 36
value for row 5 and col 4 is 300
value for row 5 and col 5 is 64
value for row 5 and col 6 is 51.4892578125

Exercises

Analyzing Health Datasets with DataFrames in Julia - Forthcoming!

References

JuliaData Contributors. (n.d.). DataFrames.jl - JuliaData. Retrieved May 1, 2024, from https://dataframes.juliadata.org/stable/

Resources

Julia Package: DataFrames.jl
Julia Package: CSV.jl
Julia Data Science: DataFrames.jl
Introducing Julia Wikibook: DataFrames
Julia DataFrames Cheat Sheets

Computing Skills

Introduction

File Directory Structures

Hierarchical Structure

GUI

Resources

Text Editors

Popular Text Editors

Microsoft Visual Studio Code (recommended)

Jupyter Notebooks

VI/VIM

Emacs

Pico

GitHub

Saving and Retrieving Code Changes

Resolving Merge Conflicts

Resources

Unix

Unix File System

Understanding Directory/File Ownership and Permissions

Unix Shell

Unix Commands

Working with Data

File and Directory Navigation

System Information

Analyzing and Searching Files

Common Shortcuts

File Content Browsing

File Permissions

Exercises

Resources

Julia

References

Resources

Installation

Resources

REPL

Julia REPL Example (local)

Julia REPL Help Pages (local)

References

Resources

Basic Syntax

"Hello, World!" Program

Variable Assignment

Comments

Print Statements

Exercises

Resources

Numbers and Math

Types of Numbers

Arithmetic Operators

Comparison Operators and Functions

Exercises

Resources

Strings and Characters

Characters and Strings

Some functions that can be performed on strings

Resources

Regular Expressions

Functions

Character Class

Anchors

Repetition and Quantifier Characters

Resources

Control Flow

Use Cases and Syntax

Conditional Statements

Loops

Comparison Operators and Functions

References

Resources

Collections and Data Structures

Arrays

Creating arrays

Creating array from string

Accessing elements

Adding and removing elements

Sort and unique

Compare arrays

Convert array to string