File system and project management

Further reading on today’s material

  • Chapter 6 of R for Data Science, available here.

  • Communicate Data With R chapter on RProjects, available here

File paths

We start with the basics: understanding how files are stored on your machine.

Why start here?

Most analysis scripts begin with something like:

my_data <- read.csv("~/Desktop/gklab/research/psf-kadumane/data/biomass.csv")
  • If your script starts this way, it is unlikely to run seemlessly on any other computer!

File paths

  • Each file on your computer has a unique address, and the way to navigate to that address is with a “file path”

  • File paths can be written as “absolute paths”, i.e. “How to get to this file, starting from your root directory?”

  • or as “relative paths”, i.e. “How to get to this file from any arbitrary location in the file system?”

Understanding file management

  • Start thinking in terms of nested hierarchies

Image source

Alternative visualization

# install.packges('fs') # if needed
library(fs)
dir_tree(path = "~", recurse = 0)
~
├── Desktop
├── Documents
├── Downloads
├── Library
├── Movies
├── Music
├── Pictures
├── Public
├── lab-admin
├── research
└── teaching

Gaurav’s setup

fs::dir_tree("~/work", 0)

Gaurav’s setup

fs::dir_tree("~/work/teaching", 0)

Gaurav’s setup

fs::dir_tree("~/work/teaching", 1)

Exercise

Consider the following directory structure:

Home/
├── coursework
│   ├── course1
│   ├── course2
│   └── course3
└── research
    ├── project1
    └── project2

You are currently working within the directory ~/research/project1, but you realize it is time to review your notes for Course 2.

What steps do you need to navigate to Course 2 notes? Note that you have to articulate the directions one step at a time.

Exercise

Consider the following directory structure:

Home/
├── coursework
│   ├── course1
│   ├── course2
│   └── course3
└── research
    ├── project1
    └── project2

You are currently working within the project1 directory, but your collaborator asks about a file related to your Project 3.

You have forgotten the names of all the files in that project. How can you quickly check what’s there?

Exercise

Consider the following directory structure:

Home/
├── coursework
│   ├── course1
│   ├── course2
│   └── course3
└── research
    ├── project1
    └── project2

You had written a script “cool-analysis.R” for your research Project 2 which you now realize is relevant for the Fall 2025 Reproducible Research course. Assuming you are currently in the Home directory, what command could you use to copy cool-analysis.R from the project2 directory into the f25-repro-res directory?

Exercise

Consider the following directory structure:

Home/
├── coursework
│   ├── course1
│   ├── course2
│   └── course3
└── research
    ├── project1
    └── project2

You got back 252 fastq sequence files from the first big sequencing run of your dissertation. Congrats! In your excitement, you downloaded and stored these under project3 even though they have to do with Project 1. From within the project1 directory, how could you move all 252 of these files over from project3?

Why are we starting the semester with file paths?

  • Many analysis scripts begin with something like this:
my_data <- read.csv("~/Desktop/gklab/research/psf-kadumane/data/biomass.csv")

  • You wouldn’t be able to run this on your computers without mucking around!

  • In a reproducible analysis context, if you can’t read in the dataset, you probably will give up with the reproduction.

  • Over the course of your dissertation (and certainly your career), you will deal with many different inter-related projects.

  • You may even use several different computing systems (e.g. right now, you might have a personal laptop, a lab desktop, and use the LSU HPC. Some time in the future, you may replace your computers or add new machines).

How to maintain an organized system?

Better project management within RStudio

  • RStudio’s Projects feature helps maintain workflows, especially as you accumulate many parallel projects.

  • Inside an RStudio Project, you will only ever use “Relative” paths

  • The “starting point” is always the directory in which the RProject file is saved.

    • This means that you can simply share an RProject directory with collaborators (or across your own computers) and rerun code without having to worry about paths.

Better project management with RStudio

  • Live coding example
    • Creating new projects
    • Anatomy of a project: .Rproj file, home directory, etc.
    • Switching between projects

Other aspects of project management

  • Internal organization
  • File names
  • README files

Internal organization

  • For any given project, conduct a “project audit” to think through the types of files that this work might entail

  • e.g. for even a “simple” plant ecology project, you might have text files for project brainstorming/literature reviews, spatial data files for locating field plots, scripts for planning experimental/sampling design, flat data files for trait measurements, sequence files, analysis scripts, figures, tables, text files for manuscripts, presentation files, (and likely more).

  • Your project’s internal structure should be set up to accommodate the complexity of that particular project.

More on this on Thursday

File names

  • As the previous slide hinted at…
    there will be a lot of files!

  • File naming as a strategy to manage the chaos

  • Principles for good file names:

. . . 

  • Human readable
  • Machine readable
  • Play well with default ordering

File names: Human readable

❌ JW7d^(2sl@*.csv

✔️plot1 fire temps 03 Feb 2025.csv

File names: Machine readable

❌ plot 1 fire temps. csv

✔️plot1-fire-temps-3Feb2025.csv

File names: Play well with default ordering

❌ plot1-fire-temps-3Feb2025.csv

✔️ 2025-02-03-plot1-fire-temps.csv