+ - 0:00:00
Notes for current slide
Notes for next slide

a disk can have multiple partitions

The file system and basic unix shell

Introduction to Data Science (BIOL7800) https://introdatasci.dlilab.com/

Daijiang Li

LSU

2021/09/21

1 / 12

A file system is a logical collection of files on a partition or disk

Unix filesystem (common folders):

Directory Description
/ Root directory
/bin Executable files for all users
/dev Device drivers
/home Home directory for users
/tmp Temporary files used between system boots
/usr Used for miscellaneous purposes, e.g. admin commands, shared files, etc.
/Users (macOS) Files of different users
2 / 12

a disk can have multiple partitions

File tree

# Debian / mint / Ubuntu
sudo apt-get install tree
# macOS & linux (with homebrew installed)
brew install tree
tree -L 3 .
.
├── libs
│   ├── header-attrs
│   │   └── header-attrs.js
│   └── remark-css
│       ├── default-fonts.css
│       └── default.css
├── presentation.Rmd
└── presentation.html
3 directories, 5 files
fs::dir_tree(path = ".") #R
## .
## ├── libs
## │   ├── header-attrs
## │   │   └── header-attrs.js
## │   └── remark-css
## │       ├── default-fonts.css
## │       └── default.css
## ├── presentation.Rmd
## └── presentation.html
3 / 12

Naming files and directories

  • All names are case sensitive
  • You can use pretty much any character except /
    • Better to avoid space in names! (looking at you, Box Sync!)
    • Also try to avoid special characters such as >, |, :, &, !
    • \ backslash-escaping the character (i.e. quote it). ls ~/Box\ Sync
    • files and directories start with . will be hidden
  • Names are generally limited to 255 characters
  • Use date format ISO: YYYYMMDD or YYYY-MM-DD
4 / 12

Naming files and directories

  • Name should be consistent (machine friendly) and descriptive (human readable)
    • data.csv (not meaningful)
    • Li_plant_diversity_WI_2020-06-08.csv (better)
  • Name files so they can ordered
    • 00_pkg_functions.R
    • 01_data.R
  • Have a file extension that matches the file format

Take time to practice and adopt these styles and strategies can save you a lot of time in the future

5 / 12

Absolute vs. relative paths

pwd to check where you are right now. Let's assume we are at /Users/dli/IntroDataScience/lectures/06_bash

Absolute path

# list files within the current folder
ls /Users/dli/IntroDataScience/lectures/06_bash
# move one level up?
cd /Users/dli/IntroDataScience/lectures 
# move two levels up?
cd /Users/dli/IntroDataScience 
# move to the home folder?
cd /Users/dli # /home/user

Relative path

# list files within the current folder
ls .
# move one level up?
cd ..
# move two levels up?
cd ../..
# move to the home folder?
cd ~
6 / 12

Some basic commands

  • man to read the manual! Press q to quit, space to read next page (--help of no man)
  • Tab for auto-completence
  • Creating, removing, changing directories
    • mkdir
    • rmdir
    • cd
  • Moving, coping, removing files
    • touch to create empty files
    • mv
    • cp
    • rm: notable options: -r, -f (dangerous rm -rf)
7 / 12

Warning: no undo button on the command line, so be extremely careful with deleting files/directories! I recommend to use mv to move files to the Trash Bin instead. (~/.Trash on Mac, ~/.local/share/Trash on Ubuntu)

8 / 12

Some basic commands

  • List files with ls: -a, -l
  • Wildcard to work with multiple files and folders
    • ls /bin/*sh
    • ls /bin/*[ab]*
    • mv /path/to/*.jpeg /new/path
  • Check history of used command history
    • !200 to rerun the 200th command from history
  • open (macOS) or start
9 / 12

The Unix Philosophy

Each program should do one thing well

Programs can be linked with the pipe operator | to do more complex tasks

ls -l | head -n 3
head -100 file.csv | tail -20 | > file_81_100.csv
10 / 12

Moving cursor inside bash

  • ^ means Ctrl key, e.g. ^A means Ctrl+A.
  • ^A move to the beginning of a line in the shell.
  • ^E move to the end of a line.
  • ^C cancel what you are doing. If it does not work, try ^\.
  • ^D end of a line. This normally will kill the job too.
  • ^L clean the screen of your shell.
  • ^h delete back one character (backspace).
  • ^w delete back one word.
  • ^u delete back to the start of line.
11 / 12

More common commands see course webpage

12 / 12

A file system is a logical collection of files on a partition or disk

Unix filesystem (common folders):

Directory Description
/ Root directory
/bin Executable files for all users
/dev Device drivers
/home Home directory for users
/tmp Temporary files used between system boots
/usr Used for miscellaneous purposes, e.g. admin commands, shared files, etc.
/Users (macOS) Files of different users
2 / 12

a disk can have multiple partitions

Paused

Help

Keyboard shortcuts

↑, ←, Pg Up, k Go to previous slide
↓, →, Pg Dn, Space, j Go to next slide
Home Go to first slide
End Go to last slide
Number + Return Go to specific slide
b / m / f Toggle blackout / mirrored / fullscreen mode
c Clone slideshow
p Toggle presenter mode
t Restart the presentation timer
?, h Toggle this help
Esc Back to slideshow