Module 2 — Linux Fundamentals How Linux is built and how to drive it from the command line: kernel vs user space, the filesystem hierarchy, essential commands, permissions, and processes/services/logs. Lesson: How Linux Is Structured What you'll learn What a kernel actually does and why every computer needs one. The difference between kernel space and user space , and why that split exists. How system calls form the controlled boundary between your programs and the hardware. What a process is and how the kernel keeps many of them running at once. What people really mean when they say "an operating system." Skill gained: a correct mental model of how Linux is layered, so the commands you learn later stop feeling like magic. The lesson You are about to spend a lot of time on Linux machines. Before you memorize commands, you need a picture of what is happening underneath. This lesson is conceptual, but everything in it is real and you can poke at it on the Jumpbox ( 10.100.100.254 , user ubuntu ). 1. What a kernel does The kernel is the core program of the operating system. It is the first real software that takes control after the machine powers on, and it stays running until you shut down. Its job is to manage the hardware and share it fairly between every program you run. Concretely, the kernel is responsible for: Processes & scheduling — deciding which program runs on the CPU and for how long. Memory — giving each program its own slice of RAM and stopping programs from reading each other's memory. Devices — talking to disks, network cards, keyboards, screens through drivers . Filesystems — turning raw blocks on a disk into files and directories. Networking — moving packets in and out of the machine. On Ubuntu you can see your kernel's version: ubuntu@Jumpbox:~$ uname -r 6.8.0-generic uname ("Unix name") prints information about the running kernel. The -r flag asks for the r elease. 2. Kernel space vs user space The kernel does powerful, dangerous things — it can touch any byte of RAM and any device. If every program could do that directly, one buggy app could crash the whole machine or read your passwords. So the CPU itself enforces two privilege levels : Kernel space — privileged mode. Only the kernel runs here. Full access to hardware. User space — restricted mode. Your programs ( ls , a web server, your editor) run here. They cannot touch hardware directly. +-----------------------------------------+ | USER SPACE | | bash nginx python your_app | <- normal programs +-----------------------------------------+ | system calls | <- the only door +-----------------------------------------+ | KERNEL SPACE | | scheduler memory mgr drivers net | +-----------------------------------------+ | HARDWARE | | CPU RAM disk NIC | +-----------------------------------------+ A program in user space that wants to do something privileged — like read a file or open a network connection — must ask the kernel. It cannot just do it. 3. System calls: the boundary A system call (syscall) is a request from a user-space program to the kernel. It is the only official door between the two worlds. When your program calls open() to open a file, that becomes an openat syscall; the CPU switches into kernel mode, the kernel checks your permissions, does the work, and switches back. You don't usually write syscalls by hand — libraries and language runtimes do it for you — but you can watch them happen with strace : ubuntu@Jumpbox:~$ strace -e trace=openat cat /etc/hostname openat(AT_FDCWD, "/etc/hostname", O_RDONLY) = 3 Jumpbox That single line shows cat asking the kernel to open a file, and getting back 3 — a file descriptor , a small number the program uses to refer to the open file. This "ask the kernel, get a number back" pattern is everywhere in Linux. 4. Processes A process is a running program — one instance of a program plus its own memory, its open files, and a numeric ID called a PID (process ID). When you type ls , the kernel creates a process, runs it, and cleans it up when it exits. The kernel runs many processes seemingly at once by giving each a tiny slice of CPU time and switching between them thousands of times a second (a context switch ). You only have a few CPU cores, but you can have hundreds of processes. Try this: ubuntu@Jumpbox:~$ echo $$ # the PID of your shell 4711 ubuntu@Jumpbox:~$ ps -o pid,ppid,comm PID PPID COMMAND 4711 4710 bash 4820 4711 ps Notice ppid — the parent PID. Every process is started by another process, forming a tree. Your ps command's parent is your shell ( bash ). This tree all the way up traces back to the very first process the kernel starts at boot (PID 1), which on Ubuntu is systemd . You'll meet systemd properly in Chapter 5. 5. So what is "an operating system"? The kernel alone is not enough to be useful. An operating system (OS) — like Ubuntu, which runs on every VM in this lab — is the kernel plus the user-space pieces that make it usable: A shell ( bash ) to type commands into. Core utilities ( ls , cp , grep ) — small programs that do one job. System libraries (like the C library glibc ) that wrap syscalls so programs don't repeat themselves. A service manager (systemd) that starts background programs at boot. A package manager ( apt on Ubuntu) to install and update software. "Ubuntu" and "Debian" and "Fedora" all use the Linux kernel; what differs is the user-space software bundled around it. That bundle is called a distribution ("distro"). 6. Tying it together When you run cat /etc/hostname on the Jumpbox: Your shell (a user-space process) creates a new process for cat . cat makes an openat syscall — crossing into kernel space . The kernel checks permissions, reads the file from disk via a driver, hands back the bytes. cat makes a write syscall to print to your screen. cat exits; the kernel reaps the process. Every command you learn from here on is a variation of this dance: user-space programs politely asking the kernel — through syscalls — to do privileged work. Hold onto that picture. Dig deeper The Linux Kernel documentation (kernel.org) What is the Linux kernel? (Red Hat) syscalls(2) — the Linux system-call list (man7.org) strace man page (man7.org) Processes and threads (Linux man page: credentials) Search terms kernel space vs user space explained what is a system call linux linux process tree pid ppid strace beginner tutorial what is a linux distribution kernel difference Check yourself Name three things the kernel is responsible for. Why can't a normal program touch the network card directly? What is a system call, and why is it described as a "door"? What does a PID identify, and what is a PPID? What is the difference between "the Linux kernel" and "a Linux distribution"? Lesson: The Linux Directory Tree What you'll learn Why Linux has one tree of directories starting at / instead of drive letters. What lives in the most important directories: /etc , /var , /home , /usr , /bin , /tmp . What the "virtual" directories /proc and /dev are and why they aren't real files on disk. The big idea that "everything is a file" in Linux. Skill gained: the ability to guess where something lives on any Linux box, so you spend less time lost and more time fixing. The lesson On Windows you have C:\ and D:\ . Linux has none of that. There is exactly one tree, and it starts at the root directory , written / (just a forward slash). Every file and every disk lives somewhere inside that single tree. This layout is standardized by the Filesystem Hierarchy Standard (FHS) , which is why Ubuntu, Debian, Fedora, and the VMs in this lab all look broadly the same. 1. The single tree / the root of everything ├── bin -> usr/bin essential user commands (ls, cp, cat) ├── boot the kernel + bootloader files ├── dev devices (disks, terminals) as files ├── etc system-wide configuration (text files) ├── home per-user home directories │ └── ubuntu your home on the Jumpbox ├── lib -> usr/lib shared libraries ├── proc live kernel/process info (virtual) ├── root the root user's home (NOT /) ├── run runtime state since last boot ├── sbin -> usr/sbin system admin commands ├── srv data served by this host ├── tmp temporary scratch space ├── usr installed programs & their files └── var data that changes: logs, caches, spools Notice some entries have -> . Those are symbolic links (shortcuts). On modern Ubuntu, /bin is just a link into /usr/bin ; treat them as the same place. 2. Your starting point: navigating Two commands anchor you: ubuntu@Jumpbox:~$ pwd # "print working directory" — where am I? /home/ubuntu ubuntu@Jumpbox:~$ cd / # change directory to root ubuntu@Jumpbox:/$ ls bin boot dev etc home lib proc root run srv tmp usr var The ~ in your prompt is shorthand for your home directory ( /home/ubuntu ). cd with no argument always takes you home. A path starting with / is absolute (from the root). A path not starting with / is relative (from where you are now). . means "here" and .. means "one level up." 3. /etc — configuration /etc (pronounced "et-cee") holds system-wide configuration files , almost all plain text. When you configure a service, you edit a file here. ubuntu@Jumpbox:~$ cat /etc/hostname # this machine's name Jumpbox ubuntu@Jumpbox:~$ cat /etc/os-release # which distro & version NAME="Ubuntu" VERSION="24.04 LTS (Noble Numbat)" Because it's all text, configuration is easy to read, edit, version-control, and automate — a core DevOps idea. 4. /var — data that varies /var holds data that changes while the system runs. The most important sub-directory for you is /var/log , where programs write their logs. ubuntu@Jumpbox:~$ ls /var/log auth.log syslog journal dpkg.log ... In this lab these local logs are also shipped to a central Loki log server at 10.100.100.5 , so you can search every host's logs in one place. But knowing logs land in /var/log first is essential when you're on the box itself. Other /var residents: /var/lib (application state, e.g. databases), /var/cache (caches), /var/spool (queues like mail/print). 5. /home — where you live Each human user gets a directory under /home . Yours on the Jumpbox is /home/ubuntu . You own your files here and can do whatever you like. The special root user's home is not under /home — it is /root . Don't confuse /root (root's home) with / (the root of the tree). 6. /usr, /bin, /sbin — the programs When you run ls , where does that program live? ubuntu@Jumpbox:~$ which ls /usr/bin/ls /usr/bin — programs for everyone ( ls , grep , python3 ). /usr/sbin — programs mainly for administrators ( useradd , systemctl ). /usr/lib — shared libraries those programs depend on. Think of /usr ("Unix System Resources") as "everything that got installed." It can be large and is generally read-only during normal operation. 7. /tmp — scratch space /tmp is a place any program can write temporary files. It is usually wiped on reboot, so never store anything you want to keep there. It's perfect for a quick test: ubuntu@Jumpbox:~$ echo "scratch" > /tmp/note.txt ubuntu@Jumpbox:~$ cat /tmp/note.txt scratch 8. /proc and /dev — files that aren't really files Here's where Linux gets clever. Some directories don't hold real files on disk — they're virtual , generated by the kernel on the fly. /proc exposes live kernel and process info as if it were files. Each running process has a directory /proc/ : ubuntu@Jumpbox:~$ cat /proc/cpuinfo # details about the CPUs ubuntu@Jumpbox:~$ cat /proc/uptime # seconds since boot 123456.78 987654.32 /dev exposes hardware devices as files. Your disk might be /dev/sda , your terminal /dev/tty , and there are useful fake devices: ubuntu@Jumpbox:~$ ls -l /dev/null /dev/zero crw-rw-rw- 1 root root 1, 3 /dev/null # discards anything written to it crw-rw-rw- 1 root root 1, 5 /dev/zero # produces endless zero bytes /dev/null is the famous "black hole" — redirect output there to throw it away: command > /dev/null . 9. Everything is a file The reason /proc and /dev work is the central Unix idea: almost everything is presented as a file. A document, a hard disk, a running process's memory, even a network connection — you read and write them with the same handful of operations ( open , read , write , close ). This is why the small toolkit of commands you're about to learn is so powerful: the same cat , grep , and > work on real files and on these virtual ones. Learn the tools once, use them everywhere. 10. Putting it to use Suppose a teammate says "the web service config is broken." Your instincts should now be: configuration → /etc ; if it logged an error → /var/log (or central Loki); the program itself → /usr/bin or /usr/sbin ; live process info → /proc . You can navigate any Linux box without a map. Dig deeper Filesystem Hierarchy Standard — hier(7) man page Filesystem Hierarchy Standard 3.0 — the official spec Everything is a file (Linux Handbook) proc(5) — the /proc filesystem null(4) — the /dev/null special file (man7.org) Search terms linux filesystem hierarchy standard explained what is /etc /var /usr in linux everything is a file unix philosophy linux /proc filesystem beginner what is /dev/null used for Check yourself Why does Linux have no C:\ drive? What replaces it? Where would you look first for a service's configuration? For its logs? What is the difference between /root and / ? Why are /proc and /dev described as "virtual"? Explain in one sentence what "everything is a file" buys you. Lesson: The Everyday Command Toolkit What you'll learn Move around the filesystem confidently with pwd , cd , and ls . Create, copy, move, and delete files and directories — safely. Look inside files with cat , less , head , and tail . Search for files with find and search inside files with grep . Combine small tools with pipes , and get help with man and --help . Skill gained: the everyday muscle memory to actually drive a Linux box from the terminal. The lesson This is the chapter you'll use every single day. Each command is a small program that does one thing well; you combine them to get big results. Type along on the Jumpbox ( 10.100.100.254 , user ubuntu ) — reading is not enough, you must build muscle memory. 1. Knowing where you are: pwd, cd, ls ubuntu@Jumpbox:~$ pwd # where am I? /home/ubuntu ubuntu@Jumpbox:~$ ls # what's here? Desktop notes.txt scripts ubuntu@Jumpbox:~$ ls -l # long format: permissions, owner, size, date -rw-r--r-- 1 ubuntu ubuntu 42 Jun 1 10:00 notes.txt drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 1 10:00 scripts ubuntu@Jumpbox:~$ ls -la /etc # -a also shows hidden files (names starting with .) Useful ls flags: -l (long), -a (all, incl. hidden), -h (human-readable sizes like 4.0K ), -t (sort by time). Combine them: ls -lah . Move around with cd : ubuntu@Jumpbox:~$ cd /var/log # absolute path ubuntu@Jumpbox:/var/log$ cd .. # up one level -> /var ubuntu@Jumpbox:/var$ cd # no argument -> back home ubuntu@Jumpbox:~$ cd - # toggle to the previous directory 2. Making and removing: mkdir, rm, rmdir ubuntu@Jumpbox:~$ mkdir project # make a directory ubuntu@Jumpbox:~$ mkdir -p a/b/c # -p makes parents as needed ubuntu@Jumpbox:~$ rmdir project # remove an EMPTY directory ubuntu@Jumpbox:~$ rm a/b/c/file.txt # remove a file ubuntu@Jumpbox:~$ rm -r a # -r removes a directory and everything in it rm is permanent. There is no Recycle Bin. Before running rm -r , look at what you're about to delete. Never run rm -rf / or rm -rf ~ — those destroy the system or your home. A safety habit: ls the target first, then rm it. 3. Copying and moving: cp, mv ubuntu@Jumpbox:~$ cp notes.txt notes.bak # copy a file ubuntu@Jumpbox:~$ cp -r scripts scripts.bak # -r to copy a whole directory ubuntu@Jumpbox:~$ mv notes.bak archive/ # move into a directory ubuntu@Jumpbox:~$ mv oldname.txt newname.txt # mv also RENAMES mv does double duty: moving and renaming are the same operation. Like rm , mv and cp will silently overwrite an existing destination — add -i to be asked first. 4. Looking inside files: cat, less, head, tail ubuntu@Jumpbox:~$ cat /etc/hostname # dump a whole (small) file to screen Jumpbox ubuntu@Jumpbox:~$ less /var/log/syslog # scroll a BIG file page by page ubuntu@Jumpbox:~$ head -n 5 /var/log/syslog # first 5 lines ubuntu@Jumpbox:~$ tail -n 20 /var/log/syslog # last 20 lines ubuntu@Jumpbox:~$ tail -f /var/log/syslog # FOLLOW: show new lines as they arrive Use cat for short files. Use less for long ones — inside less , press Space to page down, /word to search, q to quit. tail -f is your friend for watching a log live (press Ctrl-C to stop). 5. Searching inside files: grep grep searches text for lines matching a pattern. It is probably the command you'll reach for most. ubuntu@Jumpbox:~$ grep "error" /var/log/syslog # lines containing "error" ubuntu@Jumpbox:~$ grep -i "error" /var/log/syslog # -i: case-insensitive ubuntu@Jumpbox:~$ grep -r "TODO" scripts/ # -r: search a whole directory ubuntu@Jumpbox:~$ grep -n "ssh" /etc/ssh/sshd_config # -n: show line numbers 6. Searching for files: find find locates files by name, type, age, size, and more. It walks a directory tree. ubuntu@Jumpbox:~$ find /etc -name "*.conf" # files ending in .conf under /etc ubuntu@Jumpbox:~$ find . -type d # directories under here ubuntu@Jumpbox:~$ find /var/log -name "*.log" -mtime -1 # .log files changed in last day Remember the difference: grep looks inside files; find looks for files. 7. Pipes: combining tools This is the heart of the Unix philosophy. A pipe ( | ) takes the output of one command and feeds it as the input of the next. Small tools become a powerful chain. ls -l /etc grep ".conf" wc -l +---------+ | +-----------+ | +------+ | output | -----> | filter | -----> | count| +---------+ +-----------+ +------+ ubuntu@Jumpbox:~$ ls /etc | grep conf # only entries containing "conf" ubuntu@Jumpbox:~$ cat /var/log/syslog | grep -i error | wc -l # count error lines ubuntu@Jumpbox:~$ ps aux | grep ssh # find ssh among running processes wc -l counts lines. You can chain as many stages as you like. Related symbols: > writes output to a file (overwriting), >> appends to a file. command > /dev/null throws output away. ubuntu@Jumpbox:~$ grep -i error /var/log/syslog > errors.txt # save matches to a file 8. Getting help: man and --help You will not memorize every flag. You don't need to — every command documents itself. ubuntu@Jumpbox:~$ ls --help # quick summary of flags ubuntu@Jumpbox:~$ man ls # full manual page (q to quit, /text to search) man ("manual") opens the full reference. Manuals are organized in sections; man 5 proc opens the section-5 page for proc . Reading man pages is a real DevOps skill — get comfortable with them now. 9. Tab completion: type less, err less Press the Tab key and the shell completes file names, directory names, and commands for you. Press Tab twice to list the options when there's more than one match. ubuntu@Jumpbox:~$ cd /var/lo # completes to /var/log/ ubuntu@Jumpbox:~$ systemc # completes to systemctl Tab completion prevents typos (a real cause of broken commands and bad rm s) and helps you discover what's available. Use it constantly. Also handy: the Up arrow recalls your previous commands, and Ctrl-R searches your command history. 10. A worked example Find every .conf file under /etc that mentions "ssh", showing line numbers: ubuntu@Jumpbox:~$ grep -rn "ssh" /etc --include="*.conf" Or count how many users have a home directory: ubuntu@Jumpbox:~$ ls /home | wc -l None of these is special knowledge — it's the same small tools, recombined. Master these ten and you can operate any Linux machine. Dig deeper The Linux command line for beginners (Ubuntu tutorial) grep — GNU manual How to use find (Red Hat sysadmin) Bash Manual — Pipelines & Redirections less man page Search terms linux basic commands cheat sheet grep vs find difference linux pipe and redirection tutorial tail -f follow log file how to read a man page Check yourself What is the difference between grep and find ? Why should you be careful with rm -r , and what habit reduces the risk? What does the pipe | do? Give an example of a two-stage pipe. How do you watch a log file update in real time? Name two ways to get help on a command without leaving the terminal. Lesson: Who Can Do What What you'll learn The difference between users , groups , and the all-powerful root account. Why you use sudo instead of logging in as root. How to read the rwx permission string that ls -l shows you. How to change permissions with chmod (both numeric and symbolic) and ownership with chown . Why permissions are the backbone of Linux security. Skill gained: the ability to read and fix "permission denied" errors instead of fearing them. The lesson Linux is a multi-user system: many people (and many programs) share one machine. Permissions are how Linux keeps them from stepping on each other and from breaking the system. Practise on the Jumpbox ( 10.100.100.254 , user ubuntu , which has passwordless sudo ). 1. Users and groups Every account on the system is a user , identified by a name and a numeric UID (user ID). Users are organized into groups (each with a GID ) so you can grant access to several people at once. ubuntu@Jumpbox:~$ whoami # who am I logged in as? ubuntu ubuntu@Jumpbox:~$ id # my UID, GID, and group memberships uid=1000(ubuntu) gid=1000(ubuntu) groups=1000(ubuntu),27(sudo) That 27(sudo) group membership is what lets ubuntu run administrative commands. User accounts are listed in /etc/passwd and groups in /etc/group — both plain text you can cat . 2. Root: the superuser root (UID 0) is the superuser . Root bypasses all permission checks — it can read, change, or delete anything . That power is also the danger: a careless command as root can wipe the machine. The golden rule: Log in as a normal user. Become root only for the moment you need it, using sudo . 3. sudo: borrowing root for one command sudo ("superuser do") runs a single command with root privileges, if your account is allowed to. It's the safe, auditable way to do admin work — every sudo use is logged (and on this lab those logs ship to the central Loki server at 10.100.100.5 ). ubuntu@Jumpbox:~$ cat /etc/shadow # password hashes — protected cat: /etc/shadow: Permission denied ubuntu@Jumpbox:~$ sudo cat /etc/shadow # run it as root root:!:19700:... On a normal machine sudo asks for your password (not root's). On the Jumpbox sudo is passwordless for convenience, but the habit is the same: prefix the one command that needs power with sudo — don't go become root for a whole session unless you must. 4. Reading permissions: ls -l Run ls -l and look at the first column: ubuntu@Jumpbox:~$ ls -l notes.txt -rw-r--r-- 1 ubuntu ubuntu 42 Jun 1 10:00 notes.txt That -rw-r--r-- is the permission string. Break it into pieces: - rw- r-- r-- ^ ^ ^ ^ type OWNER GROUP OTHERS (ubuntu)(ubuntu) (everyone else) First character = type: - regular file, d directory, l symbolic link. Then three groups of three: permissions for the owner , the group , and others (everyone else). In each group: r = read, w = write, x = execute. A - means that permission is off. So -rw-r--r-- means: owner can read & write; group can read; others can read. Nobody can execute it (it's data, not a program). What r/w/x mean differs slightly for directories : r = list the directory's contents. w = create/delete files inside it. x = enter ( cd into) the directory. 5. chmod, numeric style chmod ("change mode") sets permissions. The numeric (octal) style encodes each rwx triple as one digit: r = 4 w = 2 x = 1 (add them up) rwx = 4+2+1 = 7 rw- = 4+2 = 6 r-x = 4+1 = 5 r-- = 4 = 4 Three digits = owner, group, others: ubuntu@Jumpbox:~$ chmod 644 notes.txt # rw-r--r-- (owner rw, others r) — typical data file ubuntu@Jumpbox:~$ chmod 600 secret.txt # rw------- (only owner) — private file ubuntu@Jumpbox:~$ chmod 755 deploy.sh # rwxr-xr-x — a script everyone may run ubuntu@Jumpbox:~$ chmod 700 scripts/ # rwx------ — private directory Memorize three: 644 (normal files), 755 (programs & directories), 600 (private files). 6. chmod, symbolic style Sometimes you want to add or remove one permission without recalculating. Symbolic style uses u ser, g roup, o ther, a ll plus + / - / = : ubuntu@Jumpbox:~$ chmod +x deploy.sh # make it executable for everyone ubuntu@Jumpbox:~$ chmod u+x deploy.sh # executable for the owner only ubuntu@Jumpbox:~$ chmod g-w notes.txt # remove write from the group ubuntu@Jumpbox:~$ chmod o= secret.txt # remove ALL permissions from others chmod +x script.sh is one of the most common commands you'll type — Linux won't run a script as a program until it has the execute bit. 7. chown: changing ownership Every file has an owner (a user) and an owning group . chown ("change owner") sets them. Because changing ownership can give power away, it usually needs sudo . ubuntu@Jumpbox:~$ sudo chown ubuntu file.txt # set owner to ubuntu ubuntu@Jumpbox:~$ sudo chown ubuntu:ubuntu file.txt # owner AND group (user:group) ubuntu@Jumpbox:~$ sudo chown -R ubuntu:ubuntu app/ # -R: recurse into a directory A very common real-world fix: a service can't write to its data directory because the files are owned by root . The cure is to chown -R the directory to the service's user. 8. Why permissions matter +---------------------------------------------------+ | file: /etc/shadow owner=root mode=640 | | | | root -> read & write (it's the owner) | | shadow grp-> read (group can read) | | ubuntu -> DENIED (not owner, not grp) | +---------------------------------------------------+ Permissions are why a normal user can't read everyone's passwords, can't edit system config, and can't delete another user's files. When you see "Permission denied," don't panic — it's the system working. Ask three questions: Who am I? ( whoami , id ) Who owns the file and what are its permissions? ( ls -l ) Do I genuinely need elevated rights? (then sudo the one command) or should I fix ownership/mode? ( chown / chmod ) That diagnostic loop will resolve the vast majority of access problems you hit as a DevOps engineer. 9. A quick practice run ubuntu@Jumpbox:~$ echo 'echo hello' > hi.sh ubuntu@Jumpbox:~$ ./hi.sh bash: ./hi.sh: Permission denied # no execute bit yet ubuntu@Jumpbox:~$ chmod +x hi.sh ubuntu@Jumpbox:~$ ./hi.sh hello ubuntu@Jumpbox:~$ ls -l hi.sh -rwxr-xr-x 1 ubuntu ubuntu 11 Jun 1 10:00 hi.sh You just hit a permission error and fixed it deliberately. That's the whole game. Dig deeper Linux permissions explained (Red Hat sysadmin) chmod man page Understanding Linux file permissions (DigitalOcean) sudo man page Users and groups (Ubuntu Server docs) Search terms linux file permissions rwx explained chmod numeric vs symbolic chown user group recursive why use sudo instead of root linux permission denied how to fix Check yourself What does -rw-r--r-- mean for owner, group, and others? Why is it safer to use sudo than to log in as root? What numeric mode makes a file readable/writable by its owner and unreadable by everyone else? What does the execute ( x ) bit mean for a directory , versus a file? A service can't write to its data folder owned by root. Which command fixes it, and how? Lesson: What's Running and Why What you'll learn Inspect running programs with ps , top , and htop . Send signals to processes and stop them with kill . Understand systemd units and what a "service" really is. Drive services with systemctl — status, start, stop, restart, enable. Read logs with journalctl and know where logs live. Skill gained: the ability to answer "what's running, is it healthy, and why did it stop?" — the daily job of operations. The lesson A running Linux box is a swarm of processes . Most of the time you care about a handful: the services that do real work (a web server, a database). This chapter teaches you to see them, control them, and read what they're telling you. Practise on the Jumpbox ( 10.100.100.254 , user ubuntu ). 1. Seeing processes: ps Recall from Chapter 1: a process is a running program with a numeric PID . ps lists processes. The most common invocation: ubuntu@Jumpbox:~$ ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.1 168000 11000 ? Ss May01 0:12 /sbin/init ubuntu 4711 0.0 0.0 9000 5000 pts/0 Ss 10:00 0:00 bash ubuntu 4990 0.0 0.0 10000 3500 pts/0 R+ 10:05 0:00 ps aux a = all users, u = user-readable columns, x = include background processes. Key columns: PID , %CPU , %MEM , and COMMAND . Notice PID 1 ( /sbin/init , which is systemd ) — the ancestor of everything. To find a specific process, pipe into grep : ubuntu@Jumpbox:~$ ps aux | grep ssh 2. Live view: top and htop ps is a snapshot. top updates continuously, sorted by CPU use — perfect for "what's eating the machine right now?" ubuntu@Jumpbox:~$ top # press 'q' to quit, 'M' to sort by memory, 'P' by CPU htop is a friendlier, colored version with scrolling and a mouse. It may need installing: ubuntu@Jumpbox:~$ sudo apt install -y htop ubuntu@Jumpbox:~$ htop # F10 or q to quit 3. Signals and kill You control a process by sending it a signal — a small message the kernel delivers. kill sends signals (despite the name, most signals don't kill). ubuntu@Jumpbox:~$ kill 4990 # sends SIGTERM (15): "please shut down nicely" ubuntu@Jumpbox:~$ kill -9 4990 # sends SIGKILL (9): "die NOW" — last resort SIGTERM (15) — the polite default. The program can clean up first. Try this first. SIGKILL (9) — forceful, cannot be ignored, no cleanup. Use only if SIGTERM fails. SIGHUP (1) — often means "reload your config." Ctrl-C in your terminal sends SIGINT (2) to the foreground program. You can target by name with pkill nginx , but be careful you don't match more than you mean to. 4. What is a service? Enter systemd A service (also called a daemon ) is a program that runs in the background, usually started automatically at boot — a web server, a logging agent, an SSH server. On Ubuntu, all of these are managed by systemd , the system and service manager that runs as PID 1. systemd organizes everything into units . The kind you'll touch most is a service unit , named like ssh.service . A unit is described by a small text file (e.g. /lib/systemd/system/ssh.service or /etc/systemd/system/... ) that tells systemd how to start the program, when, and what to do if it crashes. +------------------- systemd (PID 1) -------------------+ | | | ssh.service nginx.service cron.service | | | | | | | starts starts starts | | v v v | | sshd process nginx process cron process | +-------------------------------------------------------+ 5. Driving services: systemctl systemctl is the command to control systemd units. The five you'll use constantly: ubuntu@Jumpbox:~$ systemctl status ssh # is it running? recent log lines? ubuntu@Jumpbox:~$ sudo systemctl start ssh # start it now ubuntu@Jumpbox:~$ sudo systemctl stop ssh # stop it now ubuntu@Jumpbox:~$ sudo systemctl restart ssh # stop then start (apply new config) ubuntu@Jumpbox:~$ sudo systemctl reload ssh # re-read config WITHOUT dropping connections A typical status output tells you a lot: ● ssh.service - OpenBSD Secure Shell server Loaded: loaded (/lib/systemd/system/ssh.service; enabled; preset: enabled) Active: active (running) since Mon 2026-06-01 09:00:00 UTC; 2h ago Main PID: 812 (sshd) Read it like this: Active: active (running) = healthy. enabled = it will start automatically at boot. If you see failed , the service crashed — go read its logs (next section). Enabled vs active trips up beginners: active = running right now. enabled = configured to start on boot. You set boot behavior separately: ubuntu@Jumpbox:~$ sudo systemctl enable ssh # start automatically at every boot ubuntu@Jumpbox:~$ sudo systemctl disable ssh # don't start at boot ubuntu@Jumpbox:~$ systemctl is-enabled ssh # check ubuntu@Jumpbox:~$ sudo systemctl enable --now ssh # enable AND start in one go 6. Reading logs: journalctl systemd captures the output of every service into a central log called the journal . journalctl reads it. ubuntu@Jumpbox:~$ journalctl -u ssh # all log lines for the ssh unit ubuntu@Jumpbox:~$ journalctl -u ssh -n 50 # last 50 lines ubuntu@Jumpbox:~$ journalctl -u ssh -f # FOLLOW live (like tail -f) ubuntu@Jumpbox:~$ journalctl -u ssh --since "1 hour ago" ubuntu@Jumpbox:~$ journalctl -p err -b # only errors, since this boot (-b) -u selects a unit, -n limits lines, -f follows, -p filters by priority, -b limits to the current boot. When a service won't start, the workflow is always: systemctl status to see that it failed, then journalctl -u -n 50 to see why . 7. Where logs live There are two complementary places: The systemd journal — read with journalctl (often stored under /var/log/journal ). This is the modern default on Ubuntu. Plain text files in /var/log — e.g. /var/log/syslog , /var/log/auth.log . Many services still also write here, and you read them with cat , less , grep , tail -f . On every host in this lab, all of these logs are also shipped to a central Loki log server at 10.100.100.5 , so you can search across the whole fleet from one place — invaluable when a problem spans several machines. But when you're logged into the box itself, journalctl and /var/log are right there. 8. The diagnostic workflow Put the whole chapter together. A teammate says "the SSH service seems down." You: ubuntu@Jumpbox:~$ systemctl status ssh # 1. is it active? enabled? ubuntu@Jumpbox:~$ journalctl -u ssh -n 50 # 2. why did it fail? read the errors ubuntu@Jumpbox:~$ ps aux | grep sshd # 3. is a process actually there? ubuntu@Jumpbox:~$ sudo systemctl restart ssh # 4. try a restart ubuntu@Jumpbox:~$ journalctl -u ssh -f # 5. watch it come back (or fail again) See it, control it, read what it says. That loop — ps / top to see, systemctl to control, journalctl to understand — is the core of day-to-day operations work. Everything more advanced builds on it. Dig deeper systemctl man page journalctl man page Understanding systemd units and unit files (DigitalOcean) Linux signals explained (Julia Evans zine post) DigitalOcean — How To Use journalctl to View and Manipulate Logs Search terms ps aux vs top vs htop linux kill signals sigterm sigkill systemctl status start enable explained journalctl -u -f follow service logs systemd service unit file beginner Check yourself What is the difference between a process being active and being enabled ? Which signal should you try first to stop a process, and which is the forceful last resort? What is a systemd "service unit," and what manages it? A service shows failed in systemctl status . What's your very next command? Name the two places service logs live on an Ubuntu host, and the command to read each. Assignment 1: Explore and explain a Linux host Goal: Prove you can navigate a real Linux machine, read its layout, and explain what you find — using only the commands from Chapters 1–4. Where: The Jumpbox ( 10.100.100.254 , user ubuntu , passwordless sudo ). SSH in from your workstation, then work in your home directory /home/ubuntu . Tasks SSH into the Jumpbox. Confirm who you are and where you are with whoami , id , and pwd . Identify the machine: capture the kernel version ( uname -r ), the distro and version ( cat /etc/os-release ), and the hostname ( cat /etc/hostname ). Make a working directory ~/m2-assignment1 and cd into it. Do all your file work here. Explore the filesystem hierarchy. For each of these directories, write one short sentence (in your own words) describing what it holds: /etc , /var/log , /home , /usr/bin , /tmp , /proc , /dev . Use ls to peek inside before you describe them. Investigate /proc : read /proc/uptime and /proc/cpuinfo . Note how many CPU cores the host has (hint: grep -c processor /proc/cpuinfo ). Use the toolkit: find every file under /etc whose name ends in .conf ( find ), then count how many there are by piping into wc -l . Search inside files: use grep to find which lines of /etc/ssh/sshd_config mention Port (use -n for line numbers). You will likely need sudo . Permissions: run ls -l /etc/shadow . Record its owner, group, and permission string, then explain in one sentence why a normal user gets "Permission denied" reading it. Create a file report.md in your working directory containing all your findings (use any editor, or nano report.md ). Deliverable A single file /home/ubuntu/m2-assignment1/report.md on the Jumpbox containing: the host identity (tasks 1–2), your one-sentence description of each directory (task 4), the CPU core count (task 5), the .conf file count (task 6), the Port grep output (task 7), and the /etc/shadow permission explanation (task 8). Acceptance criteria — you're done when: You can SSH into the Jumpbox and report.md exists at /home/ubuntu/m2-assignment1/report.md . The report states the kernel version, distro+version, and hostname. The report describes all seven directories ( /etc , /var/log , /home , /usr/bin , /tmp , /proc , /dev ) in your own words. The report gives the correct CPU core count for the Jumpbox. The report includes a .conf -file count produced via find ... | wc -l . The report shows the Port line(s) from sshd_config with line numbers. The report lists /etc/shadow 's owner, group, and mode, and correctly explains the "Permission denied" error in terms of ownership/permissions. Hints Stuck on a flag? command --help or man command — every tool documents itself. Use Tab completion for long paths like /etc/ssh/sshd_config to avoid typos. To save a command's output straight into your report: find /etc -name "*.conf" | wc -l >> report.md . "Permission denied" is not a bug — re-read Chapter 4, section 8, and ask the three diagnostic questions. Blocked for >~30 min after re-reading the lessons? Bring what you've tried to your mentor. Assignment 2: Manage a service and read its logs Goal: Prove you can inspect, control, and troubleshoot a systemd service and read its logs — the core operations loop from Chapter 5. Where: The Jumpbox ( 10.100.100.254 , user ubuntu , passwordless sudo ). Work in a new directory ~/m2-assignment2 . Tasks SSH into the Jumpbox and create ~/m2-assignment2 ; keep your notes file report.md there. List running processes with ps aux and again with top (quit top with q ). In your report, name the process with PID 1 and explain what it is. Pick the cron service (the scheduler, always present on Ubuntu) as your subject. Check its state with systemctl status cron . Record whether it is active and whether it is enabled , and explain the difference between those two words in your own words. Confirm boot behavior explicitly with systemctl is-enabled cron . Read the service's logs: journalctl -u cron -n 30 . Paste the last few lines into your report. Practise the control verbs (these are safe on cron ): sudo systemctl restart cron , then immediately re-check systemctl status cron and confirm the "Active: active (running) since ..." timestamp updated to just now. Watch logs live: run journalctl -u cron -f in your terminal, note that it follows, then stop it with Ctrl-C . Record what -f does. Find logs on disk too: list /var/log , then use tail -n 20 /var/log/syslog (with sudo if needed). In your report, name the two places service logs live on this host and state where they are also centrally shipped. Signals: start a harmless background sleep with sleep 600 & , note its PID ( echo $! or ps aux | grep sleep ), stop it with kill (SIGTERM), and confirm with ps that it's gone. Record the PID and the signal name SIGTERM corresponds to. Deliverable A file /home/ubuntu/m2-assignment2/report.md documenting tasks 2–9: the PID 1 process, cron's active/enabled state with the difference explained, its is-enabled result, a snippet of its journal, evidence the restart updated the start time, what journalctl -f does, the two log locations plus the central Loki destination, and the sleep-process PID you killed with the signal name. Acceptance criteria — you're done when: report.md exists at /home/ubuntu/m2-assignment2/report.md . The report identifies PID 1 as systemd ( /sbin/init ) and explains what it is. The report states cron's active and enabled status and correctly explains active vs enabled. The report includes a journalctl -u cron log snippet. The report shows evidence (the updated "Active ... since" timestamp) that systemctl restart cron worked. The report explains what journalctl -u cron -f does. The report names both log locations (the systemd journal and /var/log/... ) and states logs also ship to the central Loki server at 10.100.100.5 . The report records a sleep PID that you killed and identifies SIGTERM as signal 15. Hints The troubleshooting loop is always: systemctl status to see that something happened, then journalctl -u -n 50 to see why . A service can be active but disabled (running now, won't survive reboot) or enabled but inactive (will start at boot, not running now) — they're independent. & puts a command in the background; echo $! prints the PID of the last backgrounded job. Only cron is your subject for restart practice — don't restart ssh while you're connected through it, or you may interrupt your own session. Blocked for >~30 min after re-reading the lessons? Bring what you've tried to your mentor.