Operating System Full Course | Operating System Tutorials for Beginners

This section introduces operating systems, defining them as a layer of software that provides abstraction and arbitration services. Abstraction hides hardware details, allowing applications to run across various configurations, while arbitration manages shared hardware resources among multiple applications. Key hardware resources managed by an OS include CPU, memory, and I/O devices. The discussion highlights the history of OS, from early single-tasking systems to modern multi-application environments, and defines the kernel as the core component for hardware sharing. It also covers different types of operating systems, with a focus on Windows and various Unix-like systems, including Linux distributions such as Red Hat, Debian, Ubuntu, and Arch Linux.

This part focuses on disk drives and their management, covering disk attachment via various bus types like IDE/PATA, SCSI, SATA, and SAS. It explains the physical properties of magnetic disks, including sectors, tracks, and cylinders, and the evolution of disk addressing from CHS to logical block addressing (LBA). The lecture also details disk partitioning schemes, Master Boot Record (MBR) and GUID Partition Table (GPT), and introduces Solid-State Drives (SSDs), discussing their advantages, NAND flash memory, wear leveling, and the 'trim' command for optimized performance and longevity, as well as the challenges of data security on SSDs.

This section dives into disk scheduling algorithms, explaining their purpose: arbitrating disk access and improving performance by reducing seek times. It covers historical algorithms like First-Come First-Serve (FIFO), Scan (Elevator), C-Scan, Look, C-Look, and Shortest Seek Time First (SSTF). The lecture then discusses modern approaches, including Linux's Noop, Anticipatory, Deadline, and Completely Fair Queuing (CFQ) schedulers. For SSDs, it emphasizes that traditional seek-time optimization algorithms are counterproductive, recommending Noop or Deadline for optimal performance.

This segment introduces file systems as mechanisms for organizing data on persistent storage. It covers file system responsibilities like data layout, reliable retrieval, providing abstractions for disk space, and managing metadata (file name, size, ownership, permissions, timestamps). The process of formatting, fragmentation, and journaling for data integrity is explained. Different internal layouts, such as File Allocation Table (FAT), inodes, and extents, are discussed, along with the concepts of mounting and unmounting file systems to make them accessible to users, contrasting UNIX mount points with Windows drive letters.

This part covers CPU features essential for supporting multiprogramming. It defines multiprogramming as running multiple processes simultaneously by rapidly switching between them. Key hardware requirements include interrupt mechanisms for preemption, a clock for timekeeping, and CPU protection levels to restrict access to sensitive operations and memory. The lecture explains CPU privilege modes (kernel/user mode) and x86 protection rings (0-3), noting that most OS use only rings 0 and 3. It also touches on mode switches and interrupts (involuntary and voluntary) as fundamental to OS event handling.

This section discusses different kernel architectures. It defines the kernel's dual functions: abstraction and arbitration. It distinguishes between mechanism (how operations are carried out) and policy (rules for access and limits). The historical RC 4000 monitor (a microkernel example) and the UNIX kernel (a monolithic kernel example) are presented. The characteristics, advantages (performance for monolithic), and disadvantages (complexity, instability for monolithic, performance for microkernel) of monolithic and microkernel architectures are explored. It concludes by noting that most modern kernels are hybrids, balancing performance and modularity.

This segment covers the basics of using the Linux command line interface. It explains how to navigate the file system using commands like `pwd` (print working directory), `cd` (change directory), and `ls` (list contents). It differentiates between absolute and relative paths, and introduces special path components like `.` (current directory) and `..` (parent directory). A detailed overview of the root directory and key top-level directories (`/bin`, `/dev`, `/etc`, `/opt`, `/usr`, `/var`, `/lib`, `/media`, `/mnt`, `/root`, `/sbin`, `/tmp`) is provided. It also touches on configuration files and the `man` command for accessing documentation.

This part continues the discussion on basic Linux system utilization, focusing on file ownership and permissions. It explains how to view file information with `ls -l` and change ownership (`chown`), group (`chgrp`), and permissions (`chmod`). Permission types (read, write, execute) and user classes (owner, group, world) are detailed. It also covers commands for managing processes (`ps`, `top`), identifying dynamic libraries (`ldd`), finding program paths (`which`), and introduces the superuser (`root`) and the `sudo` command for administrative tasks.

This section explains interrupts and their role in device input/output. It contrasts polling (CPU constantly checking devices) with interrupts (devices signaling the CPU when attention is needed). The advantages of interrupts (responsiveness, efficient CPU usage) and their implementation challenges are discussed. It details how interrupts are integrated into the CPU's fetch-execute cycle, involving mode switching, saving program counters, and using an interrupt vector table (IVT) to locate handlers. The concept of fast and slow interrupt handlers, atomic operations, and the potential for interrupt storms and live locks are also covered.

This part delves into interrupt controllers, differentiating between the old Programmable Interrupt Controller (PIC) and the newer Advanced Programmable Interrupt Controller (APIC) with message-signaled interrupts (MSI/MSI-X). It explains the PIC's dual-chip architecture, limited IRQ lines, and their direct physical connections, highlighting the issues of IRQ sharing and hardware conflicts. In contrast, APIC uses memory-mapped registers and a shared bus for message-signaled interrupts, solving sharing issues and improving efficiency, particularly for PCI Express devices. The role of direct memory access (DMA) in modern systems is also mentioned.

This segment introduces memory management, focusing on Random Access Memory (RAM) and how processes view it. It defines RAM as a volatile, dedicated hardware memory, contrasting it with persistent storage. It explains how the OS perceives memory as a large block of addressable bytes, discussing the limitations of 32-bit vs. 64-bit architectures. The memory layout of a process (stack, heap, global variables, text segment) is detailed, along with how different programming languages manage heap allocation. It highlights the kernel's self-contained memory management and the challenges of sharing memory between the kernel and user processes, leading to fixed-size memory partitioning as a basic sharing method.

This section focuses on dynamic memory allocation at the system level. It contrasts dynamic allocation with fixed memory partitioning, highlighting its benefits for multiprogramming but also its increased complexity. Key issues discussed include efficiency, free space tracking, allocation algorithms, and memory fragmentation (external and internal). External fragmentation occurs when free space is broken into non-contiguous pieces, while internal fragmentation happens when allocated blocks are larger than requested. The lecture introduces classical allocation algorithms: Best Fit, Worst Fit, First Fit, and Next Fit, analyzing their trade-offs in terms of fragmentation and search efficiency.

This part introduces advanced memory allocation techniques used in OS kernels. It explains 'power of two methods,' such as the buddy system, which use binary trees to track allocated and free blocks, improving speed and reducing external fragmentation through 'coalescence.' Trade-offs in choosing block sizes (U and L) are discussed. For kernel memory, where minimal fragmentation is crucial, 'slab allocation' is presented. This method pre-allocates fixed-size slabs for specific kernel data structures into caches, optimizing space utilization and allocation speed at compile time. It also details the evolution of Linux slab allocators, including the original 'slab' and the more efficient 'slub' allocator.

This section introduces paging as a mechanism for sharing memory, improving upon simpler algorithms by allocating fixed-sized 'pages' to processes. It explains how each process gets its own logical memory space, with logical addresses being translated by the Memory Management Unit (MMU) to physical addresses in 'frames.' Key advantages include eliminating external fragmentation and enabling non-contiguous physical storage. The lecture details the address translation process using page tables and the role of the Translation Lookaside Buffer (TLB) in speeding up this process, differentiating between TLB hits and misses.

This part continues the discussion on paging, focusing on page tables and advanced schemes. It covers how CPUs manage and search page tables (hardware vs. software managed) and introduces hierarchical page tables for reducing search time, where logical addresses are divided into multiple components for multi-level lookups. It then discusses hashed page tables as a solution for large address spaces in 64-bit systems, where a hash function maps page numbers to inner linear page tables. Finally, it introduces inverted page tables (less common) and Extended Page Tables (EPT) or Rapid Virtualization Indexing (RVI) for virtualized environments, explaining how they enable direct translation from guest to host physical frames without hypervisor intervention.

This lecture focuses on memory protection mechanisms, essential for arbitrating RAM access among processes. It explains how operating systems prevent processes from accessing memory that doesn't belong to them or from crashing the system. On non-paging systems, segmentation divides a process's memory into logical segments (text, global variables, stack, heap), with access permissions applied to each segment and enforced by a segment table. On modern paging systems, memory protection is achieved by adding permission bits (valid/invalid, read/write, no execute/NX bit) to page table entries. The NX bit, in particular, enhances security against buffer overflow attacks by preventing code execution from data memory regions. It also highlights how read-only shared pages enable memory savings by allowing multiple processes to share program code, beneficial for applications like web browsers.

This section introduces virtual memory, explaining how it further increases multiprogramming by allowing unused pages to be swapped out to a backing store (like a hard disk). This enables processes to have logical memory spaces larger than physical RAM. The address translation process remains similar, but now includes detecting if a page is in RAM or swapped out, leading to 'page faults' that the OS handles to bring the page back. It clarifies the terms 'swapping' and 'paging,' discusses swap partitions and files, and the implications for SSDs (wear limits) and data security (forensic recovery). It also introduces compressed RAM disks (zRAM/compcache) as an alternative backing store, offering faster performance and enhanced security.

This continues the virtual memory discussion, focusing on paging performance and related concepts. It explains how high page fault rates can lead to 'thrashing,' drastically reducing system performance. It discusses virtual memory 'fetch policies,' contrasting 'demand paging' (loading pages only when needed) with 'prefetching' (loading pages before needed). Pure demand paging is identified as efficient for embedded systems and power-saving. The lecture then covers 'copy-on-write,' a technique that optimizes process cloning by initially sharing memory pages and making copies only when a modification occurs, significantly improving performance in UNIX-like systems when processes fork. Finally, it explores 'memory-mapped files' for improved I/O performance and the use of 'shared libraries' for memory efficiency among multiple programs.

This lecture delves into page replacement algorithms used in virtual memory to manage memory when demand exceeds physical RAM. It discusses the need to select a 'victim frame' for swapping out and how 'present' and 'dirty' bits in page tables aid this decision. It differentiates between 'global' (any page can be replaced) and 'local' (only process-owned pages) replacement policies, noting the complexity of local policies due to 'Belady's anomaly.' Various classical algorithms are examined: Random, First-In First-Out (FIFO), Least Frequently Used (LFU), Most Frequently Used (MFU), Least Recently Used (LRU) - practical but hard to implement directly, and the theoretically optimal but impossible-to-implement OPT algorithm. It concludes with LRU approximations like Not Used Recently (NUR).

This part introduces processes in operating systems, defining them as instances of executing programs. It covers the fundamental information associated with a process, including memory segments (text, data, stack, heap, shared libraries), process ID, owner information, and various statistics maintained by the OS for scheduling. The different states a process transitions through during its lifetime (new, ready, running, waiting, terminated) are explained with a state diagram. It details process creation on UNIX-like systems via the 'fork' system call, where a child process is a clone of its parent. The concept of orphan processes and daemons is also introduced.

This section continues the discussion on processes, focusing on the `fork`, `exec`, and `wait` system calls. It provides C and Python examples demonstrating how `fork` creates a child process (with a `pid` of 0 in the child and the child's `pid` in the parent). The `exec` system call, which replaces the current process's code with a new program, is explained in detail, including its various versions (e.g., `execl`, `execv`, `execle`) and their conventions (e.g., first argument being the program name). It emphasizes that code after `exec` will not run. Finally, the `wait` system call is covered, enabling a parent process to pause its execution until a child process terminates, with examples in C and Python, and a note on orphan processes and daemons.

This part focuses on process management, detailing process contexts, context switches, and process scheduling. It defines 'process context' as the minimal state information needed to stop and restart a process (CPU registers, program counter, RAM contents). 'Context switches' (or process switches) are explained as expensive operations involving mode switches into the kernel, saving the old process's state, and restoring the new process's state. It outlines the process lifecycle flow from creation (via `fork`) to scheduling by the dispatcher, and how blocking I/O operations lead to processes yielding CPU resources. The concept of CPU-bound processes and the need for preemption by the CPU scheduler or hardware interrupts are discussed. Finally, it introduces Process Control Blocks (PCBs) as data structures storing process information, their organization in linked lists (job queue, ready list, device queues), and a high-level overview of job, midterm, and CPU scheduling.