diff --git a/docs-en/chapter_array_and_linkedlist/array.md b/docs-en/chapter_array_and_linkedlist/array.md new file mode 100755 index 000000000..e9ccf6ced --- /dev/null +++ b/docs-en/chapter_array_and_linkedlist/array.md @@ -0,0 +1,1230 @@ +--- +comments: true +--- + +# 4.1 Arrays + +The "array" is a linear data structure that stores elements of the same type in contiguous memory locations. We refer to the position of an element in the array as its "index". The following image illustrates the main terminology and concepts of an array. + +{ class="animation-figure" } + +
Figure 4-1 Array Definition and Storage Method
+ +## 4.1.1 Common Operations on Arrays + +### 1. Initializing Arrays + +There are two ways to initialize arrays depending on the requirements: without initial values and with given initial values. In cases where initial values are not specified, most programming languages will initialize the array elements to $0$: + +=== "Python" + + ```python title="array.py" + # Initialize array + arr: list[int] = [0] * 5 # [ 0, 0, 0, 0, 0 ] + nums: list[int] = [1, 3, 2, 5, 4] + ``` + +=== "C++" + + ```cpp title="array.cpp" + /* Initialize array */ + // Stored on stack + int arr[5]; + int nums[5] = { 1, 3, 2, 5, 4 }; + // Stored on heap (manual memory release needed) + int* arr1 = new int[5]; + int* nums1 = new int[5] { 1, 3, 2, 5, 4 }; + ``` + +=== "Java" + + ```java title="array.java" + /* Initialize array */ + int[] arr = new int[5]; // { 0, 0, 0, 0, 0 } + int[] nums = { 1, 3, 2, 5, 4 }; + ``` + +=== "C#" + + ```csharp title="array.cs" + /* Initialize array */ + int[] arr = new int[5]; // { 0, 0, 0, 0, 0 } + int[] nums = [1, 3, 2, 5, 4]; + ``` + +=== "Go" + + ```go title="array.go" + /* Initialize array */ + var arr [5]int + // In Go, specifying the length ([5]int) denotes an array, while not specifying it ([]int) denotes a slice. + // Since Go's arrays are designed to have compile-time fixed length, only constants can be used to specify the length. + // For convenience in implementing the extend() method, the Slice will be considered as an Array here. + nums := []int{1, 3, 2, 5, 4} + ``` + +=== "Swift" + + ```swift title="array.swift" + /* Initialize array */ + let arr = Array(repeating: 0, count: 5) // [0, 0, 0, 0, 0] + let nums = [1, 3, 2, 5, 4] + ``` + +=== "JS" + + ```javascript title="array.js" + /* Initialize array */ + var arr = new Array(5).fill(0); + var nums = [1, 3, 2, 5, 4]; + ``` + +=== "TS" + + ```typescript title="array.ts" + /* Initialize array */ + let arr: number[] = new Array(5).fill(0); + let nums: number[] = [1, 3, 2, 5, 4]; + ``` + +=== "Dart" + + ```dart title="array.dart" + /* Initialize array */ + ListFigure 4-2 Memory Address Calculation for Array Elements
+ +As observed in the above image, the index of the first element of an array is $0$, which may seem counterintuitive since counting starts from $1$. However, from the perspective of the address calculation formula, **an index is essentially an offset from the memory address**. The offset for the first element's address is $0$, making its index $0$ logical. + +Accessing elements in an array is highly efficient, allowing us to randomly access any element in $O(1)$ time. + +=== "Python" + + ```python title="array.py" + def random_access(nums: list[int]) -> int: + """随机访问元素""" + # 在区间 [0, len(nums)-1] 中随机抽取一个数字 + random_index = random.randint(0, len(nums) - 1) + # 获取并返回随机元素 + random_num = nums[random_index] + return random_num + ``` + +=== "C++" + + ```cpp title="array.cpp" + /* 随机访问元素 */ + int randomAccess(int *nums, int size) { + // 在区间 [0, size) 中随机抽取一个数字 + int randomIndex = rand() % size; + // 获取并返回随机元素 + int randomNum = nums[randomIndex]; + return randomNum; + } + ``` + +=== "Java" + + ```java title="array.java" + /* 随机访问元素 */ + int randomAccess(int[] nums) { + // 在区间 [0, nums.length) 中随机抽取一个数字 + int randomIndex = ThreadLocalRandom.current().nextInt(0, nums.length); + // 获取并返回随机元素 + int randomNum = nums[randomIndex]; + return randomNum; + } + ``` + +=== "C#" + + ```csharp title="array.cs" + /* 随机访问元素 */ + int RandomAccess(int[] nums) { + Random random = new(); + // 在区间 [0, nums.Length) 中随机抽取一个数字 + int randomIndex = random.Next(nums.Length); + // 获取并返回随机元素 + int randomNum = nums[randomIndex]; + return randomNum; + } + ``` + +=== "Go" + + ```go title="array.go" + /* 随机访问元素 */ + func randomAccess(nums []int) (randomNum int) { + // 在区间 [0, nums.length) 中随机抽取一个数字 + randomIndex := rand.Intn(len(nums)) + // 获取并返回随机元素 + randomNum = nums[randomIndex] + return + } + ``` + +=== "Swift" + + ```swift title="array.swift" + /* 随机访问元素 */ + func randomAccess(nums: [Int]) -> Int { + // 在区间 [0, nums.count) 中随机抽取一个数字 + let randomIndex = nums.indices.randomElement()! + // 获取并返回随机元素 + let randomNum = nums[randomIndex] + return randomNum + } + ``` + +=== "JS" + + ```javascript title="array.js" + /* 随机访问元素 */ + function randomAccess(nums) { + // 在区间 [0, nums.length) 中随机抽取一个数字 + const random_index = Math.floor(Math.random() * nums.length); + // 获取并返回随机元素 + const random_num = nums[random_index]; + return random_num; + } + ``` + +=== "TS" + + ```typescript title="array.ts" + /* 随机访问元素 */ + function randomAccess(nums: number[]): number { + // 在区间 [0, nums.length) 中随机抽取一个数字 + const random_index = Math.floor(Math.random() * nums.length); + // 获取并返回随机元素 + const random_num = nums[random_index]; + return random_num; + } + ``` + +=== "Dart" + + ```dart title="array.dart" + /* 随机访问元素 */ + int randomAccess(ListFigure 4-3 Array Element Insertion Example
+ +It's important to note that since the length of an array is fixed, inserting an element will inevitably lead to the loss of the last element in the array. We will discuss solutions to this problem in the "List" chapter. + +=== "Python" + + ```python title="array.py" + def insert(nums: list[int], num: int, index: int): + """在数组的索引 index 处插入元素 num""" + # 把索引 index 以及之后的所有元素向后移动一位 + for i in range(len(nums) - 1, index, -1): + nums[i] = nums[i - 1] + # 将 num 赋给 index 处的元素 + nums[index] = num + ``` + +=== "C++" + + ```cpp title="array.cpp" + /* 在数组的索引 index 处插入元素 num */ + void insert(int *nums, int size, int num, int index) { + // 把索引 index 以及之后的所有元素向后移动一位 + for (int i = size - 1; i > index; i--) { + nums[i] = nums[i - 1]; + } + // 将 num 赋给 index 处的元素 + nums[index] = num; + } + ``` + +=== "Java" + + ```java title="array.java" + /* 在数组的索引 index 处插入元素 num */ + void insert(int[] nums, int num, int index) { + // 把索引 index 以及之后的所有元素向后移动一位 + for (int i = nums.length - 1; i > index; i--) { + nums[i] = nums[i - 1]; + } + // 将 num 赋给 index 处的元素 + nums[index] = num; + } + ``` + +=== "C#" + + ```csharp title="array.cs" + /* 在数组的索引 index 处插入元素 num */ + void Insert(int[] nums, int num, int index) { + // 把索引 index 以及之后的所有元素向后移动一位 + for (int i = nums.Length - 1; i > index; i--) { + nums[i] = nums[i - 1]; + } + // 将 num 赋给 index 处的元素 + nums[index] = num; + } + ``` + +=== "Go" + + ```go title="array.go" + /* 在数组的索引 index 处插入元素 num */ + func insert(nums []int, num int, index int) { + // 把索引 index 以及之后的所有元素向后移动一位 + for i := len(nums) - 1; i > index; i-- { + nums[i] = nums[i-1] + } + // 将 num 赋给 index 处的元素 + nums[index] = num + } + ``` + +=== "Swift" + + ```swift title="array.swift" + /* 在数组的索引 index 处插入元素 num */ + func insert(nums: inout [Int], num: Int, index: Int) { + // 把索引 index 以及之后的所有元素向后移动一位 + for i in nums.indices.dropFirst(index).reversed() { + nums[i] = nums[i - 1] + } + // 将 num 赋给 index 处的元素 + nums[index] = num + } + ``` + +=== "JS" + + ```javascript title="array.js" + /* 在数组的索引 index 处插入元素 num */ + function insert(nums, num, index) { + // 把索引 index 以及之后的所有元素向后移动一位 + for (let i = nums.length - 1; i > index; i--) { + nums[i] = nums[i - 1]; + } + // 将 num 赋给 index 处的元素 + nums[index] = num; + } + ``` + +=== "TS" + + ```typescript title="array.ts" + /* 在数组的索引 index 处插入元素 num */ + function insert(nums: number[], num: number, index: number): void { + // 把索引 index 以及之后的所有元素向后移动一位 + for (let i = nums.length - 1; i > index; i--) { + nums[i] = nums[i - 1]; + } + // 将 num 赋给 index 处的元素 + nums[index] = num; + } + ``` + +=== "Dart" + + ```dart title="array.dart" + /* 在数组的索引 index 处插入元素 _num */ + void insert(ListFigure 4-4 Array Element Deletion Example
+ +Note that after deletion, the last element becomes "meaningless", so we do not need to specifically modify it. + +=== "Python" + + ```python title="array.py" + def remove(nums: list[int], index: int): + """删除索引 index 处的元素""" + # 把索引 index 之后的所有元素向前移动一位 + for i in range(index, len(nums) - 1): + nums[i] = nums[i + 1] + ``` + +=== "C++" + + ```cpp title="array.cpp" + /* 删除索引 index 处的元素 */ + void remove(int *nums, int size, int index) { + // 把索引 index 之后的所有元素向前移动一位 + for (int i = index; i < size - 1; i++) { + nums[i] = nums[i + 1]; + } + } + ``` + +=== "Java" + + ```java title="array.java" + /* 删除索引 index 处的元素 */ + void remove(int[] nums, int index) { + // 把索引 index 之后的所有元素向前移动一位 + for (int i = index; i < nums.length - 1; i++) { + nums[i] = nums[i + 1]; + } + } + ``` + +=== "C#" + + ```csharp title="array.cs" + /* 删除索引 index 处的元素 */ + void Remove(int[] nums, int index) { + // 把索引 index 之后的所有元素向前移动一位 + for (int i = index; i < nums.Length - 1; i++) { + nums[i] = nums[i + 1]; + } + } + ``` + +=== "Go" + + ```go title="array.go" + /* 删除索引 index 处的元素 */ + func remove(nums []int, index int) { + // 把索引 index 之后的所有元素向前移动一位 + for i := index; i < len(nums)-1; i++ { + nums[i] = nums[i+1] + } + } + ``` + +=== "Swift" + + ```swift title="array.swift" + /* 删除索引 index 处的元素 */ + func remove(nums: inout [Int], index: Int) { + // 把索引 index 之后的所有元素向前移动一位 + for i in nums.indices.dropFirst(index).dropLast() { + nums[i] = nums[i + 1] + } + } + ``` + +=== "JS" + + ```javascript title="array.js" + /* 删除索引 index 处的元素 */ + function remove(nums, index) { + // 把索引 index 之后的所有元素向前移动一位 + for (let i = index; i < nums.length - 1; i++) { + nums[i] = nums[i + 1]; + } + } + ``` + +=== "TS" + + ```typescript title="array.ts" + /* 删除索引 index 处的元素 */ + function remove(nums: number[], index: number): void { + // 把索引 index 之后的所有元素向前移动一位 + for (let i = index; i < nums.length - 1; i++) { + nums[i] = nums[i + 1]; + } + } + ``` + +=== "Dart" + + ```dart title="array.dart" + /* 删除索引 index 处的元素 */ + void remove(ListFigure 4-5 Linked List Definition and Storage Method
+ +Observing the image above, the fundamental unit of a linked list is the "node" object. Each node contains two pieces of data: the "value" of the node and the "reference" to the next node. + +- The first node of a linked list is known as the "head node", and the last one is called the "tail node". +- The tail node points to "null", which is represented as $\text{null}$ in Java, $\text{nullptr}$ in C++, and $\text{None}$ in Python. +- In languages that support pointers, like C, C++, Go, and Rust, the aforementioned "reference" should be replaced with a "pointer". + +As shown in the following code, a linked list node `ListNode`, apart from containing a value, also needs to store a reference (pointer). Therefore, **a linked list consumes more memory space than an array for the same amount of data**. + +=== "Python" + + ```python title="" + class ListNode: + """Linked List Node Class""" + def __init__(self, val: int): + self.val: int = val # Node value + self.next: ListNode | None = None # Reference to the next node + ``` + +=== "C++" + + ```cpp title="" + /* Linked List Node Structure */ + struct ListNode { + int val; // Node value + ListNode *next; // Pointer to the next node + ListNode(int x) : val(x), next(nullptr) {} // Constructor + }; + ``` + +=== "Java" + + ```java title="" + /* Linked List Node Class */ + class ListNode { + int val; // Node value + ListNode next; // Reference to the next node + ListNode(int x) { val = x; } // Constructor + } + ``` + +=== "C#" + + ```csharp title="" + /* Linked List Node Class */ + class ListNode(int x) { // Constructor + int val = x; // Node value + ListNode? next; // Reference to the next node + } + ``` + +=== "Go" + + ```go title="" + /* Linked List Node Structure */ + type ListNode struct { + Val int // Node value + Next *ListNode // Pointer to the next node + } + + // NewListNode Constructor, creates a new linked list + func NewListNode(val int) *ListNode { + return &ListNode{ + Val: val, + Next: nil, + } + } + ``` + +=== "Swift" + + ```swift title="" + /* Linked List Node Class */ + class ListNode { + var val: Int // Node value + var next: ListNode? // Reference to the next node + + init(x: Int) { // Constructor + val = x + } + } + ``` + +=== "JS" + + ```javascript title="" + /* Linked List Node Class */ + class ListNode { + constructor(val, next) { + this.val = (val === undefined ? 0 : val); // Node value + this.next = (next === undefined ? null : next); // Reference to the next node + } + } + ``` + +=== "TS" + + ```typescript title="" + /* Linked List Node Class */ + class ListNode { + val: number; + next: ListNode | null; + constructor(val?: number, next?: ListNode | null) { + this.val = val === undefined ? 0 : val; // Node value + this.next = next === undefined ? null : next; // Reference to the next node + } + } + ``` + +=== "Dart" + + ```dart title="" + /* 链表节点类 */ + class ListNode { + int val; // Node value + ListNode? next; // Reference to the next node + ListNode(this.val, [this.next]); // Constructor + } + ``` + +=== "Rust" + + ```rust title="" + use std::rc::Rc; + use std::cell::RefCell; + /* Linked List Node Class */ + #[derive(Debug)] + struct ListNode { + val: i32, // Node value + next: OptionFigure 4-6 Linked List Node Insertion Example
+ +=== "Python" + + ```python title="linked_list.py" + def insert(n0: ListNode, P: ListNode): + """在链表的节点 n0 之后插入节点 P""" + n1 = n0.next + P.next = n1 + n0.next = P + ``` + +=== "C++" + + ```cpp title="linked_list.cpp" + /* 在链表的节点 n0 之后插入节点 P */ + void insert(ListNode *n0, ListNode *P) { + ListNode *n1 = n0->next; + P->next = n1; + n0->next = P; + } + ``` + +=== "Java" + + ```java title="linked_list.java" + /* 在链表的节点 n0 之后插入节点 P */ + void insert(ListNode n0, ListNode P) { + ListNode n1 = n0.next; + P.next = n1; + n0.next = P; + } + ``` + +=== "C#" + + ```csharp title="linked_list.cs" + /* 在链表的节点 n0 之后插入节点 P */ + void Insert(ListNode n0, ListNode P) { + ListNode? n1 = n0.next; + P.next = n1; + n0.next = P; + } + ``` + +=== "Go" + + ```go title="linked_list.go" + /* 在链表的节点 n0 之后插入节点 P */ + func insertNode(n0 *ListNode, P *ListNode) { + n1 := n0.Next + P.Next = n1 + n0.Next = P + } + ``` + +=== "Swift" + + ```swift title="linked_list.swift" + /* 在链表的节点 n0 之后插入节点 P */ + func insert(n0: ListNode, P: ListNode) { + let n1 = n0.next + P.next = n1 + n0.next = P + } + ``` + +=== "JS" + + ```javascript title="linked_list.js" + /* 在链表的节点 n0 之后插入节点 P */ + function insert(n0, P) { + const n1 = n0.next; + P.next = n1; + n0.next = P; + } + ``` + +=== "TS" + + ```typescript title="linked_list.ts" + /* 在链表的节点 n0 之后插入节点 P */ + function insert(n0: ListNode, P: ListNode): void { + const n1 = n0.next; + P.next = n1; + n0.next = P; + } + ``` + +=== "Dart" + + ```dart title="linked_list.dart" + /* 在链表的节点 n0 之后插入节点 P */ + void insert(ListNode n0, ListNode P) { + ListNode? n1 = n0.next; + P.next = n1; + n0.next = P; + } + ``` + +=== "Rust" + + ```rust title="linked_list.rs" + /* 在链表的节点 n0 之后插入节点 P */ + #[allow(non_snake_case)] + pub fn insertFigure 4-7 Linked List Node Deletion
+ +=== "Python" + + ```python title="linked_list.py" + def remove(n0: ListNode): + """删除链表的节点 n0 之后的首个节点""" + if not n0.next: + return + # n0 -> P -> n1 + P = n0.next + n1 = P.next + n0.next = n1 + ``` + +=== "C++" + + ```cpp title="linked_list.cpp" + /* 删除链表的节点 n0 之后的首个节点 */ + void remove(ListNode *n0) { + if (n0->next == nullptr) + return; + // n0 -> P -> n1 + ListNode *P = n0->next; + ListNode *n1 = P->next; + n0->next = n1; + // 释放内存 + delete P; + } + ``` + +=== "Java" + + ```java title="linked_list.java" + /* 删除链表的节点 n0 之后的首个节点 */ + void remove(ListNode n0) { + if (n0.next == null) + return; + // n0 -> P -> n1 + ListNode P = n0.next; + ListNode n1 = P.next; + n0.next = n1; + } + ``` + +=== "C#" + + ```csharp title="linked_list.cs" + /* 删除链表的节点 n0 之后的首个节点 */ + void Remove(ListNode n0) { + if (n0.next == null) + return; + // n0 -> P -> n1 + ListNode P = n0.next; + ListNode? n1 = P.next; + n0.next = n1; + } + ``` + +=== "Go" + + ```go title="linked_list.go" + /* 删除链表的节点 n0 之后的首个节点 */ + func removeItem(n0 *ListNode) { + if n0.Next == nil { + return + } + // n0 -> P -> n1 + P := n0.Next + n1 := P.Next + n0.Next = n1 + } + ``` + +=== "Swift" + + ```swift title="linked_list.swift" + /* 删除链表的节点 n0 之后的首个节点 */ + func remove(n0: ListNode) { + if n0.next == nil { + return + } + // n0 -> P -> n1 + let P = n0.next + let n1 = P?.next + n0.next = n1 + P?.next = nil + } + ``` + +=== "JS" + + ```javascript title="linked_list.js" + /* 删除链表的节点 n0 之后的首个节点 */ + function remove(n0) { + if (!n0.next) return; + // n0 -> P -> n1 + const P = n0.next; + const n1 = P.next; + n0.next = n1; + } + ``` + +=== "TS" + + ```typescript title="linked_list.ts" + /* 删除链表的节点 n0 之后的首个节点 */ + function remove(n0: ListNode): void { + if (!n0.next) { + return; + } + // n0 -> P -> n1 + const P = n0.next; + const n1 = P.next; + n0.next = n1; + } + ``` + +=== "Dart" + + ```dart title="linked_list.dart" + /* 删除链表的节点 n0 之后的首个节点 */ + void remove(ListNode n0) { + if (n0.next == null) return; + // n0 -> P -> n1 + ListNode P = n0.next!; + ListNode? n1 = P.next; + n0.next = n1; + } + ``` + +=== "Rust" + + ```rust title="linked_list.rs" + /* 删除链表的节点 n0 之后的首个节点 */ + #[allow(non_snake_case)] + pub fn removeTable 4-1 Efficiency Comparison of Arrays and Linked Lists
+ +Figure 4-8 Common Types of Linked Lists
+ +## 4.2.4 Typical Applications of Linked Lists + +Singly linked lists are commonly used to implement stacks, queues, hash tables, and graphs. + +- **Stacks and Queues**: When insertion and deletion operations are performed at one end of the linked list, it exhibits last-in-first-out characteristics, corresponding to a stack. When insertion is at one end and deletion is at the other, it shows first-in-first-out characteristics, corresponding to a queue. +- **Hash Tables**: Chaining is one of the mainstream solutions to hash collisions, where all colliding elements are placed in a linked list. +- **Graphs**: Adjacency lists are a common way to represent graphs, where each vertex is associated with a linked list. Each element in the list represents other vertices connected to that vertex. + +Doubly linked lists are commonly used in scenarios that require quick access to the previous and next elements. + +- **Advanced Data Structures**: For example, in red-black trees and B-trees, we need to access a node's parent, which can be achieved by storing a reference to the parent node in each node, similar to a doubly linked list. +- **Browser History**: In web browsers, when a user clicks the forward or backward button, the browser needs to know the previously and next visited web pages. The properties of a doubly linked list make this operation simple. +- **LRU Algorithm**: In Least Recently Used (LRU) cache eviction algorithms, we need to quickly find the least recently used data and support rapid addition and deletion of nodes. Here, using a doubly linked list is very appropriate. + +Circular linked lists are commonly used in scenarios requiring periodic operations, such as resource scheduling in operating systems. + +- **Round-Robin Scheduling Algorithm**: In operating systems, the round-robin scheduling algorithm is a common CPU scheduling algorithm that cycles through a group of processes. Each process is assigned a time slice, and when it expires, the CPU switches to the next process. This circular operation can be implemented using a circular linked list. +- **Data Buffers**: Circular linked lists may also be used in some data buffer implementations. For instance, in audio and video players, the data stream might be divided into multiple buffer blocks placed in a circular linked list to achieve seamless playback. diff --git a/docs-en/chapter_array_and_linkedlist/list.md b/docs-en/chapter_array_and_linkedlist/list.md new file mode 100755 index 000000000..25f111409 --- /dev/null +++ b/docs-en/chapter_array_and_linkedlist/list.md @@ -0,0 +1,2120 @@ +--- +comments: true +--- + +# 4.3 List + +A "list" is an abstract data structure concept, representing an ordered collection of elements. It supports operations like element access, modification, addition, deletion, and traversal, without requiring users to consider capacity limitations. Lists can be implemented based on linked lists or arrays. + +- A linked list naturally functions as a list, supporting operations for adding, deleting, searching, and modifying elements, and can dynamically adjust its size. +- Arrays also support these operations, but due to their fixed length, they can be considered as a list with a length limit. + +When using arrays to implement lists, **the fixed length property reduces the practicality of the list**. This is because we often cannot determine in advance how much data needs to be stored, making it difficult to choose an appropriate list length. If the length is too small, it may not meet the requirements; if too large, it may waste memory space. + +To solve this problem, we can use a "dynamic array" to implement lists. It inherits the advantages of arrays and can dynamically expand during program execution. + +In fact, **many programming languages' standard libraries implement lists using dynamic arrays**, such as Python's `list`, Java's `ArrayList`, C++'s `vector`, and C#'s `List`. In the following discussion, we will consider "list" and "dynamic array" as synonymous concepts. + +## 4.3.1 Common List Operations + +### 1. Initializing a List + +We typically use two methods of initialization: "without initial values" and "with initial values". + +=== "Python" + + ```python title="list.py" + # Initialize list + # Without initial values + nums1: list[int] = [] + # With initial values + nums: list[int] = [1, 3, 2, 5, 4] + ``` + +=== "C++" + + ```cpp title="list.cpp" + /* Initialize list */ + // Note, in C++ the vector is the equivalent of nums described here + // Without initial values + vectorTable 4-2 Computer Storage Devices
+ +Figure 4-9 Computer Storage System
+ +!!! note + + The storage hierarchy of computers reflects a delicate balance between speed, capacity, and cost. In fact, this kind of trade-off is common in all industrial fields, requiring us to find the best balance between different advantages and limitations. + +Overall, **hard disks are used for long-term storage of large amounts of data, memory is used for temporary storage of data being processed during program execution, and cache is used to store frequently accessed data and instructions** to improve program execution efficiency. Together, they ensure the efficient operation of computer systems. + +As shown in the Figure 4-10 , during program execution, data is read from the hard disk into memory for CPU computation. The cache can be considered a part of the CPU, **smartly loading data from memory** to provide fast data access to the CPU, significantly enhancing program execution efficiency and reducing reliance on slower memory. + +{ class="animation-figure" } + +Figure 4-10 Data Flow Between Hard Disk, Memory, and Cache
+ +## 4.4.2 Memory Efficiency of Data Structures + +In terms of memory space utilization, arrays and linked lists have their advantages and limitations. + +On one hand, **memory is limited and cannot be shared by multiple programs**, so we hope that data structures can use space as efficiently as possible. The elements of an array are tightly packed without extra space for storing references (pointers) between linked list nodes, making them more space-efficient. However, arrays require allocating sufficient continuous memory space at once, which may lead to memory waste, and array expansion also requires additional time and space costs. In contrast, linked lists allocate and reclaim memory dynamically on a per-node basis, providing greater flexibility. + +On the other hand, during program execution, **as memory is repeatedly allocated and released, the degree of fragmentation of free memory becomes higher**, leading to reduced memory utilization efficiency. Arrays, due to their continuous storage method, are relatively less likely to cause memory fragmentation. In contrast, the elements of a linked list are dispersedly stored, and frequent insertion and deletion operations make memory fragmentation more likely. + +## 4.4.3 Cache Efficiency of Data Structures + +Although caches are much smaller in space capacity than memory, they are much faster and play a crucial role in program execution speed. Since the cache's capacity is limited and can only store a small part of frequently accessed data, when the CPU tries to access data not in the cache, a "cache miss" occurs, forcing the CPU to load the needed data from slower memory. + +Clearly, **the fewer the cache misses, the higher the CPU's data read-write efficiency**, and the better the program performance. The proportion of successful data retrieval from the cache by the CPU is called the "cache hit rate," a metric often used to measure cache efficiency. + +To achieve higher efficiency, caches adopt the following data loading mechanisms. + +- **Cache Lines**: Caches don't store and load data byte by byte but in units of cache lines. Compared to byte-by-byte transfer, the transmission of cache lines is more efficient. +- **Prefetch Mechanism**: Processors try to predict data access patterns (such as sequential access, fixed stride jumping access, etc.) and load data into the cache according to specific patterns to improve the hit rate. +- **Spatial Locality**: If data is accessed, data nearby is likely to be accessed in the near future. Therefore, when loading certain data, the cache also loads nearby data to improve the hit rate. +- **Temporal Locality**: If data is accessed, it's likely to be accessed again in the near future. Caches use this principle to retain recently accessed data to improve the hit rate. + +In fact, **arrays and linked lists have different cache utilization efficiencies**, mainly reflected in the following aspects. + +- **Occupied Space**: Linked list elements occupy more space than array elements, resulting in less effective data volume in the cache. +- **Cache Lines**: Linked list data is scattered throughout memory, and since caches load "by line," the proportion of loading invalid data is higher. +- **Prefetch Mechanism**: The data access pattern of arrays is more "predictable" than that of linked lists, meaning the system is more likely to guess which data will be loaded next. +- **Spatial Locality**: Arrays are stored in concentrated memory spaces, so the data near the loaded data is more likely to be accessed next. + +Overall, **arrays have a higher cache hit rate and are generally more efficient in operation than linked lists**. This makes data structures based on arrays more popular in solving algorithmic problems. + +It should be noted that **high cache efficiency does not mean that arrays are always better than linked lists**. Which data structure to choose in actual applications should be based on specific requirements. For example, both arrays and linked lists can implement the "stack" data structure (which will be detailed in the next chapter), but they are suitable for different scenarios. + +- In algorithm problems, we tend to choose stacks based on arrays because they provide higher operational efficiency and random access capabilities, with the only cost being the need to pre-allocate a certain amount of memory space for the array. +- If the data volume is very large, highly dynamic, and the expected size of the stack is difficult to estimate, then a stack based on a linked list is more appropriate. Linked lists can disperse a large amount of data in different parts of the memory and avoid the additional overhead of array expansion. diff --git a/docs-en/chapter_array_and_linkedlist/summary.md b/docs-en/chapter_array_and_linkedlist/summary.md new file mode 100644 index 000000000..865e4d2fe --- /dev/null +++ b/docs-en/chapter_array_and_linkedlist/summary.md @@ -0,0 +1,85 @@ +--- +comments: true +--- + +# 4.5 Summary + +### 1. Key Review + +- Arrays and linked lists are two fundamental data structures, representing two storage methods in computer memory: continuous space storage and dispersed space storage. Their characteristics complement each other. +- Arrays support random access and use less memory; however, they are inefficient in inserting and deleting elements and have a fixed length after initialization. +- Linked lists implement efficient node insertion and deletion through changing references (pointers) and can flexibly adjust their length; however, they have lower node access efficiency and use more memory. +- Common types of linked lists include singly linked lists, circular linked lists, and doubly linked lists, each with its own application scenarios. +- Lists are ordered collections of elements that support addition, deletion, and modification, typically implemented based on dynamic arrays, retaining the advantages of arrays while allowing flexible length adjustment. +- The advent of lists significantly enhanced the practicality of arrays but may lead to some memory space wastage. +- During program execution, data is mainly stored in memory. Arrays provide higher memory space efficiency, while linked lists are more flexible in memory usage. +- Caches provide fast data access to CPUs through mechanisms like cache lines, prefetching, spatial locality, and temporal locality, significantly enhancing program execution efficiency. +- Due to higher cache hit rates, arrays are generally more efficient than linked lists. When choosing a data structure, the appropriate choice should be made based on specific needs and scenarios. + +### 2. Q & A + +!!! question "Does storing arrays on the stack versus the heap affect time and space efficiency?" + + Arrays stored on both the stack and heap are stored in continuous memory spaces, and data operation efficiency is essentially the same. However, stacks and heaps have their own characteristics, leading to the following differences. + + 1. Allocation and release efficiency: The stack is a smaller memory block, allocated automatically by the compiler; the heap memory is relatively larger and can be dynamically allocated in the code, more prone to fragmentation. Therefore, allocation and release operations on the heap are generally slower than on the stack. + 2. Size limitation: Stack memory is relatively small, while the heap size is generally limited by available memory. Therefore, the heap is more suitable for storing large arrays. + 3. Flexibility: The size of arrays on the stack needs to be determined at compile-time, while the size of arrays on the heap can be dynamically determined at runtime. + +!!! question "Why do arrays require elements of the same type, while linked lists do not emphasize same-type elements?" + + Linked lists consist of nodes connected by references (pointers), and each node can store data of different types, such as int, double, string, object, etc. + + In contrast, array elements must be of the same type, allowing the calculation of offsets to access the corresponding element positions. For example, an array containing both int and long types, with single elements occupying 4 bytes and 8 bytes respectively, cannot use the following formula to calculate offsets, as the array contains elements of two different lengths. + + ```shell + # Element memory address = Array memory address + Element length * Element index + ``` + +!!! question "After deleting a node, is it necessary to set `P.next` to `None`?" + + Not modifying `P.next` is also acceptable. From the perspective of the linked list, traversing from the head node to the tail node will no longer encounter `P`. This means that node `P` has been effectively removed from the list, and where `P` points no longer affects the list. + + From a garbage collection perspective, for languages with automatic garbage collection mechanisms like Java, Python, and Go, whether node `P` is collected depends on whether there are still references pointing to it, not on the value of `P.next`. In languages like C and C++, we need to manually free the node's memory. + +!!! question "In linked lists, the time complexity for insertion and deletion operations is `O(1)`. But searching for the element before insertion or deletion takes `O(n)` time, so why isn't the time complexity `O(n)`?" + + If an element is searched first and then deleted, the time complexity is indeed `O(n)`. However, the `O(1)` advantage of linked lists in insertion and deletion can be realized in other applications. For example, in the implementation of double-ended queues using linked lists, we maintain pointers always pointing to the head and tail nodes, making each insertion and deletion operation `O(1)`. + +!!! question "In the image 'Linked List Definition and Storage Method', do the light blue storage nodes occupy a single memory address, or do they share half with the node value?" + + The diagram is just a qualitative representation; quantitative analysis depends on specific situations. + + - Different types of node values occupy different amounts of space, such as int, long, double, and object instances. + - The memory space occupied by pointer variables depends on the operating system and compilation environment used, usually 8 bytes or 4 bytes. + +!!! question "Is adding elements to the end of a list always `O(1)`?" + + If adding an element exceeds the list length, the list needs to be expanded first. The system will request a new memory block and move all elements of the original list over, in which case the time complexity becomes `O(n)`. + +!!! question "The statement 'The emergence of lists greatly improves the practicality of arrays, but may lead to some memory space wastage' - does this refer to the memory occupied by additional variables like capacity, length, and expansion multiplier?" + + The space wastage here mainly refers to two aspects: on the one hand, lists are set with an initial length, which we may not always need; on the other hand, to prevent frequent expansion, expansion usually multiplies by a coefficient, such as $\times 1.5$. This results in many empty slots, which we typically cannot fully fill. + +!!! question "In Python, after initializing `n = [1, 2, 3]`, the addresses of these 3 elements are contiguous, but initializing `m = [2, 1, 3]` shows that each element's `id` is not consecutive but identical to those in `n`. If the addresses of these elements are not contiguous, is `m` still an array?" + + If we replace list elements with linked list nodes `n = [n1, n2, n3, n4, n5]`, these 5 node objects are also typically dispersed throughout memory. However, given a list index, we can still access the node's memory address in `O(1)` time, thereby accessing the corresponding node. This is because the array stores references to the nodes, not the nodes themselves. + + Unlike many languages, in Python, numbers are also wrapped as objects, and lists store references to these numbers, not the numbers themselves. Therefore, we find that the same number in two arrays has the same `id`, and these numbers' memory addresses need not be contiguous. + +!!! question "The `std::list` in C++ STL has already implemented a doubly linked list, but it seems that some algorithm books don't directly use it. Is there any limitation?" + + On the one hand, we often prefer to use arrays to implement algorithms, only using linked lists when necessary, mainly for two reasons. + + - Space overhead: Since each element requires two additional pointers (one for the previous element and one for the next), `std::list` usually occupies more space than `std::vector`. + - Cache unfriendly: As the data is not stored continuously, `std::list` has a lower cache utilization rate. Generally, `std::vector` performs better. + + On the other hand, linked lists are primarily necessary for binary trees and graphs. Stacks and queues are often implemented using the programming language's `stack` and `queue` classes, rather than linked lists. + +!!! question "Does initializing a list `res = [0] * self.size()` result in each element of `res` referencing the same address?" + + No. However, this issue arises with two-dimensional arrays, for example, initializing a two-dimensional list `res = [[0] * self.size()]` would reference the same list `[0]` multiple times. + +!!! question "In deleting a node, is it necessary to break the reference to its successor node?" + + From the perspective of data structures and algorithms (problem-solving), it's okay not to break the link, as long as the program's logic is correct. From the perspective of standard libraries, breaking the link is safer and more logically clear. If the link is not broken, and the deleted node is not properly recycled, it could affect the recycling of the successor node's memory. diff --git a/docs-en/chapter_computational_complexity/index.md b/docs-en/chapter_computational_complexity/index.md index 5a1fd7fcb..db3800931 100644 --- a/docs-en/chapter_computational_complexity/index.md +++ b/docs-en/chapter_computational_complexity/index.md @@ -19,7 +19,7 @@ icon: material/timer-sand ## 本章内容 -- [2.1 Evaluating Algorithm Efficiency](https://www.hello-algo.com/chapter_computational_complexity/performance_evaluation/) +- [2.1 Algorithm Efficiency Assessment](https://www.hello-algo.com/chapter_computational_complexity/performance_evaluation/) - [2.2 Iteration and Recursion](https://www.hello-algo.com/chapter_computational_complexity/iteration_and_recursion/) - [2.3 Time Complexity](https://www.hello-algo.com/chapter_computational_complexity/time_complexity/) - [2.4 Space Complexity](https://www.hello-algo.com/chapter_computational_complexity/space_complexity/) diff --git a/docs-en/chapter_data_structure/basic_data_types.md b/docs-en/chapter_data_structure/basic_data_types.md new file mode 100644 index 000000000..c93d6f0d6 --- /dev/null +++ b/docs-en/chapter_data_structure/basic_data_types.md @@ -0,0 +1,172 @@ +--- +comments: true +--- + +# 3.2 Fundamental Data Types + +When we think of data in computers, we imagine various forms like text, images, videos, voice, 3D models, etc. Despite their different organizational forms, they are all composed of various fundamental data types. + +**Fundamental data types are those that the CPU can directly operate on** and are directly used in algorithms, mainly including the following. + +- Integer types: `byte`, `short`, `int`, `long`. +- Floating-point types: `float`, `double`, used to represent decimals. +- Character type: `char`, used to represent letters, punctuation, and even emojis in various languages. +- Boolean type: `bool`, used for "yes" or "no" decisions. + +**Fundamental data types are stored in computers in binary form**. One binary digit is equal to 1 bit. In most modern operating systems, 1 byte consists of 8 bits. + +The range of values for fundamental data types depends on the size of the space they occupy. Below, we take Java as an example. + +- The integer type `byte` occupies 1 byte = 8 bits and can represent $2^8$ numbers. +- The integer type `int` occupies 4 bytes = 32 bits and can represent $2^{32}$ numbers. + +The following table lists the space occupied, value range, and default values of various fundamental data types in Java. This table does not need to be memorized, but understood roughly and referred to when needed. + +Table 3-1 Space Occupied and Value Range of Fundamental Data Types
+ +Figure 3-6 ASCII Code
+ +However, **ASCII can only represent English characters**. With the globalization of computers, a character set called "EASCII" was developed to represent more languages. It expands on the 7-bit basis of ASCII to 8 bits, enabling the representation of 256 different characters. + +Globally, a series of EASCII character sets for different regions emerged. The first 128 characters of these sets are uniformly ASCII, while the remaining 128 characters are defined differently to cater to various language requirements. + +## 3.4.2 GBK Character Set + +Later, it was found that **EASCII still could not meet the character requirements of many languages**. For instance, there are nearly a hundred thousand Chinese characters, with several thousand used in everyday life. In 1980, China's National Standards Bureau released the "GB2312" character set, which included 6763 Chinese characters, essentially meeting the computer processing needs for Chinese. + +However, GB2312 could not handle some rare and traditional characters. The "GBK" character set, an expansion of GB2312, includes a total of 21886 Chinese characters. In the GBK encoding scheme, ASCII characters are represented with one byte, while Chinese characters use two bytes. + +## 3.4.3 Unicode Character Set + +With the rapid development of computer technology and a plethora of character sets and encoding standards, numerous problems arose. On one hand, these character sets generally only defined characters for specific languages and could not function properly in multilingual environments. On the other hand, the existence of multiple character set standards for the same language caused garbled text when information was exchanged between computers using different encoding standards. + +Researchers of that era thought: **What if we introduced a comprehensive character set that included all languages and symbols worldwide, wouldn't that solve the problems of cross-language environments and garbled text?** Driven by this idea, the extensive character set, Unicode, was born. + +The Chinese name for "Unicode" is "统一码" (Unified Code), theoretically capable of accommodating over a million characters. It aims to incorporate characters from all over the world into a single set, providing a universal character set for processing and displaying various languages and reducing the issues of garbled text due to different encoding standards. + +Since its release in 1991, Unicode has continually expanded to include new languages and characters. As of September 2022, Unicode contains 149,186 characters, including characters, symbols, and even emojis from various languages. In the vast Unicode character set, commonly used characters occupy 2 bytes, while some rare characters take up 3 or even 4 bytes. + +Unicode is a universal character set that assigns a number (called a "code point") to each character, **but it does not specify how these character code points should be stored in a computer**. One might ask: When Unicode code points of varying lengths appear in a text, how does the system parse the characters? For example, given a 2-byte code, how does the system determine if it represents a single 2-byte character or two 1-byte characters? + +A straightforward solution to this problem is to store all characters as equal-length encodings. As shown in the Figure 3-7 , each character in "Hello" occupies 1 byte, while each character in "算法" (algorithm) occupies 2 bytes. We could encode all characters in "Hello 算法" as 2 bytes by padding the higher bits with zeros. This way, the system can parse a character every 2 bytes, recovering the content of the phrase. + +{ class="animation-figure" } + +Figure 3-7 Unicode Encoding Example
+ +However, as ASCII has shown us, encoding English only requires 1 byte. Using the above approach would double the space occupied by English text compared to ASCII encoding, which is a waste of memory space. Therefore, a more efficient Unicode encoding method is needed. + +## 3.4.4 UTF-8 Encoding + +Currently, UTF-8 has become the most widely used Unicode encoding method internationally. **It is a variable-length encoding**, using 1 to 4 bytes to represent a character, depending on the complexity of the character. ASCII characters need only 1 byte, Latin and Greek letters require 2 bytes, commonly used Chinese characters need 3 bytes, and some other rare characters need 4 bytes. + +The encoding rules for UTF-8 are not complex and can be divided into two cases: + +- For 1-byte characters, set the highest bit to $0$, and the remaining 7 bits to the Unicode code point. Notably, ASCII characters occupy the first 128 code points in the Unicode set. This means that **UTF-8 encoding is backward compatible with ASCII**. This implies that UTF-8 can be used to parse ancient ASCII text. +- For characters of length $n$ bytes (where $n > 1$), set the highest $n$ bits of the first byte to $1$, and the $(n + 1)^{\text{th}}$ bit to $0$; starting from the second byte, set the highest 2 bits of each byte to $10$; the rest of the bits are used to fill the Unicode code point. + +The Figure 3-8 shows the UTF-8 encoding for "Hello算法". It can be observed that since the highest $n$ bits are set to $1$, the system can determine the length of the character as $n$ by counting the number of highest bits set to $1$. + +But why set the highest 2 bits of the remaining bytes to $10$? Actually, this $10$ serves as a kind of checksum. If the system starts parsing text from an incorrect byte, the $10$ at the beginning of the byte can help the system quickly detect an anomaly. + +The reason for using $10$ as a checksum is that, under UTF-8 encoding rules, it's impossible for the highest two bits of a character to be $10$. This can be proven by contradiction: If the highest two bits of a character are $10$, it indicates that the character's length is $1$, corresponding to ASCII. However, the highest bit of an ASCII character should be $0$, contradicting the assumption. + +{ class="animation-figure" } + +Figure 3-8 UTF-8 Encoding Example
+ +Apart from UTF-8, other common encoding methods include: + +- **UTF-16 Encoding**: Uses 2 or 4 bytes to represent a character. All ASCII characters and commonly used non-English characters are represented with 2 bytes; a few characters require 4 bytes. For 2-byte characters, the UTF-16 encoding is equal to the Unicode code point. +- **UTF-32 Encoding**: Every character uses 4 bytes. This means UTF-32 occupies more space than UTF-8 and UTF-16, especially for texts with a high proportion of ASCII characters. + +From the perspective of storage space, UTF-8 is highly efficient for representing English characters, requiring only 1 byte; UTF-16 might be more efficient for encoding some non-English characters (like Chinese), as it requires only 2 bytes, while UTF-8 might need 3 bytes. + +From a compatibility standpoint, UTF-8 is the most versatile, with many tools and libraries supporting UTF-8 as a priority. + +## 3.4.5 Character Encoding in Programming Languages + +In many classic programming languages, strings during program execution are encoded using fixed-length encodings like UTF-16 or UTF-32. This allows strings to be treated as arrays, offering several advantages: + +- **Random Access**: Strings encoded in UTF-16 can be accessed randomly with ease. For UTF-8, which is a variable-length encoding, locating the $i^{th}$ character requires traversing the string from the start to the $i^{th}$ position, taking $O(n)$ time. +- **Character Counting**: Similar to random access, counting the number of characters in a UTF-16 encoded string is an $O(1)$ operation. However, counting characters in a UTF-8 encoded string requires traversing the entire string. +- **String Operations**: Many string operations like splitting, concatenating, inserting, and deleting are easier on UTF-16 encoded strings. These operations generally require additional computation on UTF-8 encoded strings to ensure the validity of the UTF-8 encoding. + +The design of character encoding schemes in programming languages is an interesting topic involving various factors: + +- Java’s `String` type uses UTF-16 encoding, with each character occupying 2 bytes. This was based on the initial belief that 16 bits were sufficient to represent all possible characters, a judgment later proven incorrect. As the Unicode standard expanded beyond 16 bits, characters in Java may now be represented by a pair of 16-bit values, known as “surrogate pairs.” +- JavaScript and TypeScript use UTF-16 encoding for similar reasons as Java. When JavaScript was first introduced by Netscape in 1995, Unicode was still in its early stages, and 16-bit encoding was sufficient to represent all Unicode characters. +- C# uses UTF-16 encoding, largely because the .NET platform, designed by Microsoft, and many Microsoft technologies, including the Windows operating system, extensively use UTF-16 encoding. + +Due to the underestimation of character counts, these languages had to resort to using "surrogate pairs" to represent Unicode characters exceeding 16 bits. This approach has its drawbacks: strings containing surrogate pairs may have characters occupying 2 or 4 bytes, losing the advantage of fixed-length encoding, and handling surrogate pairs adds to the complexity and debugging difficulty of programming. + +Owing to these reasons, some programming languages have adopted different encoding schemes: + +- Python’s `str` type uses Unicode encoding with a flexible representation where the storage length of characters depends on the largest Unicode code point in the string. If all characters are ASCII, each character occupies 1 byte; if characters exceed ASCII but are within the Basic Multilingual Plane (BMP), each occupies 2 bytes; if characters exceed the BMP, each occupies 4 bytes. +- Go’s `string` type internally uses UTF-8 encoding. Go also provides the `rune` type for representing individual Unicode code points. +- Rust’s `str` and `String` types use UTF-8 encoding internally. Rust also offers the `char` type for individual Unicode code points. + +It’s important to note that the above discussion pertains to how strings are stored in programming languages, **which is a different issue from how strings are stored in files or transmitted over networks**. For file storage or network transmission, strings are usually encoded in UTF-8 format for optimal compatibility and space efficiency. diff --git a/docs-en/chapter_data_structure/classification_of_data_structure.md b/docs-en/chapter_data_structure/classification_of_data_structure.md index 4ec268b1d..766813b39 100644 --- a/docs-en/chapter_data_structure/classification_of_data_structure.md +++ b/docs-en/chapter_data_structure/classification_of_data_structure.md @@ -1,49 +1,58 @@ -# Classification Of Data Structures +--- +comments: true +--- -Common data structures include arrays, linked lists, stacks, queues, hash tables, trees, heaps, and graphs. They can be divided into two categories: logical structure and physical structure. +# 3.1 Classification of Data Structures -## Logical Structures: Linear And Non-linear +Common data structures include arrays, linked lists, stacks, queues, hash tables, trees, heaps, and graphs. They can be classified into two dimensions: "Logical Structure" and "Physical Structure". -**Logical structures reveal logical relationships between data elements**. In arrays and linked lists, data are arranged in sequential order, reflecting the linear relationship between data; while in trees, data are arranged hierarchically from the top down, showing the derived relationship between ancestors and descendants; and graphs are composed of nodes and edges, reflecting the complex network relationship. +## 3.1.1 Logical Structure: Linear and Non-Linear -As shown in the figure below, logical structures can further be divided into "linear data structure" and "non-linear data structure". Linear data structures are more intuitive, meaning that the data are arranged linearly in terms of logical relationships; non-linear data structures, on the other hand, are arranged non-linearly. +**The logical structure reveals the logical relationships between data elements**. In arrays and linked lists, data is arranged in a certain order, reflecting a linear relationship between them. In trees, data is arranged from top to bottom in layers, showing a "ancestor-descendant" hierarchical relationship. Graphs, consisting of nodes and edges, represent complex network relationships. -- **Linear data structures**: arrays, linked lists, stacks, queues, hash tables. -- **Nonlinear data structures**: trees, heaps, graphs, hash tables. +As shown in the Figure 3-1 , logical structures can be divided into two major categories: "Linear" and "Non-linear". Linear structures are more intuitive, indicating data is arranged linearly in logical relationships; non-linear structures, conversely, are arranged non-linearly. - +- **Linear Data Structures**: Arrays, Linked Lists, Stacks, Queues, Hash Tables. +- **Non-Linear Data Structures**: Trees, Heaps, Graphs, Hash Tables. -Non-linear data structures can be further divided into tree and graph structures. +{ class="animation-figure" } -- **Linear structures**: arrays, linked lists, queues, stacks, hash tables, with one-to-one sequential relationship between elements. -- **Tree structure**: tree, heap, hash table, with one-to-many relationship between elements. -- **Graph**: graph with many-to-many relationship between elements. +Figure 3-1 Linear and Non-Linear Data Structures
-## Physical Structure: Continuous vs. Dispersed +Non-linear data structures can be further divided into tree structures and network structures. -**When an algorithm is running, the data being processed is stored in memory**. The figure below shows a computer memory module where each black square represents a memory space. We can think of the memory as a giant Excel sheet in which each cell can store data of a certain size. +- **Tree Structures**: Trees, Heaps, Hash Tables, where elements have one-to-many relationships. +- **Network Structures**: Graphs, where elements have many-to-many relationships. -**The system accesses the data at the target location by means of a memory address**. As shown in the figure below, the computer assigns a unique identifier to each cell in the table according to specific rules, ensuring that each memory space has a unique memory address. With these addresses, the program can access the data in memory. +## 3.1.2 Physical Structure: Contiguous and Dispersed - +**When an algorithm program runs, the data being processed is mainly stored in memory**. The following figure shows a computer memory stick, each black block containing a memory space. We can imagine memory as a huge Excel spreadsheet, where each cell can store a certain amount of data. + +**The system accesses data at the target location through memory addresses**. As shown in the Figure 3-2 , the computer allocates numbers to each cell in the table according to specific rules, ensuring each memory space has a unique memory address. With these addresses, programs can access data in memory. + +{ class="animation-figure" } + +Figure 3-2 Memory Stick, Memory Spaces, Memory Addresses
!!! tip - It is worth noting that comparing memory to the Excel sheet is a simplified analogy. The actual memory working mechanism is more complicated, involving the concepts of address, space, memory management, cache mechanism, virtual and physical memory. + It's worth noting that comparing memory to an Excel spreadsheet is a simplified analogy. The actual working mechanism of memory is more complex, involving concepts like address space, memory management, cache mechanisms, virtual memory, and physical memory. -Memory is a shared resource for all programs, and when a block of memory is occupied by one program, it cannot be used by other programs at the same time. **Therefore, considering memory resources is crucial in designing data structures and algorithms**. For example, the algorithm's peak memory usage should not exceed the remaining free memory of the system; if there is a lack of contiguous memory blocks, then the data structure chosen must be able to be stored in non-contiguous memory blocks. +Memory is a shared resource for all programs. When a block of memory is occupied by one program, it cannot be used by others simultaneously. **Therefore, memory resources are an important consideration in the design of data structures and algorithms**. For example, the peak memory usage of an algorithm should not exceed the system's remaining free memory. If there is a lack of contiguous large memory spaces, the chosen data structure must be able to store data in dispersed memory spaces. -As shown in the figure below, **Physical structure reflects the way data is stored in computer memory and it can be divided into consecutive space storage (arrays) and distributed space storage (linked lists)**. The physical structure determines how data is accessed, updated, added, deleted, etc. Logical and physical structure complement each other in terms of time efficiency and space efficiency. +As shown in the Figure 3-3 , **the physical structure reflects how data is stored in computer memory**, which can be divided into contiguous space storage (arrays) and dispersed space storage (linked lists). The physical structure determines from the bottom level how data is accessed, updated, added, or deleted. Both types of physical structures exhibit complementary characteristics in terms of time efficiency and space efficiency. - +{ class="animation-figure" } -**It is worth stating that all data structures are implemented based on arrays, linked lists, or a combination of the two**. For example, stacks and queues can be implemented using both arrays and linked lists; and implementations of hash tables may contain both arrays and linked lists. +Figure 3-3 Contiguous Space Storage and Dispersed Space Storage
-- **Array-based structures**: stacks, queues, hash tables, trees, heaps, graphs, matrices, tensors (arrays of dimension $\geq 3$), and so on. -- **Linked list-based structures**: stacks, queues, hash tables, trees, heaps, graphs, etc. +It's important to note that **all data structures are implemented based on arrays, linked lists, or a combination of both**. For example, stacks and queues can be implemented using either arrays or linked lists; while hash tables may include both arrays and linked lists. -Data structures based on arrays are also known as "static data structures", which means that such structures' length remains constant after initialization. In contrast, data structures based on linked lists are called "dynamic data structures", meaning that their length can be adjusted during program execution after initialization. +- **Array-based Implementations**: Stacks, Queues, Hash Tables, Trees, Heaps, Graphs, Matrices, Tensors (arrays with dimensions $\geq 3$). +- **Linked List-based Implementations**: Stacks, Queues, Hash Tables, Trees, Heaps, Graphs, etc. + +Data structures implemented based on arrays are also called “Static Data Structures,” meaning their length cannot be changed after initialization. Conversely, those based on linked lists are called “Dynamic Data Structures,” which can still adjust their size during program execution. !!! tip - If you find it difficult to understand the physical structure, it is recommended that you read the next chapter, "Arrays and Linked Lists," before reviewing this section. + If you find it difficult to understand the physical structure, it's recommended to read the next chapter first and then revisit this section. diff --git a/docs-en/chapter_data_structure/index.md b/docs-en/chapter_data_structure/index.md index 147eee85c..7966e7227 100644 --- a/docs-en/chapter_data_structure/index.md +++ b/docs-en/chapter_data_structure/index.md @@ -1,13 +1,26 @@ -# Data Structure +--- +comments: true +icon: material/shape-outline +--- + +# Chapter 3. Data StructuresFigure 3-4 Conversions between Sign-Magnitude, One's Complement, and Two's Complement
+ +Although sign-magnitude is the most intuitive, it has limitations. For one, **negative numbers in sign-magnitude cannot be directly used in calculations**. For example, in sign-magnitude, calculating $1 + (-2)$ results in $-3$, which is incorrect. + +$$ +\begin{aligned} +& 1 + (-2) \newline +& \rightarrow 0000 \; 0001 + 1000 \; 0010 \newline +& = 1000 \; 0011 \newline +& \rightarrow -3 +\end{aligned} +$$ + +To address this, computers introduced the **one's complement**. If we convert to one's complement and calculate $1 + (-2)$, then convert the result back to sign-magnitude, we get the correct result of $-1$. + +$$ +\begin{aligned} +& 1 + (-2) \newline +& \rightarrow 0000 \; 0001 \; \text{(Sign-magnitude)} + 1000 \; 0010 \; \text{(Sign-magnitude)} \newline +& = 0000 \; 0001 \; \text{(One's complement)} + 1111 \; 1101 \; \text{(One's complement)} \newline +& = 1111 \; 1110 \; \text{(One's complement)} \newline +& = 1000 \; 0001 \; \text{(Sign-magnitude)} \newline +& \rightarrow -1 +\end{aligned} +$$ + +Additionally, **there are two representations of zero in sign-magnitude**: $+0$ and $-0$. This means two different binary encodings for zero, which could lead to ambiguity. For example, in conditional checks, not differentiating between positive and negative zero might result in incorrect outcomes. Addressing this ambiguity would require additional checks, potentially reducing computational efficiency. + +$$ +\begin{aligned} ++0 & \rightarrow 0000 \; 0000 \newline +-0 & \rightarrow 1000 \; 0000 +\end{aligned} +$$ + +Like sign-magnitude, one's complement also suffers from the positive and negative zero ambiguity. Therefore, computers further introduced the **two's complement**. Let's observe the conversion process for negative zero in sign-magnitude, one's complement, and two's complement: + +$$ +\begin{aligned} +-0 \rightarrow \; & 1000 \; 0000 \; \text{(Sign-magnitude)} \newline += \; & 1111 \; 1111 \; \text{(One's complement)} \newline += 1 \; & 0000 \; 0000 \; \text{(Two's complement)} \newline +\end{aligned} +$$ + +Adding $1$ to the one's complement of negative zero produces a carry, but with `byte` length being only 8 bits, the carried-over $1$ to the 9th bit is discarded. Therefore, **the two's complement of negative zero is $0000 \; 0000$**, the same as positive zero, thus resolving the ambiguity. + +One last puzzle is the $[-128, 127]$ range for `byte`, with an additional negative number, $-128$. We observe that for the interval $[-127, +127]$, all integers have corresponding sign-magnitude, one's complement, and two's complement, and these can be converted between each other. + +However, **the two's complement $1000 \; 0000$ is an exception without a corresponding sign-magnitude**. According to the conversion method, its sign-magnitude would be $0000 \; 0000$, which is a contradiction since this represents zero, and its two's complement should be itself. Computers designate this special two's complement $1000 \; 0000$ as representing $-128$. In fact, the calculation of $(-1) + (-127)$ in two's complement results in $-128$. + +$$ +\begin{aligned} +& (-127) + (-1) \newline +& \rightarrow 1111 \; 1111 \; \text{(Sign-magnitude)} + 1000 \; 0001 \; \text{(Sign-magnitude)} \newline +& = 1000 \; 0000 \; \text{(One's complement)} + 1111 \; 1110 \; \text{(One's complement)} \newline +& = 1000 \; 0001 \; \text{(Two's complement)} + 1111 \; 1111 \; \text{(Two's complement)} \newline +& = 1000 \; 0000 \; \text{(Two's complement)} \newline +& \rightarrow -128 +\end{aligned} +$$ + +As you might have noticed, all these calculations are additions, hinting at an important fact: **computers' internal hardware circuits are primarily designed around addition operations**. This is because addition is simpler to implement in hardware compared to other operations like multiplication, division, and subtraction, allowing for easier parallelization and faster computation. + +It's important to note that this doesn't mean computers can only perform addition. **By combining addition with basic logical operations, computers can execute a variety of other mathematical operations**. For example, the subtraction $a - b$ can be translated into $a + (-b)$; multiplication and division can be translated into multiple additions or subtractions. + +We can now summarize the reason for using two's complement in computers: with two's complement representation, computers can use the same circuits and operations to handle both positive and negative number addition, eliminating the need for special hardware circuits for subtraction and avoiding the ambiguity of positive and negative zero. This greatly simplifies hardware design and enhances computational efficiency. + +The design of two's complement is quite ingenious, and due to space constraints, we'll stop here. Interested readers are encouraged to explore further. + +## 3.3.2 Floating-Point Number Encoding + +You might have noticed something intriguing: despite having the same length of 4 bytes, why does a `float` have a much larger range of values compared to an `int`? This seems counterintuitive, as one would expect the range to shrink for `float` since it needs to represent fractions. + +In fact, **this is due to the different representation method used by floating-point numbers (`float`)**. Let's consider a 32-bit binary number as: + +$$ +b_{31} b_{30} b_{29} \ldots b_2 b_1 b_0 +$$ + +According to the IEEE 754 standard, a 32-bit `float` consists of the following three parts: + +- Sign bit $\mathrm{S}$: Occupies 1 bit, corresponding to $b_{31}$. +- Exponent bit $\mathrm{E}$: Occupies 8 bits, corresponding to $b_{30} b_{29} \ldots b_{23}$. +- Fraction bit $\mathrm{N}$: Occupies 23 bits, corresponding to $b_{22} b_{21} \ldots b_0$. + +The value of a binary `float` number is calculated as: + +$$ +\text{val} = (-1)^{b_{31}} \times 2^{\left(b_{30} b_{29} \ldots b_{23}\right)_2 - 127} \times \left(1 . b_{22} b_{21} \ldots b_0\right)_2 +$$ + +Converted to a decimal formula, this becomes: + +$$ +\text{val} = (-1)^{\mathrm{S}} \times 2^{\mathrm{E} - 127} \times (1 + \mathrm{N}) +$$ + +The range of each component is: + +$$ +\begin{aligned} +\mathrm{S} \in & \{ 0, 1\}, \quad \mathrm{E} \in \{ 1, 2, \dots, 254 \} \newline +(1 + \mathrm{N}) = & (1 + \sum_{i=1}^{23} b_{23-i} \times 2^{-i}) \subset [1, 2 - 2^{-23}] +\end{aligned} +$$ + +{ class="animation-figure" } + +Figure 3-5 Example Calculation of a float in IEEE 754 Standard
+ +Observing the diagram, given an example data $\mathrm{S} = 0$, $\mathrm{E} = 124$, $\mathrm{N} = 2^{-2} + 2^{-3} = 0.375$, we have: + +$$ +\text{val} = (-1)^0 \times 2^{124 - 127} \times (1 + 0.375) = 0.171875 +$$ + +Now we can answer the initial question: **The representation of `float` includes an exponent bit, leading to a much larger range than `int`**. Based on the above calculation, the maximum positive number representable by `float` is approximately $2^{254 - 127} \times (2 - 2^{-23}) \approx 3.4 \times 10^{38}$, and the minimum negative number is obtained by switching the sign bit. + +**However, the trade-off for `float`'s expanded range is a sacrifice in precision**. The integer type `int` uses all 32 bits to represent the number, with values evenly distributed; but due to the exponent bit, the larger the value of a `float`, the greater the difference between adjacent numbers. + +As shown in the Table 3-2 , exponent bits $E = 0$ and $E = 255$ have special meanings, **used to represent zero, infinity, $\mathrm{NaN}$, etc.** + +Table 3-2 Meaning of Exponent Bits
+ +