mirror of
https://github.com/krahets/hello-algo.git
synced 2025-11-05 22:56:48 +08:00
translation: Update the complexity analysis chapter to the main branch (#1001)
* Update buttons. * Update button svg * Update button * Update README.md * Update index.md * Update translation of about_the _book * Update English headings. * Update the translation of chapter comlexity analysis to main branch.
This commit is contained in:
Yudong Jin
@ -1,12 +1,12 @@
|
||||
# Time Complexity
|
||||
|
||||
Runtime can be a visual and accurate reflection of the efficiency of an algorithm. What should we do if we want to accurately predict the runtime of a piece of code?
|
||||
Time complexity is a concept used to measure how the run time of an algorithm increases with the size of the input data. Understanding time complexity is crucial for accurately assessing the efficiency of an algorithm.
|
||||
|
||||
1. **Determine the running platform**, including hardware configuration, programming language, system environment, etc., all of which affect the efficiency of the code.
|
||||
2. **Evaluates the running time** required for various computational operations, e.g., the addition operation `+` takes 1 ns, the multiplication operation `*` takes 10 ns, the print operation `print()` takes 5 ns, and so on.
|
||||
3. **Counts all the computational operations in the code** and sums the execution times of all the operations to get the runtime.
|
||||
1. **Determining the Running Platform**: This includes hardware configuration, programming language, system environment, etc., all of which can affect the efficiency of code execution.
|
||||
2. **Evaluating the Run Time for Various Computational Operations**: For instance, an addition operation `+` might take 1 ns, a multiplication operation `*` might take 10 ns, a print operation `print()` might take 5 ns, etc.
|
||||
3. **Counting All the Computational Operations in the Code**: Summing the execution times of all these operations gives the total run time.
|
||||
|
||||
For example, in the following code, the input data size is $n$ :
|
||||
For example, consider the following code with an input size of $n$:
|
||||
|
||||
=== "Python"
|
||||
|
||||
@ -186,19 +186,19 @@ For example, in the following code, the input data size is $n$ :
|
||||
}
|
||||
```
|
||||
|
||||
Based on the above method, the algorithm running time can be obtained as $6n + 12$ ns :
|
||||
Using the above method, the run time of the algorithm can be calculated as $(6n + 12)$ ns:
|
||||
|
||||
$$
|
||||
1 + 1 + 10 + (1 + 5) \times n = 6n + 12
|
||||
$$
|
||||
|
||||
In practice, however, **statistical algorithm runtimes are neither reasonable nor realistic**. First, we do not want to tie the estimation time to the operation platform, because the algorithm needs to run on a variety of different platforms. Second, it is difficult for us to be informed of the runtime of each operation, which makes the prediction process extremely difficult.
|
||||
However, in practice, **counting the run time of an algorithm is neither practical nor reasonable**. First, we don't want to tie the estimated time to the running platform, as algorithms need to run on various platforms. Second, it's challenging to know the run time for each type of operation, making the estimation process difficult.
|
||||
|
||||
## Trends In Statistical Time Growth
|
||||
## Assessing Time Growth Trend
|
||||
|
||||
The time complexity analysis counts not the algorithm running time, **but the tendency of the algorithm running time to increase as the amount of data gets larger**.
|
||||
Time complexity analysis does not count the algorithm's run time, **but rather the growth trend of the run time as the data volume increases**.
|
||||
|
||||
The concept of "time-growing trend" is rather abstract, so let's try to understand it through an example. Suppose the size of the input data is $n$, and given three algorithmic functions `A`, `B` and `C`:
|
||||
Let's understand this concept of "time growth trend" with an example. Assume the input data size is $n$, and consider three algorithms `A`, `B`, and `C`:
|
||||
|
||||
=== "Python"
|
||||
|
||||
@ -452,23 +452,23 @@ The concept of "time-growing trend" is rather abstract, so let's try to understa
|
||||
}
|
||||
```
|
||||
|
||||
The figure below shows the time complexity of the above three algorithmic functions.
|
||||
The following figure shows the time complexities of these three algorithms.
|
||||
|
||||
- Algorithm `A` has only $1$ print operations, and the running time of the algorithm does not increase with $n$. We call the time complexity of this algorithm "constant order".
|
||||
- The print operation in algorithm `B` requires $n$ cycles, and the running time of the algorithm increases linearly with $n$. The time complexity of this algorithm is called "linear order".
|
||||
- The print operation in algorithm `C` requires $1000000$ loops, which is a long runtime, but it is independent of the size of the input data $n$. Therefore, the time complexity of `C` is the same as that of `A`, which is still of "constant order".
|
||||
- Algorithm `A` has just one print operation, and its run time does not grow with $n$. Its time complexity is considered "constant order."
|
||||
- Algorithm `B` involves a print operation looping $n$ times, and its run time grows linearly with $n$. Its time complexity is "linear order."
|
||||
- Algorithm `C` has a print operation looping 1,000,000 times. Although it takes a long time, it is independent of the input data size $n$. Therefore, the time complexity of `C` is the same as `A`, which is "constant order."
|
||||
|
||||
|
||||
|
||||
|
||||
What are the characteristics of time complexity analysis compared to direct statistical algorithmic running time?
|
||||
Compared to directly counting the run time of an algorithm, what are the characteristics of time complexity analysis?
|
||||
|
||||
- The **time complexity can effectively evaluate the efficiency of an algorithm**. For example, the running time of algorithm `B` increases linearly and is slower than algorithm `A` for $n > 1$ and slower than algorithm `C` for $n > 1,000,000$. In fact, as long as the input data size $n$ is large enough, algorithms with "constant order" of complexity will always outperform algorithms with "linear order", which is exactly what the time complexity trend means.
|
||||
- The **time complexity of the projection method is simpler**. Obviously, neither the running platform nor the type of computational operation is related to the growth trend of the running time of the algorithm. Therefore, in the time complexity analysis, we can simply consider the execution time of all computation operations as the same "unit time", and thus simplify the "statistics of the running time of computation operations" to the "statistics of the number of computation operations", which is the same as the "statistics of the number of computation operations". The difficulty of the estimation is greatly reduced by considering the execution time of all operations as the same "unit time".
|
||||
- There are also some limitations of **time complexity**. For example, although algorithms `A` and `C` have the same time complexity, the actual running time varies greatly. Similarly, although the time complexity of algorithm `B` is higher than that of `C` , algorithm `B` significantly outperforms algorithm `C` when the size of the input data $n$ is small. In these cases, it is difficult to judge the efficiency of an algorithm based on time complexity alone. Of course, despite the above problems, complexity analysis is still the most effective and commonly used method to judge the efficiency of algorithms.
|
||||
- **Time complexity effectively assesses algorithm efficiency**. For instance, algorithm `B` has linearly growing run time, which is slower than algorithm `A` when $n > 1$ and slower than `C` when $n > 1,000,000$. In fact, as long as the input data size $n$ is sufficiently large, a "constant order" complexity algorithm will always be better than a "linear order" one, demonstrating the essence of time growth trend.
|
||||
- **Time complexity analysis is more straightforward**. Obviously, the running platform and the types of computational operations are irrelevant to the trend of run time growth. Therefore, in time complexity analysis, we can simply treat the execution time of all computational operations as the same "unit time," simplifying the "computational operation run time count" to a "computational operation count." This significantly reduces the complexity of estimation.
|
||||
- **Time complexity has its limitations**. For example, although algorithms `A` and `C` have the same time complexity, their actual run times can be quite different. Similarly, even though algorithm `B` has a higher time complexity than `C`, it is clearly superior when the input data size $n$ is small. In these cases, it's difficult to judge the efficiency of algorithms based solely on time complexity. Nonetheless, despite these issues, complexity analysis remains the most effective and commonly used method for evaluating algorithm efficiency.
|
||||
|
||||
## Functions Asymptotic Upper Bounds
|
||||
## Asymptotic Upper Bound
|
||||
|
||||
Given a function with input size $n$:
|
||||
Consider a function with an input size of $n$:
|
||||
|
||||
=== "Python"
|
||||
|
||||
@ -637,41 +637,39 @@ Given a function with input size $n$:
|
||||
}
|
||||
```
|
||||
|
||||
Let the number of operations of the algorithm be a function of the size of the input data $n$, denoted as $T(n)$ , then the number of operations of the above function is:
|
||||
Given a function that represents the number of operations of an algorithm as a function of the input size $n$, denoted as $T(n)$, consider the following example:
|
||||
|
||||
$$
|
||||
T(n) = 3 + 2n
|
||||
$$
|
||||
|
||||
$T(n)$ is a primary function, which indicates that the trend of its running time growth is linear, and thus its time complexity is of linear order.
|
||||
Since $T(n)$ is a linear function, its growth trend is linear, and therefore, its time complexity is of linear order, denoted as $O(n)$. This mathematical notation, known as "big-O notation," represents the "asymptotic upper bound" of the function $T(n)$.
|
||||
|
||||
We denote the time complexity of the linear order as $O(n)$ , and this mathematical notation is called the "big $O$ notation big-$O$ notation", which denotes the "asymptotic upper bound" of the function $T(n)$.
|
||||
In essence, time complexity analysis is about finding the asymptotic upper bound of the "number of operations $T(n)$". It has a precise mathematical definition.
|
||||
|
||||
Time complexity analysis is essentially the computation of asymptotic upper bounds on the "number of operations function $T(n)$", which has a clear mathematical definition.
|
||||
!!! abstract "Asymptotic Upper Bound"
|
||||
|
||||
!!! abstract "Function asymptotic upper bound"
|
||||
If there exist positive real numbers $c$ and $n_0$ such that for all $n > n_0$, $T(n) \leq c \cdot f(n)$, then $f(n)$ is considered an asymptotic upper bound of $T(n)$, denoted as $T(n) = O(f(n))$.
|
||||
|
||||
If there exists a positive real number $c$ and a real number $n_0$ such that $T(n) \leq c \cdot f(n)$ for all $n > n_0$ , then it can be argued that $f(n)$ gives an asymptotic upper bound on $T(n)$ , denoted as $T(n) = O(f(n))$ .
|
||||
As illustrated below, calculating the asymptotic upper bound involves finding a function $f(n)$ such that, as $n$ approaches infinity, $T(n)$ and $f(n)$ have the same growth order, differing only by a constant factor $c$.
|
||||
|
||||
As shown in the figure below, computing the asymptotic upper bound is a matter of finding a function $f(n)$ such that $T(n)$ and $f(n)$ are at the same growth level as $n$ tends to infinity, differing only by a multiple of the constant term $c$.
|
||||
|
||||
|
||||
|
||||
## Calculation Method
|
||||
|
||||
## Method Of Projection
|
||||
While the concept of asymptotic upper bound might seem mathematically dense, you don't need to fully grasp it right away. Let's first understand the method of calculation, which can be practiced and comprehended over time.
|
||||
|
||||
Asymptotic upper bounds are a bit heavy on math, so don't worry if you feel you don't have a full solution. Because in practice, we only need to master the projection method, and the mathematical meaning can be gradually comprehended.
|
||||
Once $f(n)$ is determined, we obtain the time complexity $O(f(n))$. But how do we determine the asymptotic upper bound $f(n)$? This process generally involves two steps: counting the number of operations and determining the asymptotic upper bound.
|
||||
|
||||
By definition, after determining $f(n)$, we can get the time complexity $O(f(n))$. So how to determine the asymptotic upper bound $f(n)$? The overall is divided into two steps: first count the number of operations, and then determine the asymptotic upper bound.
|
||||
### Step 1: Counting the Number of Operations
|
||||
|
||||
### The First Step: Counting The Number Of Operations
|
||||
This step involves going through the code line by line. However, due to the presence of the constant $c$ in $c \cdot f(n)$, **all coefficients and constant terms in $T(n)$ can be ignored**. This principle allows for simplification techniques in counting operations.
|
||||
|
||||
For the code, it is sufficient to calculate from top to bottom line by line. However, since the constant term $c \cdot f(n)$ in the above $c \cdot f(n)$ can take any size, **the various coefficients and constant terms in the number of operations $T(n)$ can be ignored**. Based on this principle, the following counting simplification techniques can be summarized.
|
||||
1. **Ignore constant terms in $T(n)$**, as they do not affect the time complexity being independent of $n$.
|
||||
2. **Omit all coefficients**. For example, looping $2n$, $5n + 1$ times, etc., can be simplified to $n$ times since the coefficient before $n$ does not impact the time complexity.
|
||||
3. **Use multiplication for nested loops**. The total number of operations equals the product of the number of operations in each loop, applying the simplification techniques from points 1 and 2 for each loop level.
|
||||
|
||||
1. **Ignore the constant terms in $T(n)$**. Since none of them are related to $n$, they have no effect on the time complexity.
|
||||
2. **omits all coefficients**. For example, loops $2n$ times, $5n + 1$ times, etc., can be simplified and notated as $n$ times because the coefficients before $n$ have no effect on the time complexity.
|
||||
3. **Use multiplication** when loops are nested. The total number of operations is equal to the product of the number of operations of the outer and inner levels of the loop, and each level of the loop can still be nested by applying the techniques in points `1.` and `2.` respectively.
|
||||
|
||||
Given a function, we can use the above trick to count the number of operations.
|
||||
Given a function, we can use these techniques to count operations:
|
||||
|
||||
=== "Python"
|
||||
|
||||
@ -901,192 +899,190 @@ Given a function, we can use the above trick to count the number of operations.
|
||||
}
|
||||
```
|
||||
|
||||
The following equations show the statistical results before and after using the above technique, both of which were introduced with a time complexity of $O(n^2)$ .
|
||||
The formula below shows the counting results before and after simplification, both leading to a time complexity of $O(n^2)$:
|
||||
|
||||
$$
|
||||
\begin{aligned}
|
||||
T(n) & = 2n(n + 1) + (5n + 1) + 2 & \text{complete statistics (-.-|||)} \newline
|
||||
T(n) & = 2n(n + 1) + (5n + 1) + 2 & \text{Complete Count (-.-|||)} \newline
|
||||
& = 2n^2 + 7n + 3 \newline
|
||||
T(n) & = n^2 + n & \text{Lazy Stats (o.O)}
|
||||
T(n) & = n^2 + n & \text{Simplified Count (o.O)}
|
||||
\end{aligned}
|
||||
$$
|
||||
|
||||
### Step 2: Judging The Asymptotic Upper Bounds
|
||||
### Step 2: Determining the Asymptotic Upper Bound
|
||||
|
||||
**The time complexity is determined by the highest order term in the polynomial $T(n)$**. This is because as $n$ tends to infinity, the highest order term will play a dominant role and the effects of all other terms can be ignored.
|
||||
**The time complexity is determined by the highest order term in $T(n)$**. This is because, as $n$ approaches infinity, the highest order term dominates, rendering the influence of other terms negligible.
|
||||
|
||||
The table below shows some examples, some of which have exaggerated values to emphasize the conclusion that "the coefficients can't touch the order". As $n$ tends to infinity, these constants become irrelevant.
|
||||
The following table illustrates examples of different operation counts and their corresponding time complexities. Some exaggerated values are used to emphasize that coefficients cannot alter the order of growth. When $n$ becomes very large, these constants become insignificant.
|
||||
|
||||
<p align="center"> Table <id> Time complexity corresponding to different number of operations </p>
|
||||
<p align="center"> Table: Time Complexity for Different Operation Counts </p>
|
||||
|
||||
| number of operations $T(n)$ | time complexity $O(f(n))$ |
|
||||
| --------------------------- | ------------------------- |
|
||||
| $100000$ | $O(1)$ |
|
||||
| $3n + 2$ | $O(n)$ |
|
||||
| $2n^2 + 3n + 2$ | $O(n^2)$ |
|
||||
| $n^3 + 10000n^2$ | $O(n^3)$ |
|
||||
| $2^n + 10000n^{10000}$ | $O(2^n)$ |
|
||||
| Operation Count $T(n)$ | Time Complexity $O(f(n))$ |
|
||||
| ---------------------- | ------------------------- |
|
||||
| $100000$ | $O(1)$ |
|
||||
| $3n + 2$ | $O(n)$ |
|
||||
| $2n^2 + 3n + 2$ | $O(n^2)$ |
|
||||
| $n^3 + 10000n^2$ | $O(n^3)$ |
|
||||
| $2^n + 10000n^{10000}$ | $O(2^n)$ |
|
||||
|
||||
## Common Types
|
||||
## Common Types of Time Complexity
|
||||
|
||||
Let the input data size be $n$ , the common types of time complexity are shown in the figure below (in descending order).
|
||||
Let's consider the input data size as $n$. The common types of time complexities are illustrated below, arranged from lowest to highest:
|
||||
|
||||
$$
|
||||
\begin{aligned}
|
||||
O(1) < O(\log n) < O(n) < O(n \log n) < O(n^2) < O(2^n) < O(n!) \newline
|
||||
\text{constant order} < \text{logarithmic order} < \text{linear order} < \text{linear logarithmic order} < \text{square order} < \text{exponential order} < \text{multiplication order}
|
||||
\text{Constant Order} < \text{Logarithmic Order} < \text{Linear Order} < \text{Linear-Logarithmic Order} < \text{Quadratic Order} < \text{Exponential Order} < \text{Factorial Order}
|
||||
\end{aligned}
|
||||
$$
|
||||
|
||||
|
||||
|
||||
|
||||
### Constant Order $O(1)$
|
||||
|
||||
The number of operations of the constant order is independent of the input data size $n$, i.e., it does not change with $n$.
|
||||
|
||||
In the following function, although the number of operations `size` may be large, the time complexity is still $O(1)$ because it is independent of the input data size $n$ :
|
||||
Constant order means the number of operations is independent of the input data size $n$. In the following function, although the number of operations `size` might be large, the time complexity remains $O(1)$ as it's unrelated to $n$:
|
||||
|
||||
```src
|
||||
[file]{time_complexity}-[class]{}-[func]{constant}
|
||||
```
|
||||
|
||||
### Linear Order $O(N)$
|
||||
### Linear Order $O(n)$
|
||||
|
||||
The number of operations in a linear order grows in linear steps relative to the input data size $n$. Linear orders are usually found in single level loops:
|
||||
Linear order indicates the number of operations grows linearly with the input data size $n$. Linear order commonly appears in single-loop structures:
|
||||
|
||||
```src
|
||||
[file]{time_complexity}-[class]{}-[func]{linear}
|
||||
```
|
||||
|
||||
The time complexity of operations such as traversing an array and traversing a linked list is $O(n)$ , where $n$ is the length of the array or linked list:
|
||||
Operations like array traversal and linked list traversal have a time complexity of $O(n)$, where $n$ is the length of the array or list:
|
||||
|
||||
```src
|
||||
[file]{time_complexity}-[class]{}-[func]{array_traversal}
|
||||
```
|
||||
|
||||
It is worth noting that **Input data size $n$ needs to be determined specifically** according to the type of input data. For example, in the first example, the variable $n$ is the input data size; in the second example, the array length $n$ is the data size.
|
||||
It's important to note that **the input data size $n$ should be determined based on the type of input data**. For example, in the first example, $n$ represents the input data size, while in the second example, the length of the array $n$ is the data size.
|
||||
|
||||
### Squared Order $O(N^2)$
|
||||
### Quadratic Order $O(n^2)$
|
||||
|
||||
The number of operations in the square order grows in square steps with respect to the size of the input data $n$. The squared order is usually found in nested loops, where both the outer and inner levels are $O(n)$ and therefore overall $O(n^2)$:
|
||||
Quadratic order means the number of operations grows quadratically with the input data size $n$. Quadratic order typically appears in nested loops, where both the outer and inner loops have a time complexity of $O(n)$, resulting in an overall complexity of $O(n^2)$:
|
||||
|
||||
```src
|
||||
[file]{time_complexity}-[class]{}-[func]{quadratic}
|
||||
```
|
||||
|
||||
The figure below compares the three time complexities of constant order, linear order and squared order.
|
||||
The following image compares constant order, linear order, and quadratic order time complexities.
|
||||
|
||||
|
||||
|
||||
|
||||
Taking bubble sort as an example, the outer loop executes $n - 1$ times, and the inner loop executes $n-1$, $n-2$, $\dots$, $2$, $1$ times, which averages out to $n / 2$ times, resulting in a time complexity of $O((n - 1) n / 2) = O(n^2)$ .
|
||||
For instance, in bubble sort, the outer loop runs $n - 1$ times, and the inner loop runs $n-1$, $n-2$, ..., $2$, $1$ times, averaging $n / 2$ times, resulting in a time complexity of $O((n - 1) n / 2) = O(n^2)$:
|
||||
|
||||
```src
|
||||
[file]{time_complexity}-[class]{}-[func]{bubble_sort}
|
||||
```
|
||||
|
||||
## Exponential Order $O(2^N)$
|
||||
### Exponential Order $O(2^n)$
|
||||
|
||||
Cell division in biology is a typical example of exponential growth: the initial state is $1$ cells, after one round of division it becomes $2$, after two rounds of division it becomes $4$, and so on, after $n$ rounds of division there are $2^n$ cells.
|
||||
Biological "cell division" is a classic example of exponential order growth: starting with one cell, it becomes two after one division, four after two divisions, and so on, resulting in $2^n$ cells after $n$ divisions.
|
||||
|
||||
The figure below and the following code simulate the process of cell division with a time complexity of $O(2^n)$ .
|
||||
The following image and code simulate the cell division process, with a time complexity of $O(2^n)$:
|
||||
|
||||
```src
|
||||
[file]{time_complexity}-[class]{}-[func]{exponential}
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
In practical algorithms, exponential orders are often found in recursion functions. For example, in the following code, it recursively splits in two and stops after $n$ splits:
|
||||
In practice, exponential order often appears in recursive functions. For example, in the code below, it recursively splits into two halves, stopping after $n$ divisions:
|
||||
|
||||
```src
|
||||
[file]{time_complexity}-[class]{}-[func]{exp_recur}
|
||||
```
|
||||
|
||||
Exponential order grows very rapidly and is more common in exhaustive methods (brute force search, backtracking, etc.). For problems with large data sizes, exponential order is unacceptable and usually requires the use of algorithms such as dynamic programming or greedy algorithms to solve.
|
||||
Exponential order growth is extremely rapid and is commonly seen in exhaustive search methods (brute force, backtracking, etc.). For large-scale problems, exponential order is unacceptable, often requiring dynamic programming or greedy algorithms as solutions.
|
||||
|
||||
### Logarithmic Order $O(\Log N)$
|
||||
### Logarithmic Order $O(\log n)$
|
||||
|
||||
In contrast to the exponential order, the logarithmic order reflects the "each round is reduced to half" case. Let the input data size be $n$, and since each round is reduced to half, the number of loops is $\log_2 n$, which is the inverse function of $2^n$.
|
||||
In contrast to exponential order, logarithmic order reflects situations where "the size is halved each round." Given an input data size $n$, since the size is halved each round, the number of iterations is $\log_2 n$, the inverse function of $2^n$.
|
||||
|
||||
The figure below and the code below simulate the process of "reducing each round to half" with a time complexity of $O(\log_2 n)$, which is abbreviated as $O(\log n)$.
|
||||
The following image and code simulate the "halving each round" process, with a time complexity of $O(\log_2 n)$, commonly abbreviated as $O(\log n)$:
|
||||
|
||||
```src
|
||||
[file]{time_complexity}-[class]{}-[func]{logarithmic}
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
Similar to the exponential order, the logarithmic order is often found in recursion functions. The following code forms a recursion tree of height $\log_2 n$:
|
||||
Like exponential order, logarithmic order also frequently appears in recursive functions. The code below forms a recursive tree of height $\log_2 n$:
|
||||
|
||||
```src
|
||||
[file]{time_complexity}-[class]{}-[func]{log_recur}
|
||||
```
|
||||
|
||||
Logarithmic order is often found in algorithms based on the divide and conquer strategy, which reflects the algorithmic ideas of "dividing one into many" and "simplifying the complexity into simplicity". It grows slowly and is the second most desirable time complexity after constant order.
|
||||
Logarithmic order is typical in algorithms based on the divide-and-conquer strategy, embodying the "split into many" and "simplify complex problems" approach. It's slow-growing and is the most ideal time complexity after constant order.
|
||||
|
||||
!!! tip "What is the base of $O(\log n)$?"
|
||||
|
||||
To be precise, the corresponding time complexity of "one divided into $m$" is $O(\log_m n)$ . And by using the logarithmic permutation formula, we can get equal time complexity with different bases:
|
||||
Technically, "splitting into $m$" corresponds to a time complexity of $O(\log_m n)$. Using the logarithm base change formula, we can equate different logarithmic complexities:
|
||||
|
||||
$$
|
||||
O(\log_m n) = O(\log_k n / \log_k m) = O(\log_k n)
|
||||
$$
|
||||
|
||||
That is, the base $m$ can be converted without affecting the complexity. Therefore we usually omit the base $m$ and write the logarithmic order directly as $O(\log n)$.
|
||||
This means the base $m$ can be changed without affecting the complexity. Therefore, we often omit the base $m$ and simply denote logarithmic order as $O(\log n)$.
|
||||
|
||||
### Linear Logarithmic Order $O(N \Log N)$
|
||||
### Linear-Logarithmic Order $O(n \log n)$
|
||||
|
||||
The linear logarithmic order is often found in nested loops, and the time complexity of the two levels of loops is $O(\log n)$ and $O(n)$ respectively. The related code is as follows:
|
||||
Linear-logarithmic order often appears in nested loops, with the complexities of the two loops being $O(\log n)$ and $O(n)$ respectively. The related code is as follows:
|
||||
|
||||
```src
|
||||
[file]{time_complexity}-[class]{}-[func]{linear_log_recur}
|
||||
```
|
||||
|
||||
The figure below shows how the linear logarithmic order is generated. The total number of operations at each level of the binary tree is $n$ , and the tree has a total of $\log_2 n + 1$ levels, resulting in a time complexity of $O(n\log n)$ .
|
||||
The image below demonstrates how linear-logarithmic order is generated. Each level of a binary tree has $n$ operations, and the tree has $\log_2 n + 1$ levels, resulting in a time complexity of $O(n \log n)$.
|
||||
|
||||
|
||||
|
||||
|
||||
Mainstream sorting algorithms typically have a time complexity of $O(n \log n)$ , such as quick sort, merge sort, heap sort, etc.
|
||||
Mainstream sorting algorithms typically have a time complexity of $O(n \log n)$, such as quicksort, mergesort, and heapsort.
|
||||
|
||||
### The Factorial Order $O(N!)$
|
||||
### Factorial Order $O(n!)$
|
||||
|
||||
The factorial order corresponds to the mathematical "permutations problem". Given $n$ elements that do not repeat each other, find all possible permutations of them, the number of permutations being:
|
||||
Factorial order corresponds to the mathematical problem of "full permutation." Given $n$ distinct elements, the total number of possible permutations is:
|
||||
|
||||
$$
|
||||
n! = n \times (n - 1) \times (n - 2) \times \dots \times 2 \times 1
|
||||
$$
|
||||
|
||||
Factorials are usually implemented using recursion. As shown in the figure below and in the code below, the first level splits $n$, the second level splits $n - 1$, and so on, until the splitting stops at the $n$th level:
|
||||
Factorials are typically implemented using recursion. As shown in the image and code below, the first level splits into $n$ branches, the second level into $n - 1$ branches, and so on, stopping after the $n$th level:
|
||||
|
||||
```src
|
||||
[file]{time_complexity}-[class]{}-[func]{factorial_recur}
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
Note that since there is always $n! > 2^n$ when $n \geq 4$, the factorial order grows faster than the exponential order, and is also unacceptable when $n$ is large.
|
||||
Note that factorial order grows even faster than exponential order; it's unacceptable for larger $n$ values.
|
||||
|
||||
## Worst, Best, Average Time Complexity
|
||||
## Worst, Best, and Average Time Complexities
|
||||
|
||||
**The time efficiency of algorithms is often not fixed, but is related to the distribution of the input data**. Suppose an array `nums` of length $n$ is input, where `nums` consists of numbers from $1$ to $n$, each of which occurs only once; however, the order of the elements is randomly upset, and the goal of the task is to return the index of element $1$. We can draw the following conclusion.
|
||||
**The time efficiency of an algorithm is often not fixed but depends on the distribution of the input data**. Assume we have an array `nums` of length $n$, consisting of numbers from $1$ to $n$, each appearing only once, but in a randomly shuffled order. The task is to return the index of the element $1$. We can draw the following conclusions:
|
||||
|
||||
- When `nums = [? , ? , ... , 1]` , i.e., when the end element is $1$, a complete traversal of the array is required, **to reach the worst time complexity $O(n)$** .
|
||||
- When `nums = [1, ? , ? , ...]` , i.e., when the first element is $1$ , there is no need to continue traversing the array no matter how long it is, **reaching the optimal time complexity $\Omega(1)$** .
|
||||
- When `nums = [?, ?, ..., 1]`, that is, when the last element is $1$, it requires a complete traversal of the array, **achieving the worst-case time complexity of $O(n)$**.
|
||||
- When `nums = [1, ?, ?, ...]`, that is, when the first element is $1$, no matter the length of the array, no further traversal is needed, **achieving the best-case time complexity of $\Omega(1)$**.
|
||||
|
||||
The "worst time complexity" corresponds to the asymptotic upper bound of the function and is denoted by the large $O$ notation. Correspondingly, the "optimal time complexity" corresponds to the asymptotic lower bound of the function and is denoted in $\Omega$ notation:
|
||||
The "worst-case time complexity" corresponds to the asymptotic upper bound, denoted by the big $O$ notation. Correspondingly, the "best-case time complexity" corresponds to the asymptotic lower bound, denoted by $\Omega$:
|
||||
|
||||
```src
|
||||
[file]{worst_best_time_complexity}-[class]{}-[func]{find_one}
|
||||
```
|
||||
|
||||
It is worth stating that we rarely use the optimal time complexity in practice because it is usually only attainable with a small probability and may be somewhat misleading. **whereas the worst time complexity is more practical because it gives a safe value for efficiency and allows us to use the algorithm with confidence**.
|
||||
It's important to note that the best-case time complexity is rarely used in practice, as it is usually only achievable under very low probabilities and might be misleading. **The worst-case time complexity is more practical as it provides a safety value for efficiency**, allowing us to confidently use the algorithm.
|
||||
|
||||
From the above examples, it can be seen that the worst or best time complexity only occurs in "special data distributions", and the probability of these cases may be very small, which does not truly reflect the efficiency of the algorithm. In contrast, **the average time complexity of can reflect the efficiency of the algorithm under random input data**, which is denoted by the $\Theta$ notation.
|
||||
From the above example, it's clear that both the worst-case and best-case time complexities only occur under "special data distributions," which may have a small probability of occurrence and may not accurately reflect the algorithm's run efficiency. In contrast, **the average time complexity can reflect the algorithm's efficiency under random input data**, denoted by the $\Theta$ notation.
|
||||
|
||||
For some algorithms, we can simply derive the average case under a random data distribution. For example, in the above example, since the input array is scrambled, the probability of an element $1$ appearing at any index is equal, so the average number of loops of the algorithm is half of the length of the array $n / 2$ , and the average time complexity is $\Theta(n / 2) = \Theta(n)$ .
|
||||
For some algorithms, we can simply estimate the average case under a random data distribution. For example, in the aforementioned example, since the input array is shuffled, the probability of element $1$ appearing at any index is equal. Therefore, the average number of loops for the algorithm is half the length of the array $n / 2$, giving an average time complexity of $\Theta(n / 2) = \Theta(n)$.
|
||||
|
||||
However, for more complex algorithms, calculating the average time complexity is often difficult because it is hard to analyze the overall mathematical expectation given the data distribution. In this case, we usually use the worst time complexity as a criterion for the efficiency of the algorithm.
|
||||
However, calculating the average time complexity for more complex algorithms can be quite difficult, as it's challenging to analyze the overall mathematical expectation under the data distribution. In such cases, we usually use the worst-case time complexity as the standard for judging the efficiency of the algorithm.
|
||||
|
||||
!!! question "Why do you rarely see the $\Theta$ symbol?"
|
||||
!!! question "Why is the $\Theta$ symbol rarely seen?"
|
||||
|
||||
Perhaps because the $O$ symbol is so catchy, we often use it to denote average time complexity. However, this practice is not standardized in the strict sense. In this book and other sources, if you encounter a statement like "average time complexity $O(n)$", please understand it as $\Theta(n)$.
|
||||
Possibly because the $O$ notation is more commonly spoken, it is often used to represent the average time complexity. However, strictly speaking, this practice is not accurate. In this book and other materials, if you encounter statements like "average time complexity $O(n)$", please understand it directly as $\Theta(n)$.
|
||||
|
||||
Reference in New Issue
Block a user