16 Differentiation, Derivative and Rules

Lecture from: 22.04.2024 | Video: Video ETHZ

Differentiable Functions

Clicker Question: Finding a Minimum

Consider the function $f(x)=x^4-3x^2+x+4$. We are at the point $x_0=1$, where $f(x_0)=1-3+1+4=3$.

Observation: A minimum for this function exists because $f$ is continuous, and as $x\to \pm \infty$, $f(x)\to \infty$ (the $x^4$ term dominates). This means the function must “turn around” somewhere.

The graph shows $f(x_0)=3$. If we take a small step $\Delta x$ to the right from $x_0$, say to $x_0+\Delta x$, the function value $f(x_0+\Delta x)$ is slightly less than $f(x_0)$. This means $\Delta f=f(x_0+\Delta x)-f(x_0)<0$.

• Question: Starting at $x_0=1$, should we move left or right to find a minimum of $f$?
• Answer: The graph suggests that moving to the right might lead us towards a minimum, because the function is decreasing in the immediate vicinity to the right of $x_0$ (since $\Delta f<0$ for a positive $\Delta x$). (This is a baby example of an idea called Gradient Descent.)

Follow-up Question: How could we have made this decision (left or right) if we didn’t have the graph? This is what we want to explore!

Let’s look at a broader view of the function in this example:

It seems our initial step to the right from $x_0=1$ led us to a minimum, but maybe not the overall (global) minimum.

Question: How can we find or describe minima if we don’t have a picture of the function?

Unfortunately, the minimum we found by moving slightly to the right from $x_0=1$ was only a local minimum, not the true “global” minimum of the function.

Key Question: What conclusions can we draw about the global behavior of a function based on its local behavior?

Goal of This Chapter

• Investigate the local change of functions. To do this, we’ll use local approximation by straight lines (these lines are called “tangents”).
• Direct Applications:
  • Determining the monotonicity of functions (where they are increasing or decreasing).
  • Finding local extrema (minima and maxima).
  • Taylor Approximation (approximating functions with polynomials of higher order).

The Derivative: Definition and Elementary Consequences

Let $D\subseteq \mathbb{R}$, $f:D\to \mathbb{R}$, and $x_0\in D$ be an accumulation point of $D$.

We want to define the “slope of the tangent line” to $f$ at $x_0$ as the limit of the “slopes of secant lines.”

The slope of the secant line between $(x_0,f(x_0))$ and $(x,f(x))$ is given by the difference quotient: $\frac{\Delta f}{\Delta x}=\frac{f(x)-f(x_0)}{x-x_0}$ This difference quotient should provide a good approximation for the slope of $f$ at $x_0$ if $x$ is close to $x_0$.

Definition: Differentiability

Let $D\subseteq \mathbb{R}$, $f:D\to \mathbb{R}$, and $x_0\in D$ be an accumulation point of $D$. The function $f$ is differentiable at $x_0$ if the limit ${\lim }_{x\to x_0}\frac{f(x)-f(x_0)}{x-x_0}$ exists in $\mathbb{R}$ (i.e., it’s a finite real number). If this limit exists, it is called the derivative of $f$ at $x_0$ and is denoted by $f^{\prime }(x_0)$.

Remark: Alternative Limit Form for the Derivative

It is often convenient to set $x=x_0+h$. As $x\to x_0$, we have $h\to 0$. Then the derivative can also be expressed as: $f^{\prime }(x_0)={\lim }_{h\to 0}\frac{f(x_0+h)-f(x_0)}{h}$

Remark: The Tangent Line

If $f:D\to \mathbb{R}$ is differentiable at $x_0\in D$, then the line $g$ defined by the equation $g(x)=f(x_0)+f^{\prime }(x_0)(x-x_0)$ is called the tangent line to $f$ at $x_0$. This line is the best linear approximation of $f$ near $x_0$.

The “approximation error” $f(x)-g(x)$ converges to $0$ faster than $x-x_0$ as $x\to x_0$. Specifically, it holds that: ${\lim }_{x\to x_0}\frac{f(x)-g(x)}{x-x_0}=0$ Exercise: Verify this limit using the definition of $g(x)$ and $f^{\prime }(x_0)$.

Theorem: Reformulation of Differentiability

Let $D\subseteq \mathbb{R}$, and let $f:D\to \mathbb{R}$ be a function. Suppose $x_0\in D$ is an accumulation point of $D$.

Then $f$ is differentiable at $x_0$ if and only if there exists a function $\Phi :D\to \mathbb{R}$ that is continuous at $x_0$ and satisfies the equation:

In this case, the value of $\Phi$ at $x_0$ gives the derivative:

Explanation: What is $\Phi$ doing?

Here’s an intuitive explanation of what this $\varphi (x)$ is doing:

1. Rearranging the Formula: For any $x\ne x_0$ in the domain $D$, you can rearrange the formula to solve for $\varphi (x)$: $\varphi (x)=\frac{f(x)-f(x_0)}{x-x_0}$ This fraction, $\frac{f(x)-f(x_0)}{x-x_0}$, is the slope of the secant line connecting the point $(x_0,f(x_0))$ and any other point $(x,f(x))$ on the graph of $f$. So, for $x\ne x_0$, $\varphi (x)$ exactly represents this average slope between $x_0$ and $x$.

2. Connecting to Differentiability: Satz 4.1.4 states that a function $f$ is differentiable at $x_0$ if and only if such a function $\varphi (x)$ exists that is defined on $D$ and is continuous at $x_0$.

3. The Role of Continuity: The condition that $\varphi (x)$ is continuous at $x_0$ means, by Definition 3.2.1 or its characterization in Satz 3.2.4, that the limit of $\varphi (x)$ as $x$ approaches $x_0$ exists and is equal to $\varphi (x_0)$. That is, ${\lim }_{x\to x_0}\varphi (x)=\varphi (x_0)$.

4. What $\varphi (x_0)$ Must Be: Now, let’s combine these ideas. Since $\varphi (x)=\frac{f(x)-f(x_0)}{x-x_0}$ for $x\ne x_0$, and ${\lim }_{x\to x_0}\varphi (x)$ exists and equals $\varphi (x_0)$, this means: ${\lim }_{x\to x_0}\frac{f(x)-f(x_0)}{x-x_0}=\varphi (x_0)$ By the definition of the derivative (Definition 4.1.1), the limit on the left is exactly $f^{\prime }(x_0)$. Therefore, if $f$ is differentiable at $x_0$ and satisfies the condition in Satz 4.1.4, it must be that $\varphi (x_0)=f^{\prime }(x_0)$.

Why is this important?

• The formula:

  $$f(x)=f(x_0)+\Phi (x)(x-x_0)$$

  looks like the linear approximation of $f$, but with a twist - the slope depends on $x$, not just a fixed derivative.

• If $\Phi$ is continuous at $x_0$, it ensures that the “slopes” smoothly approach the true derivative. This gives us differentiability.

• Key idea: Differentiability at a point is equivalent to being able to write the function locally as a linear approximation with a continuous slope function $\Phi$.

Sketch of Proof

1. Define $\Phi (x)$ as above.
2. If $f$ is differentiable at $x_0$, then the difference quotient converges to $f^{\prime }(x_0)$, so $\Phi (x)\to f^{\prime }(x_0)$. That makes $\Phi$ continuous at $x_0$, and the formula holds.
3. Conversely, if the formula holds and $\Phi$ is continuous at $x_0$, then the difference quotient equals $\Phi (x)$, and: $$\underset{x\to x_0}{\lim }\frac{f(x)-f(x_0)}{x-x_0}=\underset{x\to x_0}{\lim }\Phi (x)=\Phi (x_0)$$ So the derivative exists, and equals $\Phi (x_0)$.

Corollary: Differentiability Implies Continuity

If $f$ is differentiable at $x_0$, then $f$ is continuous at $x_0$.

Proof

If $f$ is differentiable at $x_0$, then by the theorem above, $f(x)=f(x_0)+\Phi (x)(x-x_0)$ where $\Phi$ is continuous at $x_0$.

Then ${\lim }_{x\to x_0}f(x)={\lim }_{x\to x_0}(f(x_0)+\Phi (x)(x-x_0))=f(x_0)+\Phi (x_0)\cdot 0=f(x_0)$. Since ${\lim }_{x\to x_0}f(x)=f(x_0)$, $f$ is continuous at $x_0$.

Examples of Derivatives

1. Constant Function: $f(x)=c$ (for some constant $c$) for all $x\in \mathbb{R}$. Then $f^{\prime }(x_0)={\lim }_{x\to x_0}\frac{f(x)-f(x_0)}{x-x_0}={\lim }_{x\to x_0}\frac{c-c}{x-x_0}={\lim }_{x\to x_0}0=0$. So, $f^{\prime }(x_0)=0$ for all $x_0\in \mathbb{R}$.

2. Identity Function: $f(x)=x$ for all $x\in \mathbb{R}$. Then $f^{\prime }(x_0)={\lim }_{x\to x_0}\frac{x-x_0}{x-x_0}={\lim }_{x\to x_0}1=1$. So, $f^{\prime }(x_0)=1$ for all $x_0\in \mathbb{R}$.

3. Quadratic Function: $f(x)=x^2$ for all $x\in \mathbb{R}$. $f^{\prime }(x_0)={\lim }_{x\to x_0}\frac{x^2-x_0^2}{x-x_0}={\lim }_{x\to x_0}\frac{(x-x_0)(x+x_0)}{x-x_0}={\lim }_{x\to x_0}(x+x_0)=x_0+x_0=2x_0$. So, $f^{\prime }(x_0)=2x_0$ for all $x_0\in \mathbb{R}$.

4. Absolute Value Function: $f(x)=|x|$ for all $x\in \mathbb{R}$. Is $f(x)=|x|$ differentiable at $x_0=0$? Consider the limit ${\lim }_{x\to 0}\frac{|x|-|0|}{x-0}={\lim }_{x\to 0}\frac{|x|}{x}$. If $x>0$, $\frac{|x|}{x}=\frac{x}{x}=1$. So, ${\lim }_{x\to 0^{+}}\frac{|x|}{x}=1$. If $x<0$, $\frac{|x|}{x}=\frac{-x}{x}=-1$. So, ${\lim }_{x\to 0^{-}}\frac{|x|}{x}=-1$. Since the right-sided limit (1) and the left-sided limit (-1) are not equal, the overall limit ${\lim }_{x\to 0}\frac{|x|}{x}$ does not exist. Therefore, $f(x)=|x|$ is not differentiable at $x_0=0$.

   

   For any other point $x_0\ne 0$, $f(x)=|x|$ is differentiable: If $x_0>0$, then for $x$ near $x_0$, $|x|=x$, so $f^{\prime }(x_0)=1$. If $x_0<0$, then for $x$ near $x_0$, $|x|=-x$, so $f^{\prime }(x_0)=-1$. So, for $x_0\ne 0$, $f^{\prime }(x_0)=\{\begin{array}{ll}1& \text{if }{x}_0>0\\ -1& \text{if }{x}_0<0\end{array}$.

Definition: Differentiable Function

Let $D\subseteq \mathbb{R}$, $f:D\to \mathbb{R}$. The function $f$ is differentiable (on $D$) if $f$ is differentiable at all accumulation points $x_0$ of $D$ with $x_0\in D$.

Remark: Typically, $D$ is a union of intervals $I_m$ with endpoints $a_m<b_m$. In this case, every point of $D$ is an accumulation point of $D$.

In this case, for a differentiable function $f:D\to \mathbb{R}$, we get a new function $f^{\prime }:D\to \mathbb{R}$, called the derivative of $f$.

(Example: $D=(-1,0]\cup (1,2)\cup [3,\infty )$).

Examples of Differentiable Functions

1. $\exp :\mathbb{R}\to \mathbb{R}$ is differentiable, and ${\exp }^{\prime }=\exp$.
2. $\sin ,\cos :\mathbb{R}\to \mathbb{R}$ are differentiable. ${\sin }^{\prime }=\cos$, ${\cos }^{\prime }=-\sin$.

Proof for ${\exp }^{\prime }(x_0)=\exp (x_0)$

Consider $\frac{\exp (x_0+h)-\exp (x_0)}{h}$ for $h\ne 0$. $\frac{\exp (x_0)\exp (h)-\exp (x_0)}{h}=\exp (x_0)\frac{\exp (h)-1}{h}$ For $h\ne 0$, $\frac{\exp (h)-1}{h}=\frac{(1+h+h^2/2!+h^3/3!+\dots )-1}{h}=\frac{h+h^2/2!+h^3/3!+\dots }{h}=1+\frac{h}{2!}+\frac{h^2}{3!}+\dots$

Let $r(h)=1+\frac{h}{2!}+\frac{h^2}{3!}+\dots$. This is a power series (in $h$) with radius of convergence $\infty$.

Since $r(h)$ is a convergent power series, it is continuous at $h=0$ (by Theorem 3.7.11).

Therefore, ${\lim }_{h\to 0}\frac{\exp (h)-1}{h}={\lim }_{h\to 0}r(h)=r(0)=1$. So, ${\lim }_{h\to 0}\exp (x_0)\frac{\exp (h)-1}{h}=\exp (x_0)\cdot 1=\exp (x_0)$.

Thus, ${\exp }^{\prime }(x_0)=\exp (x_0)$.

(For $\sin ,\cos$, see script.)

Rules for Differentiation

Theorem: Differentiation Rules

Let $D\subseteq \mathbb{R}$, $x_0\in D$ be an accumulation point of $D$, and $c\in \mathbb{R}$. Let $f,g:D\to \mathbb{R}$ be differentiable at $x_0$. Then:

1. Linearity of the Derivative: The functions $c\cdot f$ and $f+g$ are differentiable at $x_0$, and

   • $(c\cdot f{)}^{\prime }(x_0)=c\cdot f^{\prime }(x_0)$
   • $(f+g{)}^{\prime }(x_0)=f^{\prime }(x_0)+g^{\prime }(x_0)$

2. Product Rule: The function $f\cdot g$ is differentiable at $x_0$, and

   • $(f\cdot g{)}^{\prime }(x_0)=f^{\prime }(x_0)g(x_0)+f(x_0)g^{\prime }(x_0)$

3. Quotient Rule: If $g(x_0)\ne 0$, then $x_0$ is an accumulation point of $\{x\in D\mid g(x)\ne 0\}$, and the function $f/g$ is differentiable at $x_0$ (on this new domain), with

   • ${\left(\frac{f}{g}\right)}^{\prime }(x_0)=\frac{f^{\prime }(x_0)g(x_0)-f(x_0)g^{\prime }(x_0)}{(g(x_0){)}^2}$

Proof (using Theorem 4.1.4 - Reformulation of Differentiability)

Since $f$ and $g$ are differentiable at $x_0$, there exist functions $\Phi ,\Psi :D\to \mathbb{R}$, continuous at $x_0$, such that for all $x\in D$:

1. $f(x)=f(x_0)+\Phi (x)(x-x_0)$ with $\Phi (x_0)=f^{\prime }(x_0)$
2. $g(x)=g(x_0)+\Psi (x)(x-x_0)$ with $\Psi (x_0)=g^{\prime }(x_0)$

Sum for 1

$f(x)+g(x)=(f(x_0)+g(x_0))+(\Phi (x)+\Psi (x))(x-x_0)$. Since $\Phi$ and $\Psi$ are continuous at $x_0$, their sum $\Phi +\Psi$ is also continuous at $x_0$.

By Theorem 4.1.4, $f+g$ is differentiable at $x_0$, and $(f+g{)}^{\prime }(x_0)=(\Phi +\Psi )(x_0)=\Phi (x_0)+\Psi (x_0)=f^{\prime }(x_0)+g^{\prime }(x_0)$.

Product for 2

$f(x)g(x)=(f(x_0)+\Phi (x)(x-x_0))(g(x_0)+\Psi (x)(x-x_0))$ $f(x)g(x)=f(x_0)g(x_0)+f(x_0)\Psi (x)(x-x_0)+g(x_0)\Phi (x)(x-x_0)+\Phi (x)\Psi (x)(x-x_0{)}^2$ $f(x)g(x)=f(x_0)g(x_0)+\underset{\text{Let this be }\Theta (x)}{\underbrace{[f(x_0)\Psi (x)+g(x_0)\Phi (x)+\Phi (x)\Psi (x)(x-x_0)]}}(x-x_0)$ The term $\Theta (x)$ is continuous at $x_0$ because $\Phi ,\Psi$ are continuous at $x_0$, and $x-x_0\to 0$ as $x\to x_0$.

Specifically, $\Theta (x_0)=f(x_0)\Psi (x_0)+g(x_0)\Phi (x_0)+\Phi (x_0)\Psi (x_0)(0)=f(x_0)g^{\prime }(x_0)+g(x_0)f^{\prime }(x_0)$.

By Theorem 4.1.4, $f\cdot g$ is differentiable at $x_0$, and $(f\cdot g{)}^{\prime }(x_0)=\Theta (x_0)=f^{\prime }(x_0)g(x_0)+f(x_0)g^{\prime }(x_0)$.

Properties (1) and (2) are proven. For (3), see script.

Examples: Applying Differentiation Rules

1. Power Rule (by induction): For $n\in {\mathbb{N}}^{\ast }$, $(x^n{)}^{\prime }=n{x}^{n-1}$ for all $x\in \mathbb{R}$.

   • Base Case (n=1): $(x^1{)}^{\prime }=(x{)}^{\prime }=1=1\cdot x^{1-1}=1\cdot x^0=1$. (From Example 4.1.6)
   • Inductive Step: Assume $(x^{n-1}{)}^{\prime }=(n-1)x^{n-2}$ holds for some $n\ge 2$. Consider $(x^n{)}^{\prime }=(x\cdot x^{n-1}{)}^{\prime }$. Using the product rule: $(x\cdot x^{n-1}{)}^{\prime }=(x{)}^{\prime }\cdot x^{n-1}+x\cdot (x^{n-1}{)}^{\prime }$ $=1\cdot x^{n-1}+x\cdot ((n-1)x^{n-2})$ (by inductive hypothesis) $=x^{n-1}+(n-1)x^{n-1}=(1+n-1)x^{n-1}=n{x}^{n-1}$. The formula holds.

2. Tangent Function: $\tan (x)=\frac{\sin (x)}{\cos (x)}$ is differentiable in its domain $\mathbb{R}\setminus \{\frac{\pi }{2}+k\pi \mid k\in \mathbb{Z}\}$. Using the quotient rule: $(\tan (x){)}^{\prime }={\left(\frac{\sin (x)}{\cos (x)}\right)}^{\prime }=\frac{(\sin (x){)}^{\prime }\cos (x)-\sin (x)(\cos (x){)}^{\prime }}{(\cos (x){)}^2}$ $=\frac{\cos (x)\cos (x)-\sin (x)(-\sin (x))}{(\cos (x){)}^2}=\frac{{\cos }^2(x)+{\sin }^2(x)}{(\cos (x){)}^2}=\frac{1}{(\cos (x){)}^2}$. So, ${\tan }^{\prime }(x)=\frac{1}{{\cos }^2(x)}$. This holds for $x\ne \frac{\pi }{2}+k\pi$.

3. Cotangent Function: Analogous to (2). $\cot (x)=\frac{\cos (x)}{\sin (x)}$. ${\cot }^{\prime }(x)=-\frac{1}{{\sin }^2(x)}$ for $x\ne k\pi$.

Continue here: 17 Chain Rule, Derivatives of Inverse Functions, Central Theorems (Local Extrema, Rolle’s Theorem, Mean Value Theorem)