Differentiability

Options A, B and C use only the values and derivatives of \(f\) and \(g\) at \(c\):

• sums, differences and products of functions differentiable at \(c\) are differentiable at \(c\);
• the quotient rule for \(\frac{f}{g}\) also needs \(g(c) \neq 0\), which holds because \(g\) takes values in \((1, \infty)\).

Option D is different: \(f \circ g\) needs \(f\) to be differentiable at \(g(c)\), and \(g \circ f\) needs \(g\) to be differentiable at \(f(c)\). Neither is given. For example, take \(c = 0\), \(f(x) = |x - 2|\) and \(g(x) = 1 + e^{x}\) (both differentiable at 0, and \(g > 1\)). Then \(f(g(x)) = |e^{x} - 1|\) has a corner at 0 while \(g(f(x))\) is smooth there, so the sum is not differentiable at 0.

Answer: A, B, C

ReLU is 0 for \(x \lt 0\) and equals \(x\) for \(x \ge 0\).

• A (correct): both pieces are continuous and meet at the value 0 when \(x = 0\), so there is no jump.
• B (wrong), C (correct): at \(x = 0\) the left derivative is 0 and the right derivative is 1. They differ, so the graph has a corner there and ReLU is not differentiable at 0. (Training code simply uses 0 or 1 as the gradient at that point.)
• D (wrong): with \(x = 1\) and \(a = 2\), \(\mathrm{ReLU}(1) = 1\) but \(\mathrm{ReLU}(2) = 2\). What does hold is \(\mathrm{ReLU}(ax) = a\,\mathrm{ReLU}(x)\) for \(a \ge 0\).

Answer: A, C

For \(x \neq y\), divide the condition by \(|x - y|\):

\[ \left| \frac{f(x) - f(y)}{x - y} \right| \le |x - y| \]

As \(y \to x\) the right side tends to 0, so \(f\) is differentiable with \(f'(x) = 0\) for every \(x\). Hence \(f\) is constant and \(f(1) - f(0) = 0\).

Without derivatives: split \([0, 1]\) into \(n\) equal steps. Each step changes \(f\) by at most \(\frac{1}{n^{2}}\), so \(|f(1) - f(0)| \le n \cdot \frac{1}{n^{2}} = \frac{1}{n}\) for every \(n\), which forces the difference to be 0.

Answer: 0

This \(f\) is the logistic (sigmoid) function. Differentiating \(f(x) = (1+e^{-x})^{-1}\) gives

\[f'(x) = \frac{e^{-x}}{(1+e^{-x})^{2}} = \frac{1}{1+e^{-x}} \cdot \frac{e^{-x}}{1+e^{-x}} = f(x)\bigl(1 - f(x)\bigr).\]

So there is no need to find \(x\) itself: at the point where \(f(x) = 0.4\), the slope is \(0.4 \times (1 - 0.4) = 0.4 \times 0.6 = 0.24\). The same identity is why the sigmoid's gradient is so cheap to compute in logistic regression and neural networks.

Answer: 0.24

Let \(q(x) = ax^{2} + bx + c\). Both outer pieces equal 2 at the joins, with slope \(-1\) on the left and \(+1\) on the right.

• Continuity: \(q(-2) = 4a - 2b + c = 2\) and \(q(2) = 4a + 2b + c = 2\).
• Differentiability: \(q'(-2) = -4a + b = -1\) and \(q'(2) = 4a + b = 1\).

Adding the slope equations gives \(b = 0\), so \(a = \frac{1}{4}\), and then \(1 + c = 2\) gives \(c = 1\). The middle piece \(\frac{x^{2}}{4} + 1\) meets both lines smoothly.

Option B, \(\frac{x^{2}}{2}\), reaches the value 2 at both joins but with slopes \(-2\) and \(2\), so it is continuous but not differentiable. Options C and D are not even continuous at \(x = -2\).

Answer: A