Consider designing a linear classifier
\[y = \operatorname{sign}(f(x; w, b)), \quad f(x; w, b) = w^{\top}x + b\]
on a dataset \(D = \{(x_{1}, y_{1}), (x_{2}, y_{2}), \ldots, (x_{N}, y_{N})\}\), \(x_{i} \in \mathbb{R}^{d}, y_{i} \in \{+1, -1\}, i = 1, 2, \ldots, N\). Recall that the sign function outputs +1 if the argument is positive, and \(-1\) if the argument is non-positive. The parameters \(w\) and \(b\) are updated as per the following training algorithm:
\[w_{new} = w_{old} + y_{n}x_{n}, \quad b_{new} = b_{old} + y_{n}\]
whenever \(\operatorname{sign}(f(x_{n}; w_{old}, b_{old})) \neq y_{n}\). In other words, whenever the classifier wrongly predicts a sample \((x_{n}, y_{n})\) from the dataset, \(w_{old}\) gets updated to \(w_{new}\), and likewise \(b_{old}\) gets updated to \(b_{new}\). Consider the case \((x_{n}, +1)\), \(f(x_{n}; w_{old}, b_{old}) < 0\). Then
Answer
(A)
Explanation
Here \(y_{n} = +1\), so the update adds the sample: \(w_{new} = w_{old} + x_{n}\) and \(b_{new} = b_{old} + 1\). Evaluate the new score on the same point:
\[f(x_{n}; w_{new}, b_{new}) = (w_{old} + x_{n})^{\top}x_{n} + b_{old} + 1 = f(x_{n}; w_{old}, b_{old}) + \lVert x_{n} \rVert^{2} + 1.\]
Since \(\lVert x_{n} \rVert^{2} + 1 \geq 1\), the score strictly increases, moving towards the correct positive side, so B and C are false. Option D fails because \(y_{n}f(x_{n}; w_{old}, b_{old}) = f(x_{n}; w_{old}, b_{old}) < 0\).
Answer: A