Consider a two-class problem in \(\mathbb{R}^{d}\) with class labels red and green. Let \(\mu_{\textit{red}}\) and \(\mu_{\textit{green}}\) be the means of the two classes. Given test sample \(x \in \mathbb{R}^{d}\), a classifier calculates the squared Euclidean distance (denoted by \(\| \cdot \|^{2}\)) between \(x\) and the means of the two classes and assigns the class label that the sample \(x\) is closest to. That is, the classifier computes
\[ f(x) = \|\mu_{\textit{red}} - x\|^{2} - \|\mu_{\textit{green}} - x\|^{2} \]
and assigns the label red to \(x\) if \(f(x) < 0\), and green otherwise. Which of the following statements is/are correct?
Answer
(B), (C)
Explanation
Expand each squared distance as \(\|\mu - x\|^{2} = \|\mu\|^{2} - 2\mu^{\top}x + \|x\|^{2}\). The \(\|x\|^{2}\) terms cancel in the difference:
\[ f(x) = 2(\mu_{\textit{green}} - \mu_{\textit{red}})^{\top}x + \|\mu_{\textit{red}}\|^{2} - \|\mu_{\textit{green}}\|^{2} \]
- B and C hold: \(f(x) = w^{\top}x + b\) with \(w = 2(\mu_{\textit{green}} - \mu_{\textit{red}})\) and \(b = \|\mu_{\textit{red}}\|^{2} - \|\mu_{\textit{green}}\|^{2}\), so the decision boundary is a hyperplane.
- D fails: the quadratic term cancels.
- A fails: \(f(0) = \|\mu_{\textit{red}}\|^{2} - \|\mu_{\textit{green}}\|^{2} < 0\) when \(\|\mu_{\textit{red}}\| < \|\mu_{\textit{green}}\|\), so the origin gets the label red, the nearer mean.
Answer: B, C