GATE DA 2025 | Question 42

Consider the neural network shown in the figure with
inputs: \(u, v\)
weights: \(a, b, c, d, e, f\)
output: \(y\)
\(R\) denotes the ReLU function, \(R(x) = \max(0, x)\).

Network diagram: input u connects to the upper ReLU node R with weight a and to the lower ReLU node R with weight c, and input v connects to the upper node with weight b and to the lower node with weight d. The upper and lower nodes feed an output ReLU node R with weights e and f, and that node outputs y.

Given \(u = 2, v = 3\),
\(a = 1, b = 1, c = 1, d = -1, e = 4, f = -1\),
which one of the following is correct?

Option A: \(\frac{\partial y}{\partial a} = 8, \frac{\partial y}{\partial f} = 0\) (correct)
Option B: \(\frac{\partial y}{\partial a} = 1, \frac{\partial y}{\partial f} = 0\)
Option C: \(\frac{\partial y}{\partial a} = 1, \frac{\partial y}{\partial f} = -1\)
Option D: \(\frac{\partial y}{\partial a} = 2, \frac{\partial y}{\partial f} = -1\)

Answer

(A)

Explanation

Forward pass, with \(h_1\) and \(h_2\) the outputs of the upper and lower hidden nodes:

\(h_1 = R(au + bv) = R(2 + 3) = 5\)
\(h_2 = R(cu + dv) = R(2 - 3) = R(-1) = 0\)
\(y = R(eh_1 + fh_2) = R(20 + 0) = 20\)

The output node and the upper node have positive inputs, so their ReLU slopes are 1. By the chain rule,

\[\frac{\partial y}{\partial a} = 1 \cdot e \cdot 1 \cdot u = 4 \times 2 = 8, \qquad \frac{\partial y}{\partial f} = 1 \cdot h_2 = 0.\]

The gradient for \(f\) is 0 because \(f\) only multiplies \(h_2\), and the lower ReLU outputs 0 here.

Answer: A