Neural Networks

GATE DA previous year questions · 2 questions

GATE DA 2025 | Question 42

MCQ · 2 marks · −0.66

Consider the neural network shown in the figure with
inputs: \(u, v\)
weights: \(a, b, c, d, e, f\)
output: \(y\)
\(R\) denotes the ReLU function, \(R(x) = \max(0, x)\).

Network diagram: input u connects to the upper ReLU node R with weight a and to the lower ReLU node R with weight c, and input v connects to the upper node with weight b and to the lower node with weight d. The upper and lower nodes feed an output ReLU node R with weights e and f, and that node outputs y.

Given \(u = 2, v = 3\),
\(a = 1, b = 1, c = 1, d = -1, e = 4, f = -1\),
which one of the following is correct?

Options for GATE DA 2025 | Question 42

Explanation

Answer: (A)

Forward pass, with \(h_1\) and \(h_2\) the outputs of the upper and lower hidden nodes:

  • \(h_1 = R(au + bv) = R(2 + 3) = 5\)
  • \(h_2 = R(cu + dv) = R(2 - 3) = R(-1) = 0\)
  • \(y = R(eh_1 + fh_2) = R(20 + 0) = 20\)

The output node and the upper node have positive inputs, so their ReLU slopes are 1. By the chain rule,

\[\frac{\partial y}{\partial a} = 1 \cdot e \cdot 1 \cdot u = 4 \times 2 = 8, \qquad \frac{\partial y}{\partial f} = 1 \cdot h_2 = 0.\]

The gradient for \(f\) is 0 because \(f\) only multiplies \(h_2\), and the lower ReLU outputs 0 here.

Answer: A

GATE DA 2024 | Question 43

MCQ · 2 marks · −0.66

Consider the two neural networks (NNs) shown in Figures 1 and 2, with \(ReLU\) activation (\(ReLU(z) = \max\{0, z\}, \forall z \in \mathbb{R}\)). \(\mathbb{R}\) denotes the set of real numbers. The connections and their corresponding weights are shown in the Figures. The biases at every neuron are set to 0. For what values of \(p, q, r\) in Figure 2 are the two NNs equivalent, when \(x_{1}, x_{2}, x_{3}\) are positive?

Figure 1

Network with inputs x1, x2 and x3, a first layer of three neurons, a second layer of two neurons and one output neuron. All input edges have weight 1 (x1 connects to all three first-layer neurons, x2 to the top and bottom ones, x3 to the middle and bottom ones); every first-layer neuron connects to both second-layer neurons with weight 2, and both second-layer neurons connect to the output with weight 3.

Figure 2

Network in which inputs x1, x2 and x3 connect directly to a single output neuron with weights p, q and r respectively.
Options for GATE DA 2024 | Question 43

Explanation

Answer: (A)

The inputs and all weights are positive, so every neuron receives a positive value and \(ReLU\) acts as the identity. Both networks are then linear, and only the total weight from each input to the output matters.

  • First layer (weights 1): the neurons receive \(x_{1} + x_{2}\), \(x_{1} + x_{3}\) and \(x_{1} + x_{2} + x_{3}\), which add up to \(s = 3x_{1} + 2x_{2} + 2x_{3}\).
  • Second layer: each of its two neurons receives \(2s\).
  • Output: \(3(2s) + 3(2s) = 12s\).

So Figure 1 computes \(36x_{1} + 24x_{2} + 24x_{3}\), giving \(p = 36\), \(q = 24\), \(r = 24\). Option D would need \(x_{2}\) and \(x_{3}\) to reach all three first-layer neurons, but each has only two connections.

Answer: A