Mean squared error is the expected (squared) distance between the models predictions and the true values. It therefore is well suited to regression tasks where the label space is a continuous one. MSE assumes the data is normally distributed however, in binary classification tasks, the data is distributed according to a Bernoulli distribution.

Multiclass classification - classification task where each instance needs to be assigned to one of two or more classes.

Mulitlabel classification - assign each instance a set of target labels. Think of this as predicting a series of properties of an instance which are not mutually exclusive.

Every neuron in a fully connected neural layer is connected to every input via a weight. This creates a vector of dimensions equal to the input layer dimensions for every neuron in the current layer. Resulting in a stack of these vectors as deep as the number of neurons in the current layer. For Z=XW the neighbouring dimensions need to match.

The key with this computational graph is to apply the chain rule at each node. Please see the attached solution for more information.

i) is demonstrating high bias as 1.) the validation and training errors are high and at a similar level 2.) as we add more training data the training error increases. This happens as the capacity of the model limits its ability to fit all the modalities present in the larger dataset.

ii) is demonstrating high variance as the validation error is much higher than the training error. The model has overfit to spurious patterns in the training set which are not representative of the true distribution (represented by the validation set)