14 Sep 100 Deep Learning MCQ (Multiple Choice Questions) with Answers
1) What does CNN stand for?
Show Answer
Answer: a
Explanation
CNN stands for Convolutional Neural Network — an architecture that applies learnable convolutional filters to extract spatial features, most commonly from images.
2) Which activation function outputs values between 0 and 1?
Show Answer
Answer: c
Explanation
The sigmoid function squashes any real-valued input into the range (0, 1), which makes it useful for producing binary probabilities.
3) Which of the following is a common loss function for binary classification?
Show Answer
Answer: b
Explanation
Binary Cross-Entropy compares predicted probabilities against 0/1 labels and is the standard loss for two-class problems. Categorical cross-entropy is used when there are more than two classes.
4) What is the purpose of the backpropagation algorithm?
Show Answer
Answer: b
Explanation
Backpropagation propagates the loss backwards through the network using the chain rule to compute gradients of the loss with respect to each weight; the optimizer then uses those gradients to update the weights.
5) Which optimizer uses both momentum and adaptive learning rates?
Show Answer
Answer: c
Explanation
Adam (Adaptive Moment Estimation) combines a momentum term (first moment) with per-parameter adaptive learning rates derived from the second moment of the gradients.
6) What does ReLU stand for?
Show Answer
Answer: a
Explanation
ReLU stands for Rectified Linear Unit, defined as f(x) = max(0, x). It is simple, fast to compute and helps alleviate vanishing gradients.
7) Which layer is used to reduce spatial dimensions in CNNs?
Show Answer
Answer: b
Explanation
A pooling layer (max or average pooling) downsamples feature maps, reducing their spatial dimensions, computation and parameter count.
8) What is the vanishing gradient problem?
Show Answer
Answer: b
Explanation
In deep networks with saturating activations, gradients shrink exponentially as they propagate backwards, so early layers receive extremely small updates and learn very slowly — this is the vanishing gradient problem.
9) Which of the following is a regularization technique?
Show Answer
Answer: d
Explanation
Dropout, batch normalization and data augmentation all reduce overfitting in different ways, so all of the above is correct.
10) What is the role of the forget gate in LSTM?
Show Answer
Answer: a
Explanation
The forget gate outputs values between 0 and 1 that determine which parts of the previous cell state should be discarded and which should be retained.
11) Which of the following is NOT a type of RNN?
Show Answer
Answer: c
Explanation
CNN is a feed-forward convolutional architecture. LSTM, GRU and bidirectional RNNs are all recurrent architectures that maintain a hidden state over sequences.
12) What is the purpose of attention mechanism?
Show Answer
Answer: a
Explanation
Attention computes relevance weights over the input so the model can focus on the most relevant parts when producing each output, instead of compressing everything into a fixed vector.
13) In Transformer, what is the purpose of positional encoding?
Show Answer
Answer: a
Explanation
Transformers process all tokens in parallel and have no built-in notion of order, so positional encodings inject information about each token’s position in the sequence.
14) Which of the following is a pre-trained language model?
Show Answer
Answer: d
Explanation
BERT, GPT and RoBERTa are all large language models pre-trained on massive text corpora and then fine-tuned for downstream tasks.
15) What is the difference between autoencoder and variational autoencoder?
Show Answer
Answer: a
Explanation
A VAE is generative: its encoder maps inputs to a probability distribution over a latent space, so new samples can be drawn and decoded. A standard autoencoder only reconstructs its input.
16) In GAN, what is the role of the discriminator?
Show Answer
Answer: b
Explanation
The discriminator is a binary classifier trained to distinguish real samples from the generator’s fake samples, while the generator tries to fool it.
17) Which loss function is used in GAN?
Show Answer
Answer: c
Explanation
GANs are trained with an adversarial (minimax) loss, where the generator and discriminator compete against each other.
18) What is transfer learning?
Show Answer
Answer: b
Explanation
Transfer learning reuses the knowledge captured by a model pre-trained on a large dataset and applies it to a new, usually smaller, task.
19) What is fine-tuning?
Show Answer
Answer: c
Explanation
Fine-tuning means continuing training of a pre-trained model on new task data, typically updating some or all of its layers with a small learning rate.
20) What is the purpose of batch normalization?
Show Answer
Answer: d
Explanation
Batch normalization normalizes the inputs to each layer, reduces internal covariate shift and allows higher, more stable learning rates — hence all of the above.
21) Which of the following is a hyperparameter?
Show Answer
Answer: c
Explanation
Hyperparameters are set before training begins. The learning rate is one of them; weights and biases are learned parameters, not hyperparameters.
22) What is an epoch?
Show Answer
Answer: c
Explanation
An epoch is one complete pass through the entire training dataset. Training usually runs for many epochs, and each epoch contains several mini-batch updates.
23) What is batch size?
Show Answer
Answer: b
Explanation
Batch size is the number of samples processed before one weight update (one forward/backward pass of the mini-batch).
24) What is the purpose of dropout?
Show Answer
Answer: a
Explanation
Dropout randomly deactivates neurons during training, forcing the network to learn redundant representations. This acts as a regularizer and prevents overfitting.
25) Which of the following is a common weight initialization method?
Show Answer
Answer: d
Explanation
Xavier/Glorot, He and LeCun are all standard initialization schemes that scale initial weights according to layer size and activation type.
26) What is the dying ReLU problem?
Show Answer
Answer: a
Explanation
If a ReLU neuron only ever receives negative inputs, it outputs 0 and its gradient stays 0, so it stops updating entirely — the neuron becomes inactive (“dies”).
27) Which activation function is smooth and zero-centered?
Show Answer
Answer: b
Explanation
Tanh is smooth (continuously differentiable) and its outputs range from -1 to 1, making it zero-centered — unlike sigmoid, whose outputs are all positive.
28) What is the output of softmax?
Show Answer
Answer: a
Explanation
Softmax converts a vector of raw scores (logits) into a probability distribution whose values are all between 0 and 1 and sum to 1.
29) Which of the following is a convolutional neural network architecture?
Show Answer
Answer: a
Explanation
ResNet is a convolutional architecture built from residual blocks. LSTM and GRU are recurrent, and the Transformer is attention-based.
30) What is a skip connection?
Show Answer
Answer: a
Explanation
A skip (residual) connection adds the input of a block directly to its output, bypassing one or more layers and giving gradients a shorter path back through the network.
31) What is the purpose of pooling?
Show Answer
Answer: b
Explanation
Pooling reduces the spatial dimensions of feature maps, lowering computation and memory while providing a degree of translation invariance. It has no learnable parameters.
32) What is stride in convolution?
Show Answer
Answer: a
Explanation
Stride is the step size with which the convolution filter slides across the input. A larger stride produces a smaller output feature map.
33) What is padding?
Show Answer
Answer: a
Explanation
Padding means adding zeros (or other values) around the border of the input so that edge pixels are seen by the filter as often as interior pixels and the output size can be controlled.
34) What is the receptive field?
Show Answer
Answer: a
Explanation
The receptive field is the region of the input that influences a particular neuron’s output. It grows as layers are stacked, letting deeper neurons see more of the image.
35) Which of the following is a 1×1 convolution used for?
Show Answer
Answer: c
Explanation
A 1×1 convolution mixes information across channels, so it can reduce or increase the number of channels depending on how many filters are used.
36) What is depthwise separable convolution?
Show Answer
Answer: a
Explanation
Depthwise separable convolution splits a standard convolution into two steps: a depthwise convolution applied per channel (spatial) and a pointwise 1×1 convolution that mixes channels. It is far cheaper in computation.
37) Which architecture introduced residual connections?
Show Answer
Answer: b
Explanation
ResNet introduced residual (skip) connections, which made it possible to train networks with hundreds of layers without severe degradation.
38) What is the main idea of Inception network?
Show Answer
Answer: a
Explanation
The Inception module applies several filter sizes (1×1, 3×3, 5×5) and pooling in parallel on the same input and concatenates the results, letting the network choose the best scale.
39) What is MobileNet known for?
Show Answer
Answer: a
Explanation
MobileNet is a lightweight CNN designed for mobile and embedded devices, built mainly from depthwise separable convolutions to minimise latency and model size.
40) What is EfficientNet known for?
Show Answer
Answer: a
Explanation
EfficientNet introduced compound scaling, which balances network width, depth and input resolution together with a fixed set of scaling coefficients.
41) What is U-Net used for?
Show Answer
Answer: a
Explanation
U-Net’s symmetric encoder-decoder structure with skip connections was designed for image segmentation, especially in biomedical imaging.
42) What is YOLO used for?
Show Answer
Answer: a
Explanation
YOLO (“You Only Look Once”) performs real-time object detection, predicting bounding boxes and class probabilities in a single forward pass.
43) What is Mask R-CNN used for?
Show Answer
Answer: a
Explanation
Mask R-CNN extends Faster R-CNN with a mask branch, producing a pixel-level mask for each detected object — that is, instance segmentation.
44) What is the difference between semantic and instance segmentation?
Show Answer
Answer: a
Explanation
Semantic segmentation labels every pixel with a class (all “person” pixels look the same), while instance segmentation additionally separates individual object instances.
45) Which of the following is a generative model?
Show Answer
Answer: d
Explanation
VAEs, GANs and diffusion models all learn the data distribution and can generate new samples, so all of the above are generative models.
46) What is the key idea of diffusion models?
Show Answer
Answer: a
Explanation
Diffusion models gradually add noise to data in a forward process and learn to reverse (denoise) it, so new samples can be generated by starting from pure noise.
47) What is CLIP?
Show Answer
Answer: a
Explanation
CLIP stands for Contrastive Language-Image Pretraining. It learns a shared embedding space for images and text using a contrastive objective.
48) What is Vision Transformer (ViT)?
Show Answer
Answer: a
Explanation
ViT applies the Transformer architecture to images by splitting them into patches and treating each patch as a token.
49) What is the main component of Transformer?
Show Answer
Answer: a
Explanation
Self-attention is the core building block of the Transformer — it lets every token attend to every other token and weigh their relevance.
50) What is multi-head attention?
Show Answer
Answer: a
Explanation
Multi-head attention runs several attention operations in parallel, each with its own learned projections, and concatenates their outputs so the model can attend to different subspaces at once.
51) What is the purpose of layer normalization?
Show Answer
Answer: a
Explanation
Layer normalization normalizes across the feature dimension within each individual sample, which makes it independent of batch size and well suited to sequence models.
52) What is the difference between batch norm and layer norm?
Show Answer
Answer: a
Explanation
Batch norm computes statistics across the batch dimension (for each feature), while layer norm computes statistics across the features of each sample independently.
53) What is the purpose of gradient clipping?
Show Answer
Answer: a
Explanation
Gradient clipping caps the magnitude of gradients, which prevents exploding gradients from destabilising training (common in RNNs and deep networks).
54) What is learning rate warmup?
Show Answer
Answer: a
Explanation
Warmup starts with a very small learning rate and gradually increases it over the first few iterations, which stabilises early training of large models.
55) What is cosine annealing?
Show Answer
Answer: a
Explanation
Cosine annealing is a learning rate schedule that reduces the learning rate following a cosine curve, often with periodic restarts.
56) Which optimizer is known for adaptive learning rates?
Show Answer
Answer: b
Explanation
Adam maintains a separate adaptive learning rate for every parameter based on estimates of the first and second moments of the gradients.
57) What is AdamW?
Show Answer
Answer: a
Explanation
Explanation
AdamW decouples weight decay from the adaptive gradient update, applying it directly to the weights — this is Adam with correct weight decay and generally generalises better.
58) What is the purpose of weight decay?
Show Answer
Answer: a
Explanation
Weight decay penalises large weights, shrinking them towards zero. It is a form of regularization that reduces overfitting.
59) What is L1 regularization?
Show Answer
Answer: a
Explanation
L1 regularization adds the sum of absolute weights to the loss. It encourages sparsity, driving many weights exactly to zero.
60) What is L2 regularization?
Show Answer
Answer: a
Explanation
L2 regularization (ridge) adds the sum of squared weights to the loss, shrinking weights smoothly towards zero without making them exactly zero.
61) What is early stopping?
Show Answer
Answer: a
Explanation
Early stopping halts training as soon as the validation error starts to increase (or stops improving), which prevents overfitting.
62) What is data augmentation?
Show Answer
Answer: a
Explanation
Data augmentation increases the effective training set by applying label-preserving transformations such as flips, crops, rotations and colour jitter.
63) What is transfer learning?
Show Answer
Answer: a
Explanation
Transfer learning starts from a pre-trained model whose features were learned on a large dataset, then adapts it to a new task.
64) What is fine-tuning?
Show Answer
Answer: a
Explanation
Fine-tuning continues training and updates the weights of the pre-trained model (often all of them, or a large portion) on the new task’s data with a small learning rate.
65) What is feature extraction in transfer learning?
Show Answer
Answer: a
Explanation
In feature extraction, the pre-trained layers are frozen and only a new classifier head is trained on top of the extracted features.
66) What is multi-task learning?
Show Answer
Answer: a
Explanation
Multi-task learning trains a single model on several tasks at the same time, sharing representations so the tasks help each other.
67) What is continual learning?
Show Answer
Answer: a
Explanation
Continual (lifelong) learning aims to learn a sequence of tasks without forgetting earlier ones — overcoming catastrophic forgetting.
68) What is meta-learning?
Show Answer
Answer: a
Explanation
Meta-learning is often described as “learning to learn” — training models or algorithms so they can adapt rapidly to new tasks from very little data.
69) What is few-shot learning?
Show Answer
Answer: a
Explanation
Few-shot learning means learning a new class or task from only a few labelled examples (typically 1–5 per class).
70) What is zero-shot learning?
Show Answer
Answer: a
Explanation
Zero-shot learning recognises classes that were never seen during training, relying on auxiliary information such as attribute descriptions or text embeddings.
71) What is self-supervised learning?
Show Answer
Answer: a
Explanation
Self-supervised learning generates supervision signals from the data itself — for example predicting masked words or the next token — avoiding manual labelling.
72) What is contrastive learning?
Show Answer
Answer: a
Explanation
Contrastive learning learns representations by pulling positive (similar) pairs together and pushing negative (dissimilar) pairs apart in embedding space.
73) What is SimCLR?
Show Answer
Answer: a
Explanation
SimCLR is a contrastive learning framework that learns visual representations by maximising agreement between differently augmented views of the same image.
74) What is MoCo?
Show Answer
Answer: a
Explanation
MoCo stands for Momentum Contrast. It maintains a queue of negative samples and a momentum-updated encoder to provide consistent contrastive targets.
75) What is BYOL?
Show Answer
Answer: a
Explanation
BYOL stands for Bootstrap Your Own Latent. It learns representations using two networks (online and target) without requiring negative pairs.
76) What is the purpose of a teacher-student model?
Show Answer
Answer: a
Explanation
In a teacher-student setup, the large teacher model guides the smaller student model — the basis of knowledge distillation.
77) What is knowledge distillation?
Show Answer
Answer: a
Explanation
Knowledge distillation transfers knowledge from a large (teacher) model to a small (student) model, usually by matching the teacher’s soft output probabilities.
78) What is pruning?
Show Answer
Answer: a
Explanation
Pruning removes unnecessary weights, neurons or filters from a trained model to reduce its size and computation with minimal accuracy loss.
79) What is quantization?
Show Answer
Answer: a
Explanation
Quantization reduces the numerical precision of weights and activations (e.g. from FP32 to INT8), shrinking model size and speeding up inference.
80) What is federated learning?
Show Answer
Answer: a
Explanation
Federated learning trains a shared model across many decentralized devices, exchanging only model updates rather than raw data — preserving privacy.
81) What is an adversarial example?
Show Answer
Answer: a
Explanation
An adversarial example is an input with a small, often imperceptible perturbation crafted specifically to fool a model into making a wrong prediction.
82) What is FGSM?
Show Answer
Answer: a
Explanation
FGSM stands for Fast Gradient Sign Method — a one-step adversarial attack that perturbs the input in the direction of the sign of the loss gradient.
83) What is Grad-CAM?
Show Answer
Answer: a
Explanation
Grad-CAM stands for Gradient-weighted Class Activation Mapping, a visual explanation technique that highlights the image regions most important for a prediction.
84) What is SHAP?
Show Answer
Answer: a
Explanation
SHAP stands for SHapley Additive exPlanations, a game-theoretic method that assigns each feature an additive contribution to a model’s prediction.
85) What is LIME?
Show Answer
Answer: a
Explanation
LIME stands for Local Interpretable Model-agnostic Explanations. It fits a simple interpretable model locally around a prediction to explain it.
86) What is bias-variance tradeoff?
Show Answer
Answer: a
Explanation
The bias-variance tradeoff is the balance between underfitting (high bias) and overfitting (high variance) — minimising total expected error requires managing both.
87) What is underfitting?
Show Answer
Answer: a
Explanation
Underfitting occurs when the model is too simple to capture the underlying patterns, performing poorly on both training and validation data.
88) What is overfitting?
Show Answer
Answer: a
Explanation
Overfitting happens when the model is too complex and memorises the training data (including noise), so it performs well on training but poorly on unseen data.
89) What is regularization?
Show Answer
Answer: a
Explanation
Regularization discourages excessive model complexity, thereby helping to prevent overfitting (e.g. L1/L2, dropout, early stopping).
90) What is dropout rate?
Show Answer
Answer: a
Explanation
The dropout rate is the probability that a given neuron is dropped (set to zero) during a training step. A rate of 0.5 means each neuron is dropped half the time.
91) What is batch size effect?
Show Answer
Answer: a
Explanation
Smaller batches estimate the gradient from fewer samples, so the gradient estimate is noisier; larger batches give smoother but more expensive estimates.
92) What is learning rate schedule?
Show Answer
Answer: a
Explanation
A learning rate schedule adjusts the learning rate during training — for example step decay, cosine annealing or warmup — to improve convergence.
93) What is warmup?
Show Answer
Answer: a
Explanation
Warmup gradually increases the learning rate from a small value at the beginning of training, avoiding unstable updates in the first iterations.
94) What is gradient accumulation?
Show Answer
Answer: a
Explanation
Gradient accumulation sums gradients over several mini-batches before performing one weight update, simulating a larger effective batch size when memory is limited.
95) What is mixed precision training?
Show Answer
Answer: a
Explanation
Mixed precision training uses FP16 for most computations (fast and memory-efficient) while keeping FP32 master weights and where numerical stability is required.
96) What is model parallelism?
Show Answer
Answer: a
Explanation
Model parallelism splits the model itself across multiple devices, so different layers or parts of the network reside on different hardware.
97) What is data parallelism?
Show Answer
Answer: a
Explanation
Data parallelism splits the training data across devices, each of which holds a full copy of the model and processes its own mini-batch.
98) What is All-reduce?
Show Answer
Answer: a
Explanation
All-reduce is a collective communication operation that aggregates (sums/averages) gradients across all devices and distributes the result back to each of them.
99) What is PyTorch DDP?
Show Answer
Answer: a
Explanation
PyTorch DDP stands for Distributed Data Parallel, PyTorch’s module for multi-GPU/multi-node data-parallel training with gradient all-reduce.
100) What is TensorFlow MirroredStrategy?
Show Answer
Answer: a
Explanation
MirroredStrategy is TensorFlow’s data parallelism strategy: it replicates the model on each device (GPU) and keeps their variables synchronised via all-reduce.
No Comments