Supervised Learning Algorithms Questions and Answers for Viva

Answer-1: Supervised learning is a type of Machine Learning where the model is trained using labeled data, meaning each input is paired with the correct output. The model learns to map inputs to outputs based on this labeled data.

Answer-2: A classification problem in supervised learning involves predicting a discrete label or category. For example, classifying emails as "spam" or "not spam."

Answer-3: A regression problem involves predicting continuous values, such as predicting house prices based on various features (square footage, location, etc.).

Answer-4: Some commonly used supervised learning algorithms are Linear Regression, Logistic Regression, Support Vector Machines (SVM), Decision Trees, Random Forests, K-Nearest Neighbors (KNN), and Naive Bayes.

Answer-5: Linear regression is used to predict a continuous output variable based on one or more input features. It assumes a linear relationship between the input features and the target variable.

Answer-6: The cost function in Linear Regression is the Mean Squared Error (MSE), which calculates the average squared difference between the predicted values and the actual values.

Answer-7: Logistic regression is used for classification problems, especially binary classification, such as predicting whether an email is spam or not. It uses the logistic function (sigmoid) to model probabilities.

Answer-8: Linear regression is used for predicting continuous values, while logistic regression is used for binary classification problems where the output is a probability that can be mapped to a class label.

Answer-9: A decision tree is a supervised learning algorithm that splits the data into subsets based on feature values. It creates a tree-like structure where each node represents a decision based on a feature.

Answer-10: Random Forest is an ensemble learning method that combines multiple decision trees to make more accurate predictions by averaging the predictions of many trees to reduce overfitting.

Answer-11: Overfitting occurs when a model learns the details and noise in the training data to the extent that it negatively impacts the performance of the model on new data, causing poor generalization.

Answer-12: Overfitting can be prevented using techniques such as cross-validation, pruning decision trees, using simpler models, and applying regularization methods like L1 (Lasso) or L2 (Ridge) regularization.

Answer-13: SVM is a supervised learning algorithm that works by finding the hyperplane that best separates different classes in a feature space. It is widely used for both classification and regression tasks.

Answer-14: The kernel trick is a technique used in SVM to transform the data into a higher-dimensional space where it is easier to separate classes, without explicitly calculating the transformation.

Answer-15: In SVM, a hyperplane is a decision boundary that separates different classes in the feature space. The goal is to find the hyperplane that maximizes the margin between the classes.

Answer-16: K-Nearest Neighbors is a simple, non-parametric supervised learning algorithm used for classification and regression. It assigns a label to a data point based on the majority label of its 'K' nearest neighbors in the feature space.

Answer-17: The value of K in KNN is typically chosen through cross-validation. A small value of K makes the model sensitive to noise, while a larger K can smooth the decision boundary and reduce overfitting.

Answer-18: Cross-validation is used to assess the model's generalization ability by dividing the data into multiple subsets, training the model on some subsets, and testing it on the remaining ones. This helps to prevent overfitting and gives a better estimate of model performance.

Answer-19: Naive Bayes is a probabilistic classifier based on Bayes' Theorem. It is commonly used for text classification tasks such as spam detection and sentiment analysis.

Answer-20: Naive Bayes assumes that the features are conditionally independent given the class label, which simplifies the computation of the posterior probability.

Answer-21: A confusion matrix is a table that is used to evaluate the performance of a classification model. It shows the true positives, false positives, true negatives, and false negatives, which are used to calculate various performance metrics.

Answer-22: Precision is the ratio of correctly predicted positive observations to the total predicted positives, while recall is the ratio of correctly predicted positive observations to all actual positives.

Answer-23: The F1-score is the harmonic mean of precision and recall. It is a balanced metric for classification problems, especially when the data is imbalanced.

Answer-24: Regularization techniques like L1 and L2 regularization add a penalty to the loss function to reduce the complexity of the model and prevent overfitting.

Answer-25: Ridge regression is a type of linear regression that uses L2 regularization to penalize large coefficients, helping to prevent overfitting by shrinking the coefficients of less important features.

Answer-26: Lasso regression is a type of linear regression that uses L1 regularization. It can shrink some coefficients to zero, effectively performing feature selection by removing less important features.

Answer-27: Lasso uses L1 regularization, which can set some coefficients to zero, effectively performing feature selection. Ridge uses L2 regularization, which shrinks coefficients but doesn't eliminate them completely.

Answer-28: The bias-variance tradeoff refers to the balance between a model's simplicity (bias) and its complexity (variance). High bias can lead to underfitting, while high variance can lead to overfitting. The goal is to find a balance that minimizes both.

Answer-29: Feature scaling is important in supervised learning, especially for distance-based algorithms like KNN and SVM. It ensures that all features contribute equally to the model by normalizing or standardizing them.

Answer-30: Gradient descent is an optimization algorithm used to minimize the loss function by adjusting the model parameters in the direction of the steepest descent (negative gradient). It is commonly used in linear regression and neural networks.

Answer-31: The main types of gradient descent are batch gradient descent, stochastic gradient descent (SGD), and mini-batch gradient descent. Batch gradient descent uses the whole dataset, while SGD uses one data point, and mini-batch uses a subset of data points.

Answer-32: A decision boundary is a surface that separates different classes in the feature space. It defines the boundary where the model makes different predictions for new data points.

Answer-33: Entropy is a measure of uncertainty or disorder in a dataset. In decision trees, it is used to calculate information gain and decide the best feature to split the data at each node.

Answer-34: Information gain measures how well a feature separates the data. It is used to select the feature that reduces uncertainty (entropy) the most when splitting the data in a decision tree.

Answer-35: Decision trees are easy to interpret, can handle both numerical and categorical data, and require little data preprocessing. However, they are prone to overfitting.

Answer-36: Pruning is the process of removing branches from a decision tree that provide little predictive power to reduce the complexity of the model and avoid overfitting.

Answer-37: The ROC (Receiver Operating Characteristic) curve is a graphical representation of the tradeoff between the true positive rate and false positive rate at different classification thresholds.

Answer-38: The AUC (Area Under the Curve) score represents the area under the ROC curve. It is a metric used to evaluate the performance of a classification model, with higher values indicating better performance.

Answer-39: One-hot encoding is a technique used to represent categorical variables as binary vectors, allowing machine learning algorithms to handle categorical data.

Answer-40: Feature selection aims to identify and retain the most important features in the dataset, improving model performance and reducing computational cost by eliminating irrelevant or redundant features.

Answer-41: Bootstrapping is a technique used in ensemble learning where multiple datasets are created by sampling with replacement from the original dataset, allowing the model to learn from different subsets of data.

Answer-42: Random Forest reduces overfitting, improves model accuracy, and handles both classification and regression tasks well. It also provides feature importance scores.

Answer-43: Bagging involves training multiple models in parallel and averaging their predictions, whereas boosting trains models sequentially, where each new model focuses on correcting the errors of the previous one.

Answer-44: AdaBoost (Adaptive Boosting) is an ensemble learning method that combines multiple weak classifiers to create a strong classifier by adjusting the weights of incorrectly classified instances.

Answer-45: Gradient Boosting is an ensemble technique that builds models sequentially, with each new model focusing on correcting the residual errors (gradients) of the previous models.

Answer-46: In Naive Bayes, continuous features are typically assumed to follow a Gaussian (normal) distribution, and the probability of a continuous feature is calculated based on the likelihood function.

Answer-47: The learning rate controls the size of the steps the algorithm takes while minimizing the loss function. If it is too large, the algorithm might overshoot, and if it is too small, the algorithm might take too long to converge.

Answer-48: In KNN, categorical variables are handled by using a distance metric like Hamming distance instead of Euclidean distance, as categorical variables don't have a meaningful notion of distance.

Answer-49: Random Forest reduces the risk of overfitting by averaging the predictions of multiple decision trees, leading to better generalization and improved accuracy.

Answer-50: In Random Forests, the bias is controlled by the depth of individual trees, and variance is controlled by averaging multiple trees. The goal is to balance bias and variance to improve model generalization.