Logistic Regression Questions and Answers for Viva

Answer-1: Logistic regression is a statistical method used for binary classification problems. It models the probability of a binary outcome using the logistic function.

Answer-2: Linear regression is used for predicting continuous values, while logistic regression is used for predicting binary outcomes (e.g., yes/no, true/false) by modeling the probability.

Answer-3: The logistic function, or sigmoid function, is defined as ?(x)=11+e?x\sigma(x) = \frac{1}{1 + e^{-x}}?(x)=1+e?x1?. It outputs values between 0 and 1, making it suitable for binary classification.

Answer-4: The output of logistic regression represents the probability that a given input belongs to the positive class (usually denoted as 1).

Answer-5: The cost function in logistic regression is the Log Loss (or Binary Cross-Entropy), which measures the difference between predicted probabilities and actual class labels.

Answer-6: Linear regression predicts a continuous output, whereas logistic regression predicts the probability of a binary outcome, which is mapped to a class label (0 or 1).

Answer-7: The decision boundary is the threshold at which the predicted probability equals 0.5. It separates the two classes (0 and 1) in binary classification.

Answer-8: The coefficients in logistic regression represent the change in the log-odds of the dependent variable for a one-unit change in the independent variable.

Answer-9: Log-odds is the logarithm of the odds, where the odds are the ratio of the probability of an event happening to the probability of it not happening. It is used as the linear model in logistic regression.

Answer-10: The odds ratio in logistic regression is the exponentiation of the regression coefficient. It represents how the odds of the dependent variable being 1 change with a one-unit change in the predictor variable.

Answer-11: Assumptions in logistic regression include: the dependent variable is binary, there is no multicollinearity among predictors, and the relationship between the predictors and the log-odds is linear.

Answer-12: The sigmoid function maps any input value to a probability between 0 and 1, which is necessary for binary classification.

Answer-13: Odds are the ratio of the probability of an event happening to the probability of it not happening. The logistic regression model predicts the log-odds, which is converted into probability using the sigmoid function.

Answer-14: MLE is a method for estimating the parameters of a logistic regression model. It finds the parameters that maximize the likelihood of the observed data given the model.

Answer-15: The steps involve: 1) Prepare the data, 2) Initialize coefficients, 3) Use the logistic function to predict probabilities, 4) Compute the cost function (Log Loss), 5) Minimize the cost function using gradient descent or MLE.

Answer-16: Gradient descent is an optimization algorithm used to minimize the cost function in logistic regression by iteratively updating the coefficients in the opposite direction of the gradient.

Answer-17: The learning rate controls the step size in gradient descent. A small learning rate leads to slow convergence, while a large rate may cause overshooting and failure to converge.

Answer-18: The performance of a logistic regression model can be evaluated using metrics like accuracy, precision, recall, F1-score, ROC curve, and AUC (Area Under the Curve).

Answer-19: The confusion matrix is a table that summarizes the model?s predictions against the actual values, showing true positives, false positives, true negatives, and false negatives.

Answer-20: Precision is the proportion of true positive predictions among all positive predictions. It is calculated as TPTP+FP\frac{TP}{TP + FP}TP+FPTP?, where TP is true positives and FP is false positives.

Answer-21: Recall is the proportion of true positive predictions among all actual positives. It is calculated as TPTP+FN\frac{TP}{TP + FN}TP+FNTP?, where TP is true positives and FN is false negatives.

Answer-22: The F1-score is the harmonic mean of precision and recall, providing a balanced measure of a model?s performance when dealing with imbalanced classes.

Answer-23: The ROC (Receiver Operating Characteristic) curve plots the true positive rate (sensitivity) against the false positive rate (1-specificity) for different threshold values.

Answer-24: AUC is the area under the ROC curve. It provides a single measure of a model's ability to distinguish between the classes. A higher AUC indicates better performance.

Answer-25: Multicollinearity can be handled by removing one of the correlated variables, using regularization techniques like L1 or L2 regularization, or applying dimensionality reduction techniques like PCA.

Answer-26: Regularization in logistic regression is used to prevent overfitting by adding a penalty term to the cost function. Common types are L1 (Lasso) and L2 (Ridge) regularization.

Answer-27: L1 regularization, or Lasso, adds the absolute values of the coefficients to the cost function. It can result in sparse models by forcing some coefficients to zero.

Answer-28: L2 regularization, or Ridge, adds the squared values of the coefficients to the cost function. It prevents large coefficients but does not eliminate them.

Answer-29: The ROC curve is important for evaluating classification models, as it shows how the model performs at various thresholds, helping to choose the best threshold for classification.

Answer-30: Logistic regression can handle imbalanced data by using techniques such as class weighting, resampling (oversampling or undersampling), or using metrics like precision-recall curves instead of accuracy.

Answer-31: The log-likelihood function is used in maximum likelihood estimation (MLE) to calculate the probability of observing the given data. The goal is to maximize this function during training.

Answer-32: Yes, logistic regression can be extended to multi-class classification using methods like one-vs-rest (OvR) or softmax regression.

Answer-33: Softmax regression (or multinomial logistic regression) is an extension of logistic regression for multi-class classification problems. It calculates the probability for each class using the softmax function.

Answer-34: The softmax function is a generalization of the sigmoid function for multi-class problems. It converts raw class scores into probabilities for each class.

Answer-35: The likelihood ratio test is used to compare two models by checking if the inclusion of additional predictors improves the fit. It compares the log-likelihoods of the two models.

Answer-36: A hyperplane is a decision boundary that separates the two classes in the feature space. In logistic regression, it is defined by the linear equation formed by the coefficients.

Answer-37: The output probability from logistic regression represents the probability that the instance belongs to the positive class. A value greater than 0.5 typically indicates class 1, while a value less than 0.5 indicates class 0.

Answer-38: A decision threshold is a cut-off probability used to classify predictions as either class 0 or class 1. The default threshold is typically 0.5, but it can be adjusted based on the problem.

Answer-39: Logistic regression is easy to implement, computationally efficient, interpretable, and works well for binary classification tasks.

Answer-40: Limitations include the assumption of linearity between the predictors and the log-odds, sensitivity to outliers, and inability to capture non-linear relationships.

Answer-41: The p-value in logistic regression tests the null hypothesis that the coefficient of a predictor is zero (no effect). A small p-value (typically less than 0.05) indicates that the predictor is statistically significant.

Answer-42: AIC is a measure used to compare different logistic regression models. It balances goodness of fit with model complexity, with lower AIC values indicating better models.

Answer-43: Regularization reduces overfitting by penalizing large coefficients, leading to simpler, more generalizable models.

Answer-44: Missing data can be handled by imputing values using mean, median, or mode, or by using advanced methods like k-nearest neighbors imputation or multiple imputation.

Answer-45: Feature scaling helps in standardizing the range of the independent variables, especially when regularization is applied, ensuring the model converges faster and is more stable.

Answer-46: Logistic regression is a linear model for binary classification, while decision trees are non-linear models that recursively split data based on feature values to make predictions.

Answer-47: Overfitting in logistic regression occurs when the model learns the noise in the training data rather than the underlying pattern, leading to poor generalization on unseen data.

Answer-48: Applications include medical diagnosis (e.g., disease prediction), credit scoring, spam email classification, and marketing (e.g., customer churn prediction).

Answer-49: Logistic regression is primarily used for classification problems. For regression problems, linear regression is typically used.

Answer-50: Binary logistic regression is used for two-class problems, while multi-class logistic regression (also called multinomial logistic regression) is used for problems involving more than two classes.