Decision Trees Questions and Answers for Viva

Answer-1: A decision tree is a flowchart-like tree structure where each internal node represents a decision based on a feature, each branch represents the outcome of that decision, and each leaf node represents a final decision or classification.

Answer-2: A decision tree works by recursively splitting the data into subsets based on the feature that best separates the data. It continues until a stopping condition is met, resulting in leaf nodes with predicted class labels or values.

Answer-3: The purpose of a decision tree is to model decisions and outcomes by partitioning data into homogeneous subsets to make predictions based on input features.

Answer-4: The two main types of decision trees are: 1) Classification Trees (for classification problems) and 2) Regression Trees (for regression problems).

Answer-5: Advantages include simplicity, interpretability, ability to handle both categorical and numerical data, and the ability to handle missing values.

Answer-6: Disadvantages include overfitting, instability (small changes in data can result in a different tree), and inability to model complex relationships (non-linear patterns).

Answer-7: Overfitting in decision trees can be prevented by pruning the tree, setting a maximum depth, requiring a minimum number of samples per split, or using ensemble methods like Random Forests.

Answer-8: Pruning is the process of removing parts of the decision tree that do not provide additional predictive power, thereby reducing the complexity and improving generalization.

Answer-9: Pre-pruning involves stopping the growth of the tree early based on certain conditions (e.g., maximum depth), while post-pruning involves growing the tree fully and then removing branches that don?t improve performance.

Answer-10: Gini impurity is a measure of how often a randomly chosen element from the dataset would be incorrectly classified. It is used to decide the optimal split in classification trees. The formula is 1??pi21 - \sum p_i^21??pi2?, where pip_ipi? is the probability of each class.

Answer-11: Entropy is a measure of the disorder or uncertainty in a dataset. It is used to calculate the information gain for splitting the data. The formula is ??pilog?2pi- \sum p_i \log_2 p_i??pi?log2?pi?.

Answer-12: Information gain is the reduction in entropy or Gini impurity achieved by a particular split. It measures how well a feature separates the classes.

Answer-13: The best feature for a split is chosen based on the highest information gain (for classification) or the lowest Gini impurity (for classification). For regression, variance reduction is used.

Answer-14: The CART (Classification and Regression Trees) algorithm is a widely used decision tree algorithm that builds binary trees using Gini impurity (for classification) and variance reduction (for regression) to split nodes.

Answer-15: ID3 (Iterative Dichotomiser 3) is an algorithm used to build decision trees, which selects the feature with the highest information gain to split the data. It uses entropy and information gain for this process.

Answer-16: C4.5 is an extension of ID3 that builds decision trees using information gain ratio to handle continuous attributes and deal with missing values. It also performs post-pruning to prevent overfitting.

Answer-17: Decision trees handle categorical data by selecting the feature that best splits the data into groups based on class distribution. The splitting criterion (like Gini or entropy) is calculated based on the categories.

Answer-18: Continuous data is handled by selecting a threshold value that best splits the data into two subsets, and then recursively applying this process. The split is based on a threshold value for the continuous feature.

Answer-19: The root node is the topmost node in a decision tree and represents the entire dataset. It is the first feature chosen for splitting the data.

Answer-20: Leaf nodes represent the final decision or class label in a decision tree. They correspond to the output of the model after all splits have been made.

Answer-21: The depth of a decision tree refers to the number of levels in the tree, from the root to the deepest leaf node. It is a measure of the complexity of the tree.

Answer-22: Splitting criteria (such as Gini impurity, entropy, or variance reduction) are used to determine the best feature to split the data at each node, ensuring that the resulting subsets are as pure as possible.

Answer-23: Stopping conditions include reaching a maximum tree depth, having a minimum number of samples per leaf node, achieving pure nodes (all samples belong to the same class), or having no further improvement in information gain.

Answer-24: A binary decision tree is a tree where each internal node splits the data into exactly two child nodes based on a decision criterion, resulting in a binary branching structure.

Answer-25: Advantages include ease of interpretation, ability to handle both categorical and numerical data, ability to model non-linear relationships, and no need for data normalization.

Answer-26: Limitations include overfitting, inability to model complex relationships unless pruned, instability with small changes in data, and a tendency to favor attributes with more levels.

Answer-27: Classification trees predict discrete class labels, while regression trees predict continuous numerical values. The splitting criterion differs: classification trees use Gini or entropy, while regression trees use variance reduction.

Answer-28: Feature selection is important in decision trees to avoid overfitting, reduce complexity, and ensure that only relevant features are used for splitting the data.

Answer-29: Decision trees handle missing values by using surrogate splits or by assigning the missing values to the most probable class based on the majority class of the node.

Answer-30: A surrogate split is a backup rule used when a feature is missing in a decision tree. It uses an alternative feature that closely matches the original feature's splitting criterion.

Answer-31: The decision tree algorithm minimizes error by iteratively splitting the data based on the best feature that reduces uncertainty (entropy or Gini impurity) or variance at each step.

Answer-32: Feature importance in decision trees is a measure of how useful a feature is in predicting the target variable. It is typically calculated based on the amount of reduction in impurity or variance achieved by the feature.

Answer-33: Applications include classification tasks (e.g., customer churn prediction), regression tasks (e.g., house price prediction), medical diagnosis, fraud detection, and decision analysis.

Answer-34: A random forest is an ensemble method that combines multiple decision trees to improve classification and regression accuracy. It uses bagging to train each tree on random subsets of the data.

Answer-35: A random forest improves upon a single decision tree by reducing variance and overfitting through ensemble learning, where multiple trees are trained on random data subsets and their predictions are aggregated.

Answer-36: Bagging (Bootstrap Aggregating) is a technique used in ensemble methods like Random Forests, where multiple decision trees are trained on different random subsets of the data, and their predictions are averaged.

Answer-37: Boosting is an ensemble technique where multiple weak learners (typically decision trees) are combined, with each new tree focusing on correcting the errors of the previous trees. Popular boosting algorithms include AdaBoost and Gradient Boosting.

Answer-38: Bagging trains multiple models independently on different random subsets of data and averages their predictions, while boosting trains models sequentially, focusing on correcting the mistakes of previous models.

Answer-39: In the ID3 algorithm, entropy is used to measure the impurity of a dataset. The feature that minimizes entropy (maximizing information gain) is chosen for the split.

Answer-40: Decision trees can handle non-linearity by splitting the data at various points in the feature space, allowing them to model complex, non-linear relationships without explicitly modeling them.

Answer-41: The main criterion used for splitting data in classification trees is either Gini impurity or entropy, which measure the disorder or uncertainty within the subsets.

Answer-42: A decision tree is a non-linear model that makes decisions based on feature splitting, whereas an SVM is a linear classifier that finds a hyperplane that separates the data into classes.

Answer-43: Decision trees handle large datasets efficiently by recursively partitioning the data and making splits at each node based on the most significant feature, allowing the algorithm to scale with data size.

Answer-44: Decision trees can be used for multi-class classification by allowing multiple classes to be represented at the leaf nodes, and by splitting data based on the feature that best separates the classes.

Answer-45: Cross-validation is a technique used to evaluate the performance of a decision tree by splitting the dataset into multiple folds, training the model on some folds, and testing it on the remaining folds.

Answer-46: Decision trees split data based on feature values to create a tree structure, while KNN makes predictions by finding the nearest neighbors in the data space and taking a majority vote or averaging their values.

Answer-47: Yes, decision trees can be used for regression tasks, where they predict continuous values instead of class labels by minimizing variance at each split.

Answer-48: The root node represents the entire dataset and is the first point where the data is split based on the feature that maximizes the information gain or minimizes the impurity.

Answer-49: Decision trees can overfit noisy data, but using techniques like pruning, setting a minimum number of samples per leaf, or using ensemble methods like Random Forests can reduce the impact of noise.

Answer-50: Decision trees handle large datasets with many features by selecting the most relevant features for splitting, and using techniques like feature selection and pruning to manage complexity and reduce overfitting.