Many Trees, One Answer: Ensemble Learning, Random Forests and Boosting Explained
the problem with trusting one tree
A decision tree is one of the most intuitive models in data mining. It asks a sequence of yes-or-no questions about the attributes of a record, such as “Is the credit score above 672?” and “Is existing debt below $10,000?”, until it reaches a leaf that assigns a class. The trouble is that a tree allowed to grow without limits will keep splitting until it has carved out a separate region for almost every training record. Its training accuracy approaches 100 percent, but its accuracy on records it has never seen is noticeably lower. That gap between training performance and test performance is the classic signature of overfitting (Tan et al., 2019).
The obvious remedy is to prune the tree or cap its depth. This works, but it trades one problem for another: a shallow tree is stable, yet it cannot capture genuinely complex patterns. Ensemble learning takes a different route. Instead of searching for the one perfect tree, it builds many imperfect trees that differ from each other in useful ways and then combines their answers. A committee of a hundred slightly different trees is usually both more accurate and more stable than any single member (Dietterich, 2000a). This article walks through the three ideas at the heart of modern ensemble methods. Part 1 explains ensemble learning in general and bagging in particular, including the probability argument for why voting works. Part 2 covers random forests, which add one extra source of randomness to bagging and produce one of the most reliable off-the-shelf classifiers available. Part 3 turns to boosting, which builds its committee sequentially so that each new tree concentrates on the mistakes of the trees before it. Every part includes a worked example you can check by hand, results from a real experiment run in Python with scikit-learn (Pedregosa et al., 2011), and a frank discussion of where each method fits and where it falls short.
By the end you should be able to explain, in a sentence each, what bagging and boosting do differently; compute a bootstrap probability, a Gini split and an AdaBoost weight update by hand; read a feature-importance chart critically; and choose sensibly between a random forest and a gradient boosting model for a new project.
Part 1: Ensemble learning and bagging
What an ensemble is, and what it is not.
Ensemble learning combines the predictions of several base models into a single prediction, usually by majority vote for classification or by averaging for regression (Tan et al., 2019). The intuition is familiar from everyday life: a panel of judges is harder to fool than one judge, and a second medical opinion catches errors the first one missed. What matters is not the number of opinions but their independence. If you train the same algorithm on exactly the same data a hundred times, you get a hundred identical trees that make identical mistakes, and voting among them changes nothing. The whole craft of ensemble learning lies in making the members different enough that their errors do not line up.
Dietterich (2000a) gives three reasons why a well-built ensemble can outperform its best member. The statistical reason is that, with limited data, many different models fit the training set equally well, and averaging them reduces the risk of picking an unlucky one. The computational reason is that tree-growing algorithms are greedy and can get stuck in poor local choices, so combining several runs smooths over those choices. The representational reason is that a combination of simple models can express decision boundaries that no single member could represent on its own.
Why voting works: a worked probability example
The power of voting can be shown with a short calculation adapted from Tan et al. (2019). Suppose we have 25 binary classifiers, each with an error rate of 0.35, and suppose, crucially, that their errors are independent. The ensemble makes a mistake only when a majority, at least 13 of the 25, are wrong at the same time. The number of wrong classifiers follows a binomial distribution, so the ensemble error is the sum over i = 13 to 25 of C(25, i) × 0.35^i × 0.65^(25 − i). Working this out gives about 0.06. Twenty-five mediocre classifiers, each wrong 35 percent of the time, combine into a committee that is wrong only 6 percent of the time.
Two conditions make this work, and both matter in practice. Each member must be better than random guessing (an error rate below 0.5), otherwise voting amplifies the errors instead of cancelling them. And the members must be at least partly independent. Real trees trained on the same data are never fully independent, so real ensembles improve far less dramatically than this table suggests. That gap between the ideal and the real is exactly why bagging, random forests and boosting were invented: each is a different strategy for manufacturing diversity.
Bagging: diversity through resampling
Bagging, short for bootstrap aggregating, was introduced by Breiman (1996). It creates diversity by giving every tree a slightly different training set. Each set is a bootstrap sample: n records drawn at random, with replacement, from the original n records. Because the draw is with replacement, some records appear two or three times in a given sample while others do not appear at all. A full, unpruned tree is grown on each bootstrap sample, and at prediction time every tree casts one vote. The procedure is simple enough to state in four steps. First, choose the number of trees, B. Second, for each b from 1 to B, draw a bootstrap sample of size n and grow a decision tree on it. Third, to classify a new record, pass it down every tree and collect B predictions. Fourth, return the majority class, or the average for a regression problem.
Consider the 14-row loan dataset used in class, with records numbered 1 to 14. On any single draw, a particular record has a 1/14 chance of being picked and a 13/14 chance of being missed. Across 14 independent draws, the probability that it is never picked is (13/14)¹⁴, which equals about 0.354. In other words, roughly 35 percent of the records are left out of each bootstrap sample and about 65 percent appear at least once. As the dataset grows, (1 − 1/n)^n approaches e^(−1) ≈ 0.368, which is why textbooks quote the figure that each bootstrap sample contains about 63.2 percent of the distinct original records (Hastie et al., 2009).
One actual bootstrap sample, generated with a fixed random seed, drew the records 1, 1, 4, 4, 5, 9, 9, 10, 11, 12, 13, 13, 13, 14. It contains 14 rows but only 9 distinct records, and records 2, 3, 6, 7 and 8 were never chosen. Those five left-out records are called the out-of-bag records for that tree. They will become important in Part 2, because they give us a free test set.
Worked example: the vote and how much to trust it
Suppose a small forest of five trees classifies two new loan applicants. For Applicant A the trees vote Approve, Approve, Reject, Approve, Reject, so the result is Approve by 3 votes to 2. For Applicant B they vote Reject, Reject, Approve, Reject, Reject, so the result is Reject by 4 votes to 1. Both are clear predictions, but they deserve very different levels of trust. Applicant A’s vote share for Approve is 3/5 = 0.6, and a single tree changing its mind would flip the decision. Applicant B’s vote share for Reject is 4/5 = 0.8, a far more stable outcome. This is precisely what scikit-learn’s predict_proba() method reports. In a real model it can drive business rules: a lender might automatically approve applicants whose approval probability exceeds 0.8, automatically decline those below 0.2, and route everything in between, like Applicant A, to a human underwriter.
Bagging in the real world, and its limits
Bagging reduces variance, which makes it most useful for models that are unstable, meaning that small changes in the training data produce large changes in the fitted model. Unpruned decision trees are the textbook example (Breiman, 1996). In our breast cancer experiment, described fully in Part 2, a single unpruned tree reached 92.0 percent cross validated accuracy while bagging 100 such trees reached 95.9 percent. The same variance-reduction logic explains crowd-sourced estimation more broadly. In 1907 Francis Galton analysed about 800 guesses of the dressed weight of an ox at a country fair and found that the median guess, 1,207 pounds, was within 1 percent of the true weight of 1,198 pounds (Galton, 1907).
Bagging has clear limitations. It does little for stable, high-bias models such as linear regression or very shallow trees, because their bootstrap replicas are nearly identical. It multiplies training and prediction cost by the number of trees. It replaces one readable tree with a hundred unreadable ones. And, most importantly for what follows, when one attribute dominates, most bagged trees choose it for their first split and end up looking alike, which weakens the diversity that voting depends on.
Part 2: Random forests
The extra trick: random attribute subsets.
A random forest is bagging plus one additional source of randomness. Breiman (2001) observed that bagged trees remain correlated because every tree is allowed to choose the strongest attribute at every split. If credit score is by far the best predictor of loan default, nearly every bagged tree will place credit score at its root, and the forest becomes a crowd of near-identical voices. Random forests break this pattern by letting each split consider only a random subset of m attributes out of the p available, with a fresh subset drawn at every node. For classification the usual default is m = √p, rounded down, so a dataset with 16 attributes gives each split a choice among 4 randomly selected candidates.
The effect is that some trees are forced to build their early splits from weaker attributes. Each individual tree becomes slightly worse, but the trees become much less alike. Breiman (2001) showed formally that a forest’s generalisation error depends on two quantities: the strength of the individual trees and the correlation between them. Random attribute selection deliberately sacrifices a little strength in exchange for a large reduction in correlation, and in most datasets that trade pays off.
The algorithm in five steps.
First, choose the number of trees B and the number of candidate attributes per split, m. Second, for each tree, draw a bootstrap sample of the training data. Third, grow the tree on that sample, and at every node pick m attributes at random and choose the best split among those m only. Fourth, grow each tree fully, usually without pruning, because the averaging step will control the variance. Fifth, classify new records by majority vote across all B trees. Setting m equal to p turns the algorithm back into plain bagging, which shows that bagging is simply a special case of a random forest.
At every node, each candidate attribute is scored by how much it reduces impurity. Scikit learn’s default measure is the Gini index, defined as 1 minus the sum of squared class proportions. A perfectly pure node has a Gini of 0, and a node split evenly between two classes has a Gini of 0.5. Take the 14-row loan dataset, which contains 7 approved and 7 rejected applicants. The root node’s Gini is 1 − (0.⁵² + 0.⁵²) = 0.5. Now test the split credit_score ≤ 700. The left branch receives 9 applicants (7 rejected and 2 approved, the two approvals having scores of 690 and 700), and the right branch receives 5 applicants, all approved. The left Gini is 1 − ((7/9)² + (2/9)²) = 28/81 ≈ 0.346. The right Gini is 0 because the node is pure. The weighted Gini after the split is (9/14 × 0.346) + (5/14 × 0) ≈ 0.222, so the split reduces impurity by 0.5 − 0.222 = 0.278
A better threshold exists. The split credit_score ≤ 672.5 sends all 7 rejected applicants left and all 7 approved applicants right, giving two pure nodes and an impurity reduction of the full 0.5. The tree therefore prefers 672.5. A useful detail emerges when you check the other two attributes: income ≤ 42.5 and existing debt ≤ 6.5 also separate the classes perfectly. In a random forest with m = 1 (because √3 rounds down to 1), each tree’s root split is made on whichever attribute happens to be drawn, and all three give perfect splits. That is why the forest trained in the class lab reports almost equal importances for the three attributes (about 0.36, 0.35 and 0.28), rather than the dominant credit score shown in the illustrative lecture chart. On a 14-row toy dataset, the single tree, the random forest and the gradient boosting model all score 100 percent on the 5-row test split, a reminder that small, perfectly separable datasets cannot reveal differences between methods.
Out-of-bag error: a free validation set
Recall from Part 1 that each bootstrap sample leaves out roughly 36.8 percent of the records. For any training record, about a third of the trees never saw it. We can ask only those trees to vote on that record and compare their majority with the true label. Repeating this for every record gives the out-of-bag (OOB) error estimate, which behaves much like cross-validation but costs nothing extra (Breiman, 2001). In our breast cancer experiment, the forest’s OOB accuracy was 95.7 percent, closely matching its cross validated accuracy of 96.1 percent. For large datasets where repeated cross-validation is expensive, OOB error is a practical way to tune the forest.
A real experiment: the Wisconsin breast cancer data
To move beyond toy examples, every model in this article was evaluated on the Breast Cancer Wisconsin (Diagnostic) dataset (Wolberg et al., 1995). It contains 569 tumour samples, 212 malignant and 357 benign, each described by 30 numeric measurements computed from digitised images of cell nuclei. Each model was assessed with 10-fold cross-validation repeated three times, giving 30 test scores per model, using scikit-learn 1.8 with default settings unless stated.
Three lessons stand out. Every ensemble beat the best single tree by roughly four to five percentage points, cutting the error rate approximately in half, from about 8 percent to about 4 percent. The ensembles were also more stable, with smaller spread across folds. And capping the tree’s depth did not help here: the shallow tree’s training and test scores sat closer together, yet its test accuracy did not improve at all. The ensembles reached a combination that no single tree could.
Get ARDEEE RAYL’s stories in your inbox
Join Medium for free to get updates from this writer.
How many trees are enough?
The number of trees, n_estimators, is the most visible setting of a random forest. Using the same dataset with 5-fold cross-validation repeated twice, a forest of 1 tree scored 92.3 percent, 5 trees scored 95.9 percent, 20 trees scored 96.8 percent, and from 50 to 200 trees the score hovered between 96.3 and 96.7 percent. The curve rises steeply and then plateaus, with small fluctuations caused by random variation rather than any real decline. Adding trees to a random forest does not cause overfitting in the way that adding depth to a single tree does (Breiman, 2001). It simply stops helping, while training and prediction time keep growing linearly. In practice, a few hundred trees is a sensible default, with the OOB error curve used to confirm that the plateau has been reached.
Feature importance, and how to read it critically
A trained forest can report how much each attribute contributed. Scikit-learn’s feature_importances_ attribute gives the mean decrease in impurity: for each attribute, it adds up the Gini reductions achieved by every split on that attribute, weighted by the number of records reaching each split, averaged over all trees and normalised to sum to 1. For a 500-tree forest fitted to the full breast cancer data, the top five attributes were worst perimeter (0.133), worst concave points (0.126), worst radius (0.112), worst area (0.109) and mean concave points (0.097). Together they account for 58 percent of the total importance across 30 attributes. This turns a black-box ensemble into a sentence a clinician or manager can act on: tumour size and the concavity of the cell boundary drive the model’s decisions.
Impurity-based importance needs careful interpretation. Strobl et al. (2007) showed that it is biased toward continuous attributes and attributes with many possible split points. Correlated attributes create a second problem. In this dataset, worst radius, worst perimeter and worst area are correlated at 0.98 or above, because they all measure tumour size. The forest spreads the credit for size across all three, so each looks only moderately important, even though size as a concept dominates. When attributes are correlated, permutation importance, which measures how much accuracy falls when one attribute’s values are shuffled, offers a useful cross-check, though it too is affected by correlation. Importance scores describe what the model used. They do not prove that an attribute causes the outcome.
Where random forests are used
Random forests are one of the most widely deployed algorithms in applied data mining. Microsoft’s Kinect sensor used randomised decision forests to label each pixel of a depth image with a body part in real time, enabling skeletal tracking from a single camera (Shotton et al., 2011). In credit scoring and fraud detection, forests handle mixed numeric and categorical attributes and provide probability scores that support tiered decision rules. In bioinformatics and medical diagnosis they cope well with many attributes and few samples. In a large benchmark of 179 classifiers across 121 datasets, Fernández Delgado et al. (2014) found random forest variants among the most consistently accurate families.
Strengths and limitations
Random forests are hard to misconfigure. They need little tuning, are not sensitive to the scale of numeric attributes, handle large numbers of attributes, provide an OOB error estimate for free, and train in parallel because each tree is independent. Their limitations are equally real. A forest of hundreds of deep trees is large in memory and slower to predict than a single tree. It loses the readable if-then structure that makes a single tree attractive. In regression, it cannot predict values outside the range seen in training, because each prediction is an average of training outcomes. And on highly imbalanced data, majority voting can favour the common class unless class weights or resampling are applied.
Part 3: Boosting
A different way to build a crowd
Bagging and random forests build their trees independently and in parallel, and diversity comes from randomness. Boosting builds its trees one after another, and diversity comes from design. Each new tree is trained to concentrate on the records that the trees before it got wrong. The final prediction is a weighted vote in which more accurate trees receive a larger say (Freund & Schapire, 1997). Where bagging mainly reduces variance, boosting mainly reduces bias: it can turn a weak learner, such as a one-split tree called a decision stump, into a strong classifier by stacking many of them
AdaBoost: re-weighting the hard cases
AdaBoost, short for adaptive boosting, is the original practical boosting algorithm (Freund & Schapire, 1997). It starts by giving every training record the same weight, 1/n. In each round it trains a weak classifier on the weighted data and computes its weighted error ε, the sum of the weights of the records it misclassifies. It then gives that classifier a voting weight, using the form found in most textbooks (Tan et al., 2019; Hastie et al., 2009): α = ½ × ln((1 − ε) / ε) A classifier with ε close to 0 receives a large α, while one with ε close to 0.5, barely better than a coin toss, receives an α close to 0. Next, each record’s weight is updated. Misclassified records are multiplied by e^α and correctly classified records by e^(−α), and the weights are then rescaled so that they sum to 1. The next classifier is trained on these new weights. The final prediction for a record is the sign of the sum of α × h(x) over all rounds, where each h(x) is +1 for one class and −1 for the other.
Worked example: three rounds of weights by hand
Take the five-point exercise from class. The true labels are Approve, Reject, Approve, Reject, Approve for points 1 to 5. Round 1’s classifier, h1, predicts Approve, Approve, Approve, Reject, Reject, so it misclassifies points 2 and 5.
Round 1 weights. Every point starts at 1/5 = 0.2. The weighted error is ε1 = 0.2 + 0.2 = 0.4. The voting weight is α1 = ½ × ln(0.6 / 0.4) = ½ × ln(1.5) ≈ 0.203.
Updating to Round 2. The three correct points are multiplied by e^(−0.203) ≈ 0.816, giving 0.163 each. The two wrong points are multiplied by e^(0.203) ≈ 1.225, giving 0.245 each. These sum to 3 × 0.163 + 2 × 0.245 ≈ 0.980, so dividing by 0.980 gives Round 2 weights of 1/6 ≈ 0.167 for points 1, 3 and 4 and 0.25 for points 2 and 5. Notice an elegant property: the misclassified points now hold exactly half of the total weight (0.25 + 0.25 = 0.5). AdaBoost always rebalances so that the previous classifier would score no better than chance on the new weights, forcing the next classifier to learn something different.
Updating to Round 3. Suppose Round 2’s classifier, h2, predicts Approve, Reject, Approve, Reject, Reject. It fixes point 2 but still misses point 5. Its weighted error is ε2 = 0.25, so α2 = ½ × ln(0.75 / 0.25) = ½ × ln(3) ≈ 0.549. The correct points are multiplied by e^(−0.549) ≈ 0.577 and point 5 by e^(0.549) ≈ 1.732. After normalising, the Round 3 weights are 1/9 ≈ 0.111 for points 1, 3 and 4, 1/6 ≈ 0.167 for point 2, and 0.5 for point 5. Half of the algorithm’s attention now rests on the single record it keeps getting wrong.
The weighted vote after two rounds. Coding Approve as +1 and Reject as −1, the combined score is 0.203 × h1(x) + 0.549 × h2(x). Point 2 scores 0.203 − 0.549 = −0.346, so the ensemble now correctly predicts Reject, even though h1 alone got it wrong. Points 1, 3 and 4 remain correct. Point 5 scores −0.203 − 0.549 = −0.752 and is still wrong. The training error has fallen from 40 percent for h1 alone to 20 percent for the two-member ensemble, and Round 3 will train with half its weight on point 5. Scikit-learn’s AdaBoostClassifier uses the SAMME variant, whose α for two classes is ln((1 − ε)/ε), exactly twice the textbook value. Because every α is doubled, the ranking of votes and the direction of every weight change stay the same.
Gradient boosting: fitting the residuals
Friedman (2001) generalised boosting into gradient boosting. Instead of re-weighting records, each new tree is fitted to the errors that remain, known as residuals for squared error loss and, more generally, as negative gradients of a loss function. The model starts from a constant prediction, adds a small tree that predicts the residuals, multiplies that tree’s output by a learning rate η to take only a cautious step, recomputes the residuals, and repeats. Because the framework works with any differentiable loss function, the same idea handles regression, classification and ranking.
Worked example: gradient boosting a tiny regression Three houses have floor areas of 100, 200 and 300 square metres and prices of $300k, $500k and $1,000k. We use squared-error loss, decision stumps, and a learning rate of 0.5 so that the effect is visible by hand (scikit-learn’s default is 0.1). Step 0. The best constant prediction under squared error is the mean, F0 = $600k for every house. The residuals are −300, −100 and +400, in thousands. The mean squared error (MSE) is (30⁰² + 10⁰² + 40⁰²) / 3 ≈ 86,667, in thousands of dollars squared. Step 1. A stump fitted to the residuals splits at area ≤ 250. Its left leaf predicts the mean residual of the first two houses, −200, and its right leaf predicts +400. With η = 0.5 the predictions become 600 − 100 = 500, 500 and 600 + 200 = 800. The new residuals are −200, 0 and +200, and the MSE falls to 26,667. Step 2. A stump fitted to the new residuals splits at area ≤ 150, predicting −200 on the left and +100 (the mean of 0 and 200) on the right. With η = 0.5 the predictions become 400, 550 and 850. The residuals are −100, −50 and +150, and the MSE falls to about 11,667. Each step closes part of the remaining gap without closing all of it. A small learning rate means more trees are needed, but the model generalises better because no single tree is trusted too much (Friedman, 2001; Hastie et al., 2009). This explains a striking number in our breast cancer experiment. A gradient boosting model with a single tree scored 62.7 percent, which equals 357/569, exactly the share of benign samples. With a learning rate of 0.1, one tree moves the starting prediction only slightly, so almost every sample still falls on the majority-class side. The score climbed to 93.8 percent with 5 trees, 95.3 percent with 100 trees, and 95.6 percent with 150 and 200 trees. Compared with the random forest, which was already at 95.9 percent with 5 trees, boosting climbs more slowly because it builds its answer step by step.
Modern gradient boosting libraries
Several engineered implementations dominate applied work on tabular data. XGBoost adds explicit regularisation of tree complexity, efficient handling of missing values, and a parallel, cache-aware design (Chen & Guestrin, 2016). LightGBM speeds up training on large datasets with histogram-based splitting and by growing trees leaf by leaf (Ke et al., 2017). CatBoost introduces ordered boosting and native encoding of categorical attributes to reduce a subtle form of target leakage (Prokhorenkova et al., 2018). A large benchmark by Grinsztajn et al. (2022) found that tree-based ensembles such as XGBoost and random forests still outperformed deep neural networks on typical medium-sized tabular datasets.
Where boosting is used
One of the best-known applications is the Viola–Jones face detector, which used AdaBoost to select a small number of informative image features from a very large pool and arranged boosted classifiers in a cascade fast enough for real-time detection (Viola & Jones, 2001). Gradient boosted trees are widely used for credit risk scoring, insurance pricing, customer churn prediction, demand forecasting, click-through-rate prediction and search ranking, wherever data comes as rows and columns and the last few percentage points of accuracy have real financial value.
Strengths and limitations
Boosting often achieves the highest accuracy on structured data. In our experiment AdaBoost with 100 stumps reached 97.4 percent, the best of all six models. It works well with simple weak learners, and modern libraries scale to millions of rows. Its weaknesses mirror bagging’s strengths. Because the trees depend on each other, they must be trained in sequence, although libraries such as XGBoost parallelise the work inside each tree. Boosting has more settings that interact, including the number of trees, learning rate, tree depth and subsampling rate, and a poor combination can overfit. It is also sensitive to label noise: since mislabelled records are, by definition, hard to fit, boosting keeps increasing their weight. Dietterich (2000b) found that when classification noise was added to the training data, bagging became the stronger method while AdaBoost degraded. The standard defences are a small learning rate, shallow trees, row subsampling, and early stopping on a validation set.
Bagging and boosting share a goal but pursue it in opposite ways, and the differences translate directly into practical advice.
A sensible workflow follows from this table. Start a new tabular project with a random forest, because it is difficult to configure badly, gives an honest out-of-bag error estimate, and provides a strong baseline within minutes. If the project needs the last one or two percentage points of accuracy, and you have time to tune and a clean validation set for early stopping, move to gradient boosting with a library such as XGBoost or LightGBM. If the labels are known to be noisy, if training must be heavily parallelised, or if the people maintaining the model are not machine learning specialists, the random forest is usually the safer long-term choice. In regulated settings where every individual decision must be explained, a single shallow tree or a linear model may still be required, with an ensemble used alongside it as a benchmark.
In our breast cancer experiment the differences among the four ensembles, between 95.9 and 97.4 percent, were small compared with their shared advantage over a single tree, and they fall within one standard deviation of the fold-to-fold variation. That pattern is typical: choosing to use an ensemble at all matters more than choosing which ensemble to use.
Ensemble learning rests on one idea: many imperfect models, combined well, beat one carefully built model. The binomial calculation in Part 1 showed why this works when errors are independent, and the rest of the article showed three practical ways to create that independence. Bagging trains each tree on a different bootstrap sample. Random forests add random attribute subsets so that even the strongest predictor cannot dominate every tree. Boosting abandons independence altogether and builds trees in sequence, each one aimed at the errors left by the last.
The experiment on real diagnostic data made these ideas concrete. A single tree, pruned or not, plateaued at about 92 percent accuracy. Every ensemble reached between 96 and 97.4 percent and varied less from one test fold to the next. The forest’s out-of-bag estimate matched cross-validation without any extra computation, and its feature importances pointed to tumour size and boundary concavity, while also reminding us that correlated attributes split their credit.
If you remember one sentence from this article, make it this one: bagging builds its trees independently and lets them vote to reduce variance, while boosting builds its trees in sequence so that each corrects the last to reduce bias. Everything else, including random attribute subsets, learning rates, out-of-bag errors and feature importances, is a refinement of that single contrast. The next step in most data mining courses is to leave labelled data behind entirely and explore clustering, where the challenge is to find structure without any target to predict.
— Rhejel Dave Crisporo
