What is an ensemble learning?
Ensemble learning is a method within machine learning that involves merging multiple individual models to construct a more potent and precise predictive model. This approach enhances overall performance, resulting in improved generalization and increased robustness.
Working of ensemble methods
Ensemble methods work by combining the predictions of different models. If individual models create an error, this helps to correct errors and make more accurate predictions overall. This is particularly helpful when dealing with complex data or when models have different strengths and weaknesses.
The following are some ensemble methods:
Bootstrap aggregating: This method trains multiple base models on different parts of the data created by random sampling. The final prediction combines all base models' predictions through averaging or voting.
Boosting: This method trains models one after the other, where each new model corrects the errors of the previous ones. The end prediction is a mix of all models, giving more importance to the better-performing ones.
Random forest: Random Forest is a unique mix of bagging and decision trees. It creates trees on different data parts and combines their predictions to improve accuracy and prevent overfitting.
Adaptive boosting: This is a boosting algorithm that learns from its mistakes. It focuses on fixing the errors by giving more importance to the wrongly predicted data in each round, which improves accuracy.
Gradient boosting: Similar to adaptive boosting, gradient boosting creates models step by step, but it's more about reducing mistakes directly by improving a loss function.
Example
Let's look at an easy example of ensemble learning using Python's scikit-learn library and the RandomForestClassifier:
from sklearn.datasets import load_irisfrom sklearn.model_selection import train_test_splitfrom sklearn.ensemble import RandomForestClassifierfrom sklearn.metrics import accuracy_scoreiris = load_iris()X = iris.datay = iris.targetX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)ensemble_model = RandomForestClassifier(n_estimators=100, random_state=42)ensemble_model.fit(X_train, y_train)predictions = ensemble_model.predict(X_test)accuracy = accuracy_score(y_test, predictions)print(f"Accuracy of the ensemble model: {accuracy:.2f}")
Explanation
Lines 1–4: We import the required modules from the
scikit-learnlibrary. These modules include dataset loading, data splitting, random forest classifier, and accuracy calculation.Lines 6–8: We load the
Irisdataset. The dataset contains features inX(iris data) and target labels iny(iris target).Line 10: We split the dataset into training and testing sets. This ensures that the model is trained on a portion of the data and tested on a separate portion.
Line 12: We create an ensemble using
RandomForestClassifier. This model comprises 100 decision trees, which will work together to make predictions.Line 14: We fit the ensemble model on the training data.
Line 16: We make predictions using the ensemble model.
Line 18: We calculate the accuracy of the ensemble model's predictions by comparing the
predictionswith the actual label (y_test).Line 19: We print the calculated accuracy of the ensemble model.
Conclusion
Ensemble learning can improve predictions, reduce overfitting, and strengthen the model. However, it needs careful adjustments and can use computer power because it trains many models. Popular machine learning libraries like scikit-learn and XGBoost have built-in help for different ensemble methods.
Free Resources
