
Gain insights into Scikit-Learn's datasets, feature engineering, linear/logistic regression, and unsupervised learning. Delve into k-means clustering and neural networks to enhance your machine learning expertise.

mk1.tar.gz

data_preprocessing

dataset

decision_tree_classification

feature_extraction

feature_selection

gradient_boost

kmeans

knn_classification

sk_lr_classification

sk_lr_regression

sk_metrics

sk_missing_value

sk_naive_bayes_classification

sk_nn

sk_parameter_search

sk_pca

sk_pipeline

sk_rf

sk_tsne

sk_svm_classification

jupyter_job

python_updated

sk_naive_bayes_classification-copy

Scikit-Learn is a powerful library that provides a handful of supervised and unsupervised learning algorithms. If you’re serious about having a career in machine learning, then scikit-learn is a must know.

In this course, you will start by learning the various built-in datasets that scikit-learn offers, such as iris and mnist. You will then learn about feature engineering and more specifically, feature selection, feature extraction, and dimension reduction.

In the latter half of the course, you will dive into linear and logistic regression where you’ll work through a few challenges to test your understanding. Lastly, you will focus on unsupervised learning and deep learning where you’ll get into k-means clustering and neural networks.

By the end of this course, you will have a great new skill to add to your resume, and you’ll be ready to start working on your own projects that will utilize scikit-learn.

Hands-on Machine Learning with Scikit-Learn

# What is Decision Tree

A **Decision Tree** is a non-parametric supervised learning method used for classification and regression. The goal is to create a model that predicts the value of a target variable by learning simple decision rules inferred from the data features. Decision trees are commonly used in operations research, specifically in decision analysis, to help identify a strategy most likely to reach a goal. They are also a popular tool in Machine Learning.

Decision Trees are a very intuitive approach. You are just asking a series of questions about your target, then choose a branch based on your choice. When you split the tree, each node has two branches to split.

Unlike other models, the decision tree is the one with very good interpretability, because you know why you chose the left branch, not the right branch. Therefore, decision trees are widely used in the fields of finance, insurance, and medicine, because these fields want to know the reason for a certain decision when deciding.

# Compare with other models

- Simple to understand and to interpret.
- Can handle the category type data.
- Can handle the missing value.
- The reasoning is fast.
- Easy to be overfitting.
- Performance is not good to compare with some models.
- Some tasks can't be learned.

# Modeling on the Iris

Let's skip the data loading and splitting as you can see the complete example later. Creating a decision tree is very simple, just create a `DecisionTreeClassifier` object from the `tree` module. Then, use the `fit` to train your tree.

```python
from sklearn.tree import DecisionTreeClassifier

tree = DecisionTreeClassifier()
# train_x,train_y is the training data and targets.
tree.fit(train_x,train_y)
```

You can evaluate this model with the confusion matrix like below. `tree` is the model, `text_x` and `test_y` are test data.

```python
import sklearn.metrics as metrics

metrics.plot_confusion_matrix(tree, test_x, test_y)
```

import sklearn.datasets as datasets
from sklearn.tree import DecisionTreeClassifier
import sklearn.tree as sktree
from sklearn.model_selection import train_test_split
import sklearn.metrics as metrics

X, y = datasets.load_iris(return_X_y=True)
print("iris data size is {}".format(X.shape))
print("iris target size is {}".format(y.shape))
print("The first five samples of iris {}".format(X[:5]))

train_x, test_x, train_y, test_y = train_test_split(X,
                                                    y,
                                                    test_size=0.2,
                                                    random_state=42)

tree = DecisionTreeClassifier()
tree.fit(train_x, train_y)
pred_y = tree.predict(test_x)
cr = metrics.classification_report(test_y, pred_y)
print(cr)

- From the `line 7` to `line 12`, the iris dataset is loaded and split into two parts.

- A decision tree object is created from `DecisionTreeClassifier()` at `line 17` and trained at `line 18`.

- This example uses the `classification_report` to evaluate the performance of our model, you can check the output at `line 21`.

As a tree model, this model should look like a `real tree` with branches and leaves. `sklearn` provides a very useful function, `plot_tree`, to plot your tree. Below is a result of a tree (please ignore the actual data, this just shows how this tree looks like).

# How the decision tree splits

As you can see from the output above, a decision tree actually is a binary tree. You start at the root node, choose the branch based on the conditions on the non-leaf nodes, and go all the way down to a leaf node. The label on the leaf node is the classification of the sample. 

How do you decide what condition is in one non-leaf node? What metric do we use to split the tree? In the original decision tree, **Gini impurity** is used to split the tree. `Gini` impurity is a measure of how often a randomly chosen element from the set would be incorrectly labeled if it was randomly labeled according to the distribution of labels in the subset.

If a target is a classification outcome taking on values 0,1,…,K-1, for node $m$, representing a region $R_{m}$ with $N_{M}$ observations, let

$$p_{mk} = \frac{1}{N_{m}} \sum I(y_{i}=k)$$

be the proportion of class k observations in node $m$.

A common measure of impurity is Gini:

$$H(X_{m})=\sum_{k}p_{mk}(1-p_{mk})$$

We recommend that you launch the widget to open the `Jupyter` file below which contains more content and interactive operations.

In this lesson, a non-parametric supervised learning model is introduced, which is Decision Tree.

Preliminaries

Working with Datasets

Feature Engineering

General Concepts

Linear Regression

Logistic Regression

Support Vector Machine

Tree Model and Ensemble Method

Unsupervised Learning

Deep Learning

Others

What's Next

Decision Tree

What is Decision Tree