
Gain insights into Scikit-Learn's datasets, feature engineering, linear/logistic regression, and unsupervised learning. Delve into k-means clustering and neural networks to enhance your machine learning expertise.

mk1.tar.gz

data_preprocessing

dataset

decision_tree_classification

feature_extraction

feature_selection

gradient_boost

kmeans

knn_classification

sk_lr_classification

sk_lr_regression

sk_metrics

sk_missing_value

sk_naive_bayes_classification

sk_nn

sk_parameter_search

sk_pca

sk_pipeline

sk_rf

sk_tsne

sk_svm_classification

jupyter_job

python_updated

sk_naive_bayes_classification-copy

Scikit-Learn is a powerful library that provides a handful of supervised and unsupervised learning algorithms. If you’re serious about having a career in machine learning, then scikit-learn is a must know.

In this course, you will start by learning the various built-in datasets that scikit-learn offers, such as iris and mnist. You will then learn about feature engineering and more specifically, feature selection, feature extraction, and dimension reduction.

In the latter half of the course, you will dive into linear and logistic regression where you’ll work through a few challenges to test your understanding. Lastly, you will focus on unsupervised learning and deep learning where you’ll get into k-means clustering and neural networks.

By the end of this course, you will have a great new skill to add to your resume, and you’ll be ready to start working on your own projects that will utilize scikit-learn.

Hands-on Machine Learning with Scikit-Learn

Fine-tuning parameters is an important part of a Machine Learning project, especially in Deep Learning. Although this course focuses on traditional Machine Learning, a few parameters need to be adjusted and the parameter space is limited. However, some models still require lots of parameters, such as a tree-based model.

In practice, people fine-tune by hand. They try a series of values for one parameter, evaluate models, and then choose the best one. If there are more than two parameters, they repeat the process until all parameters are chosen. The process is time-consuming and tedious and it can be completely automated.

`sklearn` provides some useful functions to help us. In this lesson, we will only focus on the `GridSearchCV`. The principle is similar in other ways.

# Grid Search

The grid search provided by `GridSearchCV` exhaustively generates candidates from a grid of parameter values specified with the `param_grid` parameter. In this example, we work on the tree-based model that has lots of parameters.

Let's skip the data loading and splitting part, we'll jump to model building and parameter searching.

Now, let’s create the estimator we want to fine-tune. In this example, we want to create a classifier that is a `GBDT`. Then, we define a map that is `param_grid`. Then, we create a `GridSearchCV` object and `fit` it. It could take 30 seconds to get results; to accelerate the training, we set the `n_jobs=4` which enables multiple processors. From the `param_grid` above, you can figure out the number of combinations is $6 * 5 * 4 * 3 = 360$. This means that the `GridSearchCV` would train 360 models and evaluate each of them. Hence, `GridSearchCV` is a very inefficient approach. Of course, if you have a small amount of data, it can save a lot of repetitive work and time.

```python
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.model_selection import GridSearchCV

gb = GradientBoostingClassifier(random_state=10)
param_grid = [{"n_estimators": [1, 2, 4, 16, 32, 64], "learning_rate": [0.05, 0.1, 0.2, 0.4],
               "min_samples_leaf": [1, 2, 4, 8, 16], "max_depth": [1, 2, 3]}]
cv = GridSearchCV(gb, param_grid=param_grid, scoring="f1", n_jobs=4)
cv.fit(train_x, train_y)
```

The `GridSearchCV` object does have some useful attributes. According to the `best_parameter_` attribute, you can get the best parameter. Moreover, you could get the evaluation metric for each set of parameters from `cv_results_`. 

```python
print("Best parameters set found on development set:")
print(cv.best_params_)
print("Grid scores on development set:")
means = cv.cv_results_['mean_test_score']
stds = cv.cv_results_['std_test_score']
for mean, std, params in zip(means, stds, cv.cv_results_['params']):
    print("%0.3f (+/-%0.03f) for %r" % (mean, std * 2, params))
```

Here is the complete code so you can have a try.

In this lesson, learn how to do parameter searching in sklearn.

Parameter Searching

Preliminaries

Working with Datasets

Feature Engineering

General Concepts

Linear Regression

Logistic Regression

Support Vector Machine

Tree Model and Ensemble Method

Unsupervised Learning

Deep Learning

Others

What's Next

Parameter Searching