Delve into effective data manipulation with pandas. Discover filtering, merging, grouping, and reshaping techniques through hands-on examples, best practices, and performance tips for real-world analysis.

pandas.tar.gz

pandas

jupyter

jupyter-display

Python

pandas-with-sqlite3

jupyter-live

The course is a comprehensive guide to using pandas for data analysis. It covers a wide range of topics related to data manipulation, including filtering, merging, grouping, pivoting, and reshaping data.

In this course, you’ll learn about the best practices and efficient techniques for working with data. You’ll be provided with numerous examples of real-world data analysis problems and gain hands-on experience with the pandas library to solve them effectively. This course will also cover performance considerations and provide insights on common pitfalls to avoid when using pandas.

The course is suitable for both beginners and experienced users of pandas. It includes detailed explanations of pandas concepts and functions, as well as tips and tricks for optimizing your code. By the time you’re done with this course, you’ll be able to effectively handle data using pandas.

Effective Data Manipulation with pandas

Please be aware that we can pass in a boolean array and a function into `loc`. Here, we select rows with `Average_rank` less than ten and the first three columns:

print(pres
 .loc[pres.Average_rank < 10, lambda df_: df_.columns[:3]]
)

An advantage of passing a function into `loc` is that the function will receive the current state of the DataFrame. If we have `loc` in a chain of operations, the column names or rows may change so that if we filter based on the original DataFrame that began the chain, we might not be able to get the data we need.

# `query` vs `loc`

There is often more than one way to do things in pandas. You may wonder if we should use `query` or `loc`.
If we do a lot of chaining (which is recommended), `query` has the advantage of working on the intermediate DataFrame. One could argue that `loc` does as well, but often when using boolean arrays with `loc`, users insert a boolean array based on the original data, not the intermediate data. We need to use a function with `loc` to get access to the original DataFrame.

On the flip side, `query` does not support column selection, but `loc` does. There is no situation where we should only learn one of these constructs and neglect the other. Learn them both and figure out which one is appropriate given the requirements.

# Summary
We explored filtering with functions and `loc`. We also compared `loc` and `query`. 

See filtering with function and .loc in DataFrames.

Introduction

Series Deep Dive

DataFrames

Manipulating Data

Wrapping Up

Appendix

Filtering with Function and .loc