Expanding Data Manipulation and Preparation Skills

Data scientists spend the majority of their time cleaning data, reformatting it, and reshaping it. We have seen this in action!

We saw several times how much code and mental effort, and importantly, domain knowledge goes into just getting our data into a certain format. Once we have our data in a standardized format, for example, a long-form (tidy) DataFrame, then our lives become easier.

We might want to learn more pandas and NumPy for a more complete set of techniques that help us reshape our data however we want.

• As mentioned at the beginning of the section, learning new pandas techniques without a practical purpose in mind can help us expand our imagination.
• Learning regular expressions can help a lot with text analysis because text is typically unstructured, and finding and extracting certain patterns can help a lot in our data cleaning process.
• Statistical and numeric techniques can definitely make a big difference. At the end of the day, we’re basically crunching numbers here.

Improving data manipulation skills naturally leads us to easier and better visualization.

Exploring more data visualization techniques

We saw how easy it is to work with Plotly Express and how powerful it can be. We also saw the extensive options available to us. At the same time, we’re constrained by the requirement to have our data in a certain format, which Plotly Express can’t help with. This is where we have to step in as data scientists.