Challenge: Create a Data Management System

Problem statement

In the bustling city of Techville, an innovative startup named DataWizards has emerged as a prominent player in the field of data analysis and visualization. As their operations expand, they are faced with the challenge of efficiently handling and analyzing large amounts of data stored in various file formats. Your task is to develop a program that tackles this challenge by implementing various string manipulation techniques, utilizing regular expressions, managing file system paths, and performing object serialization.

Tasks

Understand the context and requirements of the problem before proceeding with the implementation.

Process data files

• DataWizards has acquired multiple data files containing textual data in CSV format.
• Develop a program to read these data files and extract relevant information using string manipulation techniques.
• Implement the custom formatter to ensure the extracted data is correctly formatted and organized.

Data validation

• Use regular expressions to identify and correct patterns in the data that do not adhere to predefined rules.
• Group and categorize data based on specific patterns using regular expression grouping.

Generate reports

• Utilize string formatting and f-strings to create comprehensive reports.
• Design a user-friendly interface that displays the generated reports.

Managing file system paths

• Develop a module to handle file system paths, ensuring seamless navigation through different directories.
• Implement functions to locate and retrieve required data files based on user input.

Utilizing JSON for serialization

• Adapt the existing code to serialize and deserialize data objects using JSON.
• Evaluate the benefits and drawbacks of using JSON over custom pickling for DataWizards’ specific needs.

Note: You may practice writing your code in the playground below before moving to its solution lesson.