Exercise: Incremental Load
Practice using the COPY command to incrementally load data into a PostgreSQL database.
In this exercise, we’ll practice using the COPY command to incrementally load data. In incremental loading, we only load the recent changes in the source into the destination repository. As always, the logic by which we do it depends on the business requirement and context.
Example
As data engineers working for a retail company, we’re tasked with creating an ETL pipeline for loading customers’ log activity data from a transactional database into a PostgreSQL database for analysis.
According to the business requirements, new batches of data should be loaded ...