...
In the context of ETL, a delta load (also known as a incremental load) refers to loading/refreshing only the changes or updates that have occurred since the last load. Also known as incremental loads. This is in contrast to an initial load, which involves loading the entire dataset from the source. The difference in performance between delta load and initial load depends on factors such as data volume, structure, target, and load type.
An incremental/delta load is the selective movement of data from one system to another. An incremental load pattern will attempt to identify the data that was created or modified since the last time the load process ran. This differs from the conventional full data load, which copies the entire set of data from a given source. The selectivity of the incremental design usually reduces the system overhead required for the ETL process.
Initial Loads vs Incremental/Delta Loads.
Initial loads (Load All in ETL+) loads the entire source table from the source into the target data warehouse table. The difference in performance between delta load and initial load depends on factors such as data volume, structure, target, and load type.
WaterMarks
A watermark is a column that has the last updated time stamp or an incrementing key. The delta loading solution loads the changed data between an old watermark and a new watermark.
...