Help required to reload data in star schema

View previous topic View next topic Go down

Help required to reload data in star schema

Post  gokul_ifs on Fri Jul 10, 2015 1:58 am

Hi,

I am new to data warehouse technologies. Looking for exports help to understand how to load data for the scenario described below.

In my DW system, I will be getting files on daily basis which contains sales information, and I will be loading these files to star schema.

Letís says, I have received to files on in the past 10 days and loaded all to files to the data warehouse. I am performing SCD type 3 load with flag variable.

Now, on Day 11, I have to undo all the changes done by loading data on day 6 and reload the data with a new file( got revised file for day 6).

Please can you help me to understand how should I approach this scenario

Thanks,
Gokul

gokul_ifs

Posts : 1
Join date : 2015-07-10

View user profile

Back to top Go down

Re: Help required to reload data in star schema

Post  nick_white on Mon Jul 13, 2015 7:44 am

Two options, both very similar...

1. Roll back all changes made from the load being changed and everything subsequent to it. Process the updated file and then all the subsequent files.
Simple to implement but obviously a lot of data to process.

2. Rollback all changes related to amended records in the new version of the file. Reload the amendments and all records in the subsequent files that relate to the amended records.
More complicated to implement but may be the only solution if you have a tight load window and only a small number of records in the amended file have changed.

Alternatively, you may be able to treat the changes as genuine changes rather than corrections of errors. You'd need to work with your users to determine the correct logic for processing these changes and then implement it in your ETL

nick_white

Posts : 366
Join date : 2014-01-06
Location : London

View user profile

Back to top Go down

View previous topic View next topic Back to top

- Similar topics

 
Permissions in this forum:
You cannot reply to topics in this forum