Ideas for "make" for data replication

We are working on a data warehous project which basically involves replicating data from a production OLTP system to a DSS system. It is a given fact that data model on both systems are slightly different, but most tables could be replicated from OLTP to DSS using views respectively snapshots. Data on DSS are refreshed periodically, basically on a dayly cycle. However there will be a time dependency for the replication. E.g. the FACTS table on DSS must be refreshed before the SUMMARIES table can be refreshed, TRENDS should only be refreshed after SUMMARIES has been refreshed. Note that SUMMARIES is based on FACTS and TRENDS is based on SUMMARIES.

Of course this synchronization can be managed by scheduling the refresh accordingly. But the 3 tables example here is greatly simplified. The real data model is far more complex and makes the scheduling a tough job. Specially when one of the tables failed to refresh, all the other tables depending on it will be out of sync. To manually resync them is apparently a nightmare.

Therefore it strikes me as natural to have something simular to the "make" utiliy know to Unix developers: one just define a hierarchical tree of dependencies, and with a "make all" the whole build process would be triggered in the right order.

I am looking for ideas on how this could be implemented. Of course it would be even better if there already exists some tool that does exactly that.

Thanks for any hints, pointers. Received on Mon Aug 28 2000 - 04:50:07 CDT