Re: Data Purging Strategy

Hello Jay

How about building an historical DB and keeping the data there. It will not overload the production instance, will be available online if you need something, you will migrate it to new versions of Oracle so compatibility will not be an issue and you can implement table changes on the historical data so the structure will remain the same as in production.

We are doing it in ADABAS on the mainframe.

Yechiel Adar
Mehish
----- Original Message -----
To: Multiple recipients of list ORACLE-L <ORACLE-L_at_fatcity.com> Sent: Thursday, November 07, 2002 11:24 PM

> Well, if worst comes to worst we can always install an earlier version on
a
> box and import it there.
> But the reason we can't get more storage approved still has me shaking my
> head...
>
> -----Original Message-----
> Sent: Thursday, November 07, 2002 2:19 PM
> To: Multiple recipients of list ORACLE-L
>
>
> Jay,
>
> just make sure you are not around when, after several Oracle upgrades, and
> they want to "import" one of these files back that they discover that the
> current release of import can no longer read the older version of the .dmp
> file.
>
> now what are these senior damagers going to do? blame the DBA, that's
what!
>
>
> duck and cover... duck and cover...
>
> Tom Mercadante
> Oracle Certified Professional
>
>
> -----Original Message-----
> Sent: Thursday, November 07, 2002 1:55 PM
> To: Multiple recipients of list ORACLE-L
>
>
> FWIW, what we just implemented (because senior management refuses to
approve
> additional storage on the grounds that "making the database larger will
> affect performance" - aaargh!) is
>
> 1) Confirmed with business how long data needs to be online for various
> tables (they're all partitioned so that makes it a lot easier)
> 2) Export partitions older than that once/month (this is generated off a
> table that lists each partitioned table and how long data should be kep)
> 3) After confirming that all export files are valid we drop the old
> partitions (this will be done by script but is being done manually for the
> first few months)
> 4) Leave dmp files on server for 2 end of months (our end of month backup
> tapes are stored for 7 years)
> 5) Maintain a table in database saying what exported partitions are on
what
> date's tapes
>
>
> And I really long for the days in this company when senior management made
> technical decisions by asking the technical people instead of just making
> things up...
>
> Jay Miller
>
>
> -----Original Message-----
> Sent: Wednesday, November 06, 2002 11:54 AM
> To: Multiple recipients of list ORACLE-L
>
>
> Someone asked about this 3 weeks ago. Here's my take
> on archiving data. I don't expect everyone to agree with this,
> but nonetheless, I have an opinion. :)
>
> Here's an email from last month. You can undoubtedly find
> some other ideas on this by searching the archives of this
> list at fatcity.com
>
> Jared
>
> ==================================================
>
> I'm not a proponent of purging data.
>
> Unless of course, you expect to never see it again.
>
> That word 'archive' rolls of the tongues of managers
> and consultants pretty easily, but what's behind it?
>
> There are a few gotchas with purging and archiving.
>
> Let's assume you have some 3 year old data that
> you need to see again, and it has been purged.
>
> Here are some of the possible problems:
>
> * Your backup tapes are corrupted
> * Your new backup hardware can't read the old tapes
> * Your software no longer understands the format that
> the data is in.
> * You have the correct software, but it won't work on the
> current version of OS on your hardware.
> * The data format/software/whatever is not well documented
> * The employees that understood the data 3 years ago
> have been laid off.
> * ... lots more stuff
>
> Read Bryon Bergeron's "Dark Ages II: When the Digital Data Die"
> http://www.powells.com/cgi-bin/biblio?inkey=2-0130661074-0
>
> Perhaps much better than archiving the data, is to stick with the
> idea of moving it to another database, and using lots of cheap
> disk storage (NAS) or a heirarchical file system to store it.
>
> The point being that if it's online somewhere, it will be maintained.
>
> Don't purge it till Finance, HR, the IRS and any other stakeholder
> says it's ok. Only then purge it and archive it to offline tape with the
> knowledge that you may never see that data again.
>
> Jared
>
>
>
>
>
> prem_at_ibsplc.com
> Sent by: root_at_fatcity.com
> 11/06/2002 01:13 AM
> Please respond to ORACLE-L
>
>
> To: Multiple recipients of list ORACLE-L
<ORACLE-L_at_fatcity.com>
> cc:
> Subject: Data Purging Strategy
>
>
>
> Dear List,
>
> I need some inputs from you all regarding purging data from the database.
>
> This is the requirement
>
>
> We define a retention period for all the data in the system.
> When the retention period is reached, the data should be deleted, but
> then at a later time, some user might request for this purged data. So it
> must be possible to retrieve this data.
>
> This is the strategy we have designed for this.
>
> When the retention period is reached, move the data from the main database
> to an offline database. Then delete the data from the main database.
>
> In the offline database, we cannot again keep it from long, so it has to
> moved to tapes. Now my question, how can we move this data to tapes and at
> the same time retrieve data from the tapes based on dates.
> i.e, the user will ask for the data on a particular date, so it must be
> possible to retrieve data from the tapes based on a date and load it to
> the database tables.
>
> Regards
> Prem
>
>
>
>
> --
> Please see the official ORACLE-L FAQ: http://www.orafaq.com
> --
> Author:
> INET: Jared.Still_at_radisys.com
>
> Fat City Network Services -- 858-538-5051 http://www.fatcity.com
> San Diego, California -- Mailing list and web hosting services
> ---------------------------------------------------------------------
> To REMOVE yourself from this mailing list, send an E-Mail message
> to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in
> the message BODY, include a line containing: UNSUB ORACLE-L
> (or the name of mailing list you want to be removed from). You may
> also send the HELP command for other information (like subscribing).
> --
> Please see the official ORACLE-L FAQ: http://www.orafaq.com
> --
> Author: Miller, Jay
> INET: JayMiller_at_TDWaterhouse.com
>
> Fat City Network Services -- 858-538-5051 http://www.fatcity.com
> San Diego, California -- Mailing list and web hosting services
> ---------------------------------------------------------------------
> To REMOVE yourself from this mailing list, send an E-Mail message
> to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in
> the message BODY, include a line containing: UNSUB ORACLE-L
> (or the name of mailing list you want to be removed from). You may
> also send the HELP command for other information (like subscribing).
> --
> Please see the official ORACLE-L FAQ: http://www.orafaq.com
> --
> Author: Mercadante, Thomas F
> INET: NDATFM_at_labor.state.ny.us
>
> Fat City Network Services -- 858-538-5051 http://www.fatcity.com
> San Diego, California -- Mailing list and web hosting services
> ---------------------------------------------------------------------
> To REMOVE yourself from this mailing list, send an E-Mail message
> to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in
> the message BODY, include a line containing: UNSUB ORACLE-L
> (or the name of mailing list you want to be removed from). You may
> also send the HELP command for other information (like subscribing).
> --
> Please see the official ORACLE-L FAQ: http://www.orafaq.com
> --
> Author: Miller, Jay
> INET: JayMiller_at_TDWaterhouse.com
>
> Fat City Network Services -- 858-538-5051 http://www.fatcity.com
> San Diego, California -- Mailing list and web hosting services
> ---------------------------------------------------------------------
> To REMOVE yourself from this mailing list, send an E-Mail message
> to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in
> the message BODY, include a line containing: UNSUB ORACLE-L
> (or the name of mailing list you want to be removed from). You may
> also send the HELP command for other information (like subscribing).

-- 
Please see the official ORACLE-L FAQ: http://www.orafaq.com
-- 
Author: Yechiel Adar
  INET: adar76_at_inter.net.il

Fat City Network Services    -- 858-538-5051 http://www.fatcity.com
San Diego, California        -- Mailing list and web hosting services
---------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from).  You may
also send the HELP command for other information (like subscribing).