Re: AWS S3 Backups

Hi all,

> or can I just mount the bucket as a file system using fuse

No one is talking about using fuse, so I'll try. I don't think it's going to be possible to use fuse for this purpose. End of the day S3 is object storage, and not a file system. You can upload a file (object), you can replace a file and you can delete a file from it, but you can't change an existing file. Changing an existing object actually replaces the whole object (reads it, modifies it, and upload the new version)
My understanding is that fuse is just a layer on top of S3, that presents it to you as a file system, but it's still limited by the same API calls and behavior of S3 underneath.
I shouldn't be much wrong to assume that RMAN will write the backup pieces using 1 MB writes (maybe it's a different value, but it should still be relatively small), so you'll get a lot of tiny changes to the backup piece on S3 (via fuse), and each of them will undergo the read/modify/upload process. It will be slow and inefficient.

OSB has a parameter - OSB_WS_CHUNK_SIZE, that "*Specifies the object size, in bytes, that will used when storing backups to Amazon S3. The default size is 100MB.*". I assume that OSB caches the chunks locally, and then uploads the whole piece in a single operation to S3. This is why it can perform much better.

P.S Needless to say, I haven't used any of these approaches :D

---
Maris Elsins




On Thu, Feb 6, 2020 at 11:44 PM Mark J. Bobak <mark_at_bobak.net> wrote:



> There are a few ways to accomplish this.  If you want to backup to S3 and


> want all the features of RMAN, you should use the OSB module to write

> backups directly to S3.

> Advantages here include:

> - all RMAN features, including:

>   - block change tracking

>   - block recovery

>   - compression

>   - multiple channels (if on EE)

>   - and monay others

> Disadvantage:

>  - OSB is separately licenseable, on a per channel basis (if you run

> parallel, it can get expensive)

>

> You could mount S3 via file storage gateway.  In this case, you can backup

> using RMAN, and don't need OSB.  I've experimented with this, and it works,

> but, it seems some versions of Oracle don't like the NFS mount as a target

> for the RMAN backup.  Also, danger here is that your backups are on a

> mounted, writable filesystem.  There is a danger of accidentally deleting

> your backups on the S3 bucket.

>

> You could do AMI backup of the entire server.  This can be done w/ DB in

> archivelog mode, and put DB in backup mode, then take AMI snapshot and

> bring DB out of backup mode, or without putting DB in backup mode

> (archivelog mode not required) and just snapshot everything.  Note that

> behind the scenes, snapshots are stored on S3, though you cannot see them

> in the S3 console, only in the snapshots console.

>

> Differences:

> w/ backup mode, you are taking a 'recoverable' backup.  This is

> essentially the same as a traditional non-RMAN backup.  You can (must)

> apply archive logs and recover db to open it.

>

> w/o backup mode, you are taking a 'restartable' backup.  This is where the

> DB will do instance recovery, and restart automatically, but it will *only*

> have data up to the point in time the AMI was taken.  Consider *carefully*

> the recovery implications and data loss implications of this decision.

>

> Currently, I'm running RMAN w/ OSB and backing up to a dedicated S3 bucket

> that way.

>

> I'm working on a set of scripts where I can mount S3 through file gateway,

> write archive logs directly to S3, and implement script to put DB in backup

> mode, take AMI snapshot, bring DB out of backup mode.  Sadly, this is a

> work in progress, and I never seem to have time to finish it.

>

> Hope that helps,

>

> -Mark

>

> On Thu, Feb 6, 2020 at 1:50 PM Mladen Gogala <gogala.mladen_at_gmail.com>

> wrote:

>

>> Hi Charlotte!

>>

>> You can backup from S3 to another S3 LUN using normal device type disk

>> channels. Just mount the LUN, create a file system for your OS and plough

>> away. In addition to that, I stand corrected: OSB apparently works with AWS

>> and can be used for storing stuff to glacier. S3 can do even fancier stuff,

>> with snapshots. You can put your DB in the backup mode, snap the database

>> LUN(s) take the database out of the backup mode and backup the database

>> files on the LUN snapshot using OS backup like tar or cpio.

>>

>> Regards

>> On 2/6/20 12:52 PM, Charlotte Hammond wrote:

>>

>> Hi Mladen,

>>

>> OSB is intended to work with AWS:

>> https://docs.oracle.com/database/121/RCMRF/web_services.htm#RCMRF90489

>>

>> That document gives advantages over using tape but not over just mounting

>> the S3 bucket as a file system.   S3 is our target rather than glacier.

>>

>> Thanks,

>> Charlotte

>> On Thursday, February 6, 2020, 05:29:41 PM GMT, Mladen Gogala

>> <gogala.mladen_at_gmail.com> <gogala.mladen_at_gmail.com> wrote:

>>

>>

>> IMHO, OSB Cloud Module is specific for Oracle Cloud and will not help

>> with AWS. The problem is in the letter "A" which doesn't get along with the

>> big red "*O*". What is your backup destination? Another S3 LUN? Local

>> storage? Glacier?

>>

>> You can do rman to another S3 LUN by allocating disk channels (no device

>> type SBT). For Glacier, you need some kind of an enterprise backup utility

>> which can write to Glacier (Commvault, Rubrik, Veeam, Cohesity) and the

>> configuration is product specific. Oracle Secure Backup is just another

>> enterprise backup tool, much like DD Boost, TSM, NetBackup or the

>> previously mentioned utilities.

>>

>>

>> On 2/6/20 11:34 AM, Charlotte Hammond (Redacted sender

>> charlottejanehammond for DMARC) wrote:

>>

>> Hello All,

>>

>> AWS backups question: if I want to backup by database to an S3 bucket

>> using RMAN do I need to use the OSB Cloud Module or can I just mount the

>> bucket as a file system using fuse?   If the latter is possible, what would

>> the benefits of using OSB be?

>>

>> Many thanks,

>> Charlotte

>>

>> --

>> Mladen Gogala

>> Database Consultant

>> Tel: (347) 321-1217

>>

>> --

>> Mladen Gogala

>> Database Consultant

>> Tel: (347) 321-1217

>>

>>


--
http://www.freelists.org/webpage/oracle-l