RE: Oracle configuration on SAN

Yeah, write-back cache is better than write-through for performance. Mirrored cache is nice too, except that some implementations (and I have zero idea whether this applies to HP or IBM) used to do silly things like dispatch mirrored writes in serial rather than in parallel, and don't load-balance reads on mirrored objects.

On the FastT900, are you sure the battery backup actually destages to disk upon power failure? If I remember correctly, the FastT line deals with power failure the same way most mid-range storage arrays do - the system loses power and a battery supplies power to the cache to keep it persistent. When the array is brought back online, before servicing any new I/O, the cache is destaged to the disk. The only problem with that is extended power failures or hardware problems can create data loss (though generally you can move the cache from the failed box to the new box without losing data).

The other possible way they do it is the same way the Clariion does it, which is the first drive shelf has a glorified UPS attached to it, and when power goes, the cache is destaged to a "vault" drive, which is then spun down.

Bigger arrays, like the HDS 9900 series, the Symmetrix, and the Shark all have big honking batteries in them (and I mean big - over a hundred pounds as I recall). When power is lost, they gracefully destage all cache to disk, carefully park the drive heads, spin the drives down, and then shut down. Much nicer than power suddenly being cut to drives spinning at 10 or 15 thousand rpms.

Matt

--
Matthew Zito
GridApp Systems
Email: mzito_at_gridapp.com
Cell: 646-220-3551
Phone: 212-358-8211 x 359
http://www.gridapp.com



> -----Original Message-----

> From: ml-errors_at_fatcity.com [mailto:ml-errors_at_fatcity.com] On 

> Behalf Of Jesse, Rich

> Sent: Monday, July 21, 2003 3:49 PM

> To: Multiple recipients of list ORACLE-L

> Subject: RE: Oracle configuration on SAN

> 

> 

> In the same cache vein, make sure you know what type of 

> caching is being done at each LUN (or how ever the HP's 

> setup).  Write-thru caching won't help your write speed at 

> all, while write-back will.  The trade off is that since 

> write-back acknowledges the write from the cache (write-thru 

> won't acknowledge the write until it physically hits the much 

> slower disk), there is a small chance that it may not be 

> flushed to disk in the event of failure (e.g. power).

> 

> I don't know about HP's offering, but the FastT900 from IBM 

> allows you to mirror your cache, plus they have a battery 

> backup for them to flush the cache to disk in the case of 

> power failure.  So, for us, I imagine we'll head for the 

> mirrored write-back cache option w/BBU.  I think it's an 

> acceptable risk for the gains for our particular situation.

> 

> Rich

> 

> Rich Jesse                           System/Database Administrator

> rjesse_at_qtiworld.com                  Quad/Tech Inc, Sussex, WI USA

> 

> > -----Original Message-----

> > From: Matthew Zito [mailto:mzito_at_gridapp.com]

> > Sent: Monday, July 21, 2003 3:04 PM

> > To: Multiple recipients of list ORACLE-L

> > Subject: RE: Oracle configuration on SAN

> > 

> > 

> > 

> > A 14-drive RAID-5 set is very large.  It's certainly

> > functional, but the

> > two problems you'll run into is problems with spindle contention and

> > rebuild times.  With a 14-drive set, your drive is getting 

> cut into 14

> > columns, so that there's 14 different disk regions per 

> drive it might

> > have to seek to in order to service any given I/O.  That can 

> > negatively

> > impact performance on random writes. Have you tested failing 

> > out a drive

> > under load?  On a 14-drive set the rebuild time is going to 

> be pretty

> > horrendous, and your performance will likely be impacted unless your

> > cache hit numbers are really great.

> > 

> > The other problem is that by carving luns globally out of a single 

> > RAID-5 set, differing i/o patterns on the luns can create hot spots 

> > much more easily, since your small (comparatively, anyway) redo log 

> > volume (for exmaple) ends up on only four columns of the disks, and 

> > other volumes on other columns on those disks can be hurt by the 

> > constant writing.

> > 

> > While I'm not necessarily as anti-RAID 5 as some (though I

> > give all due

> > respect and worship to our mighty BAARF leaders), you need to keep a

> > very close eye on your array in this configuration.  If you have a

> > normal OLTP workload (whatever "normal" is), play with your cache

> > allocations - the read v. write cache, and if you can do per-lun

> > tweaking, weight the redo and archive log lun(s) very 

> heavily towards

> > write cache. 

> > 

> > If you're set on RAID-5, I would recommend taking two of 

> the disks and 

> > making them a mirrored pair for redo and archive logs.  Since the 

> > writes tend to be reasonably contiguous, the fact you're 

> hitting just 

> > one set of spindles shouldn't hurt quite as bad, and cache should

> > take the edge

> > off a bit.  

> > 

> > This all being said, my knowledge of that particular HP array

> > is limited

> > at best, so I can't offer vendor-specific 

> > recommendations/thoughts that

> > might invalidate some of these concerns.  Good luck.

> > 

> > Thanks,

> > Matt

> > 

> > --

> > Matthew Zito

> -- 

> Please see the official ORACLE-L FAQ: http://www.orafaq.net

> -- 

> Author: Jesse, Rich

>   INET: Rich.Jesse_at_qtiworld.com

> 

> Fat City Network Services    -- 858-538-5051 http://www.fatcity.com

> San Diego, California        -- Mailing list and web hosting services

> ---------------------------------------------------------------------

> To REMOVE yourself from this mailing list, send an E-Mail message

> to: ListGuru_at_fatcity.com (note EXACT spelling of 'ListGuru') 

> and in the message BODY, include a line containing: UNSUB 

> ORACLE-L (or the name of mailing list you want to be removed