Message-Id: <10701.123666@fatcity.com> From: bruce.taneja@mcd.com Date: Tue, 5 Dec 2000 09:59:46 -0600 Subject: Re: Raw Vs. File systems, your opinion? Hi guys, thanks for all your responses.. However while you guys were writing, I was digging through the web over= other sites.. Here is a compilation of other "useful" messages I found.. -Bruce Taneja =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D HAPPY READING =3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Our database is about 8G and only about 3 tablespaces are updated/inser= ted > frequently. The DB is running on a shared RS/6000 which uses RAID5 fo= r most > of the filesystems. I heard that RAID5 is not suitable for redo logs = plus > write intensive tablespaces. > > Is it possible to have some datafiles resided on cooked and some othe= rs, say > online/archived logs on raw devices ?? > > TIA > Kevin Yes you can.... In fact in high activity databases, very good performance gains can be = made simply by making the log files raw...After all, you virtually never cha= nge their size, never back them - so there is nil impact in having them raw= . ?---- I'm not a sysop, so here goes nothing: RS/6000's and AIX have a pretty good LVM, that takes away a lot of "all the trouble" you a re referring to. At least, so I'm told= (not very long ago). But it may be worth to see what a "normal, multi-spindle, mirrored logi= cal volume" does. Mirroring takes forever, BTW, on SSA controllers, but I'm= in no way in favor of removing mirrors for the sake of performance. Frank -- As you already know, you should put your redo logs and other write intensive datafiles on non-RAID 5 volumes. But you do not need to go through the trouble of using raw file systems. Just use a non-RAID file= system to handle this for you. ----- I am looking at using Oracle Parallel server - it seams that it requires RAW file system to be setup. I have not had any experience wit= h this. Does anybody have any experience using raw devices. What are the advantages ?? What are the disadvantages ?? What about backup ?? Disk failure normall= y protected by RAID 1 or RAID 5 Keep in mind this would be on a NT platform... -=3D-=3D-=3D-- In article <8q81nq$36m$1@newsg3.svr.pol.co.uk>, "Dave Leather" wrote: > Thanks Mark, > I didn't intend to use autoextend, it was just a query. But the question > about control files still stands. How large should the control file device > be given that they can grow ( to quite a size, the ref says 20000 oracle > blocks ). > > Dave Leather > > "Mark D Powell" wrote in message > news:8q7qvi$gpe$1@nnrp1.deja.com... > > In article <8q7l6q$vps$1@newsg1.svr.pol.co.uk>, > > "Dave Leather" wrote: > > > All, > > > > > > When you implement raw devices ( as you must for OPS),how does th= e > > > autoextend feature happen, is is disabled for raw dev's . Also > > control files > > > grow, what should be allocated to manage the growth , or is the growth > > > different in OPS ? > > > ( if you have any good references for OPS could you let me have them ) > > > > > > Thanks in advance > > > > > > Dave Leather > > > > > > > > I would not attempt to use auto-extend with OPS and raw partitions. We > > run OPS on raw partitions and have never tried auto-extend. Since raw > > partitions are fixed in size at creation it makes no sense to allocate > > less than the entire partition. I would think that it would work as= > > long as the rdbms did not try to extend past the partition boundries, > > but I would hope that Oracle is smart enough not to allow auto- extend > > on raw partitions. Post if you find out it works. > > -- > > Mark D. Powell -- The only advice that counts is the advice that > > you follow so follow your own advice -- > > Well, if you chosen Oracle Block size is 8K then 20,000 blocks would be= about 160M if my mental math is correct. Plus you should leave room for= future growth needed by the next release. The more I think about it the more it seems that I have read something = on how to size a control file. I though maybe I could use v$controlfile_record_section to get an idea, but it is way short. DDC1> select sum(record_size * records_total) 2 from v$controlfile_record_section 3 / SUM(RECORD_SIZE*RECORDS_TOTAL) ------------------------------ 1178830 If you know how many v$log_history entries you plan to keep, plus how m= any database files you plan to allow for, the number of redo logs members, = and the space per record they require you should be able to make a decent guess. But again I seem to be coming up way short as 1024 files X 255 b= ytes per file name is only 255K. Even doubling it for the history and adding= overhead of 255 bytes per row we are only at a megabyte. You also have to account for rman if you plan on using it. There is an init.ora parameter to control how many days/runs of information it reta= ins in the control file that can help limit it consumption. This is a really good question: how do you estimate the control file si= ze? -=3D-=3D-=3D-=3D In article <8q55hf$c1c$1@nnrp1.deja.com>, new_dba@my-deja.com wrote: > Hi, > We have IBM RS 6000 machine running on AIX 4.3. The data files of our= > production database are on unix file system. I want to know whether i= t > was right to create database on unix file system. Is it possible to > create Oracle database on raw device (starting with "c") or block > device (starting with "b") ? Could somebody suggest which is better > and why ? On AIX as well as HP-UX. > > I have worked earlier on Sybase and Sybase always recommended to use > block device. But it looks different for Oracle. Pls advise. Thanks. > > Sent via Deja.com http://www.deja.com/ > Before you buy. > We have Sybase and Oracle both on AIX. Her eis the rundown as I understand it. - Sybase recommends against JFS mostly due to the write caching. Not as= big a problem with Oracle, but we are still looking to see if there is a wa= y to disable the write cache. - Raw is a bit faster. On the order of a few percent. In itself, ususally not enought to bother with unless your DB is a VL VLDB= . - JFS adds some memory management overhead due to the way file memory v= s working memory is handled. You need to adjust maxperm and minperm to he= lp, but it's still a bit of an issue IMO. We do run all our AIX Sybase Raw and Oracle JFS. The choice is up to you; Oracle will work well with either block or charater(raw) devices. I have worked with both. Apparently Sybase is optimized for block devices (most likely due to the fixed 2K block size= ). The theory is that Oracle will work better with raw devices simply beca= use the block size of the reads and writes is the same as the database bloc= k size -- there is no OS-defined constraint. With block devices the datab= ase block size is a multiple of the OS block size (512 bytes, 1K, 2K, etc.)= ; the raw devices eliminate the translation from one block size to anothe= r. Oracle can be tuned to operate well under either configuration. As I st= ated before, the choice is up to you. vasarpota@my-deja.com wrote: > > A few questions on the use of raw partitions. > > Do you guys use them? I know they can give you a performance boost if= > I/O is the bottleneck of the database and that data integrity is bett= er > if using them since always know that Oracle writes going to disk and > not in OS cache but do you guys use them. I see that Oracle does not > reccomend that you use them. Are raw devices very difficult to manage= > (e.g. backups, sizing/planning, etc.)? Are they worth the trouble? > > Do you guys use cooked files to overcome the data integrity issue > instead of raw partitions? > > Thanks for your help > > V- > > Sent via Deja.com http://www.deja.com/ > Before you buy. Ideally I'd go raw on everything but there is a little more management involved... You can possibly work through a compromise which should give you good improvements over a file system based database. 1) Make your redo logs raw - you're not meant to back them up anymore anyway (assuming you're in archivelog mode) 2) Mount your file systems for direct IO. 3) If you are doing a lot of sorting, then make your temp tspace raw or= at least in several (not one) file. Similarly, with tempfiles nowadays you= don't need to back them up either. Then you are "half way" toward a raw system with much of the manageabil= ity still retained. HTH -- The issue is, as you say, one of I/O. Raws come into their own for Onli= ne Redo Logs. If you mirror your redo logs (and you'd be mental not to) th= en poor ol' LGWR has to mnake the same entries in both (or all) members of= each group -which could result in slower performance for the entire database unless you can somehow get the writes to the separate members = to happen truly in parallel. Most file systems do not support true asynchronous i/o, and accordingly= it is quite often recommended that online redo logs should reside on raw devices (the lack of a file system means that true asynchronous i/o is almost a given). However, they are actively discouraged for archived redo logs (there's actually an explicit 'prohibition' against putting archives on a raw de= vice in the Oracle training materials). And they are also not much use for t= he data files. And, I gather, most unix variants these days do a reasonable job of simulating asynchronous i/o (though not being a unix guru, I couldn't v= ouch for this for sure) -so inside Oracle these days its always a bit iffy whether to tall people to use raw devices even for online redo. The problem as I understand it is that without a file system, you can't= very easily manage your logs -there's nothing handy like 'mv' or 'rm' o= r 'cp' -you have to use 'dd' all the time, and I am constantly told by students on my courses that managing raw devices is (comparatively speaking) a pain in the butt. I'd avoid 'em like the plague for anything other than redo logs, and I'= d only bother using them for the logs if you can demonstrate that multiplexing your logs has caused significant performance degradation. I came from Informix DS and DB/2 Common Server dependent application. T= hose engines recommends cooked files only for home work and tests and they'r= e right. I'm really surprised most of Oracle users doesn't use raw device= s avoiding filesystem overhead. Think raw devices are sth new in Oracle a= nd have not been tested enough, that's why. -=3D-=3D-=3D=3D- >> In oracle, if you want to speed up the I/O operations, you can use r= aw device rather than file system -=3D-=3D-=3D-=3D wrote in message news:8m9sdg$9ga$1@nnrp1.deja.com... > A few questions on the use of raw partitions. > > Do you guys use them? I know they can give you a performance boost if= > I/O is the bottleneck of the database and that data integrity is bett= er > if using them since always know that Oracle writes going to disk and > not in OS cache but do you guys use them. I see that Oracle does not > reccomend that you use them. Are raw devices very difficult to manage= > (e.g. backups, sizing/planning, etc.)? Are they worth the trouble? > The issue is, as you say, one of I/O. Raws come into their own for Onli= ne Redo Logs. If you mirror your redo logs (and you'd be mental not to) th= en poor ol' LGWR has to mnake the same entries in both (or all) members of= each group -which could result in slower performance for the entire database unless you can somehow get the writes to the separate members = to happen truly in parallel. Most file systems do not support true asynchronous i/o, and accordingly= it is quite often recommended that online redo logs should reside on raw devices (the lack of a file system means that true asynchronous i/o is almost a given). However, they are actively discouraged for archived redo logs (there's actually an explicit 'prohibition' against putting archives on a raw de= vice in the Oracle training materials). And they are also not much use for t= he data files. And, I gather, most unix variants these days do a reasonable job of simulating asynchronous i/o (though not being a unix guru, I couldn't v= ouch for this for sure) -so inside Oracle these days its always a bit iffy whether to tall people to use raw devices even for online redo. The problem as I understand it is that without a file system, you can't= very easily manage your logs -there's nothing handy like 'mv' or 'rm' o= r 'cp' -you have to use 'dd' all the time, and I am constantly told by students on my courses that managing raw devices is (comparatively speaking) a pain in the butt. I'd avoid 'em like the plague for anything other than redo logs, and I'= d only bother using them for the logs if you can demonstrate that multiplexing your logs has caused significant performance degradation. Regards HJR > Do you guys use cooked files to overcome the data integrity issue > instead of raw partitions? > > Thanks for your help -=3D-=3D-=3D- As I understand Async I/O is a raw device. On HPUX boxes you can create= these raw devices as storage areas for database files. There is also a kernel parameter within HPUX that must be set to allow async i/o. I don= 't use it myself as I inherited my systems, but I would like look into it also. JH wrote in message news:39177120.189F04EC@mail.dotcom.fr... > Walter Dorninger a =E9crit : > > > > Hi, > > > > I have a question concerning asyncronous I/O on any platform. In th= e Oracle > > documentation there is stated that if I set the oraini-Parameter fo= r async > > I/O to true async I/O is used IF THE OS SUPPORTS async I/O - If the= OS does > > NOT support async I/O the setting is ignored. So I wonder how can I= figure > > out if async I/O is used or not ? > > > > Thanx, Walter > > WARNING ! > There's a corruption alert relative to ASYNC_IO and AIX 4.3.x... asynchronous i/o (for Oracle 8.0.6 on HP-UX 11.0) works only for "raw" partitions, it does not work for file systems. instructions for enabling async i/o are included in the Oracle installation guide. HP has a more complete document titled something like "Configuring MC/LockManager for Oracle Parallel Server" that goes into more detail of "raw" partitions, how to set them= up, configuring async i/o.. a lot of the info is applicable even if you're not running OPS. -=3D-=3D-=3D- I just got in a client site to replace a dba for a short period > of time(2-3 months), I find the datafiles are in ufs filesystem > instead of raw devices > OS Solaris 2.6 > Oracle 8.0.5 > How much of a performance gain would I get by moving all the > datafiles to a raw devices, assuming the db operations here > quite a I/O intensive, the client cannot afford a big downtime, > not really quite easy to justify to the client to buy more > hardware due to some internal situations > Is it worth the trouble? > Will a datafile created under ufs would work as expected if > transferred to a raw device directly? > > Your suggestion and comments would be greatly appreciated. > > * Sent from RemarQ http://www.remarq.com The Internet's Discussion Network * > The fastest and easiest way to search and participate in Usenet - Fre= e! Check the other posts .. although I've never really understood the supp= osed 'complexities' of raw devices - I've always found them easy enough to setup/admin etc... One quick fix would be to turn on the 'forcedirectio' option on the fil= e systems (in vfstab). This bypasses the unix file buffers and thus gets = you some of the way toward raw performance. One area to look at is the amount of sorts to disk - if there is a lot,= make sure you have many (as opposed to one) datafiles for the TEMP tablespace - since inode locking in ufs can cause nasties. What you don't mention is: The type of high I/O... Raw is better suited toward write access, than is ufs. Note, you can try using the readv option to increase reads with ufs. I agree with the others, and for a somewhat different reason. You are t= he temporary DBA, changing to raw requires a more care and feeding, and th= e new DBA may not have those abilities on day one. In all the reading I have done, most believe that using the Unix buffer= s is ultimately better, than turning them off. This is due to the read ahead= capabilities. Again I would concur with the previous posters, and besid= es an extra CPU(which is always great), I would check the RAM usage and ma= ke sure you are correctly using the Unix/Oracle buffers. Unless your client is well-versed in the administration of raw devices, I would discourage this. Place in front of your client some "what-if" scenarios having to do with database backups, database restorations, disk failures, and so on, and ask him if he would know how to deal with them, especially when his database disks= do not appear in the output of the df, mount, et al shell commands. Remember, the purchase of one additional CPU card to get more performance out of a mounted filesystem may be more than offset by the trouble and lost business (downtime) caused by a dba or sysop who is confounded by some peculiarity of raw devices and who does just one thing wrong at one particular instant. Just my $.02 -- if the shop knows raw devices inside and out, then none of this is applicable to your situation. But in many cases in my own experience (admittedly on other platforms) the sysops wish they'd never traded the admin ease of filesystems for the m= inor performance increases they got from re-creating the database on raw devices. One more thought... a few hours' worth of tuning might get you the same or more performance increase than going to raw ever would; I've seen cases where fixing just ONE BOGUS SQL STATEMENT using explain plan has made huge performance improvements! After managing both environments on similar platforms on IBM SP. I can = tell you that the raw DB is not worth the trouble ( unless you are going to = run Parallel Server ) the performance difference was non-existant when runn= ing Oracle Apps 10.7 /7.3.4.3. With the new disk solutions available for performance and availability , go that route and save the headache of managing RAW partitions. Also, keep in mind that you will likely have m= ore that one DB. Do you want your entire DB environment RAW ? It can be a P= IA if you want to copy non-raw to raw or vice versa regards DPC It depends. If you are running without an LVM raw files will usually give you better performance. When running with an LVM results appear to be mixed. Unless you are running Oracle Parallel Server you need not use raw devices. Based on your comment it would appear that Oracle is "operating system-friendly" when compared to Sybase since Oracle runs quite well w= ith cooked files. Oracle does not run better on cooked, normal OS, files; but most Unix administrators and DBA's probably find ufx file systems easier to understand and work with. We use Unix files for all our test systems an= d both Unix files and raw partitions for production. The more important t= he database the more likely we are using raw partitions for the small but still important performance improvement. Depending on your platform you may be able to use direct i/o to bypass = the Unix buffer pool instead of raw partitions and get much of the same performance. Note that if you use raw partitions you want to be sure to= configure async io to get the full performance benefit. On some systems= you have to manually configure both the OS and Oracle to use async and on others Oracle will automatically use async io if the file is a raw partiton. Check your platform specific manuals. Be careful about assertions that raw devices give better performance th= an cooked ones as this is not always the case. Take a look at what your application is doing, if it does lots of full table scans, using raw devices may actually reduce overall performance, because you lose the benefit of the Disk buffer cache. And there's always the possibility that your machine has 64 Gb of RAM of which Oracle can only use 1.7 Gb ! The filesystem buffer could then be the best place to use (a lot of) the rest. With RAW its hard to get things wrong.. With Cooked there's a lot of thing that you can get wrong - inode locki= ng, block sizes, file cache etc... Everyone keeps telling me how hard raw is - but so far it seems to be limited to "you have to use the dd command"...doesn't seem that hard to= me ... ;-) We use raw devices because our tests proved 10 % performance gain on Solaris 2.6, Sun 450. Further, you cannot inadvertently delete your raw= disks (as you may with your cooked files). Backup is not a problem beca= use I use the famous "dd" command to a pipe (and gzip from the pipe). But you _only_ have dd available to you. With cooked you can use dd, tar, cpio, ufsdump (or vxfsdump), cp, rdist, etc... Yay! The famous "dd" command! Nice to know it's still useful for someth= ing other than tape monkey work... ;) =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Just FYI, If you will use RMAN on Solaris then make sure that your oracle version= should be 8.0.5.1 or higher. RMAN has a bug when you use RMAN for resto= ring RAW devices with version prior to 8.0.5.1 on Solaris only. Otherwise RMAN works fine. I did all kind of Backup and Restore tests w= ith RAW devices. Sanjeev K. Sagar Alex Hudghton wrote: > wrote in message news:7rlma8$k35$1@nnrp1.deja.com... > > Has anybody successfully done Oracle Hot Backups (putting tablespac= e in > > backup mode etc.) from an instance based on Raw File Systems. > > > > Have you managed to successfully restored the database using this method > > > > How did you manage to backup the 'data-files' did you use DD. > > > > Are there scripts available which we could use to automate this > > > > If you're running version 8 then RMAN simplifies this task. We origin= ally > backed up to disk then saved the whole system on tape, but now use Le= gato to > backup straight to tape. > > Alex > I am pretty new to Oracle DBA field. I just finished installing ORACLE > on AIX. Now I am ready to create database. Some are saying to create the > databse in File System, but some are saying to create it in Raw device > system. Anybody have any recommendations. What are the pros and cons of > each system. > > One more question, a consultant is supposed to come and install couple > of user software which will access the Oracle on Unix. He gave me a call > yesterday, and asked me whether the Oracle Client is ready in a NT > machine. What does Oracle Client mean ? Does he mean Net8,sql Net, or > Oracle on NT ? Hi, the File System vs Raw Device question is one of trade-off between convenience and performance. The file system has more overhead than the raw device, but it is much easier to deal with than raw devices. However: - the file system implementation in AIX (JFS) is not too bad, so the performance hit will be acceptable, unless the size of the DB is > +/- 10 GB - unless your DB is already well optimized, the difference of performance between file system and raw device will be smaller than what you can gain by: * choosing the right block size * sizing and allocating the data cache correctly * allocating separate disks for: data, indexes, redo log, system segment, temporary * and of course DB design, indexes, etc In summary, go for file system unless performance is critical and you have already (or will) optimize(d) all the other aspects. With respect to Oracle Client question, the consultant is probably talking about Sql Net, unless he is talking about the client part of one of Oracle applications (Oracle Financials, ...). I hope this helps -=3D-=3D- > I have seen Oracle consultants come and go. Each one seems to have a > different style of his/her own when making recommendations for settin= g up > an Oracle database. One will say use 64kb strip as opposed to 8kb. On= e will > create large VGs with Raid 5 as opposed to Raid 1 with VGs reflecting= the > drive it is created on. Now the question before us is why does Oracle= > recommend the use of filesystems as opposed to raw logical volumes. C= ould > someone address this issue and if possible some of the others mention= ed in > this posting? > > Thanks very much in advance, > > TOM Define "best". Ofcourse, there's always s trade off between security/availability and speed. I thinks it's safe to say that decreasing one will increase the other. Having a RAID5 solution will be very secure as far as data is concerned= (it's not likely to loose data), but will not yield highest throuhput. Raw, unmirrored LVs, striped over many disks will yield highest through= put, but if one disk fails, lots of data(files) can be lost. As an example, take the record breaking TpC configurations: no archive logging, raw, striped LV's, everything destined for speed. Would you like a 24*7 shop without archive logging? So, it's up to you (or any Oracle consultant) to decide what is best. A= nd you may decide different every time requirements change. As far as stripe size is concerned, 32k to 64k is recommended (by Oracl= e). If you have the luxury of a test system and have plenty of time, you could tes= t different settings. -=3D-=3D-=3D-=3D I'de like to say I have vast experience with Solaris, but I don't. I th= ink I can however give you some info to help you make your decision. Raw devices are faster, but not as much as you would think. 3% to 15%. NT w= ould benefit the least from RAW while AIX the most. Raw partitions are a real real real pain to work with. If you have a VL= DB (ie > 200GB) you might consider them. Other than that, its's probably m= ore trouble than it's worth. DO NOT USE RAID 5. USE RAID 0+1 for performance Raw devices are not that Hard to Manage these days with various Volume Managers and Backup Softwares Available. Besides you should consider getting as much performance gains as possible especially if they provid= e like 20% better Performance than File Systems..But again , you may need= to Bench Mark before deciding .. FYI Oracle Recommends Using Raw DEvices for Redo Log files becos they a= re serial in nature . But you can also close in the Performance gaps between FileSystem and R=