linux raw devices and filesystem on one phys. device

From: Volker Apelt <volker_apelt_at_yahoo.de>
Date: Sun, 25 May 2003 14:57:49 +0200
Message-ID: <x1-m3znlbp4j6.fsf@apelt.gw.tgnet.de>

ORA i920, Linux Suse 8.0  

Here is our problem:

CREATE DATABASE hangs after filling the raw partitions and creating the system_01.dbf and control files. I had to kill oracle and sqlplus with -9 after some hours during which the system was idle (checked with vmstat and top).

The same HW was used before to run Oracle with tablespaces on a filesystem and redo logs on raw devices on separate harddisks.

Our question:
Is it possible to use one partition (sda5) through a filesystem and access the other partitions (sda6 ...) through raw devices on the same physical device at the same time?

Setup:
We have a HW raid-10 (adaptec adapter) partitioned in non-overlapping partitions. (eg: sda5 and sda6, sda7, ..)

The first partition is formatted with an ext3 filesystem (noatime) and will contain the SYSTEM tablespace. The remaining space in it will serve as scratch space for the files of experimental and short lived tablespaces.

The other partitions on the same RAID will contain the tablespaces for our main application nd will be accessed as raw devices.

Log files and redo are on raw partitions on other hard disks.

eg:
.. partition /dev/sda with cfdisk

  mkfs.ext3 /dev/sda5
  raw /dev/raw/raw1 /dev/sda6
.. bind the other raw devices

  mount /dev/sda5 -o noatime /path/to/oracledata
.. copy and delete stuff on /path/to/oracledata
..

  sqlplus /nolog
  startup nomount;
  CREATE DATABASE xxx

     ...
     DATAFILE '/path/to/oracledata/system_01.dbf' SIZE 500M
        AUTOEXTEND on MAXSIZE  1024M
     ... 
     DEFAULT TEMPORARY TABLESPACE temp
        TEMPFILE '/dev/raw/raw2'
        SIZE xxxk  AUTOEXTEND off
     UNDO TABLESPACE undotbs1
        DATAFILE '/dev/raw/raw1' SIZE  xxxK AUTOEXTEND off
     ; 
     << THIS POINT NEVER REACHED!
     #.. run oracle installation scripts 

  #.. shutdown database
  #umount /path/to/oracledata

The tablespaces on the raw devices have a fixed size of (size-of-partition - 2*db_block_size - 2*raid-stripe-size) rounded down to the next even number.

The 'raw' man page warns:

       Raw I/O devices do not maintain cache coherency  with  the
       Linux  block  device  buffer cache.  If you use raw I/O to
       overwrite data already in the  buffer  cache,  the  buffer
       cache  will  no  longer  correspond to the contents of the
       actual storage device underneath.  This is deliberate, but
       is regarded either a bug or a feature depending on who you
       ask!

So, it it is obvously not allowed to access one partition by filesystem and raw device and expect healthy information on disk. But that describes a situation where _one_ partition is accessed by _two_ programs.

But how about two _separate_ partitions, which are on a HW RAID-10? I'm not trying to read one partition in both ways.

When I try to unmount the fs partition after oracle has been killed that file system hangs forever in unmount.

Do you know of any reason why the fs partition should hang, if the other is used in ORACLE?

Do you know of such a setup that is working or reasons why it should not work?

Thanks,

Volker

-- 
Volker Apelt   volker._apelt  .@.  yahoo.de (remove the first 3 dots, please)
Dipl. Chem.    +49 6172 31126   

Received on Sun May 25 2003 - 07:57:49 CDT