Re: 2Gb Exp problems 8.1.7

From: Tim X <timx_at_spamto.devnul.com>
Date: 15 Feb 2003 01:47:42 +1100
Message-ID: <87ptpuzzox.fsf@tiger.rapttech.com.au>

>>>>> "DA" == DA Morgan <damorgan_at_exesolutions.com> writes:

 DA> Billy Verreynne wrote:
>> Frank wrote:
>>
>> > Could some OS guru elaborate on why some OS's only handle 2GB?
>> > Seems to me these must be 31 bit then. (AIX 4.x had a special >
>> 'add-on' to go beyond 2GB, looked like the MSB was enabled, so to
>> speak)
>>
>> Won't call myself an OS guru, but I will have a bash (don't like
>> ksh or csh that much ;-).
>>
>> Firstly, it has _nothing_ to do with operating system and
>> _everything_ to do with the file system.
>>
>> And this type of problem has been around for a long while. Back in
>> my mainframe days, the filesystem used on the Siemens BS2000 o/s
>> (still IMO one of the best mainframe o/s's ever) used a single
>> byte for storing extent number. Thus you were limited to 255
>> extents. (wasn't this also the case with Oracle extents a few
>> versions ago?)
>>
>> Using DOS old FAT16 filesystem is no different. The File
>> Allocation Table has also size restrictions. Thus, a completed
>> path of a subdirectory can only be (if I recall correctly) also
>> 255-256 characters in total. When you reached that limit, you can
>> not create any more subdirectories.
>>
>> Okay, now for ionodes and Unix. A file is represented by a
>> structure called an inode (similar to a FAT entry).
>>
>> Each inode contains stuff like file type, access rights,
>> timestamps, size, pointers to data blocks, and so on. A bunch of
>> data blocks make up a file. Their addresses are stored in that
>> file's inode.
>>
>> An ionode is of a certain size. Thus, you are limited to the
>> number of data block pointers (or addresses) you can store in the
>> ionode of a file.
>>
>> Here are a table listing Unix filesystems used by Linux through
>> the years: Minix ext ext2 Max fileystem size 64MB 2GB 4TB Max file
>> size 64MB 2GB 2GB
>>
>> Because operating systems are closely tied to specific filesystems
>> (often only running on a single specific filesystem), filesystem
>> limitation such as the 2GB filesize limit is seen as an operating
>> system issue. Which is not really the case as that limitation
>> belongs solely to the filesystem.
>>
>> -- Billy

 DA> You are correct but I don't see the separation as clearly as you
 DA> do. If an O/S doesn't support a file system it is the O/S that is
 DA> the issue.

 DA> In the case of some O/Ss they default to 2GB files and default

 DA> parameters in the O/S can be changed to allow larger files. The  DA> makes it look, again, like the O/S to me.

Yes, I don't think you can isolate the two as much either. Though I have not checked this, but I believe the limitations in the ext2 filesystem used on Linux was different depending on the underlying architecture of the filesystem - when run on a 64 bit machine, it was much larger than on a 32 bit system. In this case, you have the same OS and filesystems, but with different limits due to the base size of the platform.

Note that ext2 has been re-written since these times and can now supports a 4TB limit, even on x86 based platforms and the filename size can be increased from 256 to 1024 characters (but I think this does require setting of a define in the source and a re-compile.

Tim

-- 
Tim Cross
The e-mail address on this message is FALSE (obviously!). My real e-mail is
to a company in Australia called rapttech and my login is tcross - if you 
really need to send mail, you should be able to work it out!

Received on Fri Feb 14 2003 - 08:47:42 CST