From oracle-l-bounce@freelists.org Wed Jun 22 00:22:11 2005 Return-Path: Received: from air891.startdedicated.com (root@localhost) by orafaq.com (8.12.10/8.12.10) with ESMTP id j5M5MAPl019030 for ; Wed, 22 Jun 2005 00:22:10 -0500 X-ClientAddr: 206.53.239.180 Received: from turing.freelists.org (freelists-180.iquest.net [206.53.239.180]) by air891.startdedicated.com (8.12.10/8.12.10) with ESMTP id j5M5MANi019026 for ; Wed, 22 Jun 2005 00:22:10 -0500 Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id BEE461C24E4; Tue, 21 Jun 2005 23:18:45 -0500 (EST) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 14398-10; Tue, 21 Jun 2005 23:18:45 -0500 (EST) Received: from turing (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 4069A1C2348; Tue, 21 Jun 2005 23:18:45 -0500 (EST) Date: Wed, 22 Jun 2005 04:16:56 +0000 From: Mladen Gogala Subject: Re: 64 node Oracle RAC Cluster (The reality of...) To: Kevin Closson Cc: Mladen Gogala , peter.sharman@oracle.com, kevinc@polyserve.com, mwf@rsiz.com, Rich.Jesse@quadtechworld.com, oracle-l@freelists.org, Peter Ross Sharman References: In-Reply-To: (from kevinc@polyserve.com on Tue Jun 21 23:53:35 2005) Message-Id: <1119413816l.6007l.0l@medo.noip.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by Ecartis X-archive-position: 21469 X-ecartis-version: Ecartis v1.0.0 Sender: oracle-l-bounce@freelists.org Errors-To: oracle-l-bounce@freelists.org X-original-sender: gogala@sbcglobal.net Precedence: normal Reply-To: gogala@sbcglobal.net X-list: oracle-l X-Virus-Scanned: by amavisd-new-20030616-p9 (Debian) at avenirtech.net X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on air891.startdedicated.com X-Spam-Status: No, hits=-3.0 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=2.63 On 06/21/2005 11:53:35 PM, Kevin Closson wrote: > this is not true for a real CFS. A proxy-cfs or nfs > exhibits the characteristics you fear, but not a > fully symmetric, concurrent read:write CFS. Demand > paging is nothing more than the internals of mmap > which in turn is really nothing more than an IO. > Oh, with one exception, it is entirely 100% read only > (a major text fault that is). That concern is a > red herring. Binaries execute just fine from a CFS. Kevin, I heard your name before and I am fully aware that you know much more then me. Truly symmetric clustered FS that I used to work with was known as Files-11. DEC representative used to tell us, I quote, "not to do image activation from remote nodes". As you probably know, VMS did not do file caching before version 6. The only question I have is synchronization of caches across the nodes. How can you achieve speed similar to the local FS? OS utilities usually do not support direct I/O. Oracle released a plug-in replacement for Linux utilities, so that you have ftp, cp, dd, ls, tar and cpio being able to utilize direct I/O. The thing still doesn't work for scp and sftp. So, how do you get around synchronizing caches on different nodes? I assume that there is a concept similar to SCN which gets increased with each transaction and if local SCN is higher then the global one, then you know that there was a change on your side and send all buffers with the increased SCN to the other guy or write the blocks down to the disk and have the other nodes re-read them (OPS technique). It still looks like a significant overhead which would slow down normal file operations significantly and make things like "vi $ORACLE_HOME/network/admin/listener.ora" fairly expensive. That would also interfere with the database operation and compete for the same bandwidth, wouldn't it? -- Mladen Gogala Oracle DBA -- http://www.freelists.org/webpage/oracle-l