Re: cluster,performance, 6k clients and lobs
Date: Sat, 10 Jul 2010 14:12:11 -0700
Message-ID: <l8lh365c58jggd5210vtb8d9h82o8hs637_at_4ax.com>
On Sat, 10 Jul 2010 06:08:55 -0700 (PDT), Mark D Powell <Mark.Powell2_at_hp.com> wrote:
>On Jul 9, 2:52 am, "hehehe" <antoni..._at_poczta.onet.pl> wrote:
>> Hello!
>>
>> Wa are trying to create application for archive lots of documents about
>> 15mln(1mb each). We will use oracle 11g database with cluster and
>> partitioning. There wiill be 3k to 6k parallel clients connection. All
>> documdnts will be in blob tables. So my question is what kind of servers and
>> diskk arreys we need to buy to meet the demands and to develop application
>> in the future? What is very important and what is not ? We weill use redhat
>> 5.x 64bit server. We will create middle tier applications in cluster to. So
>> maybe someone have similar production app ? Every opinion,info etc will be
>> very helpfull
>>
>> regards -MJ
>
>MJ, your description is a little unclear but what you want to do is
>estimate how many IO per second you need to handle and then spec the
>hardware especially the disk farm to be able to handle the load. The
>number of concurrent sessions is very important for determining the
>server specifications: number cpu and memory.
>
>Being that I work for HP I will recommend that you look over our
>offerings. If you want Intel base X86 processors we have the DL380
>line which is pretty good and we also have Itanium based product. We
>also offer storage solutions and network switches. We can provide the
>complete package which can make our solutions very cost effective.
>Just visit our web site.
>
>But to get truely useful sizing help you need to determine your IO
>requirements and concurrent user load. You will also need your total
>storage requirements. With this you can make informed decisions.
>Without decent estimates you might as well just buy as much hardware
>as you can afford. Once you have this information you can contact
>various vendors and ask them to submit bids with appropriate equipment
>to meet your performance specifications.
>
>HTH -- Mark D Powell --
>
>
>
MJ,
I too work for HP (on a large records management project), so I won't mention any more about the HP product line :-) but will agree that Mark has provided the correct response i.e. you need, among other things (like a business plan), a detailed requirements analysis.
It sounds like you've picked a database and an OS, plan to write an in-house app, and know how many users there will be, along with a very rough and initial storage estimate.
Here's a basic condieration - how will you be recording, documenting, and communicating your design process? How will you be presenting this as a business proposal? You have a long way to go - and once under way, how do you plan to identify and measure milestones?
It's important that you quantify the specifics of the [concurrent or otherwise] projected load, testing/researching towards identifying your resource requirements - and to perform ongoing testing (for a range of things) throughout your development and deployment. Along the lines of simple storage requirements, what about projected growth and retention?
Add in your uptime, failover, available facilities, etc requirements. Realize that downtime is necessary, and can be expensive to minimize.
Factor in ongoing maintenance and upgrade requirements etc.
Spend some time considering any third-party multivendor software/firmware/driver etc requirements.
Will you have separate development, test, and production environments? How would those environments be isolated and managed? Think about development life cycles, revision control, and change management.
Consider virtualization and how it might help to add agility and elbow room to your infrastructure.
Think about application/web load balancing, IO mulitpathing from server to storage, differing performance levels of storage being used for in specific places for specific needs.
How about backups, and offsite storage? Start thinking about disaster recovery.
Early on, design your network - don't just think about servers and disks. Plan for security, firewalls etc. How and when will you need to move files around for daily app and admin operations - and how will that be facilitated, managed, and controlled? How and under what conditions wll developers and users access the systems, using what tools and applications?
Consider the existing skill sets of your organization and where you may need external technical support/consulting. There are companies, who shall remain nameless :-) who design, implement, and maintain these types of systems for a living.
From there, it's a business decision$
Happy Trails,
Eric Stahl
Received on Sat Jul 10 2010 - 16:12:11 CDT