Could someone validate this product idea

From: <ttandom_at_hotmail.com>
Date: Fri, 29 Jan 1999 06:38:59 GMT
Message-ID: <78rl22$q5n$1@nnrp1.dejanews.com>

Hello,

Myself, along with a bunch of qualified friends are trying to put our entrepreneurial skills to test. We envision building a product in the Internet space ( Ohh yaa, just like everyone else ).

The product idea in question is to build a highly scalable, intelligent data cache to help large scale distributed internet applications.

In today’s enterprise a large number of data intensive applications have to be designed with the Internet in mind. In the client server era, typical DB activities like capacity planning, data growth rates, choice of server hardware were typically well known and DB’s designed around these facts. But today, these calculations have become a huge challenge, due to vast user/customer base coming in to access data through the internet which pragmatically cannot be determined at design time. Add to this complexity the fact that corporations are constantly merging with traditional competitors to better their stock value. Such mergers could suddenly increase the data size and the number of users multi-fold. So, we believe that tools which can help DB’s distribute their load permitting the DB’s to scale will have a tremendous value. A Relational DB cache which can be distributed and can cache frequently requested data can be one such product.

I am looking for any advise, validation of the idea in terms of its usefulness in the real world, any related products, etc. The paragraph below is primarily a brain dump and I can try to address any specific questions u may have.

The solution would be a 100% java solution allowing it run in a wide variety of platforms. The solution will be quite transparent allowing the application deployer to plug this product in to boost performance. We think we will be able to provide this using one of the following two options: - Implement as a JDBC driver. The client Java app simply makes Java JDBC calls to the driver which faults to the DB if the request cannot be satisfied by the cache. In fact, the cache will be implemented as a JDBC driver, which wraps any JDBC driver available in the market. - Intercept DB vendor specific network traffic ( For instance intercepting Oracle SQL*Net traffic) , parse the same checking for queries. Similar to the previous case, fault to the DB only if the requested data is unavailable in the cache. This approach gives us the advantage to allow any application written in any language to interface with the cache.

Predictive prefetching ( THIS IS THE AREA THAT REALLY NEEDS SOME HEAVY-DUTY VALIDATION) Typical DB cache's are either LRU or MRU based. We plan to use a variation of the LRU scheme where the system will automatically try to learn querying patterns and pre-fetch data based on certain policies and free CPU cycles. As query requests come in, the cache gains knowledge and maintains an online DB containing querying patterns. The logic continuously calculates the next probable query based on the knowledge gained, along with other user specific data(??). If for a given user, the system does not detect any repeatable pattern even after some period of time it can stop gaining any additional knowledge about the user.

QoS ( Quality of Service ) Certain users may be configured to have better quality of service than others. For instance - a Quota system can be

configured to control the memory usage by individual users/ groups.	- A
preference treatment when servicing requests to some users over others.     -

Query and associated patterns can be statically configured into the system overriding and/or complementing the auto pre-fetching for certain users. - Prefetching can be disabled for individual users and/or groups. - etc...

The cache will maintain cached data as Java object collections especially if we decide to serve the Java community.

I could address more aspects of the product, but enough for now. Our main concern is in the area of pre-fetching

How valuable will this be ??
Have any of u come across repeating querying patterns in your apps ??
Another big area of concern is in maintaining data consistency in the cache. Several thoughts have been floated:
- Ignore this for the first phase, provide necessary hooks in the product and
add support in the future.
- Data consistency need not be real time, but use configurable time out’s to
age data and re-fetch.
- Provide API/ framework and have the user manage the data consistency .
I would love to hear what u think ???

Thanking u in anticipation

-----------== Posted via Deja News, The Discussion Network ==---------- http://www.dejanews.com/ Search, Read, Discuss, or Start Your Own Received on Fri Jan 29 1999 - 00:38:59 CST