Re: Internet search engines and databases

Thank you Tony for your interests. To answer your question, no, we do not query beyond what is given on each site published data contents. The reasons are obvious, one, it is illegal to do that without consent from each site, two, it will involve a hell lot of additional efforts if we can do that. BUT, don't restrict yourself when you talk about accessing databases across the Internet to just conventional databases. Think about the published HTML pages from each site are actually databases themselves, just in somewhat un-regulated format and far from relational. Our site http://www.mallagent.com is a working example of querying the entire Internet and aggregating different data sources (un-related HTML pages). Non-confidential description? We don't have one yet. We may take this off the board and exchange emails later.

In article <38E3382E.3109D085_at_attglobal.net>, tony_fulford_at_attglobal.net wrote:
> I did visit your site (http://www.mallagent.com) and it was
> worthwhile.
>
> It appears you utilize other search engines in obtaining
> some of your results. Do you actually access any of the
> "databases" of the web sites or only the content which is
> readily available?
>
> Myself, and others, would like to see a non-confidential
> description of your methods and techniques.
>
> Thanks for the response.
> ___________________________________________________________
>
> mallagent_at_my-deja.com wrote:
>
> > You have raised some excellent points. Take a
> > look at www.mallagent.com. We are doing exactly
> > what you are asking for, an XML-based dynamic
> > querying site to bring multiple database search
> > results on one page. Our database is the whole
> > Internet.
> >
> > In article <08ec726e.812ee423_at_usw-ex0102-
> > 014.remarq.com>,
> > Tony Fulford
> > <tony_fulfordNOtoSPAM_at_ibm.net.invalid> wrote:
> > > I would like to start a new thread to discuss
 the future of
> > > searching
> > > the internet.
> > >
> > > Internet search engines utilize the accessible
 content of a
> > > website to
> > > generate their search results. More and more
 website content is
> > > being
> > > placed in databases and served dynamically to
 users as required.
> > > The
> > > information in these databases is not available
 to search engines
> > > because the databases have security mechanisms
 (e.g., username &
> > > password) to prevent access. As more and more
 websites become
> > > based on
> > > information contained in databases, the search
 results be come
> > > increasingly sparse and inaccurate.
> > >
> > > I have already found that when I want to find
 an item, a
> > > particular
> > > model computer printer (using model number for
 the search) for
> > > example,
> > > I will not get a hit from many websites via the
 search engines,
> > > yet if I
> > > go to the websites and search, I do get a
 search hit. So where
> > > will this
> > > leave us for comparison shopping for products.
> > >
> > > Are web search engines going to be relegated to
 providing search
> > > results
> > > from websites which only have static content?
> > >
> > > Are there ways to let search engines have
 access to databases
> > > without
> > > giving away the store (i.e., family jewels)?
> > >
> > > If we give search engines access to the
 databases, how do they
> > > know how
> > > to search them (e.g., databases, tables,
 columns, etc.)?
> > >
> > > Is it possible to generate an XML standard
 which would allow
> > > general
> > > access to most databases?
> > >
> > > Do we need some standards for databases which
 want to participate
> > > with
> > > search engines?
> > >
> > > The future of internet navigation and true
 competitive x-commerce
> > > depends on resolution of this problem. Let's
 get started!
> > >
> > > Tony Fulford
> > > Business User Software
> > >
> > > * Sent from RemarQ http://www.remarq.com The
 Internet's Discussion Network *
> > > The fastest and easiest way to search and
 participate in Usenet - Free!
> > >
> > >
> >
> > Sent via Deja.com http://www.deja.com/
> > Before you buy.
>
>

Sent via Deja.com http://www.deja.com/
Before you buy. Received on Thu Mar 30 2000 - 00:00:00 CEST