Re: Development Trends in Web and Oracle

Mark C. Stock apparently said,on my timestamp of 15/03/2005 12:50 AM:

> gettting it off the wire and into your application... all the layers (ie.
> i'm the non-techie manager that just found out the biggest customer we've
> got just standardized on XML for file transfers -- what's your solution?)

TCP/IP? ;)
> sorry, 'a comms program' sounds vague to me.
> -- how would you receive the document?

with whatever program I need to receive what was transmitted. For example, if they sent it via ftp, I'd need to use ftp. If they sent it with any other technology (highly recommended!), then I'd have to use the proper program. There is no such thing as one single way of sending a doc. Although of course there are recommendations. (and even SOAP!)

> (business manager talking here) we need a permanent record of the original
> received transmision. where do you propose storing it?

compressed in a file, which I'll make accessible to the applications, while keeping some categorisation data in a relational database.

> how do you propose
> tracking it -- ie. if i'm looking at your extracted data through your
> interface, will you be able to show me the original document in case of
> customer dispute?

Sure. Keep the tracking info in the database, with the actual doc in a compressed format somewhere else. That's what directories and bfiles were created for: to keep raw arbitrary data off the database in a convenient format and storage method.

> (please realize, i'm just trying to walk through a scenario, not trying skew
> this as justification for one technology over the other)

Of course. No sweat.

> fair enough. becomes a question then of which layer to you want to do it in,
> then which tools are available, understood, and significantly performant

Bingo. And therein lies another world of argumentation... :D

> oh, please. let's call it 'one of my many customers' then. it's a simple use
> case that certainly fits into you more complex scenario -- just multply it
> by #customers and #formats

Not really, Mark. You see: this is not a "scalability" issue where presumably one just keeps throwing CPU at an inherently inefficient process hoping like crazy to make it cope with large volumes.

The reason I invoked the multiple clients is not just volume.

It is also the issue of multiple universes. You will not have ALL your clients talking XML for a loooooooong time. And even when you do, I'll bet each will talk a slightly different idiom or "proprietary" lingo that will throw the scalability out the window.

*If* you base your solution on a "uber-XML" approach.

> Which is exactly what XML DB is designed to do. simply register the schema
> (once), then either create an XML resource from the XML Document via your
> favorite programming language, or just let the customer FTP the XML document
> directly into the database, or drag and drop it via a WebDav folder
>
> Depending on how you register the schema, XML DB automaticaly extracts the
> data into object relational tables, which can be accessed directly and made
> the subect of constraints and triggers, or it simple stores the original
> document data in CLOBs (as advocated by a few in this thread) allowing
> access to the data elements via SQL, using XPATH expressions. FBI and
> hierarchical indexes are used for performance.

I hear what you're saying and part of me wants to agree. Always been for keeping all in the db and processing it all there. However if I try to make that stick with most "architects" nowadays, sparks will fly. And with good reason: channeling the automatic extraction through the db is a recipe for non-scalability. And a recipe for BIG problems when I hit the first customer with no XML or a slightly non-standard implementation.

So I take the defensive design approach: move the data storage to a relational schema which I can integrate with the rest of the "legacy relational" (Yegawds!!!!) corporate data and leave the translation from/to XML (or whatever) outside of the db: where I can make full use of a plethora of tools out there.

See what I mean? It's all about defending myself against the vagaries of an imperfect but real world.

> XML DB supports schema evolution... although I'm not real familiar with the
> way that works.

Believe me: I spent a year looking deeply into XML and I still can't see how it can happen. Unless someone invents xsl for schemas. Now, *that* would be real complexity!

> However, the issue of schema change is unavoidable... no matter what your
> approach, if the partner changes the schema, you have to change your process
> to conform to the new version of the schema, and perhaps remain backward
> compatible with the old version(s). No matter what tool you choose.

Not really. I'll have to update my XML schema JUST FOR THAT client. In the translation layer. Because what I want to capture for further processing in relational is locked away in my db thank you very much! And shielded from external changes by the translation layer that I've built *outside* of the db. Using XML or whatever else is needed to understand ALL my clients' interchange data.

> Case studies and usage scenarios are a starting point for either further
> discussion or further analysis. You've nicely augmented the scenario, thank
> you.

That was the intention. I'm all for new technologies - wouldn't be in IT after all these years if I wasn't!

But I'd rather walk into a new technology fully aware of ALL the angles and gotchas. Not just the "tech-du-jour" that Oracle (or any other maker) wants me to swallow.

-- 
Cheers
Nuno Souto
in sunny Sydney, Australia
wizofoz2k_at_yahoo.com.au.nospam