Re: What is Pick anyway?

David Cressey wrote:
> "dawn" <dawnwolthuis_at_gmail.com> wrote in message
> news:1133116247.420778.41000_at_z14g2000cwz.googlegroups.com...
> >
> > David Cressey wrote:
> > <snip>
> > > How does the Pick model translate into an environment where many
> application
> > > programmers, possibly working for entirely separate companies, all
> write
> > > data into a common database?
> >
> > Can you give an example of an application where there is not a single
> > company or organization that controls the integrity of a database? I
> > have been curious about this before because the answer to your question
> > is that your implication is correct -- you would never want to have a
> > free-for-all where multiple companies use the PICK API directly. You
> > would use a more service-oriented approach where one company controlled
> > all services writing to the database.
>
> I can't speak to PICK. That's why I'm asking the question.
> I'm comparing to a database situation, where a major goal is data
> integration.
>
> It's fairly common, in the development of new databases, that you are
> integrating data that, in the status quo ante,
> are managed by separate departments, using separate packages, purchased from
> separate vendors, and maintained by separate teams of programmers.
>
> For example, the SALES department uses one package for All order processing
> functions, from order placement
> right up through billing. The PURCHASING department uses a different
> package, from a different vendor, for
> managing inventory, and tracking the replacement of items sold, including
> costs. A database is desired that integrates these two bodies of data, so
> as to be able to analyze MARGIN, at a wide variety of granularities, using
> some kind of
> graphic, interactive, drill down tool.
>
>
> >
> > I don't doubt that this is a requirement somewhere, but I have never
> > seen a situation where multiple companies must write directly to a
> > database without one of the companies controlling the integrity. So,
> > I'd be interested in an example.
>
> In the scenario I'm outlining, you may be dependent on the expertise of
> programmers in both the SALES and the PURCHASING departments to provide data
> feeds. Those feeds may or may not be consistent and/or reliable.

Garbage in, garbage out.

> You want to protect the database from bad feeds. There are two ways to go:
> Screen the data programmatically,
> by preprocessing it before loading into the new database, or declare
> constraints in the database itself, that will cause the database itself to
> reject data that violates the constraints. Generally, in a complex real
> world example, both approaches are used. They each have merits.
>

Most companies running Multi Valued (aka Pick) systems run all the departments on one database, ie one set of interrelated files.

However to handle the situation you describe in a classic MV system, a screening program would be used to validate the data. If one doesn't use SQL with a MV database, there are no restraints on what you can store. I don't use SQL so I don't know how well it handles constraints.
>
> >
> > > Doesn't the implicit coordination that results
> > > from "the programmer and the DBA are the same person" break down in
> that
> > > circumstance?
> >
> > No, it really doesn't. You might identify one developer as responsible
> > for a data access layer and have others use that proprietary API to get
> > consistent error handling and such.
>
> SQL plays the same role as a "proprietary API" except that it is more
> ubiquitous. Learn it once,
> and you've mastered the intrface between hundreds of programming
> environments and hundreds of
> database environments.
>

My guess is the proprietary MV program can be written quicker and with finer details than a combination of database constraints and SQL whatif's. I think the problem you and other SQL programmers have in understanding MV is that you are trapped in the SQL box. ;-)
>
> > Each developer would still prepare
> > any schema changes, however. The reason this works is that the schema
> > is less brittle than in a typical SQL DBMS. It is quite easy to extend
> > the schema for new purposes without harming anything there already.
> > This is related to the weak typing.
> >
>
> It's easy to build a multischema database, and let each programming team
> have it's own schema.
> However, when it's desired to combine the separate schemas into data that
> managed together,
> a schema has to be devised that meets the needs of all the stakeholders. In
> the environment I'm used to
> this is where database design gets interesting.
>
> All the stuff we discuss endlessly in here: logical and physical design,
> normalization, NULLS, etc. etc. is just the nuts and bolts. You have to
> be competent with that stuff, or you are going to make mistakes. Some of
> those mistakes have large consequences. But technical competence is jsut
> the beginning.
>
> You have to get PEOPLE to collaborate, or else you don't really have a
> database. All you have is some files dressed up to look like relational
> tables. An interesting exercise, but it hardly pays the rent.
>

Right, managing the people can be a lot harder than managing the data.
>
> > > How can the DBA or database designer of a Pick or UniVerse
> > > database protect the programmers from each other?
> >
> > Programmers typically have standards they agree upon and libraries
> > (sub-routines) they agree upon, but just like JavaScript and other
> > duck-typing languages, the programmer is given enough rope to hang
> > themselves. The flip side is that I have never had a more productive
> > team of developers than those working in PICK. The risk must be
> > weighed, but for almost all typical sets of data processing
> > requirements, I am quite sure I would get higher productivity from a
> > team of professionals developing in PICK than most other environments.
> >
> The above comment seems to me to be about the PICK programming language,
> and not about the PICK data model.
>

I normally don't think of these as separate entities. Many MV shops don't have strict standards. They may have only a couple of programmers, so it's not two hard to control things. Also, there are MV based and non-MV based source control systems that let you control and track program changes. I consider source control a must.
>
> That's fine, but unless you can explain how the Pick data model works in
> the scenario I'm outlining, I can't see
> how I'd apply the same model to enterprise wide data integration. And that's
> where my head is at with regard to the application of "database theory".
> (not "application programming language theory")
>

The MV database model basically is, you can store anything anyway you want. The dictionary and program logic defines the meaning of the data. As I stated in a previous post, the dictionary is optional if you want to program EVERYTHING. However, I consider a good dictionary a must. The dictionary is requried for access from outside the MV environment such as via ODBC.

> I may not have described the scenario I've outlined very well, but I regard
> the scenario as typical of database work.

As a side note, one of the most popular ETL (Extract, Translate, Load) programs, Datastage, uses Universe as it's engine. Such a program could be used to validate and load data from and to multiple sources including relational databases.

Cheers,
Don Received on Wed Dec 28 2005 - 05:33:31 CET