Database engines as extensions to applications

Correction below - I misinterpreted you.


"Eric Kaun" wrote in message



  > > Its not the DBMS theorist's or
  > > vendor's position to tell the application programmer how they must solve
  > > their problem.
 >
 > Uh, if it means not telling them when they're violating the information
 > needs of the business, then you're wrong - it IS the business of the DBMS
to
 > vomit back in the application developer's face, so to speak.

What I should have said:

It is the data theorist's job to suggest good ways to model data. It is the
DBMS designer's job to model the structure and behavior of the DBMS that
will process data according to some theory (like design, the theory is
there - you can either be explicit or not, or use a solid theory or not, but
the theory is always there).

I don't recall any theorists anywhere wrestling developers to the ground to
prevent them from being stupid. But it is certainly their prerogative and
within their purview to mention the stupidity, and to present a better way.

In response to the implicit "there is no one right way" and "there's more
than one way to skin a cat" argument: relational has presented arguments as
to why it's better, and illustrations of relational algebras and calculi
indicate their expressive power, their useful properties, and the value of
the principles upon which they rest. I have yet to see another theory do the
same. Compare the writings of the relational theorists to those attempting
to formalize nonsense like XQuery, and describe to me what additional power
the theory has. It's excess complexity with no increase in power; at best,
XPath expressions have a place as operations on a data type (XML type, that
is).

- erk
 >> Stay informed about: Database engines as extensions to applications 

"Eric Kaun" wrote in message

<...>
 > In response to the implicit "there is no one right way" and "there's more
 > than one way to skin a cat" argument: relational has presented arguments
as
 > to why it's better, and illustrations of relational algebras and calculi
 > indicate their expressive power, their useful properties, and the value of
 > the principles upon which they rest. I have yet to see another theory do
the same.

Sure... I don't think anyone on this forum has argued that relational does
not have a solid theoretical base. What has been said is that a real world
implementation may benefit from using other tools for persistence (like an
OODB). These tools may not have the full expressiveness of a relational
solution but the application may not need it ... and the OODB probably
solves a niche problem better than a general relational solution.

If I had many applications hitting a common data store, I'd probably also
want to enforce data integrity rules in the DB. But if that's not the
situation, why introduce the overhead of having two areas semantic control
(constraints in the database and behaviour in the application)?

In our application, we program with objects (not 'data' and 'programs') so
*all* of our behaviour is defined in our object database. 'Applications'
are just different views of the same object model. I'd rather do that then
split behaviour and data.
--
Bob Nemec
Northwater Objects
 >> Stay informed about: Database engines as extensions to applications 

"Eric Kaun" wrote in message

 > Correction below - I misinterpreted you.
 >
 >




   > > > Its not the DBMS theorist's or
   > > > vendor's position to tell the application programmer how they must
solve
   > > > their problem.
  > >
  > > Uh, if it means not telling them when they're violating the information
  > > needs of the business, then you're wrong - it IS the business of the
DBMS
 > to
  > > vomit back in the application developer's face, so to speak.
 >
 > What I should have said:
 >
 > It is the data theorist's job to suggest good ways to model data. It is
the
 > DBMS designer's job to model the structure and behavior of the DBMS that
 > will process data according to some theory (like design, the theory is
 > there - you can either be explicit or not, or use a solid theory or not,
but
 > the theory is always there).
 >
I totally agree.

 > I don't recall any theorists anywhere wrestling developers to the ground
to
 > prevent them from being stupid. But it is certainly their prerogative and
 > within their purview to mention the stupidity, and to present a better
way.
 >
I would argue that this statement of yours is just that - wrestling
developers to the ground. Your method is verbal, and it happens all the
time. Subdued developers, who could otherwise complete their assignments,
have become convinced that they are not qualified to use a DBMS.

 > In response to the implicit "there is no one right way" and "there's more
 > than one way to skin a cat" argument: relational has presented arguments
as
 > to why it's better, and illustrations of relational algebras and calculi
 > indicate their expressive power, their useful properties, and the value of
 > the principles upon which they rest. I have yet to see another theory do
the
 > same. Compare the writings of the relational theorists to those attempting
 > to formalize nonsense like XQuery, and describe to me what additional
power
 > the theory has. It's excess complexity with no increase in power; at best,
 > XPath expressions have a place as operations on a data type (XML type,
that
 > is).
 >
I have no gripe against relational algebras and calculi, and nothing
superior to offer. But I qualified my comments as referring to "my world,"
which is a bit different than yours. My world is not the Enterprise with an
array of business applicaitons and ever-changing requirements, or rogue
developers.

Database engines are being used in places where they weren't practical
before, in embedded computers or within industrial processes. Their
services are useful in real-time systems which until recently couldn't use a
DBMS because of storage or performance limitations. They are also being
used in mass distribution, where its not practical to license the
full-featured but expensive DBMS products.

I will welcome the day when relational solutions are available in these
other places. But practically speaking, the simpler DBMS solutions are the
ones that fit, and happen to work (do the job?). Technology will eventually
catch up to the theory. In the mean time, the small footprint computer
systems of today are much like the big computers of the '60s, and some of
the same old (flat file, hierarchical, network) database solutions are the
best answer.

 > - erk
 >
 >
 >> Stay informed about: Database engines as extensions to applications 

Eric Kaun wrote:



 >
  >>Hi, I have only been reading posts in some of the database newsgroups for
 >
 > a
 >
  >>few weeks, but I have been involved in database engine development for
 >
 > over
 >
  >>20 years.
  >>
  >>In my world, where software is developed to solve problems, database
 >
 > engines
 >
  >>are purchased/used if they are a part of the solution. Then, the database
  >>software's purpose is to follow the instructions of the application as it
  >>accomplishes its purpose.
 >
 >
 > I disagree completely. The database, like the applications, support the
 > business. It can, and should, prevent the application programmers from
 > making stupid mistakes, and from violating the information needs of the
 > business. Even in application code I firewall certain things, and make
 > classes immutable, to prevent stupid mistakes. The database also serves to
 > "formally document" the meaning of the data.

Eric, if you're talking about things like integrity constraints,
you have my agreement. Much like a compiler prevents you
from writing invalid constructs in your code, the DBMS
should prevent you from violating referential integrity.

On the other hand, I have agree with Wayne that, as
programmers we have to solve a certain set of problems
which are defined by the application at hand. If, at
the same time, we can address problems of data redundancy
et. al. which relational algebra addresses, well and good.

As to stopping programmers from making stupid mistakes,
you might as well try to stop world hunger, baldness
and "the heartbreak of psoriasis" at the same time.
As one person put it ...

"The two most ubiquitous things in the universe are
hydrogen and stupidity, not necessarily in that order."


 >
 >
  >>Perhaps I am misinterpreting many of the discussions I am reading, but it
  >>seems that many participants define the world the other way around, where
  >>the data model or the theory behind the model assume superiority over the
  >>job to be done.
 >
 >
 > Yes. The business has a greater need: valid data. Leaving validity up to
 > each of the N applications that touch the database is foolish, as it simply
 > invites mistakes and misinterpretations and inconsistency.

This is the referential integrity argument again. These checks
could be left to the DBMS engine or encapsulated in library
calls which everyone *must* use on a discplined project.
Whether or not this library code would be more or less
efficient than the DBMS-engine is debatable.

 >
 >
  >>Is an application "wrong" if it uses a flat file as its database, when
 >
 > that
 >
  >>is entirely sufficient to do the job?
 >
 >
 > Not necessarily, but there are costs that climb exponentially as the number
 > of applications rises above 1. Most businesses have important data that's
 > "touched" by more than one application; thus the need for consistency and
 > validation.

This is a very real problem which, to my knowledge has
never been solved in the real world, although many
attempts have been made. If you are developing a
"one of" application, for a fixed price and a deadline,
the use of flat files may be exactly the right choice.
Once you get past 1 application needing the same data,
you are right (but exponentially is probably stretching
the point).

Note that sharing of data between applications need
NOT be instantaneous. A nightly download is quite
acceptable for many cases. Each case has to be
examined individually. Primary-secondary relationships
have to be well-defined.

 >
 >
  >>Is it wrong if it uses a network
  >>model database because space constraints rule out relational languages?
 >
 >
 > Which relational language? There are very low-footprint SQL engines. Very
 > few (meaning zero) relational ones, though.

At the risk of putting words into Wayne's mouth
(sorry if I misinterpreted you Wayne), using
the currently available SQL implementations
and having the need to apply multiple indices to
tables could use up considerable space
over and above the basic needs of data
storage. In a CODASYL model, where access
paths are well defined and known a-prior, the
only overhead is for the pointers to the related
records. (G_d help you if your navigation ever
needs to change though, but let's not bring
up the network vs. relational debates of
20+ years ago again.) This space limitation
may well constrain the choice of DBMS engine
to something other than relational.

Then again, 1NF set of tables is also
wasteful of space, so you have to make
tradeoffs.

 >
 >
  >>My experience shows that applications have needs for all kinds of data
  >>storage and manipulation extensions.
 >
 >
 > What do you mean by "extensions" - to the application language?
 >
 >
  >>Its not the DBMS theorist's or
  >>vendor's position to tell the application programmer how they must solve
  >>their problem.

Here, I will take a middle ground. Something like
the relational model provides a starting point
which can be used for analysis. Using that
base of knowledge rather than reinventing the
wheel regarding data-modeling is a "good thing".

Assuming that you then must implement the physcial
data structures on disk (e.g.) exactly as the
model dictates is often overkill. My philosophy
is probably similar to Wayne's. Do what's necessary
for the current job at hand.

 >
 >
 > Uh, if it means not telling them when they're violating the information
 > needs of the business, then you're wrong - it IS the business of the DBMS to
 > vomit back in the application developer's face, so to speak.

So, who write the integrity constraints? Another programmer?
They are just as likely to be wrong (initially) as the
application developer. (Granted, usually these are
checked much more throughly than application code.
But that's a process issue when you come to think
of it.)

 >
 >
  >>That's not to say that development of theory and improvement
  >>of implementation are not important.
 >
 >
 > This isn't theory - DBMSs, and relational in particular, was created for
 > PRACTICAL reasons.
 >
 >
  >>But the test of their importance is
  >>when they are actually used to do something real.
 >
 >
 > I consider telling the developer of an application that they're breaking
 > data integrity constraints that are important for year-end financial
 > reporting (one example only) "real". Don't you?
 >
 >
  >>The best ideas and
  >>implementations will prevail
 >
 >
 > In other words, you're proposing a Darwinian approach to correctness? Most
 > businesses don't have the luxury of experimenting with every possible theory
 > and implementation.
 >
 >
  >>because they really help to do the job.
 >
 >
 > "The job" is your weasel-phrase here. What job? Jobs in a business can
 > conflict. Why do you suppose the accounting, auditing, corporate security
 > and other departments exist in a company? How about labor relations?

And each of these departments has different requirements
on how they use the intersecting set of data, and at what
frequency. Try telling your customer for your application
that the reason it's going to take 10+ minutes (e.g.)
to update an Employee record is that someone from
accounting is running the year-end financials.

 >
 > Checks and balances. An app developer has to do a job, but if in the process
 > they violate data requirements of the company, many other jobs could be in
 > jeopardy.
 >
 > - erk
 >
 >


--
"It is impossible to make anything foolproof
because fools are so ingenious"
- A. Bloch
 >> Stay informed about: Database engines as extensions to applications 

And I didn't see this followup before I posted my
rim shots, either


Eric Kaun wrote:

 > Correction below - I misinterpreted you.
 >
 >


 >


  >>
   >>>Its not the DBMS theorist's or
   >>>vendor's position to tell the application programmer how they must solve
   >>>their problem.
  >>
  >>Uh, if it means not telling them when they're violating the information
  >>needs of the business, then you're wrong - it IS the business of the DBMS
 >
 > to
 >
  >>vomit back in the application developer's face, so to speak.
 >
 >
 > What I should have said:
 >
 > It is the data theorist's job to suggest good ways to model data. It is the
 > DBMS designer's job to model the structure and behavior of the DBMS that
 > will process data according to some theory (like design, the theory is
 > there - you can either be explicit or not, or use a solid theory or not, but
 > the theory is always there).
 >
 > I don't recall any theorists anywhere wrestling developers to the ground to
 > prevent them from being stupid. But it is certainly their prerogative and
 > within their purview to mention the stupidity, and to present a better way.

Maybe not in this forum there haven't been, at least
recently. But I have worked in places where the
so-called Data Modeller had the final say as to
the layout of the database. In some cases, it was
the data modellers who were stupid and would not
budge from the relational algebra even when
presented with de-facto evidence that their model
could not meet, for example, performance
requirements. I guess that's one of the reasons
I still have a job. Fixing the physical data
layouts to match actual access patterns and optimize
those.

 >
 > In response to the implicit "there is no one right way" and "there's more
 > than one way to skin a cat" argument: relational has presented arguments as
 > to why it's better, and illustrations of relational algebras and calculi
 > indicate their expressive power, their useful properties, and the value of
 > the principles upon which they rest. I have yet to see another theory do the
 > same. Compare the writings of the relational theorists to those attempting
 > to formalize nonsense like XQuery, and describe to me what additional power
 > the theory has. It's excess complexity with no increase in power; at best,
 > XPath expressions have a place as operations on a data type (XML type, that
 > is).
 >
 > - erk
 >
 >


--
"It is impossible to make anything foolproof
because fools are so ingenious"
- A. Bloch
 >> Stay informed about: Database engines as extensions to applications 

 > relational has presented arguments as to why it's better, and
 > illustrations of relational algebras and calculi indicate their expressive
 > power, their useful properties, and the value of the principles upon which
 > they rest. I have yet to see another theory do the same.

RDM is a subset of the expressiveness afforded by pure relational
algebra. RDM doen't have sufficient expressiveness to implement the
higher levels of data/code structures/integration that people loosely
associate with OO. RDM is optimal for its limited scope. For problems
outside of RDM's scope, RDM will be inefficient and impractical. For
now, RDM is well suited for many common applications.

Lets test the expressiveness of RDM on a simple problem. Suppose we
are modeling persons. A person may (or may not) have the following
attributes: skin and hair. Skin may (or may not) have the attributes
of color(s) and texture(s). Likewise hair may (or may not) have the
attributes of color(s) and texture(s). Thus example data might be:

Joe.
Bob with black hair.
John with black skin and red, blue hair.
Mary with smooth, white skin and red, silky hair.

Could you show how RDM is as expressive as XDb. By that I mean, as
normalized and NULL-less (or do you consider NULLs to be expressive).
And could you demonstrate how RDM's calculi is simpler at finding
persons than XDb's relational expression based queries shown at
<a rel="nofollow" style='text-decoration: none;' href="http://www.xdb1.com/Example/Ex005.asp" target="_blank">www.xdb1.com/Example/Ex005.asp</a>

Since RDMers had trouble getting started will the last problem posed
in another thread, below are possible base tables to get one started:

T_Person
T_PersonSkin
T_SkinColor
T_SkinTexture
T_PersonHair
T_HairColor
T_HairTexture
T_Color
T_Texture

PS. The answer to your question from a different post: The
underpinning of OO is the same as that of RDM, relational algebra.
 >> Stay informed about: Database engines as extensions to applications 

 >In my world, where software is developed to solve problems, database engines
 >are purchased/used if they are a part of the solution. Then, the database
 >software's purpose is to follow the instructions of the application as it
 >accomplishes its purpose.

You assume that the application owns the data, but this is a realy
rare case. For common data this would trigger a import operation of
the data when the application starts and an export operation when the
application ends as soon as a second application uses the same input
data or needs the result of this application.

 >Perhaps I am misinterpreting many of the discussions I am reading, but it
 >seems that many participants define the world the other way around, where
 >the data model or the theory behind the model assume superiority over the
 >job to be done.

They start from the point that the data represents the business and
that there are a lot of different applications which work on the SAME
data pool. In this scenario you tune the overal acceess speed and
not the access speed of a single application.

 >Is an application "wrong" if it uses a flat file as its database, when that
 >is entirely sufficient to do the job? Is it wrong if it uses a network
 >model database because space constraints rule out relational languages?

As long as the application owns the data, i.e is the only one that
uses it, it can store the data in any form it wants.

Regards from Germany

Franz-Leo
 >> Stay informed about: Database engines as extensions to applications 

Franz-Leo Chomse wrote:

  >>In my world, where software is developed to solve problems, database engines
  >>are purchased/used if they are a part of the solution. Then, the database
  >>software's purpose is to follow the instructions of the application as it
  >>accomplishes its purpose.
 >
 >
 > You assume that the application owns the data, but this is a realy
 > rare case. For common data this would trigger a import operation of
 > the data when the application starts and an export operation when the
 > application ends as soon as a second application uses the same input
 > data or needs the result of this application.
 >
 >
  >>Perhaps I am misinterpreting many of the discussions I am reading, but it
  >>seems that many participants define the world the other way around, where
  >>the data model or the theory behind the model assume superiority over the
  >>job to be done.
 >
 >
 > They start from the point that the data represents the business and
 > that there are a lot of different applications which work on the SAME
 > data pool. In this scenario you tune the overal acceess speed and
 > not the access speed of a single application.
 >
 >
  >>Is an application "wrong" if it uses a flat file as its database, when that
  >>is entirely sufficient to do the job? Is it wrong if it uses a network
  >>model database because space constraints rule out relational languages?
 >
 >
 > As long as the application owns the data, i.e is the only one that
 > uses it, it can store the data in any form it wants.
 >
 > Regards from Germany
 >
 > Franz-Leo
 >

Quite reasonable, actually.

I think this points out wide differences in experiences
between the different folks here.

From one standpoint, the concept that the data
represents the business is very much valid.
What many folks do, however, is make the
(unsubstantiated) leap of faith that every
application which uses any of the data must
have the latest "up to the second" data
to work with. (I am not implying that you do,
Franz-Leo, just that many people do.)

For certain classes of applications, it
is perfectly acceptable to use "last
night's data" (or pick another time
period, if suits you). Thus, what you
suggested above regarding export/import
is quite a viable alternative for these.
(Caveat: the primary-secondary relationship
between data source and data sink should
be well understood and never violated,
but that's another issue.)

For certain other classes of applications,
having to access the huge corporate data-store
is just not practical. As an example,
assume an application which is generating
revenue for the business, such as a
telephone switching system. When a call
is placed, some sub-set of the callers and
called party's data needs to be available to
this system in order to complete that call.
Every call completed generates revenue for
Deutsche Telekomm (my apologies if I
misspelled that, Franz-Leo) therefor
the company wants & needs to have these calls
complete as quickly as possible. Many,
if not all, of these kinds of applications
have a need for reading (order of magnitude)
  >1000 records per second and updating >500
records per second. For practical reasons then,
these applications does *NOT* use the corporate data
repository to do its job but has a copy
of only the subset it needs (and may
actually have more information than
the corporate data repository, e.g.
speed-dial lists, etc.) In this case,
I think you would agree, that the
application needs to behave /as if/
it owned the data it was using in
order to meet its requirements, and
thus is free to choose any data representation
or storage scheme it deems best for
its needs. (It, in fact, may be
implemented as several embedded
processors within the physical enclosure,
each of which operates on a yet a smaller
subset of the switches data and each
of which may have its own representation
of the data.)

(Note: This application also
generates billing records which are
forwarded, at periodic intervals,
to the corporate repository so that
the appropriate accounts-receivable
application may issue bills and so forth,
so that it indirectly updates the
corporate data.)

This is quite a different environment
from what I think of as the corporate
data repository. (Correct me please
if I make any mis-statements below.)
There, applications such as accounts
payable, accounts receivable, sales,
inventory control, payroll, etc., etc.
are all using intersecting sets of
the same data. In fact, in order
to ensure consistency of the "books",
it is almost a given that they *have*
to be accessing consistent data, thus
must be accessing the same (logical)
data store. In this case, as you say,
the data store is not optimized for
any of the individual applications,
but for overall access speed.
(Sort of like politics, you make
everyone a little unhappy, but no
major group extremely happy or
unhappy.)

Sorry for such a long post, everyone.


--
"It is impossible to make anything foolproof
because fools are so ingenious"
- A. Bloch
 >> Stay informed about: Database engines as extensions to applications 

"Bob Nemec" wrote in message



 > Sure... I don't think anyone on this forum has argued that relational does
 > not have a solid theoretical base. What has been said is that a real
world
 > implementation may benefit from using other tools for persistence (like an
 > OODB).

Persistence tools are one thing, but when the "tool" impacts the data model
itself, there's a problem. There are OO-SQL mappings which rely on a SQL
data store, and thus preserve a model which is lame but better than OODBMSs.

 > These tools may not have the full expressiveness of a relational
 > solution but the application may not need it ...

I would argue that most programming languages could benefit greatly from a
built-in relational algebra, not just for data processing but for (drumroll
please) logic itself. But even if App X doesn't need relational, what about
Apps Y and Z and... ?

 > and the OODB probably
 > solves a niche problem better than a general relational solution.

Solves one problem, but creates others.

 > If I had many applications hitting a common data store, I'd probably also
 > want to enforce data integrity rules in the DB. But if that's not the
 > situation,

I can't think of a single case where I'd want to make that gamble,
considering that the cost of change would be very high. Every tiny little
app that I've ever written has grown - although in actuality, it was the use
of the data that grew, as additional apps wanted to tie in and use what App
X started. By the way, that's a sign of success ("they like your data! they
really like your data!"), and I certainly prefer to design for success.

 > why introduce the overhead of having two areas semantic control
 > (constraints in the database and behaviour in the application)?

Behavior is different from control, and you put constraints in the DB so
that multiple apps don't step on the data and each other.

 > In our application, we program with objects (not 'data' and 'programs')

Objects are data and programs - they're just welded together, sort of like
in the movie "Tetsuo: Iron Man".

 > so *all* of our behaviour is defined in our object database.
'Applications'
 > are just different views of the same object model.

So what does a view of an object model entail? A root object somewhere?

 > I'd rather do that then split behaviour and data.

Behavior and data are split, naturally. Objects function well to the extent
that they're considered values of types, stored in variables. But object
trees are hard to manage, and when you couple that with the fact that not
all operations are just naturally part of a one object, but are in fact
multi-class polymorphic, you get a mess. Sure you can introduce helpers and
factories and everything else to ease this, but why bother?

- erk
 >> Stay informed about: Database engines as extensions to applications 

<...>
  > > In our application, we program with objects (not 'data' and 'programs')
 > Objects are data and programs - they're just welded together, sort of like
 > in the movie "Tetsuo: Iron Man".

Well, that's a whole OO or not OO debate (I believe that objects are more
then just the sum of their parts, but that's not for this forum). Let's
just say that if you want to develop a large application and want to use
objects to do it, an OODB will add noticeably less overhead then object
decomposition into a relation DB.

  > > so *all* of our behaviour is defined in our object database.
  > > 'Applications' are just different views of the same object model.
 > So what does a view of an object model entail? A root object somewhere?

I think of it as a 'perspective' of the domain model. For example, we have
multiple groups using our financial app. The users that do the analytics
have a different perspective of the model then those that do the trades.
But they're both updating the same domain objects.


  > > I'd rather do that then split behaviour and data.
 >
 > Behavior and data are split, naturally. Objects function well to the
extent
 > that they're considered values of types, stored in variables. But object
 > trees are hard to manage, and when you couple that with the fact that not
 > all operations are just naturally part of a one object, but are in fact
 > multi-class polymorphic, you get a mess. Sure you can introduce helpers
and
 > factories and everything else to ease this, but why bother?

We, that's were our experience significantly diverges. I found managing all
the programs that manipulated data to be much harder than managing an object
model. I've never found object trees 'hard to manage' and as for behaviours
not being a 'natural part' of one object... that's common; it's why OO
models implement transaction (or complex update) objects. Transferring
money between a savings account and a checking account should not be the
responsibility of either account. Just like you'd have a farmer milk a cow:
the cow does not milk itself and the milk does not un-cow itself.

There's a whole world of OO modelling that we could get into (let's not).
It works for me, and an active OODB supports an OO model well.
--
Bob Nemec
Northwater Objects
 >> Stay informed about: Database engines as extensions to applications 

"Wayne Warren" wrote in message



  > > I don't recall any theorists anywhere wrestling developers to the ground
 > to
  > > prevent them from being stupid. But it is certainly their prerogative
and
  > > within their purview to mention the stupidity, and to present a better
 > way.
  > >
 > I would argue that this statement of yours is just that - wrestling
 > developers to the ground. Your method is verbal, and it happens all the
 > time. Subdued developers, who could otherwise complete their assignments,
 > have become convinced that they are not qualified to use a DBMS.

I didn't mean to be combative, but I was. Developers can be qualified to use
a DBMS, but I don't think it's too much to ask for them to understand
relational theory - if for no other reason than to have a qualified basis
for comparison with whatever they're using. However, since our schools are
de-emphasizing anything that's not language or product specific, we're in a
great deal of trouble - even basic logic and discrete math courses are
endangered, not to mention anything non-procedural (e.g. logic and
functional languages). That's the real shame.

 > I have no gripe against relational algebras and calculi, and nothing
 > superior to offer. But I qualified my comments as referring to "my
world,"
 > which is a bit different than yours. My world is not the Enterprise with
an
 > array of business applicaitons and ever-changing requirements, or rogue
 > developers.

True, your world is different, and I was overly dogmatic.

 > Database engines are being used in places where they weren't practical
 > before, in embedded computers or within industrial processes. Their
 > services are useful in real-time systems which until recently couldn't use
a
 > DBMS because of storage or performance limitations. They are also being
 > used in mass distribution, where its not practical to license the
 > full-featured but expensive DBMS products.

I'd just say that until we have a good relational engine (for general data
manipulation, not just persistence), we won't know how good such would be in
apps like yours. I'd still wager they have general applicability, but
understand your performance constraints (to some degree).

 > I will welcome the day when relational solutions are available in these
 > other places. But practically speaking, the simpler DBMS solutions are
the
 > ones that fit, and happen to work (do the job?). Technology will
eventually
 > catch up to the theory. In the mean time, the small footprint computer
 > systems of today are much like the big computers of the '60s, and some of
 > the same old (flat file, hierarchical, network) database solutions are the
 > best answer.

Yup. Just keeping my eye on the theory...

- erk
 >> Stay informed about: Database engines as extensions to applications 

"Nick Landsberg" wrote in message

 > Eric, if you're talking about things like integrity constraints,
 > you have my agreement. Much like a compiler prevents you
 > from writing invalid constructs in your code, the DBMS
 > should prevent you from violating referential integrity.

Say Uncle! UNCLE!

 > On the other hand, I have agree with Wayne that, as
 > programmers we have to solve a certain set of problems
 > which are defined by the application at hand. If, at
 > the same time, we can address problems of data redundancy
 > et. al. which relational algebra addresses, well and good.

I propose that a good declarative language (e.g. relational) will reduce
code bulk, increase clarity, and help solve those problems faster. Of
course, I understand the constraints of commercial availability,
performance, and company standards.

 > As to stopping programmers from making stupid mistakes,
 > you might as well try to stop world hunger, baldness
 > and "the heartbreak of psoriasis" at the same time.
 > As one person put it ...
 >
 > "The two most ubiquitous things in the universe are
 > hydrogen and stupidity, not necessarily in that order."

Well... true, but you can at least make it more difficult. For example, I
have a habit of returning unmodifiable collections from my Java objects,
since I usually want the owning object to... well, own its collection. Turns
out some developers were trying to modify the collection directly, which
would have caused problems. My "firewall" had practical benefits, and
discouraged harmful aliasing in the app. So you can't ever stop stupidity,
but you can sound an alert and limit its effects.

  > > Yes. The business has a greater need: valid data. Leaving validity up to
  > > each of the N applications that touch the database is foolish, as it
simply
  > > invites mistakes and misinterpretations and inconsistency.
 >
 > This is the referential integrity argument again. These checks
 > could be left to the DBMS engine or encapsulated in library
 > calls which everyone *must* use on a discplined project.
 > Whether or not this library code would be more or less
 > efficient than the DBMS-engine is debatable.

My preference for a relational engine stems not only for the
single-checkpoint you mention, but also its clarity of expression (e.g.
declarative constraints rather than slogging through constraint-maintaining
code, which tends to be longer and more implementation-driven).

  > > Not necessarily, but there are costs that climb exponentially as the
number
  > > of applications rises above 1. Most businesses have important data
that's
  > > "touched" by more than one application; thus the need for consistency
and
  > > validation.
 >
 > This is a very real problem which, to my knowledge has
 > never been solved in the real world, although many
 > attempts have been made. If you are developing a
 > "one off" application, for a fixed price and a deadline,
 > the use of flat files may be exactly the right choice.
 > Once you get past 1 application needing the same data,
 > you are right (but exponentially is probably stretching
 > the point).

Probably true.

  > > Which relational language? There are very low-footprint SQL engines.
Very
  > > few (meaning zero) relational ones, though.
 >
 > At the risk of putting words into Wayne's mouth
 > (sorry if I misinterpreted you Wayne), using
 > the currently available SQL implementations
 > and having the need to apply multiple indices to
 > tables could use up considerable space
 > over and above the basic needs of data
 > storage. In a CODASYL model, where access
 > paths are well defined and known a-prior, the
 > only overhead is for the pointers to the related
 > records. (G_d help you if your navigation ever
 > needs to change though, but let's not bring
 > up the network vs. relational debates of
 > 20+ years ago again.) This space limitation
 > may well constrain the choice of DBMS engine
 > to something other than relational.

Agreed. Hopefully TransRelational (which theoretically reduces storage size
as well as indices and access time) will improve this situation.

   > >>Its not the DBMS theorist's or
   > >>vendor's position to tell the application programmer how they must solve
   > >>their problem.
 >
 > Here, I will take a middle ground. Something like
 > the relational model provides a starting point
 > which can be used for analysis. Using that
 > base of knowledge rather than reinventing the
 > wheel regarding data-modeling is a "good thing".
 >
 > Assuming that you then must implement the physcial
 > data structures on disk (e.g.) exactly as the
 > model dictates is often overkill. My philosophy
 > is probably similar to Wayne's. Do what's necessary
 > for the current job at hand.

At the end of the day, that's true. I just get annoyed when people
completely skip the good design, using theory as a reference point, out of
ignorance or apathy rather than a need to get the job done using the tools
available.

  > > Uh, if it means not telling them when they're violating the information
  > > needs of the business, then you're wrong - it IS the business of the
DBMS to
  > > vomit back in the application developer's face, so to speak.
 >
 > So, who write the integrity constraints? Another programmer?
 > They are just as likely to be wrong (initially) as the
 > application developer. (Granted, usually these are
 > checked much more throughly than application code.
 > But that's a process issue when you come to think
 > of it.)

I disagree primarily because although declarative constraints are somewhat
alien, and might take getting used to, they're shorter and easier to debug
(e.g. by using them in queries to "see what happens") than implementation
code. I agree my point here might be debatable...

  > > "The job" is your weasel-phrase here. What job? Jobs in a business can
  > > conflict. Why do you suppose the accounting, auditing, corporate
security
  > > and other departments exist in a company? How about labor relations?
 >
 > And each of these departments has different requirements
 > on how they use the intersecting set of data, and at what
 > frequency. Try telling your customer for your application
 > that the reason it's going to take 10+ minutes (e.g.)
 > to update an Employee record is that someone from
 > accounting is running the year-end financials.

Sorry, the "weasel-phrase" was overly harsh, and wasn't meant to be
accusatory.

- erk
 >> Stay informed about: Database engines as extensions to applications 

In article ,
"Wayne Warren" <wwarren.RemoveThis@birdstep.com> wrote:

 > In my world, where software is developed to solve problems, database engines
 > are purchased/used if they are a part of the solution. ...
 >
 > seems that many participants define the world the other way around, where
 > the data model or the theory behind the model assume superiority over the
 > job to be done.

Most people on this forum argue from the position of corporate-style
shared databases. It's not often that initial assumptions like that are
explained.

FWIW I come from an application development background too. The tools I
use and develop are almost always for a product-oriented application. I
have very rarely worked with a shared database repository - even in
corporate settings data has generally been shared by import/export (with
validation).

Maybe that's why I'm pragmatic and why OOFILE is more of an OO tool than
a "true relational" database.

I do get sincerely confused about some people's viewpoints however
because even when I've seen relational databases in use, the
applications on top are either
a) dealing with one flat table, or
b) inherently navigational.

 > Is an application "wrong" if it uses a flat file as its database
Nope.

Nowadays it may be more sensible (and vastly more politically correct)
to use an XML file.

To give some examples, in the last 6 years I've used XML files for
- preferences in a distributed 3d Stereoscopic movie architecture, where
multiple external applications could control QuickTime components to
adjust 3D rendering on the fly

- preferences in a geological data graphic program

- a label printing application, storing vector objects

- client-server brainwave broadcasting, where XML fragments were sent
via a BEEP-based protocol to a server to setup triggers, request data
and XML sent back. The same XML settings schema as used to persist
settings on disk was used in the network requests to change settings.

As an extension of the flat file argument, there are many applications
out there where people want to save to a single desktop document that
can be copied and emailed around, even if internally it is a database.

I don't think you'll find any useful discussion of that kind of
application on this forum - it should really be named
comp.databases.object.bigSharedCorporateServers.

BTW I came very close to choosing dbVista over c-tree at one point in
the past, can't remember why I didn't now, although given the changes in
company direction I'm glad I did as a lot of my work is still on Mac.

--
Andy Dent BSc MACS AACM
OOFILE - Cross-Platform Database, Reports, Graphs, GUI in C++
PP2MFC - PowerPlant->MFC portability
<a rel="nofollow" style='text-decoration: none;' href="http://www.oofile.com.au/" target="_blank">http://www.oofile.com.au/</a>
 >> Stay informed about: Database engines as extensions to applications