Nearest Common Ancestor Report (XDb1's $1000 Challenge)

  > > Among other things, the difference in normalization between the
  > > implementations is quite different.
 >
 > It should not matter HOW my implementation generates the report.

It does matter because a less generic and unnormalized implementation
will cause more problems or even fail under a broader scope.

Here is another example. Suppose the user enters 'john' in tbl_thing
and then enters 'john' in other tables. Later he tries to change
'john' to 'johnny' in tbl_thing. I tried, but I cant because of
constraints to keep redundant data in multiple tables from becoming
unsynchronized (UPDATE statement conflicted with COL REF constraint
'FK_types_thing_0D'. This conflict occured in db 'Test', table,
'types', col 'thing'. Statement terminated.)

In XDb1, user can change 'john' to 'johnny' and the report runs as
before.

Using data itself to link redundant data is a non-generic solution.
 >> Stay informed about: Nearest Common Ancestor Report (XDb1's $1000 Challenge) 

On Tue, 18 May 2004 11:27:12 +0200, Hugo Kornelis
wrote:


 >
 > Your explanation does raise another question. It looks as if the same
 > syntax is used to specify both intension and extension of the model,
 > thereby eliminating the distinction between schema and population. My
 > guess is that XDb1 would accept the following without complaining:
 >
 > person isa thing.
 > john isa person.
 > mary isa person.
 > neo isa john.
 >
 > XDb1 will probably gladly store it - but what does it mean?

Interesting question. It probably means that anything can be anything.
Which might have an application in text/html processing. You could easily
create XML documents and maybe even schema's by extending XDb1. However,
looking at the onerous licensing page on the site, I'm not sure if you
would want to do that.

There might be situations where an untyped database can have
 > it's use (I admit that my original choice of words was too harsh), but
 > outside of those niches, the schema should be strictly seperate from the
 > data and the datatype of the data should be known.

While this is true for structural/relational models, in the OO model you
can enforce the datatype or even data behaviour by using member functions.
By overloading functions or operators in C++ you can compare 7 to eight,
over-the-hill to very-young and anything else you desire. Such
functionality may soon make it's way into SQL server as well, so perhaps
you don't require the data to be strongly typed as long as the operators
on the data members can be overloaded.

Just how this can be done with XDb1, I don't know, as I am concerned about
downloading it due to factors mentioned earlier.

 >
 > And this is exactly the reason why I'd never use XDb1 for serious work,
 > unless I encounter a problem area where the advantages of allowing
 > untyped
 > data outweigh the disadvantages.

It is said that if all you have is a hammer, everything looks like a nail.
There are a range of applications that require more flexibility than
strongly typed languages provide. Often the language of choice used is
Perl, and those who use it take the work seriously. But is a language a
database, or a database a language? After all, SQL Server stores the data
on disk as binary, whether or not it is integer or string, and it is the
database engine that reads/writes from/to the disk that enforces the
datatype. How is this different from using member functions/overloaded
operators in an object database?

 >
 > In 99.9% of all applications that store a person's age, comparisons have
 > to be made: who is younger than 45 years? Who is older, John, Mary or
 > Fido? How can XDb1 (or any other type-less database) anser the last
 > question is the user has provided the following input:
 >
 > over-the-hill isa age.
 > very-young isa age.
 > 7 isa age.
 > john is over-the-hill.
 > mary is very-young.
 > fido is 7.
 >
 > Most computers I have used will classify very-young as greater than
 > over-the-hill and will refuse to compare either of these to 7.

Er, it's the software (or operating system) that does the
classification/comparison, not the computer. You did mean software right?
And that is not an indication of how computers work, only a limitation of
the current software.



 >
 > Like I said - there may be specific situations where a product such as
 > XDb1 has it's use. But it's not (to quote the web site) "the future of
 > databases" - not even remotely!

Maybe not XDb1. After all, the customer service skills shown by Neo so far
precludes that possibility. But Object databases I would not be so quick
to dismiss. XDb1 is the beginnings of an Object database, only it's a
technology that I actually saw more than 15 years ago, and which has
evolved into a product called Cache (www.intersystems.com)

 > is not defined for the character domain and would result in an error. I
 > know all this, and I can rely on it - if and only if the database ensures
 > that only numeric data will be accepted in the Age column and only
 > character data in the Color column.

See my comment above about operator overloading. Also, your Colour could
be an RGB value. Where are you then? In weakly typed languages/databases
you can accept black and 0 and white and 16777216 as Colours.

Anyway, have fun

Ian
 >> Stay informed about: Nearest Common Ancestor Report (XDb1's $1000 Challenge) 

On Fri, 21 May 2004 18:05:20 +1000, thirdrock <iktaccounts at optusnet dot
com dot au> wrote:

(snip stuff I agree with)

  >> And this is exactly the reason why I'd never use XDb1 for serious work,
  >> unless I encounter a problem area where the advantages of allowing
  >> untyped
  >> data outweigh the disadvantages.
 >
 >It is said that if all you have is a hammer, everything looks like a nail.
 >There are a range of applications that require more flexibility than
 >strongly typed languages provide. Often the language of choice used is
 >Perl, and those who use it take the work seriously. But is a language a
 >database, or a database a language? After all, SQL Server stores the data
 >on disk as binary, whether or not it is integer or string, and it is the
 >database engine that reads/writes from/to the disk that enforces the
 >datatype. How is this different from using member functions/overloaded
 >operators in an object database?

I'm afraid I don't see what point you're trying to make.


  >> In 99.9% of all applications that store a person's age, comparisons have
  >> to be made: who is younger than 45 years? Who is older, John, Mary or
  >> Fido? How can XDb1 (or any other type-less database) anser the last
  >> question is the user has provided the following input:
  >>
  >> over-the-hill isa age.
  >> very-young isa age.
  >> 7 isa age.
  >> john is over-the-hill.
  >> mary is very-young.
  >> fido is 7.
  >>
  >> Most computers I have used will classify very-young as greater than
  >> over-the-hill and will refuse to compare either of these to 7.
 >
 >Er, it's the software (or operating system) that does the
 >classification/comparison, not the computer. You did mean software right?
 >And that is not an indication of how computers work, only a limitation of
 >the current software.

Yes, I did mean the software. Sorry for the sloppy choice of words.

(snip some more stuff I agree with)


  >> is not defined for the character domain and would result in an error. I
  >> know all this, and I can rely on it - if and only if the database ensures
  >> that only numeric data will be accepted in the Age column and only
  >> character data in the Color column.
 >
 >See my comment above about operator overloading. Also, your Colour could
 >be an RGB value. Where are you then? In weakly typed languages/databases
 >you can accept black and 0 and white and 16777216 as Colours.

In a relational database, I can define Colour with a datatype that accepts
RGB values instead of english names. If overloading of operators is added
as a feature in the future, you could accept create a datatype colour that
accepts both, that will overload the = operator to return true if 'red' is
compared to 0xFF0000 and that will gladly accept the expression 'green' +
'orange' and return 'yellow' (green = 0x00FF00; orange = 0xFF5000; the
result of adding (mixing) these colors is the bitwise or of these RGB
values: 0xFFFF00, or yellow).

But I (the developer) have to do this. I'll have to write the code for the
overloaded = and + operators. If I do that, I can extend the range of
acceptable values for the "colour" domain to include all English color
names, all 3-byte values from 0x000000 to 0xFFFFFF and all integers that
are a decimal representation of one of those 3-byte values. After that,
the end user will still be unable to enter 'banana' in the color field of
his application screen and expect the software to know that (s)he means a
slightly dark yellow.

Overloading gives the designer the possibility to create lots of extra
possibilities for the user. But the user still can't use things that are
not pre-defined by the designer. That's the point I was trying to make:
XDb1's parser will gladly accept ages of 7, "over the top", "slightly
older than I am" or even "I think therefore I am" without bothering to
check if it has any overloaded operators that are able to do anything
useful with that data.

Best, Hugo
--

(Remove _NO_ and _SPAM_ to get my e-mail address)
 >> Stay informed about: Nearest Common Ancestor Report (XDb1's $1000 Challenge) 

On 20 May 2004 14:22:24 -0700, Neo wrote:

Hi Neo,

   >> > > This indicates an elapsed time of less than 3 ms ...
   >> >
   >> > Among other things, the difference in normalization between the
   >> > implementations is quite different.
   >> > In an initial attempt to make them more comparable...
  >>
  >> Now you're changing the rules after the competition has started.
  >> Not a sign of good sportsmanship, Neo!
 >
 >The provided implementation was not similarly generic

Yes, I have to admit that my implementation was not "similarly" generic,
but even MORE generic than the XDb1 implementation, as I didn't limit the
possible hierarchies to a few predefined hierarchy types (as XDb1 does),
but to allow ANY hierarchies (as required in the challenge).

 > or normalized.

This argument is getting stale. In another message, I quoted Date's
definition of normal forms, as well as a description of Domain Key Normal
Form (DKNF), said to be <<(theoretically, at least) the "ultimate normal
form">>. Instead of pointing out which of theses normal forms you think my
implementation violates, your message only repeats uninteresting stuff
about XDb1's internals, followed by an unbacked claim that there is
redundancy in my table and that it's not normalized.

Backup your claim, if you can. But please leave XDb1 out of it - the
challenge was to use the relational model, not XDb1's model.


 >To make a very rough comparision, I simplified my implementation to
 >that which was provided.

Are you referring to this quote from another message you wrote: "I
modified XDb1 algorithm so as to not resolve things' names and simply
prints their IDs."

If so, I don't accept this simplification. The challenge was to create a
report (that contains names) from some input data (that contains names).
There were no IDs in either the input or the report. If there are IDs
under the cover is irrelevant.

If you want to start a new challenge to create a report showing only IDs
from input data containing names, by all means go ahead. Heck, I might
even try and modify my winning entry for this competition to see if I can
rid you of another $1000.


 > In general, XDb1 can't compete with RM in the
 >scope of a single table's worth of data since XDb1 has the overhead of
 >variable storage structures for each table cell. An yes, IDs are
 >faster than text so the comparision is far from perfect. XDb1 should
 >gain advantage as more tables are joined. The provided solution
 >doesn't join any tables since it is less generic and unnormalized.

The provided solution is not less, but even more generic. And it is
completely normalized.

The reason that it doesn't join any tables is that I made proper use of my
RDBMS' tools, instead of the brainless automatic insertion of an
autonumber/identity/guid table_ID column in every table that so many of my
fellow RDBMS developers fall victim to.


 >As I said earlier, in the simplfied case, it takes XDb1 2 or 3 ms on a
 >500 Mhz, 512MB Dell PowerEdge Server.

As I said earlier, this is irrelevant to the challenge. In the original
form, XDb1 took 16 ms on your machine. Interesting, it also took 16 ms on
my machine (AMD Athlon 1.3 GHz, 256MB)

 > I ran the provided SQL Server
 >script on the same machine and the report generation takes 65 ms
 >(based on the avg of the 3 runs below):
 >
 >Starting time Ending time Time Elapsed
 >------------- ------------ ------------
 >14:44:57.670 14:44:57.733 67 ms
 >15:02:26.233 15:02:26.297 64 ms
 >15:07:57.780 15:07:57.843 63 ms
 >
 >What might I be doing wrong to get results different than yours?

I assume that you copied and pasted my script straight into Query Analyzer
and executed it without any modification.

It's hard to diagnose the cause of this difference without access to your
computer. But I did think of one thing that might cause this difference.

When I installed SQL Server, I chose the collation "Latin1_General_BIN" as
deafult. That means that character data comparisons are done by straight
comparing of ASCII values. This is the quickest, but it has the
disadvantage that 'john', 'John' and 'JOHN' are treated as three distinct
values. If I wanted to, I could have a person named 'john' and a dog named
'John'.

I don't know what the default collation is, but I do know that it is one
of the case insensitive collations. One that treats 'john', 'John' and
'JOHN' as equal. This will of course introduce some overhead for string
comparisons - which are used quite extensively in my implementation.

Since this is the default collation, I assume that your database will use
case insensitive comparisons. There is a simple way to find out: change
one of the occurences of 'fido' in the insert statements for hierarchies
to 'Fido'. I accept that your database will accept this, showing that it
uses a case sensitive collation. If it is not accepted, your databse uses
a case insensitive collation; it might be a binary collation but it need
not - it might still be accent insensitive or even double-byte character
set!


Another possible explanation might be the other activity on the machine
that runs the SQL Server process. I have SQL Server on my desktop; when I
timed the queries I first shut down all other applications. (SQL Server is
a "friendly" process - it will grab extra memory or processor time if it
needs more, but only if that won't hinder other processes. Reducing other
processes will generally improve SQL Server's performance). Earlier
testing (with several other applications active) resulted in elapsed times
between 10 and 20 ms.

If you have SQL Server running on another server, make sure that server is
dedicated to SQL Server only, if you want the best performance. If you
have SQL Server running on your desktop, close all applications except
Query Analyzer when running this query.

Best, Hugo
--

(Remove _NO_ and _SPAM_ to get my e-mail address)
 >> Stay informed about: Nearest Common Ancestor Report (XDb1's $1000 Challenge) 

On 21 May 2004 13:21:24 -0700, Neo wrote:

Hi Neo,

   >> > Among other things, the difference in normalization between the
   >> > implementations is quite different.
  >>
  >> It should not matter HOW my implementation generates the report.
 >
 >It does matter because a less generic and unnormalized implementation
 >will cause more problems or even fail under a broader scope.
 >
 >Here is another example. Suppose the user enters 'john' in tbl_thing
 >and then enters 'john' in other tables. Later he tries to change
 >'john' to 'johnny' in tbl_thing. I tried, but I cant because of
 >constraints to keep redundant data in multiple tables from becoming
 >unsynchronized (UPDATE statement conflicted with COL REF constraint
 >'FK_types_thing_0D'. This conflict occured in db 'Test', table,
 >'types', col 'thing'. Statement terminated.)

You're right. I had forgotten to include DRI actions when I posted the
script for my implementation. Everywhere I wrote "references things", I
should have written "references things on update cascade on delete
cascade". Due to a silly limitation of SQL Server, this can only be done
for one of the foreign keys in the hierarchies table; I'd have to write a
trigger to get the same result for the other one. (Both the DRI actions
and the trigger won't affect the report writing performance in any way.)

I'm perfectly willing to supply the code for the trigger if necessary to
convince you, but not now - it's already 2 AM over here, so I'm off to
bed.


 >In XDb1, user can change 'john' to 'johnny' and the report runs as
 >before.
 >
 >Using data itself to link redundant data is a non-generic solution.

Huh? Please elaborate.

Best, Hugo
--

(Remove _NO_ and _SPAM_ to get my e-mail address)
 >> Stay informed about: Nearest Common Ancestor Report (XDb1's $1000 Challenge) 

  > > Among other things, the difference in normalization between the
  > > implementations is quite different.
 >
 > It should not matter HOW my implementation generates the report.

It matters because a non-generic/unnormalized implementation
is more likely to experience problems over a broader scope. Here is
another example, consider what happens when a thing has no name. Since
the provided solution specifically uses a thing's name to represent it
and link between tables, the provided solution will fail to represent
a non-named thing (ie a person whose name is unknown or doesn't have
one).

On the other hand, the XDb1 db's schema (created by the simple
english-like script) allows a thing to have 0 to many names. In XDb1's
db, the name is an attribute of a thing, the name isn't the thing
itself. The easiest way to check this functionality is to unname john
in a completed db. To unname john, click thing/person/john twice with
a small delay to enable edit mode. Clear out 'john' and press enter.
Note, when a thing has no name, XDb1 displays it's secondary classes,
if any and its properties, if any, thus after unnaming john, one will
see 35, his age. In addition, the report will run as before, except
'john' will be replaced with 'person 35', the unnamed things classes
and properties.

If you insist that unnaming a thing is not the same as creating an
unnamed thing in the first place, you can create the db with the
original script, except replace all instances of the word 'john' with
'*'. XDb1's NLI takes '* isa person' to mean, create an unnamed
instance of person class. In succeeding statements, '*' continues to
refer to the last unnamed thing created (XDb1's NLI is currently far
from complete and its GUI or API is required for many operations).

The main point I wanted to emphasize is not what the GUI has or hasn't
implemented, but the broader scope afforded by representing data with
high degree of flexibility and normalization (replacing redundant
things with a suitable ref to the original; things' names are not
suitable refs).
 >> Stay informed about: Nearest Common Ancestor Report (XDb1's $1000 Challenge) 

On 22 May 2004 13:50:44 -0700, Neo wrote:

   >> > Among other things, the difference in normalization between the
   >> > implementations is quite different.
  >>
  >> It should not matter HOW my implementation generates the report.
 >
 >It matters because a non-generic/unnormalized implementation
 >is more likely to experience problems over a broader scope. Here is
 >another example, consider what happens when a thing has no name. Since
 >the provided solution specifically uses a thing's name to represent it
 >and link between tables, the provided solution will fail to represent
 >a non-named thing (ie a person whose name is unknown or doesn't have
 >one).

Hi Neo,

Yes, that's correct. If the challenge had included the requirement to
store unnamed things as well, I wouldn't have designed my tables the way
they are now. Note however that things without name tend to be a PITA when
you want to do anything useful with it, unless other attributes are stored
that can be used to identify exactly which thing one wants to discuss.


 >On the other hand, the XDb1 db's schema (created by the simple
 >english-like script) allows a thing to have 0 to many names. In XDb1's
 >db, the name is an attribute of a thing, the name isn't the thing
 >itself.

The same holds for a relational database. In my design, the name is also
an attribute (aka column). Since I declared it as primary key, it has to
disallow nulls. If the challenge had called for nameless things, I would
have used a different primary key (probably an autogenerated integer that
would never be exposed to the end users).


 > The easiest way to check this functionality is to unname john
 >in a completed db. To unname john, click thing/person/john twice with
 >a small delay to enable edit mode. Clear out 'john' and press enter.
 >Note, when a thing has no name, XDb1 displays it's secondary classes,
 >if any and its properties, if any, thus after unnaming john, one will
 >see 35, his age. In addition, the report will run as before, except
 >'john' will be replaced with 'person 35', the unnamed things classes
 >and properties.

I checked, and this does indeed work as advertised.

 >If you insist that unnaming a thing is not the same as creating an
 >unnamed thing in the first place, you can create the db with the
 >original script, except replace all instances of the word 'john' with
 >'*'. XDb1's NLI takes '* isa person' to mean, create an unnamed
 >instance of person class. In succeeding statements, '*' continues to
 >refer to the last unnamed thing created (XDb1's NLI is currently far
 >from complete and its GUI or API is required for many operations).

This incompleteness shows if you try to enter an unnamed person, an
unnamed dog and an unnamed computer. Changing all 'john', 'fido' and
'laptop1' to '*' in the script is a good way to mess things up big time.
While multiple unnamed things are indeed allowed, it becomes impossible to
explain to XDb1 which of the unnamed things I'm referring to. As far as
I'm concerned, this shows why it's generally better to insist on naming
things.

Of course, you already pointed out that the NLI is incomplete. I'm sure
that you'll find a way to improve this. But please do note that I can
change my SQL Server database to allow nameless things without having to
contact the databse vendor, whereas an XDb1 user is stuck with the current
functionality until you find the time to fix this.

Both implementations do not provide full support for things without name,
things sharing a name or things having multiple names. All these points
can be addressed in my implementation by adding and/or changing some
tables and/or columns, with no need to contact the vendor. All these
points can only be changed by the vendor in XDb1's implementation. I'd say
that, unless you happen to *be* the vendor, my implementation should be
considered more generic.


 >The main point I wanted to emphasize is not what the GUI has or hasn't
 >implemented, but the broader scope afforded by representing data with
 >high degree of flexibility and normalization (replacing redundant
 >things with a suitable ref to the original; things' names are not
 >suitable refs).

**IF** the problems' requirements are such that one might expect the need
to store unnamed things, then the name should not be considered as
possible primary key. But since that requirement was not clear from your
original $1000 challenge post, it doesn't invalidate my claim.

Best, Hugo
--

(Remove _NO_ and _SPAM_ to get my e-mail address)
 >> Stay informed about: Nearest Common Ancestor Report (XDb1's $1000 Challenge) 

(Neo) writes:

 > Please prove me a fraud [...]

The problem isn't that you're a fraud, but that you're an idiot.
You're a fraud too, and a weasel, but that just makes you
entertaining. Heck, you've turned comp.databases.object into a
newsgroup that should really be under rec.humor in the hierarchy.

However, I do think you should stop and consider whether you want to
continue to make a fool of yourself in public. With every posting,
you demonstrate that you don't understand databases, and, lately, that
you're fundamentally dishonest to boot. My advice to you would be to
drop off USENET for a while, and spend some time reading database
theory (I recommend Joe Celko's books), and, as I've suggested before,
an introductory book on the programming language Prolog, which is the
language XDb1 looks like a retarded cousin of.

-tih
--
Tom Ivar Helbekkmo, Senior System Administrator, EUnet Norway
<a rel="nofollow" style='text-decoration: none;' href="http://www.eunet.no" target="_blank">www.eunet.no</a> T: +47-22092958 M: +47-93013940 F: +47-22092901
 >> Stay informed about: Nearest Common Ancestor Report (XDb1's $1000 Challenge) 

Why would this matter to anybody???

"Neo" wrote in message

  > > Tony wrote: If your analogy holds any water at all (to give you the
  > > benefit of very large doubt), it suggests that relational theory will do
  > > just fine for pretty much anything we ever want to do "in the real
world".
 >
 > $1000 to the first person who replicates the equivalent of
 > <a rel="nofollow" style='text-decoration: none;' href="http://www.xdb1.com/Example/Ex076.asp" target="_blank">www.xdb1.com/Example/Ex076.asp</a> using the relational model. To claim
 > the prize, one needs to produce the equivalent Nearest Common Ancestor
 > Report from normalized and NULL-less data and the solution must be as
 > generic, meaning allow the user to create any hierarchy, consisting of
 > different types of things (each type to allow different attributes)
 > and each thing in the hierarchy to have any number of parents. Report
 > generation must not be more than 2X slower than XDb1 on equivalent
 > hardware.
 >> Stay informed about: Nearest Common Ancestor Report (XDb1's $1000 Challenge) 

 > Why would this matter to anybody???

Because the ability to represent things in a NULL-less, normalized and
generic manner is a measure of a data model's scope and completeness.
 >> Stay informed about: Nearest Common Ancestor Report (XDb1's $1000 Challenge)