native xml processing vs what Postgres and Oracle offer

paul c wrote:

>rpost wrote:
>...
>>
>> To which he replied: but a forum message is often a reply, and in that case,
>> a reply to a specific other message; this is not a presentation feature
>> but a basic structural property of his forum (and of USENET as well);
>
>For all we know, the OP's forum could be some idiosyncratic mutant, eg.,
>one-user-at-a-time and synchronous. I'd say it would be more useful to
>consider USENET. Regarding whatever a "basic structural property" is,
>to be more accurate, the basic structure of USENET is a message. As far
>as USENET is concerned, a message isn't complete when a user submits it,
>it is complete when some server has massaged the user message and
>introduced various "headers" to it. Those headers are the relevant
>"basic structural property" (attributes, to use Codd's lingo).

Yes.

>> not just of the implementation but at the functional requirement level.
>> You seemed to be flat-out denying this, which raised the question:
>> how would *you* model USENET or his forum?
>> ...
>
>I'm not denying any such thing. As far as how I would model USENET
>goes, first, I'd list my "functional requirements" and single out the
>ones that are expressible in terms of formal constraints, as well as
>apply the Information Principle and identify the attributes (properties
>if you like) that I wanted to record for each message, then form
>predicates and constraints that would admit whatever display
>presentation or presentations I desired.

Yes, and my specific question was: how would you deal with the requirement
that messages can be replies to specific other messages, or do you deny
that requirement?

>I doubt if after that, there would remain any pertinent reason, ie.,
>need, for the word "forum" other than as a label for the resulting
>application. Probably the constraints would have to counter various
>loopholes in the RFC's as well.

Perhaps, but I'm not interested in covering loopholes in RFCs, but
in the basic question that the OP implied in my eyes, namely, how to
model and work with the is-reply-to relationship between messages in a
relational framework.

--
Reinier
 >> Stay informed about: native xml processing vs what Postgres and Oracle offer 

paul c wrote:

>Bob Badour wrote:
>...
>> A much bigger problem with google groups is the inability to filter the
>> self-aggrandizing ignorants.
>
>Filtering looks like a functional requirement to me, I guess it would
>amount to restriction in relational terms. With the hierarchy if person
> B answers (person) A and C filters B, then it gets murky if person D
>responds to B. Maybe there are now two disconnected trees, maybe there
>is one, who knows?

Filtering commonly happens on threads, i.e. the transitive closure
of the is-reply-to relationship between messages. E.g. in my newsreader
I can just enter

/Bob Badour/f:,

to omit all messages from Bob Badour with all direct or indirect replies.
So the modelling language and/or query language must be capable of
expressing this; e.g. one solution would be to explicitly add the
is-eventually-reply-to relation with constraints to express its being
the closure of is-reply-to.

--
Reinier
 >> Stay informed about: native xml processing vs what Postgres and Oracle offer 

JOG wrote:

>On Nov 26, 9:30 am, rp....DeleteThis@pcwin518.campus.tue.nl (rpost) wrote:
[...]
>> From a relational perspective, the protocol spec should indeed have
>> been preceded by a separate logical design.  In the NNTP design, the
>> decision was made to postulate the unique identifiability of messages
>> regardless of their contents or other attributes; the alternative
>> (which most here generally advocate) is to identify entities based on
>> their attributes.
>
>While I am one of those advocates, it would be silly to ignore the
>efficiency of using a surrogate ID's in practical situations (and with
>the tools we currently have).
>
>However, to the OP: in terms of /the theory/ (this is a theory
>newsgroup after all) a good design takes a "message entity" and asks
>what it is exactly that defines its identity (what is the ID a
>surrogate for?). Note that there is no single "true" answer to this -
>that's important - because what a "message" actually is can be a whole
>host of things:
>
>1) {author, timestamp}: a message as a submission from an author at a
>certain time. If the content is edited later on it is still viewed as
>the same message as the original.

For USENET this would suffice: it does not allow messages to be edited;
it allow them to be superseded, but that doesn't work well in practice.
For a web-based forum it would also work. The issue for USENET is to
what extent the <author,timestamp> that will actually be used can
be guaranteed to be accurate, or at least unique.

>2) {author, timestamp, content}: a message as a piece of text,
>submitted by someone at a certain time. If the content is edited it is
>then viewed as a different message to the original.

This is only necessary if the same author can post multiple messages
at the same time, which as far as I can see NNTP doesn't allow, and
neither does web forum software. So even if messages can be edited, 1
is a better idea.

>3) {author, timestamp, parent}: a message is a position in a thread
>tree. If it is moved it is viewed as a different message. If this is
>not desired, you can still have a separate positioning table of
>course. Position just becomes a normal attribute however, and not part
>of the message's identity.

The same remark applies: even if messages can be moved (which some web
forum software supports), author and timestamp suffice for identification,
if they are reliable in the first place. If they don't, then adding the
parent won't fix it unless the posting protocol has some really unusual
properties.

There is also a fundamental objection: a parent is itself a message.

>4) {author, timestamp, content, parent}: a message is a piece of
>content at some position in a thread.

The same objections apply.

In short, I think the main issue in picking attributes and keys here is
not in determining how the data is to be used, but in determining
realistic commitments from the supporting software on the accuracy
of the attribute values supplied. For a web forum, <author, timestamp>
seems a good choice of key, even if they aren't always accurate, as long
as the server never accepts multiple messages with the same author and
timestamp.

--
Reinier
 >> Stay informed about: native xml processing vs what Postgres and Oracle offer 

rpost wrote:
....
> Yes, and my specific question was: how would you deal with the requirement
> that messages can be replies to specific other messages, or do you deny
> that requirement?
> ...

When you talk of "replies to replies", as you have done in this thread,
it makes me think that nothing I can say will cause you to think more
precisely, which is what is needed. Clearly there are messages that
arise that aren't replies. Clearly there are replies to such messages.
If you want replies to replies, then you are actually talking about
replies to replies to such messages. Obviously there are least three
predicates here. If you are "in the trees", I invite you to try to put
all three in one hierarchy, but I don't think I want you to show me the
result3.
 >> Stay informed about: native xml processing vs what Postgres and Oracle offer 

paul c wrote:

>rpost wrote:
>...
>> Yes, and my specific question was: how would you deal with the requirement
>> that messages can be replies to specific other messages, or do you deny
>> that requirement?
>> ...
>
>When you talk of "replies to replies", as you have done in this thread,
>it makes me think that nothing I can say will cause you to think more
>precisely, which is what is needed.

It would help focus discussions if you could bring yourself
to omit stuff like this. It tempts me to comment on its accuracy,
which is not what this newsgroup is for.

> Clearly there are messages that
>arise that aren't replies. Clearly there are replies to such messages.
>If you want replies to replies, then you are actually talking about
>replies to replies to such messages.

Yes, and replies to replies to replies to replies to ... to replies
to such messages. That's how discussion fora, including USENET, work.

> Obviously there are least three
>predicates here.

No, two will do: message and reply, where reply is-a message (i.e.
same primary key with a dependency key(reply) \subseteq key(message)).

An auxiliary predicate may be used to keep reply's transitive closure.

>If you are "in the trees", I invite you to try to put
>all three in one hierarchy, but I don't think I want you to show me the
>result3.

I don't understand this remark.

--
Reinier
 >> Stay informed about: native xml processing vs what Postgres and Oracle offer 

rpost wrote:
> paul c wrote:
....
>> When you talk of "replies to replies", as you have done in this thread,
>> it makes me think that nothing I can say will cause you to think more
>> precisely, which is what is needed.
>
> It would help focus discussions if you could bring yourself
> to omit stuff like this. It tempts me to comment on its accuracy,
> which is not what this newsgroup is for.
> ...

You could do well to be so tempted. The opposite is dooming oneself to
endless dead-ends (apologies to Yogi Berra). Bring it on!
....
>> Obviously there are least three
>> predicates here.
>
> No, two will do: message and reply, where reply is-a message (i.e.
> same primary key with a dependency key(reply) \subseteq key(message)).
>
> An auxiliary predicate may be used to keep reply's transitive closure.
> ...

In other words, 2 + 1 = 3 predicates.

>> If you are "in the trees", I invite you to try to put
>> all three in one hierarchy, but I don't think I want you to show me the
>> result3.
>
> I don't understand this remark.
>

I know you don't.
 >> Stay informed about: native xml processing vs what Postgres and Oracle offer 

rpost wrote:

> In short, I think the main issue in picking attributes and keys here is
> not in determining how the data is to be used, but in determining
> realistic commitments from the supporting software on the accuracy
> of the attribute values supplied. For a web forum, <author, timestamp>
> seems a good choice of key, even if they aren't always accurate, as long
> as the server never accepts multiple messages with the same author and
> timestamp.
>

This is why web-based forum coders (working with a relational database
at the back end) don't like "threading."

....it's hard to do and you pay a huge performance penalty to make it
happen (with relational modeling anyway). So relational programmers
fool themselves into thinking hierarchies are bad design.
Hierarchies are not bad design, they are the weak underbelly of the
relational model.
Hierarchies are part of the real world. They just don't fit well into
the relational scheme of things.

With XML querying hierarchies is a snap.
So if you have a hierarchical problem, XML is a better technology.

Someone referred to XML as messy technology that couldn't be optimised.
But SleepyCat and XPath is faster than any relational system running any
one of the ugly, complex and slow-as-mollases "relational solutions" to
the hierarchical problem.

For some problems you don't need a database at all: grep or perhaps
lucene or HyperEstaier are all that's needed.

For some problems XML is the best choice, particularly if the data
is naturally hierarchical.

For other problems--particularly for *large* data problems--relational
systems are the best choice....but almost never when hierarchies are
involved.
 >> Stay informed about: native xml processing vs what Postgres and Oracle offer 

salmobytes wrote:

> rpost wrote:
>
>> In short, I think the main issue in picking attributes and keys here is
>> not in determining how the data is to be used, but in determining
>> realistic commitments from the supporting software on the accuracy
>> of the attribute values supplied. For a web forum, <author, timestamp>
>> seems a good choice of key, even if they aren't always accurate, as long
>> as the server never accepts multiple messages with the same author and
>> timestamp.
>>
>
> This is why web-based forum coders (working with a relational database
> at the back end) don't like "threading."
>
> ...it's hard to do and you pay a huge performance penalty to make it
> happen (with relational modeling anyway). So relational programmers
> fool themselves into thinking hierarchies are bad design.

You are an idiot. Have a Happy New Year!
 >> Stay informed about: native xml processing vs what Postgres and Oracle offer 

On Dec 12 2008, 11:25 am, rp....DeleteThis@pcwin518.campus.tue.nl (rpost) wrote:
> Yes, and replies to replies to replies to replies to ... to replies
> to such messages.  That's how discussion fora, including USENET, work.

"Fora"? Bah humbug to faux latin pluralization of good, honest, hard
workin' english words....

That is unless, of course, you use the dative form when you post "to"
a USENET foro. Then hats off
 >> Stay informed about: native xml processing vs what Postgres and Oracle offer 

JOG wrote:

> On Dec 12 2008, 11:25 am, rp... RemoveThis @pcwin518.campus.tue.nl (rpost) wrote:
>
>>Yes, and replies to replies to replies to replies to ... to replies
>>to such messages. That's how discussion fora, including USENET, work.
>
> "Fora"? Bah humbug to faux latin pluralization of good, honest, hard
> workin' english words....

Speaking of good, honest, hard workin' english words, what's wrong with
"fake" or "false" ?
 >> Stay informed about: native xml processing vs what Postgres and Oracle offer 

Bob Badour wrote:

>JOG wrote:
>
>> On Dec 12 2008, 11:25 am, rp....DeleteThis@pcwin518.campus.tue.nl (rpost) wrote:
>>
>>>Yes, and replies to replies to replies to replies to ... to replies
>>>to such messages. That's how discussion fora, including USENET, work.
>>
>> "Fora"? Bah humbug to faux latin pluralization of good, honest, hard
>> workin' english words....
>
>Speaking of good, honest, hard workin' english words, what's wrong with
>"fake" or "false" ?

"The problem with defending the purity of the English language is that
English is about as pure as a cribhouse whore. We don't just borrow
words; on occasion, English has pursued other languages down alleyways
to beat them unconscious and rifle their pockets for new vocabulary."
-- James D. Nicoll

Sincerely,

Gene Wirchenko

Computerese Irregular Verb Conjugation:
I have preferences.
You have biases.
He/She has prejudices.
 >> Stay informed about: native xml processing vs what Postgres and Oracle offer 

salmobytes wrote:

[...]

>Hierarchies are part of the real world. They just don't fit well into
>the relational scheme of things.

This is a broad statement. What is needed as far as I can see is
efficient traversal of relations (i.e. arbitrarily wide, very selective
joins). This can be supported, even if many existing RDMBSes don't.

>With XML querying hierarchies is a snap.
>So if you have a hierarchical problem, XML is a better technology.

Not so fast.

XML itself is just a standard for serializing labelled trees. In my
experience, most of my "trees" are really arbitrary graphs (relations),
and while XML supports crosslinks as well, XML definition and manipulation
languages tend not to support their traversal well.
For a forum this need not be an issue.

But XML also assumes that all data is stored as documents and processed
by operating on documents one at a time. USENET does the same thing,
but it really isn't very practical.

Some issues from a database perspective: What if we want to query
or manipulate across the whole collection? Why do we always have
to parse documents whenever we want to use the data they contain?
Why do I, when writing queries or transformations on my data
(e.g. with XPath, XQuery or XSLT) or schema definition (XML Schema - please)
I always have to concern myself with stupid serialization and document
management issues such as the consistent use of file names and URLs,
file system limitations, character encodings, etcetera?

Not to mention that XPath, XSLT and XQuery are still pretty hideous
languages, both syntactically and semantically, although they have much
improved. Try representing an arbitrary relation (graph) in XML, then
writing, say, an XPath expression to compute its connected components.

Not such a good match for a discussion forum, if you ask me.
XPath queries may be expressive enough, but what about speed?
Do you want to represent the whole forum contents as a single XML
document that is updated whenever some posting or edit is performed?
Or are you thinking of some solution that keeps the whole thing in memory
in parsed form? How to make it scale?

>Someone referred to XML as messy technology that couldn't be optimised.
>But SleepyCat and XPath is faster than any relational system running any
>one of the ugly, complex and slow-as-mollases "relational solutions" to
>the hierarchical problem.

I'm not familiar with this, but does it work well for a big discusion forum?
How sophisticated is the querying you allow?

>For some problems you don't need a database at all: grep or perhaps
>lucene or HyperEstaier are all that's needed.
>
>For some problems XML is the best choice, particularly if the data
>is naturally hierarchical.

.... and consists of small enough bits (documents) that don't need to be
queried or manipulated collectively.

>For other problems--particularly for *large* data problems--relational
>systems are the best choice....but almost never when hierarchies are
>involved.

I think this is far too strong a statement.

--
Reinier
 >> Stay informed about: native xml processing vs what Postgres and Oracle offer