Обсуждение: Dreaming About Redesigning SQL
Hi, This is for relational database theory experts on one hand and imlementers of real-world alications on the other hand. If there was a chance to start again and design SQL afresh, for best cleaness/power/performance what changes would you make? What would _your_ query language (and the underlying database concept) look like? Seun Osewa PS: I should want to post my ideas too for review but more experienced/qualified people should come first
> -----Original Message----- > From: Seun Osewa [mailto:seunosewa@inaira.com] > Sent: Friday, October 03, 2003 11:52 AM > To: pgsql-hackers@postgresql.org > Subject: [HACKERS] Dreaming About Redesigning SQL > > > Hi, > > This is for relational database theory experts on one hand > and imlementers of real-world alications on the other hand. > If there was a chance to start again and design SQL afresh, > for best cleaness/power/performance what changes would you > make? What would _your_ query language (and the underlying > database concept) look like? > > Seun Osewa > PS: I should want to post my ideas too for review but more > experienced/qualified people should come first I imagine you have read the 3rd database manifesto by Codd. I think he's gone off the deep end a bit. You don't just throw away a trillion dollars worth of effort and tools to make things mathematically orthogonal. However, on some things he is clearly right. For instance, null values are evil. Programmers understand it, but end users will *always* be surprised that: SELECT COUNT(shirts) FROM clothing WHERE color = 'blue' SELECT COUNT(shirts) FROM clothing WHERE NOT color = 'blue' Is not equal to the number of shirts in the inventory if any color fields are null. Therefore, his idea of using default values instead and never using null is a good one. If SQL vendors would follow the ANSI/ISO standard to the letter, and implement the latest iteration, that would solve all of the problems that SQL tool users have to face.
A long time ago, in a galaxy far, far away, dwolt@iserv.net (Dawn M. Wolthuis) wrote:
> Good question. Although I would want to move away from relational
> databases too, if there is an RDBMS and one wants to query it, what
> would I aim for? If you look at XQuery, you will see an example of
> what I would definitely NOT aim for. Although the user of such a
> language might very well be a technical person, instead of starting
> with mathematics (relational calculus, relational algebra) I would
> suggest starting with language. The mathematics of language is more
> complex than the mathematics of relations, particularly simple
> relations (such as 1NF tables).
No, that _very much_ gets things backwards.
You need to have a clearly defined model of how the data is to be
manipulated before it makes any sense to try to make up a language.
--
output = reverse("moc.enworbbc" "@" "enworbbc")
http://cbbrowne.com/info/rdbms.html
To iterate is human; to recurse, divine.
DCorbit@connx.com ("Dann Corbit") wrote in message
news:<D90A5A6C612A39408103E6ECDD77B829408BE9@voyager.corporate.connx.com>...
> > -----Original Message-----
> > From: Seun Osewa [mailto:seunosewa@inaira.com]
> > Sent: Friday, October 03, 2003 11:52 AM
> > To: pgsql-hackers@postgresql.org
> > Subject: [HACKERS] Dreaming About Redesigning SQL
> >
> >
> > Hi,
> >
> > This is for relational database theory experts on one hand
> > and imlementers of real-world alications on the other hand.
> > If there was a chance to start again and design SQL afresh,
> > for best cleaness/power/performance what changes would you
> > make? What would your query language (and the underlying
> > database concept) look like?
> >
> > Seun Osewa
> > PS: I should want to post my ideas too for review but more
> > experienced/qualified people should come first
>
> I imagine you have read the 3rd database manifesto by Codd. I think
> he's gone off the deep end a bit.
Dann, you are showing your ignorance. While Dr. Codd recently died, if
you think he wrote a third database manifesto, you have definitely
gone off the deep end yourself.
> You don't just throw away a trillion
> dollars worth of effort and tools to make things mathematically
> orthogonal.
Again, you are showing your ignorance. Nobody has ever suggested
anything even remotely resembling the above.
> However, on some things he is clearly right. For instance, null values
> are evil.
Dr. Codd believed we need two NULLs. You ascribe correctness to the
one thing I think he clearly got wrong.
> Programmers understand it
That's an absurd assertion.
> Therefore, his idea of using default values instead and never using null
> is a good one.
That is not his idea.
> If SQL vendors would follow the ANSI/ISO standard to the letter, and
> implement the latest iteration, that would solve all of the problems
> that SQL tool users have to face.
Upon what do you base this ridiculous opinion?
>> The mathematics of language is more complex than the mathematics of relations, particularly simple relations (such as 1NF tables). << Are you sure, you know what you are talking about? >> I would suggest ditching the entire relational model (as both overly simplistic in its theory and overly complex in its implementation.. << Incredible! How about reading some books on the subject? -- -- Anith ( Please reply to newsgroups only )
Good question. Although I would want to move away from relational databases too, if there is an RDBMS and one wants to query it, what would I aim for? If you look at XQuery, you will see an example of what I would definitely NOT aim for. Although the user of such a language might very well be a technical person, instead of starting with mathematics (relational calculus, relational algebra) I would suggest starting with language. The mathematics of language is more complex than the mathematics of relations, particularly simple relations (such as 1NF tables). If you look at the history of data persistence prior to Codd's 1970 ACM paper, you will see several attempts at this. One I have studied of late is GIRLS (Generalized Information Retrieval Language and System), specified by Don Nelson and implemented by several folks with the most famous being Dick Pick. This GIRLS language was specified a full 40 years ago and lives today in many IT shops under a variety of about 10 different names, including IBM's UniQuery and Retrieve (for UniData and Universe respectively). This language is flawed, as are all, but so very close to what I would think would be a good approach.It was written at TRW in order to make it so that themilitary in Viet Nam could query their data without technical folks in the field. It went into production in 1969 with the US Army. Prior to the end of the cold war, it was used by the CIA to track (the associated database) and query about Russion spies in the US. I would suggest ditching the entire relational model (as both overly simplistic in its theory and overly complex in its implementation) and start with English (that is one of the other names for the GIRLS language). Note that language is also the starting point for putting data in XML documents, but it sure doesn't seem to be the starting point for XQuery, eh? --dawn Dawn M. Wolthuis www.tincat-group.com
With all due respect, Dawn, you are an idiot. "Dawn M. Wolthuis" <dwolt@iserv.net> wrote in message news:6db906b2.0310091212.4f967cf5@posting.google.com... > Good question. Although I would want to move away from relational > databases too, if there is an RDBMS and one wants to query it, what > would I aim for? If you look at XQuery, you will see an example of > what I would definitely NOT aim for. Although the user of such a > language might very well be a technical person, instead of starting > with mathematics (relational calculus, relational algebra) I would > suggest starting with language. The mathematics of language is more > complex than the mathematics of relations, particularly simple > relations (such as 1NF tables). > > If you look at the history of data persistence prior to Codd's 1970 > ACM paper, you will see several attempts at this. One I have studied > of late is GIRLS (Generalized Information Retrieval Language and > System), specified by Don Nelson and implemented by several folks with > the most famous being Dick Pick. This GIRLS language was specified a > full 40 years ago and lives today in many IT shops under a variety of > about 10 different names, including IBM's UniQuery and Retrieve (for > UniData and Universe respectively). This language is flawed, as are > all, but so very close to what I would think would be a good approach. > It was written at TRW in order to make it so that the military in > Viet Nam could query their data without technical folks in the field. > It went into production in 1969 with the US Army. Prior to the end of > the cold war, it was used by the CIA to track (the associated > database) and query about Russion spies in the US. > > I would suggest ditching the entire relational model (as both overly > simplistic in its theory and overly complex in its implementation) and > start with English (that is one of the other names for the GIRLS > language). Note that language is also the starting point for putting > data in XML documents, but it sure doesn't seem to be the starting > point for XQuery, eh? > > --dawn > Dawn M. Wolthuis > www.tincat-group.com
dwolt@iserv.net (Dawn M. Wolthuis) wrote in message news:<6db906b2.0310091212.4f967cf5@posting.google.com>... > I would suggest ditching the entire relational model (as both overly > simplistic in its theory and overly complex in its implementation) and > start with English (that is one of the other names for the GIRLS > language). Note that language is also the starting point for putting > data in XML documents, but it sure doesn't seem to be the starting > point for XQuery, eh? > > --dawn > Dawn M. Wolthuis > www.tincat-group.com Please explain further. What do you really mean? Its natural for everyone here to think every word in that post was a troll unless you explain your views more clearly. You could not have expressed a more unpopular/unsupportable combination of ideas! Exactly how would we go about using language as a query tool? Is this AI? What would the underlying model be knowing how redundant and imprecise language can be? Tell what we may have missed. Seun Osewa
"Mike Preece" <michael@preece.net> wrote in message news:1b0b566c.0310131551.47ca0d47@posting.google.com... > dwolt@iserv.net (Dawn M. Wolthuis) wrote in message news:<6db906b2.0310130618.6920e737@posting.google.com>... > > Bob Badour" <bbadour@golden.net> wrote in message news:<O6WdnZ4P-_6I4RWiXTWJlg@golden.net>... > > > "Dawn M. Wolthuis" <dwolt@iserv.net> wrote in message > > > news:6db906b2.0310100813.65b872d8@posting.google.com... > > > > Thank you, Seun, for asking your question with a bit of logic and not > > > > gut-reaction emotional baggage (and for also asking a question of me > > > > off-list so I could ramble). I'll try to make this more suscinct. > > > > > > With all due respect, Dawn, you are an idiot. That statement has nothing to > > > do with my viscera or with my emotions; I write it with complete dispassion. > > > You simply are an idiot, and Seun simply lacks sufficient education to > > > recognize that fact from what you have written. > > > > Thanks for the clarification, Bob. What seemed like an emotional > > outburst from me was actually a logically-derived statement regarding > > my capacity then, right? If I were you, I might be just a tad bit > > concerned that I have taught many engineers calculus and you are > > flying on planes or driving over bridges they designed. Let's hope > > they learned it well in spite of me, eh? smiles. > > --dawn > > > > Dawn M. Wolthuis > > Dawn: I'm sure you're not an idiot. Mike, you have expressed certainty contradicting the patently obvious before. Your opinion is only a useful gauge insofar as you are wrong so much more frequently than you are correct.
dwolt@iserv.net (Dawn M. Wolthuis) wrote in message news:<6db906b2.0310130618.6920e737@posting.google.com>... > Bob Badour" <bbadour@golden.net> wrote in message news:<O6WdnZ4P-_6I4RWiXTWJlg@golden.net>... > > "Dawn M. Wolthuis" <dwolt@iserv.net> wrote in message > > news:6db906b2.0310100813.65b872d8@posting.google.com... > > > Thank you, Seun, for asking your question with a bit of logic and not > > > gut-reaction emotional baggage (and for also asking a question of me > > > off-list so I could ramble). I'll try to make this more suscinct. > > > > With all due respect, Dawn, you are an idiot. That statement has nothing to > > do with my viscera or with my emotions; I write it with complete dispassion. > > You simply are an idiot, and Seun simply lacks sufficient education to > > recognize that fact from what you have written. > > Thanks for the clarification, Bob. What seemed like an emotional > outburst from me was actually a logically-derived statement regarding > my capacity then, right? If I were you, I might be just a tad bit > concerned that I have taught many engineers calculus and you are > flying on planes or driving over bridges they designed. Let's hope > they learned it well in spite of me, eh? smiles. > --dawn > > Dawn M. Wolthuis Dawn: I'm sure you're not an idiot. Bob: Do pigs really like people to wrestle in mud with them?
Bob Badour" <bbadour@golden.net> wrote in message news:<O6WdnZ4P-_6I4RWiXTWJlg@golden.net>... > "Dawn M. Wolthuis" <dwolt@iserv.net> wrote in message > news:6db906b2.0310100813.65b872d8@posting.google.com... > > Thank you, Seun, for asking your question with a bit of logic and not > > gut-reaction emotional baggage (and for also asking a question of me > > off-list so I could ramble). I'll try to make this more suscinct. > > With all due respect, Dawn, you are an idiot. That statement has nothing to > do with my viscera or with my emotions; I write it with complete dispassion. > You simply are an idiot, and Seun simply lacks sufficient education to > recognize that fact from what you have written. Thanks for the clarification, Bob. What seemed like an emotional outburst from me was actually a logically-derived statement regarding my capacity then, right? If I were you, I might be just a tad bit concerned that I have taught many engineers calculus and you are flying on planes or driving over bridges they designed. Let's hope they learned it well in spite of me, eh? smiles. --dawn Dawn M. Wolthuis
"Dawn M. Wolthuis" <dwolt@iserv.net> wrote in message news:6db906b2.0310100813.65b872d8@posting.google.com... > Thank you, Seun, for asking your question with a bit of logic and not > gut-reaction emotional baggage (and for also asking a question of me > off-list so I could ramble). I'll try to make this more suscinct. With all due respect, Dawn, you are an idiot. That statement has nothing to do with my viscera or with my emotions; I write it with complete dispassion. You simply are an idiot, and Seun simply lacks sufficient education to recognize that fact from what you have written.
Thank you, Seun, for asking your question with a bit of logic and not gut-reaction emotional baggage (and for also asking a question of me off-list so I could ramble). I'll try to make this more suscinct. First of all, I have read Codd's 1970 & 1974 ACM papers, as well as his "The Relational Model for Database Management, Version 2" book. I have also read several books by Chris Date and by Fabian Pascal. I held a dialog (that reads more like a monologue) with Pascal and it is reproduced in total if you scroll to the bottom of the http://store.tincat-group.com page and click on the Dick Pick / Ted Codd Blue Brothers parody picture. I had not been reading this news group until lately, but it strikes me that this is a group that might be very entertained by that dialog. I have a master's degree in mathematics and my father is a linguist. I find the language of mathematics and the mathematics of language both fascinating. My experience, however, is that I have run IT project teams working with a variety of databases (and languages) and have never seen any environment that is as agile for develoeprs (both productive from the start and easy to maintain) than the teams I have led that worked with the UniData database. I have worked with SQL as well as both older and newer database languages. So, nope, I'm not trolling. I've been doing some research the past couple of years and I'm convinced that it is time to do something new (and yet old) with data persistence. I favor using Java for a variety of reasons, but am comfortable with other languages as well, and think that using Java both for the software application and for the constraints on the data, rather than encoding constraints in some other language within a database, makes for both a more agile development approach AND, surprisingly enough, tends to make for better data integrity, although a lousy software developer can certainly mess up either environment. Separating the DBA from the software developer has definitely had a negative affect on the speed with which software is developed and maintained, but my experience (and intuition -- I don't, yet, have scientific evidence) tells me that the benefits purported by the approach of having a dba work on some centralized constraints on the persisted data outside of the context of the use of that data have not really come to fruition and/or are not worth the costs of using this approach (more on that some other time). So, while some might classify me as an idiot (men can be so emotional sometimes ;-), I have several graduate classes in logic to my credit and believe that I am approaching this topic quite logically, even if my summaries skip some of the logical steps in the process. I have thought about how to prove my points and since my point is really about agility and quality in application software development and maintenance, a competition to see what tools and techniques and what data persistence approaches win such a competition might be the best proof. The current industry benchmarks for databases tend to be SQL-based and highly political, so let's put different approaches to the test. Thanks for asking your question and not just assuming I'm a nut because I disagree with the current state of the industry on this topic. I'm sure there are gaps in my thinking and I know some of my opinions are based on intuition that arises from my experience, but I do hope to have more proof in the future. I am also very willing to adjust my opinions with convincing arguments and evidence and trust that there are some on this list who work similarly. --dawn seunosewa@inaira.com (Seun Osewa) wrote in message news:<ba87a3cf.0310092217.72098544@posting.google.com>... > dwolt@iserv.net (Dawn M. Wolthuis) wrote in message news:<6db906b2.0310091212.4f967cf5@posting.google.com>... > > I would suggest ditching the entire relational model (as both overly > > simplistic in its theory and overly complex in its implementation) and > > start with English (that is one of the other names for the GIRLS > > language). Note that language is also the starting point for putting > > data in XML documents, but it sure doesn't seem to be the starting > > point for XQuery, eh? > > > > --dawn > > Dawn M. Wolthuis > > www.tincat-group.com > > Please explain further. What do you really mean? Its natural for > everyone here to think every word in that post was a troll unless you > explain your views more clearly. You could not have expressed a more > unpopular/unsupportable combination of ideas! Exactly how would we go > about using language as a query tool? Is this AI? What would the > underlying model be knowing how redundant and imprecise language can > be? Tell what we may have missed. > > Seun Osewa
"Bob Badour" <bbadour@golden.net> wrote in message news:<XJGdnSwNRq8UxxaiU-KYuA@golden.net>... > "Mike Preece" <michael@preece.net> wrote in message > news:1b0b566c.0310131551.47ca0d47@posting.google.com... > > dwolt@iserv.net (Dawn M. Wolthuis) wrote in message > news:<6db906b2.0310130618.6920e737@posting.google.com>... > > > Bob Badour" <bbadour@golden.net> wrote in message > news:<O6WdnZ4P-_6I4RWiXTWJlg@golden.net>... > > > > "Dawn M. Wolthuis" <dwolt@iserv.net> wrote in message > > > > news:6db906b2.0310100813.65b872d8@posting.google.com... > > > > > Thank you, Seun, for asking your question with a bit of logic and > not > > > > > gut-reaction emotional baggage (and for also asking a question of me > > > > > off-list so I could ramble). I'll try to make this more suscinct. > > > > > > > > With all due respect, Dawn, you are an idiot. That statement has > nothing to > > > > do with my viscera or with my emotions; I write it with complete > dispassion. > > > > You simply are an idiot, and Seun simply lacks sufficient education to > > > > recognize that fact from what you have written. > > > > > > Thanks for the clarification, Bob. What seemed like an emotional > > > outburst from me was actually a logically-derived statement regarding > > > my capacity then, right? If I were you, I might be just a tad bit > > > concerned that I have taught many engineers calculus and you are > > > flying on planes or driving over bridges they designed. Let's hope > > > they learned it well in spite of me, eh? smiles. > > > --dawn > > > > > > Dawn M. Wolthuis > > > > Dawn: I'm sure you're not an idiot. > > Mike, you have expressed certainty contradicting the patently obvious > before. Your opinion is only a useful gauge insofar as you are wrong so much > more frequently than you are correct. Dawn: Bob professes to respect referential integrity!?
seunosewa@inaira.com (Seun Osewa) wrote in message news:<ba87a3cf.0310031052.77315052@posting.google.com>... > Hi, > > This is for relational database theory experts on one hand and > imlementers of real-world alications on the other hand. If there was > a chance to start again and design SQL afresh, for best > cleaness/power/performance what changes would you make? What would > _your_ query language (and the underlying database concept) look like? > > Seun Osewa > PS: I should want to post my ideas too for review but more > experienced/qualified people should come first Some ideas, links, and complaints about SQL and remaking or replacing it: http://www.c2.com/cgi/wiki?SqlFlaws -T-
Anthony, > And don't other databases have both theory and model? Actually, no, the "new" databases do not. The relational model is backed by relational algebra and relational calculus, plus a series of postulates and laws which have been refined and tested over 20 years. Not Object-Oriented databases nor XML "databases", nor Multi-Value databases have any body of theory behind them, mathematical or otherwise. I defy you to post a single paper that has a mathematical theory for MV or OODB, or even a firm set of laws that govern such a database. Nor is the industry moving toward developing such a theory; instead the marketeers of commercial OODB and XMLDB use a lot of ink to denigrate the idea of mathematical theory as antiquated and stuffy, and in one case even using their advertising clout to drive critical theorists off the pages of IT magazines (see Fabian Pascal's web page). Actually, amusingly enough, there is a body of theory backing XML databases, but it's not one any current devloper would want to admit to: the XML database is functionally identical to the Network Databases of the 1960's. Of course, Network Databases gave way, historically, to Relational for good reason. And MV databases, despite decades of existence, never developed any theory behind them at all, possibly because one is not possible; MV databases are entirely an ad-hoc creation designed to work around decade-old limits in computer processing. Pick is merely TextMagic revived and put on the web. Now, OODB could certainly *develop* a model and theory, and I think it's high time it did. The Zope project has amply demonstrated the usefulness of OODB for certain applications. But as long as there is no OODB calculus, and no industry-agreed model, and no ANSI standard language or interface, each and every OODB will be 100% incompatible with every other one ... severely limiting their utility. The importance of theory, model, and standards is *not* to be overstated in an industry where every year the industry-favorite commerical databases get more ad-hoc, further from the theory, and more callous in their disregard of international standards. FWIW, I share your dissatisfaction with SQL, but because it's not relational enough rather than the other way around. -- Josh Berkus Aglio Database Solutions San Francisco
>>>>> "Josh" == Josh Berkus <josh@agliodbs.com> writes: This is an unfair characterization of XML databases, and I can say this without accusations of bias for I vastly prefer working with the relational model. Josh> Actually, amusingly enough, there is a body of theory Josh> backing XML databases, but it's not one any currentdevloper Josh> would want to admit to: the XML database is functionally Josh> identical to the Network Databasesof the 1960's. Of Josh> course, Network Databases gave way, historically, to Josh> Relational for good reason. If you look at the academic research work, there have been gazillions of recent papers on XML database technology. All the major database vendors (Oracle, IBM and Microsoft) are investing fairly heavily in core-engine XMLDB technology. Finally, while it is true that some of XML db technology is evocative of network databases, XML databases are certainly more than network databases. For one, they are semi-structure .. in addition they present query language access to their data (although I'm not a big fan of XQuery). -- Pip-pip Sailesh http://www.cs.berkeley.edu/~sailesh
In an attempt to throw the authorities off his trail,
sailesh@cs.berkeley.edu (Sailesh Krishnamurthy) transmitted:
>>>>>> "Josh" == Josh Berkus <josh@agliodbs.com> writes:
> This is an unfair characterization of XML databases, and I can say
> this without accusations of bias for I vastly prefer working with
> the relational model.
>
> Josh> Actually, amusingly enough, there is a body of theory
> Josh> backing XML databases, but it's not one any current devloper
> Josh> would want to admit to: the XML database is functionally
> Josh> identical to the Network Databases of the 1960's. Of
> Josh> course, Network Databases gave way, historically, to
> Josh> Relational for good reason.
>
> If you look at the academic research work, there have been
> gazillions of recent papers on XML database technology. All the
> major database vendors (Oracle, IBM and Microsoft) are investing
> fairly heavily in core-engine XMLDB technology.
Ah, but do "papers" honestly indicate the emergence of some underlying
theoretical model for which fidelity could be evaluated?
Or merely that academics are looking to write papers on whatever
topics can attract research funding?
Half the articles in SIGOS have been about pretend applications of
Java to operating systems; why does it seem likely that the "database
academics" are any less affected by this?
I haven't yet seen a coherent "XML theory" emerge from the chaos.
It's not too dissimilar from the "object chaos;" the only works I am
aware of that try hard to provide theory behind "object orientedness"
are _A Theory of Objects_ by Abadi and Cardelli, and Benjamin Pierce's
book, _Types and Programming Languages_. After twenty-odd years of
object oriented programming, I find it quite appalling that there is
as little theoretical OO literature as there is.
> Finally, while it is true that some of XML db technology is
> evocative of network databases, XML databases are certainly more
> than network databases. For one, they are semi-structure .. in
> addition they present query language access to their data (although
> I'm not a big fan of XQuery).
CODASYL had a query system, albeit something that looked more like
assembly language than anything else...
--
output = reverse("gro.gultn" "@" "enworbbc")
http://www.ntlug.org/~cbbrowne/nonrdbms.html
Rules of the Evil Overlord #63. "Bulk trash will be disposed of in
incinerators, not compactors. And they will be kept hot, with none of
that nonsense about flames going through accessible tunnels at
predictable intervals." <http://www.eviloverlord.com/>
Sailesh, Warning: I get carried away in this response. I'm afraid that I'm a fond reader of Fabian Pascal and CJ Date, so I have far too much to say on the topic. So if you really care about XML databases, you should probably hold off on reading the rest until you're well-caffinated and in a cheerful frame of mind. Also, let me clarify that there is a distinction between using XML *as a* database, and putting XML documents into databases of other types. I find the latter obvious and sensible, but the former a silly and wrong-headed idea, and it's the pure-XML-database which I attack below. If you want to really have this out, I live in San Francisco and I love to argue. Coffee at Intermezzo? I'll buy. ------------------------------- > If you look at the academic research work, there have been gazillions > of recent papers on XML database technology. Point me to one which presents an algebra, calculus, or other mathematical underpinning of XML databases, and I will be happy to eat my words on this list. I can easily find lots of papers using google, but all of them are about *technical implementation* and do not provide a theoretical underpinning for XML databases. A few (such as Dan Suciu's paper) present some theory to back XQuery but it is presented entirely as an XML-based data access extension to SQL ... a role which seems fine to me. Others, even those cited by xmldb.org like the below, have rather lukewarm things to say on the topic, such as David Mertz, PhD: (http://www-106.ibm.com/developerworks/library/x-matters8/index.html) "XML is an extremely versatile data transport format, but despite high hopes for it, XML is mediocre to poor as a data storage and access format. ..." <snip> " ...XML has no inherent mechanism for enforcing constraints of this sort (DTDs and schemas are constraints of a different, more limited sort). Without constraints, you just have data, not a data model (to slightly oversimplify matters). ..." <snip> " ... In other words, go ahead and be excited by XML's promise of a universal data transport mechanism, but keep your backend data on something designed for it, like DB2 or Oracle (or on Postgres or MySQL for smaller-scale systems)." And this guy is cited by XMLDB.org? Perhaps not surprising, as among the 5 goals of XMLDB.org, development of a standard theory of XML databases is not present. > All the major database > vendors (Oracle, IBM and Microsoft) are investing fairly heavily in > core-engine XMLDB technology. So? Oracle, IBM and Microsoft also have SQL databases that do a terrible job of upholding the SQL standard, and their (at least Oracle's and Microsoft's) adherence is getting worse with successive versions rather than better. I wouldn't look to them for guidance. If they're spending millions on XML Databases, it's becuase it, however wrong-headed, is a fad and fads mean sales, and they don't want to take a chance on missing out. And these companies have backed plenty of useless technologies before; remember Microsoft's "Periodicals on CD"? Not that I'm against XML; as far as I'm concerned, for interchangable, searchable, and archival documents, XML is the greatest thing since sliced Beatles. I love XML-RPC for pushing data through HTTP, and I will happily be in the cheering squad for anyone who writes a set of OSS tools to extract data from XML docs stored in a PostgreSQL database, or to automate some-standard-XML-to-relational-data-and-back conversion. That is a good application of XML+Database ideas. XML databases, on the other hand, are an example of taking a good idea too far. XML is a great data transmission tool; it's a great document transformation tool; it's a good way to store documents. It is not, however, a good database. ------------------------------------------------------ -- Josh Berkus Aglio Database Solutions San Francisco
>>>>> "Christopher" == Christopher Browne <cbbrowne@acm.org> writes: Christopher> Ah, but do "papers" honestly indicate the emergence Christopher> of some underlying theoretical model forwhich Christopher> fidelity could be evaluated? Certainly. The model is that of semi-structured data, where often times schema isn't very clearly defined. It's a different model from the relational model - which I'm partial to. There are situations where the XML model does make sense. Christopher> Or merely that academics are looking to write papers Christopher> on whatever topics can attract researchfunding? Bash academics if you want. The truth is that industry is also working on it. As I said before, I have no axe to grind in this. I might be in academia now, but frankly I couldn't give a toss about XML. Christopher> Half the articles in SIGOS have been about pretend Christopher> applications of Java to operating systems;why does Christopher> it seem likely that the "database academics" are any Christopher> less affected by this? I think you are looking at the wrong publishing location :-) The premier venue for the OS community are the SOSP and OSDI conferences. Please look at the SOSP04 papers - you'll find fairly good systems work. BTW, I'm not sure which database papers you read - the premeer venues for database systems work are the SIGMOD and VLDB conferences. Christopher> CODASYL had a query system, albeit something that Christopher> looked more like assembly language thananything Please take a fair look at the XQuery data model and the XQuery language before comparing it with CODASYL. I will not admit (at least in public :-) to being a big fan of XQuery but that is because of certain details, not anything fundamental. -- Pip-pip Sailesh http://www.cs.berkeley.edu/~sailesh
sailesh@cs.berkeley.edu (Sailesh Krishnamurthy) writes: >>>>>> "Christopher" == Christopher Browne <cbbrowne@acm.org> writes: > Christopher> Ah, but do "papers" honestly indicate the emergence > Christopher> of some underlying theoretical model for which > Christopher> fidelity could be evaluated? > > Certainly. The model is that of semi-structured data, where often > times schema isn't very clearly defined. It's a different model from > the relational model - which I'm partial to. There are situations > where the XML model does make sense. > > Christopher> Or merely that academics are looking to write > Christopher> papers on whatever topics can attract research > Christopher> funding? > > Bash academics if you want. The truth is that industry is also > working on it. > > As I said before, I have no axe to grind in this. I might be in > academia now, but frankly I couldn't give a toss about XML. Fifteen years ago, there were _vital_ things going on in academia; Stonebraker's group was assortedly working on Postgres, Ingres, and Cohera. _Real_ OS research was going on, with active work on such systems as Sprite, BSD, Mach, with _many_ schools building their own OS kernels to explore different edges. The problem is that with the dearth of funding for "basic research," academia seems to be getting used as a partly-government-funded "development" area for the developments that industry hopes might work but where they would rather evade responsibility for the failures. > Christopher> Half the articles in SIGOS have been about pretend > Christopher> applications of Java to operating systems; why does > Christopher> it seem likely that the "database academics" are any > Christopher> less affected by this? > > I think you are looking at the wrong publishing location :-) The > premier venue for the OS community are the SOSP and OSDI > conferences. Please look at the SOSP04 papers - you'll find fairly > good systems work. When the first paragraph of the annual report contains the phrase ... despite the "operating systems" in our name ... I can't see this boding well for the area of "operating systems research" being particularly vital. They have had to expand it to some notion of "systems" that is sufficiently wide open that I'm not sure what it _wouldn't_ admit as "relevant." > BTW, I'm not sure which database papers you read - the premeer > venues for database systems work are the SIGMOD and VLDB > conferences. I haven't been subscribing to anything for a couple years now, TODS having pretty much fallen into irrelevance. I suppose I need to look to see if there is something worth following again. > Christopher> CODASYL had a query system, albeit something that > Christopher> looked more like assembly language than anything > > Please take a fair look at the XQuery data model and the XQuery > language before comparing it with CODASYL. I will not admit (at > least in public :-) to being a big fan of XQuery but that is because > of certain details, not anything fundamental. I have used it a bit for extracting data out of SOAP requests; I wasn't terribly impressed with it, finding that with the way it was remapping data, I would commonly have to code my own "tree walker" to do things _right_. It's not hard to do; when I had to modify my Galeon "bookmark walker" a few months ago when they changed from a custom XML form to RDF, I was pleasantly surprised to find that most of the changes involved changing names of tags in DEFCONSTANT declarations. (Guess the language :-).) Of course, "web bookmark" systems are likely to be structured similarly, so that this change would be easy shouldn't be much of a surprise. But when you need to write code, that indicates that there is some weakness to the model... -- let name="cbbrowne" and tld="libertyrms.info" in String.concat "@" [name;tld];; <http://dev6.int.libertyrms.com/> Christopher Browne (416) 646 3304 x124 (land)
> -----Original Message----- > From: Lauri Pietarinen [mailto:lauri.pietarinen@atbusiness.com] > Sent: Sunday, October 19, 2003 1:50 PM > To: pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] Dreaming About Redesigning SQL > > > Anthony W. Youngman wrote: > > >Well, as far as we MV'ers are concerned, performance IS a > problem with > >the relational approach. The attitude (as far as I can tell) with > >relational is to hide the actual DB implementation from the > >programmers. So it is a design "flaw" that it is extremely > easy for a > >programmer to do something stupid. And you need a DBA to try and > >protect the database from the programmers! > > > >As soon as a requirement for a database specifies extraction of the > >maximum power from the box, it OUGHT to rule out all the current > >relational databases. MV flattens it for it for performance. > As an MV > >programmer, I *KNOW* that I can find any thing I'm looking > for (or find > >out it doesn't exist) with just ONE disk seek. A relational > programmer > >has to ask the db "does this exist" and hope the db is > optimised to be > >able to return the result quickly. To quote the Pick FAQ > "SQL optimises > >the easy task of finding stuff in memory. Pick optimises the > hard task > >of getting it into memory in the first place". > > > So in your opinion, is the problem > > 1) SQL is so hard that the average programmer will not know > how to use it efficiently or This same average programmer will have the same difficulty with a pure, orthogonal and mathematically rigorous language. > 2) Relational (or SQL-) DBMS'es are just too slow > > If 2) then why don't we get a bit more concrete. Could you > give an example of a query that in your experience would be > too slow using a standard SQL database (e.g. Oracle, or > MySQL). We could then actually try it out on some machine and > compare. I suggest using the > customer-order-order_detail-product database > > If 1) I would like to hear some concrete examples. I once worked on a telephone customer support system that had 5 million lines of C++ and 5 million lines of SQL. It was too complex for the average person to comprehend. There are always going to be some queries that are too slow. If you put a key on everything, data modifications will go into the toilet. If you lack a key on a field required for a filtered query, you will have a table scan. Hence, there is no free lunch, and there will have to be compromises. However, due to the exponential expansion of compute power, the efficiency of SQL is the least of our worries. See this link, it makes for excellent reading: http://www.kurzweilai.net/articles/art0134.html?printable=1 > best regards, > Lauri Pietarinen > > > > ---------------------------(end of > broadcast)--------------------------- > TIP 9: the planner will ignore your desire to choose an index > scan if your > joining column's datatypes do not match >
Wol, > I think one MAJOR problem is that most (if not all) MV practitioners are > not formally qualified in computing ... <snip> > "Relational" is all about theory and proving things mathematically > correct. "MV" is all about engineering and getting the result. And if > that means pinching all the best ideas we can find from relational, then > we're engineers - of course we'll do it :-) So what you're saying is that you use MV databases becuase you find them easier to understand and set up than relational databases. Sure. For that matter, spreadsheets are easier to understand and set up than relational databases -- or MV databases, for that matter. I've no problem with the idea that MV databases are easier for the neophyte to understand. But you seem to be making this bizarre logical leap that "easier to understand without training" == "technically superior". And I noticed that you have completely backed off from your assertion that there was some kind of MV database theory without ever addressing it. > As soon as a requirement for a database specifies extraction of the > maximum power from the box, it OUGHT to rule out all the current > relational databases. MV flattens it for it for performance. As an MV > programmer, I *KNOW* that I can find any thing I'm looking for (or find > out it doesn't exist) with just ONE disk seek. A relational programmer > has to ask the db "does this exist" and hope the db is optimised to be > able to return the result quickly. Well, frankly, no. You're confusing your floundering because you find it hard to grasp relational database design and/or have a badly designed database with a performance problem. There are several people in our community running large genetics databases on PostgreSQL, using good relational design, and doing well with them. > To quote the Pick FAQ "SQL optimises > the easy task of finding stuff in memory. Pick optimises the hard task > of getting it into memory in the first place". And hope to the gods that nobody pulls the power cord. > "Relational" is all about theory and proving things mathematically > correct. "MV" is all about engineering and getting the result. And if > that means pinching all the best ideas we can find from relational, then > we're engineers - of course we'll do it :-) "Relational" is all about preserving the *long-term* integrity, utility, and accessability of your data. "MV" is all about getting an expedient result immediately and to heck with the future. Read some of E.F. Codd's original papers ... he was not concerned with expanding some obscure batch of mathematics, he was concerned with the problems of data that wasn't clean, or wasn't the same when you queried it twice, or data with rules that changed eraticaly over time, and wasn't portable at all. These are real, "engineering" problems that needed solving and relational theory solved it. > . Unless you can use set theory to predict > the future, relational has nothing to do with science ... I wasn't aware that clairvoyance was a tenet of the scientific method. Science is about reproducing reliable results, which is also what relational theory is about. To conclude: Your entire advocacy of Pick can be summed up as "I like Pick and find it easy to program, therefore it is superior." Well, Wol, that makes it superior for *you* but not for the rest of us. If you want to use Pick and not PostgreSQL, go for it; but if you barge in here and try to convince us that Pick is superior based strictly on your say-so, you've become one of the crazy preachers on Sproul Plaza. -- Josh Berkus Aglio Database Solutions San Francisco
>>>>> "Josh" == Josh Berkus <josh@agliodbs.com> writes: >> "Relational" is all about theory and proving things >> mathematically correct. "MV" is all about engineering and >> getting the result. And if that means pinching all the best >> ideas we can find from relational, then we're engineers- of >> course we'll do it :-) Josh> "Relational" is all about preserving the *long-term* Josh> integrity, utility, and accessability of your data. "MV" is Josh> all about getting an expedient result immediately and to Josh> heck with the future. To emphasize the *long-term* - relational is all about separating physical database representation from a logical view of the data. Josh, I'll be happy to meet at Intermezzo - after Nov 24 perhaps. I have a conference deadline .. the papers that we write "just to secure funding" - your tax dollars at work. Long live the NSF ! BTW, I'm not necessarily that far from your views. There is, however, more to an XML database than just storing data - relational databases do just fine there. The tools that are used to build database systems have other uses - for instance XML message brokers. -- Pip-pip Sailesh http://www.cs.berkeley.edu/~sailesh
Cygwin, 7.3.4 This thing is really KILLING us and our customers. In pgerr.log this always go together: WARNING: ShmemAlloc: out of memory ERROR: FreeSpaceMap hashtable out of memory Theses errors usually take place on INSERT statements like this one: INSERT INTO params (param_id,map_id,param_key,param_value) VALUES (0,858,'MAP_NAME','New Map') params is just a simple table (no triggers or rules involved): CREATE TABLE map_11.params ( param_id int4 NOT NULL, map_id int4 NOT NULL, param_key varchar(1024), param_value text, CONSTRAINT params_pkey PRIMARYKEY (param_id) ) WITH OIDS; The db server machine has 4GB of physical memory and more than half is always available. Number of connection is 64 Number of buffers was increased to 81920 (but this does not help) Is there any parameters that can be configured to increase available shared memory size? Any help is greatly appreciated.
> "Bob" <bbadour@golden.net> spewed forth... > > We achieved 8 times the performance with exactly the same > hardware. What the hell is this idiot talking about us > relying on hardware? He is a moron. You will do everyone > a favour if you just bounce him off the bottom of your > killfile. > ... > [profanity-laced responses deleted] > ... It's about time to take this discussion private, guys. It has long since stopped being on-topic to hacking PostgreSql internals. Darren
Darren King wrote: >>"Bob" <bbadour@golden.net> spewed forth... >> >>We achieved 8 times the performance with exactly the same >>hardware. What the hell is this idiot talking about us >>relying on hardware? He is a moron. You will do everyone >>a favour if you just bounce him off the bottom of your >>killfile. >>... >>[profanity-laced responses deleted] >>... > > > It's about time to take this discussion private, guys. > It has long since stopped being on-topic to hacking PostgreSql > internals. > > Darren I've learned that a feed into the postgresql-hackers mailing list from comp.databases.postgresql.hackers can be easily spotted by its astonishing lack of civility and intelligent discourse... :-( Mike Mascari mascarm@mascari.com
Dawn M. Wolthuis wrote: > So, nope, I'm not trolling. I've been doing some research the past > couple of years and I'm convinced that it is time to do something new > (and yet old) with data persistence. Perhaps. But before you go down that road, you have to answer the following simple, yet possibly difficult-to-answer, question: What problem are you trying to solve? "Data persistence" is far too vague a term to be meaningful in its own right -- the term needs some context to have any real meaning here. We store data for a reason. Similarly, we retrieve it for a reason. The data we're interested in looking for and the way we are interested in looking for it will have a huge impact on any data retrieval solution one could craft. The relational model of data storage and retrieval is just a model, highly suitable to some things and not suitable at all to others. The amount of development work that has gone into it and the amount of use it has gotten shows that the relational model is actually reasonably good at meeting many of the data storage and retrieval needs that people have. As with any method, it has tradeoffs and drawbacks, There is no magic bullet and there never will be (or so experience says). I have no reason to believe that the problem of data persistence and retrieval is any exception to that. If you have a particular set of data retrieval problems in mind that you wish to solve, by all means feel free to develop the mathematical foundation to solve them. Feel free to tell us that the relational model is not suitable for that set of problems -- we might even agree with you on that. But don't make the claim that the relational model is lacking as a result of not being a storage and retrieval method that is suitable to all problems, and that there is a Better Way that will Solve Everything. Many have made such claims about many different technologies. They were wrong, too. I may be misreading you and responding to arguments you aren't making or implying, but if so I doubt I'm alone, based on some of the other responses I've seen here. By the way, language is only a means of expression, and the only sort of question (relevant to this discussion, anyway) that a language is the answer to is "what's the best way for me to express X?". It is neither an answer to the question of how to retrieve data nor is it a solution to the problem of storing data in a persistent manner. The answer to the question of how best to query data is certainly going to be a language, but the specific language one comes up with in answer to the question will depend on what the person asking wants. "English" is likely to be the best answer only under certain circumstances. SQL is likely to be the best answer (or, at least, a very good answer) only under other circumstances. It just depends. But as with any solution to any problem, there is no one-size-fits-all solution. As a mathematician, you should know this: the English language is horribly inadequate for expressing mathematical concepts. That's why mathematicians don't use it as their primary language for doing math. Why, then, should we expect English, or Java, or any other language to be any better for performing certain kinds of queries against data than some other, more directed language? Say what you want about SQL, but at least it was designed with querying table-based data in mind and is at least somewhat good at its job. -- Kevin Brown kevin@sysexperts.com
"Yurgis Baykshtis" <ybaykshtis@micropat.com> writes:
> In pgerr.log this always go together:
> WARNING: ShmemAlloc: out of memory
> ERROR: FreeSpaceMap hashtable out of memory
If you have a large number of tables in your database, it might be that
you need to increase max_locks_per_transaction.
regards, tom lane