Обсуждение: work on extending PostgreSQL to data integration systems

Поиск
Список
Период
Сортировка

work on extending PostgreSQL to data integration systems

От
"Lee Riquelmei"
Дата:
Hello, everybody,<br /><br />I am developing a postgresql-based distributed relational data integration system which is
likethe IBM DB2 Information Integrator. And I have started a pgfoundry project: <a
href="http://pgfoundry.org/projects/ignite">http://pgfoundry.org/projects/ignite</a> .(codes on it have already been
obsolete.)<br/><br />The goal of the system is to provide a collection of virtual views in a pgsql database to
integratedata from various distributed, heterogeneous and autonomous data sources. To achieve this goal, I did my work
intwo aspects: <br />(1) outside pgsql:  The system utilizes various wrappers to interact with data sources. A
well-definedwrapper interface is presented, which allows for implementations of concrete wrappers using any language ,
suchas pl/java. In nature, a wrapper is a collection of pgsql functions which consists of the function for obtaining
metadataof a data source, the function for exectuing SQL in data source and etc. <br /><br />(2) inside pgsql: Several
modifictionsto pgsql is used to improve the performance of processing data integration queries including the pipelined
FunctionScanoperator and support of query shipping(selection/projection/sort). <br /><br />The developing work is still
ongoing,although an initial version of the system has already been used in a real application in China. This version
consistsof a wrapper for PostgreSQL using libpq and a wrapper for MS SQL SERVER using pl/java and JDBC.  Currently, I
havenot published all sources and not provided good documents in the pgfoundry projects. But I will do it quickly. If
anyoneis interested in the system, pls contact me. <br /><br />I have a question. Does the PostgreSQL project have any
planof adding data integration functionalities in the PostgreSQL system as other DBMS vendors did?  <br /><br /><br
/>Thanks.<br/><br />Yours sincerely,<br /><br />Lee Rubao <br /><a
href="mailto:lirubao@software.ict.ac.cn">lirubao@software.ict.ac.cn</a><br/><br />2006-11-1<br /><br /><br /><br /> 

Re: work on extending PostgreSQL to data integration systems

От
David Fetter
Дата:
On Wed, Nov 01, 2006 at 02:13:47AM +0800, Lee Riquelmei wrote:
> 
>    Hello, everybody,
>    I am developing a postgresql-based distributed relational data integration
>    system which is like the IBM DB2 Information Integrator. And I have started
>    a pgfoundry project: [1]http://pgfoundry.org/projects/ignite .(codes on it
>    have already been obsolete.)
>    The goal of the system is to provide a collection of virtual views in a
>    pgsql database to integrate data from various distributed, heterogeneous and
>    autonomous data sources.

You can do this now in a crude way using DBI-Link, which I am
developing now.
<http://pgfoundry.org/projects/dbi-link>

I'm very interested in your work, as DBI-Link's methods, all of which
act in user space, are somewhat crude.  It has no hooks into the
planner, the optimizer, etc.  It does have facilities which could use
such hooks if they were made available.

>    To achieve this goal, I did my work in two aspects:
>    (1) outside pgsql:  The system utilizes various wrappers to interact with
>    data sources. A well-defined wrapper interface is presented, which allows
>    for  implementations of concrete wrappers using any language , such as
>    pl/java. In nature, a wrapper is a collection of pgsql functions which
>    consists  of the function for obtaining metadata of a data source, the
>    function for exectuing SQL in data source and etc.
>    (2) inside pgsql: Several modifictions to pgsql is used to improve the
>    performance of processing data integration queries including the pipelined
>    FunctionScan operator and support of query
>    shipping(selection/projection/sort).
>    The developing work is still ongoing, although an initial version of the
>    system has already been used in a real application in China. This version
>    consists of a wrapper for PostgreSQL using libpq and a wrapper for MS SQL
>    SERVER using pl/java and JDBC.  Currently, I have not published all sources
>    and not provided good documents in the pgfoundry projects. But I will do it
>    quickly. If anyone is interested in the system, pls contact me.
>    I have a question. Does the PostgreSQL project have any plan of adding data
>    integration functionalities in the PostgreSQL system as other DBMS vendors
>    did?

The SQL:2003 standard includes a section called SQL/MED (Management of
External Data).  I suspect that your implementation would be more
likely to go into the core if it were made compatible with this
standard.

Regards,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
phone: +1 415 235 3778        AIM: dfetter666                             Skype: davidfetter

Remember to vote!