Обсуждение: [PROPOSAL] Add SCTP network protocol to postgresql backend and frontend

Hello,

I want to propose to add sctp network protocol to backend and frontend code.

Light Sctp description:

Sctp is a layer 3 network protocol over ip networks, similar to tcp, udp and others. It's message oriented protocol,
unliketcp and udp that are data stream oriented, and it ensures that a message reach its destination, as tcp does with
packets.Each sctp packet can have multiple messages or parts of messages inside. It provides multihoming, dinamically
bindand unbind network devices (eth0, eth1, wifi...), mix ipv4 and ipv6, congestion control algorithms similar to tcp,
messagereliability and administration (message time to live, retries on failure...). A connection between server and
clientis called association.
 

Sctp support exists in Linux since 2.4+, FreeBSD 7+, Solaris10+ and Cisco, Juniper, F5 and others routers. Windows and
MacOSXneeds (AFAIK) third-party drivers.
 

Benefits:

Dynamic multihoming, modifiable at run time, don't need aggregate links at OS level or shutdown servers/clients for a
hardwareor topology network change.
 
Message oriented connection.
Message reliability.
Inmune to SYN floods that affect tcp.
Assimetric multihoming, a client with 4 links(3x 1GbEth + wifi) can connect to a server with 1 link (10GbEth).
Metadata connection messages.

Problems:

Windows and MacOSX needs a third-party drivers.
Can't use TLS, encryption of tls record n depends on previous record. In sctp order and reliability of packets is not
assuredand packet content may change. As sctp is message oriented it needs DTLS-SCTP (not in this proposal) where
encryptionof packet n is independent from any other. See RFC 6083
 
Some network cards and drivers don't support hardware acceleration, cpu does crc check/calculation.
Firewalls needs sctp rules.

Implementation:

The tcp code resides in src/backend/libpq/pqcomm.c (unix/windows sockets and SSL socket portion) and
src/backend/libpq/ip.c(ipv6). Similar in frontend. 
 
Need new GUCs for sctp configuration, and a way to modify them at runtime.

I don't know how you want it, if you accept this proposal:

a) replicate pqcomm.c replacing tcp code with sctp in a new file pqcommsctp.c (full sctp, not compatibility tcp hack),
b) inside pqcomm.c and ip.c,
c) other

My main develop environment is FreeBSD10 and my contractor uses FreeBSD10 too but I'll check it works on some Linux
distros.


Useful links:

http://www.bsdcan.org/2008/schedule/attachments/44_bsdcan_sctp.pdf
https://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol
https://tools.ietf.org/html/rfc6083

Thanks

---   ---
Eduardo Morras <emorrasg@yahoo.es>

On 03/23/2016 01:55 PM, Eduardo Morras wrote:
> Benefits:
>
> Dynamic multihoming, modifiable at run time, don't need aggregate links at OS level or shutdown servers/clients for a
hardwareor topology network change.
 
> Message oriented connection.
> Message reliability.
> Inmune to SYN floods that affect tcp.
> Assimetric multihoming, a client with 4 links(3x 1GbEth + wifi) can connect to a server with 1 link (10GbEth).
> Metadata connection messages.

While SCTP has some nice advantages in general (I think it is a pity it 
is not used more) I wonder how well these benefits translate into the 
database space. Many databases are run either in a controlled server 
environment with no direct access from the Internet, or locally on the 
same machine as the application. In those environments you generally do 
not have to worry about SYN floods or asymmetric links.

Do you have any specific use case in mind?

Andreas

Andreas Karlsson escribió:
> On 03/23/2016 01:55 PM, Eduardo Morras wrote:
> >Benefits:
> >
> >Dynamic multihoming, modifiable at run time, don't need aggregate links at OS level or shutdown servers/clients for
ahardware or topology network change.
 
> >Message oriented connection.
> >Message reliability.
> >Inmune to SYN floods that affect tcp.
> >Assimetric multihoming, a client with 4 links(3x 1GbEth + wifi) can connect to a server with 1 link (10GbEth).
> >Metadata connection messages.
> 
> While SCTP has some nice advantages in general (I think it is a pity it is
> not used more) I wonder how well these benefits translate into the database
> space. Many databases are run either in a controlled server environment with
> no direct access from the Internet, or locally on the same machine as the
> application. In those environments you generally do not have to worry about
> SYN floods or asymmetric links.

That might or might not be the most common cases, but replication across
the ocean and similar long-range setups are a reality today and their use
will only increase.

I wonder about message ordering.  Is it possible to get messages out of
order in SCTP?  Say if you have an ordered resultset stream from the
server, it would be disastrous to get the data messages out of order.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 03/23/2016 02:13 PM, Alvaro Herrera wrote:
> Andreas Karlsson escribió:
>> On 03/23/2016 01:55 PM, Eduardo Morras wrote:
>>> Benefits:
>>>
>>> Dynamic multihoming, modifiable at run time, don't need aggregate links at OS level or shutdown servers/clients for
ahardware or topology network change.
 
>>> Message oriented connection.
>>> Message reliability.
>>> Inmune to SYN floods that affect tcp.
>>> Assimetric multihoming, a client with 4 links(3x 1GbEth + wifi) can connect to a server with 1 link (10GbEth).
>>> Metadata connection messages.
>>
>> While SCTP has some nice advantages in general (I think it is a pity it is
>> not used more) I wonder how well these benefits translate into the database
>> space. Many databases are run either in a controlled server environment with
>> no direct access from the Internet, or locally on the same machine as the
>> application. In those environments you generally do not have to worry about
>> SYN floods or asymmetric links.
>
> That might or might not be the most common cases, but replication across
> the ocean and similar long-range setups are a reality today and their use
> will only increase.

Agreed. When I reread my message I realized that I implied things I did 
not mean. People run databases today in the cloud and, as you said, long 
distance replication will only get more common. What I am actually 
curious about is how the advantages of SCTP translate into the database 
space.

> I wonder about message ordering.  Is it possible to get messages out of
> order in SCTP?  Say if you have an ordered resultset stream from the
> server, it would be disastrous to get the data messages out of order.

Message ordering is an optional feature in SCTP, so if you need message 
ordering you can get it.

Andreas

On Wed, 23 Mar 2016 14:03:31 +0100
Andreas Karlsson <andreas@proxel.se> wrote:

> On 03/23/2016 01:55 PM, Eduardo Morras wrote:
> > Benefits:
> >
> > Dynamic multihoming, modifiable at run time, don't need aggregate
> > links at OS level or shutdown servers/clients for a hardware or
> > topology network change. Message oriented connection. Message
> > reliability. Inmune to SYN floods that affect tcp.
> > Assimetric multihoming, a client with 4 links(3x 1GbEth + wifi) can
> > connect to a server with 1 link (10GbEth). Metadata connection
> > messages.
>
> While SCTP has some nice advantages in general (I think it is a pity
> it is not used more) I wonder how well these benefits translate into
> the database space. Many databases are run either in a controlled
> server environment with no direct access from the Internet, or
> locally on the same machine as the application. In those environments
> you generally do not have to worry about SYN floods or asymmetric
> links.
>
> Do you have any specific use case in mind?

The main use case is change the network topology on the fly, without shutting down postgresql server, postgresql
middleware,or any of the applications that uses it through libpq.  

Specific use case, backup is backup server on OS level or pgdump, not postgresql slave, (hope it don't wraps)

backup <-> postgresql <-> middleware <-> client apps <-> backup

At peak times you need all nics connected between postgresql servers and middleware and client apps,

backup <-> postgresql <=> middleware <=> client apps <-> backup

at night or idle time or while backup, you can reassign the nics to get more network bandwith to backup server

backup <=> postgresql <-> middleware <-> client apps <=> backup

On a crash restore, all nics are used from backup to servers

backup <?> postgresql < > middleware < > client apps <?> backup

> Andreas


---   ---
Eduardo Morras <emorrasg@yahoo.es>

On Wed, 23 Mar 2016 10:13:42 -0300
Alvaro Herrera <alvherre@2ndquadrant.com> wrote:

> Andreas Karlsson escribió:
> > On 03/23/2016 01:55 PM, Eduardo Morras wrote:
> > >Benefits:
> > >
> > >Dynamic multihoming, modifiable at run time, don't need aggregate
> > >links at OS level or shutdown servers/clients for a hardware or
> > >topology network change. Message oriented connection. Message
> > >reliability. Inmune to SYN floods that affect tcp.
> > >Assimetric multihoming, a client with 4 links(3x 1GbEth + wifi)
> > >can connect to a server with 1 link (10GbEth). Metadata connection
> > >messages.
> >
> > While SCTP has some nice advantages in general (I think it is a
> > pity it is not used more) I wonder how well these benefits
> > translate into the database space. Many databases are run either in
> > a controlled server environment with no direct access from the
> > Internet, or locally on the same machine as the application. In
> > those environments you generally do not have to worry about SYN
> > floods or asymmetric links.
>
> That might or might not be the most common cases, but replication
> across the ocean and similar long-range setups are a reality today
> and their use will only increase.
>
> I wonder about message ordering.  Is it possible to get messages out
> of order in SCTP?  Say if you have an ordered resultset stream from
> the server, it would be disastrous to get the data messages out of
> order.

Message ordering is optional, server decides if clients can use messages out of order as received or strictly in the
sameorder as sended. 

> --
> Álvaro Herrera                http://www.2ndQuadrant.com/
> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

---   ---
Eduardo Morras <emorrasg@yahoo.es>