Обсуждение: Feature Proposal: Add ssltermination parameter for SNI-based LoadBalancing

Поиск
Список
Период
Сортировка

Feature Proposal: Add ssltermination parameter for SNI-based LoadBalancing

От
Lukas Meisegeier
Дата:
Hi,

I try to host multiple postgresql-servers on the same ip and the same
port through SNI-based load-balancing.
Currently this is not possible because of two issues:
1. The psql client won't set the tls-sni-extension correctly
(https://www.postgresql.org/message-id/20181211145240.GL20222%40redhat.com)
2. The psql connection protocol implements a SSLRequest in plain text
before actually opening a connection.

The first issue is easily solvable by calling
`SSL_set_tlsext_host_name(conn->ssl,
conn->connhost[conn->whichhost].host)` before opening the connection.

The second issue is also solvable through a new parameter
"ssltermination" which if set to "proxy" will skip the initial
SSLRequest and connects directly through ssl.
The default value would be "server" which changes nothing on the
existing behaviour.

I compiled the psql-client with these changes and was able to connect to
2 different databases through the same ip and port just by changing the
hostname.

This fix is important to allow multiple postgres instances on one ip
without having to add a port number.

I implemented this change on a fork of the postgres mirror on github:
https://github.com/klg71/mayope_postgres

The  affected files are:
- src/interfaces/libpq/fe-connect.c (added ssltermination parameter)
- src/interfaces/libpq/libpq-int.h (added ssltermination parameter)
- src/interfaces/libpq/fe-secure-openssl.c (added tls-sni-extension)

I appended the relevant diff.

Best Regards
Lukas

Вложения

Re: Feature Proposal: Add ssltermination parameter for SNI-based LoadBalancing

От
Heikki Linnakangas
Дата:
On 10/12/2020 17:49, Lukas Meisegeier wrote:
> I try to host multiple postgresql-servers on the same ip and the same
> port through SNI-based load-balancing.
> Currently this is not possible because of two issues:
> 1. The psql client won't set the tls-sni-extension correctly
> (https://www.postgresql.org/message-id/20181211145240.GL20222%40redhat.com)
> 2. The psql connection protocol implements a SSLRequest in plain text
> before actually opening a connection.
> 
> The first issue is easily solvable by calling
> `SSL_set_tlsext_host_name(conn->ssl,
> conn->connhost[conn->whichhost].host)` before opening the connection.
> 
> The second issue is also solvable through a new parameter
> "ssltermination" which if set to "proxy" will skip the initial
> SSLRequest and connects directly through ssl.
> The default value would be "server" which changes nothing on the
> existing behaviour.

Don't you need backend changes as well? The backend will still expect 
the client to send an SSLRequest. Or is the connection from the proxy to 
the actual server unencrypted?

It's not very nice that the client needs to set special options, 
depending on whether the server is behind a proxy or not. Could you 
teach the proxy to deal with the SSLRequest message?

Perhaps we should teach the backend to accept a TLS ClientHello 
directly, without the SSLRequest message. That way, the client could 
send the ClientHello without SSLRequest, and the proxy wouldn't need to 
care about SSLRequest. It would also eliminate one round-trip from the 
protocol handshake, which would be nice. A long deprecation/transition 
period would be needed before we could make that the default behavior, 
but that's ok.

- Heikki



Re: Feature Proposal: Add ssltermination parameter for SNI-based LoadBalancing

От
Lukas Meisegeier
Дата:
Hey Heikki,

thanks for providing feedback :)
The traffic between proxy and psql-server is unencrypted thats why I
don't need to patch the server.
I tried returning a fixed response on the first plain SSLRequest
forwarding it to a psql-server with ssl enabled an tried to switch then
on the ssl connection startup but that didn't work out. I guess its
because the psql-server won't accept an ssl connection if its not
requested via SSLRequest.
I would definitly appreciate if the psql-server could accept the
TLS-client hello directly but we would still need to set the
tls-sni-extension correctly.
Perhaps we could rename the parameter to "sslplainrequest(yes/no)" and
start with making the plain SSLRequest optional in the psql-server.

Best Regards
Lukas


Am 11-Dec-20 um 14:26 schrieb Heikki Linnakangas:
> On 10/12/2020 17:49, Lukas Meisegeier wrote:
>> I try to host multiple postgresql-servers on the same ip and the same
>> port through SNI-based load-balancing.
>> Currently this is not possible because of two issues:
>> 1. The psql client won't set the tls-sni-extension correctly
>> (https://www.postgresql.org/message-id/20181211145240.GL20222%40redhat.com)
>>
>> 2. The psql connection protocol implements a SSLRequest in plain text
>> before actually opening a connection.
>>
>> The first issue is easily solvable by calling
>> `SSL_set_tlsext_host_name(conn->ssl,
>> conn->connhost[conn->whichhost].host)` before opening the connection.
>>
>> The second issue is also solvable through a new parameter
>> "ssltermination" which if set to "proxy" will skip the initial
>> SSLRequest and connects directly through ssl.
>> The default value would be "server" which changes nothing on the
>> existing behaviour.
>
> Don't you need backend changes as well? The backend will still expect
> the client to send an SSLRequest. Or is the connection from the proxy to
> the actual server unencrypted?
>
> It's not very nice that the client needs to set special options,
> depending on whether the server is behind a proxy or not. Could you
> teach the proxy to deal with the SSLRequest message?
>
> Perhaps we should teach the backend to accept a TLS ClientHello
> directly, without the SSLRequest message. That way, the client could
> send the ClientHello without SSLRequest, and the proxy wouldn't need to
> care about SSLRequest. It would also eliminate one round-trip from the
> protocol handshake, which would be nice. A long deprecation/transition
> period would be needed before we could make that the default behavior,
> but that's ok.
>
> - Heikki



Re: Feature Proposal: Add ssltermination parameter for SNI-based LoadBalancing

От
Heikki Linnakangas
Дата:
On 11/12/2020 16:46, Lukas Meisegeier wrote:
> Hey Heikki,
> 
> thanks for providing feedback :)
> The traffic between proxy and psql-server is unencrypted thats why I
> don't need to patch the server.

Ok.

> I tried returning a fixed response on the first plain SSLRequest
> forwarding it to a psql-server with ssl enabled an tried to switch then
> on the ssl connection startup but that didn't work out. I guess its
> because the psql-server won't accept an ssl connection if its not
> requested via SSLRequest.

Your proxy could receive the client's SSLRequest message, and respond 
with a single byte 'S'. You don't need to forward that to the real 
PostgreSQL server, since the connection to the PostgreSQL server is 
unencrypted. Then perform the TLS handshake, and forward all traffic to 
the real server only after that.

Client: -> SSLRequest
  Proxy: <- 'S'
Client: -> TLS ClientHello
  Proxy: [finish TLS handshake]

- Heikki



Re: Feature Proposal: Add ssltermination parameter for SNI-based LoadBalancing

От
Lukas Meisegeier
Дата:
Thanks for the provided ideas :)
I use HaProxy for my load-balancing and unfortunately I can't define
that I want to listen on a port for both ssl and non ssl requests.
That means if I try to return a fixed response 'S' on the SSLRequest it
fails with an SSL-Handshake failure cause the server expects a ssl message.

I searched for some way to forward to a default backend on ssl failure
but this seems to be no use-case for haproxy and isn't supported.

I also didn't find any other tcp-loadbalancer, which could handle this
type of ssl-failure fallback.

My only option would therefore be to write a custom loadbalancer for
this usecase, which is not really feasible given the amount of features
of haproxy I plan to use.

I have to say the psql ssl handshake procedure is really unique and
challenging :D

The tool stunnel is capable of this protocol but I can't do sni-based
loadbalancing with it so this is kind of a dead end here.

Lukas

Am 11-Dec-20 um 16:44 schrieb Heikki Linnakangas:
> On 11/12/2020 16:46, Lukas Meisegeier wrote:
>> Hey Heikki,
>>
>> thanks for providing feedback :)
>> The traffic between proxy and psql-server is unencrypted thats why I
>> don't need to patch the server.
>
> Ok.
>
>> I tried returning a fixed response on the first plain SSLRequest
>> forwarding it to a psql-server with ssl enabled an tried to switch then
>> on the ssl connection startup but that didn't work out. I guess its
>> because the psql-server won't accept an ssl connection if its not
>> requested via SSLRequest.
>
> Your proxy could receive the client's SSLRequest message, and respond
> with a single byte 'S'. You don't need to forward that to the real
> PostgreSQL server, since the connection to the PostgreSQL server is
> unencrypted. Then perform the TLS handshake, and forward all traffic to
> the real server only after that.
>
> Client: -> SSLRequest
>   Proxy: <- 'S'
> Client: -> TLS ClientHello
>   Proxy: [finish TLS handshake]
>
> - Heikki



Re: Feature Proposal: Add ssltermination parameter for SNI-based LoadBalancing

От
Heikki Linnakangas
Дата:
On 12/12/2020 13:52, Lukas Meisegeier wrote:
> Thanks for the provided ideas :)
> I use HaProxy for my load-balancing and unfortunately I can't define
> that I want to listen on a port for both ssl and non ssl requests.

Could you configure HaProxy to listen on separate ports for SSL and 
non-SSL connections, then? And forward both to the same Postgres server.

> That means if I try to return a fixed response 'S' on the SSLRequest it
> fails with an SSL-Handshake failure cause the server expects a ssl message.

That doesn't sound right to me, or perhaps I have misunderstood what you 
mean. If you don't send the SSLRequest to the Postgres server, but "eat" 
it in the proxy, the Postgres server will not try to do an SSL handshake.

> I have to say the psql ssl handshake procedure is really unique and
> challenging :D

Yeah. IMAP and SMTP can use "STARTTLS" to switch an unencrypted 
connection to encrypted, though. That's pretty similar to the 
'SSLRequest' message used in the postgres protocol.

- Heikki



Re: Feature Proposal: Add ssltermination parameter for SNI-based LoadBalancing

От
Lukas Meisegeier
Дата:
I liked the idea with separate ports for ssl and non ssl requests and
tried it with haproxy.
The psql-client connects with haproxy and receives the fixed 'S' byte
response. After that he tried to continue on the same connection and
doens't open a new one. This crashes the connection because haproxy
expects a new tcp connection.


psqlClient: opens connection (ARP: SYN)
haproxy: accepts connection (ARP: SYN ACK)
psqlClient: confirmes the connection (ARP: ACK)
psqlClient: sends SSLRequest
haproxy: sends confirmation (ARP: ACK)
haproxy: sends fixed byte response ('S')
haproxy: closes connection (ARP: FIN, ACK)
psqlclient: confirmed fixed byte response (ARP: ACK)
psqlclient: sends ssl hello request --> error connection already
closed("psql: error: SSL SYSCALL error: No error (0x00000000/0))

In my eyes the problem lies in upgrading the connection rather then
opening a new one.

Am 14-Dec-20 um 14:50 schrieb Heikki Linnakangas:
> On 12/12/2020 13:52, Lukas Meisegeier wrote:
>> Thanks for the provided ideas :)
>> I use HaProxy for my load-balancing and unfortunately I can't define
>> that I want to listen on a port for both ssl and non ssl requests.
>
> Could you configure HaProxy to listen on separate ports for SSL and
> non-SSL connections, then? And forward both to the same Postgres server.
>
>> That means if I try to return a fixed response 'S' on the SSLRequest it
>> fails with an SSL-Handshake failure cause the server expects a ssl
>> message.
>
> That doesn't sound right to me, or perhaps I have misunderstood what you
> mean. If you don't send the SSLRequest to the Postgres server, but "eat"
> it in the proxy, the Postgres server will not try to do an SSL handshake.
>
>> I have to say the psql ssl handshake procedure is really unique and
>> challenging :D
>
> Yeah. IMAP and SMTP can use "STARTTLS" to switch an unencrypted
> connection to encrypted, though. That's pretty similar to the
> 'SSLRequest' message used in the postgres protocol.
>
> - Heikki



Re: Feature Proposal: Add ssltermination parameter for SNI-based LoadBalancing

От
Lukas Meisegeier
Дата:
Hey,

whats the state of this? Can we start working out a plan to remove the
inital SSLRequest from the connection protocol or is there any reason to
keep it?

I would start by removing the need of the SSLRequest in the psql-server
if its started with a special parameter(ssl-only or so).
Simultaneously I would add a parameter to this disable the SSLRequest in
the client as well.

Later we could make this behaviour default for psql-server with
ssl-enabled and clients and some far time ahead we could remove the
implementation of SSLRequest in both server and client.

What are your thaughts about this?

Best Regards