Обсуждение: [BUGS] BUG #14686: OpenSSL 1.1.0+ breaks PostgreSQL's sslcompressionassumption, defaults to SSL_OP_NO_COMPRESSION
[BUGS] BUG #14686: OpenSSL 1.1.0+ breaks PostgreSQL's sslcompressionassumption, defaults to SSL_OP_NO_COMPRESSION
От
greenreaper@hotmail.com
Дата:
The following bug has been logged on the website: Bug reference: 14686 Logged by: Laurence Parry Email address: greenreaper@hotmail.com PostgreSQL version: 9.6.3 Operating system: Debian 9 'stretch', Linux 4.9.25, libc 2.24 Description: PostgreSQL is documented to compress data over SSL connections by default where possible: https://www.postgresql.org/docs/9.6/static/libpq-connect.html#LIBPQ-CONNECT-SSLCOMPRESSION PostgreSQL assumes that OpenSSL 0.9.8+ will compress with zlib if it can. However, OpenSSL 1.1.0+ - already in Manjaro, and the upcoming Debian 9 'Stretch', Fedora 26, and Tails 3.0 - sets SSL_OP_NO_COMPRESSION by default to discourage the CRIME attack: https://www.openssl.org/news/changelog.html#x7 *) CRIME protection: disable compression by default, even if OpenSSL is compiled with zlib enabled. Applications canstill enable compression by calling SSL_CTX_clear_options(ctx, SSL_OP_NO_COMPRESSION), or by using the SSL_CONFlibrary to configure compression. PostgreSQL does not anticipate this, and so fails to compress the connection even if zlib support is compiled into OpenSSL. This is not immediately obvious, unless you are watching networking graphs. Restoring compression in PostgreSQL requires patching out the new default in the depths of OpenSSL and recompiling (rather than just reconfiguring and recompiling). == Possible PostgreSQL code fix == If the connection string includes sslcompression=0, initialize_SSL() in pqlib disables SSL compression by setting SSL_OP_NO_COMPRESSION: https://doxygen.postgresql.org/fe-secure-openssl_8c_source.html#l01278 I believe there also needs to be an 'else' case to *clear* SSL_OP_NO_COMPRESSION with: SSL_clear_options(conn->ssl, SSL_OP_NO_COMPRESSION); in the case that sslcompression is unset or not '0', rather than rely on the default. PostgreSQL _could_ clear it if sslcompression is '1', rather than unset or not '0', but that would mean more people not using SSL compression. From 9.2 onwards, the documentation has 1 as the default; so many will not have set it, assuming that it will [continue to] work. Distributions will not use PostgreSQL 10 until it is released, while OpenSSL 1.1.0 is already in use with PostgreSQL 9.6 (and might be used with older versions): https://distrowatch.com/search.php?pkg=openssl&relation=similar&pkgver=1.1&distrorange=InAny#pkgsearch [In addition, PostgreSQL currently won't work as described if a distribution or end-user has made a similar change to the default in OpenSSL 1.0.x, although that seems less likely.] I therefore suggest leaving compression on by default, and back-patching a fix to 9.2 and above, where sslcompression was introduced, to clear this flag if it may have been set. The same code is in fe-secure.c rather than fe-secure-openssl.c in PostgreSQL 9.4 and below. This would be in line with the back-porting of other support for 1.1.0 in: https://www.postgresql.org/message-id/flat/20160627151604.GD1051%40msg.df7cb.de#20160627151604.GD1051@msg.df7cb.de According to a Python SSL comment: /* SSL_CTX_clear_options() and SSL_clear_options() were first added in* OpenSSL 0.9.8m but do not appear in some 0.9.9-devversions such the* 0.9.9 from "May 2008" that NetBSD 5.0 uses. */ Since SSL_OP_NO_COMPRESSION was added in 1.0.0, it should be fine to use SSL_clear_options() within the existing #ifdef. It may also help to mention in the documentation that this compression requires a version of openssl/libssl compiled with zlib[-dynamic], and potentially environment variables to enable it, such as OPENSSL_DEFAULT_ZLIB in Ubuntu 12.04 (but not 14.04+): https://bazaar.launchpad.net/~ubuntu-branches/ubuntu/vivid/openssl/vivid/revision/95 This was suggested previously in: https://www.postgresql.org/message-id/flat/CAKwe89Cj7KQ3BZDoUXLF5KBZ8X6icKXHi2Y1mDzTut3PNrH2VA%40mail.gmail.com == Workaround: recompiling openssl == You can check whether openssl has been compiled with zlib support by checking for -DZLIB / -DZLIB_SHARED in: $ openssl version -f but either way, it will not be on by default with openssl 1.1.0+ For those wishing to fix compression on Debian stretch (or jessie, or Ubuntu 14.04+): # apt-get install build-essential # apt-get build-dep openssl From an unprivileged account: $ apt-get source openssl $ cd openssl-1.1.0f $ nano debian/rules [ replace 'no-zlib' with 'zlib' or 'zlib-dynamic' in CONFARGS ] For OpenSSL 1.1.0+ (stretch et. al), also apply the following patch to ssl/ssl_lib.c: ###START### --- ssl/ssl_lib.c.orig 2017-06-02 22:34:20.524405847 +0200 +++ ssl/ssl_lib.c 2017-06-02 22:34:36.764406166 +0200 @@ -2471,13 +2471,6 @@ * deployed might change this. */ ret->options |= SSL_OP_LEGACY_SERVER_CONNECT; - /* - * Disable compression by default to prevent CRIME. Applications can - * re-enable compression by configuring - * SSL_CTX_clear_options(ctx, SSL_OP_NO_COMPRESSION); - * or by using the SSL_CONF library. - */ - ret->options |= SSL_OP_NO_COMPRESSION; ret->tlsext_status_type = -1; ###END### $ nice dpkg-buildpackage -b -j4 -nc (note: build tests will fail if root) Then, from a privileged account [adjust for version/arch] # dpkg -I ../libssl1.1_1.1.0f-1_amd64.deb # dpkg -I ../openssl_1.1.0f-1_amd64.deb Check that ZLIB is no longer a disabled algorithm: # openssl list -disabled It should also be listed as -DZLIB and -DZLIB_SHARED in: # openssl version -f # service postgresql restart == Future thoughts == If compression is unavailable it may become a competitive issue (MySQL/MariaDB offer zlib/deflate-based compression toggled via slave_compressed_protocol), or a reason to avoid streaming physical replication - possibly logical as well, I don't have experience there. Uncompressed replication roughly triples the required bandwidth/transfer, which can translate into significant costs or performance issues. In my case, compression means I can remain within a 10TB/month budget on a server used for cascading replication. The feeling in 2012 was that transport compression isn't within PostgreSQL's wheelhouse (which is why it remains important to enable it where possible in OpenSSL): https://www.postgresql.org/message-id/flat/4FD9E70B.9040607%40timbira.com#4FD9E70B.9040607@timbira.com This is a fair argument; however, TLS 1.3 removes compression: https://tools.ietf.org/html/draft-ietf-tls-tls13-20 - "Other cryptographic improvements including the removal ofcompression and custom DHE groups..." ...and this will be included in OpensSSL 1.1.1+, likely this year: https://www.openssl.org/blog/blog/2017/05/04/tlsv1.3/ So it may become necessary to use *something* in PostgreSQL 11+, be it zlib, PGLZ ( http://paquier.xyz/postgresql-2/postgres-9-5-feature-highlight-pglz-compression-libpqcommon/ ), or another algorithm. Or specify TLS 1.2. But that would need further code changes, too - and you have to use an outdated security protocol for compression" does not sound compelling. Best regards, -- Laurence "GreenReaper" Parry - Inkbunny administrator greenreaper.co.uk - wikifur.com - flayrah.com - inkbunny.net "Eternity lies ahead of us, and behind. Have you drunk your fill?" -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
greenreaper@hotmail.com writes: > PostgreSQL is documented to compress data over SSL connections by default > where possible: > https://www.postgresql.org/docs/9.6/static/libpq-connect.html#LIBPQ-CONNECT-SSLCOMPRESSION > PostgreSQL assumes that OpenSSL 0.9.8+ will compress with zlib if it can. > However, OpenSSL 1.1.0+ - already in Manjaro, and the upcoming Debian 9 > 'Stretch', Fedora 26, and Tails 3.0 - sets SSL_OP_NO_COMPRESSION by default > to discourage the CRIME attack: So the actual issue here, which you've not addressed in this otherwise extensive screed, is whether it's a wise thing for us to override that decision. Should we not just change sslcompression into a no-op? Is there some reason why PG data is immune to the CRIME attack? We might in the future consider compressing the data for ourselves inside the SSL stream, but (a) that would be a protocol break, with a pile of unpleasant compatibility consequences, and (b) if compression by OpenSSL itself makes the stream more decryptable, wouldn't app-provided compression inside the stream also do that? In short I'm wondering whether app-driven transport data compression is an idea whose time has passed. regards, tom lane -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
> So the actual issue here, which you've not addressed in
> this otherwise extensive screed, is whether it's a wise
> thing for us to override that decision.
>
> Is there some reason why PG data is immune to the CRIME attack?
I didn't address the security implications because I am not a security expert.
It would be a good idea for a security professional to evaluate the protocol.
If you want, I can make a few guesses, to be taken with a large pinch of salt:1) Many PostgreSQL connections operate over networks which are believed to be secure. This came up last year in postgres-hackers, and there were others who felt that CRIME was not a significant concern in most of their deployments:
In this case, administrators may be using SSL *only* for the compression benefits, because there is no separate compression feature. There is still an argument for defence-in-depth, but it is not as compelling in such situations.
[Bear in mind that unexpected loss of compression - say, after upgrading a standby - can impact the stability of a replication setup with specified bandwidth limits. Suddenly, the load triples - perhaps 5Mbps -> 15Mbps, on a 10Mbps leased line. Replication lags, WAL backs up - and before you know it, you're out of disk space on the *primary*, without even touching it. This may only be noticed once the server gets busy - say, after the maintenance window in which you upgraded the standby - and is compounded by the fact that the only way you know what's happened is by capturing the SSL handshakes, or deduce it from the change in bandwidth usage.]
2) HTTPS is particularly vulnerable to CRIME because unrelated information (much of which the attacker knows and/or controls) is regularly compressed and transferred with a static authentication cookie which they want to discover.
PostgreSQL does not handle authentication in quite the same way. It is done only at connection startup; credentials are transferred separately from other data, and the authentication persists for the connection lifetime, which is often extensive:
Now, ZLIB-based SSL compression is stateful, so if an attacker can get a client to open many connections, they might be vulnerable if they can also get information into the first 32KB transmitted and draw an inference from there. But it's likely to be harder to determine a match, especially if they don't know the queries being made.
3) Applications which are separated from the database over a network are also likely to be using a connection pooler, which holds open a set of of active connections and does not consistently repeat authentication when new requests are made.
4) In my own use-case, network connections are only used for replication, which doesn't have the multiple-connections issue - if a slot is in use, the master will refuse additional connections (I know this well from using a somewhat unreliable WAN). Maybe there is some MITM attack which could be used to repeatedly drop and re-establish the connection, but this is likely to be noticeable in the logs/performance. If using certificates for identification, that kind of attack should be infeasible.
5) A WAL stream or other connection might contain other data which they could try to match. But even if specific plaintext can be inserted, *and* they can get a secret included too, an active connection is likely transferring other content as well. It lacks the same consistency as as a stream of POSTs which the attacker controls, containing a constant secret at a constant location. (synchronous_commit='off' may make this harder, though I don't know how immediate streaming replication is.)
> Should we not just change sslcompression into a no-op?
Well, it effectively *is* a no-op on OpenSSL 1.1.0+. That's what I want to fix! Losing the ability to compress the transport should, in itself, be considered a regression. I don't think every PostgreSQL user on a modern OS should have to patch their copy of OpenSSL because PostgreSQL isn't doing what it claims to. :-)
Making sslcompression a no-op would also stop administrators from disabling SSL compression just for PostgreSQL on platforms where it *is* on by default - i.e. OpenSSL 0.9.8x/1.0.x where the distro hasn't patched-out ZLIB - which is currently the point of the parameter.
I guess maybe you meant it should always set no-compression? There's an argument that PostgreSQL's defaults should be the safest possible. From that perspective, it may be reasonable to change the default to not be compressed, at least for 10. But even if this is done, PostgresSQL should clear the no-compression flag if sslcompression=1, as that is a clear expression of intent by the administrator.
PostgreSQL is used in a variety of situations. In some of those, SSL compression may be seen as inadvisable. In others, it is beneficial. This is about making sure administrators can continue to choose it as an option based on their situation.
PostgreSQL already trusts administrators to make decisions about security trade-offs, otherwise it might forbid connections without SSL entirely - or at least have 'require' rather than 'prefer' as a default. It _would_ be a good idea to communicate concerns in the documentation, so administrators can make more-educated decisions.
> In short I'm wondering whether app-driven transport data
> compression is an idea whose time has passed.
If anything it seems far *more* suitable for the app to do it vs. delegating to the transport, because it has knowledge of the sensitivity of the data.
For example, it might use a stateless ZLIB replacement ( http://www.libslz.org/ ) during the authentication phase, and revert to regular ZLIB operation during WAL streaming. Or just use SLZ all the time, accepting that compression will be worse (but still with a ~55% saving), and faster to boot.
But that would be a separate feature. This bug is about preserving an existing documented feature, which is seen as useful in many use-cases:
...at least until there is a good replacement for it.--
Laurence "GreenReaper" Parry