Обсуждение: Python 3.x versus PG 9.1 branch

In view of our rather embarrassing failure to cover the back branches
with Python 3.5-related regression test adjustments, I think there is
a clear need for a buildfarm critter that's testing with Python 3.5,
and I've been working on setting one up.  It's passing at the moment
for 9.2 and up, but not for 9.1, because we've repeatedly not bothered
to back-port regression test fixes for newer Pythons into that branch.
I could just omit Python 3 coverage for that branch in the critter's
configuration, but I wonder exactly why things are that way.

For clarity, to cover 9.1 I think we'd need to back-patch some subset
of these commits:

f16d52269 ff2faeec5 d0765d50f 6bff0e7d9 527ea6684 8182ffde5
45d1f1e02 2cfb1c6f7

The precedent of not fixing 9.1 started with the last of these.

I haven't looked into the details of which changes would actually
apply to 9.1, I just searched for commits that touched the Python
regression tests.  We'd really only need enough changes to address
the regression failures I'm getting, which are attached below.

Or we could just blow it off on the grounds that 9.1 is not long
for this world anyhow.

Opinions anyone?

            regards, tom lane

*** /Users/tgl/pgsql/src/pl/plpython/./python3/expected/plpython_do.out    Wed Jan 13 00:46:31 2016
--- /Users/tgl/pgsql/src/pl/plpython/./python3/results/plpython_do.out    Wed Jan 13 00:46:31 2016
***************
*** 2,8 ****
  NOTICE:  This is plpythonu.
  CONTEXT:  PL/Python anonymous code block
  DO $$ nonsense $$ LANGUAGE plpython3u;
! ERROR:  NameError: global name 'nonsense' is not defined
  CONTEXT:  Traceback (most recent call last):
    PL/Python anonymous code block, line 1, in <module>
      nonsense
--- 2,8 ----
  NOTICE:  This is plpythonu.
  CONTEXT:  PL/Python anonymous code block
  DO $$ nonsense $$ LANGUAGE plpython3u;
! ERROR:  NameError: name 'nonsense' is not defined
  CONTEXT:  Traceback (most recent call last):
    PL/Python anonymous code block, line 1, in <module>
      nonsense

======================================================================

*** /Users/tgl/pgsql/src/pl/plpython/./python3/expected/plpython_import.out    Wed Jan 13 00:46:31 2016
--- /Users/tgl/pgsql/src/pl/plpython/./python3/results/plpython_import.out    Wed Jan 13 00:46:32 2016
***************
*** 51,57 ****
  -- import python modules
  --
  SELECT import_fail();
! NOTICE:  import socket failed -- No module named foosocket
  CONTEXT:  PL/Python function "import_fail"
      import_fail
  --------------------
--- 51,57 ----
  -- import python modules
  --
  SELECT import_fail();
! NOTICE:  import socket failed -- No module named 'foosocket'
  CONTEXT:  PL/Python function "import_fail"
      import_fail
  --------------------

======================================================================

*** /Users/tgl/pgsql/src/pl/plpython/./python3/expected/plpython_params.out    Wed Jan 13 00:46:31 2016
--- /Users/tgl/pgsql/src/pl/plpython/./python3/results/plpython_params.out    Wed Jan 13 00:46:32 2016
***************
*** 37,46 ****
  SELECT test_param_names2(users) from users;
                             test_param_names2
  -----------------------------------------------------------------------
!  {'lname': 'doe', 'username': 'j_doe', 'userid': 1, 'fname': 'jane'}
!  {'lname': 'doe', 'username': 'johnd', 'userid': 2, 'fname': 'john'}
!  {'lname': 'doe', 'username': 'w_doe', 'userid': 3, 'fname': 'willem'}
!  {'lname': 'smith', 'username': 'slash', 'userid': 4, 'fname': 'rick'}
  (4 rows)

  SELECT test_param_names2(NULL);
--- 37,46 ----
  SELECT test_param_names2(users) from users;
                             test_param_names2
  -----------------------------------------------------------------------
!  {'userid': 1, 'username': 'j_doe', 'lname': 'doe', 'fname': 'jane'}
!  {'userid': 2, 'username': 'johnd', 'lname': 'doe', 'fname': 'john'}
!  {'userid': 3, 'username': 'w_doe', 'lname': 'doe', 'fname': 'willem'}
!  {'userid': 4, 'username': 'slash', 'lname': 'smith', 'fname': 'rick'}
  (4 rows)

  SELECT test_param_names2(NULL);

======================================================================

*** /Users/tgl/pgsql/src/pl/plpython/./python3/expected/plpython_trigger.out    Wed Jan 13 00:46:31 2016
--- /Users/tgl/pgsql/src/pl/plpython/./python3/results/plpython_trigger.out    Wed Jan 13 00:46:32 2016
***************
*** 119,125 ****
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[name] => show_trigger_data_trig_before
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[new] => {'i': 1, 'v': 'insert'}
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[old] => None
  CONTEXT:  PL/Python function "trigger_data"
--- 119,125 ----
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[name] => show_trigger_data_trig_before
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[new] => {'v': 'insert', 'i': 1}
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[old] => None
  CONTEXT:  PL/Python function "trigger_data"
***************
*** 139,145 ****
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[name] => show_trigger_data_trig_after
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[new] => {'i': 1, 'v': 'insert'}
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[old] => None
  CONTEXT:  PL/Python function "trigger_data"
--- 139,145 ----
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[name] => show_trigger_data_trig_after
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[new] => {'v': 'insert', 'i': 1}
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[old] => None
  CONTEXT:  PL/Python function "trigger_data"
***************
*** 180,188 ****
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[name] => show_trigger_data_trig_before
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[new] => {'i': 1, 'v': 'update'}
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[old] => {'i': 1, 'v': 'insert'}
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[relid] => bogus:12345
  CONTEXT:  PL/Python function "trigger_data"
--- 180,188 ----
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[name] => show_trigger_data_trig_before
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[new] => {'v': 'update', 'i': 1}
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[old] => {'v': 'insert', 'i': 1}
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[relid] => bogus:12345
  CONTEXT:  PL/Python function "trigger_data"
***************
*** 200,208 ****
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[name] => show_trigger_data_trig_after
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[new] => {'i': 1, 'v': 'update'}
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[old] => {'i': 1, 'v': 'insert'}
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[relid] => bogus:12345
  CONTEXT:  PL/Python function "trigger_data"
--- 200,208 ----
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[name] => show_trigger_data_trig_after
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[new] => {'v': 'update', 'i': 1}
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[old] => {'v': 'insert', 'i': 1}
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[relid] => bogus:12345
  CONTEXT:  PL/Python function "trigger_data"
***************
*** 243,249 ****
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[new] => None
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[old] => {'i': 1, 'v': 'update'}
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[relid] => bogus:12345
  CONTEXT:  PL/Python function "trigger_data"
--- 243,249 ----
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[new] => None
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[old] => {'v': 'update', 'i': 1}
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[relid] => bogus:12345
  CONTEXT:  PL/Python function "trigger_data"
***************
*** 263,269 ****
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[new] => None
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[old] => {'i': 1, 'v': 'update'}
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[relid] => bogus:12345
  CONTEXT:  PL/Python function "trigger_data"
--- 263,269 ----
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[new] => None
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[old] => {'v': 'update', 'i': 1}
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[relid] => bogus:12345
  CONTEXT:  PL/Python function "trigger_data"
***************
*** 311,317 ****
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[name] => show_trigger_data_trig
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[new] => {'i': 2, 'v': 'insert'}
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[old] => None
  CONTEXT:  PL/Python function "trigger_data"
--- 311,317 ----
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[name] => show_trigger_data_trig
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[new] => {'v': 'insert', 'i': 2}
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[old] => None
  CONTEXT:  PL/Python function "trigger_data"
***************
*** 332,340 ****
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[name] => show_trigger_data_trig
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[new] => {'i': 1, 'v': 'update'}
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[old] => {'i': 1, 'v': 'insert'}
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[relid] => bogus:12345
  CONTEXT:  PL/Python function "trigger_data"
--- 332,340 ----
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[name] => show_trigger_data_trig
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[new] => {'v': 'update', 'i': 1}
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[old] => {'v': 'insert', 'i': 1}
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[relid] => bogus:12345
  CONTEXT:  PL/Python function "trigger_data"
***************
*** 355,361 ****
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[new] => None
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[old] => {'i': 1, 'v': 'insert'}
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[relid] => bogus:12345
  CONTEXT:  PL/Python function "trigger_data"
--- 355,361 ----
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[new] => None
  CONTEXT:  PL/Python function "trigger_data"
! NOTICE:  TD[old] => {'v': 'insert', 'i': 1}
  CONTEXT:  PL/Python function "trigger_data"
  NOTICE:  TD[relid] => bogus:12345
  CONTEXT:  PL/Python function "trigger_data"

======================================================================

*** /Users/tgl/pgsql/src/pl/plpython/./python3/expected/plpython_error.out    Wed Jan 13 00:46:31 2016
--- /Users/tgl/pgsql/src/pl/plpython/./python3/results/plpython_error.out    Wed Jan 13 00:46:32 2016
***************
*** 245,251 ****
  plpy.nonexistent
  $$ LANGUAGE plpython3u;
  SELECT toplevel_attribute_error();
! ERROR:  AttributeError: 'module' object has no attribute 'nonexistent'
  CONTEXT:  Traceback (most recent call last):
    PL/Python function "toplevel_attribute_error", line 2, in <module>
      plpy.nonexistent
--- 245,251 ----
  plpy.nonexistent
  $$ LANGUAGE plpython3u;
  SELECT toplevel_attribute_error();
! ERROR:  AttributeError: module 'plpy' has no attribute 'nonexistent'
  CONTEXT:  Traceback (most recent call last):
    PL/Python function "toplevel_attribute_error", line 2, in <module>
      plpy.nonexistent

======================================================================

*** /Users/tgl/pgsql/src/pl/plpython/./python3/expected/plpython_subtransaction.out    Wed Jan 13 00:46:31 2016
--- /Users/tgl/pgsql/src/pl/plpython/./python3/results/plpython_subtransaction.out    Wed Jan 13 00:46:32 2016
***************
*** 58,64 ****

  TRUNCATE subtransaction_tbl;
  SELECT subtransaction_test('Python');
! ERROR:  AttributeError: 'module' object has no attribute 'attribute_error'
  CONTEXT:  Traceback (most recent call last):
    PL/Python function "subtransaction_test", line 13, in <module>
      plpy.attribute_error
--- 58,64 ----

  TRUNCATE subtransaction_tbl;
  SELECT subtransaction_test('Python');
! ERROR:  AttributeError: module 'plpy' has no attribute 'attribute_error'
  CONTEXT:  Traceback (most recent call last):
    PL/Python function "subtransaction_test", line 13, in <module>
      plpy.attribute_error
***************
*** 110,116 ****

  TRUNCATE subtransaction_tbl;
  SELECT subtransaction_ctx_test('Python');
! ERROR:  AttributeError: 'module' object has no attribute 'attribute_error'
  CONTEXT:  Traceback (most recent call last):
    PL/Python function "subtransaction_ctx_test", line 8, in <module>
      plpy.attribute_error
--- 110,116 ----

  TRUNCATE subtransaction_tbl;
  SELECT subtransaction_ctx_test('Python');
! ERROR:  AttributeError: module 'plpy' has no attribute 'attribute_error'
  CONTEXT:  Traceback (most recent call last):
    PL/Python function "subtransaction_ctx_test", line 8, in <module>
      plpy.attribute_error

======================================================================

On Wed, Jan 13, 2016 at 11:46:07AM -0500, Tom Lane wrote:
> [...] we've repeatedly not bothered
> to back-port regression test fixes for newer Pythons into that branch.
> I could just omit Python 3 coverage for that branch in the critter's
> configuration, but I wonder exactly why things are that way.
> 
> For clarity, to cover 9.1 I think we'd need to back-patch some subset
> of these commits:
> 
> f16d52269 ff2faeec5 d0765d50f 6bff0e7d9 527ea6684 8182ffde5
> 45d1f1e02 2cfb1c6f7
> 
> The precedent of not fixing 9.1 started with the last of these.

> Or we could just blow it off on the grounds that 9.1 is not long
> for this world anyhow.
> 
> Opinions anyone?

I respect the 2012-era decision to have 9.1 not support newer Python, and I
think the lack of user complaints validates it.  I wouldn't object to
overturning the decision, either.  The biggest risk, albeit still a small
risk, is that newer Python is incompatible with 9.1 in a way that the test
suite does not catch.

On Thu, Jan 14, 2016 at 12:37 PM, Noah Misch <noah@leadboat.com> wrote:
> On Wed, Jan 13, 2016 at 11:46:07AM -0500, Tom Lane wrote:
>> [...] we've repeatedly not bothered
>> to back-port regression test fixes for newer Pythons into that branch.
>> I could just omit Python 3 coverage for that branch in the critter's
>> configuration, but I wonder exactly why things are that way.
>>
>> For clarity, to cover 9.1 I think we'd need to back-patch some subset
>> of these commits:
>>
>> f16d52269 ff2faeec5 d0765d50f 6bff0e7d9 527ea6684 8182ffde5
>> 45d1f1e02 2cfb1c6f7
>>
>> The precedent of not fixing 9.1 started with the last of these.
>
>> Or we could just blow it off on the grounds that 9.1 is not long
>> for this world anyhow.
>>
>> Opinions anyone?
>
> I respect the 2012-era decision to have 9.1 not support newer Python, and I
> think the lack of user complaints validates it.  I wouldn't object to
> overturning the decision, either.  The biggest risk, albeit still a small
> risk, is that newer Python is incompatible with 9.1 in a way that the test
> suite does not catch.

The lack of user complaints regarding 9.1 using Python 3.5 and the
fact that 9.1 will be EOL in 8~9 months does not sound worth it to me.

A couple of days ago I bumped into this article, leading to the
thought that Python 4.0 may induce as much breakage as 3.5 did :(
http://astrofrog.github.io/blog/2016/01/12/stop-writing-python-4-incompatible-code/
Just something to keep in mind.
-- 
Michael

On Wed, Jan 13, 2016 at 11:46 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Or we could just blow it off on the grounds that 9.1 is not long
> for this world anyhow.

+1 for blowing it off.  I can't see the point in putting effort into
this.  Nobody should be spinning up new PostgreSQL 9.1 deployments at
this point, and whatever PostgreSQL 9.1 deployments already exist are
evidently OK with what we've been doing up until now.  So it seems
unlikely to help anyone.

Also, if it does help someone, it will be helping them to deploy a
nearly-obsolete PostgreSQL version at a time when, really, it would be
much better if they were thinking about how to get off that version.

Moreover, it's not inconceivable that back-porting all of those
commits could break something that works now, in which event we would
end up worse off than we are today.

So I really don't see any upside.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company