Обсуждение: PL/Java 1.5.1_BETA2

Поиск
Список
Период
Сортировка

PL/Java 1.5.1_BETA2

От
Chapman Flack
Дата:
Hello packagers,

There is a PL/Java 1.5.1_BETA2 release at github:
https://github.com/tada/pljava/releases/tag/V1_5_1b2

A lot of time has passed since BETA1, so there is more
new functionality than would be typical for a BETA2.
In retrospect, I should have bumped the number higher
and called it another beta1, which would allow it to
be announced on pgsql-announce, but that policy had
slipped my mind.

Anyway, as poorly announced as it may be, there is
a new beta release. It would be great if there could
be packages for it.

Thanks!
-Chap


Re: PL/Java 1.5.1_BETA2

От
Christoph Berg
Дата:
Re: Chapman Flack 2018-09-10 <5B96B226.2080803@anastigmatix.net>
> Hello packagers,
> 
> There is a PL/Java 1.5.1_BETA2 release at github:
> https://github.com/tada/pljava/releases/tag/V1_5_1b2

Hi Chap,

you got me interested again :)

I tried building pljava, but I seem to fail at a very early stage:

$ JAVA_HOME=/usr/lib/jvm/default-java/ PATH=/usr/lib/jvm/default-java/bin:$PATH mvn compile

[...]
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ pljava-api ---
[INFO] Compiling 24 source files to /srv/projects/postgresql/pljava/postgresql-pljava/pljava-api/target/classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] Failure executing javac, but could not parse the error:
Fatal Error: Unable to find package java.lang in classpath or bootclasspath

[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] PostgreSQL PL/Java 1.5.1-BETA2 ..................... SUCCESS [  0.023 s]
[INFO] PL/Java API ........................................ FAILURE [  0.934 s]
[INFO] PL/Java backend Java code .......................... SKIPPED
[INFO] PL/Java backend native code ........................ SKIPPED
[INFO] PL/Java Deploy ..................................... SKIPPED
[INFO] PL/Java Ant tasks .................................. SKIPPED
[INFO] PL/Java examples ................................... SKIPPED
[INFO] PL/Java packaging 1.5.1-BETA2 ...................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.931 s
[INFO] Finished at: 2018-09-15T22:26:25+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile) on
projectpljava-api: Compilation failure
 
[ERROR] Failure executing javac, but could not parse the error:
[ERROR] Fatal Error: Unable to find package java.lang in classpath or bootclasspath
[...]

openjdk-10-jdk is installed. Any idea what is missing?

Christoph


Re: PL/Java 1.5.1_BETA2

От
Christoph Berg
Дата:
Re: Chapman Flack 2018-09-10 <5B96B226.2080803@anastigmatix.net>
> Hello packagers,
> 
> There is a PL/Java 1.5.1_BETA2 release at github:
> https://github.com/tada/pljava/releases/tag/V1_5_1b2

Hi Chap,

you got me interested again :)

I tried building pljava, but I seem to fail at a very early stage:

$ JAVA_HOME=/usr/lib/jvm/default-java/ PATH=/usr/lib/jvm/default-java/bin:$PATH mvn compile

[...]
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ pljava-api ---
[INFO] Compiling 24 source files to /srv/projects/postgresql/pljava/postgresql-pljava/pljava-api/target/classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] Failure executing javac, but could not parse the error:
Fatal Error: Unable to find package java.lang in classpath or bootclasspath

[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] PostgreSQL PL/Java 1.5.1-BETA2 ..................... SUCCESS [  0.023 s]
[INFO] PL/Java API ........................................ FAILURE [  0.934 s]
[INFO] PL/Java backend Java code .......................... SKIPPED
[INFO] PL/Java backend native code ........................ SKIPPED
[INFO] PL/Java Deploy ..................................... SKIPPED
[INFO] PL/Java Ant tasks .................................. SKIPPED
[INFO] PL/Java examples ................................... SKIPPED
[INFO] PL/Java packaging 1.5.1-BETA2 ...................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.931 s
[INFO] Finished at: 2018-09-15T22:26:25+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile) on
projectpljava-api: Compilation failure
 
[ERROR] Failure executing javac, but could not parse the error:
[ERROR] Fatal Error: Unable to find package java.lang in classpath or bootclasspath
[...]

openjdk-10-jdk is installed. Any idea what is missing?

Christoph


Re: PL/Java 1.5.1_BETA2

От
Chapman Flack
Дата:
On 09/15/18 16:32, Christoph Berg wrote:
> Fatal Error: Unable to find package java.lang in classpath or bootclasspath
> ...
> openjdk-10-jdk is installed. Any idea what is missing?

With Java 9, a completely new modular structure was brought to the JDK,
with deep implications for build procedures. I will have to rework the
build process heavily for a future PL/Java release, but for 1.5.1, I
have not.

So, it needs a Java 8 JDK (or 7, or 6, but let's just say 8) to build.

Once built, it will _run_ on later Java runtimes (and users can write
PL/Java functions using the latest language features, compile them with
later JDKs, and run them in PL/Java, as long as it is running on a new-
enough runtime).

The Java runtime that actually gets used, at runtime, is determined by
the pljava.libjvm_location variable, which can be given a default at
build time with -Dpljava.libjvmdefault=....

So if you are packaging for a platform where you know Java 10 will be
the default, and you know what the conventional path to its libjvm.so will
be, then by all means, use -Dpljava.libjvmdefault= to build in that
default for use at runtime. But the build, for now, must be done with 8.

There is a new(-ish?) page in the documentation with information intended
for packagers:

http://tada.github.io/pljava/build/package.html

-Chap


Re: PL/Java 1.5.1_BETA2

От
Chapman Flack
Дата:
On 09/15/18 16:32, Christoph Berg wrote:
> Fatal Error: Unable to find package java.lang in classpath or bootclasspath
> ...
> openjdk-10-jdk is installed. Any idea what is missing?

With Java 9, a completely new modular structure was brought to the JDK,
with deep implications for build procedures. I will have to rework the
build process heavily for a future PL/Java release, but for 1.5.1, I
have not.

So, it needs a Java 8 JDK (or 7, or 6, but let's just say 8) to build.

Once built, it will _run_ on later Java runtimes (and users can write
PL/Java functions using the latest language features, compile them with
later JDKs, and run them in PL/Java, as long as it is running on a new-
enough runtime).

The Java runtime that actually gets used, at runtime, is determined by
the pljava.libjvm_location variable, which can be given a default at
build time with -Dpljava.libjvmdefault=....

So if you are packaging for a platform where you know Java 10 will be
the default, and you know what the conventional path to its libjvm.so will
be, then by all means, use -Dpljava.libjvmdefault= to build in that
default for use at runtime. But the build, for now, must be done with 8.

There is a new(-ish?) page in the documentation with information intended
for packagers:

http://tada.github.io/pljava/build/package.html

-Chap


Re: PL/Java 1.5.1_BETA2

От
Christoph Berg
Дата:
Re: Chapman Flack 2018-09-15 <5B9D73A2.2090402@anastigmatix.net>
> On 09/15/18 16:32, Christoph Berg wrote:
> > Fatal Error: Unable to find package java.lang in classpath or bootclasspath
> > ...
> > openjdk-10-jdk is installed. Any idea what is missing?
> 
> With Java 9, a completely new modular structure was brought to the JDK,
> with deep implications for build procedures. I will have to rework the
> build process heavily for a future PL/Java release, but for 1.5.1, I
> have not.
> 
> So, it needs a Java 8 JDK (or 7, or 6, but let's just say 8) to build.

Ok, openjdk-8 worked, thanks. I was actually skimming the docs for the
supported Java versions, but couldn't find anything, so went with the
default-jdk in Debian, which is 10. A note in the docs would be
appreciated, or if it's already in there, it needs to be more visible.

> Once built, it will _run_ on later Java runtimes (and users can write
> PL/Java functions using the latest language features, compile them with
> later JDKs, and run them in PL/Java, as long as it is running on a new-
> enough runtime).
> 
> The Java runtime that actually gets used, at runtime, is determined by
> the pljava.libjvm_location variable, which can be given a default at
> build time with -Dpljava.libjvmdefault=....

I got that working just fine on sid/amd64, but have a series of
problems on i386 (i.e. 32-bit x86). The build is running with
-Dpljava.libjvmdefault=/usr/lib/jvm/java-8-openjdk-i386/jre/lib/i386/server/libjvm.so,
but then actually using the extension yields this:

https://pgdgbuild.dus.dg-i.net/job/postgresql-pljava-binaries/3/architecture=i386,distribution=sid/

20:14:15 + psql -c 'CREATE EXTENSION pljava'
20:14:15 WARNING:  Java virtual machine not yet loaded
20:14:15 DETAIL:  /usr/lib/jvm/java-8-openjdk-1/jre/lib/1/server/libjvm.so: cannot open shared object file: No such
fileor directory
 
20:14:15 HINT:  SET pljava.libjvm_location TO the correct path to the jvm library (libjvm.so or jvm.dll, etc.)

i.e. it's replacing both occurrences of "i386" by "1".

If I manually fix that, there's more weirdness:
(same session was running "create extension pljava" earlier)
# set pljava.libjvm_location = '/usr/lib/jvm/java-8-openjdk-i386/jre/lib/i386/server/libjvm.so';
HINWEIS:  00000: PL/Java loaded
DETAIL:  versions:
PL/Java native code (1.5.1-BETA2)
PL/Java common code (1.5.1-BETA2)
Built for (PostgreSQL 10.5 (Debian 10.5-2~30.git84cd21f.pgdg+1) on i686-pc-linux-gnu, compiled by gcc (Debian 8.2.0-4)
8.2.0,32-bit)
 
Loaded in (PostgreSQL 10.5 (Debian 10.5-2~30.git84cd21f.pgdg+1) on i686-pc-linux-gnu, compiled by gcc (Debian 8.2.0-4)
8.2.0,32-bit)
 
OpenJDK Runtime Environment (1.8.0_181-8u181-b13-1-b13)
OpenJDK Server VM (25.181-b13, mixed mode)
ORT:  initsequencer, Backend.c:597
HINWEIS:  00000: PL/Java successfully started after adjusting settings
TIP:  The settings that worked should be saved (using ALTER DATABASE postgres SET ... FROM CURRENT or in the
"/home/cbe/tmp/pg_virtualenv.m9FB0r/postgresql/10/regress/postgresql.conf"file). For a reminder of what has been set,
try:SELECT name, setting FROM pg_settings WHERE name LIKE 'pljava.%' AND source = 'session'
 
ORT:  initsequencer, Backend.c:639
HINWEIS:  00000: PL/Java load successful after failed CREATE EXTENSION
DETAIL:  PL/Java is now installed, but not as an extension.
TIP:  To correct that, either COMMIT or ROLLBACK, make sure the working settings are saved, exit this session, and in a
newsession, either: 1. if committed, run "CREATE EXTENSION pljava FROM unpackaged", or 2. if rolled back, simply
"CREATEEXTENSION pljava" again.
 
ORT:  initsequencer, Backend.c:654
SET

Why does tell me that this installation is not from an extension?

Proceeding to load the examples:

# SELECT sqlj.install_jar('file:/usr/share/postgresql/10/pljava/pljava-examples-1.5.1-BETA2.jar', 'examples', true)
;
HINWEIS:  42809: Rückgabetyp javatest.udtscalariotest ist nur eine Hülle
ORT:  compute_return_type, functioncmds.c:105
[...]
ORT:  Java_org_postgresql_pljava_internal_Backend__1log, Backend.c:1621
INFO:  00000: 16 Sep 18 19:54:06 org.postgresql.pljava.example.LoggerTest timestamptz roundtrip passes
ORT:  Java_org_postgresql_pljava_internal_Backend__1log, Backend.c:1621
INFO:  00000: 16 Sep 18 19:54:09 org.postgresql.pljava.example.LoggerTest all Unicode codepoint ranges roundtripped
successfully.
ORT:  Java_org_postgresql_pljava_internal_Backend__1log, Backend.c:1621
Server beendete die Verbindung unerwartet
    Das heißt wahrscheinlich, dass der Server abnormal beendete
    bevor oder während die Anweisung bearbeitet wurde.
Die Verbindung zum Server wurde verloren.  Versuche Reset: Fehlgeschlagen.

... it crashes.

> There is a new(-ish?) page in the documentation with information intended
> for packagers:
> 
> http://tada.github.io/pljava/build/package.html

Aye, I remember having seen that 2.5 years ago when I last tried
looking at 1.5.


Another question: Docs say
  mvn -Dpgsql.pgconfig=...
  java -Dpgconfig=/local/pgsql/bin/pg_config -jar pljava-packaging/...
I had first tried mvn -Dpgconfig, which didn't work - is it really two
different variable names?

And is -Dpgconfig even used if for the installer, when
-Dpgconfig.pkglibdir and -Dpgconfig.sharedir are given? I need to
point these to debian/postgresql-$version-pljava/usr/... A "DESTDIR"
mechanism would be nice to have here.


On stretch/amd64, install_jar fails on 9.6 (9.3/4/5 worked):

20:07:05 + psql -c 'CREATE EXTENSION pljava'
20:07:05 CREATE EXTENSION
20:07:05 + psql -c 'SELECT
sqlj.install_jar('\''file:/usr/share/postgresql/9.6/pljava/pljava-examples-1.5.1-BETA2.jar'\'','\''examples'\'',
true)'
[...]
20:07:07 INFO:  16 Sep 18 18:07:07 org.postgresql.pljava.management.Commands Type id = OID(16686)
20:07:07 ERROR:  java.lang.ClassNotFoundException: org.postgresql.pljava.example.annotation.ComplexTuple
20:07:07 CONTEXT:  SQL statement "SELECT javatest.assertHasValues( CAST('(1,2)' AS javatest.complextuple), 1, 2)"

https://pgdgbuild.dus.dg-i.net/job/postgresql-pljava-binaries/3/architecture=amd64,distribution=stretch/


On sid/ppc64el, compilation dies:

https://pgdgbuild.dus.dg-i.net/job/postgresql-pljava-binaries/3/architecture=ppc64el,distribution=sid/console

20:00:43 [ERROR] Failed to execute goal com.github.maven-nar:nar-maven-plugin:3.2.3:nar-validate (default-nar-validate)
onproject pljava-so: NAR: One of two things may be wrong here:
 
20:00:43 [ERROR] 1. <Name> tag is missing inside the <Linker> tag of your NAR configuration
20:00:43 [ERROR] 2. no linker is defined in the aol.properties file for 'ppc64le.Linux.linker'

Is that a problem in pljava, or a general java-on-that-platform
problem?


(I can open GitHub issues for any of these items if you confirm which
make sense.)

Christoph


Re: PL/Java 1.5.1_BETA2

От
Christoph Berg
Дата:
Re: To Chapman Flack 2018-09-16 <20180916201808.GA30681@msg.df7cb.de>
> On stretch/amd64, install_jar fails on 9.6 (9.3/4/5 worked):
> 
> 20:07:05 + psql -c 'CREATE EXTENSION pljava'
> 20:07:05 CREATE EXTENSION
> 20:07:05 + psql -c 'SELECT
sqlj.install_jar('\''file:/usr/share/postgresql/9.6/pljava/pljava-examples-1.5.1-BETA2.jar'\'','\''examples'\'',
true)'
> [...]
> 20:07:07 INFO:  16 Sep 18 18:07:07 org.postgresql.pljava.management.Commands Type id = OID(16686)
> 20:07:07 ERROR:  java.lang.ClassNotFoundException: org.postgresql.pljava.example.annotation.ComplexTuple
> 20:07:07 CONTEXT:  SQL statement "SELECT javatest.assertHasValues( CAST('(1,2)' AS javatest.complextuple), 1, 2)"
> 
> https://pgdgbuild.dus.dg-i.net/job/postgresql-pljava-binaries/3/architecture=amd64,distribution=stretch/

The problem on bionic/amd64 is similar, here 9.4 fails after 9.3
passed the tests:

20:06:37 + psql -c 'SELECT
sqlj.install_jar('\''file:/usr/share/postgresql/9.4/pljava/pljava-examples-1.5.1-BETA2.jar'\'','\''examples'\'',
true)'
[...]
20:06:38 ERROR:  java.lang.ClassNotFoundException: org.postgresql.pljava.example.annotation.Point
20:06:38 CONTEXT:  SQL statement "SELECT javatest.assertHasValues(CAST('(1,2)' AS point), 1, 2)"

https://pgdgbuild.dus.dg-i.net/job/postgresql-pljava-binaries/3/architecture=amd64,distribution=bionic/console


Also, does it have to be that verbose all the time, even if things
worked?


This was all on openjdk-8. On jessie/amd64, openjdk-7 is used:

20:22:24 + java -Dpgsql.pgconfig=/usr/lib/postgresql/9.3/bin/pg_config
-Dpgconfig.pkglibdir=debian/postgresql-9.3-pljava/usr/lib/postgresql/9.3/lib
-Dpgconfig.sharedir=debian/postgresql-9.3-pljava/usr/share/postgresql/9.3-jar
build-9.3/pljava-pg9.3-amd64-Linux-gpp.jar
20:22:25 Exception in thread "main" java.lang.ClassCastException: sun.org.mozilla.javascript.ConsString cannot be cast
tojava.lang.String
 
20:22:25     at org.gjt.cuspy.JarX.extract(JarX.java:670)
20:22:25     at org.gjt.cuspy.JarX.extract(JarX.java:445)
20:22:25     at org.gjt.cuspy.JarX.main(JarX.java:419)

https://pgdgbuild.dus.dg-i.net/job/postgresql-pljava-binaries/3/architecture=amd64,distribution=jessie/console


The full set of dist/architecture combinations can be see at
https://pgdgbuild.dus.dg-i.net/job/postgresql-pljava-binaries/3/

Please tell me which issues you want a problem report on GitHub for :)

Cheers,
Christoph


Re: PL/Java 1.5.1_BETA2

От
Chapman Flack
Дата:
On 09/16/18 16:18, Christoph Berg wrote:

> i.e. it's replacing both occurrences of "i386" by "1".

Ok, that's irritating. I can guess what is happening. I implement the
-Dpljava.libjvmdefault by passing the string into cpp with a -D, and that
gets converted to a string using the CppAsString2 macro, and that apparently
is not protected from expansion of other macros, and I bet something is
defining i386 to be 1.

Phooey.

I don't know if that will be easy to fix using cpp. An alternative would be
to pass it to cpp already quoted according to C rules, but that would be
somewhat ugly to express in a Maven build script.

> (same session was running "create extension pljava" earlier)
> # set pljava.libjvm_location = '/usr/lib/jvm/java-8-openjdk-i386/jre/lib/i386/server/libjvm.so';
> HINWEIS:  00000: PL/Java load successful after failed CREATE EXTENSION
> DETAIL:  PL/Java is now installed, but not as an extension.
> TIP:  To correct that, either COMMIT or ROLLBACK, make sure the working settings are saved, exit this session, and in
anew session, either: 1. if committed, run "CREATE EXTENSION pljava FROM unpackaged", or 2. if rolled back, simply
"CREATEEXTENSION pljava" again.
 
> ORT:  initsequencer, Backend.c:654
> 
> Why does tell me that this installation is not from an extension?

Your original CREATE EXTENSION command did not succeed, reporting the error
trying to load the library.

Your SET command to correct the library name completed the installation
of PL/Java, but at that point it was no longer within a CREATE EXTENSION
command, so the various functions, tables, etc., have just been installed
'loose' (exactly as they would if you were installing in 9.0 or earlier
with no 'extension' mechanism).

The TIP describes how to proceed so they will all be grouped together
as a named extension, just as would have happened had the jvm path been
correct when the CREATE EXTENSION command was first given.

> Proceeding to load the examples:

> Server beendete die Verbindung unerwartet

Is there anything in the server log with details of the crash?
In many cases there will be a lengthy message from the JVM in the server
log, and that message will contain a file name (something like
/tmp/hs_err_pidnnnn.log) which will contain *even more* detail.
I would be interested to see that. Was this still on i386 ?

> Another question: Docs say
>   mvn -Dpgsql.pgconfig=...
>   java -Dpgconfig=/local/pgsql/bin/pg_config -jar pljava-packaging/...
> I had first tried mvn -Dpgconfig, which didn't work - is it really two
> different variable names?

They are different. Maven has a very large number of named properties,
often in prefix.foo form, so I followed that pattern, but the self-install
jar has very few.

> And is -Dpgconfig even used if for the installer, when
> -Dpgconfig.pkglibdir and -Dpgconfig.sharedir are given? I need to
> point these to debian/postgresql-$version-pljava/usr/... A "DESTDIR"
> mechanism would be nice to have here.

I think -Dpgconfig will not be used, if explicit -Dpgconfig.foo values
are given for all pljava/foo directory prefixes appearing in the jar.
As pkglibdir and sharedir are the only ones I see in the jar, it
should be sufficient to explicitly supply those two.

> 20:07:07 ERROR:  java.lang.ClassNotFoundException: org.postgresql.pljava.example.annotation.ComplexTuple
> 20:07:07 CONTEXT:  SQL statement "SELECT javatest.assertHasValues( CAST('(1,2)' AS javatest.complextuple), 1, 2)"

I believe you'll find this a Heisenbug, and it will go away if you repeat
the build and the test. It results from an interaction between the order
of commands in the jar's pljava.ddr file and the way PostgreSQL decides when
to get an updated snapshot. It will go away if you repeat the build because
pljava.ddr is not generated in a fully determined order, but only one that
satisfies the annotated provides/requires relationships. Most orderings do
not show the problem. :)

I had added workarounds already to ComplexScalar and UDTScalarIOTest,
but I must have missed ComplexTuple.


https://github.com/tada/pljava/blob/V1_5_1b2/pljava-examples/src/main/java/org/postgresql/pljava/example/annotation/ComplexScalar.java#L153

https://github.com/tada/pljava/blob/V1_5_1b2/pljava-examples/src/main/java/org/postgresql/pljava/example/annotation/UDTScalarIOTest.java#L221

That can be a github issue. I'll work on that before the release.

> 20:00:43 [ERROR] Failed to execute goal com.github.maven-nar:nar-maven-plugin:3.2.3:nar-validate
(default-nar-validate)on project pljava-so: NAR: One of two things may be wrong here:
 
> 20:00:43 [ERROR] 1. <Name> tag is missing inside the <Linker> tag of your NAR configuration
> 20:00:43 [ERROR] 2. no linker is defined in the aol.properties file for 'ppc64le.Linux.linker'
> 
> Is that a problem in pljava, or a general java-on-that-platform
> problem?

I think it is probably a Maven-on-that-platform problem.
The nar-maven-plugin does not seem to have any toolchain definition
for "ppc64le.Linux". As far as ppc, I only see "ppc64.AIX" in its
toolchain properties file:

https://github.com/maven-nar/nar-maven-plugin/blob/master/src/main/resources/com/github/maven_nar/aol.properties#L1048

("aol" for the nar-maven-plugin means architecture-OS-linker, not an early
internet service provider.)

PL/Java includes a contributed toolchain properties file for Solaris gcc:
https://github.com/tada/pljava/blob/V1_5_1b2/pljava-so/aol.solaris-gcc.properties

A similar contributed file for ppc64le.Linux should work. It could be
a github pull request.

-Chap


Re: PL/Java 1.5.1_BETA2

От
Chapman Flack
Дата:
On 09/16/18 16:27, Christoph Berg wrote:

>> 20:07:07 CONTEXT:  SQL statement "SELECT javatest.assertHasValues( CAST('(1,2)' AS javatest.complextuple), 1, 2)"

Yup, ComplexTuple again....


> 20:06:38 CONTEXT:  SQL statement "SELECT javatest.assertHasValues(CAST('(1,2)' AS point), 1, 2)"

Point this time. Sigh. Same issue.

Instead of the silly boot functions, I should probably add to PL/Java a way
to request a CommandCounterIncrement, and have install_jar always use it,
after loading the jar, before running the DDR commands.

> Also, does it have to be that verbose all the time, even if things
> worked?

I don't have very good control yet. There's a page on the wiki for
how I *want* the logging to work:

  https://github.com/tada/pljava/wiki/Thoughts-on-logging

... and I also want, eventually, the testing to happen in a more
standard way, such as TAP (tap4j and pgtap). But those goals still
need a substantial development effort. For now, I am stuck with one
example function that writes verbose log messages, and using INFO
for the succeeding ones and WARNING for the failing ones. And even
different PostgreSQL versions have different amounts of verbose
logging that they do for CREATE TYPE commands, etc.

> 20:22:25 Exception in thread "main" java.lang.ClassCastException: sun.org.mozilla.javascript.ConsString cannot be
castto java.lang.String
 

Could you try changing pljava-packaging/build.xml at line 353 from

computedPath = replacement + computedPath.slice(plen);

to

computedPath = new java.lang.String(replacement + computedPath.slice(plen));

I keep being surprised by differences in the JavaScript interpreter
supplied in different Java runtimes; the unchanged code has worked
without problems on so many. But I *think* this should work on your jdk7
and not break others....

-Chap


Re: PL/Java 1.5.1_BETA2

От
Chapman Flack
Дата:
On 09/16/18 17:36, Chapman Flack wrote:
> The nar-maven-plugin does not seem to have any toolchain definition
> for "ppc64le.Linux". As far as ppc, I only see "ppc64.AIX" in its
> toolchain properties file:
> 
>
https://github.com/maven-nar/nar-maven-plugin/blob/master/src/main/resources/com/github/maven_nar/aol.properties#L1048
> 
> ("aol" for the nar-maven-plugin means architecture-OS-linker, not an early
> internet service provider.)
> 
> PL/Java includes a contributed toolchain properties file for Solaris gcc:
> https://github.com/tada/pljava/blob/V1_5_1b2/pljava-so/aol.solaris-gcc.properties
> 
> A similar contributed file for ppc64le.Linux should work. It could be
> a github pull request.

Ideally, such a pull request would add documentation too, like
https://github.com/tada/pljava/commit/9a76b4a

-Chap


Re: PL/Java 1.5.1_BETA2

От
Christoph Berg
Дата:
Re: Chapman Flack 2018-09-17 <5B9ED559.5080208@anastigmatix.net>
> > 20:22:25 Exception in thread "main" java.lang.ClassCastException: sun.org.mozilla.javascript.ConsString cannot be
castto java.lang.String
 
> 
> Could you try changing pljava-packaging/build.xml at line 353 from
> 
> computedPath = replacement + computedPath.slice(plen);
> 
> to
> 
> computedPath = new java.lang.String(replacement + computedPath.slice(plen));

That shifted the problem a bit:

+ java -Dpgsql.pgconfig=/usr/lib/postgresql/9.4/bin/pg_config
-Dpgconfig.pkglibdir=debian/postgresql-9.4-pljava/usr/lib/postgresql/9.4/lib
-Dpgconfig.sharedir=debian/postgresql-9.4-pljava/usr/share/postgresql/9.4-jar
build-9.4/pljava-pg9.4-amd64-Linux-gpp.jar
Exception in thread "main" java.lang.ClassCastException: sun.org.mozilla.javascript.NativeJavaObject cannot be cast to
java.lang.String
    at org.gjt.cuspy.JarX.extract(JarX.java:670)
    at org.gjt.cuspy.JarX.extract(JarX.java:445)
    at org.gjt.cuspy.JarX.main(JarX.java:419)

> I keep being surprised by differences in the JavaScript interpreter
> supplied in different Java runtimes; the unchanged code has worked
> without problems on so many. But I *think* this should work on your jdk7
> and not break others....

We could just declare openjdk-8 to be required on the packaging side,
that excludes jessie and trusty which will be EOL soonish anyway.

Christoph


Re: PL/Java 1.5.1_BETA2

От
Chapman Flack
Дата:
On 09/17/18 10:47, Christoph Berg wrote:
> Re: Chapman Flack 2018-09-17 <5B9ED559.5080208@anastigmatix.net>
>> computedPath = replacement + computedPath.slice(plen);
>> to
>> computedPath = new java.lang.String(replacement + computedPath.slice(plen));
> 
> ClassCastException: sun.org.mozilla.javascript.NativeJavaObject cannot be cast to java.lang.String

I have no words. :)

One more (less intuitive) thing might be worth trying:

computedPath = String(replacement + computedPath.slice(plen));


Would you have time to try it that way? I can't make the exception
happen here, even with an Oracle jdk7 that is installed.

Correction ... I can make it happen (prior to this change) in an
icedtea-6 build installed here, with a dubious version of the
Rhino-to-JSR223 adapter jar.

And the change above to String(...) does seem to fix it (without
breaking any of the other versions I can test).

I was first guessing the issue would be the change of the bundled
JavaScript implementation from Rhino in jdk6/7 to Nashorn in jdk8.
But in both cases there is also an adapter layer so that the Java
JSR223 API is the same whether Rhino or Nashorn is underneath. And
the pre-jdk8 OpenJDK builds seem to have ended up with an older,
incomplete version of the Rhino adapter (rhino-js-engine.jar) than
what was in the Oracle builds, and it doesn't handle all of the data
type conversions as it should.

Apparently it will correctly convert a JavaScript String to a Java
String. Apparently it does *not* know what to do with a JavaScript
wrapper of an actual Java string. :) And it seems Rhino has a
clever object, this ConsString, that is an implementation detail of
a JavaScript string created efficiently by concatenation, and of
course the adapter doesn't know what to do with that either.

So using that to construct a brand new ordinary JavaScript String
seems to be the one way to get a successful conversion.

-Chap


Re: PL/Java 1.5.1_BETA2

От
Christoph Berg
Дата:
Re: Chapman Flack 2018-09-18 <5BA058C6.8010807@anastigmatix.net>
> One more (less intuitive) thing might be worth trying:
> 
> computedPath = String(replacement + computedPath.slice(plen));

That worked, tested on openjdk-7/trusty and openjdk-8/sid.

(Now it doesn't pass the "install_jar" test anymore because of
java.lang.ClassNotFoundException: org.postgresql.pljava.example.annotation.ComplexTuple...)

Submitted as https://github.com/tada/pljava/pull/175

Christoph


Re: PL/Java 1.5.1_BETA2

От
Christoph Berg
Дата:
Re: Chapman Flack 2018-09-17 <5B9F039A.5030701@anastigmatix.net>
> > The nar-maven-plugin does not seem to have any toolchain definition
> > for "ppc64le.Linux". As far as ppc, I only see "ppc64.AIX" in its
> > toolchain properties file:
> > 
> >
https://github.com/maven-nar/nar-maven-plugin/blob/master/src/main/resources/com/github/maven_nar/aol.properties#L1048
> > 
> > ("aol" for the nar-maven-plugin means architecture-OS-linker, not an early
> > internet service provider.)

:)

> > PL/Java includes a contributed toolchain properties file for Solaris gcc:
> > https://github.com/tada/pljava/blob/V1_5_1b2/pljava-so/aol.solaris-gcc.properties
> > 
> > A similar contributed file for ppc64le.Linux should work. It could be
> > a github pull request.
> 
> Ideally, such a pull request would add documentation too, like
> https://github.com/tada/pljava/commit/9a76b4a

I got it working:

https://salsa.debian.org/postgresql/postgresql-pljava/blob/master/debian/patches/aol-properties

It probably makes more sense to have this merged into nar-maven-plugin
than to have it live as a wart in pljava. I'll submit a PR there and
stick with the patch in the Debian package until there's a nar release
that has it.

Christoph


Re: PL/Java 1.5.1_BETA2

От
Christoph Berg
Дата:
Re: To Chapman Flack 2018-09-18 <20180918092141.GC6196@msg.df7cb.de>
> It probably makes more sense to have this merged into nar-maven-plugin
> than to have it live as a wart in pljava. I'll submit a PR there and
> stick with the patch in the Debian package until there's a nar release
> that has it.

FTR, https://github.com/maven-nar/nar-maven-plugin/pull/328

Christoph


Re: PL/Java 1.5.1_BETA2

От
Christoph Berg
Дата:
Re: Chapman Flack 2018-09-16 <5B9ECCD1.1050309@anastigmatix.net>
> > i.e. it's replacing both occurrences of "i386" by "1".
> 
> Ok, that's irritating. I can guess what is happening. I implement the
> -Dpljava.libjvmdefault by passing the string into cpp with a -D, and that
> gets converted to a string using the CppAsString2 macro, and that apparently
> is not protected from expansion of other macros, and I bet something is
> defining i386 to be 1.
> 
> Phooey.
> 
> I don't know if that will be easy to fix using cpp. An alternative would be
> to pass it to cpp already quoted according to C rules, but that would be
> somewhat ugly to express in a Maven build script.

We could try "-Ui386" if there's a way to pass that flag into the
build process?

Filed at https://github.com/tada/pljava/issues/176, let's continue
there.

> Your SET command to correct the library name completed the installation
> of PL/Java, but at that point it was no longer within a CREATE EXTENSION
> command, so the various functions, tables, etc., have just been installed
> 'loose' (exactly as they would if you were installing in 9.0 or earlier
> with no 'extension' mechanism).

Oh, that's unexpected. Wouldn't it be time to unsupport that and move
to a "normal" extension .sql script?

> > Proceeding to load the examples:
> 
> > Server beendete die Verbindung unerwartet
> 
> Is there anything in the server log with details of the crash?
> In many cases there will be a lengthy message from the JVM in the server
> log, and that message will contain a file name (something like
> /tmp/hs_err_pidnnnn.log) which will contain *even more* detail.
> I would be interested to see that. Was this still on i386 ?

Filed as https://github.com/tada/pljava/issues/177 with attachment.

> > 20:07:07 ERROR:  java.lang.ClassNotFoundException: org.postgresql.pljava.example.annotation.ComplexTuple
> > 20:07:07 CONTEXT:  SQL statement "SELECT javatest.assertHasValues( CAST('(1,2)' AS javatest.complextuple), 1, 2)"
> 
> I believe you'll find this a Heisenbug, and it will go away if you repeat
> the build and the test.

https://github.com/tada/pljava/issues/178

Christoph


PL/Java 1.5.1

От
Chapman Flack
Дата:
I expect to shortly tag a 1.5.1 release (just in time to say it builds
with PG 11).

I had a few last-minute thoughts about the .deb builds (in order of
outlandishness, maybe).

1. It looks as if they are being built with libjvmdefault => java 8.
   Do some of the distro releases they are being built for have a later
   version of Java as the default? (If I'm looking in the right places,
   it seems that the Ubuntu default-jdk-headless is 8 for xenial and
   artful, 10 for bionic, 11 for cosmic. Debian's default-jdk seems to
   be 7 in wheezy and jessie, 8 in stretch, 10 in buster and sid,
   11 in experimental).

   It might be nice to build pljava packages where libjvmdefault => the
   default version for the release. It still has to be /built/ with 8, but
   can then run with the later versions on releases that have them. The
   user can always set pljava.libjvm_location to a different version, but
   this would have the property that leaving it alone means getting the
   expected Java version, and (in recent releases) having access to newer
   Java features, without having to take extra steps.

2. You seem to be building packages for PG from 11 back to 9.3. For that
   range of PG versions, it would be possible to add the -Psaxon-examples
   option on the build command (if you can stomach having the 5 MB
   Saxon-HE-9.8.0.14.jar as a build dependency). The jar can be left out
   of the package as an optional runtime dependency. The Saxon examples
   would just be included in the pljava-examples jar, they would get
   automatically loaded in PG if the user installs the jar. They wouldn't
   work unless the user also installed the Saxon-HE jar, but that would be
   the only step needed for someone who might want to try them.

   It's actually safe to add -Psaxon-examples when building for any PG
   version back to 8.4. Before 8.4, those examples can't be included
   in pljava-examples.jar because their deployment SQL commands would
   be syntax errors and prevent installing the jar, which is why I made
   the build option.

3. Here is the possibly outlandish thought: I see in
   pool/main/p/postgresql-pljava/ that there are existing .debs in there
   for PG 9.2 back to 8.1, and they are based on really ancient PL/Java
   versions with serious known bugs. I have been taking pains to make sure
   PL/Java 1.5.x stays buildable back to 8.2 (but not 8.1, sadly). It would
   be great if there could be packages in the repo so that those old buggy
   ones would not be the only ones available. I understand that your build
   arrangements probably don't even have those six PG versions present to
   build against, and there is probably no justification for doing ongoing,
   automated builds for them. Even if it were possible to just do one round
   of builds and leave them there (as it appears those old packages are
   simply left in the directory from years ago), that would benefit anyone
   who needs, for some reason, to use such an old PG version.

Someone has actually opened a PL/Java issue this week who is using
PG 8.3. :)  I did not press for details about why.... That's on Windows
though.

Cheers,
-Chap


Re: PL/Java 1.5.1

От
Chapman Flack
Дата:
On 10/15/18 22:38, Chapman Flack wrote:
>    artful, 10 for bionic, 11 for cosmic. Debian's default-jdk seems to
>    be 7 in wheezy and jessie, 8 in stretch, 10 in buster and sid,
>    11 in experimental).
> 
>    It might be nice to build pljava packages where libjvmdefault => the
>    default version for the release. It still has to be /built/ with 8, but

... except in this scheme, it would have to be built with 7 for wheezy or
jessie (the build JDK can't be /newer/ than the runtime). Slipped my mind.

-Chap


Re: PL/Java 1.5.1

От
Christoph Berg
Дата:
Re: Chapman Flack 2018-10-16 <5BC54F32.7020705@anastigmatix.net>
> 1. It looks as if they are being built with libjvmdefault => java 8.
>    Do some of the distro releases they are being built for have a later
>    version of Java as the default? (If I'm looking in the right places,
>    it seems that the Ubuntu default-jdk-headless is 8 for xenial and
>    artful, 10 for bionic, 11 for cosmic. Debian's default-jdk seems to
>    be 7 in wheezy and jessie, 8 in stretch, 10 in buster and sid,
>    11 in experimental).

Seems about right.

>    It might be nice to build pljava packages where libjvmdefault => the
>    default version for the release. It still has to be /built/ with 8, but
>    can then run with the later versions on releases that have them. The
>    user can always set pljava.libjvm_location to a different version, but
>    this would have the property that leaving it alone means getting the
>    expected Java version, and (in recent releases) having access to newer
>    Java features, without having to take extra steps.

Currently pljava.libjvm_location is set to the jvm version the package
was built with. The .deb will have this dependency:

Depends: ${jre:Depends} | default-jre-headless

The first problem here is that this is missing "| java-runtime-headless"
to allow arbitrary jre versions.

What we probably should to is to change this to

Depends: default-jre-headless | ${jre:Depends} | java-runtime-headless

Apt doesn't consistently pick the first alternative, but there's rule
that the preferred one should come first.

... and also point pljava.libjvm_location at the default version.

> 2. You seem to be building packages for PG from 11 back to 9.3. For that
>    range of PG versions, it would be possible to add the -Psaxon-examples
>    option on the build command (if you can stomach having the 5 MB
>    Saxon-HE-9.8.0.14.jar as a build dependency). The jar can be left out
>    of the package as an optional runtime dependency. The Saxon examples
>    would just be included in the pljava-examples jar, they would get
>    automatically loaded in PG if the user installs the jar. They wouldn't
>    work unless the user also installed the Saxon-HE jar, but that would be
>    the only step needed for someone who might want to try them.
> 
>    It's actually safe to add -Psaxon-examples when building for any PG
>    version back to 8.4. Before 8.4, those examples can't be included
>    in pljava-examples.jar because their deployment SQL commands would
>    be syntax errors and prevent installing the jar, which is why I made
>    the build option.

I'm not really following how that relates to the range of PG versions?
Because it's "only" 8.4+ ?

> 3. Here is the possibly outlandish thought: I see in
>    pool/main/p/postgresql-pljava/ that there are existing .debs in there
>    for PG 9.2 back to 8.1, and they are based on really ancient PL/Java
>    versions with serious known bugs. I have been taking pains to make sure
>    PL/Java 1.5.x stays buildable back to 8.2 (but not 8.1, sadly). It would
>    be great if there could be packages in the repo so that those old buggy
>    ones would not be the only ones available. I understand that your build
>    arrangements probably don't even have those six PG versions present to
>    build against, and there is probably no justification for doing ongoing,
>    automated builds for them. Even if it were possible to just do one round
>    of builds and leave them there (as it appears those old packages are
>    simply left in the directory from years ago), that would benefit anyone
>    who needs, for some reason, to use such an old PG version.

Some of the very old packages were not built on/for apt.pg.o but
copied over from Debian. The oldest PG version we have is 8.2, which
I'm still keeping alive, but in sid-pgdg only. The other distributions
have what was supported while they were alive, so the window of
package versions updated shifted over time.

We are usually not updating modules for PG versions that aren't
supported anymore, just to cut the maintenance down, but we could
probably give the older PG versions a one-time try. (This wouldn't
extend to unsupported distributions, though - the build chroots are
gone.)

> Someone has actually opened a PL/Java issue this week who is using
> PG 8.3. :)  I did not press for details about why.... That's on Windows
> though.

... that doesn't make it any better :P

Christoph


Re: PL/Java 1.5.1

От
Chapman Flack
Дата:
On 10/16/18 07:32, Christoph Berg wrote:

>> 2. You seem to be building packages for PG from 11 back to 9.3. For that
>>    range of PG versions, it would be possible to add the -Psaxon-examples
...
>>    It's actually safe to add -Psaxon-examples when building for any PG
>>    version back to 8.4. Before 8.4, those examples can't be included

> I'm not really following how that relates to the range of PG versions?
> Because it's "only" 8.4+ ?

The saxon-examples functions are declared with SQL syntax that didn't
exist pre-8.4, so they would just be rejected as syntax errors in 8.3
or 8.2.

The documentation illustrating how to call them uses named-parameter
notation with => so it works as far back as 9.5, and with a simple
substitution of := for => they can be called the same way as far back
as 9.0. But the illustrations also show FROM clauses with LATERAL or
with a function referring to earlier entries, which is only possible
in 9.3+.

So the saxon-examples PG version range dependency boils down to:

* the functions can be built and installed in any version 8.4 and up
* they are easy to use in 9.5 and up, following exactly the illustrations
    in the docs
* they are almost as easy to use in 9.3 and 9.4, changing => to :=
* from 9.2 back to 8.4, they work, but require increasing devilish
    cleverness to write SQL queries that use them.

-Chap


Re: PL/Java 1.5.1

От
Chapman Flack
Дата:
On 10/15/2018 11:32 PM, Chapman Flack wrote:
> On 10/15/18 22:38, Chapman Flack wrote:
>>    artful, 10 for bionic, 11 for cosmic. Debian's default-jdk seems to
>>    be 7 in wheezy and jessie, 8 in stretch, 10 in buster and sid,
>>    11 in experimental).
>>
>>    It might be nice to build pljava packages where libjvmdefault => the
>>    default version for the release. It still has to be /built/ with 8, but
> 
> ... except in this scheme, it would have to be built with 7 for wheezy or
> jessie (the build JDK can't be /newer/ than the runtime). Slipped my mind.

Hmm, it also slipped my mind that the pom.xml contains
<target>1.6</target>, so it should be ok to build with 8 everywhere,
the built classes should still work in Java 6/7. I don't think
I've confirmed that recently, but I can check easily enough later today.

-Chap


Re: PL/Java 1.5.1

От
Christoph Berg
Дата:
Jessie doesn't have Java 8, so that question does not matter for packaging.

Re: PL/Java 1.5.1

От
Chapman Flack
Дата:
Hi Christoph,

Was there some objection in principle (or maybe just download size) to
building with -Psaxon-examples ?

If it is not objectionable, my preference would be to build with that
option (whenever the PG version >= 8.4 anyway).

A lot of the examples are just toy learning aids or tests, but the Saxon
examples are there for a useful, non-trivial purpose: to give real SQL:2006+
XML Query functionality, which core PG still lacks.

So I would like to make it as easy as possible for the user to try out
those examples. Even leaving the Saxon library as an optional download,
so one extra thing would have to be downloaded before those examples
work, that would still be less fuss than having to build the example
code separately AND download the library.

What do you think?

Thanks,
-Chap


On 10/16/18 08:49, Chapman Flack wrote:
> On 10/16/18 07:32, Christoph Berg wrote:
> 
>>> 2. You seem to be building packages for PG from 11 back to 9.3. For that
>>>    range of PG versions, it would be possible to add the -Psaxon-examples
> ...
>>>    It's actually safe to add -Psaxon-examples when building for any PG
>>>    version back to 8.4. Before 8.4, those examples can't be included
> 
>> I'm not really following how that relates to the range of PG versions?
>> Because it's "only" 8.4+ ?
> 
> The saxon-examples functions are declared with SQL syntax that didn't
> exist pre-8.4, so they would just be rejected as syntax errors in 8.3
> or 8.2.
> 
> The documentation illustrating how to call them uses named-parameter
> notation with => so it works as far back as 9.5, and with a simple
> substitution of := for => they can be called the same way as far back
> as 9.0. But the illustrations also show FROM clauses with LATERAL or
> with a function referring to earlier entries, which is only possible
> in 9.3+.
> 
> So the saxon-examples PG version range dependency boils down to:
> 
> * the functions can be built and installed in any version 8.4 and up
> * they are easy to use in 9.5 and up, following exactly the illustrations
>     in the docs
> * they are almost as easy to use in 9.3 and 9.4, changing => to :=
> * from 9.2 back to 8.4, they work, but require increasing devilish
>     cleverness to write SQL queries that use them.
> 
> -Chap
> 


Re: PL/Java 1.5.1

От
Christoph Berg
Дата:
Re: Chapman Flack 2018-10-19 <5BC92CB0.2020007@anastigmatix.net>
> Was there some objection in principle (or maybe just download size) to
> building with -Psaxon-examples ?

I'm just lacking time atm, I filed your last mail as
https://salsa.debian.org/postgresql/postgresql-pljava/issues/1
so it doesn't get lost completely.

> If it is not objectionable, my preference would be to build with that
> option (whenever the PG version >= 8.4 anyway).
> 
> A lot of the examples are just toy learning aids or tests, but the Saxon
> examples are there for a useful, non-trivial purpose: to give real SQL:2006+
> XML Query functionality, which core PG still lacks.
> 
> So I would like to make it as easy as possible for the user to try out
> those examples. Even leaving the Saxon library as an optional download,
> so one extra thing would have to be downloaded before those examples
> work, that would still be less fuss than having to build the example
> code separately AND download the library.
> 
> What do you think?

It makes a lot of sense. If I should forget, could you ping me again
once the whole upload-everything-for-PG11 dust has settled in a few
weeks?

Christoph