Обсуждение: Making Vars outer-join aware
[ Before settling into commitfest mode, I wanted to put out a snapshot
of what I've been working on for the past few weeks. This is not
anywhere near committable, but I think people might be interested
in looking at it now anyway. ]
We've had many discussions (eg [1][2]) about the need to treat outer
joins more honestly in parsed queries, so that the planner's reasoning
about things like equivalence classes can stand on a firmer foundation.
The attached patch series makes a start at doing that, and carries the
idea as far as a working system in which all Vars are labeled as to
which outer joins can null them. I have not yet gotten to the fun part
of fixing or ripping out all the higher-level planner logic that could
now be simplified or removed entirely --- but as examples, I believe
that reconsider_outer_join_clause no longer does anything useful, and
a lot of the logic in deconstruct_jointree and distribute_qual_to_rels
could be simplified, and we shouldn't need the notion of
second-class-citizen EquivalenceClasses for "below outer join" cases.
Another thing that could be built on this infrastructure, but I've
not tackled it yet, is fixing the known semantic bugs in grouping sets
[3][4]. What I have in mind there is to invent a dummy RTE representing
the action of grouping, and use Vars that are marked as nullable by that
RTE to represent possibly-nullable grouping-set expressions.
The main thing here that differs from my previous ideas is that the
nulling-rel labeling is placed directly on Vars or PlaceHolderVars,
whereas I had been advocating to use some sort of wrapper node instead.
After several failed attempts I decided that it was too complicated
to separate the labeling from the Var itself. (I'll just mention one
weak spot in that idea: the entire API concept of replace_rte_variables
breaks down, because many of the callbacks using it need to manipulate
nulling-rel labeling along the way, which they can only do if they
see it on the Var they're passed.) Of course, the objection to doing it
like this is that it bloats struct Var, which is a mighty common node
type, even in cases where there's no outer join anywhere. However, on
a 64-bit machine struct Var would widen from 40 to 48 bytes, which is
basically free considering that palloc will round the allocation up to
64 bytes. There's a more valid consideration that the pg_node_tree
representation of a Var will get longer; but really, if you're worried
about that you should be designing a more compact storage representation
for node trees. There's also reason to fear that the distributed cost
of maintaining these extra Bitmapsets will pose a noticeable drag on
parsing or planning speed. However, I see little point in doing
performance measurements when we've not yet reaped any of the
foreseeable planner improvements.
Anyway, on to the patch series. I've broken it up a little bit
for review, but note I'm not claiming that the intermediate states
would compile or pass regression testing.
0000: Write some overview documentation in optimizer/README.
This might be worth reading even if you lack time to look at the code.
I went into some detail about Var semantics, and also added a discussion
of PlaceHolderVars, which Robert has rightfully complained are badly
underdocumented. (At one point I'd thought maybe we could get rid of
PlaceHolderVars, but now I see them as complementary to this work ---
indeed, arguably the reason for them to exist is so that a Var's
nullingrels markers are not lost when replacing it with a pulled-up
expression from a subquery.) The changes in the section about
EquivalenceClasses are pretty rough and speculative, since I've not
actually coded those changes yet.
0001: add infrastructure, namely add new fields to assorted data
structures and update places like backend/nodes/*.c. This is mostly
pretty boring, except for the commentary changes in *nodes.h.
0002: change the parser to correctly label emitted Vars with the
sets of outer joins that can null them, according to the query text
as-written. (That is, we don't account here for the possibility
of join strength reduction or anything like that.)
0003: fix the planner to cope, including adjusting nullingrel labeling
for join elimination, join strength reduction, join reordering, etc.
This is still WIP to some extent. In particular note all the XXX
comments in setrefs.c complaining about how we're not verifying that the
nullingrel states agree when matching upper-level Vars to lower-level
ones. This is partly due to setrefs.c not having readily-available info
about which outer joins are applied at which plan nodes (should we
expend the space to mark them?), and partly because I'm not sure
that we can enforce 100% consistency there anyway. Because of the
compromises involved in implementing outer-join identity 3 (see 0000),
there will be cases where an upper Var that "should" have a nullingrel
bit set will not. I don't know how to make a hole in the check that
will allow those cases without rendering such checking mostly useless.
Is there a way that we can do the identity-3 transformation without
being squishy about the nullability state of Vars in the moved qual?
I've not thought of one, but it's very annoying considering that the
whole point of this patch series is to not be squishy about that.
I guess the good news is that the squishiness only seems to be needed
during final transformation of the plan, where all we are losing is
the ability to detect bugs in earlier planner stages. All of the
decisions that actually count seem to work fine without compromises.
So far the patch series changes no regression test results, and I've
not added any new tests either. The next steps will probably have
visible effects in the form of improved plans for some test queries.
Anyway, even though this is far from done, I'm pretty pleased with
the results so far, so I thought I'd put it out for review by
anyone who cares to take a look. I'll add it to the September CF
in hopes that it might be more or less finished by then, and so
that the cfbot will check it out.
regards, tom lane
[1] https://www.postgresql.org/message-id/7771.1576452845%40sss.pgh.pa.us
[2] https://www.postgresql.org/message-id/flat/15848.1576515643%40sss.pgh.pa.us
[3] https://www.postgresql.org/message-id/17071-24dc13fbfa29672d%40postgresql.org
[4] https://www.postgresql.org/message-id/CAMbWs48AtQTQGk37MSyDk_EAgDO3Y0iA_LzvuvGQ2uO_Wh2muw%40mail.gmail.com
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index 41c120e0cd..2b30d22aed 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -295,6 +295,191 @@ Therefore, we don't merge FROM-lists if the result would have too many
FROM-items in one list.
+Vars and PlaceHolderVars
+------------------------
+
+A Var node is simply the parse-tree representation of a table column
+reference. However, in the presence of outer joins, that concept is
+more subtle than it might seem. We need to distinguish the values of
+a Var "above" and "below" any outer join that could force the Var to
+null. As an example, consider
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE foo(t2.z)
+
+(Assume foo() is not strict, so that we can't reduce the left join to
+a plain join.) A naive implementation might try to push the foo(t2.z)
+call down to the scan of t2, but that is not correct because
+(a) what foo() should actually see for a null-extended join row is NULL,
+and (b) if foo() returns false, we should suppress the t1 row from the
+join altogether, not emit it with a null-extended t2 row. On the other
+hand, it *would* be correct (and desirable) to push the call down to
+the scan level if the query were
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y AND foo(t2.z))
+
+This motivates considering "t2.z" within the left join's ON clause
+to be a different value from "t2.z" outside the JOIN clause. The
+former can be identified with t2.z as seen at the relation scan level,
+but the latter can't.
+
+Another example occurs in connection with EquivalenceClasses (discussed
+below). Given
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE t1.x = 42
+
+we would like to put t1.x and t2.y and 42 into the same EquivalenceClass
+and then derive "t2.y = 42" to use as a restriction clause for the scan
+of t2. However, it'd be wrong to conclude that t2.y will always have
+the value 42, or that it's equal to t1.x in every joined row. We can
+solve this problem by deeming that "t2.y" in the ON clause refers to
+the relation-scan-level value of t2.y, but not to the value that y will
+have in joined rows, where it might be NULL rather than equal to t1.x.
+
+Therefore, Var nodes are decorated with "varnullingrels", which are sets
+of the rangetable indexes of outer joins that potentially null this Var
+at the point where it appears in the query. (Using a set, not an
+ordered list, is fine since it doesn't matter which join forced the
+value to null; and that avoids having to change the representation when
+we consider different outer-join orders.) In the examples above, all
+occurrences of t1.x would have empty varnullingrels, since the left join
+doesn't null t1. The t2 references within the JOIN ON clauses would
+also have empty varnullingrels, but other references to t2 columns would
+be labeled with the index of the JOIN's rangetable entry (RTE), so that
+they'd be understood as potentially different from the t2 values seen at
+scan level. Labeling t2.z in the WHERE clause with the JOIN's RT index
+lets us recognize that that occurrence of foo(t2.z) cannot be pushed
+down to the t2 scan level: we cannot evaluate that value at the scan
+level, but only after the join has been done.
+
+For LEFT and RIGHT outer joins, only Vars coming from the nullable side
+of the join are marked with that join's RT index. For FULL joins, all
+Vars are marked. (Such marking doesn't let us tell which side of the
+full join a Var came from; but that information can be found elsewhere
+at need.)
+
+Notionally, a Var having nonempty varnullingrels can be thought of as
+ CASE WHEN any-of-these-outer-joins-produced-a-null-extended-row
+ THEN NULL
+ ELSE the-scan-level-value-of-the-column
+ END
+It's only notional, because no such calculation is ever done explicitly.
+In a finished plan, Vars occurring in scan-level plan nodes represent
+the actual table column values, but upper-level Vars are always
+references to outputs of lower-level plan nodes. When a join node emits
+a null-extended row, it just returns nulls for the relevant output
+columns rather than copying up values from its input. Because we don't
+ever have to do this calculation explicitly, it's not necessary to
+distinguish which side of an outer join got null-extended, which'd
+otherwise be essential information for FULL JOIN cases.
+
+Outer join identity 3 (discussed above) complicates this picture
+a bit. In the form
+ A leftjoin (B leftjoin C on (Pbc)) on (Pab)
+all of the Vars in clauses Pbc and Pab will have empty varnullingrels,
+but if we start with
+ (A leftjoin B on (Pab)) leftjoin C on (Pbc)
+then the parser will have marked Pbc's B Vars with the A/B join's
+RT index, making this form artificially different from the first.
+We resolve this by, after noting that Pbc is strict, running
+through that clause and removing any varnullingrels references to
+left joins in the lefthand side. That makes the clause equivalent
+to what it would have looked like if the first form were presented,
+so that we can freely consider both join orders. However, because
+we have done this, if we do construct a plan based on the second
+join order then we cannot cross-check that B Vars appearing above
+the A/B join are all marked with that join's RT index. That would
+be a useful cross-check to have to catch planner bugs, but it
+doesn't seem useful enough to justify the extra complication of
+devising a representation that would support it.
+
+Outer joins also complicate handling of subquery pull-up. Consider
+
+ SELECT ..., ss.x FROM tab1
+ LEFT JOIN (SELECT *, 42 AS x FROM tab2) ss ON ...
+
+We want to be able to pull up the subquery as discussed previously,
+but we can't just replace the "ss.x" Var in the top-level SELECT list
+with the constant 42. That'd result in always emitting 42, rather
+than emitting NULL in null-extended join rows.
+
+To solve this, we introduce the concept of PlaceHolderVars.
+A PlaceHolderVar is somewhat like a Var, in that its value originates
+at a relation scan level and can then be forced to null by higher-level
+outer joins; hence PlaceHolderVars carry a set of nulling rel IDs just
+like Vars. Unlike a Var, whose original value comes from a table,
+a PlaceHolderVar's original value is defined by a query-determined
+expression ("42" in this example); so we represent the PlaceHolderVar
+as a node with that expression as child. We insert a PlaceHolderVar
+whenever subquery pullup needs to replace a subquery-referencing Var
+that has nonempty varnullingrels with an expression that is not simply a
+Var. (When the replacement expression is a pulled-up Var, we can just
+add the replaced Var's varnullingrels to its set. Also, if the replaced
+Var has empty varnullingrels, we don't need a PlaceHolderVar: there is
+nothing that'd force the value to null, so the pulled-up expression is
+fine to use as-is.) In a finished plan, a PlaceHolderVar becomes just
+the contained expression at whatever plan level it's supposed to be
+evaluated at, and then upper-level occurrences are replaced by
+references to that output column of the lower plan level. That causes
+the value to go to null when appropriate at an outer join, in the same
+way as for Vars. Thus, PlaceHolderVars are never seen outside the
+planner.
+
+PlaceHolderVars (PHVs) are more complicated than Vars in another way:
+their original value might need to be calculated at a join, not a
+base-level relation scan. This can happen if a pulled-up subquery
+contains a join. Because of this, a PHV can create a join order
+constraint that wouldn't otherwise exist, to ensure that it can
+be calculated before it is used. A PHV's expression can also contain
+LATERAL references, adding complications that are discussed below.
+
+
+Relation Identification and Qual Clause Placement
+-------------------------------------------------
+
+A qual clause obtained from WHERE or JOIN/ON can be enforced at the lowest
+scan or join level that includes all relations used in the clause. For
+this purpose we consider that outer joins listed in varnullingrels or
+phnullingrels are used in the clause, since we can't compute the qual's
+result correctly until we know whether such Vars have gone to null.
+
+The one exception to this general rule is that a non-degenerate outer
+JOIN/ON qual (one that references the non-nullable side of the join)
+cannot be enforced below that join, even if it doesn't reference the
+nullable side. Pushing it down into the non-nullable side would result
+in rows disappearing from the join's result, rather than appearing as
+null-extended rows. To handle that, when we identify such a qual we
+artificially add the join's minimum input relid set to the set of
+relations it is considered to use, forcing it to be evaluated exactly at
+that join level. The same happens for outer-join quals that mention no
+relations at all.
+
+When attaching a qual clause to a join plan node that is performing
+an outer join, the qual clause is considered a "join clause" (that
+is, it is applied before the join) if it does not use that specific
+outer join, or a "filter clause" (applied after the join) if it does
+use that outer join.
+
+These things lead us to identify join relations within the planner
+by the sets of base relation RT indexes plus outer join RT indexes
+that they include. In that way, the sets of relations used by qual
+clauses can be directly compared to join relations' relid sets to
+see where to place the clauses. These identifying sets are unique
+because, for any given collection of base relations, there is only
+one valid set of outer joins to have performed along the way to
+joining that set of base relations (although the order of applying
+them could vary, as discussed above).
+
+SEMI joins do not have RT indexes, because they are artifacts made by
+the planner rather than the parser. (We could create rangetable
+entries for them, but there seems no need at present.) This does not
+cause a problem for qual placement, because the nullable side of a
+semijoin is not referenceable from above the join, so there is never a
+need to cite it in varnullingrels or phnullingrels. It does not cause
+a problem for join relation identification either, since again whether
+a semijoin has been completed is implicit in the set of base relations
+included in the join.
+
+
Optimizer Functions
-------------------
@@ -437,11 +622,10 @@ inputs.
EquivalenceClasses
------------------
-During the deconstruct_jointree() scan of the query's qual clauses, we look
-for mergejoinable equality clauses A = B whose applicability is not delayed
-by an outer join; these are called "equivalence clauses". When we find
-one, we create an EquivalenceClass containing the expressions A and B to
-record this knowledge. If we later find another equivalence clause B = C,
+During the deconstruct_jointree() scan of the query's qual clauses, we
+look for mergejoinable equality clauses A = B. When we find one, we
+create an EquivalenceClass containing the expressions A and B to record
+that they are equal. If we later find another equivalence clause B = C,
we add C to the existing EquivalenceClass for {A B}; this may require
merging two existing EquivalenceClasses. At the end of the scan, we have
sets of values that are known all transitively equal to each other. We can
@@ -473,15 +657,26 @@ asserts that at any plan node where more than one of its member values
can be computed, output rows in which the values are not all equal may
be discarded without affecting the query result. (We require all levels
of the plan to enforce EquivalenceClasses, hence a join need not recheck
-equality of values that were computable by one of its children.) For an
-ordinary EquivalenceClass that is "valid everywhere", we can further infer
-that the values are all non-null, because all mergejoinable operators are
-strict. However, we also allow equivalence clauses that appear below the
-nullable side of an outer join to form EquivalenceClasses; for these
-classes, the interpretation is that either all the values are equal, or
-all (except pseudo-constants) have gone to null. (This requires a
-limitation that non-constant members be strict, else they might not go
-to null when the other members do.) Consider for example
+equality of values that were computable by one of its children.)
+
+We can further infer that the values are all non-null, because all
+mergejoinable operators are strict. This is a little tricky in the
+presence of outer joins. Consider
+
+ SELECT *
+ FROM a LEFT JOIN
+ (SELECT * FROM b LEFT JOIN c ON b.y = c.z WHERE b.y = 10) ss
+ ON a.x = ss.y
+ WHERE a.x = 42;
+
+We can form the EquivalenceClass {b.y c.z 10} and thereby apply c.z = 10
+while scanning c. However it would be incorrect to conclude that a.x
+is also a member of that EquivalenceClass. Instead, we form a second
+EquivalenceClass {a.x ss.y 42}, where (as discussed earlier) ss.y
+references the same table column as b.y but has a different
+varnullingrels label and is therefore considered a distinct Var.
+
+If the lower join were INNER:
SELECT *
FROM a LEFT JOIN
@@ -489,40 +684,23 @@ to null when the other members do.) Consider for example
ON a.x = ss.y
WHERE a.x = 42;
-We can form the below-outer-join EquivalenceClass {b.y c.z 10} and thereby
-apply c.z = 10 while scanning c. (The reason we disallow outerjoin-delayed
-clauses from forming EquivalenceClasses is exactly that we want to be able
-to push any derived clauses as far down as possible.) But once above the
-outer join it's no longer necessarily the case that b.y = 10, and thus we
-cannot use such EquivalenceClasses to conclude that sorting is unnecessary
-(see discussion of PathKeys below).
-
-In this example, notice also that a.x = ss.y (really a.x = b.y) is not an
-equivalence clause because its applicability to b is delayed by the outer
-join; thus we do not try to insert b.y into the equivalence class {a.x 42}.
-But since we see that a.x has been equated to 42 above the outer join, we
-are able to form a below-outer-join class {b.y 42}; this restriction can be
-added because no b/c row not having b.y = 42 can contribute to the result
-of the outer join, and so we need not compute such rows. Now this class
-will get merged with {b.y c.z 10}, leading to the contradiction 10 = 42,
-which lets the planner deduce that the b/c join need not be computed at all
-because none of its rows can contribute to the outer join. (This gets
-implemented as a gating Result filter, since more usually the potential
-contradiction involves Param values rather than just Consts, and thus has
-to be checked at runtime.)
+then ss.y is not any different from b.y and we'd end up with the
+EquivalenceClass {a.x b.y c.z 10 42}. This leads to the contradiction
+10 = 42, which lets the planner deduce that the b/c join need not be
+computed at all because none of its rows can contribute to the outer
+join. (This gets implemented as a gating Result filter, since more
+usually the potential contradiction involves Param values rather than
+just Consts, and thus has to be checked at runtime.)
To aid in determining the sort ordering(s) that can work with a mergejoin,
we mark each mergejoinable clause with the EquivalenceClasses of its left
-and right inputs. For an equivalence clause, these are of course the same
-EquivalenceClass. For a non-equivalence mergejoinable clause (such as an
-outer-join qualification), we generate two separate EquivalenceClasses for
-the left and right inputs. This may result in creating single-item
-equivalence "classes", though of course these are still subject to merging
-if other equivalence clauses are later found to bear on the same
-expressions.
-
-Another way that we may form a single-item EquivalenceClass is in creation
-of a PathKey to represent a desired sort order (see below). This is a bit
+and right inputs. (These are in fact always the same EquivalenceClass.)
+
+In some cases we will form single-item EquivalenceClasses. This happens
+if an ORDER BY or GROUP BY key is not mentioned in any equivalence
+clause. We need to reason about sort orders in such queries, and our
+representation of sort ordering is a PathKey (see below) which uses an
+EquivalenceClass, so we have to make an EquivalenceClass. This is a bit
different from the above cases because such an EquivalenceClass might
contain an aggregate function or volatile expression. (A clause containing
a volatile function will never be considered mergejoinable, even if its top
@@ -579,7 +757,7 @@ Index scans have Path.pathkeys that represent the chosen index's ordering,
if any. A single-key index would create a single-PathKey list, while a
multi-column index generates a list with one element per key index column.
Non-key columns specified in the INCLUDE clause of covering indexes don't
-have corresponding PathKeys in the list, because the have no influence on
+have corresponding PathKeys in the list, because they have no influence on
index ordering. (Actually, since an index can be scanned either forward or
backward, there are two possible sort orders and two possible PathKey lists
it can generate.)
@@ -655,14 +833,9 @@ redundancy, we save time and improve planning, since the planner will more
easily recognize equivalent orderings as being equivalent.
Another interesting property is that if the underlying EquivalenceClass
-contains a constant and is not below an outer join, then the pathkey is
-completely redundant and need not be sorted by at all! Every row must
-contain the same constant value, so there's no need to sort. (If the EC is
-below an outer join, we still have to sort, since some of the rows might
-have gone to null and others not. In this case we must be careful to pick
-a non-const member to sort by. The assumption that all the non-const
-members go to null at the same plan level is critical here, else they might
-not produce the same sort order.) This might seem pointless because users
+contains a constant, then the pathkey is completely redundant and need
+not be sorted by at all! Every row must contain the same value, so
+there's no need to sort. This might seem pointless because users
are unlikely to write "... WHERE x = 42 ORDER BY x", but it allows us to
recognize when particular index columns are irrelevant to the sort order:
if we have "... WHERE x = 42 ORDER BY y", scanning an index on (x,y)
@@ -670,15 +843,6 @@ produces correctly ordered data without a sort step. We used to have very
ugly ad-hoc code to recognize that in limited contexts, but discarding
constant ECs from pathkeys makes it happen cleanly and automatically.
-You might object that a below-outer-join EquivalenceClass doesn't always
-represent the same values at every level of the join tree, and so using
-it to uniquely identify a sort order is dubious. This is true, but we
-can avoid dealing with the fact explicitly because we always consider that
-an outer join destroys any ordering of its nullable inputs. Thus, even
-if a path was sorted by {a.x} below an outer join, we'll re-sort if that
-sort ordering was important; and so using the same PathKey for both sort
-orderings doesn't create any real problem.
-
Order of processing for EquivalenceClasses and PathKeys
-------------------------------------------------------
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 51d630fa89..a34e7643d7 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -789,6 +789,7 @@ _copyForeignScan(const ForeignScan *from)
COPY_NODE_FIELD(fdw_scan_tlist);
COPY_NODE_FIELD(fdw_recheck_quals);
COPY_BITMAPSET_FIELD(fs_relids);
+ COPY_BITMAPSET_FIELD(fs_base_relids);
COPY_SCALAR_FIELD(fsSystemCol);
return newnode;
@@ -1458,6 +1459,7 @@ _copyVar(const Var *from)
COPY_SCALAR_FIELD(vartype);
COPY_SCALAR_FIELD(vartypmod);
COPY_SCALAR_FIELD(varcollid);
+ COPY_BITMAPSET_FIELD(varnullingrels);
COPY_SCALAR_FIELD(varlevelsup);
COPY_SCALAR_FIELD(varnosyn);
COPY_SCALAR_FIELD(varattnosyn);
@@ -2825,6 +2827,7 @@ _copyRestrictInfo(const RestrictInfo *from)
COPY_SCALAR_FIELD(leakproof);
COPY_SCALAR_FIELD(has_volatile);
COPY_SCALAR_FIELD(security_level);
+ COPY_SCALAR_FIELD(num_base_rels);
COPY_BITMAPSET_FIELD(clause_relids);
COPY_BITMAPSET_FIELD(required_relids);
COPY_BITMAPSET_FIELD(outer_relids);
@@ -2867,6 +2870,7 @@ _copyPlaceHolderVar(const PlaceHolderVar *from)
COPY_NODE_FIELD(phexpr);
COPY_BITMAPSET_FIELD(phrels);
+ COPY_BITMAPSET_FIELD(phnullingrels);
COPY_SCALAR_FIELD(phid);
COPY_SCALAR_FIELD(phlevelsup);
@@ -2886,6 +2890,8 @@ _copySpecialJoinInfo(const SpecialJoinInfo *from)
COPY_BITMAPSET_FIELD(syn_lefthand);
COPY_BITMAPSET_FIELD(syn_righthand);
COPY_SCALAR_FIELD(jointype);
+ COPY_SCALAR_FIELD(ojrelid);
+ COPY_BITMAPSET_FIELD(strict_relids);
COPY_SCALAR_FIELD(lhs_strict);
COPY_SCALAR_FIELD(delay_upper_joins);
COPY_SCALAR_FIELD(semi_can_btree);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e747e1667d..d8d1d6cbae 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -231,6 +231,7 @@ _equalVar(const Var *a, const Var *b)
COMPARE_SCALAR_FIELD(vartype);
COMPARE_SCALAR_FIELD(vartypmod);
COMPARE_SCALAR_FIELD(varcollid);
+ COMPARE_BITMAPSET_FIELD(varnullingrels);
COMPARE_SCALAR_FIELD(varlevelsup);
/*
@@ -1231,12 +1232,15 @@ _equalPlaceHolderVar(const PlaceHolderVar *a, const PlaceHolderVar *b)
* could get replaced by differently-numbered Params when sublink folding
* is done. (The end result of such a situation would be some
* unreferenced initplans, which is annoying but not really a problem.) On
- * the same reasoning, there is no need to examine phrels.
+ * the same reasoning, there is no need to examine phrels. But we do need
+ * to compare phnullingrels, as that is in some sense external to the
+ * value of the PHV proper.
*
* COMPARE_NODE_FIELD(phexpr);
*
* COMPARE_BITMAPSET_FIELD(phrels);
*/
+ COMPARE_BITMAPSET_FIELD(phnullingrels);
COMPARE_SCALAR_FIELD(phid);
COMPARE_SCALAR_FIELD(phlevelsup);
@@ -1251,6 +1255,8 @@ _equalSpecialJoinInfo(const SpecialJoinInfo *a, const SpecialJoinInfo *b)
COMPARE_BITMAPSET_FIELD(syn_lefthand);
COMPARE_BITMAPSET_FIELD(syn_righthand);
COMPARE_SCALAR_FIELD(jointype);
+ COMPARE_SCALAR_FIELD(ojrelid);
+ COMPARE_BITMAPSET_FIELD(strict_relids);
COMPARE_SCALAR_FIELD(lhs_strict);
COMPARE_SCALAR_FIELD(delay_upper_joins);
COMPARE_SCALAR_FIELD(semi_can_btree);
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index 28288dcfc1..19606c495f 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -81,11 +81,13 @@ makeVar(int varno,
var->varlevelsup = varlevelsup;
/*
- * Only a few callers need to make Var nodes with varnosyn/varattnosyn
- * different from varno/varattno. We don't provide separate arguments for
- * them, but just initialize them to the given varno/varattno. This
- * reduces code clutter and chance of error for most callers.
+ * Only a few callers need to make Var nodes with non-null varnullingrels,
+ * or with varnosyn/varattnosyn different from varno/varattno. We don't
+ * provide separate arguments for them, but just initialize them to NULL
+ * and the given varno/varattno. This reduces code clutter and chance of
+ * error for most callers.
*/
+ var->varnullingrels = NULL;
var->varnosyn = (Index) varno;
var->varattnosyn = varattno;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 4cb1744da6..ccf63515fa 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -2847,6 +2847,7 @@ expression_tree_mutator(Node *node,
Var *newnode;
FLATCOPY(newnode, var, Var);
+ /* Assume we need not copy the varnullingrels bitmapset */
return (Node *) newnode;
}
break;
@@ -3442,7 +3443,7 @@ expression_tree_mutator(Node *node,
FLATCOPY(newnode, phv, PlaceHolderVar);
MUTATE(newnode->phexpr, phv->phexpr, Expr *);
- /* Assume we need not copy the relids bitmapset */
+ /* Assume we need not copy the relids bitmapsets */
return (Node *) newnode;
}
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index ce12915592..408d8ace34 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -725,6 +725,7 @@ _outForeignScan(StringInfo str, const ForeignScan *node)
WRITE_NODE_FIELD(fdw_scan_tlist);
WRITE_NODE_FIELD(fdw_recheck_quals);
WRITE_BITMAPSET_FIELD(fs_relids);
+ WRITE_BITMAPSET_FIELD(fs_base_relids);
WRITE_BOOL_FIELD(fsSystemCol);
}
@@ -1146,6 +1147,7 @@ _outVar(StringInfo str, const Var *node)
WRITE_OID_FIELD(vartype);
WRITE_INT_FIELD(vartypmod);
WRITE_OID_FIELD(varcollid);
+ WRITE_BITMAPSET_FIELD(varnullingrels);
WRITE_UINT_FIELD(varlevelsup);
WRITE_UINT_FIELD(varnosyn);
WRITE_INT_FIELD(varattnosyn);
@@ -2459,6 +2461,8 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(plan_params);
WRITE_BITMAPSET_FIELD(outer_params);
WRITE_BITMAPSET_FIELD(all_baserels);
+ WRITE_BITMAPSET_FIELD(outer_join_rels);
+ WRITE_BITMAPSET_FIELD(all_query_rels);
WRITE_BITMAPSET_FIELD(nullable_baserels);
WRITE_NODE_FIELD(join_rel_list);
WRITE_INT_FIELD(join_cur_level);
@@ -2470,7 +2474,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(canon_pathkeys);
WRITE_NODE_FIELD(left_join_clauses);
WRITE_NODE_FIELD(right_join_clauses);
- WRITE_NODE_FIELD(full_join_clauses);
+ /* can't dump full_join_clauses because its contents are not Nodes */
WRITE_NODE_FIELD(join_info_list);
WRITE_BITMAPSET_FIELD(all_result_relids);
WRITE_BITMAPSET_FIELD(leaf_result_relids);
@@ -2552,7 +2556,6 @@ _outRelOptInfo(StringInfo str, const RelOptInfo *node)
WRITE_NODE_FIELD(joininfo);
WRITE_BOOL_FIELD(has_eclass_joins);
WRITE_BOOL_FIELD(consider_partitionwise_join);
- WRITE_BITMAPSET_FIELD(top_parent_relids);
WRITE_BOOL_FIELD(partbounds_merged);
WRITE_BITMAPSET_FIELD(live_parts);
WRITE_BITMAPSET_FIELD(all_partrels);
@@ -2709,6 +2712,7 @@ _outRestrictInfo(StringInfo str, const RestrictInfo *node)
WRITE_BOOL_FIELD(leakproof);
WRITE_ENUM_FIELD(has_volatile, VolatileFunctionStatus);
WRITE_UINT_FIELD(security_level);
+ WRITE_INT_FIELD(num_base_rels);
WRITE_BITMAPSET_FIELD(clause_relids);
WRITE_BITMAPSET_FIELD(required_relids);
WRITE_BITMAPSET_FIELD(outer_relids);
@@ -2749,6 +2753,7 @@ _outPlaceHolderVar(StringInfo str, const PlaceHolderVar *node)
WRITE_NODE_FIELD(phexpr);
WRITE_BITMAPSET_FIELD(phrels);
+ WRITE_BITMAPSET_FIELD(phnullingrels);
WRITE_UINT_FIELD(phid);
WRITE_UINT_FIELD(phlevelsup);
}
@@ -2763,6 +2768,8 @@ _outSpecialJoinInfo(StringInfo str, const SpecialJoinInfo *node)
WRITE_BITMAPSET_FIELD(syn_lefthand);
WRITE_BITMAPSET_FIELD(syn_righthand);
WRITE_ENUM_FIELD(jointype, JoinType);
+ WRITE_UINT_FIELD(ojrelid);
+ WRITE_BITMAPSET_FIELD(strict_relids);
WRITE_BOOL_FIELD(lhs_strict);
WRITE_BOOL_FIELD(delay_upper_joins);
WRITE_BOOL_FIELD(semi_can_btree);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 6a05b69415..08b8ca78f0 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -622,6 +622,7 @@ _readVar(void)
READ_OID_FIELD(vartype);
READ_INT_FIELD(vartypmod);
READ_OID_FIELD(varcollid);
+ READ_BITMAPSET_FIELD(varnullingrels);
READ_UINT_FIELD(varlevelsup);
READ_UINT_FIELD(varnosyn);
READ_INT_FIELD(varattnosyn);
@@ -2312,6 +2313,7 @@ _readForeignScan(void)
READ_NODE_FIELD(fdw_scan_tlist);
READ_NODE_FIELD(fdw_recheck_quals);
READ_BITMAPSET_FIELD(fs_relids);
+ READ_BITMAPSET_FIELD(fs_base_relids);
READ_BOOL_FIELD(fsSystemCol);
READ_DONE();
diff --git a/src/backend/rewrite/rewriteManip.c b/src/backend/rewrite/rewriteManip.c
index 101c39553a..a0a0026469 100644
--- a/src/backend/rewrite/rewriteManip.c
+++ b/src/backend/rewrite/rewriteManip.c
@@ -40,6 +40,13 @@ typedef struct
int win_location;
} locate_windowfunc_context;
+typedef struct
+{
+ Bitmapset *removable_relids;
+ Bitmapset *except_relids;
+ int sublevels_up;
+} remove_nulling_relids_context;
+
static bool contain_aggs_of_level_walker(Node *node,
contain_aggs_of_level_context *context);
static bool locate_agg_of_level_walker(Node *node,
@@ -50,6 +57,9 @@ static bool locate_windowfunc_walker(Node *node,
static bool checkExprHasSubLink_walker(Node *node, void *context);
static Relids offset_relid_set(Relids relids, int offset);
static Relids adjust_relid_set(Relids relids, int oldrelid, int newrelid);
+static bool get_nulling_relids_walker(Node *node, Bitmapset **context);
+static Node *remove_nulling_relids_mutator(Node *node,
+ remove_nulling_relids_context *context);
/*
@@ -348,6 +358,8 @@ OffsetVarNodes_walker(Node *node, OffsetVarNodes_context *context)
if (var->varlevelsup == context->sublevels_up)
{
var->varno += context->offset;
+ var->varnullingrels = offset_relid_set(var->varnullingrels,
+ context->offset);
if (var->varnosyn > 0)
var->varnosyn += context->offset;
}
@@ -386,6 +398,8 @@ OffsetVarNodes_walker(Node *node, OffsetVarNodes_context *context)
{
phv->phrels = offset_relid_set(phv->phrels,
context->offset);
+ phv->phnullingrels = offset_relid_set(phv->phnullingrels,
+ context->offset);
}
/* fall through to examine children */
}
@@ -510,11 +524,13 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context)
{
Var *var = (Var *) node;
- if (var->varlevelsup == context->sublevels_up &&
- var->varno == context->rt_index)
+ if (var->varlevelsup == context->sublevels_up)
{
- var->varno = context->new_index;
- /* If the syntactic referent is same RTE, fix it too */
+ if (var->varno == context->rt_index)
+ var->varno = context->new_index;
+ var->varnullingrels = adjust_relid_set(var->varnullingrels,
+ context->rt_index,
+ context->new_index);
if (var->varnosyn == context->rt_index)
var->varnosyn = context->new_index;
}
@@ -557,6 +573,9 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context)
phv->phrels = adjust_relid_set(phv->phrels,
context->rt_index,
context->new_index);
+ phv->phnullingrels = adjust_relid_set(phv->phnullingrels,
+ context->rt_index,
+ context->new_index);
}
/* fall through to examine children */
}
@@ -833,7 +852,8 @@ rangeTableEntry_used_walker(Node *node,
Var *var = (Var *) node;
if (var->varlevelsup == context->sublevels_up &&
- var->varno == context->rt_index)
+ (var->varno == context->rt_index ||
+ bms_is_member(context->rt_index, var->varnullingrels)))
return true;
return false;
}
@@ -1061,6 +1081,154 @@ AddInvertedQual(Query *parsetree, Node *qual)
}
+/*
+ * get_nulling_relids collects all the level-zero RT indexes mentioned in
+ * Var.varnullingrels and PlaceHolderVar.phnullingrels fields within the
+ * given expression.
+ */
+Bitmapset *
+get_nulling_relids(Node *node)
+{
+ Bitmapset *result = NULL;
+
+ (void) get_nulling_relids_walker(node, &result);
+ return result;
+}
+
+static bool
+get_nulling_relids_walker(Node *node, Bitmapset **context)
+{
+ if (node == NULL)
+ return false;
+ if (IsA(node, Var))
+ {
+ Var *var = (Var *) node;
+
+ if (var->varlevelsup == 0)
+ *context = bms_add_members(*context, var->varnullingrels);
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ if (phv->phlevelsup == 0)
+ *context = bms_add_members(*context, phv->phnullingrels);
+ }
+
+ /*
+ * Currently, this is only used after the planner has converted SubLinks
+ * to SubPlans, so we don't need to support recursion into sub-Queries; so
+ * no sublevels_up counting is needed.
+ */
+ Assert(!IsA(node, SubLink));
+ Assert(!IsA(node, Query));
+ return expression_tree_walker(node, get_nulling_relids_walker, context);
+}
+
+/*
+ * remove_nulling_relids removes mentions of the specified RT index(es)
+ * in Var.varnullingrels and PlaceHolderVar.phnullingrels fields within
+ * the given expression, except in nodes belonging to rels listed in
+ * except_relids.
+ *
+ * XXX consider making this a destructive walker.
+ */
+Node *
+remove_nulling_relids(Node *node, Bitmapset *removable_relids,
+ Bitmapset *except_relids)
+{
+ remove_nulling_relids_context context;
+
+ context.removable_relids = removable_relids;
+ context.except_relids = except_relids;
+ context.sublevels_up = 0;
+ return query_or_expression_tree_mutator(node,
+ remove_nulling_relids_mutator,
+ &context,
+ 0);
+}
+
+static Node *
+remove_nulling_relids_mutator(Node *node,
+ remove_nulling_relids_context *context)
+{
+ if (node == NULL)
+ return NULL;
+ if (IsA(node, Var))
+ {
+ Var *var = (Var *) node;
+
+ if (var->varlevelsup == context->sublevels_up &&
+ !bms_is_member(var->varno, context->except_relids) &&
+ bms_overlap(var->varnullingrels, context->removable_relids))
+ {
+ Relids newnullingrels = bms_difference(var->varnullingrels,
+ context->removable_relids);
+
+ /* Micro-optimization: ensure nullingrels is NULL if empty */
+ if (bms_is_empty(newnullingrels))
+ newnullingrels = NULL;
+ /* Copy the Var ... */
+ var = copyObject(var);
+ /* ... and replace the copy's varnullingrels field */
+ var->varnullingrels = newnullingrels;
+ return (Node *) var;
+ }
+ /* Otherwise fall through to copy the Var normally */
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ if (phv->phlevelsup == context->sublevels_up &&
+ !bms_overlap(phv->phrels, context->except_relids))
+ {
+ Relids newnullingrels = bms_difference(phv->phnullingrels,
+ context->removable_relids);
+
+ /*
+ * Micro-optimization: ensure nullingrels is NULL if empty.
+ *
+ * Note: it might seem desirable to remove the PHV altogether if
+ * phnullingrels goes to empty. Currently we dare not do that
+ * because we use PHVs in some cases to enforce separate identity
+ * of subexpressions; see wrap_non_vars usages in prepjointree.c.
+ */
+ if (bms_is_empty(newnullingrels))
+ newnullingrels = NULL;
+ /* Copy the PlaceHolderVar and mutate what's below ... */
+ phv = (PlaceHolderVar *)
+ expression_tree_mutator(node,
+ remove_nulling_relids_mutator,
+ (void *) context);
+ /* ... and replace the copy's phnullingrels field */
+ phv->phnullingrels = newnullingrels;
+ /* We must also update phrels, if it contains a removable RTI */
+ phv->phrels = bms_difference(phv->phrels,
+ context->removable_relids);
+ Assert(!bms_is_empty(phv->phrels));
+ return (Node *) phv;
+ }
+ /* Otherwise fall through to copy the PlaceHolderVar normally */
+ }
+ else if (IsA(node, Query))
+ {
+ /* Recurse into RTE or sublink subquery */
+ Query *newnode;
+
+ context->sublevels_up++;
+ newnode = query_tree_mutator((Query *) node,
+ remove_nulling_relids_mutator,
+ (void *) context,
+ 0);
+ context->sublevels_up--;
+ return (Node *) newnode;
+ }
+ return expression_tree_mutator(node, remove_nulling_relids_mutator,
+ (void *) context);
+}
+
+
/*
* replace_rte_variables() finds all Vars in an expression tree
* that reference a particular RTE, and replaces them with substitute
diff --git a/src/backend/utils/misc/queryjumble.c b/src/backend/utils/misc/queryjumble.c
index eeaa0b31fe..e517e0363c 100644
--- a/src/backend/utils/misc/queryjumble.c
+++ b/src/backend/utils/misc/queryjumble.c
@@ -381,6 +381,11 @@ JumbleExpr(JumbleState *jstate, Node *node)
APP_JUMB(var->varno);
APP_JUMB(var->varattno);
APP_JUMB(var->varlevelsup);
+
+ /*
+ * We can omit varnullingrels, because it's fully determined
+ * by varno/varlevelsup plus the Var's query location.
+ */
}
break;
case T_Const:
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 73f635b455..78e6d93bf5 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1067,6 +1067,14 @@ typedef struct RangeTblEntry
* alias Vars are generated only for merged columns). We keep these
* entries only because they're needed in expandRTE() and similar code.
*
+ * Vars appearing within joinaliasvars are marked with varnullingrels sets
+ * that describe the nulling effects of this join and lower ones. This is
+ * essential for FULL JOIN cases, because the COALESCE expression only
+ * describes the semantics correctly if its inputs have been nulled by the
+ * join. For other cases, it allows expandRTE() to generate a valid
+ * representation of the join's output without consulting additional
+ * parser state.
+ *
* Within a Query loaded from a stored rule, it is possible for non-merged
* joinaliasvars items to be null pointers, which are placeholders for
* (necessarily unreferenced) columns dropped since the rule was made.
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index a6e5db4eec..b697a00839 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -202,13 +202,26 @@ struct PlannerInfo
struct AppendRelInfo **append_rel_array;
/*
- * all_baserels is a Relids set of all base relids (but not "other"
- * relids) in the query; that is, the Relids identifier of the final join
- * we need to form. This is computed in make_one_rel, just before we
- * start making Paths.
+ * all_baserels is a Relids set of all base relids (but not joins or
+ * "other" relids) in the query. This is computed in make_one_rel, just
+ * before we start making Paths.
*/
Relids all_baserels;
+ /*
+ * outer_join_rels is a Relids set of all outer-join relids in the query.
+ * This is computed in deconstruct_jointree.
+ */
+ Relids outer_join_rels;
+
+ /*
+ * all_query_rels is a Relids set of all base relids and outer join relids
+ * (but not "other" relids) in the query. This is the Relids identifier
+ * of the final join we need to form. This is computed in make_one_rel,
+ * just before we start making Paths.
+ */
+ Relids all_query_rels;
+
/*
* nullable_baserels is a Relids set of base relids that are nullable by
* some outer join in the jointree; these are rels that are potentially
@@ -261,8 +274,8 @@ struct PlannerInfo
* outer join clauses w/nonnullable var on
* right */
- List *full_join_clauses; /* list of RestrictInfos for mergejoinable
- * full join clauses */
+ List *full_join_clauses; /* list of FullJoinClauseInfos for
+ * mergejoinable full join clauses */
List *join_info_list; /* list of SpecialJoinInfos */
@@ -430,9 +443,10 @@ typedef struct PartitionSchemeData *PartitionScheme;
* or the output of a sub-SELECT or function that appears in the range table.
* In either case it is uniquely identified by an RT index. A "joinrel"
* is the joining of two or more base rels. A joinrel is identified by
- * the set of RT indexes for its component baserels. We create RelOptInfo
- * nodes for each baserel and joinrel, and store them in the PlannerInfo's
- * simple_rel_array and join_rel_list respectively.
+ * the set of RT indexes for its component baserels, along with RT indexes
+ * for any outer joins it has computed. We create RelOptInfo nodes for each
+ * baserel and joinrel, and store them in the PlannerInfo's simple_rel_array
+ * and join_rel_list respectively.
*
* Note that there is only one joinrel for any given set of component
* baserels, no matter what order we assemble them in; so an unordered
@@ -471,8 +485,10 @@ typedef struct PartitionSchemeData *PartitionScheme;
* Parts of this data structure are specific to various scan and join
* mechanisms. It didn't seem worth creating new node types for them.
*
- * relids - Set of base-relation identifiers; it is a base relation
- * if there is just one, a join relation if more than one
+ * relids - Set of relation identifiers (RT indexes). This is a base
+ * relation if there is just one, a join relation if more;
+ * in the join case, RT indexes of any outer joins formed
+ * at or below this join are included along with baserels
* rows - estimated number of tuples in the relation after restriction
* clauses have been applied (ie, output rows of a plan for it)
* consider_startup - true if there is any value in keeping plain paths for
@@ -679,7 +695,7 @@ typedef struct RelOptInfo
RelOptKind reloptkind;
/* all relations included in this RelOptInfo */
- Relids relids; /* set of base relids (rangetable indexes) */
+ Relids relids; /* base + OJ relids (rangetable indexes) */
/* size estimates generated by planner */
Cardinality rows; /* estimated number of result tuples */
@@ -754,8 +770,10 @@ typedef struct RelOptInfo
/* used by partitionwise joins: */
bool consider_partitionwise_join; /* consider partitionwise join
* paths? (if partitioned rel) */
- Relids top_parent_relids; /* Relids of topmost parents (if "other"
- * rel) */
+
+ /* inheritance links, if this is an otherrel (otherwise NULL): */
+ struct RelOptInfo *parent; /* immediate parent */
+ struct RelOptInfo *top_parent; /* topmost parent */
/* used for partitioned relations: */
PartitionScheme part_scheme; /* Partitioning scheme */
@@ -1940,17 +1958,17 @@ typedef struct LimitPath
* If a restriction clause references a single base relation, it will appear
* in the baserestrictinfo list of the RelOptInfo for that base rel.
*
- * If a restriction clause references more than one base rel, it will
+ * If a restriction clause references more than one base+OJ relation, it will
* appear in the joininfo list of every RelOptInfo that describes a strict
- * subset of the base rels mentioned in the clause. The joininfo lists are
+ * subset of the relations mentioned in the clause. The joininfo lists are
* used to drive join tree building by selecting plausible join candidates.
* The clause cannot actually be applied until we have built a join rel
- * containing all the base rels it references, however.
+ * containing all the relations it references, however.
*
- * When we construct a join rel that includes all the base rels referenced
+ * When we construct a join rel that includes all the relations referenced
* in a multi-relation restriction clause, we place that clause into the
* joinrestrictinfo lists of paths for the join rel, if neither left nor
- * right sub-path includes all base rels referenced in the clause. The clause
+ * right sub-path includes all relations referenced in the clause. The clause
* will be applied at that join level, and will not propagate any further up
* the join tree. (Note: the "predicate migration" code was once intended to
* push restriction clauses up and down the plan tree based on evaluation
@@ -1971,12 +1989,15 @@ typedef struct LimitPath
* or join to enforce that all members of each EquivalenceClass are in fact
* equal in all rows emitted by the scan or join.
*
- * When dealing with outer joins we have to be very careful about pushing qual
- * clauses up and down the tree. An outer join's own JOIN/ON conditions must
- * be evaluated exactly at that join node, unless they are "degenerate"
- * conditions that reference only Vars from the nullable side of the join.
- * Quals appearing in WHERE or in a JOIN above the outer join cannot be pushed
- * down below the outer join, if they reference any nullable Vars.
+ * The clause_relids field lists the base plus outer-join RT indexes that
+ * actually appear in the clause. required_relids lists the minimum set of
+ * relids needed to evaluate the clause; while this is often equal to
+ * clause_relids, it can be more. We will add relids to required_relids when
+ * we need to force an outer join ON clause to be evaluated exactly at the
+ * level of the outer join, which is true except when it is a "degenerate"
+ * condition that references only Vars from the nullable side of the join.
+ *
+ * XXX rewrite or remove me:
* RestrictInfo nodes contain a flag to indicate whether a qual has been
* pushed down to a lower level than its original syntactic placement in the
* join tree would suggest. If an outer join prevents us from pushing a qual
@@ -2084,12 +2105,14 @@ typedef struct RestrictInfo
bool leakproof; /* true if known to contain no leaked Vars */
- VolatileFunctionStatus has_volatile; /* to indicate if clause contains
- * any volatile functions. */
+ VolatileFunctionStatus has_volatile; /* indicates if clause contains
+ * any volatile functions */
Index security_level; /* see comment above */
- /* The set of relids (varnos) actually referenced in the clause: */
+ int num_base_rels; /* number of base rels in clause_relids */
+
+ /* The relids (varnos+varnullingrels) actually referenced in the clause: */
Relids clause_relids;
/* The set of relids required to evaluate the clause: */
@@ -2147,6 +2170,7 @@ typedef struct RestrictInfo
} RestrictInfo;
/*
+ * XXX this will need work:
* This macro embodies the correct way to test whether a RestrictInfo is
* "pushed down" to a given outer join, that is, should be treated as a filter
* clause rather than a join clause at that outer join. This is certainly so
@@ -2186,10 +2210,15 @@ typedef struct MergeScanSelCache
* of a plan tree. This is used during planning to represent the contained
* expression. At the end of the planning process it is replaced by either
* the contained expression or a Var referring to a lower-level evaluation of
- * the contained expression. Typically the evaluation occurs below an outer
+ * the contained expression. Generally the evaluation occurs below an outer
* join, and Var references above the outer join might thereby yield NULL
* instead of the expression value.
*
+ * phrels and phlevelsup correspond to the varno/varlevelsup fields of a
+ * plain Var, except that phrels has to be a relid set since the evaluation
+ * level of a PlaceHolderVar might be a join rather than a base relation.
+ * Likewise, phnullingrels corresponds to varnullingrels.
+ *
* Although the planner treats this as an expression node type, it is not
* recognized by the parser or executor, so we declare it here rather than
* in primnodes.h.
@@ -2199,7 +2228,8 @@ typedef struct PlaceHolderVar
{
Expr xpr;
Expr *phexpr; /* the represented expression */
- Relids phrels; /* base relids syntactically within expr src */
+ Relids phrels; /* relids syntactically within expr src */
+ Relids phnullingrels; /* RT indexes of joins that can null PHV */
Index phid; /* ID for PHV (unique within planner run) */
Index phlevelsup; /* > 0 if PHV belongs to outer query */
} PlaceHolderVar;
@@ -2220,17 +2250,20 @@ typedef struct PlaceHolderVar
* We make SpecialJoinInfos for FULL JOINs even though there is no flexibility
* of planning for them, because this simplifies make_join_rel()'s API.
*
- * min_lefthand and min_righthand are the sets of base relids that must be
- * available on each side when performing the special join. lhs_strict is
- * true if the special join's condition cannot succeed when the LHS variables
- * are all NULL (this means that an outer join can commute with upper-level
+ * min_lefthand and min_righthand are the sets of base+OJ relids that must be
+ * available on each side when performing the special join.
+ *
+ * strict_relids is the set of base+OJ relids for which the special join's
+ * condition is strict, ie it cannot succeed if any of those rels produce
+ * an all-NULL row. lhs_strict reports whether any LHS rels appear in
+ * strict_relids (this means that an outer join can commute with upper-level
* outer joins even if it appears in their RHS). We don't bother to set
- * lhs_strict for FULL JOINs, however.
+ * strict_relids or lhs_strict for FULL JOINs, however.
*
* It is not valid for either min_lefthand or min_righthand to be empty sets;
* if they were, this would break the logic that enforces join order.
*
- * syn_lefthand and syn_righthand are the sets of base relids that are
+ * syn_lefthand and syn_righthand are the sets of base+OJ relids that are
* syntactically below this special join. (These are needed to help compute
* min_lefthand and min_righthand for higher joins.)
*
@@ -2252,14 +2285,18 @@ typedef struct PlaceHolderVar
* the inputs to make it a LEFT JOIN. So the allowed values of jointype
* in a join_info_list member are only LEFT, FULL, SEMI, or ANTI.
*
+ * ojrelid is the RT index of the join RTE representing this outer join,
+ * if there is one. It is zero when jointype is INNER or SEMI.
+ *
* For purposes of join selectivity estimation, we create transient
* SpecialJoinInfo structures for regular inner joins; so it is possible
* to have jointype == JOIN_INNER in such a structure, even though this is
* not allowed within join_info_list. We also create transient
* SpecialJoinInfos with jointype == JOIN_INNER for outer joins, since for
* cost estimation purposes it is sometimes useful to know the join size under
- * plain innerjoin semantics. Note that lhs_strict, delay_upper_joins, and
- * of course the semi_xxx fields are not set meaningfully within such structs.
+ * plain innerjoin semantics. Note that strict_relids, lhs_strict,
+ * delay_upper_joins, and of course the semi_xxx fields are not set
+ * meaningfully within such structs.
*/
#ifndef HAVE_SPECIALJOININFO_TYPEDEF
typedef struct SpecialJoinInfo SpecialJoinInfo;
@@ -2269,11 +2306,13 @@ typedef struct SpecialJoinInfo SpecialJoinInfo;
struct SpecialJoinInfo
{
NodeTag type;
- Relids min_lefthand; /* base relids in minimum LHS for join */
- Relids min_righthand; /* base relids in minimum RHS for join */
- Relids syn_lefthand; /* base relids syntactically within LHS */
- Relids syn_righthand; /* base relids syntactically within RHS */
+ Relids min_lefthand; /* base+OJ relids in minimum LHS for join */
+ Relids min_righthand; /* base+OJ relids in minimum RHS for join */
+ Relids syn_lefthand; /* base+OJ relids syntactically within LHS */
+ Relids syn_righthand; /* base+OJ relids syntactically within RHS */
JoinType jointype; /* always INNER, LEFT, FULL, SEMI, or ANTI */
+ Index ojrelid; /* outer join's RT index; 0 if none */
+ Relids strict_relids; /* joinclause is strict for these relids */
bool lhs_strict; /* joinclause is strict for some LHS rel */
bool delay_upper_joins; /* can't commute with upper RHS */
/* Remaining fields are set only for JOIN_SEMI jointype: */
@@ -2283,6 +2322,18 @@ struct SpecialJoinInfo
List *semi_rhs_exprs; /* righthand-side expressions of these ops */
};
+/*
+ * FULL JOIN clause info.
+ *
+ * We set aside every FULL JOIN ON clause that looks mergejoinable, and
+ * process it specially at the end of qual distribution.
+ */
+typedef struct FullJoinClauseInfo
+{
+ RestrictInfo *rinfo; /* a mergejoinable FULL JOIN clause */
+ SpecialJoinInfo *sjinfo; /* the FULL JOIN's SpecialJoinInfo */
+} FullJoinClauseInfo;
+
/*
* Append-relation info.
*
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 0ea9a22dfb..5ca0314c8f 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -652,6 +652,7 @@ typedef struct WorkTableScan
* When the plan node represents a foreign join, scan.scanrelid is zero and
* fs_relids must be consulted to identify the join relation. (fs_relids
* is valid for simple scans as well, but will always match scan.scanrelid.)
+ * fs_relids includes outer joins; fs_base_relids does not.
*
* If the FDW's PlanDirectModify() callback decides to repurpose a ForeignScan
* node to perform the UPDATE or DELETE operation directly in the remote
@@ -671,7 +672,8 @@ typedef struct ForeignScan
List *fdw_private; /* private data for FDW */
List *fdw_scan_tlist; /* optional tlist describing scan tuple */
List *fdw_recheck_quals; /* original quals not in scan.plan.qual */
- Bitmapset *fs_relids; /* RTIs generated by this scan */
+ Bitmapset *fs_relids; /* base+OJ RTIs generated by this scan */
+ Bitmapset *fs_base_relids; /* base RTIs generated by this scan */
bool fsSystemCol; /* true if any "system column" is needed */
} ForeignScan;
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index 51505eee85..eba47ecbff 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -171,6 +171,14 @@ typedef struct Expr
* row identity information during UPDATE/DELETE. This value should never
* be seen outside the planner.
*
+ * varnullingrels is the set of RT indexes of outer joins that can force
+ * the Var's value to null (at the point where it appears in the query).
+ * See optimizer/README for discussion of that.
+ *
+ * varlevelsup is greater than zero in Vars that represent outer references.
+ * Note that it affects all of varno, varnullingrels, and varnosyn, all of
+ * which refer to the range table of that query level.
+ *
* In the parser, varnosyn and varattnosyn are either identical to
* varno/varattno, or they specify the column's position in an aliased JOIN
* RTE that hides the semantic referent RTE's refname. This is a syntactic
@@ -202,6 +210,7 @@ typedef struct Var
Oid vartype; /* pg_type OID for the type of this var */
int32 vartypmod; /* pg_attribute typmod value */
Oid varcollid; /* OID of collation, or InvalidOid if none */
+ Bitmapset *varnullingrels; /* RT indexes of joins that can null var */
Index varlevelsup; /* for subquery variables referencing outer
* relations; 0 in a normal var, >0 means N
* levels up */
diff --git a/src/include/parser/parse_node.h b/src/include/parser/parse_node.h
index cf9c759025..8bef98487d 100644
--- a/src/include/parser/parse_node.h
+++ b/src/include/parser/parse_node.h
@@ -115,6 +115,13 @@ typedef Node *(*CoerceParamHook) (ParseState *pstate, Param *param,
* This is one-for-one with p_rtable, but contains NULLs for non-join
* RTEs, and may be shorter than p_rtable if the last RTE(s) aren't joins.
*
+ * p_nullingrels: list of Bitmapsets associated with p_rtable entries, each
+ * containing the set of outer-join RTE indexes that can null that relation
+ * at the current point in the parse tree. This is one-for-one with p_rtable,
+ * but may be shorter than p_rtable, in which case the missing entries are
+ * implicitly empty (NULL). That rule allows us to save work when the query
+ * contains no outer joins.
+ *
* p_joinlist: list of join items (RangeTblRef and JoinExpr nodes) that
* will become the fromlist of the query's top-level FromExpr node.
*
@@ -182,6 +189,7 @@ struct ParseState
const char *p_sourcetext; /* source text, or NULL if not available */
List *p_rtable; /* range table so far */
List *p_joinexprs; /* JoinExprs for RTE_JOIN p_rtable entries */
+ List *p_nullingrels; /* Bitmapsets showing nulling outer joins */
List *p_joinlist; /* join items so far (will become FromExpr
* node's fromlist) */
List *p_namespace; /* currently-referenceable RTEs (List of
diff --git a/src/include/rewrite/rewriteManip.h b/src/include/rewrite/rewriteManip.h
index 98b9b3a288..a3f902c1bb 100644
--- a/src/include/rewrite/rewriteManip.h
+++ b/src/include/rewrite/rewriteManip.h
@@ -63,6 +63,10 @@ extern bool contain_windowfuncs(Node *node);
extern int locate_windowfunc(Node *node);
extern bool checkExprHasSubLink(Node *node);
+extern Bitmapset *get_nulling_relids(Node *node);
+extern Node *remove_nulling_relids(Node *node, Bitmapset *removable_relids,
+ Bitmapset *except_relids);
+
extern Node *replace_rte_variables(Node *node,
int target_varno, int sublevels_up,
replace_rte_variables_callback callback,
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 1bcb875507..d8eca5bc68 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -670,6 +670,7 @@ transformInsertStmt(ParseState *pstate, InsertStmt *stmt)
*/
sub_pstate->p_rtable = sub_rtable;
sub_pstate->p_joinexprs = NIL; /* sub_rtable has no joins */
+ sub_pstate->p_nullingrels = NIL;
sub_pstate->p_namespace = sub_namespace;
sub_pstate->p_resolve_unknowns = false;
@@ -851,7 +852,7 @@ transformInsertStmt(ParseState *pstate, InsertStmt *stmt)
/*
* Generate list of Vars referencing the RTE
*/
- exprList = expandNSItemVars(nsitem, 0, -1, NULL);
+ exprList = expandNSItemVars(pstate, nsitem, 0, -1, NULL);
/*
* Re-apply any indirection on the target column specs to the Vars
diff --git a/src/backend/parser/parse_agg.c b/src/backend/parser/parse_agg.c
index 3ef9e8ee5e..c15fab0f68 100644
--- a/src/backend/parser/parse_agg.c
+++ b/src/backend/parser/parse_agg.c
@@ -1162,7 +1162,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
* entries are RTE_JOIN kind.
*/
if (hasJoinRTEs)
- groupClauses = (List *) flatten_join_alias_vars(qry,
+ groupClauses = (List *) flatten_join_alias_vars(NULL, qry,
(Node *) groupClauses);
/*
@@ -1206,7 +1206,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
groupClauses, hasJoinRTEs,
have_non_var_grouping);
if (hasJoinRTEs)
- clause = flatten_join_alias_vars(qry, clause);
+ clause = flatten_join_alias_vars(NULL, qry, clause);
check_ungrouped_columns(clause, pstate, qry,
groupClauses, groupClauseCommonVars,
have_non_var_grouping,
@@ -1217,7 +1217,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
groupClauses, hasJoinRTEs,
have_non_var_grouping);
if (hasJoinRTEs)
- clause = flatten_join_alias_vars(qry, clause);
+ clause = flatten_join_alias_vars(NULL, qry, clause);
check_ungrouped_columns(clause, pstate, qry,
groupClauses, groupClauseCommonVars,
have_non_var_grouping,
@@ -1546,7 +1546,7 @@ finalize_grouping_exprs_walker(Node *node,
Index ref = 0;
if (context->hasJoinRTEs)
- expr = flatten_join_alias_vars(context->qry, expr);
+ expr = flatten_join_alias_vars(NULL, context->qry, expr);
/*
* Each expression must match a grouping entry at the current
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index c655d188c7..2abd164380 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -52,7 +52,8 @@
#include "utils/syscache.h"
-static int extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
+static int extractRemainingColumns(ParseState *pstate,
+ ParseNamespaceColumn *src_nscolumns,
List *src_colnames,
List **src_colnos,
List **res_colnames, List **res_colvars,
@@ -75,9 +76,11 @@ static ParseNamespaceItem *getNSItemForSpecialRelationTypes(ParseState *pstate,
static Node *transformFromClauseItem(ParseState *pstate, Node *n,
ParseNamespaceItem **top_nsitem,
List **namespace);
-static Var *buildVarFromNSColumn(ParseNamespaceColumn *nscol);
+static Var *buildVarFromNSColumn(ParseState *pstate,
+ ParseNamespaceColumn *nscol);
static Node *buildMergedJoinVar(ParseState *pstate, JoinType jointype,
Var *l_colvar, Var *r_colvar);
+static void markRelsAsNulledBy(ParseState *pstate, Node *n, int jindex);
static void setNamespaceColumnVisibility(List *namespace, bool cols_visible);
static void setNamespaceLateralState(List *namespace,
bool lateral_only, bool lateral_ok);
@@ -249,7 +252,8 @@ setTargetTable(ParseState *pstate, RangeVar *relation,
* Returns the number of columns added.
*/
static int
-extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
+extractRemainingColumns(ParseState *pstate,
+ ParseNamespaceColumn *src_nscolumns,
List *src_colnames,
List **src_colnos,
List **res_colnames, List **res_colvars,
@@ -285,7 +289,8 @@ extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
*src_colnos = lappend_int(*src_colnos, attnum);
*res_colnames = lappend(*res_colnames, lfirst(lc));
*res_colvars = lappend(*res_colvars,
- buildVarFromNSColumn(src_nscolumns + attnum - 1));
+ buildVarFromNSColumn(pstate,
+ src_nscolumns + attnum - 1));
/* Copy the input relation's nscolumn data for this column */
res_nscolumns[colcount] = src_nscolumns[attnum - 1];
colcount++;
@@ -1295,8 +1300,7 @@ transformFromClauseItem(ParseState *pstate, Node *n,
{
/*
* JOIN/USING (or NATURAL JOIN, as transformed above). Transform
- * the list into an explicit ON-condition, and generate a list of
- * merged result columns.
+ * the list into an explicit ON-condition.
*/
List *ucols = j->usingClause;
List *l_usingvars = NIL;
@@ -1314,8 +1318,6 @@ transformFromClauseItem(ParseState *pstate, Node *n,
int r_index = -1;
Var *l_colvar,
*r_colvar;
- Node *u_colvar;
- ParseNamespaceColumn *res_nscolumn;
Assert(u_colname[0] != '\0');
@@ -1379,17 +1381,109 @@ transformFromClauseItem(ParseState *pstate, Node *n,
u_colname)));
r_colnos = lappend_int(r_colnos, r_index + 1);
- l_colvar = buildVarFromNSColumn(l_nscolumns + l_index);
+ /* Build Vars to use in the generated JOIN ON clause */
+ l_colvar = buildVarFromNSColumn(pstate, l_nscolumns + l_index);
l_usingvars = lappend(l_usingvars, l_colvar);
- r_colvar = buildVarFromNSColumn(r_nscolumns + r_index);
+ r_colvar = buildVarFromNSColumn(pstate, r_nscolumns + r_index);
r_usingvars = lappend(r_usingvars, r_colvar);
+ /*
+ * While we're here, add column names to the res_colnames
+ * list. It's a bit ugly to do this here while the
+ * corresponding res_colvars entries are not made till later,
+ * but doing this later would require an additional traversal
+ * of the usingClause list.
+ */
res_colnames = lappend(res_colnames, lfirst(ucol));
+ }
+
+ /* Construct the generated JOIN ON clause */
+ j->quals = transformJoinUsingClause(pstate,
+ l_usingvars,
+ r_usingvars);
+ }
+ else if (j->quals)
+ {
+ /* User-written ON-condition; transform it */
+ j->quals = transformJoinOnClause(pstate, j, my_namespace);
+ }
+ else
+ {
+ /* CROSS JOIN: no quals */
+ }
+
+ /*
+ * If this is an outer join, now mark the appropriate child RTEs as
+ * being nulled by this join. We have finished processing the child
+ * join expressions as well as the current join's quals, which deal in
+ * non-nulled input columns. All future references to those RTEs will
+ * see possibly-nulled values, and we should mark generated Vars to
+ * account for that. In particular, the join alias Vars that we're
+ * about to build should reflect the nulling effects of this join.
+ *
+ * A difficulty with doing this is that we need the join's RT index,
+ * which we don't officially have yet. However, no other RTE can get
+ * made between here and the addRangeTableEntryForJoin call, so we can
+ * predict what the assignment will be. (Alternatively, we could call
+ * addRangeTableEntryForJoin before we have all the data computed, but
+ * this seems less ugly.)
+ */
+ j->rtindex = list_length(pstate->p_rtable) + 1;
+
+ switch (j->jointype)
+ {
+ case JOIN_INNER:
+ break;
+ case JOIN_LEFT:
+ markRelsAsNulledBy(pstate, j->rarg, j->rtindex);
+ break;
+ case JOIN_FULL:
+ markRelsAsNulledBy(pstate, j->larg, j->rtindex);
+ markRelsAsNulledBy(pstate, j->rarg, j->rtindex);
+ break;
+ case JOIN_RIGHT:
+ markRelsAsNulledBy(pstate, j->larg, j->rtindex);
+ break;
+ default:
+ /* shouldn't see any other types here */
+ elog(ERROR, "unrecognized join type: %d",
+ (int) j->jointype);
+ break;
+ }
+
+ /*
+ * Now we can construct join alias expressions for the USING columns.
+ */
+ if (j->usingClause)
+ {
+ ListCell *lc1,
+ *lc2;
+
+ /* Scan the colnos lists to recover info from the previous loop */
+ forboth(lc1, l_colnos, lc2, r_colnos)
+ {
+ int l_index = lfirst_int(lc1) - 1;
+ int r_index = lfirst_int(lc2) - 1;
+ Var *l_colvar,
+ *r_colvar;
+ Node *u_colvar;
+ ParseNamespaceColumn *res_nscolumn;
+
+ /*
+ * Note we re-build these Vars: they might have different
+ * varnullingrels than the ones made in the previous loop.
+ */
+ l_colvar = buildVarFromNSColumn(pstate, l_nscolumns + l_index);
+ r_colvar = buildVarFromNSColumn(pstate, r_nscolumns + r_index);
+
+ /* Construct the join alias Var for this column */
u_colvar = buildMergedJoinVar(pstate,
j->jointype,
l_colvar,
r_colvar);
res_colvars = lappend(res_colvars, u_colvar);
+
+ /* Construct column's res_nscolumns[] entry */
res_nscolumn = res_nscolumns + res_colindex;
res_colindex++;
if (u_colvar == (Node *) l_colvar)
@@ -1407,47 +1501,45 @@ transformFromClauseItem(ParseState *pstate, Node *n,
/*
* Merged column is not semantically equivalent to either
* input, so it needs to be referenced as the join output
- * column. We don't know the join's varno yet, so we'll
- * replace these zeroes below.
+ * column.
*/
- res_nscolumn->p_varno = 0;
+ res_nscolumn->p_varno = j->rtindex;
res_nscolumn->p_varattno = res_colindex;
res_nscolumn->p_vartype = exprType(u_colvar);
res_nscolumn->p_vartypmod = exprTypmod(u_colvar);
res_nscolumn->p_varcollid = exprCollation(u_colvar);
- res_nscolumn->p_varnosyn = 0;
+ res_nscolumn->p_varnosyn = j->rtindex;
res_nscolumn->p_varattnosyn = res_colindex;
}
}
-
- j->quals = transformJoinUsingClause(pstate,
- l_usingvars,
- r_usingvars);
- }
- else if (j->quals)
- {
- /* User-written ON-condition; transform it */
- j->quals = transformJoinOnClause(pstate, j, my_namespace);
- }
- else
- {
- /* CROSS JOIN: no quals */
}
/* Add remaining columns from each side to the output columns */
res_colindex +=
- extractRemainingColumns(l_nscolumns, l_colnames, &l_colnos,
+ extractRemainingColumns(pstate,
+ l_nscolumns, l_colnames, &l_colnos,
&res_colnames, &res_colvars,
res_nscolumns + res_colindex);
res_colindex +=
- extractRemainingColumns(r_nscolumns, r_colnames, &r_colnos,
+ extractRemainingColumns(pstate,
+ r_nscolumns, r_colnames, &r_colnos,
&res_colnames, &res_colvars,
res_nscolumns + res_colindex);
+ /* If join has an alias, it syntactically hides all inputs */
+ if (j->alias)
+ {
+ for (k = 0; k < res_colindex; k++)
+ {
+ ParseNamespaceColumn *nscol = res_nscolumns + k;
+
+ nscol->p_varnosyn = j->rtindex;
+ nscol->p_varattnosyn = k + 1;
+ }
+ }
+
/*
* Now build an RTE and nsitem for the result of the join.
- * res_nscolumns isn't totally done yet, but that's OK because
- * addRangeTableEntryForJoin doesn't examine it, only store a pointer.
*/
nsitem = addRangeTableEntryForJoin(pstate,
res_colnames,
@@ -1461,31 +1553,16 @@ transformFromClauseItem(ParseState *pstate, Node *n,
j->alias,
true);
- j->rtindex = nsitem->p_rtindex;
+ /* Verify that we correctly predicted the join's RT index */
+ Assert(j->rtindex == nsitem->p_rtindex);
+ /* Cross-check number of columns, too */
+ Assert(res_colindex == list_length(nsitem->p_names->colnames));
/*
- * Now that we know the join RTE's rangetable index, we can fix up the
- * res_nscolumns data in places where it should contain that.
+ * Save a link to the JoinExpr in the proper element of p_joinexprs.
+ * Since we maintain that list lazily, it may be necessary to fill in
+ * empty entries before we can add the JoinExpr in the right place.
*/
- Assert(res_colindex == list_length(nsitem->p_names->colnames));
- for (k = 0; k < res_colindex; k++)
- {
- ParseNamespaceColumn *nscol = res_nscolumns + k;
-
- /* fill in join RTI for merged columns */
- if (nscol->p_varno == 0)
- nscol->p_varno = j->rtindex;
- if (nscol->p_varnosyn == 0)
- nscol->p_varnosyn = j->rtindex;
- /* if join has an alias, it syntactically hides all inputs */
- if (j->alias)
- {
- nscol->p_varnosyn = j->rtindex;
- nscol->p_varattnosyn = k + 1;
- }
- }
-
- /* make a matching link to the JoinExpr for later use */
for (k = list_length(pstate->p_joinexprs) + 1; k < j->rtindex; k++)
pstate->p_joinexprs = lappend(pstate->p_joinexprs, NULL);
pstate->p_joinexprs = lappend(pstate->p_joinexprs, j);
@@ -1554,10 +1631,13 @@ transformFromClauseItem(ParseState *pstate, Node *n,
* buildVarFromNSColumn -
* build a Var node using ParseNamespaceColumn data
*
- * We assume varlevelsup should be 0, and no location is specified
+ * This is used to construct joinaliasvars entries.
+ * We can assume varlevelsup should be 0, and no location is specified.
+ * Note also that no column SELECT privilege is requested here; that would
+ * happen only if the column is actually referenced in the query.
*/
static Var *
-buildVarFromNSColumn(ParseNamespaceColumn *nscol)
+buildVarFromNSColumn(ParseState *pstate, ParseNamespaceColumn *nscol)
{
Var *var;
@@ -1571,6 +1651,10 @@ buildVarFromNSColumn(ParseNamespaceColumn *nscol)
/* makeVar doesn't offer parameters for these, so set by hand: */
var->varnosyn = nscol->p_varnosyn;
var->varattnosyn = nscol->p_varattnosyn;
+
+ /* ... and update varnullingrels */
+ markNullableIfNeeded(pstate, var);
+
return var;
}
@@ -1682,6 +1766,47 @@ buildMergedJoinVar(ParseState *pstate, JoinType jointype,
return res_node;
}
+/*
+ * markRelsAsNulledBy -
+ * Mark the given jointree node and its children as nulled by join jindex
+ */
+static void
+markRelsAsNulledBy(ParseState *pstate, Node *n, int jindex)
+{
+ int varno;
+ ListCell *lc;
+
+ /* Note: we can't see FromExpr here */
+ if (IsA(n, RangeTblRef))
+ {
+ varno = ((RangeTblRef *) n)->rtindex;
+ }
+ else if (IsA(n, JoinExpr))
+ {
+ JoinExpr *j = (JoinExpr *) n;
+
+ /* recurse to children */
+ markRelsAsNulledBy(pstate, j->larg, jindex);
+ markRelsAsNulledBy(pstate, j->rarg, jindex);
+ varno = j->rtindex;
+ }
+ else
+ {
+ elog(ERROR, "unrecognized node type: %d", (int) nodeTag(n));
+ varno = 0; /* keep compiler quiet */
+ }
+
+ /*
+ * Now add jindex to the p_nullingrels set for relation varno. Since we
+ * maintain the p_nullingrels list lazily, we might need to extend it to
+ * make the varno'th entry exist.
+ */
+ while (list_length(pstate->p_nullingrels) < varno)
+ pstate->p_nullingrels = lappend(pstate->p_nullingrels, NULL);
+ lc = list_nth_cell(pstate->p_nullingrels, varno - 1);
+ lfirst(lc) = bms_add_member((Bitmapset *) lfirst(lc), jindex);
+}
+
/*
* setNamespaceColumnVisibility -
* Convenience subroutine to update cols_visible flags in a namespace list.
diff --git a/src/backend/parser/parse_coerce.c b/src/backend/parser/parse_coerce.c
index c4e958e4aa..4ded12e873 100644
--- a/src/backend/parser/parse_coerce.c
+++ b/src/backend/parser/parse_coerce.c
@@ -1042,7 +1042,7 @@ coerce_record_to_complex(ParseState *pstate, Node *node,
ParseNamespaceItem *nsitem;
nsitem = GetNSItemByRangeTablePosn(pstate, rtindex, sublevels_up);
- args = expandNSItemVars(nsitem, sublevels_up, vlocation, NULL);
+ args = expandNSItemVars(pstate, nsitem, sublevels_up, vlocation, NULL);
}
else
ereport(ERROR,
diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c
index 0dc2fc472e..9c36109257 100644
--- a/src/backend/parser/parse_expr.c
+++ b/src/backend/parser/parse_expr.c
@@ -2602,6 +2602,9 @@ transformWholeRowRef(ParseState *pstate, ParseNamespaceItem *nsitem,
/* location is not filled in by makeWholeRowVar */
result->location = location;
+ /* mark Var if it's nulled by any outer joins */
+ markNullableIfNeeded(pstate, result);
+
/* mark relation as requiring whole-row SELECT access */
markVarForSelectPriv(pstate, result);
@@ -2629,6 +2632,8 @@ transformWholeRowRef(ParseState *pstate, ParseNamespaceItem *nsitem,
rowexpr->colnames = copyObject(nsitem->p_names->colnames);
rowexpr->location = location;
+ /* XXX we ought to mark the row as possibly nullable */
+
return (Node *) rowexpr;
}
}
diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c
index 926dcbf30e..deec58e4b1 100644
--- a/src/backend/parser/parse_relation.c
+++ b/src/backend/parser/parse_relation.c
@@ -751,6 +751,9 @@ scanNSItemForColumn(ParseState *pstate, ParseNamespaceItem *nsitem,
}
var->location = location;
+ /* Mark Var if it's nulled by any outer joins */
+ markNullableIfNeeded(pstate, var);
+
/* Require read access to the column */
markVarForSelectPriv(pstate, var);
@@ -1007,6 +1010,35 @@ searchRangeTableForCol(ParseState *pstate, const char *alias, const char *colnam
return fuzzystate;
}
+/*
+ * markNullableIfNeeded
+ * If the RTE referenced by the Var is nullable by outer join(s)
+ * at this point in the query, set var->varnullingrels to show that.
+ */
+void
+markNullableIfNeeded(ParseState *pstate, Var *var)
+{
+ int rtindex = var->varno;
+ Bitmapset *relids;
+
+ /* Find the appropriate pstate */
+ for (int lv = 0; lv < var->varlevelsup; lv++)
+ pstate = pstate->parentParseState;
+
+ /* Find currently-relevant join relids for the Var's rel */
+ if (rtindex > 0 && rtindex <= list_length(pstate->p_nullingrels))
+ relids = (Bitmapset *) list_nth(pstate->p_nullingrels, rtindex - 1);
+ else
+ relids = NULL;
+
+ /*
+ * Merge with any already-declared nulling rels. (Typically there won't
+ * be any, but let's get it right if there are.)
+ */
+ if (relids != NULL)
+ var->varnullingrels = bms_union(var->varnullingrels, relids);
+}
+
/*
* markRTEForSelectPriv
* Mark the specified column of the RTE with index rtindex
@@ -3066,7 +3098,7 @@ expandTupleDesc(TupleDesc tupdesc, Alias *eref, int count, int offset,
* the list elements mustn't be modified.
*/
List *
-expandNSItemVars(ParseNamespaceItem *nsitem,
+expandNSItemVars(ParseState *pstate, ParseNamespaceItem *nsitem,
int sublevels_up, int location,
List **colnames)
{
@@ -3102,6 +3134,10 @@ expandNSItemVars(ParseNamespaceItem *nsitem,
var->varnosyn = nscol->p_varnosyn;
var->varattnosyn = nscol->p_varattnosyn;
var->location = location;
+
+ /* ... and update varnullingrels */
+ markNullableIfNeeded(pstate, var);
+
result = lappend(result, var);
if (colnames)
*colnames = lappend(*colnames, colnameval);
@@ -3136,7 +3172,7 @@ expandNSItemAttrs(ParseState *pstate, ParseNamespaceItem *nsitem,
*var;
List *te_list = NIL;
- vars = expandNSItemVars(nsitem, sublevels_up, location, &names);
+ vars = expandNSItemVars(pstate, nsitem, sublevels_up, location, &names);
/*
* Require read access to the table. This is normally redundant with the
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index 2a1d44b813..22834a5bf3 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -1379,7 +1379,7 @@ ExpandSingleTable(ParseState *pstate, ParseNamespaceItem *nsitem,
List *vars;
ListCell *l;
- vars = expandNSItemVars(nsitem, sublevels_up, location, NULL);
+ vars = expandNSItemVars(pstate, nsitem, sublevels_up, location, NULL);
/*
* Require read access to the table. This is normally redundant with
diff --git a/src/include/parser/parse_relation.h b/src/include/parser/parse_relation.h
index de21c3c649..85d96563f3 100644
--- a/src/include/parser/parse_relation.h
+++ b/src/include/parser/parse_relation.h
@@ -41,6 +41,7 @@ extern Node *scanNSItemForColumn(ParseState *pstate, ParseNamespaceItem *nsitem,
int location);
extern Node *colNameToVar(ParseState *pstate, const char *colname, bool localonly,
int location);
+extern void markNullableIfNeeded(ParseState *pstate, Var *var);
extern void markVarForSelectPriv(ParseState *pstate, Var *var);
extern Relation parserOpenTable(ParseState *pstate, const RangeVar *relation,
int lockmode);
@@ -109,7 +110,7 @@ extern void errorMissingColumn(ParseState *pstate,
extern void expandRTE(RangeTblEntry *rte, int rtindex, int sublevels_up,
int location, bool include_dropped,
List **colnames, List **colvars);
-extern List *expandNSItemVars(ParseNamespaceItem *nsitem,
+extern List *expandNSItemVars(ParseState *pstate, ParseNamespaceItem *nsitem,
int sublevels_up, int location,
List **colnames);
extern List *expandNSItemAttrs(ParseState *pstate, ParseNamespaceItem *nsitem,
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
index 8f4d8a5022..cfac1ea57b 100644
--- a/contrib/postgres_fdw/deparse.c
+++ b/contrib/postgres_fdw/deparse.c
@@ -3880,7 +3880,17 @@ get_relation_column_alias_ids(Var *node, RelOptInfo *foreignrel,
i = 1;
foreach(lc, foreignrel->reltarget->exprs)
{
- if (equal(lfirst(lc), (Node *) node))
+ Var *tlvar = (Var *) lfirst(lc);
+
+ /*
+ * As in setrefs.c, we match only on varno/varattno. Ideally there
+ * would be some cross-check on varnullingrels, but it's unclear what
+ * to do exactly; we don't have enough context to know what that value
+ * should be.
+ */
+ if (IsA(tlvar, Var) &&
+ tlvar->varno == node->varno &&
+ tlvar->varattno == node->varattno)
{
*colno = i;
return;
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index d56951153b..53f6e22f39 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -834,10 +834,7 @@ get_useful_ecs_for_relation(PlannerInfo *root, RelOptInfo *rel)
/* If this is a child rel, we must use the topmost parent rel to search. */
if (IS_OTHER_REL(rel))
- {
- Assert(!bms_is_empty(rel->top_parent_relids));
- relids = rel->top_parent_relids;
- }
+ relids = rel->top_parent->relids;
else
relids = rel->relids;
@@ -1513,13 +1510,13 @@ postgresBeginForeignScan(ForeignScanState *node, int eflags)
/*
* Identify which user to do the remote access as. This should match what
* ExecCheckRTEPerms() does. In case of a join or aggregate, use the
- * lowest-numbered member RTE as a representative; we would get the same
- * result from any.
+ * lowest-numbered member base RTE as a representative; we would get the
+ * same result from any.
*/
if (fsplan->scan.scanrelid > 0)
rtindex = fsplan->scan.scanrelid;
else
- rtindex = bms_next_member(fsplan->fs_relids, -1);
+ rtindex = bms_next_member(fsplan->fs_base_relids, -1);
rte = exec_rt_fetch(rtindex, estate);
userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
@@ -2405,7 +2402,7 @@ find_modifytable_subplan(PlannerInfo *root,
{
ForeignScan *fscan = (ForeignScan *) subplan;
- if (bms_is_member(rtindex, fscan->fs_relids))
+ if (bms_is_member(rtindex, fscan->fs_base_relids))
return fscan;
}
@@ -2832,8 +2829,8 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
* that setrefs.c won't update the string when flattening the
* rangetable. To find out what rtoffset was applied, identify the
* minimum RT index appearing in the string and compare it to the
- * minimum member of plan->fs_relids. (We expect all the relids in
- * the join will have been offset by the same amount; the Asserts
+ * minimum member of plan->fs_base_relids. (We expect all the relids
+ * in the join will have been offset by the same amount; the Asserts
* below should catch it if that ever changes.)
*/
minrti = INT_MAX;
@@ -2850,7 +2847,7 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
else
ptr++;
}
- rtoffset = bms_next_member(plan->fs_relids, -1) - minrti;
+ rtoffset = bms_next_member(plan->fs_base_relids, -1) - minrti;
/* Now we can translate the string */
relations = makeStringInfo();
@@ -2865,7 +2862,7 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
char *refname;
rti += rtoffset;
- Assert(bms_is_member(rti, plan->fs_relids));
+ Assert(bms_is_member(rti, plan->fs_base_relids));
rte = rt_fetch(rti, es->rtable);
Assert(rte->rtekind == RTE_RELATION);
/* This logic should agree with explain.c's ExplainTargetRel */
@@ -5614,7 +5611,7 @@ foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
/* PlaceHolderInfo refers to parent relids, not child relids. */
relids = IS_OTHER_REL(joinrel) ?
- joinrel->top_parent_relids : joinrel->relids;
+ joinrel->top_parent->relids : joinrel->relids;
if (bms_is_subset(phinfo->ph_eval_at, relids) &&
bms_nonempty_difference(relids, phinfo->ph_eval_at))
diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml
index d0b5951019..329affa30b 100644
--- a/doc/src/sgml/fdwhandler.sgml
+++ b/doc/src/sgml/fdwhandler.sgml
@@ -351,6 +351,17 @@ GetForeignJoinPaths(PlannerInfo *root,
it will supply at run time in the tuples it returns.
</para>
+ <note>
+ <para>
+ Beginning with <productname>PostgreSQL</productname> 16,
+ <structfield>fs_relids</structfield> includes the rangetable indexes
+ of outer joins, if any were involved in this join. The new field
+ <structfield>fs_base_relids</structfield> includes only base
+ relation indexes, and thus
+ mimics <structfield>fs_relids</structfield>'s old semantics.
+ </para>
+ </note>
+
<para>
See <xref linkend="fdw-planning"/> for additional information.
</para>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 5d1f7089da..7d58443b07 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1092,7 +1092,7 @@ ExplainPreScanNode(PlanState *planstate, Bitmapset **rels_used)
break;
case T_ForeignScan:
*rels_used = bms_add_members(*rels_used,
- ((ForeignScan *) plan)->fs_relids);
+ ((ForeignScan *) plan)->fs_base_relids);
break;
case T_CustomScan:
*rels_used = bms_add_members(*rels_used,
diff --git a/src/backend/executor/execScan.c b/src/backend/executor/execScan.c
index 043bb83f55..2b37266b6a 100644
--- a/src/backend/executor/execScan.c
+++ b/src/backend/executor/execScan.c
@@ -325,7 +325,7 @@ ExecScanReScan(ScanState *node)
* all of them.
*/
if (IsA(node->ps.plan, ForeignScan))
- relids = ((ForeignScan *) node->ps.plan)->fs_relids;
+ relids = ((ForeignScan *) node->ps.plan)->fs_base_relids;
else if (IsA(node->ps.plan, CustomScan))
relids = ((CustomScan *) node->ps.plan)->custom_relids;
else
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index e9342097e5..97c8cd0711 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -179,6 +179,9 @@ make_one_rel(PlannerInfo *root, List *joinlist)
root->all_baserels = bms_add_member(root->all_baserels, brel->relid);
}
+ /* Now we can form the value of all_query_rels, too */
+ root->all_query_rels = bms_union(root->all_baserels, root->outer_join_rels);
+
/* Mark base rels as to whether we care about fast-start plans */
set_base_rel_consider_startup(root);
@@ -230,9 +233,9 @@ make_one_rel(PlannerInfo *root, List *joinlist)
rel = make_rel_from_joinlist(root, joinlist);
/*
- * The result should join all and only the query's base rels.
+ * The result should join all and only the query's base + outer-join rels.
*/
- Assert(bms_equal(rel->relids, root->all_baserels));
+ Assert(bms_equal(rel->relids, root->all_query_rels));
return rel;
}
@@ -558,7 +561,7 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* (see grouping_planner).
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- bms_membership(root->all_baserels) != BMS_SINGLETON)
+ bms_membership(root->all_query_rels) != BMS_SINGLETON)
generate_useful_gather_paths(root, rel, false);
/* Now find the cheapest of the paths for this rel */
@@ -879,7 +882,7 @@ set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *
* to support an uncommon usage of second-rate sampling methods. Instead,
* if there is a risk that the query might perform an unsafe join, just
* wrap the SampleScan in a Materialize node. We can check for joins by
- * counting the membership of all_baserels (note that this correctly
+ * counting the membership of all_query_rels (note that this correctly
* counts inheritance trees as single rels). If we're inside a subquery,
* we can't easily check whether a join might occur in the outer query, so
* just assume one is possible.
@@ -888,7 +891,7 @@ set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *
* so check repeatable_across_scans last, even though that's a bit odd.
*/
if ((root->query_level > 1 ||
- bms_membership(root->all_baserels) != BMS_SINGLETON) &&
+ bms_membership(root->all_query_rels) != BMS_SINGLETON) &&
!(GetTsmRoutine(rte->tablesample->tsmhandler)->repeatable_across_scans))
{
path = (Path *) create_material_path(rel, path);
diff --git a/src/backend/optimizer/path/clausesel.c b/src/backend/optimizer/path/clausesel.c
index 06f836308d..6a5182c0d7 100644
--- a/src/backend/optimizer/path/clausesel.c
+++ b/src/backend/optimizer/path/clausesel.c
@@ -218,7 +218,7 @@ clauselist_selectivity_ext(PlannerInfo *root,
if (rinfo)
{
- ok = (bms_membership(rinfo->clause_relids) == BMS_SINGLETON) &&
+ ok = (rinfo->num_base_rels == 1) &&
(is_pseudo_constant_clause_relids(lsecond(expr->args),
rinfo->right_relids) ||
(varonleft = false,
@@ -579,30 +579,6 @@ find_single_rel_for_clauses(PlannerInfo *root, List *clauses)
return NULL; /* no clauses */
}
-/*
- * bms_is_subset_singleton
- *
- * Same result as bms_is_subset(s, bms_make_singleton(x)),
- * but a little faster and doesn't leak memory.
- *
- * Is this of use anywhere else? If so move to bitmapset.c ...
- */
-static bool
-bms_is_subset_singleton(const Bitmapset *s, int x)
-{
- switch (bms_membership(s))
- {
- case BMS_EMPTY_SET:
- return true;
- case BMS_SINGLETON:
- return bms_is_member(x, s);
- case BMS_MULTIPLE:
- return false;
- }
- /* can't get here... */
- return false;
-}
-
/*
* treat_as_join_clause -
* Decide whether an operator clause is to be handled by the
@@ -631,17 +607,20 @@ treat_as_join_clause(PlannerInfo *root, Node *clause, RestrictInfo *rinfo,
else
{
/*
- * Otherwise, it's a join if there's more than one relation used. We
- * can optimize this calculation if an rinfo was passed.
+ * Otherwise, it's a join if there's more than one base relation used.
+ * We can optimize this calculation if an rinfo was passed.
*
* XXX Since we know the clause is being evaluated at a join, the
* only way it could be single-relation is if it was delayed by outer
- * joins. Although we can make use of the restriction qual estimators
- * anyway, it seems likely that we ought to account for the
- * probability of injected nulls somehow.
+ * joins. We intentionally count only baserels here, not OJs that
+ * might be present in rinfo->clause_relids, so that we direct such
+ * cases to the restriction qual estimators not join estimators.
+ * Eventually some notice should be taken of the possibility of
+ * injected nulls, but we'll likely want to do that in the restriction
+ * estimators rather than starting to treat such cases as join quals.
*/
if (rinfo)
- return (bms_membership(rinfo->clause_relids) == BMS_MULTIPLE);
+ return (rinfo->num_base_rels > 1);
else
return (NumRelids(root, clause) > 1);
}
@@ -753,8 +732,7 @@ clause_selectivity_ext(PlannerInfo *root,
* considering a unique-ified case, so we only need one cache variable
* for all non-JOIN_INNER cases.
*/
- if (varRelid == 0 ||
- bms_is_subset_singleton(rinfo->clause_relids, varRelid))
+ if (varRelid == 0 || rinfo->num_base_rels <= 1)
{
/* Cacheable --- do we already have the result? */
if (jointype == JOIN_INNER)
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index fcc26b01a4..3ca598830e 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -5083,7 +5083,9 @@ compute_semi_anti_join_factors(PlannerInfo *root,
norm_sjinfo.syn_lefthand = outerrel->relids;
norm_sjinfo.syn_righthand = innerrel->relids;
norm_sjinfo.jointype = JOIN_INNER;
+ norm_sjinfo.ojrelid = 0;
/* we don't bother trying to make the remaining fields valid */
+ norm_sjinfo.strict_relids = NULL;
norm_sjinfo.lhs_strict = false;
norm_sjinfo.delay_upper_joins = false;
norm_sjinfo.semi_can_btree = false;
@@ -5248,7 +5250,9 @@ approx_tuple_count(PlannerInfo *root, JoinPath *path, List *quals)
sjinfo.syn_lefthand = path->outerjoinpath->parent->relids;
sjinfo.syn_righthand = path->innerjoinpath->parent->relids;
sjinfo.jointype = JOIN_INNER;
+ sjinfo.ojrelid = 0;
/* we don't bother trying to make the remaining fields valid */
+ sjinfo.strict_relids = NULL;
sjinfo.lhs_strict = false;
sjinfo.delay_upper_joins = false;
sjinfo.semi_can_btree = false;
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 60c0e3f108..257ac1273c 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -29,6 +29,7 @@
#include "optimizer/paths.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "rewrite/rewriteManip.h"
#include "utils/lsyscache.h"
@@ -64,7 +65,7 @@ static bool reconsider_outer_join_clause(PlannerInfo *root,
RestrictInfo *rinfo,
bool outer_on_left);
static bool reconsider_full_join_clause(PlannerInfo *root,
- RestrictInfo *rinfo);
+ FullJoinClauseInfo *fjinfo);
static Bitmapset *get_eclass_indexes_for_relids(PlannerInfo *root,
Relids relids);
static Bitmapset *get_common_eclass_indexes(PlannerInfo *root, Relids relids1,
@@ -768,6 +769,9 @@ get_eclass_for_sort_expr(PlannerInfo *root,
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL)
+ continue; /* must be an outer join */
+
Assert(rel->reloptkind == RELOPT_BASEREL ||
rel->reloptkind == RELOPT_DEADREL);
@@ -936,7 +940,36 @@ is_exprlist_member(Expr *node, List *exprs)
if (expr && IsA(expr, TargetEntry))
expr = ((TargetEntry *) expr)->expr;
- if (equal(node, expr))
+ /*
+ * For Vars and PlaceHolderVars, match using the same rules as
+ * setrefs.c will, in particular ignoring nullingrels. XXX when that
+ * gets tightened up, this should too.
+ */
+ if (IsA(node, Var))
+ {
+ if (expr && IsA(expr, Var))
+ {
+ Var *v1 = (Var *) node;
+ Var *v2 = (Var *) expr;
+
+ if (v1->varno == v2->varno &&
+ v1->varattno == v2->varattno &&
+ v1->varlevelsup == v2->varlevelsup)
+ return true;
+ }
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ if (expr && IsA(expr, PlaceHolderVar))
+ {
+ PlaceHolderVar *v1 = (PlaceHolderVar *) node;
+ PlaceHolderVar *v2 = (PlaceHolderVar *) expr;
+
+ if (v1->phid == v2->phid)
+ return true;
+ }
+ }
+ else if (equal(node, expr))
return true;
}
return false;
@@ -1124,6 +1157,9 @@ generate_base_implied_equalities(PlannerInfo *root)
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL)
+ continue; /* must be an outer join */
+
Assert(rel->reloptkind == RELOPT_BASEREL);
rel->eclass_indexes = bms_add_member(rel->eclass_indexes,
@@ -1415,10 +1451,10 @@ generate_join_implied_equalities(PlannerInfo *root,
/* If inner rel is a child, extra setup work is needed */
if (IS_OTHER_REL(inner_rel))
{
- Assert(!bms_is_empty(inner_rel->top_parent_relids));
+ Assert(inner_rel->top_parent != NULL);
/* Fetch relid set for the topmost parent rel */
- nominal_inner_relids = inner_rel->top_parent_relids;
+ nominal_inner_relids = inner_rel->top_parent->relids;
/* ECs will be marked with the parent's relid, not the child's */
nominal_join_relids = bms_union(outer_relids, nominal_inner_relids);
}
@@ -1493,10 +1529,10 @@ generate_join_implied_equalities_for_ecs(PlannerInfo *root,
/* If inner rel is a child, extra setup work is needed */
if (IS_OTHER_REL(inner_rel))
{
- Assert(!bms_is_empty(inner_rel->top_parent_relids));
+ Assert(inner_rel->top_parent != NULL);
/* Fetch relid set for the topmost parent rel */
- nominal_inner_relids = inner_rel->top_parent_relids;
+ nominal_inner_relids = inner_rel->top_parent->relids;
/* ECs will be marked with the parent's relid, not the child's */
nominal_join_relids = bms_union(outer_relids, nominal_inner_relids);
}
@@ -1760,8 +1796,8 @@ generate_join_implied_equalities_broken(PlannerInfo *root,
if (IS_OTHER_REL(inner_rel) && result != NIL)
result = (List *) adjust_appendrel_attrs_multilevel(root,
(Node *) result,
- inner_rel->relids,
- inner_rel->top_parent_relids);
+ inner_rel,
+ inner_rel->top_parent);
return result;
}
@@ -2014,10 +2050,12 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* Process the FULL JOIN clauses */
foreach(cell, root->full_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ FullJoinClauseInfo *fjinfo = (FullJoinClauseInfo *) lfirst(cell);
- if (reconsider_full_join_clause(root, rinfo))
+ if (reconsider_full_join_clause(root, fjinfo))
{
+ RestrictInfo *rinfo = fjinfo->rinfo;
+
found = true;
/* remove it from the list */
root->full_join_clauses =
@@ -2046,9 +2084,9 @@ reconsider_outer_join_clauses(PlannerInfo *root)
}
foreach(cell, root->full_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ FullJoinClauseInfo *fjinfo = (FullJoinClauseInfo *) lfirst(cell);
- distribute_restrictinfo_to_rels(root, rinfo);
+ distribute_restrictinfo_to_rels(root, fjinfo->rinfo);
}
}
@@ -2184,8 +2222,11 @@ reconsider_outer_join_clause(PlannerInfo *root, RestrictInfo *rinfo,
* Returns true if we were able to propagate a constant through the clause.
*/
static bool
-reconsider_full_join_clause(PlannerInfo *root, RestrictInfo *rinfo)
+reconsider_full_join_clause(PlannerInfo *root, FullJoinClauseInfo *fjinfo)
{
+ RestrictInfo *rinfo = fjinfo->rinfo;
+ SpecialJoinInfo *sjinfo = fjinfo->sjinfo;
+ Relids fjrelids = bms_make_singleton(sjinfo->ojrelid);
Expr *leftvar;
Expr *rightvar;
Oid opno,
@@ -2267,6 +2308,18 @@ reconsider_full_join_clause(PlannerInfo *root, RestrictInfo *rinfo)
cfirst = (Node *) linitial(cexpr->args);
csecond = (Node *) lsecond(cexpr->args);
+ /*
+ * The COALESCE arguments will be marked as possibly nulled by
+ * the full join, while we wish to generate clauses that apply
+ * to the join's inputs. So we must strip the join from the
+ * nullingrels fields of cfirst/csecond before comparing them
+ * to leftvar/rightvar. (Perhaps with a less hokey
+ * representation for FULL JOIN USING output columns, this
+ * wouldn't be needed?)
+ */
+ cfirst = remove_nulling_relids(cfirst, fjrelids, NULL);
+ csecond = remove_nulling_relids(csecond, fjrelids, NULL);
+
if (equal(leftvar, cfirst) && equal(rightvar, csecond))
{
coal_idx = foreach_current_index(lc2);
@@ -2552,7 +2605,7 @@ add_child_rel_equivalences(PlannerInfo *root,
RelOptInfo *parent_rel,
RelOptInfo *child_rel)
{
- Relids top_parent_relids = child_rel->top_parent_relids;
+ Relids top_parent_relids = child_rel->top_parent->relids;
Relids child_relids = child_rel->relids;
int i;
@@ -2626,8 +2679,8 @@ add_child_rel_equivalences(PlannerInfo *root,
child_expr = (Expr *)
adjust_appendrel_attrs_multilevel(root,
(Node *) cur_em->em_expr,
- child_relids,
- top_parent_relids);
+ child_rel,
+ child_rel->top_parent);
}
/*
@@ -2680,7 +2733,7 @@ add_child_join_rel_equivalences(PlannerInfo *root,
RelOptInfo *parent_joinrel,
RelOptInfo *child_joinrel)
{
- Relids top_parent_relids = child_joinrel->top_parent_relids;
+ Relids top_parent_relids = child_joinrel->top_parent->relids;
Relids child_relids = child_joinrel->relids;
Bitmapset *matching_ecs;
MemoryContext oldcontext;
@@ -2768,8 +2821,8 @@ add_child_join_rel_equivalences(PlannerInfo *root,
child_expr = (Expr *)
adjust_appendrel_attrs_multilevel(root,
(Node *) cur_em->em_expr,
- child_relids,
- top_parent_relids);
+ child_joinrel,
+ child_joinrel->top_parent);
}
/*
@@ -2791,8 +2844,8 @@ add_child_join_rel_equivalences(PlannerInfo *root,
new_nullable_relids =
adjust_child_relids_multilevel(root,
new_nullable_relids,
- child_relids,
- top_parent_relids);
+ child_joinrel,
+ child_joinrel->top_parent);
(void) add_eq_member(cur_ec, child_expr,
new_relids, new_nullable_relids,
@@ -3094,10 +3147,7 @@ eclass_useful_for_merging(PlannerInfo *root,
/* If specified rel is a child, we must consider the topmost parent rel */
if (IS_OTHER_REL(rel))
- {
- Assert(!bms_is_empty(rel->top_parent_relids));
- relids = rel->top_parent_relids;
- }
+ relids = rel->top_parent->relids;
else
relids = rel->relids;
@@ -3203,6 +3253,8 @@ get_eclass_indexes_for_relids(PlannerInfo *root, Relids relids)
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL)
+ continue; /* must be an outer join */
ec_indexes = bms_add_members(ec_indexes, rel->eclass_indexes);
}
return ec_indexes;
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index 0ef70ad7f1..ba451f8952 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -3357,13 +3357,13 @@ check_index_predicates(PlannerInfo *root, RelOptInfo *rel)
* Add on any equivalence-derivable join clauses. Computing the correct
* relid sets for generate_join_implied_equalities is slightly tricky
* because the rel could be a child rel rather than a true baserel, and in
- * that case we must remove its parents' relid(s) from all_baserels.
+ * that case we must subtract its parents' relid(s) from all_query_rels.
*/
if (rel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- otherrels = bms_difference(root->all_baserels,
+ otherrels = bms_difference(root->all_query_rels,
find_childrel_parents(root, rel));
else
- otherrels = bms_difference(root->all_baserels, rel->relids);
+ otherrels = bms_difference(root->all_query_rels, rel->relids);
if (!bms_is_empty(otherrels))
clauselist =
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 2a3f0ab7bf..855948f192 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -34,8 +34,8 @@ set_join_pathlist_hook_type set_join_pathlist_hook = NULL;
* any of its child.
*/
#define PATH_PARAM_BY_PARENT(path, rel) \
- ((path)->param_info && bms_overlap(PATH_REQ_OUTER(path), \
- (rel)->top_parent_relids))
+ ((path)->param_info && (rel)->top_parent && \
+ bms_overlap(PATH_REQ_OUTER(path), (rel)->top_parent->relids))
#define PATH_PARAM_BY_REL_SELF(path, rel) \
((path)->param_info && bms_overlap(PATH_REQ_OUTER(path), (rel)->relids))
@@ -141,7 +141,7 @@ add_paths_to_joinrel(PlannerInfo *root,
* partitions.
*/
if (joinrel->reloptkind == RELOPT_OTHER_JOINREL)
- joinrelids = joinrel->top_parent_relids;
+ joinrelids = joinrel->top_parent->relids;
else
joinrelids = joinrel->relids;
@@ -250,7 +250,7 @@ add_paths_to_joinrel(PlannerInfo *root,
if (bms_overlap(joinrelids, sjinfo2->min_righthand) &&
!bms_overlap(joinrelids, sjinfo2->min_lefthand))
extra.param_source_rels = bms_join(extra.param_source_rels,
- bms_difference(root->all_baserels,
+ bms_difference(root->all_query_rels,
sjinfo2->min_righthand));
/* full joins constrain both sides symmetrically */
@@ -258,7 +258,7 @@ add_paths_to_joinrel(PlannerInfo *root,
bms_overlap(joinrelids, sjinfo2->min_lefthand) &&
!bms_overlap(joinrelids, sjinfo2->min_righthand))
extra.param_source_rels = bms_join(extra.param_source_rels,
- bms_difference(root->all_baserels,
+ bms_difference(root->all_query_rels,
sjinfo2->min_lefthand));
}
@@ -643,13 +643,13 @@ try_nestloop_path(PlannerInfo *root,
* Paths are parameterized by top-level parents, so run parameterization
* tests on the parent relids.
*/
- if (innerrel->top_parent_relids)
- innerrelids = innerrel->top_parent_relids;
+ if (innerrel->top_parent)
+ innerrelids = innerrel->top_parent->relids;
else
innerrelids = innerrel->relids;
- if (outerrel->top_parent_relids)
- outerrelids = outerrel->top_parent_relids;
+ if (outerrel->top_parent)
+ outerrelids = outerrel->top_parent->relids;
else
outerrelids = outerrel->relids;
@@ -762,8 +762,8 @@ try_partial_nestloop_path(PlannerInfo *root,
* level parents, not the child relations, so we must use those relids
* for our parameterization tests.
*/
- if (outerrel->top_parent_relids)
- outerrelids = outerrel->top_parent_relids;
+ if (outerrel->top_parent)
+ outerrelids = outerrel->top_parent->relids;
else
outerrelids = outerrel->relids;
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 9da3ff2f9a..b64c37f089 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -353,7 +353,10 @@ make_rels_by_clauseless_joins(PlannerInfo *root,
*
* Caller must supply not only the two rels, but the union of their relids.
* (We could simplify the API by computing joinrelids locally, but this
- * would be redundant work in the normal path through make_join_rel.)
+ * would be redundant work in the normal path through make_join_rel.
+ * Note that this value does NOT include the RT index of any outer join that
+ * might need to be performed here, so it's not the canonical identifier
+ * of the join relation.)
*
* On success, *sjinfo_p is set to NULL if this is to be a plain inner join,
* else it's set to point to the associated SpecialJoinInfo node. Also,
@@ -695,7 +698,7 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(rel1->relids, rel2->relids));
- /* Construct Relids set that identifies the joinrel. */
+ /* Construct Relids set that identifies the joinrel (without OJ as yet). */
joinrelids = bms_union(rel1->relids, rel2->relids);
/* Check validity and determine join type. */
@@ -707,6 +710,10 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
return NULL;
}
+ /* If we have an outer join, add its RTI to form the canonical relids. */
+ if (sjinfo && sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
+
/* Swap rels if needed to match the join info. */
if (reversed)
{
@@ -730,7 +737,9 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
sjinfo->syn_lefthand = rel1->relids;
sjinfo->syn_righthand = rel2->relids;
sjinfo->jointype = JOIN_INNER;
+ sjinfo->ojrelid = 0;
/* we don't bother trying to make the remaining fields valid */
+ sjinfo->strict_relids = NULL;
sjinfo->lhs_strict = false;
sjinfo->delay_upper_joins = false;
sjinfo->semi_can_btree = false;
@@ -1510,8 +1519,6 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(child_rel1->relids, child_rel2->relids));
- child_joinrelids = bms_union(child_rel1->relids, child_rel2->relids);
- appinfos = find_appinfos_by_relids(root, child_joinrelids, &nappinfos);
/*
* Construct SpecialJoinInfo from parent join relations's
@@ -1521,6 +1528,15 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
child_rel1->relids,
child_rel2->relids);
+ /* Build correct join relids for child join */
+ child_joinrelids = bms_union(child_rel1->relids, child_rel2->relids);
+ if (child_sjinfo->ojrelid != 0)
+ child_joinrelids = bms_add_member(child_joinrelids,
+ child_sjinfo->ojrelid);
+
+ /* Find the AppendRelInfo structures */
+ appinfos = find_appinfos_by_relids(root, child_joinrelids, &nappinfos);
+
/*
* Construct restrictions applicable to the child join from those
* applicable to the parent join.
@@ -1536,8 +1552,7 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
{
child_joinrel = build_child_join_rel(root, child_rel1, child_rel2,
joinrel, child_restrictlist,
- child_sjinfo,
- child_sjinfo->jointype);
+ child_sjinfo);
joinrel->part_rels[cnt_parts] = child_joinrel;
joinrel->live_parts = bms_add_member(joinrel->live_parts, cnt_parts);
joinrel->all_partrels = bms_add_members(joinrel->all_partrels,
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index 337f470d58..bbe31c03fe 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -34,7 +34,7 @@
/* local functions */
static bool join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo);
-static void remove_rel_from_query(PlannerInfo *root, int relid,
+static void remove_rel_from_query(PlannerInfo *root, int relid, int ojrelid,
Relids joinrelids);
static List *remove_rel_from_joinlist(List *joinlist, int relid, int *nremoved);
static bool rel_supports_distinctness(PlannerInfo *root, RelOptInfo *rel);
@@ -70,6 +70,7 @@ restart:
foreach(lc, root->join_info_list)
{
SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+ Relids joinrelids;
int innerrelid;
int nremoved;
@@ -84,9 +85,12 @@ restart:
*/
innerrelid = bms_singleton_member(sjinfo->min_righthand);
- remove_rel_from_query(root, innerrelid,
- bms_union(sjinfo->min_lefthand,
- sjinfo->min_righthand));
+ /* Compute the relid set for the join we are considering */
+ joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ if (sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
+
+ remove_rel_from_query(root, innerrelid, sjinfo->ojrelid, joinrelids);
/* We verify that exactly one reference gets removed from joinlist */
nremoved = 0;
@@ -188,6 +192,8 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
/* Compute the relid set for the join we are considering */
joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ if (sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
/*
* We can't remove the join if any inner-rel attributes are used above the
@@ -306,10 +312,12 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
* no longer treated as a baserel, and that attributes of other baserels
* are no longer marked as being needed at joins involving this rel.
* Also, join quals involving the rel have to be removed from the joininfo
- * lists, but only if they belong to the outer join identified by joinrelids.
+ * lists, but only if they belong to the outer join identified by ojrelid
+ * and joinrelids.
*/
static void
-remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
+remove_rel_from_query(PlannerInfo *root, int relid, int ojrelid,
+ Relids joinrelids)
{
RelOptInfo *rel = find_base_rel(root, relid);
List *joininfos;
@@ -349,6 +357,13 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
}
}
+ /*
+ * The removed outer join has to be dropped from root->outer_join_rels.
+ * (We'd need to update all_baserels and all_query_rels too, but those
+ * haven't been computed yet.)
+ */
+ root->outer_join_rels = bms_del_member(root->outer_join_rels, ojrelid);
+
/*
* Likewise remove references from SpecialJoinInfo data structures.
*
@@ -365,6 +380,10 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
sjinfo->min_righthand = bms_del_member(sjinfo->min_righthand, relid);
sjinfo->syn_lefthand = bms_del_member(sjinfo->syn_lefthand, relid);
sjinfo->syn_righthand = bms_del_member(sjinfo->syn_righthand, relid);
+ sjinfo->min_lefthand = bms_del_member(sjinfo->min_lefthand, ojrelid);
+ sjinfo->min_righthand = bms_del_member(sjinfo->min_righthand, ojrelid);
+ sjinfo->syn_lefthand = bms_del_member(sjinfo->syn_lefthand, ojrelid);
+ sjinfo->syn_righthand = bms_del_member(sjinfo->syn_righthand, ojrelid);
}
/*
@@ -393,8 +412,10 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
else
{
phinfo->ph_eval_at = bms_del_member(phinfo->ph_eval_at, relid);
+ phinfo->ph_eval_at = bms_del_member(phinfo->ph_eval_at, ojrelid);
Assert(!bms_is_empty(phinfo->ph_eval_at));
phinfo->ph_needed = bms_del_member(phinfo->ph_needed, relid);
+ phinfo->ph_needed = bms_del_member(phinfo->ph_needed, ojrelid);
}
}
@@ -431,6 +452,8 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
rinfo->required_relids = bms_copy(rinfo->required_relids);
rinfo->required_relids = bms_del_member(rinfo->required_relids,
relid);
+ rinfo->required_relids = bms_del_member(rinfo->required_relids,
+ ojrelid);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
@@ -545,6 +568,7 @@ reduce_unique_semijoins(PlannerInfo *root)
/* Compute the relid set for the join we are considering */
joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ Assert(sjinfo->ojrelid == 0); /* SEMI joins don't have RT indexes */
/*
* Since we're only considering a single-rel RHS, any join clauses it
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 76606faa3e..16247dd9ce 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -29,6 +29,7 @@
#include "optimizer/cost.h"
#include "optimizer/optimizer.h"
#include "optimizer/paramassign.h"
+#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/placeholder.h"
#include "optimizer/plancat.h"
@@ -4106,6 +4107,8 @@ create_foreignscan_plan(PlannerInfo *root, ForeignPath *best_path,
Index scan_relid = rel->relid;
Oid rel_oid = InvalidOid;
Plan *outer_plan = NULL;
+ Relids fs_base_relids;
+ int rtindex;
Assert(rel->fdwroutine != NULL);
@@ -4154,14 +4157,28 @@ create_foreignscan_plan(PlannerInfo *root, ForeignPath *best_path,
/*
* Likewise, copy the relids that are represented by this foreign scan. An
- * upper rel doesn't have relids set, but it covers all the base relations
- * participating in the underlying scan, so use root's all_baserels.
+ * upper rel doesn't have relids set, but it covers all the relations
+ * participating in the underlying scan/join, so use root->all_query_rels.
*/
if (rel->reloptkind == RELOPT_UPPER_REL)
- scan_plan->fs_relids = root->all_baserels;
+ scan_plan->fs_relids = root->all_query_rels;
else
scan_plan->fs_relids = best_path->path.parent->relids;
+ /*
+ * Join relid sets include relevant outer joins, but FDWs may need to know
+ * which are the included base rels. That's a bit tedious to get without
+ * access to the plan-time data structures, so compute it here.
+ */
+ fs_base_relids = NULL;
+ rtindex = -1;
+ while ((rtindex = bms_next_member(scan_plan->fs_relids, rtindex)) >= 0)
+ {
+ if (find_base_rel_ignore_join(root, rtindex) != NULL)
+ fs_base_relids = bms_add_member(fs_base_relids, rtindex);
+ }
+ scan_plan->fs_base_relids = fs_base_relids;
+
/*
* If this is a foreign join, and to make it valid to push down we had to
* assume that the current user is the same as some user explicitly named
@@ -5806,8 +5823,9 @@ make_foreignscan(List *qptlist,
node->fdw_private = fdw_private;
node->fdw_scan_tlist = fdw_scan_tlist;
node->fdw_recheck_quals = fdw_recheck_quals;
- /* fs_relids will be filled in by create_foreignscan_plan */
+ /* fs_relids, fs_base_relids will be filled by create_foreignscan_plan */
node->fs_relids = NULL;
+ node->fs_base_relids = NULL;
/* fsSystemCol will be filled in by create_foreignscan_plan */
node->fsSystemCol = false;
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 023efbaf09..31911ecc42 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -60,12 +60,15 @@ static void process_security_barrier_quals(PlannerInfo *root,
static SpecialJoinInfo *make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
- JoinType jointype, List *clause);
+ JoinType jointype, Index ojrelid,
+ List *clause);
static void compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo,
List *clause);
+static List *remove_unneeded_nulling_relids(PlannerInfo *root, List *quals,
+ SpecialJoinInfo *sjinfo);
static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool below_outer_join,
- JoinType jointype,
+ SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
Relids ojscope,
@@ -250,10 +253,16 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
attno -= rel->min_attr;
if (rel->attr_needed[attno] == NULL)
{
- /* Variable not yet requested, so add to rel's targetlist */
- /* XXX is copyObject necessary here? */
- rel->reltarget->exprs = lappend(rel->reltarget->exprs,
- copyObject(var));
+ /*
+ * Variable not yet requested, so add to rel's targetlist.
+ *
+ * The value available at the rel's scan level has not been
+ * nulled by any outer join, so drop its varnullingrels.
+ * (We'll put those back as we climb up the join tree.)
+ */
+ var = copyObject(var);
+ var->varnullingrels = NULL;
+ rel->reltarget->exprs = lappend(rel->reltarget->exprs, var);
/* reltarget cost and width will be computed later */
}
rel->attr_needed[attno] = bms_add_members(rel->attr_needed[attno],
@@ -551,8 +560,10 @@ create_lateral_join_info(PlannerInfo *root)
varno = -1;
while ((varno = bms_next_member(eval_at, varno)) >= 0)
{
- RelOptInfo *brel = find_base_rel(root, varno);
+ RelOptInfo *brel = find_base_rel_ignore_join(root, varno);
+ if (brel == NULL)
+ continue; /* ignore outer joins in eval_at */
brel->lateral_relids = bms_add_members(brel->lateral_relids,
phinfo->ph_lateral);
}
@@ -643,7 +654,10 @@ create_lateral_join_info(PlannerInfo *root)
{
RelOptInfo *brel2 = root->simple_rel_array[rti2];
- Assert(brel2 != NULL && brel2->reloptkind == RELOPT_BASEREL);
+ if (brel2 == NULL)
+ continue; /* must be an OJ */
+
+ Assert(brel2->reloptkind == RELOPT_BASEREL);
brel2->lateral_referencers =
bms_add_member(brel2->lateral_referencers, rti);
}
@@ -695,7 +709,8 @@ deconstruct_jointree(PlannerInfo *root)
Assert(root->parse->jointree != NULL &&
IsA(root->parse->jointree, FromExpr));
- /* this is filled as we scan the jointree */
+ /* These are filled as we scan the jointree */
+ root->outer_join_rels = NULL;
root->nullable_baserels = NULL;
result = deconstruct_recurse(root, (Node *) root->parse->jointree, false,
@@ -717,7 +732,7 @@ deconstruct_jointree(PlannerInfo *root)
* below_outer_join is true if this node is within the nullable side of a
* higher-level outer join
* Outputs:
- * *qualscope gets the set of base Relids syntactically included in this
+ * *qualscope gets the set of base+OJ Relids syntactically included in this
* jointree node (do not modify or free this, as it may also be pointed
* to by RestrictInfo and SpecialJoinInfo nodes)
* *inner_join_rels gets the set of base Relids syntactically included in
@@ -802,6 +817,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
* there was exactly one element, we should (and already did) report
* whatever its inner_join_rels were. If there were no elements (is
* that still possible?) the initialization before the loop fixed it.
+ *
+ * XXX now wrong, do we care?
*/
if (list_length(f->fromlist) > 1)
*inner_join_rels = *qualscope;
@@ -816,7 +833,7 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
if (bms_is_subset(pq->relids, *qualscope))
distribute_qual_to_rels(root, pq->qual,
- below_outer_join, JOIN_INNER,
+ below_outer_join, NULL,
root->qual_security_level,
*qualscope, NULL, NULL,
NULL);
@@ -832,7 +849,7 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
Node *qual = (Node *) lfirst(l);
distribute_qual_to_rels(root, qual,
- below_outer_join, JOIN_INNER,
+ below_outer_join, NULL,
root->qual_security_level,
*qualscope, NULL, NULL,
postponed_qual_list);
@@ -896,6 +913,13 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ /* caution: ANTI join derived from SEMI will lack rtindex */
+ if (j->rtindex != 0)
+ {
+ *qualscope = bms_add_member(*qualscope, j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
+ j->rtindex);
+ }
*inner_join_rels = bms_union(left_inners, right_inners);
nonnullable_rels = leftids;
nullable_rels = rightids;
@@ -910,6 +934,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ /* SEMI join never has rtindex, so don't add to qualscope */
+ Assert(j->rtindex == 0);
*inner_join_rels = bms_union(left_inners, right_inners);
/* Semi join adds no restrictions for quals */
nonnullable_rels = NULL;
@@ -931,6 +957,10 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ Assert(j->rtindex != 0);
+ *qualscope = bms_add_member(*qualscope, j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
+ j->rtindex);
*inner_join_rels = bms_union(left_inners, right_inners);
/* each side is both outer and inner */
nonnullable_rels = *qualscope;
@@ -976,32 +1006,44 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
my_quals = list_concat(my_quals, (List *) j->quals);
/*
- * For an OJ, form the SpecialJoinInfo now, because we need the OJ's
- * semantic scope (ojscope) to pass to distribute_qual_to_rels. But
- * we mustn't add it to join_info_list just yet, because we don't want
- * distribute_qual_to_rels to think it is an outer join below us.
- *
- * Semijoins are a bit of a hybrid: we build a SpecialJoinInfo, but we
- * want ojscope = NULL for distribute_qual_to_rels.
+ * For an OJ, form the SpecialJoinInfo now, because we need it for
+ * distribute_qual_to_rels. But we mustn't add it to join_info_list
+ * just yet, because we don't want distribute_qual_to_rels to think it
+ * is an outer join below us.
*/
if (j->jointype != JOIN_INNER)
- {
sjinfo = make_outerjoininfo(root,
leftids, rightids,
*inner_join_rels,
j->jointype,
+ j->rtindex,
my_quals);
- if (j->jointype == JOIN_SEMI)
- ojscope = NULL;
- else
- ojscope = bms_union(sjinfo->min_lefthand,
- sjinfo->min_righthand);
- }
else
- {
sjinfo = NULL;
+
+ /*
+ * If we have a LEFT JOIN whose ON qual is strict for any LHS
+ * relations, we may be able to commute the join with lower outer
+ * joins that null those relations. To do that, we must remove such
+ * lower outer joins from Var.varnullingrels fields within the qual,
+ * else subsequent processing will think that the qual has to be
+ * evaluated above such lower outer joins.
+ */
+ if (j->jointype == JOIN_LEFT && sjinfo->lhs_strict)
+ my_quals = remove_unneeded_nulling_relids(root, my_quals, sjinfo);
+
+ /*
+ * Now we can compute ojscope (we can't do it earlier, because
+ * remove_unneeded_nulling_relids might change the scope).
+ *
+ * Semijoins are a bit of a hybrid: we build a SpecialJoinInfo, but we
+ * want ojscope = NULL for distribute_qual_to_rels.
+ */
+ if (j->jointype == JOIN_INNER || j->jointype == JOIN_SEMI)
ojscope = NULL;
- }
+ else
+ ojscope = bms_union(sjinfo->min_lefthand,
+ sjinfo->min_righthand);
/* Process the JOIN's qual clauses */
foreach(l, my_quals)
@@ -1009,7 +1051,7 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
Node *qual = (Node *) lfirst(l);
distribute_qual_to_rels(root, qual,
- below_outer_join, j->jointype,
+ below_outer_join, sjinfo,
root->qual_security_level,
*qualscope,
ojscope, nonnullable_rels,
@@ -1112,7 +1154,7 @@ process_security_barrier_quals(PlannerInfo *root,
*/
distribute_qual_to_rels(root, qual,
below_outer_join,
- JOIN_INNER,
+ NULL,
security_level,
qualscope,
qualscope,
@@ -1135,6 +1177,7 @@ process_security_barrier_quals(PlannerInfo *root,
* right_rels: the base Relids syntactically on inner side of join
* inner_join_rels: base Relids participating in inner joins below this one
* jointype: what it says (must always be LEFT, FULL, SEMI, or ANTI)
+ * ojrelid: RT index of the join RTE (0 for SEMI, which isn't in the RT list)
* clause: the outer join's join condition (in implicit-AND format)
*
* The node should eventually be appended to root->join_info_list, but we
@@ -1148,7 +1191,8 @@ static SpecialJoinInfo *
make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
- JoinType jointype, List *clause)
+ JoinType jointype, Index ojrelid,
+ List *clause)
{
SpecialJoinInfo *sjinfo = makeNode(SpecialJoinInfo);
Relids clause_relids;
@@ -1196,6 +1240,7 @@ make_outerjoininfo(PlannerInfo *root,
sjinfo->syn_lefthand = left_rels;
sjinfo->syn_righthand = right_rels;
sjinfo->jointype = jointype;
+ sjinfo->ojrelid = ojrelid;
/* this always starts out false */
sjinfo->delay_upper_joins = false;
@@ -1206,6 +1251,7 @@ make_outerjoininfo(PlannerInfo *root,
{
sjinfo->min_lefthand = bms_copy(left_rels);
sjinfo->min_righthand = bms_copy(right_rels);
+ sjinfo->strict_relids = NULL; /* don't care about this */
sjinfo->lhs_strict = false; /* don't care about this */
return sjinfo;
}
@@ -1220,6 +1266,7 @@ make_outerjoininfo(PlannerInfo *root,
* rel's columns are all NULL?
*/
strict_relids = find_nonnullable_rels((Node *) clause);
+ sjinfo->strict_relids = strict_relids;
/* Remember whether the clause is strict for any LHS relations */
sjinfo->lhs_strict = bms_overlap(strict_relids, left_rels);
@@ -1258,6 +1305,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_lefthand = bms_add_member(min_lefthand,
+ otherinfo->ojrelid);
}
if (bms_overlap(right_rels, otherinfo->syn_lefthand) ||
bms_overlap(right_rels, otherinfo->syn_righthand))
@@ -1266,6 +1316,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_righthand = bms_add_member(min_righthand,
+ otherinfo->ojrelid);
}
/* Needn't do anything else with the full join */
continue;
@@ -1295,6 +1348,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_lefthand = bms_add_member(min_lefthand,
+ otherinfo->ojrelid);
}
}
@@ -1337,6 +1393,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_righthand = bms_add_member(min_righthand,
+ otherinfo->ojrelid);
}
}
}
@@ -1561,6 +1620,62 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
sjinfo->semi_rhs_exprs = semi_rhs_exprs;
}
+/*
+ * remove_unneeded_nulling_relids
+ * Remove lower outer joins from Vars (& PHVs) in the quals, if possible
+ *
+ * This paves the way to apply outer join identity 3 to commute the current
+ * LEFT JOIN with lower outer joins. We already know that the quals are
+ * strict for at least one LHS relation.
+ */
+static List *
+remove_unneeded_nulling_relids(PlannerInfo *root, List *quals,
+ SpecialJoinInfo *sjinfo)
+{
+ Relids old_nulling_relids;
+ Relids removable_relids;
+ ListCell *lc;
+
+ /*
+ * Find outer joins mentioned in nullingrel fields in the quals. If there
+ * aren't any (the common case), there's no need to work hard.
+ */
+ old_nulling_relids = get_nulling_relids((Node *) quals);
+ if (bms_is_empty(old_nulling_relids))
+ return quals;
+
+ /*
+ * Thumb through the existing SpecialJoinInfos (which describe all outer
+ * joins below this one, but not yet this one) to find the ones mentioned
+ * in the quals. If the current join's quals are strict for any rel of
+ * one's RHS, we can commute this join with that one, so remove it from
+ * the current join's min_lefthand and from the quals' nullingrel fields.
+ */
+ removable_relids = NULL;
+ foreach(lc, root->join_info_list)
+ {
+ SpecialJoinInfo *sjinfo2 = (SpecialJoinInfo *) lfirst(lc);
+
+ if (sjinfo2->jointype != JOIN_LEFT ||
+ !bms_is_member(sjinfo2->ojrelid, old_nulling_relids))
+ continue; /* it's not relevant */
+ if (bms_is_subset(sjinfo2->syn_righthand, sjinfo->syn_lefthand) &&
+ bms_overlap(sjinfo->strict_relids, sjinfo2->min_righthand))
+ {
+ sjinfo->min_lefthand = bms_del_member(sjinfo->min_lefthand,
+ sjinfo2->ojrelid);
+ removable_relids = bms_add_member(removable_relids,
+ sjinfo2->ojrelid);
+ }
+ }
+
+ if (removable_relids == NULL)
+ return quals; /* no hits, nothing to do */
+
+ return (List *) remove_nulling_relids((Node *) quals,
+ removable_relids, NULL);
+}
+
/*****************************************************************************
*
@@ -1582,7 +1697,7 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
* 'clause': the qual clause to be distributed
* 'below_outer_join': true if the qual is from a JOIN/ON that is below the
* nullable side of a higher-level outer join
- * 'jointype': type of join the qual is from (JOIN_INNER for a WHERE clause)
+ * 'sjinfo': join's SpecialJoinInfo (NULL for an inner join or WHERE clause)
* 'security_level': security_level to assign to the qual
* 'qualscope': set of baserels the qual's syntactic scope covers
* 'ojscope': NULL if not an outer-join qual, else the minimum set of baserels
@@ -1600,12 +1715,13 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
* level, which will be ojscope not necessarily qualscope.
*
* At the time this is called, root->join_info_list must contain entries for
- * all and only those special joins that are syntactically below this qual.
+ * all and only those special joins that are syntactically below this qual;
+ * in particular, the passed-in SpecialJoinInfo isn't yet in that list.
*/
static void
distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool below_outer_join,
- JoinType jointype,
+ SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
Relids ojscope,
@@ -1642,7 +1758,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
PostponedQual *pq = (PostponedQual *) palloc(sizeof(PostponedQual));
Assert(root->hasLateralRTEs); /* shouldn't happen otherwise */
- Assert(jointype == JOIN_INNER); /* mustn't postpone past outer join */
+ Assert(sjinfo == NULL); /* mustn't postpone past outer join */
pq->qual = clause;
pq->relids = relids;
*postponed_qual_list = lappend(*postponed_qual_list, pq);
@@ -1704,7 +1820,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
{
relids =
get_relids_in_jointree((Node *) root->parse->jointree,
- false);
+ true, false);
qualscope = bms_copy(relids);
}
}
@@ -1946,11 +2062,15 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
restrictinfo);
return;
}
- if (jointype == JOIN_FULL)
+ if (sjinfo && sjinfo->jointype == JOIN_FULL)
{
/* FULL JOIN (above tests cannot match in this case) */
+ FullJoinClauseInfo *fjinfo = palloc(sizeof(FullJoinClauseInfo));
+
+ fjinfo->rinfo = restrictinfo;
+ fjinfo->sjinfo = sjinfo;
root->full_join_clauses = lappend(root->full_join_clauses,
- restrictinfo);
+ fjinfo);
return;
}
/* nope, so fall through to distribute_restrictinfo_to_rels */
@@ -2344,7 +2464,7 @@ process_implied_equality(PlannerInfo *root,
{
relids =
get_relids_in_jointree((Node *) root->parse->jointree,
- false);
+ true, false);
}
}
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index a0f2390334..593c3c9fcc 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -898,7 +898,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
*/
if (rte->lateral && root->hasJoinRTEs)
rte->subquery = (Query *)
- flatten_join_alias_vars(root->parse,
+ flatten_join_alias_vars(root, root->parse,
(Node *) rte->subquery);
}
else if (rte->rtekind == RTE_FUNCTION)
@@ -1099,7 +1099,7 @@ preprocess_expression(PlannerInfo *root, Node *expr, int kind)
kind == EXPRKIND_VALUES ||
kind == EXPRKIND_TABLESAMPLE ||
kind == EXPRKIND_TABLEFUNC))
- expr = flatten_join_alias_vars(root->parse, expr);
+ expr = flatten_join_alias_vars(root, root->parse, expr);
/*
* Simplify constant expressions. For function RTEs, this was already
@@ -1791,8 +1791,8 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
withCheckOptions = (List *)
adjust_appendrel_attrs_multilevel(root,
(Node *) withCheckOptions,
- this_result_rel->relids,
- top_result_rel->relids);
+ this_result_rel,
+ top_result_rel);
withCheckOptionLists = lappend(withCheckOptionLists,
withCheckOptions);
}
@@ -1804,8 +1804,8 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
returningList = (List *)
adjust_appendrel_attrs_multilevel(root,
(Node *) returningList,
- this_result_rel->relids,
- top_result_rel->relids);
+ this_result_rel,
+ top_result_rel);
returningLists = lappend(returningLists,
returningList);
}
@@ -1826,13 +1826,13 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
leaf_action->qual =
adjust_appendrel_attrs_multilevel(root,
(Node *) action->qual,
- this_result_rel->relids,
- top_result_rel->relids);
+ this_result_rel,
+ top_result_rel);
leaf_action->targetList = (List *)
adjust_appendrel_attrs_multilevel(root,
(Node *) action->targetList,
- this_result_rel->relids,
- top_result_rel->relids);
+ this_result_rel,
+ top_result_rel);
if (leaf_action->commandType == CMD_UPDATE)
leaf_action->updateColnos =
adjust_inherited_attnums_multilevel(root,
@@ -2221,7 +2221,7 @@ preprocess_rowmarks(PlannerInfo *root)
* make a bitmapset of all base rels and then remove the items we don't
* need or have FOR [KEY] UPDATE/SHARE marks for.
*/
- rels = get_relids_in_jointree((Node *) parse->jointree, false);
+ rels = get_relids_in_jointree((Node *) parse->jointree, false, false);
if (parse->resultRelation)
rels = bms_del_member(rels, parse->resultRelation);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 9cef92cab2..3b720ec5d6 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -151,6 +151,9 @@ static Var *search_indexed_tlist_for_var(Var *var,
indexed_tlist *itlist,
int newvarno,
int rtoffset);
+static Var *search_indexed_tlist_for_phv(PlaceHolderVar *phv,
+ indexed_tlist *itlist,
+ int newvarno);
static Var *search_indexed_tlist_for_non_var(Expr *node,
indexed_tlist *itlist,
int newvarno);
@@ -1530,6 +1533,7 @@ set_foreignscan_references(PlannerInfo *root,
}
fscan->fs_relids = offset_relid_set(fscan->fs_relids, rtoffset);
+ fscan->fs_base_relids = offset_relid_set(fscan->fs_base_relids, rtoffset);
/* Adjust resultRelation if it's valid */
if (fscan->resultRelation > 0)
@@ -2108,6 +2112,7 @@ fix_scan_expr_mutator(Node *node, fix_scan_expr_context *context)
/* At scan level, we should always just evaluate the contained expr */
PlaceHolderVar *phv = (PlaceHolderVar *) node;
+ Assert(phv->phnullingrels == NULL);
return fix_scan_expr_mutator((Node *) phv->phexpr, context);
}
if (IsA(node, AlternativeSubPlan))
@@ -2228,33 +2233,12 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
/*
* Now we need to fix up the targetlist and qpqual, which are logically
* above the join. This means they should not re-use any input expression
- * that was computed in the nullable side of an outer join. Vars and
- * PlaceHolderVars are fine, so we can implement this restriction just by
- * clearing has_non_vars in the indexed_tlist structs.
+ * that was computed in the nullable side of an outer join.
*
- * XXX This is a grotty workaround for the fact that we don't clearly
- * distinguish between a Var appearing below an outer join and the "same"
- * Var appearing above it. If we did, we'd not need to hack the matching
- * rules this way.
+ * XXX we will probably need to pass some flag down to indicate that this
+ * context applies, so that search_indexed_tlist_for_var() and siblings
+ * can correctly check for varnullingrels matches.
*/
- switch (join->jointype)
- {
- case JOIN_LEFT:
- case JOIN_SEMI:
- case JOIN_ANTI:
- inner_itlist->has_non_vars = false;
- break;
- case JOIN_RIGHT:
- outer_itlist->has_non_vars = false;
- break;
- case JOIN_FULL:
- outer_itlist->has_non_vars = false;
- inner_itlist->has_non_vars = false;
- break;
- default:
- break;
- }
-
join->plan.targetlist = fix_join_expr(root,
join->plan.targetlist,
outer_itlist,
@@ -2543,7 +2527,7 @@ set_dummy_tlist_references(Plan *plan, int rtoffset)
* tlist_member() searches.
*
* The result of this function is an indexed_tlist struct to pass to
- * search_indexed_tlist_for_var() or search_indexed_tlist_for_non_var().
+ * search_indexed_tlist_for_var() and siblings.
* When done, the indexed_tlist may be freed with a single pfree().
*/
static indexed_tlist *
@@ -2665,6 +2649,8 @@ search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
/* Found a match */
Var *newvar = copyVar(var);
+ /* XXX we oughta check varnullingrels match here ... */
+
newvar->varno = newvarno;
newvar->varattno = vinfo->resno;
if (newvar->varnosyn > 0)
@@ -2677,15 +2663,55 @@ search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
}
/*
- * search_indexed_tlist_for_non_var --- find a non-Var in an indexed tlist
+ * search_indexed_tlist_for_phv --- find a PlaceHolderVar in an indexed tlist
*
* If a match is found, return a Var constructed to reference the tlist item.
* If no match, return NULL.
*
- * NOTE: it is a waste of time to call this unless itlist->has_ph_vars or
- * itlist->has_non_vars. Furthermore, set_join_references() relies on being
- * able to prevent matching of non-Vars by clearing itlist->has_non_vars,
- * so there's a correctness reason not to call it unless that's set.
+ * NOTE: it is a waste of time to call this unless itlist->has_ph_vars.
+ */
+static Var *
+search_indexed_tlist_for_phv(PlaceHolderVar *phv,
+ indexed_tlist *itlist, int newvarno)
+{
+ ListCell *lc;
+
+ foreach(lc, itlist->tlist)
+ {
+ TargetEntry *tle = (TargetEntry *) lfirst(lc);
+
+ if (tle->expr && IsA(tle->expr, PlaceHolderVar))
+ {
+ PlaceHolderVar *subphv = (PlaceHolderVar *) tle->expr;
+ Var *newvar;
+
+ /*
+ * Analogously to search_indexed_tlist_for_var, we match on phid
+ * only. We don't use equal(), partially for speed but mostly
+ * because phnullingrels might not be exactly equal.
+ *
+ * XXX we really oughta verify phnullingrels.
+ */
+ if (phv->phid != subphv->phid)
+ continue;
+
+ /* Found a matching subplan output expression */
+ newvar = makeVarFromTargetEntry(newvarno, tle);
+ newvar->varnosyn = 0; /* wasn't ever a plain Var */
+ newvar->varattnosyn = 0;
+ return newvar;
+ }
+ }
+ return NULL; /* no match */
+}
+
+/*
+ * search_indexed_tlist_for_non_var --- find a non-Var/PHV in an indexed tlist
+ *
+ * If a match is found, return a Var constructed to reference the tlist item.
+ * If no match, return NULL.
+ *
+ * NOTE: it is a waste of time to call this unless itlist->has_non_vars.
*/
static Var *
search_indexed_tlist_for_non_var(Expr *node,
@@ -2870,22 +2896,23 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* See if the PlaceHolderVar has bubbled up from a lower plan node */
if (context->outer_itlist && context->outer_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->outer_itlist,
- OUTER_VAR);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->outer_itlist,
+ OUTER_VAR);
if (newvar)
return (Node *) newvar;
}
if (context->inner_itlist && context->inner_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->inner_itlist,
- INNER_VAR);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->inner_itlist,
+ INNER_VAR);
if (newvar)
return (Node *) newvar;
}
/* If not supplied by input plans, evaluate the contained expr */
+ /* XXX assert something about phnullingrels */
return fix_join_expr_mutator((Node *) phv->phexpr, context);
}
/* Try matching more complex expressions too, if tlists have any */
@@ -2994,13 +3021,14 @@ fix_upper_expr_mutator(Node *node, fix_upper_expr_context *context)
/* See if the PlaceHolderVar has bubbled up from a lower plan node */
if (context->subplan_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->subplan_itlist,
- context->newvarno);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->subplan_itlist,
+ context->newvarno);
if (newvar)
return (Node *) newvar;
}
/* If not supplied by input plan, evaluate the contained expr */
+ /* XXX assert something about phnullingrels */
return fix_upper_expr_mutator((Node *) phv->phexpr, context);
}
/* Try matching more complex expressions too, if tlist has any */
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 0bd99acf83..7b22254173 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -49,17 +49,28 @@ typedef struct pullup_replace_vars_context
* pullup (set only if target_rte->lateral) */
bool *outer_hasSubLinks; /* -> outer query's hasSubLinks */
int varno; /* varno of subquery */
- bool need_phvs; /* do we need PlaceHolderVars? */
- bool wrap_non_vars; /* do we need 'em on *all* non-Vars? */
+ bool wrap_non_vars; /* do we need all non-Var outputs to be PHVs? */
Node **rv_cache; /* cache for results with PHVs */
} pullup_replace_vars_context;
-typedef struct reduce_outer_joins_state
+typedef struct reduce_outer_joins_pass1_state
{
Relids relids; /* base relids within this subtree */
bool contains_outer; /* does subtree contain outer join(s)? */
List *sub_states; /* List of states for subtree components */
-} reduce_outer_joins_state;
+} reduce_outer_joins_pass1_state;
+
+typedef struct reduce_outer_joins_pass2_state
+{
+ Relids inner_reduced; /* OJ relids reduced to plain inner joins */
+ List *partial_reduced; /* List of partially reduced FULL joins */
+} reduce_outer_joins_pass2_state;
+
+typedef struct reduce_outer_joins_partial_state
+{
+ int full_join_rti; /* RT index of a formerly-FULL join */
+ Relids unreduced_side; /* relids in its still-nullable side */
+} reduce_outer_joins_partial_state;
static Node *pull_up_sublinks_jointree_recurse(PlannerInfo *root, Node *jtnode,
Relids *relids);
@@ -68,12 +79,10 @@ static Node *pull_up_sublinks_qual_recurse(PlannerInfo *root, Node *node,
Node **jtlink2, Relids available_rels2);
static Node *pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static Node *pull_up_simple_subquery(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static Node *pull_up_simple_union_all(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte);
@@ -90,7 +99,6 @@ static Node *pull_up_simple_values(PlannerInfo *root, Node *jtnode,
static bool is_simple_values(PlannerInfo *root, RangeTblEntry *rte);
static Node *pull_up_constant_function(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static bool is_simple_union_all(Query *subquery);
static bool is_simple_union_all_recurse(Node *setOp, Query *setOpQuery,
@@ -101,25 +109,27 @@ static bool jointree_contains_lateral_outer_refs(PlannerInfo *root,
Relids safe_upper_varnos);
static void perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars_context *rvcontext,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static void replace_vars_in_jointree(Node *jtnode,
- pullup_replace_vars_context *context,
- JoinExpr *lowest_nulling_outer_join);
+ pullup_replace_vars_context *context);
static Node *pullup_replace_vars(Node *expr,
pullup_replace_vars_context *context);
static Node *pullup_replace_vars_callback(Var *var,
replace_rte_variables_context *context);
static Query *pullup_replace_vars_subquery(Query *query,
pullup_replace_vars_context *context);
-static reduce_outer_joins_state *reduce_outer_joins_pass1(Node *jtnode);
+static reduce_outer_joins_pass1_state *reduce_outer_joins_pass1(Node *jtnode);
static void reduce_outer_joins_pass2(Node *jtnode,
- reduce_outer_joins_state *state,
+ reduce_outer_joins_pass1_state *state1,
+ reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
List *nonnullable_vars,
List *forced_null_vars);
-static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode);
+static void report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
+ int rtindex, Relids relids);
+static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Relids *dropped_outer_joins);
static int get_result_relid(PlannerInfo *root, Node *jtnode);
static void remove_result_refs(PlannerInfo *root, int varno, Node *newjtloc);
static bool find_dependent_phvs(PlannerInfo *root, int varno);
@@ -764,7 +774,7 @@ pull_up_subqueries(PlannerInfo *root)
/* Recursion starts with no containing join nor appendrel */
root->parse->jointree = (FromExpr *)
pull_up_subqueries_recurse(root, (Node *) root->parse->jointree,
- NULL, NULL, NULL);
+ NULL, NULL);
/* We should still have a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
}
@@ -779,12 +789,6 @@ pull_up_subqueries(PlannerInfo *root)
* lowest_outer_join references the lowest such JoinExpr node; otherwise
* it is NULL. We use this to constrain the effects of LATERAL subqueries.
*
- * If this jointree node is within the nullable side of an outer join, then
- * lowest_nulling_outer_join references the lowest such JoinExpr node;
- * otherwise it is NULL. This forces use of the PlaceHolderVar mechanism for
- * references to non-nullable targetlist items, but only for references above
- * that join.
- *
* If we are looking at a member subquery of an append relation,
* containing_appendrel describes that relation; else it is NULL.
* This forces use of the PlaceHolderVar mechanism for all non-Var targetlist
@@ -801,15 +805,14 @@ pull_up_subqueries(PlannerInfo *root)
* Notice also that we can't turn pullup_replace_vars loose on the whole
* jointree, because it'd return a mutated copy of the tree; we have to
* invoke it just on the quals, instead. This behavior is what makes it
- * reasonable to pass lowest_outer_join and lowest_nulling_outer_join as
- * pointers rather than some more-indirect way of identifying the lowest
- * OJs. Likewise, we don't replace append_rel_list members but only their
- * substructure, so the containing_appendrel reference is safe to use.
+ * reasonable to pass lowest_outer_join as a pointer rather than some
+ * more-indirect way of identifying the lowest OJ. Likewise, we don't
+ * replace append_rel_list members but only their substructure, so the
+ * containing_appendrel reference is safe to use.
*/
static Node *
pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Assert(jtnode != NULL);
@@ -831,7 +834,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
is_safe_append_member(rte->subquery)))
return pull_up_simple_subquery(root, jtnode, rte,
lowest_outer_join,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -864,7 +866,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
*/
if (rte->rtekind == RTE_FUNCTION)
return pull_up_constant_function(root, jtnode, rte,
- lowest_nulling_outer_join,
containing_appendrel);
/* Otherwise, do nothing at this node. */
@@ -880,7 +881,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
{
lfirst(l) = pull_up_subqueries_recurse(root, lfirst(l),
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
}
}
@@ -895,11 +895,9 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
case JOIN_INNER:
j->larg = pull_up_subqueries_recurse(root, j->larg,
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
break;
case JOIN_LEFT:
@@ -907,31 +905,25 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
case JOIN_ANTI:
j->larg = pull_up_subqueries_recurse(root, j->larg,
j,
- lowest_nulling_outer_join,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
- j,
j,
NULL);
break;
case JOIN_FULL:
j->larg = pull_up_subqueries_recurse(root, j->larg,
- j,
j,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
- j,
j,
NULL);
break;
case JOIN_RIGHT:
j->larg = pull_up_subqueries_recurse(root, j->larg,
- j,
j,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
j,
- lowest_nulling_outer_join,
NULL);
break;
default:
@@ -961,7 +953,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
static Node *
pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -1085,7 +1076,8 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* maybe even in the rewriter; but for now let's just fix this case here.)
*/
subquery->targetList = (List *)
- flatten_join_alias_vars(subroot->parse, (Node *) subquery->targetList);
+ flatten_join_alias_vars(subroot, subroot->parse,
+ (Node *) subquery->targetList);
/*
* Adjust level-0 varnos in subquery so that we can append its rangetable
@@ -1107,31 +1099,25 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* The subquery's targetlist items are now in the appropriate form to
* insert into the top query, except that we may need to wrap them in
* PlaceHolderVars. Set up required context data for pullup_replace_vars.
+ * (Note that we should include the subquery's inner joins in relids,
+ * since it may include join alias vars referencing them.)
*/
rvcontext.root = root;
rvcontext.targetlist = subquery->targetList;
rvcontext.target_rte = rte;
if (rte->lateral)
rvcontext.relids = get_relids_in_jointree((Node *) subquery->jointree,
- true);
+ true, true);
else /* won't need relids */
rvcontext.relids = NULL;
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = varno;
- /* these flags will be set below, if needed */
- rvcontext.need_phvs = false;
+ /* this flag will be set below, if needed */
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(subquery->targetList) + 1) *
sizeof(Node *));
- /*
- * If we are under an outer join then non-nullable items and lateral
- * references may have to be turned into PlaceHolderVars.
- */
- if (lowest_nulling_outer_join != NULL)
- rvcontext.need_phvs = true;
-
/*
* If we are dealing with an appendrel member then anything that's not a
* simple Var has to be turned into a PlaceHolderVar. We force this to
@@ -1140,10 +1126,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* expression actually available from the appendrel.
*/
if (containing_appendrel != NULL)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* If the parent query uses grouping sets, we need a PlaceHolderVar for
@@ -1155,10 +1138,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* that pullup_replace_vars hasn't currently got.)
*/
if (parse->groupingSets)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* Replace all of the top query's references to the subquery's outputs
@@ -1166,7 +1146,6 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* replace any of the jointree structure.
*/
perform_pullup_replace_vars(root, &rvcontext,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -1233,7 +1212,8 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
{
Relids subrelids;
- subrelids = get_relids_in_jointree((Node *) subquery->jointree, false);
+ subrelids = get_relids_in_jointree((Node *) subquery->jointree,
+ true, false);
substitute_phv_relids((Node *) parse, varno, subrelids);
fix_append_rel_relids(root->append_rel_list, varno, subrelids);
}
@@ -1424,7 +1404,7 @@ pull_up_union_leaf_queries(Node *setOp, PlannerInfo *root, int parentRTindex,
rtr = makeNode(RangeTblRef);
rtr->rtindex = childRTindex;
(void) pull_up_subqueries_recurse(root, (Node *) rtr,
- NULL, NULL, appinfo);
+ NULL, appinfo);
}
else if (IsA(setOp, SetOperationStmt))
{
@@ -1561,7 +1541,7 @@ is_simple_subquery(PlannerInfo *root, Query *subquery, RangeTblEntry *rte,
{
restricted = true;
safe_upper_varnos = get_relids_in_jointree((Node *) lowest_outer_join,
- true);
+ true, true);
}
else
{
@@ -1673,7 +1653,6 @@ pull_up_simple_values(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte)
rvcontext.relids = NULL;
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = varno;
- rvcontext.need_phvs = false;
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(tlist) + 1) *
@@ -1685,7 +1664,7 @@ pull_up_simple_values(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte)
* any of the jointree structure. We can assume there's no outer joins or
* appendrels in the dummy Query that surrounds a VALUES RTE.
*/
- perform_pullup_replace_vars(root, &rvcontext, NULL, NULL);
+ perform_pullup_replace_vars(root, &rvcontext, NULL);
/*
* There should be no appendrels to fix, nor any outer joins and hence no
@@ -1784,7 +1763,6 @@ is_simple_values(PlannerInfo *root, RangeTblEntry *rte)
static Node *
pull_up_constant_function(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -1836,40 +1814,26 @@ pull_up_constant_function(PlannerInfo *root, Node *jtnode,
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = ((RangeTblRef *) jtnode)->rtindex;
- /* these flags will be set below, if needed */
- rvcontext.need_phvs = false;
+ /* this flag will be set below, if needed */
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(rvcontext.targetlist) + 1) *
sizeof(Node *));
- /*
- * If we are under an outer join then non-nullable items and lateral
- * references may have to be turned into PlaceHolderVars.
- */
- if (lowest_nulling_outer_join != NULL)
- rvcontext.need_phvs = true;
-
/*
* If we are dealing with an appendrel member then anything that's not a
* simple Var has to be turned into a PlaceHolderVar. (See comments in
* pull_up_simple_subquery().)
*/
if (containing_appendrel != NULL)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* If the parent query uses grouping sets, we need a PlaceHolderVar for
* anything that's not a simple Var.
*/
if (parse->groupingSets)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* Replace all of the top query's references to the RTE's output with
@@ -1877,7 +1841,6 @@ pull_up_constant_function(PlannerInfo *root, Node *jtnode,
* jointree structure.
*/
perform_pullup_replace_vars(root, &rvcontext,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -2099,13 +2062,11 @@ jointree_contains_lateral_outer_refs(PlannerInfo *root, Node *jtnode,
*
* Caller has already filled *rvcontext with data describing what to
* substitute for Vars referencing the target subquery. In addition
- * we need the identity of the lowest outer join that can null the
- * target subquery, and its containing appendrel if any.
+ * we need the identity of the containing appendrel if any.
*/
static void
perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars_context *rvcontext,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -2149,38 +2110,31 @@ perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars((Node *) action->targetList, rvcontext);
}
}
- replace_vars_in_jointree((Node *) parse->jointree, rvcontext,
- lowest_nulling_outer_join);
+ replace_vars_in_jointree((Node *) parse->jointree, rvcontext);
Assert(parse->setOperations == NULL);
parse->havingQual = pullup_replace_vars(parse->havingQual, rvcontext);
/*
* Replace references in the translated_vars lists of appendrels. When
- * pulling up an appendrel member, we do not need PHVs in the list of the
- * parent appendrel --- there isn't any outer join between. Elsewhere,
- * use PHVs for safety. (This analysis could be made tighter but it seems
- * unlikely to be worth much trouble.)
+ * pulling up an appendrel member, we do not want to force PHVs in the
+ * list of the parent appendrel --- there isn't any outer join between.
+ * Elsewhere, use PHVs for safety. (This analysis could be made tighter
+ * but it seems unlikely to be worth much trouble.)
*/
foreach(lc, root->append_rel_list)
{
AppendRelInfo *appinfo = (AppendRelInfo *) lfirst(lc);
- bool save_need_phvs = rvcontext->need_phvs;
+ bool save_wrap_non_vars = rvcontext->wrap_non_vars;
if (appinfo == containing_appendrel)
- rvcontext->need_phvs = false;
+ rvcontext->wrap_non_vars = false;
appinfo->translated_vars = (List *)
pullup_replace_vars((Node *) appinfo->translated_vars, rvcontext);
- rvcontext->need_phvs = save_need_phvs;
+ rvcontext->wrap_non_vars = save_wrap_non_vars;
}
/*
* Replace references in the joinaliasvars lists of join RTEs.
- *
- * You might think that we could avoid using PHVs for alias vars of joins
- * below lowest_nulling_outer_join, but that doesn't work because the
- * alias vars could be referenced above that join; we need the PHVs to be
- * present in such references after the alias vars get flattened. (It
- * might be worth trying to be smarter here, someday.)
*/
foreach(lc, parse->rtable)
{
@@ -2197,14 +2151,10 @@ perform_pullup_replace_vars(PlannerInfo *root,
* Helper routine for perform_pullup_replace_vars: do pullup_replace_vars on
* every expression in the jointree, without changing the jointree structure
* itself. Ugly, but there's no other way...
- *
- * If we are at or below lowest_nulling_outer_join, we can suppress use of
- * PlaceHolderVars wrapped around the replacement expressions.
*/
static void
replace_vars_in_jointree(Node *jtnode,
- pullup_replace_vars_context *context,
- JoinExpr *lowest_nulling_outer_join)
+ pullup_replace_vars_context *context)
{
if (jtnode == NULL)
return;
@@ -2217,7 +2167,7 @@ replace_vars_in_jointree(Node *jtnode,
* jointree scan, rather than a scan of the rtable, for a couple of
* reasons: we can avoid processing no-longer-referenced RTEs, and we
* can use the appropriate setting of need_phvs depending on whether
- * the RTE is above possibly-nulling outer joins or not.
+ * the RTE is above possibly-nulling outer joins or not. XXX fix
*/
int varno = ((RangeTblRef *) jtnode)->rtindex;
@@ -2274,42 +2224,30 @@ replace_vars_in_jointree(Node *jtnode,
ListCell *l;
foreach(l, f->fromlist)
- replace_vars_in_jointree(lfirst(l), context,
- lowest_nulling_outer_join);
+ replace_vars_in_jointree(lfirst(l), context);
f->quals = pullup_replace_vars(f->quals, context);
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- bool save_need_phvs = context->need_phvs;
+ bool save_wrap_non_vars = context->wrap_non_vars;
- if (j == lowest_nulling_outer_join)
- {
- /* no more PHVs in or below this join */
- context->need_phvs = false;
- lowest_nulling_outer_join = NULL;
- }
- replace_vars_in_jointree(j->larg, context, lowest_nulling_outer_join);
- replace_vars_in_jointree(j->rarg, context, lowest_nulling_outer_join);
+ replace_vars_in_jointree(j->larg, context);
+ replace_vars_in_jointree(j->rarg, context);
/*
- * Use PHVs within the join quals of a full join, even when it's the
- * lowest nulling outer join. Otherwise, we cannot identify which
- * side of the join a pulled-up var-free expression came from, which
- * can lead to failure to make a plan at all because none of the quals
- * appear to be mergeable or hashable conditions. For this purpose we
- * don't care about the state of wrap_non_vars, so leave it alone.
+ * Use PHVs within the join quals of a full join. Otherwise, we
+ * cannot identify which side of the join a pulled-up var-free
+ * expression came from, which can lead to failure to make a plan at
+ * all because none of the quals appear to be mergeable or hashable
+ * conditions.
*/
if (j->jointype == JOIN_FULL)
- context->need_phvs = true;
+ context->wrap_non_vars = true;
j->quals = pullup_replace_vars(j->quals, context);
- /*
- * We don't bother to update the colvars list, since it won't be used
- * again ...
- */
- context->need_phvs = save_need_phvs;
+ context->wrap_non_vars = save_wrap_non_vars;
}
else
elog(ERROR, "unrecognized node type: %d",
@@ -2338,8 +2276,18 @@ pullup_replace_vars_callback(Var *var,
{
pullup_replace_vars_context *rcon = (pullup_replace_vars_context *) context->callback_arg;
int varattno = var->varattno;
+ bool need_phv;
Node *newnode;
+ /*
+ * We need a PlaceHolderVar if the Var-to-be-replaced has nonempty
+ * varnullingrels (unless we find below that the replacement expression is
+ * a Var or PlaceHolderVar that we can just add the nullingrels to). We
+ * also need one if the caller has instructed us that all non-Var/PHV
+ * replacements need to be wrapped for identification purposes.
+ */
+ need_phv = (var->varnullingrels != NULL) || rcon->wrap_non_vars;
+
/*
* If PlaceHolderVars are needed, we cache the modified expressions in
* rcon->rv_cache[]. This is not in hopes of any material speed gain
@@ -2348,13 +2296,16 @@ pullup_replace_vars_callback(Var *var,
* and possibly prevent optimizations that rely on recognizing different
* references to the same subquery output as being equal(). So it's worth
* a bit of extra effort to avoid it.
+ *
+ * The cached items have phlevelsup = 0 and phnullingrels = NULL; we'll
+ * copy them and adjust those values for this reference site below.
*/
- if (rcon->need_phvs &&
+ if (need_phv &&
varattno >= InvalidAttrNumber &&
varattno <= list_length(rcon->targetlist) &&
rcon->rv_cache[varattno] != NULL)
{
- /* Just copy the entry and fall through to adjust its varlevelsup */
+ /* Just copy the entry and fall through to adjust phlevelsup etc */
newnode = copyObject(rcon->rv_cache[varattno]);
}
else if (varattno == InvalidAttrNumber)
@@ -2363,7 +2314,7 @@ pullup_replace_vars_callback(Var *var,
RowExpr *rowexpr;
List *colnames;
List *fields;
- bool save_need_phvs = rcon->need_phvs;
+ bool save_wrap_non_vars = rcon->wrap_non_vars;
int save_sublevelsup = context->sublevels_up;
/*
@@ -2374,18 +2325,18 @@ pullup_replace_vars_callback(Var *var,
* the RowExpr for use of the executor and ruleutils.c.
*
* In order to be able to cache the results, we always generate the
- * expansion with varlevelsup = 0, and then adjust if needed.
+ * expansion with varlevelsup = 0, and then adjust below if needed.
*/
expandRTE(rcon->target_rte,
var->varno, 0 /* not varlevelsup */ , var->location,
(var->vartype != RECORDOID),
&colnames, &fields);
- /* Adjust the generated per-field Vars, but don't insert PHVs */
- rcon->need_phvs = false;
+ /* Expand the generated per-field Vars, but don't insert PHVs there */
+ rcon->wrap_non_vars = false;
context->sublevels_up = 0; /* to match the expandRTE output */
fields = (List *) replace_rte_variables_mutator((Node *) fields,
context);
- rcon->need_phvs = save_need_phvs;
+ rcon->wrap_non_vars = save_wrap_non_vars;
context->sublevels_up = save_sublevelsup;
rowexpr = makeNode(RowExpr);
@@ -2403,14 +2354,13 @@ pullup_replace_vars_callback(Var *var,
* expression to yield NULL, not ROW(NULL,NULL,...) when it is forced
* to null by an outer join.
*/
- if (rcon->need_phvs)
+ if (need_phv)
{
- /* RowExpr is certainly not strict, so always need PHV */
newnode = (Node *)
make_placeholder_expr(rcon->root,
(Expr *) newnode,
bms_make_singleton(rcon->varno));
- /* cache it with the PHV, and with varlevelsup still zero */
+ /* cache it with the PHV, and with phlevelsup etc not set yet */
rcon->rv_cache[InvalidAttrNumber] = copyObject(newnode);
}
}
@@ -2427,7 +2377,7 @@ pullup_replace_vars_callback(Var *var,
newnode = (Node *) copyObject(tle->expr);
/* Insert PlaceHolderVar if needed */
- if (rcon->need_phvs)
+ if (need_phv)
{
bool wrap;
@@ -2453,69 +2403,61 @@ pullup_replace_vars_callback(Var *var,
/* No need to wrap a PlaceHolderVar with another one, either */
wrap = false;
}
- else if (rcon->wrap_non_vars)
- {
- /* Wrap all non-Vars in a PlaceHolderVar */
- wrap = true;
- }
else
{
/*
- * If it contains a Var of the subquery being pulled up, and
- * does not contain any non-strict constructs, then it's
- * certainly nullable so we don't need to insert a
- * PlaceHolderVar.
- *
- * This analysis could be tighter: in particular, a non-strict
- * construct hidden within a lower-level PlaceHolderVar is not
- * reason to add another PHV. But for now it doesn't seem
- * worth the code to be more exact.
- *
- * Note: in future maybe we should insert a PlaceHolderVar
- * anyway, if the tlist item is expensive to evaluate?
- *
- * For a LATERAL subquery, we have to check the actual var
- * membership of the node, but if it's non-lateral then any
- * level-zero var must belong to the subquery.
+ * Must wrap, either because we need a place to insert
+ * varnullingrels or because caller told us to wrap
+ * everything.
*/
- if ((rcon->target_rte->lateral ?
- bms_overlap(pull_varnos(rcon->root, (Node *) newnode),
- rcon->relids) :
- contain_vars_of_level((Node *) newnode, 0)) &&
- !contain_nonstrict_functions((Node *) newnode))
- {
- /* No wrap needed */
- wrap = false;
- }
- else
- {
- /* Else wrap it in a PlaceHolderVar */
- wrap = true;
- }
+ wrap = true;
}
if (wrap)
+ {
newnode = (Node *)
make_placeholder_expr(rcon->root,
(Expr *) newnode,
bms_make_singleton(rcon->varno));
- /*
- * Cache it if possible (ie, if the attno is in range, which it
- * probably always should be). We can cache the value even if we
- * decided we didn't need a PHV, since this result will be
- * suitable for any request that has need_phvs.
- */
- if (varattno > InvalidAttrNumber &&
- varattno <= list_length(rcon->targetlist))
- rcon->rv_cache[varattno] = copyObject(newnode);
+ /*
+ * Cache it if possible (ie, if the attno is in range, which
+ * it probably always should be).
+ */
+ if (varattno > InvalidAttrNumber &&
+ varattno <= list_length(rcon->targetlist))
+ rcon->rv_cache[varattno] = copyObject(newnode);
+ }
}
}
- /* Must adjust varlevelsup if tlist item is from higher query */
+ /* Must adjust varlevelsup if replaced Var is within a subquery */
if (var->varlevelsup > 0)
IncrementVarSublevelsUp(newnode, var->varlevelsup, 0);
+ /* Propagate any varnullingrels into the replacement Var or PHV */
+ if (var->varnullingrels != NULL)
+ {
+ if (IsA(newnode, Var))
+ {
+ Var *newvar = (Var *) newnode;
+
+ Assert(newvar->varlevelsup == var->varlevelsup);
+ newvar->varnullingrels = bms_add_members(newvar->varnullingrels,
+ var->varnullingrels);
+ }
+ else if (IsA(newnode, PlaceHolderVar))
+ {
+ PlaceHolderVar *newphv = (PlaceHolderVar *) newnode;
+
+ Assert(newphv->phlevelsup == var->varlevelsup);
+ newphv->phnullingrels = bms_add_members(newphv->phnullingrels,
+ var->varnullingrels);
+ }
+ else
+ elog(ERROR, "failed to wrap a non-Var");
+ }
+
return newnode;
}
@@ -2674,7 +2616,9 @@ flatten_simple_union_all(PlannerInfo *root)
void
reduce_outer_joins(PlannerInfo *root)
{
- reduce_outer_joins_state *state;
+ reduce_outer_joins_pass1_state *state1;
+ reduce_outer_joins_pass2_state state2;
+ ListCell *lc;
/*
* To avoid doing strictness checks on more quals than necessary, we want
@@ -2685,14 +2629,44 @@ reduce_outer_joins(PlannerInfo *root)
* join(s) below each side of each join clause. The second pass examines
* qual clauses and changes join types as it descends the tree.
*/
- state = reduce_outer_joins_pass1((Node *) root->parse->jointree);
+ state1 = reduce_outer_joins_pass1((Node *) root->parse->jointree);
/* planner.c shouldn't have called me if no outer joins */
- if (state == NULL || !state->contains_outer)
+ if (state1 == NULL || !state1->contains_outer)
elog(ERROR, "so where are the outer joins?");
+ state2.inner_reduced = NULL;
+ state2.partial_reduced = NIL;
+
reduce_outer_joins_pass2((Node *) root->parse->jointree,
- state, root, NULL, NIL, NIL);
+ state1, &state2,
+ root, NULL, NIL, NIL);
+
+ /*
+ * If we successfully reduced the strength of any outer joins, we must
+ * remove references to those joins as nulling rels. This is handled as
+ * an additional pass, for simplicity and because we can handle all
+ * fully-reduced joins in a single pass over the parse tree.
+ */
+ if (!bms_is_empty(state2.inner_reduced))
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ state2.inner_reduced,
+ NULL);
+
+ /*
+ * Partially-reduced full joins have to be done one at a time, since
+ * they'll each need a different setting of except_relids.
+ */
+ foreach(lc, state2.partial_reduced)
+ {
+ reduce_outer_joins_partial_state *statep = lfirst(lc);
+
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ bms_make_singleton(statep->full_join_rti),
+ statep->unreduced_side);
+ }
}
/*
@@ -2700,13 +2674,13 @@ reduce_outer_joins(PlannerInfo *root)
*
* Returns a state node describing the given jointree node.
*/
-static reduce_outer_joins_state *
+static reduce_outer_joins_pass1_state *
reduce_outer_joins_pass1(Node *jtnode)
{
- reduce_outer_joins_state *result;
+ reduce_outer_joins_pass1_state *result;
- result = (reduce_outer_joins_state *)
- palloc(sizeof(reduce_outer_joins_state));
+ result = (reduce_outer_joins_pass1_state *)
+ palloc(sizeof(reduce_outer_joins_pass1_state));
result->relids = NULL;
result->contains_outer = false;
result->sub_states = NIL;
@@ -2726,7 +2700,7 @@ reduce_outer_joins_pass1(Node *jtnode)
foreach(l, f->fromlist)
{
- reduce_outer_joins_state *sub_state;
+ reduce_outer_joins_pass1_state *sub_state;
sub_state = reduce_outer_joins_pass1(lfirst(l));
result->relids = bms_add_members(result->relids,
@@ -2738,7 +2712,7 @@ reduce_outer_joins_pass1(Node *jtnode)
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- reduce_outer_joins_state *sub_state;
+ reduce_outer_joins_pass1_state *sub_state;
/* join's own RT index is not wanted in result->relids */
if (IS_OUTER_JOIN(j->jointype))
@@ -2766,15 +2740,23 @@ reduce_outer_joins_pass1(Node *jtnode)
* reduce_outer_joins_pass2 - phase 2 processing
*
* jtnode: current jointree node
- * state: state data collected by phase 1 for this node
+ * state1: state data collected by phase 1 for this node
+ * state2: where to accumulate info about successfully-reduced joins
* root: toplevel planner state
* nonnullable_rels: set of base relids forced non-null by upper quals
* nonnullable_vars: list of Vars forced non-null by upper quals
* forced_null_vars: list of Vars forced null by upper quals
+ *
+ * Returns info in state2 about outer joins that were successfully simplified.
+ * Joins that were fully reduced to inner joins are all added to
+ * state2->inner_reduced. If a full join is reduced to a left join,
+ * it needs its own entry in state2->partial_reduced, since that will
+ * require custom processing to remove only the correct nullingrel markers.
*/
static void
reduce_outer_joins_pass2(Node *jtnode,
- reduce_outer_joins_state *state,
+ reduce_outer_joins_pass1_state *state1,
+ reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
List *nonnullable_vars,
@@ -2808,13 +2790,14 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_forced_null_vars = list_concat(pass_forced_null_vars,
forced_null_vars);
/* And recurse --- but only into interesting subtrees */
- Assert(list_length(f->fromlist) == list_length(state->sub_states));
- forboth(l, f->fromlist, s, state->sub_states)
+ Assert(list_length(f->fromlist) == list_length(state1->sub_states));
+ forboth(l, f->fromlist, s, state1->sub_states)
{
- reduce_outer_joins_state *sub_state = lfirst(s);
+ reduce_outer_joins_pass1_state *sub_state = lfirst(s);
if (sub_state->contains_outer)
- reduce_outer_joins_pass2(lfirst(l), sub_state, root,
+ reduce_outer_joins_pass2(lfirst(l), sub_state,
+ state2, root,
pass_nonnullable_rels,
pass_nonnullable_vars,
pass_forced_null_vars);
@@ -2827,8 +2810,8 @@ reduce_outer_joins_pass2(Node *jtnode,
JoinExpr *j = (JoinExpr *) jtnode;
int rtindex = j->rtindex;
JoinType jointype = j->jointype;
- reduce_outer_joins_state *left_state = linitial(state->sub_states);
- reduce_outer_joins_state *right_state = lsecond(state->sub_states);
+ reduce_outer_joins_pass1_state *left_state = linitial(state1->sub_states);
+ reduce_outer_joins_pass1_state *right_state = lsecond(state1->sub_states);
List *local_nonnullable_vars = NIL;
bool computed_local_nonnullable_vars = false;
@@ -2851,12 +2834,22 @@ reduce_outer_joins_pass2(Node *jtnode,
if (bms_overlap(nonnullable_rels, right_state->relids))
jointype = JOIN_INNER;
else
+ {
jointype = JOIN_LEFT;
+ /* Also report partial reduction in state2 */
+ report_reduced_full_join(state2, rtindex,
+ right_state->relids);
+ }
}
else
{
if (bms_overlap(nonnullable_rels, right_state->relids))
+ {
jointype = JOIN_RIGHT;
+ /* Also report partial reduction in state2 */
+ report_reduced_full_join(state2, rtindex,
+ left_state->relids);
+ }
}
break;
case JOIN_SEMI:
@@ -2889,8 +2882,8 @@ reduce_outer_joins_pass2(Node *jtnode,
j->larg = j->rarg;
j->rarg = tmparg;
jointype = JOIN_LEFT;
- right_state = linitial(state->sub_states);
- left_state = lsecond(state->sub_states);
+ right_state = linitial(state1->sub_states);
+ left_state = lsecond(state1->sub_states);
}
/*
@@ -2923,7 +2916,10 @@ reduce_outer_joins_pass2(Node *jtnode,
jointype = JOIN_ANTI;
}
- /* Apply the jointype change, if any, to both jointree node and RTE */
+ /*
+ * Apply the jointype change, if any, to both jointree node and RTE.
+ * Also, if we changed an RTE to INNER, add its RTI to inner_reduced.
+ */
if (rtindex && jointype != j->jointype)
{
RangeTblEntry *rte = rt_fetch(rtindex, root->parse->rtable);
@@ -2931,6 +2927,9 @@ reduce_outer_joins_pass2(Node *jtnode,
Assert(rte->rtekind == RTE_JOIN);
Assert(rte->jointype == j->jointype);
rte->jointype = jointype;
+ if (jointype == JOIN_INNER)
+ state2->inner_reduced = bms_add_member(state2->inner_reduced,
+ rtindex);
}
j->jointype = jointype;
@@ -3011,7 +3010,8 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_nonnullable_vars = NIL;
pass_forced_null_vars = NIL;
}
- reduce_outer_joins_pass2(j->larg, left_state, root,
+ reduce_outer_joins_pass2(j->larg, left_state,
+ state2, root,
pass_nonnullable_rels,
pass_nonnullable_vars,
pass_forced_null_vars);
@@ -3033,7 +3033,8 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_nonnullable_vars = NIL;
pass_forced_null_vars = NIL;
}
- reduce_outer_joins_pass2(j->rarg, right_state, root,
+ reduce_outer_joins_pass2(j->rarg, right_state,
+ state2, root,
pass_nonnullable_rels,
pass_nonnullable_vars,
pass_forced_null_vars);
@@ -3046,6 +3047,19 @@ reduce_outer_joins_pass2(Node *jtnode,
(int) nodeTag(jtnode));
}
+/* Helper for reduce_outer_joins_pass2 */
+static void
+report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
+ int rtindex, Relids relids)
+{
+ reduce_outer_joins_partial_state *statep;
+
+ statep = palloc(sizeof(reduce_outer_joins_partial_state));
+ statep->full_join_rti = rtindex;
+ statep->unreduced_side = relids;
+ state2->partial_reduced = lappend(state2->partial_reduced, statep);
+}
+
/*
* remove_useless_result_rtes
@@ -3087,16 +3101,34 @@ reduce_outer_joins_pass2(Node *jtnode,
void
remove_useless_result_rtes(PlannerInfo *root)
{
+ Relids dropped_outer_joins = NULL;
ListCell *cell;
/* Top level of jointree must always be a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
/* Recurse ... */
root->parse->jointree = (FromExpr *)
- remove_useless_results_recurse(root, (Node *) root->parse->jointree);
+ remove_useless_results_recurse(root,
+ (Node *) root->parse->jointree,
+ &dropped_outer_joins);
/* We should still have a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
+ /*
+ * If we removed any outer-join nodes from the jointree, run around and
+ * remove references to those joins as nulling rels. (There could be such
+ * references in PHVs that we pulled up out of the original subquery that
+ * the RESULT rel replaced. This is kosher on the grounds that we now
+ * know that such an outer join wouldn't really have nulled anything.) We
+ * don't do this during the main recursion, for simplicity and because we
+ * can handle all such joins in a single pass over the parse tree.
+ */
+ if (!bms_is_empty(dropped_outer_joins))
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ dropped_outer_joins,
+ NULL);
+
/*
* Remove any PlanRowMark referencing an RTE_RESULT RTE. We obviously
* must do that for any RTE_RESULT that we just removed. But one for a
@@ -3122,9 +3154,12 @@ remove_useless_result_rtes(PlannerInfo *root)
* Recursive guts of remove_useless_result_rtes.
*
* This recursively processes the jointree and returns a modified jointree.
+ * In addition, the RT indexes of any removed outer-join nodes are added to
+ * *dropped_outer_joins.
*/
static Node *
-remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
+remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Relids *dropped_outer_joins)
{
Assert(jtnode != NULL);
if (IsA(jtnode, RangeTblRef))
@@ -3152,7 +3187,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
int varno;
/* Recursively transform child ... */
- child = remove_useless_results_recurse(root, child);
+ child = remove_useless_results_recurse(root, child,
+ dropped_outer_joins);
/* ... and stick it back into the tree */
lfirst(cell) = child;
@@ -3201,8 +3237,10 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
int varno;
/* First, recurse */
- j->larg = remove_useless_results_recurse(root, j->larg);
- j->rarg = remove_useless_results_recurse(root, j->rarg);
+ j->larg = remove_useless_results_recurse(root, j->larg,
+ dropped_outer_joins);
+ j->rarg = remove_useless_results_recurse(root, j->rarg,
+ dropped_outer_joins);
/* Apply join-type-specific optimization rules */
switch (j->jointype)
@@ -3270,6 +3308,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
!find_dependent_phvs(root, varno)))
{
remove_result_refs(root, varno, j->larg);
+ *dropped_outer_joins = bms_add_member(*dropped_outer_joins,
+ j->rtindex);
jtnode = j->larg;
}
break;
@@ -3280,6 +3320,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
!find_dependent_phvs(root, varno)))
{
remove_result_refs(root, varno, j->rarg);
+ *dropped_outer_joins = bms_add_member(*dropped_outer_joins,
+ j->rtindex);
jtnode = j->rarg;
}
break;
@@ -3294,11 +3336,14 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
* Unlike the LEFT/RIGHT cases, we just Assert that there are
* no PHVs that need to be evaluated at the semijoin's RHS,
* since the rest of the query couldn't reference any outputs
- * of the semijoin's RHS.
+ * of the semijoin's RHS. Also, we don't need to worry about
+ * removing traces of the join's rtindex, since it hasn't got
+ * one.
*/
if ((varno = get_result_relid(root, j->rarg)) != 0)
{
Assert(!find_dependent_phvs(root, varno));
+ Assert(j->rtindex == 0);
remove_result_refs(root, varno, j->larg);
if (j->quals)
jtnode = (Node *)
@@ -3367,7 +3412,7 @@ remove_result_refs(PlannerInfo *root, int varno, Node *newjtloc)
{
Relids subrelids;
- subrelids = get_relids_in_jointree(newjtloc, false);
+ subrelids = get_relids_in_jointree(newjtloc, true, false);
Assert(!bms_is_empty(subrelids));
substitute_phv_relids((Node *) root->parse, varno, subrelids);
fix_append_rel_relids(root->append_rel_list, varno, subrelids);
@@ -3479,7 +3524,7 @@ find_dependent_phvs_in_jointree(PlannerInfo *root, Node *node, int varno)
* are not marked LATERAL, though, since they couldn't possibly contain
* any cross-references to other RTEs.
*/
- subrelids = get_relids_in_jointree(node, false);
+ subrelids = get_relids_in_jointree(node, false, false);
relid = -1;
while ((relid = bms_next_member(subrelids, relid)) >= 0)
{
@@ -3623,11 +3668,17 @@ fix_append_rel_relids(List *append_rel_list, int varno, Relids subrelids)
/*
* get_relids_in_jointree: get set of RT indexes present in a jointree
*
- * If include_joins is true, join RT indexes are included; if false,
- * only base rels are included.
+ * Base-relation relids are always included in the result.
+ * If include_outer_joins is true, outer-join RT indexes are included.
+ * If include_inner_joins is true, inner-join RT indexes are included.
+ *
+ * Note that for most purposes in the planner, outer joins are included
+ * in standard relid sets. Setting include_inner_joins true is only
+ * appropriate for special purposes during subquery flattening.
*/
Relids
-get_relids_in_jointree(Node *jtnode, bool include_joins)
+get_relids_in_jointree(Node *jtnode, bool include_outer_joins,
+ bool include_inner_joins)
{
Relids result = NULL;
@@ -3648,18 +3699,34 @@ get_relids_in_jointree(Node *jtnode, bool include_joins)
{
result = bms_join(result,
get_relids_in_jointree(lfirst(l),
- include_joins));
+ include_outer_joins,
+ include_inner_joins));
}
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- result = get_relids_in_jointree(j->larg, include_joins);
+ result = get_relids_in_jointree(j->larg,
+ include_outer_joins,
+ include_inner_joins);
result = bms_join(result,
- get_relids_in_jointree(j->rarg, include_joins));
- if (include_joins && j->rtindex)
- result = bms_add_member(result, j->rtindex);
+ get_relids_in_jointree(j->rarg,
+ include_outer_joins,
+ include_inner_joins));
+ if (j->rtindex)
+ {
+ if (j->jointype == JOIN_INNER)
+ {
+ if (include_inner_joins)
+ result = bms_add_member(result, j->rtindex);
+ }
+ else
+ {
+ if (include_outer_joins)
+ result = bms_add_member(result, j->rtindex);
+ }
+ }
}
else
elog(ERROR, "unrecognized node type: %d",
@@ -3668,7 +3735,7 @@ get_relids_in_jointree(Node *jtnode, bool include_joins)
}
/*
- * get_relids_for_join: get set of base RT indexes making up a join
+ * get_relids_for_join: get set of base+OJ RT indexes making up a join
*/
Relids
get_relids_for_join(Query *query, int joinrelid)
@@ -3679,7 +3746,7 @@ get_relids_for_join(Query *query, int joinrelid)
joinrelid);
if (!jtnode)
elog(ERROR, "could not find join node %d", joinrelid);
- return get_relids_in_jointree(jtnode, false);
+ return get_relids_in_jointree(jtnode, true, false);
}
/*
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index 9d4bb47027..4d8da9f6e2 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -228,6 +228,12 @@ adjust_appendrel_attrs_mutator(Node *node,
if (var->varlevelsup != 0)
return (Node *) var; /* no changes needed */
+ /*
+ * You might think we need to adjust var->varnullingrels, but that
+ * shouldn't need any changes. It will contain outer-join relids,
+ * while the transformation we are making affects only baserels.
+ */
+
for (cnt = 0; cnt < nappinfos; cnt++)
{
if (var->varno == appinfos[cnt]->parent_relid)
@@ -348,6 +354,8 @@ adjust_appendrel_attrs_mutator(Node *node,
var = copyObject(ridinfo->rowidvar);
/* ... but use the correct relid */
var->varno = leaf_relid;
+ /* identity vars shouldn't have nulling rels */
+ Assert(var->varnullingrels == NULL);
/* varnosyn in the RowIdentityVarInfo is probably wrong */
var->varnosyn = 0;
var->varattnosyn = 0;
@@ -392,8 +400,11 @@ adjust_appendrel_attrs_mutator(Node *node,
(void *) context);
/* now fix PlaceHolderVar's relid sets */
if (phv->phlevelsup == 0)
- phv->phrels = adjust_child_relids(phv->phrels, context->nappinfos,
- context->appinfos);
+ {
+ phv->phrels = adjust_child_relids(phv->phrels,
+ nappinfos, appinfos);
+ /* as above, we needn't touch phnullingrels */
+ }
return (Node *) phv;
}
/* Shouldn't need to handle planner auxiliary nodes here */
@@ -486,32 +497,26 @@ adjust_appendrel_attrs_mutator(Node *node,
*/
Node *
adjust_appendrel_attrs_multilevel(PlannerInfo *root, Node *node,
- Relids child_relids,
- Relids top_parent_relids)
+ RelOptInfo *childrel,
+ RelOptInfo *parentrel)
{
AppendRelInfo **appinfos;
- Bitmapset *parent_relids = NULL;
int nappinfos;
- int cnt;
-
- Assert(bms_num_members(child_relids) == bms_num_members(top_parent_relids));
-
- appinfos = find_appinfos_by_relids(root, child_relids, &nappinfos);
- /* Construct relids set for the immediate parent of given child. */
- for (cnt = 0; cnt < nappinfos; cnt++)
+ /* Recurse if immediate parent is not the top parent. */
+ if (childrel->parent != parentrel)
{
- AppendRelInfo *appinfo = appinfos[cnt];
-
- parent_relids = bms_add_member(parent_relids, appinfo->parent_relid);
+ if (childrel->parent)
+ node = adjust_appendrel_attrs_multilevel(root, node,
+ childrel->parent,
+ parentrel);
+ else
+ elog(ERROR, "presented child rel is not a child of parent rel");
}
- /* Recurse if immediate parent is not the top parent. */
- if (!bms_equal(parent_relids, top_parent_relids))
- node = adjust_appendrel_attrs_multilevel(root, node, parent_relids,
- top_parent_relids);
+ /* Now translate for this child. */
+ appinfos = find_appinfos_by_relids(root, childrel->relids, &nappinfos);
- /* Now translate for this child */
node = adjust_appendrel_attrs(root, node, nappinfos, appinfos);
pfree(appinfos);
@@ -554,56 +559,43 @@ adjust_child_relids(Relids relids, int nappinfos, AppendRelInfo **appinfos)
}
/*
- * Replace any relid present in top_parent_relids with its child in
- * child_relids. Members of child_relids can be multiple levels below top
- * parent in the partition hierarchy.
+ * Substitute child relids for parent relids in a Relid set.
+ * The childrel can be multiple inheritance levels below the parent.
*/
Relids
adjust_child_relids_multilevel(PlannerInfo *root, Relids relids,
- Relids child_relids, Relids top_parent_relids)
+ RelOptInfo *childrel,
+ RelOptInfo *parentrel)
{
AppendRelInfo **appinfos;
int nappinfos;
- Relids parent_relids = NULL;
- Relids result;
- Relids tmp_result = NULL;
- int cnt;
/*
- * If the given relids set doesn't contain any of the top parent relids,
- * it will remain unchanged.
+ * If the given relids set doesn't contain any of the parent relids, it
+ * will remain unchanged.
*/
- if (!bms_overlap(relids, top_parent_relids))
+ if (!bms_overlap(relids, parentrel->relids))
return relids;
- appinfos = find_appinfos_by_relids(root, child_relids, &nappinfos);
-
- /* Construct relids set for the immediate parent of the given child. */
- for (cnt = 0; cnt < nappinfos; cnt++)
- {
- AppendRelInfo *appinfo = appinfos[cnt];
-
- parent_relids = bms_add_member(parent_relids, appinfo->parent_relid);
- }
-
/* Recurse if immediate parent is not the top parent. */
- if (!bms_equal(parent_relids, top_parent_relids))
+ if (childrel->parent != parentrel)
{
- tmp_result = adjust_child_relids_multilevel(root, relids,
- parent_relids,
- top_parent_relids);
- relids = tmp_result;
+ if (childrel->parent)
+ relids = adjust_child_relids_multilevel(root, relids,
+ childrel->parent,
+ parentrel);
+ else
+ elog(ERROR, "presented child rel is not a child of parent rel");
}
- result = adjust_child_relids(relids, nappinfos, appinfos);
+ /* Now translate for this child. */
+ appinfos = find_appinfos_by_relids(root, childrel->relids, &nappinfos);
+
+ relids = adjust_child_relids(relids, nappinfos, appinfos);
- /* Free memory consumed by any intermediate result. */
- if (tmp_result)
- bms_free(tmp_result);
- bms_free(parent_relids);
pfree(appinfos);
- return result;
+ return relids;
}
/*
@@ -694,8 +686,8 @@ get_translated_update_targetlist(PlannerInfo *root, Index relid,
*processed_tlist = (List *)
adjust_appendrel_attrs_multilevel(root,
(Node *) root->processed_tlist,
- bms_make_singleton(relid),
- bms_make_singleton(root->parse->resultRelation));
+ find_base_rel(root, relid),
+ find_base_rel(root, root->parse->resultRelation));
if (update_colnos)
*update_colnos =
adjust_inherited_attnums_multilevel(root, root->update_colnos,
@@ -706,7 +698,11 @@ get_translated_update_targetlist(PlannerInfo *root, Index relid,
/*
* find_appinfos_by_relids
- * Find AppendRelInfo structures for all relations specified by relids.
+ * Find AppendRelInfo structures for base relations listed in relids.
+ *
+ * The relids argument is typically a join relation's relids, which can
+ * include outer-join RT indexes in addition to baserels. We silently
+ * ignore the outer joins.
*
* The AppendRelInfos are returned in an array, which can be pfree'd by the
* caller. *nappinfos is set to the number of entries in the array.
@@ -718,8 +714,9 @@ find_appinfos_by_relids(PlannerInfo *root, Relids relids, int *nappinfos)
int cnt = 0;
int i;
- *nappinfos = bms_num_members(relids);
- appinfos = (AppendRelInfo **) palloc(sizeof(AppendRelInfo *) * *nappinfos);
+ /* Allocate an array that's certainly big enough */
+ appinfos = (AppendRelInfo **)
+ palloc(sizeof(AppendRelInfo *) * bms_num_members(relids));
i = -1;
while ((i = bms_next_member(relids, i)) >= 0)
@@ -727,10 +724,17 @@ find_appinfos_by_relids(PlannerInfo *root, Relids relids, int *nappinfos)
AppendRelInfo *appinfo = root->append_rel_array[i];
if (!appinfo)
+ {
+ /* Probably i is an OJ index, but let's check */
+ if (find_base_rel_ignore_join(root, i) == NULL)
+ continue;
+ /* It's a base rel, but we lack an append_rel_array entry */
elog(ERROR, "child rel %d not found in append_rel_array", i);
+ }
appinfos[cnt++] = appinfo;
}
+ *nappinfos = cnt;
return appinfos;
}
@@ -772,6 +776,7 @@ add_row_identity_var(PlannerInfo *root, Var *orig_var,
Assert(IsA(orig_var, Var));
Assert(orig_var->varno == rtindex);
Assert(orig_var->varlevelsup == 0);
+ Assert(orig_var->varnullingrels == NULL);
/*
* If we're doing non-inherited UPDATE/DELETE/MERGE, there's little need
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 533df86ff7..c82fd451b2 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -2022,14 +2022,16 @@ is_pseudo_constant_clause_relids(Node *clause, Relids relids)
* NumRelids
* (formerly clause_relids)
*
- * Returns the number of different relations referenced in 'clause'.
+ * Returns the number of different base relations referenced in 'clause'.
*/
int
NumRelids(PlannerInfo *root, Node *clause)
{
+ int result;
Relids varnos = pull_varnos(root, clause);
- int result = bms_num_members(varnos);
+ varnos = bms_del_members(varnos, root->outer_join_rels);
+ result = bms_num_members(varnos);
bms_free(varnos);
return result;
}
diff --git a/src/backend/optimizer/util/joininfo.c b/src/backend/optimizer/util/joininfo.c
index d4cffdb198..afd243f5d8 100644
--- a/src/backend/optimizer/util/joininfo.c
+++ b/src/backend/optimizer/util/joininfo.c
@@ -88,8 +88,8 @@ have_relevant_joinclause(PlannerInfo *root,
* not depend on context).
*
* 'restrictinfo' describes the join clause
- * 'join_relids' is the list of relations participating in the join clause
- * (there must be more than one)
+ * 'join_relids' is the set of relations participating in the join clause
+ * (some of these could be outer joins)
*/
void
add_join_clause_to_rels(PlannerInfo *root,
@@ -101,8 +101,11 @@ add_join_clause_to_rels(PlannerInfo *root,
cur_relid = -1;
while ((cur_relid = bms_next_member(join_relids, cur_relid)) >= 0)
{
- RelOptInfo *rel = find_base_rel(root, cur_relid);
+ RelOptInfo *rel = find_base_rel_ignore_join(root, cur_relid);
+ /* We only need to add the clause to baserels */
+ if (rel == NULL)
+ continue;
rel->joininfo = lappend(rel->joininfo, restrictinfo);
}
}
@@ -115,8 +118,8 @@ add_join_clause_to_rels(PlannerInfo *root,
* discover that a relation need not be joined at all.
*
* 'restrictinfo' describes the join clause
- * 'join_relids' is the list of relations participating in the join clause
- * (there must be more than one)
+ * 'join_relids' is the set of relations participating in the join clause
+ * (some of these could be outer joins)
*/
void
remove_join_clause_from_rels(PlannerInfo *root,
@@ -128,7 +131,11 @@ remove_join_clause_from_rels(PlannerInfo *root,
cur_relid = -1;
while ((cur_relid = bms_next_member(join_relids, cur_relid)) >= 0)
{
- RelOptInfo *rel = find_base_rel(root, cur_relid);
+ RelOptInfo *rel = find_base_rel_ignore_join(root, cur_relid);
+
+ /* We would only have added the clause to baserels */
+ if (rel == NULL)
+ continue;
/*
* Remove the restrictinfo from the list. Pointer comparison is
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index b1363df065..a62d4587ea 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -338,7 +338,9 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
sjinfo.syn_lefthand = sjinfo.min_lefthand;
sjinfo.syn_righthand = sjinfo.min_righthand;
sjinfo.jointype = JOIN_INNER;
+ sjinfo.ojrelid = 0;
/* we don't bother trying to make the remaining fields valid */
+ sjinfo.strict_relids = NULL;
sjinfo.lhs_strict = false;
sjinfo.delay_upper_joins = false;
sjinfo.semi_can_btree = false;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index e2a3c110ce..93235ee3e2 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -1307,7 +1307,7 @@ create_append_path(PlannerInfo *root,
* Apply query-wide LIMIT if known and path is for sole base relation.
* (Handling this at this low level is a bit klugy.)
*/
- if (root != NULL && bms_equal(rel->relids, root->all_baserels))
+ if (root != NULL && bms_equal(rel->relids, root->all_query_rels))
pathnode->limit_tuples = root->limit_tuples;
else
pathnode->limit_tuples = -1.0;
@@ -1427,7 +1427,7 @@ create_merge_append_path(PlannerInfo *root,
* Apply query-wide LIMIT if known and path is for sole base relation.
* (Handling this at this low level is a bit klugy.)
*/
- if (bms_equal(rel->relids, root->all_baserels))
+ if (bms_equal(rel->relids, root->all_query_rels))
pathnode->limit_tuples = root->limit_tuples;
else
pathnode->limit_tuples = -1.0;
@@ -3996,8 +3996,8 @@ reparameterize_path_by_child(PlannerInfo *root, Path *path,
#define ADJUST_CHILD_ATTRS(node) \
((node) = \
(List *) adjust_appendrel_attrs_multilevel(root, (Node *) (node), \
- child_rel->relids, \
- child_rel->top_parent_relids))
+ child_rel, \
+ child_rel->top_parent))
#define REPARAMETERIZE_CHILD_PATH(path) \
do { \
@@ -4027,7 +4027,7 @@ do { \
* doesn't need reparameterization.
*/
if (!path->param_info ||
- !bms_overlap(PATH_REQ_OUTER(path), child_rel->top_parent_relids))
+ !bms_overlap(PATH_REQ_OUTER(path), child_rel->top_parent->relids))
return path;
/*
@@ -4214,8 +4214,8 @@ do { \
old_ppi = new_path->param_info;
required_outer =
adjust_child_relids_multilevel(root, old_ppi->ppi_req_outer,
- child_rel->relids,
- child_rel->top_parent_relids);
+ child_rel,
+ child_rel->top_parent);
/* If we already have a PPI for this parameterization, just return it */
new_ppi = find_param_path_info(new_path->parent, required_outer);
@@ -4251,7 +4251,7 @@ do { \
* outer relation is laterally referenced in this relation.
*/
if (bms_overlap(path->parent->lateral_relids,
- child_rel->top_parent_relids))
+ child_rel->top_parent->relids))
{
new_path->pathtarget = copy_pathtarget(new_path->pathtarget);
ADJUST_CHILD_ATTRS(new_path->pathtarget->exprs);
diff --git a/src/backend/optimizer/util/placeholder.c b/src/backend/optimizer/util/placeholder.c
index 3b0f0584f0..8226e6c0f7 100644
--- a/src/backend/optimizer/util/placeholder.c
+++ b/src/backend/optimizer/util/placeholder.c
@@ -32,8 +32,14 @@ static void find_placeholders_in_expr(PlannerInfo *root, Node *expr);
* make_placeholder_expr
* Make a PlaceHolderVar for the given expression.
*
- * phrels is the syntactic location (as a set of baserels) to attribute
+ * phrels is the syntactic location (as a set of relids) to attribute
* to the expression.
+ *
+ * The caller is responsible for adjusting phlevelsup and phnullingrels
+ * as needed. Because we do not know here which query level the PHV
+ * will be associated with, it's important that this function touches
+ * only root->glob; messing with other parts of PlannerInfo would be
+ * likely to do the wrong thing.
*/
PlaceHolderVar *
make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
@@ -42,8 +48,9 @@ make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
phv->phexpr = expr;
phv->phrels = phrels;
+ phv->phnullingrels = NULL; /* caller may change this later */
phv->phid = ++(root->glob->lastPHId);
- phv->phlevelsup = 0;
+ phv->phlevelsup = 0; /* caller may change this later */
return phv;
}
@@ -317,6 +324,8 @@ update_placeholder_eval_levels(PlannerInfo *root, SpecialJoinInfo *new_sjinfo)
sjinfo->min_lefthand);
eval_at = bms_add_members(eval_at,
sjinfo->min_righthand);
+ if (sjinfo->ojrelid)
+ eval_at = bms_add_member(eval_at, sjinfo->ojrelid);
/* we'll need another iteration */
found_some = true;
}
@@ -390,9 +399,16 @@ add_placeholders_to_base_rels(PlannerInfo *root)
bms_nonempty_difference(phinfo->ph_needed, eval_at))
{
RelOptInfo *rel = find_base_rel(root, varno);
+ PlaceHolderVar *phv;
- rel->reltarget->exprs = lappend(rel->reltarget->exprs,
- copyObject(phinfo->ph_var));
+ /*
+ * As in add_vars_to_targetlist(), a value computed at scan level
+ * has not yet been nulled by any outer join, so set its
+ * phnullingrels to empty.
+ */
+ phv = copyObject(phinfo->ph_var);
+ phv->phnullingrels = NULL;
+ rel->reltarget->exprs = lappend(rel->reltarget->exprs, phv);
/* reltarget's cost and width fields will be updated later */
}
}
@@ -411,7 +427,8 @@ add_placeholders_to_base_rels(PlannerInfo *root)
*/
void
add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *outer_rel, RelOptInfo *inner_rel)
+ RelOptInfo *outer_rel, RelOptInfo *inner_rel,
+ SpecialJoinInfo *sjinfo)
{
Relids relids = joinrel->relids;
ListCell *lc;
@@ -426,10 +443,11 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
/* Is it still needed above this joinrel? */
if (bms_nonempty_difference(phinfo->ph_needed, relids))
{
- /* Yup, add it to the output */
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- phinfo->ph_var);
- joinrel->reltarget->width += phinfo->ph_width;
+ /*
+ * Yup, we must add it to the output. Make a copy so we can
+ * adjust phnullingrels if needed.
+ */
+ PlaceHolderVar *phv = copyObject(phinfo->ph_var);
/*
* Charge the cost of evaluating the contained expression if
@@ -442,16 +460,42 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
* with that; but we might want to improve it later by
* refiguring the reltarget costs for each pair of inputs.
*/
- if (!bms_is_subset(phinfo->ph_eval_at, outer_rel->relids) &&
- !bms_is_subset(phinfo->ph_eval_at, inner_rel->relids))
+ if (bms_is_subset(phinfo->ph_eval_at, outer_rel->relids))
+ {
+ if (sjinfo->jointype == JOIN_FULL && sjinfo->ojrelid != 0)
+ {
+ /* PHV's value can be nulled at this join */
+ phv->phnullingrels = bms_add_member(phv->phnullingrels,
+ sjinfo->ojrelid);
+ }
+ }
+ else if (bms_is_subset(phinfo->ph_eval_at, inner_rel->relids))
{
+ if (sjinfo->jointype != JOIN_INNER && sjinfo->ojrelid != 0)
+ {
+ /* PHV's value can be nulled at this join */
+ phv->phnullingrels = bms_add_member(phv->phnullingrels,
+ sjinfo->ojrelid);
+ }
+ }
+ else
+ {
+ /* It must be computed here. */
QualCost cost;
+ /* It'll start out not nulled by anything */
+ phv->phnullingrels = NULL;
+ /* Add the appropriate cost */
cost_qual_eval_node(&cost, (Node *) phinfo->ph_var->phexpr,
root);
joinrel->reltarget->cost.startup += cost.startup;
joinrel->reltarget->cost.per_tuple += cost.per_tuple;
}
+
+ joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
+ phv);
+ /* Update width estimate, too */
+ joinrel->reltarget->width += phinfo->ph_width;
}
/*
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 520409f4ba..5284ec367c 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -39,7 +39,7 @@ typedef struct JoinHashEntry
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *input_rel);
+ RelOptInfo *input_rel, int ojrelid);
static List *build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outer_rel,
@@ -58,7 +58,8 @@ static void set_foreign_rel_properties(RelOptInfo *joinrel,
static void add_join_rel(PlannerInfo *root, RelOptInfo *joinrel);
static void build_joinrel_partition_info(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
- List *restrictlist, JoinType jointype);
+ SpecialJoinInfo *sjinfo,
+ List *restrictlist);
static bool have_partkey_equi_join(RelOptInfo *joinrel,
RelOptInfo *rel1, RelOptInfo *rel2,
JoinType jointype, List *restrictlist);
@@ -66,7 +67,8 @@ static int match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel,
bool strict_op);
static void set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
- JoinType jointype);
+ SpecialJoinInfo *sjinfo);
+static Node *add_nullingrel_to(Node *node, int relid);
static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *parentrel,
RelOptInfo *childrel,
@@ -265,14 +267,9 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
*/
if (parent)
{
- /*
- * Each direct or indirect child wants to know the relids of its
- * topmost parent.
- */
- if (parent->top_parent_relids)
- rel->top_parent_relids = parent->top_parent_relids;
- else
- rel->top_parent_relids = bms_copy(parent->relids);
+ /* We keep back-links to immediate parent and topmost parent. */
+ rel->parent = parent;
+ rel->top_parent = parent->top_parent ? parent->top_parent : parent;
/*
* Also propagate lateral-reference information from appendrel parent
@@ -294,7 +291,8 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
}
else
{
- rel->top_parent_relids = NULL;
+ rel->parent = NULL;
+ rel->top_parent = NULL;
rel->direct_lateral_relids = NULL;
rel->lateral_relids = NULL;
rel->lateral_referencers = NULL;
@@ -369,7 +367,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
/*
* find_base_rel
- * Find a base or other relation entry, which must already exist.
+ * Find a base or otherrel relation entry, which must already exist.
*/
RelOptInfo *
find_base_rel(PlannerInfo *root, int relid)
@@ -390,6 +388,44 @@ find_base_rel(PlannerInfo *root, int relid)
return NULL; /* keep compiler quiet */
}
+/*
+ * find_base_rel_ignore_join
+ * Find a base or otherrel relation entry, which must already exist.
+ *
+ * Unlike find_base_rel, if relid references an outer join then this
+ * will return NULL rather than raising an error. This is convenient
+ * for callers that must deal with relid sets including both base and
+ * outer joins.
+ */
+RelOptInfo *
+find_base_rel_ignore_join(PlannerInfo *root, int relid)
+{
+ Assert(relid > 0);
+
+ if (relid < root->simple_rel_array_size)
+ {
+ RelOptInfo *rel;
+ RangeTblEntry *rte;
+
+ rel = root->simple_rel_array[relid];
+ if (rel)
+ return rel;
+
+ /*
+ * We could just return NULL here, but for debugging purposes it seems
+ * best to actually verify that the relid is an outer join and not
+ * something weird.
+ */
+ rte = root->simple_rte_array[relid];
+ if (rte && rte->rtekind == RTE_JOIN && rte->jointype != JOIN_INNER)
+ return NULL;
+ }
+
+ elog(ERROR, "no relation entry for relid %d", relid);
+
+ return NULL; /* keep compiler quiet */
+}
+
/*
* build_join_rel_hash
* Construct the auxiliary hash table for join relations.
@@ -663,7 +699,8 @@ build_join_rel(PlannerInfo *root,
joinrel->joininfo = NIL;
joinrel->has_eclass_joins = false;
joinrel->consider_partitionwise_join = false; /* might get changed later */
- joinrel->top_parent_relids = NULL;
+ joinrel->parent = NULL;
+ joinrel->top_parent = NULL;
joinrel->part_scheme = NULL;
joinrel->nparts = -1;
joinrel->boundinfo = NULL;
@@ -686,9 +723,11 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ build_joinrel_tlist(root, joinrel, outer_rel,
+ (sjinfo->jointype == JOIN_FULL) ? sjinfo->ojrelid : 0);
+ build_joinrel_tlist(root, joinrel, inner_rel,
+ (sjinfo->jointype != JOIN_INNER) ? sjinfo->ojrelid : 0);
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel, sjinfo);
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -720,8 +759,8 @@ build_join_rel(PlannerInfo *root,
joinrel->has_eclass_joins = has_relevant_eclass_joinclause(root, joinrel);
/* Store the partition information. */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- sjinfo->jointype);
+ build_joinrel_partition_info(joinrel, outer_rel, inner_rel, sjinfo,
+ restrictlist);
/*
* Set estimates of the joinrel's size.
@@ -777,16 +816,14 @@ build_join_rel(PlannerInfo *root,
* 'parent_joinrel' is the RelOptInfo representing the join between parent
* relations. Some of the members of new RelOptInfo are produced by
* translating corresponding members of this RelOptInfo
- * 'sjinfo': child-join context info
* 'restrictlist': list of RestrictInfo nodes that apply to this particular
* pair of joinable relations
- * 'jointype' is the join type (inner, left, full, etc)
+ * 'sjinfo': child join's join-type details
*/
RelOptInfo *
build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
RelOptInfo *inner_rel, RelOptInfo *parent_joinrel,
- List *restrictlist, SpecialJoinInfo *sjinfo,
- JoinType jointype)
+ List *restrictlist, SpecialJoinInfo *sjinfo)
{
RelOptInfo *joinrel = makeNode(RelOptInfo);
AppendRelInfo **appinfos;
@@ -800,6 +837,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->reloptkind = RELOPT_OTHER_JOINREL;
joinrel->relids = bms_union(outer_rel->relids, inner_rel->relids);
+ if (sjinfo->ojrelid != 0)
+ joinrel->relids = bms_add_member(joinrel->relids, sjinfo->ojrelid);
joinrel->rows = 0;
/* cheap startup cost is interesting iff not all tuples to be retrieved */
joinrel->consider_startup = (root->tuple_fraction > 0);
@@ -842,7 +881,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->joininfo = NIL;
joinrel->has_eclass_joins = false;
joinrel->consider_partitionwise_join = false; /* might get changed later */
- joinrel->top_parent_relids = NULL;
+ joinrel->parent = parent_joinrel;
+ joinrel->top_parent = parent_joinrel->top_parent ? parent_joinrel->top_parent : parent_joinrel;
joinrel->part_scheme = NULL;
joinrel->nparts = -1;
joinrel->boundinfo = NULL;
@@ -854,9 +894,6 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->partexprs = NULL;
joinrel->nullable_partexprs = NULL;
- joinrel->top_parent_relids = bms_union(outer_rel->top_parent_relids,
- inner_rel->top_parent_relids);
-
/* Compute information relevant to foreign relations. */
set_foreign_rel_properties(joinrel, outer_rel, inner_rel);
@@ -887,8 +924,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->has_eclass_joins = parent_joinrel->has_eclass_joins;
/* Is the join between partitions itself partitioned? */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- jointype);
+ build_joinrel_partition_info(joinrel, outer_rel, inner_rel, sjinfo,
+ restrictlist);
/* Child joinrel is parallel safe if parent is parallel safe. */
joinrel->consider_parallel = parent_joinrel->consider_parallel;
@@ -967,12 +1004,15 @@ min_join_parameterization(PlannerInfo *root,
* will still be needed above the join. This subroutine adds all such
* Vars from the specified input rel's tlist to the join rel's tlist.
*
+ * If the join can null Vars from this input relation, pass its RT index
+ * (if any) as ojrelid; if not, pass zero.
+ *
* We also compute the expected width of the join's output, making use
* of data that was cached at the baserel level by set_rel_width().
*/
static void
build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *input_rel)
+ RelOptInfo *input_rel, int ojrelid)
{
Relids relids = joinrel->relids;
ListCell *vars;
@@ -1003,9 +1043,7 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
RowIdentityVarInfo *ridinfo = (RowIdentityVarInfo *)
list_nth(root->row_identity_vars, var->varattno - 1);
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- var);
- /* Vars have cost zero, so no need to adjust reltarget->cost */
+ /* Update reltarget width estimate from RowIdentityVarInfo */
joinrel->reltarget->width += ridinfo->rowidwidth;
}
else
@@ -1018,15 +1056,28 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
/* Is it still needed above this joinrel? */
ndx = var->varattno - baserel->min_attr;
- if (bms_nonempty_difference(baserel->attr_needed[ndx], relids))
- {
- /* Yup, add it to the output */
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- var);
- /* Vars have cost zero, so no need to adjust reltarget->cost */
- joinrel->reltarget->width += baserel->attr_widths[ndx];
- }
+ if (!bms_nonempty_difference(baserel->attr_needed[ndx], relids))
+ continue; /* nope, skip it */
+
+ /* Update reltarget width estimate from baserel's attr_widths */
+ joinrel->reltarget->width += baserel->attr_widths[ndx];
+ }
+
+ /*
+ * Add the Var to the output. If this join potentially nulls this
+ * input, we have to update the Var's varnullingrels, which means
+ * making a copy.
+ */
+ if (ojrelid != 0)
+ {
+ var = copyObject(var);
+ var->varnullingrels = bms_add_member(var->varnullingrels, ojrelid);
}
+
+ joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
+ var);
+
+ /* Vars have cost zero, so no need to adjust reltarget->cost */
}
}
@@ -1045,7 +1096,7 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
* is not handled in the sub-relations, so it depends on which
* sub-relations are considered.
*
- * If a join clause from an input relation refers to base rels still not
+ * If a join clause from an input relation refers to base+OJ rels still not
* present in the joinrel, then it is still a join clause for the joinrel;
* we put it into the joininfo list for the joinrel. Otherwise,
* the clause is now a restrict clause for the joined relation, and we
@@ -1646,8 +1697,8 @@ find_param_path_info(RelOptInfo *rel, Relids required_outer)
*/
static void
build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
- RelOptInfo *inner_rel, List *restrictlist,
- JoinType jointype)
+ RelOptInfo *inner_rel, SpecialJoinInfo *sjinfo,
+ List *restrictlist)
{
PartitionScheme part_scheme;
@@ -1674,7 +1725,7 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
!inner_rel->consider_partitionwise_join ||
outer_rel->part_scheme != inner_rel->part_scheme ||
!have_partkey_equi_join(joinrel, outer_rel, inner_rel,
- jointype, restrictlist))
+ sjinfo->jointype, restrictlist))
{
Assert(!IS_PARTITIONED_REL(joinrel));
return;
@@ -1698,7 +1749,7 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* child-join relations of the join relation in try_partitionwise_join().
*/
joinrel->part_scheme = part_scheme;
- set_joinrel_partition_key_exprs(joinrel, outer_rel, inner_rel, jointype);
+ set_joinrel_partition_key_exprs(joinrel, outer_rel, inner_rel, sjinfo);
/*
* Set the consider_partitionwise_join flag.
@@ -1878,6 +1929,23 @@ match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel, bool strict_op)
{
if (equal(lfirst(lc), expr))
return cnt;
+
+ /*
+ * XXX For the moment, also allow a match if we have Vars that
+ * match except for varnullingrels. This may be indicative of a
+ * bug, although given the restriction to strict join operators,
+ * it could be okay.
+ */
+ if (IsA(expr, Var) && IsA(lfirst(lc), Var))
+ {
+ Var *v1 = (Var *) expr;
+ Var *v2 = (Var *) lfirst(lc);
+
+ if (v1->varno == v2->varno &&
+ v1->varattno == v2->varattno &&
+ v1->varlevelsup == v2->varlevelsup)
+ return cnt;
+ }
}
}
@@ -1891,7 +1959,7 @@ match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel, bool strict_op)
static void
set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
- JoinType jointype)
+ SpecialJoinInfo *sjinfo)
{
PartitionScheme part_scheme = joinrel->part_scheme;
int partnatts = part_scheme->partnatts;
@@ -1917,7 +1985,7 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
List *nullable_partexpr = NIL;
ListCell *lc;
- switch (jointype)
+ switch (sjinfo->jointype)
{
/*
* A join relation resulting from an INNER join may be
@@ -1993,18 +2061,37 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
* partitionwise nesting of any outer join.) We assume no
* type coercions are needed to make the coalesce expressions,
* since columns of different types won't have gotten
- * classified as the same PartitionScheme.
+ * classified as the same PartitionScheme. However, we do
+ * have to worry about marking the COALESCE inputs as nullable
+ * by the full join, else these won't match the real thing.
*/
foreach(lc, list_concat_copy(outer_expr, outer_null_expr))
{
Node *larg = (Node *) lfirst(lc);
ListCell *lc2;
+ /* Insert nullingrel, or skip it if we can't */
+ larg = add_nullingrel_to(larg, sjinfo->ojrelid);
+ if (larg == NULL)
+ continue;
+
foreach(lc2, list_concat_copy(inner_expr, inner_null_expr))
{
Node *rarg = (Node *) lfirst(lc2);
- CoalesceExpr *c = makeNode(CoalesceExpr);
+ CoalesceExpr *c;
+
+ /* Forget it if coercions would be needed */
+ if (exprType(larg) != exprType(rarg) ||
+ exprCollation(larg) != exprCollation(rarg))
+ continue;
+ /* Insert nullingrel, or skip it if we can't */
+ rarg = add_nullingrel_to(rarg, sjinfo->ojrelid);
+ if (rarg == NULL)
+ continue;
+
+ /* Now we can build a valid merged join variable */
+ c = makeNode(CoalesceExpr);
c->coalescetype = exprType(larg);
c->coalescecollid = exprCollation(larg);
c->args = list_make2(larg, rarg);
@@ -2015,7 +2102,8 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
break;
default:
- elog(ERROR, "unrecognized join type: %d", (int) jointype);
+ elog(ERROR, "unrecognized join type: %d",
+ (int) sjinfo->jointype);
}
joinrel->partexprs[cnt] = partexpr;
@@ -2023,6 +2111,54 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
}
}
+/*
+ * Attempt to add relid to nullingrels of a FULL JOIN USING variable.
+ * Returns the modified expression if successful, or NULL if we failed.
+ *
+ * We currently don't support any cases where type coercion is involved,
+ * so only plain Vars and COALESCE nodes need be handled. However, we
+ * do need to support nested COALESCEs, so recursion is required.
+ */
+static Node *
+add_nullingrel_to(Node *node, int relid)
+{
+ if (IsA(node, Var))
+ {
+ /* Copy so we can modify it... */
+ Var *var = (Var *) copyObject(node);
+
+ /* ... and insert the correct nullingrel marker */
+ var->varnullingrels = bms_add_member(var->varnullingrels,
+ relid);
+ return (Node *) var;
+ }
+ if (IsA(node, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) node;
+ CoalesceExpr *newcexpr;
+ List *newargs = NIL;
+ ListCell *lc;
+
+ /* Try to modify each argument ... */
+ foreach(lc, cexpr->args)
+ {
+ Node *newarg = add_nullingrel_to((Node *) lfirst(lc), relid);
+
+ if (newarg == NULL)
+ return NULL;
+ newargs = lappend(newargs, newarg);
+ }
+ /* Success, so make the result node */
+ newcexpr = makeNode(CoalesceExpr);
+ newcexpr->coalescetype = cexpr->coalescetype;
+ newcexpr->coalescecollid = cexpr->coalescecollid;
+ newcexpr->args = newargs;
+ newcexpr->location = cexpr->location;
+ return (Node *) newcexpr;
+ }
+ return NULL;
+}
+
/*
* build_child_join_reltarget
* Set up a child-join relation's reltarget from a parent-join relation.
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index ef8df3d098..6902c2a9d7 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -116,6 +116,7 @@ make_restrictinfo_internal(PlannerInfo *root,
Relids nullable_relids)
{
RestrictInfo *restrictinfo = makeNode(RestrictInfo);
+ Relids baserels;
restrictinfo->clause = clause;
restrictinfo->orclause = orclause;
@@ -187,6 +188,20 @@ make_restrictinfo_internal(PlannerInfo *root,
else
restrictinfo->required_relids = restrictinfo->clause_relids;
+ /*
+ * Count the number of base rels appearing in clause_relids. To do this,
+ * we just delete rels mentioned in root->outer_join_rels and count the
+ * survivors. Because we are called during deconstruct_jointree which is
+ * the same tree walk that populates outer_join_rels, this is a little bit
+ * unsafe-looking; but it should be fine because the recursion in
+ * deconstruct_jointree should already have visited any outer join that
+ * could be mentioned in this clause.
+ */
+ baserels = bms_difference(restrictinfo->clause_relids,
+ root->outer_join_rels);
+ restrictinfo->num_base_rels = bms_num_members(baserels);
+ bms_free(baserels);
+
/*
* Fill in all the cacheable fields with "not yet set" markers. None of
* these will be computed until/unless needed. Note in particular that we
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
index ebc6ce84b0..81fc3002fe 100644
--- a/src/backend/optimizer/util/var.c
+++ b/src/backend/optimizer/util/var.c
@@ -62,6 +62,7 @@ typedef struct
typedef struct
{
+ PlannerInfo *root; /* could be NULL! */
Query *query; /* outer Query */
int sublevels_up;
bool possible_sublink; /* could aliases include a SubLink? */
@@ -80,6 +81,10 @@ static bool pull_var_clause_walker(Node *node,
pull_var_clause_context *context);
static Node *flatten_join_alias_vars_mutator(Node *node,
flatten_join_alias_vars_context *context);
+static Node *add_nullingrels_if_needed(PlannerInfo *root, Node *newnode,
+ Var *oldvar);
+static bool is_standard_join_alias_expression(Node *newnode, Var *oldvar);
+static void adjust_standard_join_alias_expression(Node *newnode, Var *oldvar);
static Relids alias_relid_set(Query *query, Relids relids);
@@ -88,6 +93,9 @@ static Relids alias_relid_set(Query *query, Relids relids);
* Create a set of all the distinct varnos present in a parsetree.
* Only varnos that reference level-zero rtable entries are considered.
*
+ * The result includes outer-join relids mentioned in Var.varnullingrels and
+ * PlaceHolderVar.phnullingrels fields in the parsetree.
+ *
* "root" can be passed as NULL if it is not necessary to process
* PlaceHolderVars.
*
@@ -153,7 +161,11 @@ pull_varnos_walker(Node *node, pull_varnos_context *context)
Var *var = (Var *) node;
if (var->varlevelsup == context->sublevels_up)
+ {
context->varnos = bms_add_member(context->varnos, var->varno);
+ context->varnos = bms_add_members(context->varnos,
+ var->varnullingrels);
+ }
return false;
}
if (IsA(node, CurrentOfExpr))
@@ -251,6 +263,14 @@ pull_varnos_walker(Node *node, pull_varnos_context *context)
context->varnos = bms_join(context->varnos,
newevalat);
}
+
+ /*
+ * In all three cases, include phnullingrels in the result. We
+ * don't worry about possibly needing to translate it, because
+ * appendrels only translate varnos of baserels, not outer joins.
+ */
+ context->varnos = bms_add_members(context->varnos,
+ phv->phnullingrels);
return false; /* don't recurse into expression */
}
}
@@ -714,26 +734,42 @@ pull_var_clause_walker(Node *node, pull_var_clause_context *context)
* is the only way that the executor can directly handle whole-row Vars.
*
* This also adjusts relid sets found in some expression node types to
- * substitute the contained base rels for any join relid.
+ * substitute the contained base+OJ rels for any join relid.
*
* If a JOIN contains sub-selects that have been flattened, its join alias
* entries might now be arbitrary expressions, not just Vars. This affects
- * this function in one important way: we might find ourselves inserting
- * SubLink expressions into subqueries, and we must make sure that their
- * Query.hasSubLinks fields get set to true if so. If there are any
+ * this function in two important ways. First, we might find ourselves
+ * inserting SubLink expressions into subqueries, and we must make sure that
+ * their Query.hasSubLinks fields get set to true if so. If there are any
* SubLinks in the join alias lists, the outer Query should already have
* hasSubLinks = true, so this is only relevant to un-flattened subqueries.
+ * Second, we have to preserve any varnullingrels info attached to the
+ * alias Vars we're replacing. If the replacement expression is a Var or
+ * PlaceHolderVar or constructed from those, we can just add the
+ * varnullingrels bits to the existing nullingrels field(s); otherwise
+ * we have to add a PlaceHolderVar wrapper.
*
- * NOTE: this is used on not-yet-planned expressions. We do not expect it
- * to be applied directly to the whole Query, so if we see a Query to start
- * with, we do want to increment sublevels_up (this occurs for LATERAL
- * subqueries).
+ * NOTE: this is also used by the parser, to expand join alias Vars before
+ * checking GROUP BY validity. For that use-case, root will be NULL, which
+ * is why we have to pass the Query separately. We need the root itself only
+ * for making PlaceHolderVars. We can avoid making PlaceHolderVars in the
+ * parser's usage because it won't be dealing with arbitrary expressions:
+ * so long as adjust_standard_join_alias_expression can handle everything
+ * the parser would make as a join alias expression, we're OK.
*/
Node *
-flatten_join_alias_vars(Query *query, Node *node)
+flatten_join_alias_vars(PlannerInfo *root, Query *query, Node *node)
{
flatten_join_alias_vars_context context;
+ /*
+ * We do not expect this to be applied to the whole Query, only to
+ * expressions or LATERAL subqueries. Hence, if the top node is a Query,
+ * it's okay to immediately increment sublevels_up.
+ */
+ Assert(node != (Node *) query);
+
+ context.root = root;
context.query = query;
context.sublevels_up = 0;
/* flag whether join aliases could possibly contain SubLinks */
@@ -804,7 +840,9 @@ flatten_join_alias_vars_mutator(Node *node,
rowexpr->colnames = colnames;
rowexpr->location = var->location;
- return (Node *) rowexpr;
+ /* Lastly, add any varnullingrels to the replacement expression */
+ return add_nullingrels_if_needed(context->root, (Node *) rowexpr,
+ var);
}
/* Expand join alias reference */
@@ -831,7 +869,8 @@ flatten_join_alias_vars_mutator(Node *node,
if (context->possible_sublink && !context->inserted_sublink)
context->inserted_sublink = checkExprHasSubLink(newvar);
- return newvar;
+ /* Lastly, add any varnullingrels to the replacement expression */
+ return add_nullingrels_if_needed(context->root, newvar, var);
}
if (IsA(node, PlaceHolderVar))
{
@@ -846,6 +885,7 @@ flatten_join_alias_vars_mutator(Node *node,
{
phv->phrels = alias_relid_set(context->query,
phv->phrels);
+ /* we *don't* change phnullingrels */
}
return (Node *) phv;
}
@@ -879,9 +919,145 @@ flatten_join_alias_vars_mutator(Node *node,
(void *) context);
}
+/*
+ * Add oldvar's varnullingrels, if any, to a flattened join alias expression.
+ * The newnode has been copied, so we can modify it freely.
+ */
+static Node *
+add_nullingrels_if_needed(PlannerInfo *root, Node *newnode, Var *oldvar)
+{
+ if (oldvar->varnullingrels == NULL)
+ return newnode; /* nothing to do */
+ /* If possible, do it by adding to existing nullingrel fields */
+ if (is_standard_join_alias_expression(newnode, oldvar))
+ adjust_standard_join_alias_expression(newnode, oldvar);
+ else if (root)
+ {
+ /* We can insert a PlaceHolderVar to carry the nullingrels */
+ PlaceHolderVar *newphv;
+ Relids phrels = pull_varnos(root, newnode);
+
+ /* XXX what if phrels is empty? */
+ Assert(!bms_is_empty(phrels)); /* probably wrong */
+ newphv = make_placeholder_expr(root, (Expr *) newnode, phrels);
+ /* newphv has zero phlevelsup and NULL phnullingrels; fix it */
+ newphv->phlevelsup = oldvar->varlevelsup;
+ newphv->phnullingrels = bms_copy(oldvar->varnullingrels);
+ newnode = (Node *) newphv;
+ }
+ else
+ {
+ /* ooops, we're missing support for something the parser can make */
+ elog(ERROR, "unsupported join alias expression");
+ }
+ return newnode;
+}
+
+/*
+ * Check to see if we can insert nullingrels into this join alias expression
+ * without use of a separate PlaceHolderVar.
+ *
+ * This will handle Vars, PlaceHolderVars, and implicit-coercion and COALESCE
+ * expressions built from those. This coverage needs to handle anything
+ * that the parser would put into joinaliasvars.
+ * XXX it's probably incomplete at the moment.
+ */
+static bool
+is_standard_join_alias_expression(Node *newnode, Var *oldvar)
+{
+ if (newnode == NULL)
+ return false;
+ if (IsA(newnode, Var) &&
+ ((Var *) newnode)->varlevelsup == oldvar->varlevelsup)
+ return true;
+ else if (IsA(newnode, PlaceHolderVar) &&
+ ((PlaceHolderVar *) newnode)->phlevelsup == oldvar->varlevelsup)
+ return true;
+ else if (IsA(newnode, FuncExpr))
+ {
+ FuncExpr *fexpr = (FuncExpr *) newnode;
+
+ /*
+ * We need to assume that the function wouldn't produce non-NULL from
+ * NULL, which is reasonable for implicit coercions but otherwise not
+ * so much. (Looking at its strictness is likely overkill, and anyway
+ * it would cause us to fail if someone forgot to mark an implicit
+ * coercion as strict.)
+ */
+ if (fexpr->funcformat != COERCE_IMPLICIT_CAST ||
+ fexpr->args == NIL)
+ return false;
+
+ /*
+ * Examine only the first argument --- coercions might have additional
+ * arguments that are constants.
+ */
+ return is_standard_join_alias_expression(linitial(fexpr->args), oldvar);
+ }
+ else if (IsA(newnode, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) newnode;
+ ListCell *lc;
+
+ Assert(cexpr->args != NIL);
+ foreach(lc, cexpr->args)
+ {
+ if (!is_standard_join_alias_expression(lfirst(lc), oldvar))
+ return false;
+ }
+ return true;
+ }
+ else
+ return false;
+}
+
+/*
+ * Insert nullingrels into an expression accepted by
+ * is_standard_join_alias_expression.
+ */
+static void
+adjust_standard_join_alias_expression(Node *newnode, Var *oldvar)
+{
+ if (IsA(newnode, Var) &&
+ ((Var *) newnode)->varlevelsup == oldvar->varlevelsup)
+ {
+ Var *newvar = (Var *) newnode;
+
+ newvar->varnullingrels = bms_add_members(newvar->varnullingrels,
+ oldvar->varnullingrels);
+ }
+ else if (IsA(newnode, PlaceHolderVar) &&
+ ((PlaceHolderVar *) newnode)->phlevelsup == oldvar->varlevelsup)
+ {
+ PlaceHolderVar *newphv = (PlaceHolderVar *) newnode;
+
+ newphv->phnullingrels = bms_add_members(newphv->phnullingrels,
+ oldvar->varnullingrels);
+ }
+ else if (IsA(newnode, FuncExpr))
+ {
+ FuncExpr *fexpr = (FuncExpr *) newnode;
+
+ adjust_standard_join_alias_expression(linitial(fexpr->args), oldvar);
+ }
+ else if (IsA(newnode, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) newnode;
+ ListCell *lc;
+
+ Assert(cexpr->args != NIL);
+ foreach(lc, cexpr->args)
+ {
+ adjust_standard_join_alias_expression(lfirst(lc), oldvar);
+ }
+ }
+ else
+ Assert(false);
+}
+
/*
* alias_relid_set: in a set of RT indexes, replace joins by their
- * underlying base relids
+ * underlying base+OJ relids
*/
static Relids
alias_relid_set(Query *query, Relids relids)
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 9d3c05aed3..aeabc2aca9 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -529,8 +529,8 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
partprunequal = (List *)
adjust_appendrel_attrs_multilevel(root,
(Node *) prunequal,
- subpart->relids,
- targetpart->relids);
+ subpart,
+ targetpart);
}
/*
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index fa1f589fad..e4033b1572 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -2204,7 +2204,7 @@ rowcomparesel(PlannerInfo *root,
else
{
/*
- * Otherwise, it's a join if there's more than one relation used.
+ * Otherwise, it's a join if there's more than one base relation used.
*/
is_join_clause = (NumRelids(root, (Node *) opargs) > 1);
}
diff --git a/src/include/optimizer/appendinfo.h b/src/include/optimizer/appendinfo.h
index fc808dcd27..5e80a741a4 100644
--- a/src/include/optimizer/appendinfo.h
+++ b/src/include/optimizer/appendinfo.h
@@ -23,13 +23,13 @@ extern AppendRelInfo *make_append_rel_info(Relation parentrel,
extern Node *adjust_appendrel_attrs(PlannerInfo *root, Node *node,
int nappinfos, AppendRelInfo **appinfos);
extern Node *adjust_appendrel_attrs_multilevel(PlannerInfo *root, Node *node,
- Relids child_relids,
- Relids top_parent_relids);
+ RelOptInfo *childrel,
+ RelOptInfo *parentrel);
extern Relids adjust_child_relids(Relids relids, int nappinfos,
AppendRelInfo **appinfos);
extern Relids adjust_child_relids_multilevel(PlannerInfo *root, Relids relids,
- Relids child_relids,
- Relids top_parent_relids);
+ RelOptInfo *childrel,
+ RelOptInfo *parentrel);
extern List *adjust_inherited_attnums(List *attnums, AppendRelInfo *context);
extern List *adjust_inherited_attnums_multilevel(PlannerInfo *root,
List *attnums,
diff --git a/src/include/optimizer/optimizer.h b/src/include/optimizer/optimizer.h
index 7be1e5906b..1f5e0b24ca 100644
--- a/src/include/optimizer/optimizer.h
+++ b/src/include/optimizer/optimizer.h
@@ -202,6 +202,6 @@ extern bool contain_var_clause(Node *node);
extern bool contain_vars_of_level(Node *node, int levelsup);
extern int locate_var_of_level(Node *node, int levelsup);
extern List *pull_var_clause(Node *node, int flags);
-extern Node *flatten_join_alias_vars(Query *query, Node *node);
+extern Node *flatten_join_alias_vars(PlannerInfo *root, Query *query, Node *node);
#endif /* OPTIMIZER_H */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index d2d46b15df..2dc4433985 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -302,6 +302,7 @@ extern void expand_planner_arrays(PlannerInfo *root, int add_size);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
+extern RelOptInfo *find_base_rel_ignore_join(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
@@ -333,6 +334,6 @@ extern ParamPathInfo *find_param_path_info(RelOptInfo *rel,
extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
- SpecialJoinInfo *sjinfo, JoinType jointype);
+ SpecialJoinInfo *sjinfo);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/placeholder.h b/src/include/optimizer/placeholder.h
index 39803ea41f..34b118a5c9 100644
--- a/src/include/optimizer/placeholder.h
+++ b/src/include/optimizer/placeholder.h
@@ -27,6 +27,7 @@ extern void update_placeholder_eval_levels(PlannerInfo *root,
extern void fix_placeholder_input_needed_levels(PlannerInfo *root);
extern void add_placeholders_to_base_rels(PlannerInfo *root);
extern void add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *outer_rel, RelOptInfo *inner_rel);
+ RelOptInfo *outer_rel, RelOptInfo *inner_rel,
+ SpecialJoinInfo *sjinfo);
#endif /* PLACEHOLDER_H */
diff --git a/src/include/optimizer/prep.h b/src/include/optimizer/prep.h
index 2b11ff1d1f..ca03f32174 100644
--- a/src/include/optimizer/prep.h
+++ b/src/include/optimizer/prep.h
@@ -29,7 +29,8 @@ extern void pull_up_subqueries(PlannerInfo *root);
extern void flatten_simple_union_all(PlannerInfo *root);
extern void reduce_outer_joins(PlannerInfo *root);
extern void remove_useless_result_rtes(PlannerInfo *root);
-extern Relids get_relids_in_jointree(Node *jtnode, bool include_joins);
+extern Relids get_relids_in_jointree(Node *jtnode, bool include_outer_joins,
+ bool include_inner_joins);
extern Relids get_relids_for_join(Query *query, int joinrelid);
/*
Tom, two quick questions before attempting to read the patch:
Given that views are represented in a parsed representation, does anything need to happen to the Vars inside a view
whenthat view is outer-joined to?
If an outer join is converted to an inner join, must this information get propagated to all the affected Vars,
potentiallyacross query block levels?
"Finnerty, Jim" <jfinnert@amazon.com> writes:
> Given that views are represented in a parsed representation, does anything need to happen to the Vars inside a
viewwhen that view is outer-joined to?
No. The markings only refer to what is in the same Query tree as the Var
itself.
Subquery flattening during planning does deal with this: if we pull up a
subquery (possibly inserted from a view) that was underneath an outer
join, the nullingrel marks on the upper-level Vars referring to subquery
outputs will get merged into what is pulled up, either by unioning the
varnullingrel bitmaps if what is pulled up is just a Var, or if what is
pulled up isn't a Var, by wrapping it in a PlaceHolderVar that carries
the old outer Var's markings. We had essentially this same behavior
with PlaceHolderVars before, but I think this way makes it a lot more
principled and intelligible (and I suspect there are now cases where we
manage to avoid inserting unnecessary PlaceHolderVars that the old code
couldn't avoid).
> If an outer join is converted to an inner join, must this information get propagated to all the affected Vars,
potentiallyacross query block levels?
Yes. The code is there in the patch to run around and remove nullingrel
bits from affected Vars.
One thing that doesn't happen (and didn't before, so this is not a
regression) is that if we strength-reduce a FULL JOIN USING to an outer
or plain join, it'd be nice if the "COALESCE" hack we represent the
merged USING column with could be replaced with the same lower-relation
Var that the parser would have used if the join weren't FULL to begin
with. Without that, we're leaving optimization opportunities on the
table. I'm hesitant to try to do that though as long as the COALESCE
structures look exactly like something a user could write. It'd be
safer if we used some bespoke node structure for this purpose ...
but nobody's bothered to invent that.
regards, tom lane
On Sat, Jul 2, 2022 at 12:42 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Anyway, even though this is far from done, I'm pretty pleased with
the results so far, so I thought I'd put it out for review by
anyone who cares to take a look. I'll add it to the September CF
in hopes that it might be more or less finished by then, and so
that the cfbot will check it out.
Thanks for the work! I have a question about qual clause placement.
For the query in the example
SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE foo(t2.z)
(foo() is not strict.) We want to avoid pushing foo(t2.z) down to the t2
scan level. Previously we do that with check_outerjoin_delay() by
scanning all the outer joins below and check if the qual references any
nullable rels of the OJ, and if so include the OJ's rels into the qual.
So as a result we'd get that foo(t2.z) is referencing t1 and t2, and
we'd put the qual into the join lists of t1 and t2.
Now there is the 'varnullingrels' marker in the t2.z, which is the LEFT
JOIN below (with RTI 3). So we consider the qual is referencing RTE 2
(which is t2) and RTE 3 (which is the OJ). Do we still need to include
RTE 1, i.e. t1 into the qual's required relids? How should we do that?
Thanks
Richard
For the query in the example
SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE foo(t2.z)
(foo() is not strict.) We want to avoid pushing foo(t2.z) down to the t2
scan level. Previously we do that with check_outerjoin_delay() by
scanning all the outer joins below and check if the qual references any
nullable rels of the OJ, and if so include the OJ's rels into the qual.
So as a result we'd get that foo(t2.z) is referencing t1 and t2, and
we'd put the qual into the join lists of t1 and t2.
Now there is the 'varnullingrels' marker in the t2.z, which is the LEFT
JOIN below (with RTI 3). So we consider the qual is referencing RTE 2
(which is t2) and RTE 3 (which is the OJ). Do we still need to include
RTE 1, i.e. t1 into the qual's required relids? How should we do that?
Thanks
Richard
Richard Guo <guofenglinux@gmail.com> writes:
> For the query in the example
> SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE foo(t2.z)
> (foo() is not strict.) We want to avoid pushing foo(t2.z) down to the t2
> scan level. Previously we do that with check_outerjoin_delay() by
> scanning all the outer joins below and check if the qual references any
> nullable rels of the OJ, and if so include the OJ's rels into the qual.
> So as a result we'd get that foo(t2.z) is referencing t1 and t2, and
> we'd put the qual into the join lists of t1 and t2.
> Now there is the 'varnullingrels' marker in the t2.z, which is the LEFT
> JOIN below (with RTI 3). So we consider the qual is referencing RTE 2
> (which is t2) and RTE 3 (which is the OJ). Do we still need to include
> RTE 1, i.e. t1 into the qual's required relids? How should we do that?
It seems likely to me that we could leave the qual's required_relids
as just {2,3} and not have to bother ORing any additional bits into
that. However, in the case of a Var-free JOIN/ON clause it'd still
be necessary to artificially add some relids to its initially empty
relids. Since I've not yet tried to rewrite distribute_qual_to_rels
I'm not sure how the details will shake out.
regards, tom lane
Here's v2 of this patch series. It's functionally identical to v1,
but I've rebased it over the recent auto-node-support-generation
changes, and also extracted a few separable bits in hopes of making
the main planner patch smaller. (It's still pretty durn large,
unfortunately.) Unlike the original submission, each step will
compile on its own, though the intermediate states mostly don't
pass all regression tests.
regards, tom lane
commit 59846b51f350064b05e4a7bf643c4ce73b7f629a
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sun Jul 10 11:16:38 2022 -0400
Add overview documentation.
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index 41c120e0cd..2b30d22aed 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -295,6 +295,191 @@ Therefore, we don't merge FROM-lists if the result would have too many
FROM-items in one list.
+Vars and PlaceHolderVars
+------------------------
+
+A Var node is simply the parse-tree representation of a table column
+reference. However, in the presence of outer joins, that concept is
+more subtle than it might seem. We need to distinguish the values of
+a Var "above" and "below" any outer join that could force the Var to
+null. As an example, consider
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE foo(t2.z)
+
+(Assume foo() is not strict, so that we can't reduce the left join to
+a plain join.) A naive implementation might try to push the foo(t2.z)
+call down to the scan of t2, but that is not correct because
+(a) what foo() should actually see for a null-extended join row is NULL,
+and (b) if foo() returns false, we should suppress the t1 row from the
+join altogether, not emit it with a null-extended t2 row. On the other
+hand, it *would* be correct (and desirable) to push the call down to
+the scan level if the query were
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y AND foo(t2.z))
+
+This motivates considering "t2.z" within the left join's ON clause
+to be a different value from "t2.z" outside the JOIN clause. The
+former can be identified with t2.z as seen at the relation scan level,
+but the latter can't.
+
+Another example occurs in connection with EquivalenceClasses (discussed
+below). Given
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE t1.x = 42
+
+we would like to put t1.x and t2.y and 42 into the same EquivalenceClass
+and then derive "t2.y = 42" to use as a restriction clause for the scan
+of t2. However, it'd be wrong to conclude that t2.y will always have
+the value 42, or that it's equal to t1.x in every joined row. We can
+solve this problem by deeming that "t2.y" in the ON clause refers to
+the relation-scan-level value of t2.y, but not to the value that y will
+have in joined rows, where it might be NULL rather than equal to t1.x.
+
+Therefore, Var nodes are decorated with "varnullingrels", which are sets
+of the rangetable indexes of outer joins that potentially null this Var
+at the point where it appears in the query. (Using a set, not an
+ordered list, is fine since it doesn't matter which join forced the
+value to null; and that avoids having to change the representation when
+we consider different outer-join orders.) In the examples above, all
+occurrences of t1.x would have empty varnullingrels, since the left join
+doesn't null t1. The t2 references within the JOIN ON clauses would
+also have empty varnullingrels, but other references to t2 columns would
+be labeled with the index of the JOIN's rangetable entry (RTE), so that
+they'd be understood as potentially different from the t2 values seen at
+scan level. Labeling t2.z in the WHERE clause with the JOIN's RT index
+lets us recognize that that occurrence of foo(t2.z) cannot be pushed
+down to the t2 scan level: we cannot evaluate that value at the scan
+level, but only after the join has been done.
+
+For LEFT and RIGHT outer joins, only Vars coming from the nullable side
+of the join are marked with that join's RT index. For FULL joins, all
+Vars are marked. (Such marking doesn't let us tell which side of the
+full join a Var came from; but that information can be found elsewhere
+at need.)
+
+Notionally, a Var having nonempty varnullingrels can be thought of as
+ CASE WHEN any-of-these-outer-joins-produced-a-null-extended-row
+ THEN NULL
+ ELSE the-scan-level-value-of-the-column
+ END
+It's only notional, because no such calculation is ever done explicitly.
+In a finished plan, Vars occurring in scan-level plan nodes represent
+the actual table column values, but upper-level Vars are always
+references to outputs of lower-level plan nodes. When a join node emits
+a null-extended row, it just returns nulls for the relevant output
+columns rather than copying up values from its input. Because we don't
+ever have to do this calculation explicitly, it's not necessary to
+distinguish which side of an outer join got null-extended, which'd
+otherwise be essential information for FULL JOIN cases.
+
+Outer join identity 3 (discussed above) complicates this picture
+a bit. In the form
+ A leftjoin (B leftjoin C on (Pbc)) on (Pab)
+all of the Vars in clauses Pbc and Pab will have empty varnullingrels,
+but if we start with
+ (A leftjoin B on (Pab)) leftjoin C on (Pbc)
+then the parser will have marked Pbc's B Vars with the A/B join's
+RT index, making this form artificially different from the first.
+We resolve this by, after noting that Pbc is strict, running
+through that clause and removing any varnullingrels references to
+left joins in the lefthand side. That makes the clause equivalent
+to what it would have looked like if the first form were presented,
+so that we can freely consider both join orders. However, because
+we have done this, if we do construct a plan based on the second
+join order then we cannot cross-check that B Vars appearing above
+the A/B join are all marked with that join's RT index. That would
+be a useful cross-check to have to catch planner bugs, but it
+doesn't seem useful enough to justify the extra complication of
+devising a representation that would support it.
+
+Outer joins also complicate handling of subquery pull-up. Consider
+
+ SELECT ..., ss.x FROM tab1
+ LEFT JOIN (SELECT *, 42 AS x FROM tab2) ss ON ...
+
+We want to be able to pull up the subquery as discussed previously,
+but we can't just replace the "ss.x" Var in the top-level SELECT list
+with the constant 42. That'd result in always emitting 42, rather
+than emitting NULL in null-extended join rows.
+
+To solve this, we introduce the concept of PlaceHolderVars.
+A PlaceHolderVar is somewhat like a Var, in that its value originates
+at a relation scan level and can then be forced to null by higher-level
+outer joins; hence PlaceHolderVars carry a set of nulling rel IDs just
+like Vars. Unlike a Var, whose original value comes from a table,
+a PlaceHolderVar's original value is defined by a query-determined
+expression ("42" in this example); so we represent the PlaceHolderVar
+as a node with that expression as child. We insert a PlaceHolderVar
+whenever subquery pullup needs to replace a subquery-referencing Var
+that has nonempty varnullingrels with an expression that is not simply a
+Var. (When the replacement expression is a pulled-up Var, we can just
+add the replaced Var's varnullingrels to its set. Also, if the replaced
+Var has empty varnullingrels, we don't need a PlaceHolderVar: there is
+nothing that'd force the value to null, so the pulled-up expression is
+fine to use as-is.) In a finished plan, a PlaceHolderVar becomes just
+the contained expression at whatever plan level it's supposed to be
+evaluated at, and then upper-level occurrences are replaced by
+references to that output column of the lower plan level. That causes
+the value to go to null when appropriate at an outer join, in the same
+way as for Vars. Thus, PlaceHolderVars are never seen outside the
+planner.
+
+PlaceHolderVars (PHVs) are more complicated than Vars in another way:
+their original value might need to be calculated at a join, not a
+base-level relation scan. This can happen if a pulled-up subquery
+contains a join. Because of this, a PHV can create a join order
+constraint that wouldn't otherwise exist, to ensure that it can
+be calculated before it is used. A PHV's expression can also contain
+LATERAL references, adding complications that are discussed below.
+
+
+Relation Identification and Qual Clause Placement
+-------------------------------------------------
+
+A qual clause obtained from WHERE or JOIN/ON can be enforced at the lowest
+scan or join level that includes all relations used in the clause. For
+this purpose we consider that outer joins listed in varnullingrels or
+phnullingrels are used in the clause, since we can't compute the qual's
+result correctly until we know whether such Vars have gone to null.
+
+The one exception to this general rule is that a non-degenerate outer
+JOIN/ON qual (one that references the non-nullable side of the join)
+cannot be enforced below that join, even if it doesn't reference the
+nullable side. Pushing it down into the non-nullable side would result
+in rows disappearing from the join's result, rather than appearing as
+null-extended rows. To handle that, when we identify such a qual we
+artificially add the join's minimum input relid set to the set of
+relations it is considered to use, forcing it to be evaluated exactly at
+that join level. The same happens for outer-join quals that mention no
+relations at all.
+
+When attaching a qual clause to a join plan node that is performing
+an outer join, the qual clause is considered a "join clause" (that
+is, it is applied before the join) if it does not use that specific
+outer join, or a "filter clause" (applied after the join) if it does
+use that outer join.
+
+These things lead us to identify join relations within the planner
+by the sets of base relation RT indexes plus outer join RT indexes
+that they include. In that way, the sets of relations used by qual
+clauses can be directly compared to join relations' relid sets to
+see where to place the clauses. These identifying sets are unique
+because, for any given collection of base relations, there is only
+one valid set of outer joins to have performed along the way to
+joining that set of base relations (although the order of applying
+them could vary, as discussed above).
+
+SEMI joins do not have RT indexes, because they are artifacts made by
+the planner rather than the parser. (We could create rangetable
+entries for them, but there seems no need at present.) This does not
+cause a problem for qual placement, because the nullable side of a
+semijoin is not referenceable from above the join, so there is never a
+need to cite it in varnullingrels or phnullingrels. It does not cause
+a problem for join relation identification either, since again whether
+a semijoin has been completed is implicit in the set of base relations
+included in the join.
+
+
Optimizer Functions
-------------------
@@ -437,11 +622,10 @@ inputs.
EquivalenceClasses
------------------
-During the deconstruct_jointree() scan of the query's qual clauses, we look
-for mergejoinable equality clauses A = B whose applicability is not delayed
-by an outer join; these are called "equivalence clauses". When we find
-one, we create an EquivalenceClass containing the expressions A and B to
-record this knowledge. If we later find another equivalence clause B = C,
+During the deconstruct_jointree() scan of the query's qual clauses, we
+look for mergejoinable equality clauses A = B. When we find one, we
+create an EquivalenceClass containing the expressions A and B to record
+that they are equal. If we later find another equivalence clause B = C,
we add C to the existing EquivalenceClass for {A B}; this may require
merging two existing EquivalenceClasses. At the end of the scan, we have
sets of values that are known all transitively equal to each other. We can
@@ -473,15 +657,26 @@ asserts that at any plan node where more than one of its member values
can be computed, output rows in which the values are not all equal may
be discarded without affecting the query result. (We require all levels
of the plan to enforce EquivalenceClasses, hence a join need not recheck
-equality of values that were computable by one of its children.) For an
-ordinary EquivalenceClass that is "valid everywhere", we can further infer
-that the values are all non-null, because all mergejoinable operators are
-strict. However, we also allow equivalence clauses that appear below the
-nullable side of an outer join to form EquivalenceClasses; for these
-classes, the interpretation is that either all the values are equal, or
-all (except pseudo-constants) have gone to null. (This requires a
-limitation that non-constant members be strict, else they might not go
-to null when the other members do.) Consider for example
+equality of values that were computable by one of its children.)
+
+We can further infer that the values are all non-null, because all
+mergejoinable operators are strict. This is a little tricky in the
+presence of outer joins. Consider
+
+ SELECT *
+ FROM a LEFT JOIN
+ (SELECT * FROM b LEFT JOIN c ON b.y = c.z WHERE b.y = 10) ss
+ ON a.x = ss.y
+ WHERE a.x = 42;
+
+We can form the EquivalenceClass {b.y c.z 10} and thereby apply c.z = 10
+while scanning c. However it would be incorrect to conclude that a.x
+is also a member of that EquivalenceClass. Instead, we form a second
+EquivalenceClass {a.x ss.y 42}, where (as discussed earlier) ss.y
+references the same table column as b.y but has a different
+varnullingrels label and is therefore considered a distinct Var.
+
+If the lower join were INNER:
SELECT *
FROM a LEFT JOIN
@@ -489,40 +684,23 @@ to null when the other members do.) Consider for example
ON a.x = ss.y
WHERE a.x = 42;
-We can form the below-outer-join EquivalenceClass {b.y c.z 10} and thereby
-apply c.z = 10 while scanning c. (The reason we disallow outerjoin-delayed
-clauses from forming EquivalenceClasses is exactly that we want to be able
-to push any derived clauses as far down as possible.) But once above the
-outer join it's no longer necessarily the case that b.y = 10, and thus we
-cannot use such EquivalenceClasses to conclude that sorting is unnecessary
-(see discussion of PathKeys below).
-
-In this example, notice also that a.x = ss.y (really a.x = b.y) is not an
-equivalence clause because its applicability to b is delayed by the outer
-join; thus we do not try to insert b.y into the equivalence class {a.x 42}.
-But since we see that a.x has been equated to 42 above the outer join, we
-are able to form a below-outer-join class {b.y 42}; this restriction can be
-added because no b/c row not having b.y = 42 can contribute to the result
-of the outer join, and so we need not compute such rows. Now this class
-will get merged with {b.y c.z 10}, leading to the contradiction 10 = 42,
-which lets the planner deduce that the b/c join need not be computed at all
-because none of its rows can contribute to the outer join. (This gets
-implemented as a gating Result filter, since more usually the potential
-contradiction involves Param values rather than just Consts, and thus has
-to be checked at runtime.)
+then ss.y is not any different from b.y and we'd end up with the
+EquivalenceClass {a.x b.y c.z 10 42}. This leads to the contradiction
+10 = 42, which lets the planner deduce that the b/c join need not be
+computed at all because none of its rows can contribute to the outer
+join. (This gets implemented as a gating Result filter, since more
+usually the potential contradiction involves Param values rather than
+just Consts, and thus has to be checked at runtime.)
To aid in determining the sort ordering(s) that can work with a mergejoin,
we mark each mergejoinable clause with the EquivalenceClasses of its left
-and right inputs. For an equivalence clause, these are of course the same
-EquivalenceClass. For a non-equivalence mergejoinable clause (such as an
-outer-join qualification), we generate two separate EquivalenceClasses for
-the left and right inputs. This may result in creating single-item
-equivalence "classes", though of course these are still subject to merging
-if other equivalence clauses are later found to bear on the same
-expressions.
-
-Another way that we may form a single-item EquivalenceClass is in creation
-of a PathKey to represent a desired sort order (see below). This is a bit
+and right inputs. (These are in fact always the same EquivalenceClass.)
+
+In some cases we will form single-item EquivalenceClasses. This happens
+if an ORDER BY or GROUP BY key is not mentioned in any equivalence
+clause. We need to reason about sort orders in such queries, and our
+representation of sort ordering is a PathKey (see below) which uses an
+EquivalenceClass, so we have to make an EquivalenceClass. This is a bit
different from the above cases because such an EquivalenceClass might
contain an aggregate function or volatile expression. (A clause containing
a volatile function will never be considered mergejoinable, even if its top
@@ -579,7 +757,7 @@ Index scans have Path.pathkeys that represent the chosen index's ordering,
if any. A single-key index would create a single-PathKey list, while a
multi-column index generates a list with one element per key index column.
Non-key columns specified in the INCLUDE clause of covering indexes don't
-have corresponding PathKeys in the list, because the have no influence on
+have corresponding PathKeys in the list, because they have no influence on
index ordering. (Actually, since an index can be scanned either forward or
backward, there are two possible sort orders and two possible PathKey lists
it can generate.)
@@ -655,14 +833,9 @@ redundancy, we save time and improve planning, since the planner will more
easily recognize equivalent orderings as being equivalent.
Another interesting property is that if the underlying EquivalenceClass
-contains a constant and is not below an outer join, then the pathkey is
-completely redundant and need not be sorted by at all! Every row must
-contain the same constant value, so there's no need to sort. (If the EC is
-below an outer join, we still have to sort, since some of the rows might
-have gone to null and others not. In this case we must be careful to pick
-a non-const member to sort by. The assumption that all the non-const
-members go to null at the same plan level is critical here, else they might
-not produce the same sort order.) This might seem pointless because users
+contains a constant, then the pathkey is completely redundant and need
+not be sorted by at all! Every row must contain the same value, so
+there's no need to sort. This might seem pointless because users
are unlikely to write "... WHERE x = 42 ORDER BY x", but it allows us to
recognize when particular index columns are irrelevant to the sort order:
if we have "... WHERE x = 42 ORDER BY y", scanning an index on (x,y)
@@ -670,15 +843,6 @@ produces correctly ordered data without a sort step. We used to have very
ugly ad-hoc code to recognize that in limited contexts, but discarding
constant ECs from pathkeys makes it happen cleanly and automatically.
-You might object that a below-outer-join EquivalenceClass doesn't always
-represent the same values at every level of the join tree, and so using
-it to uniquely identify a sort order is dubious. This is true, but we
-can avoid dealing with the fact explicitly because we always consider that
-an outer join destroys any ordering of its nullable inputs. Thus, even
-if a path was sorted by {a.x} below an outer join, we'll re-sort if that
-sort ordering was important; and so using the same PathKey for both sort
-orderings doesn't create any real problem.
-
Order of processing for EquivalenceClasses and PathKeys
-------------------------------------------------------
commit 7ae3b66331c268862ef9058fb7ca5419949efbd9
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sun Jul 10 12:04:07 2022 -0400
Improve performance of adjust_appendrel_attrs_multilevel.
The present implementations of adjust_appendrel_attrs_multilevel and
its sibling adjust_child_relids_multilevel are very messy, because
they work by reconstructing the relids of the child's immediate
parent and then seeing if that's bms_equal to the relids of the
target parent. Aside from being quite inefficient, this will not
work for joinrels whose relids contain outer-join relids in addition
to baserels.
The whole thing can be solved at a stroke by adding explicit parent
and top_parent links to child RelOptInfos, and making these functions
work with RelOptInfo pointers instead of relids. Doing that is
simpler for most callers, too.
In my original version of this patch, I got rid of
RelOptInfo.top_parent_relids on the grounds that it was now redundant.
However, that adds a lot of code churn in places that otherwise would
not need changing, and arguably the extra indirection needed to fetch
top_parent->relids in those places costs something. So this version
leaves that field in place.
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 60c0e3f108..f8a97622b1 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -1760,8 +1760,8 @@ generate_join_implied_equalities_broken(PlannerInfo *root,
if (IS_OTHER_REL(inner_rel) && result != NIL)
result = (List *) adjust_appendrel_attrs_multilevel(root,
(Node *) result,
- inner_rel->relids,
- inner_rel->top_parent_relids);
+ inner_rel,
+ inner_rel->top_parent);
return result;
}
@@ -2626,8 +2626,8 @@ add_child_rel_equivalences(PlannerInfo *root,
child_expr = (Expr *)
adjust_appendrel_attrs_multilevel(root,
(Node *) cur_em->em_expr,
- child_relids,
- top_parent_relids);
+ child_rel,
+ child_rel->top_parent);
}
/*
@@ -2768,8 +2768,8 @@ add_child_join_rel_equivalences(PlannerInfo *root,
child_expr = (Expr *)
adjust_appendrel_attrs_multilevel(root,
(Node *) cur_em->em_expr,
- child_relids,
- top_parent_relids);
+ child_joinrel,
+ child_joinrel->top_parent);
}
/*
@@ -2791,8 +2791,8 @@ add_child_join_rel_equivalences(PlannerInfo *root,
new_nullable_relids =
adjust_child_relids_multilevel(root,
new_nullable_relids,
- child_relids,
- top_parent_relids);
+ child_joinrel,
+ child_joinrel->top_parent);
(void) add_eq_member(cur_ec, child_expr,
new_relids, new_nullable_relids,
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 06ad856eac..f6baa2a765 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -1791,8 +1791,8 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
withCheckOptions = (List *)
adjust_appendrel_attrs_multilevel(root,
(Node *) withCheckOptions,
- this_result_rel->relids,
- top_result_rel->relids);
+ this_result_rel,
+ top_result_rel);
withCheckOptionLists = lappend(withCheckOptionLists,
withCheckOptions);
}
@@ -1804,8 +1804,8 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
returningList = (List *)
adjust_appendrel_attrs_multilevel(root,
(Node *) returningList,
- this_result_rel->relids,
- top_result_rel->relids);
+ this_result_rel,
+ top_result_rel);
returningLists = lappend(returningLists,
returningList);
}
@@ -1826,13 +1826,13 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
leaf_action->qual =
adjust_appendrel_attrs_multilevel(root,
(Node *) action->qual,
- this_result_rel->relids,
- top_result_rel->relids);
+ this_result_rel,
+ top_result_rel);
leaf_action->targetList = (List *)
adjust_appendrel_attrs_multilevel(root,
(Node *) action->targetList,
- this_result_rel->relids,
- top_result_rel->relids);
+ this_result_rel,
+ top_result_rel);
if (leaf_action->commandType == CMD_UPDATE)
leaf_action->updateColnos =
adjust_inherited_attnums_multilevel(root,
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index 9d4bb47027..62cccf9d87 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -479,39 +479,34 @@ adjust_appendrel_attrs_mutator(Node *node,
/*
* adjust_appendrel_attrs_multilevel
- * Apply Var translations from a toplevel appendrel parent down to a child.
+ * Apply Var translations from an appendrel parent down to a child.
*
- * In some cases we need to translate expressions referencing a parent relation
- * to reference an appendrel child that's multiple levels removed from it.
+ * Replace Vars in the "node" expression that reference "parentrel" with
+ * the appropriate Vars for "childrel". childrel can be more than one
+ * inheritance level removed from parentrel.
*/
Node *
adjust_appendrel_attrs_multilevel(PlannerInfo *root, Node *node,
- Relids child_relids,
- Relids top_parent_relids)
+ RelOptInfo *childrel,
+ RelOptInfo *parentrel)
{
AppendRelInfo **appinfos;
- Bitmapset *parent_relids = NULL;
int nappinfos;
- int cnt;
-
- Assert(bms_num_members(child_relids) == bms_num_members(top_parent_relids));
-
- appinfos = find_appinfos_by_relids(root, child_relids, &nappinfos);
- /* Construct relids set for the immediate parent of given child. */
- for (cnt = 0; cnt < nappinfos; cnt++)
+ /* Recurse if immediate parent is not the top parent. */
+ if (childrel->parent != parentrel)
{
- AppendRelInfo *appinfo = appinfos[cnt];
-
- parent_relids = bms_add_member(parent_relids, appinfo->parent_relid);
+ if (childrel->parent)
+ node = adjust_appendrel_attrs_multilevel(root, node,
+ childrel->parent,
+ parentrel);
+ else
+ elog(ERROR, "childrel is not a child of parentrel");
}
- /* Recurse if immediate parent is not the top parent. */
- if (!bms_equal(parent_relids, top_parent_relids))
- node = adjust_appendrel_attrs_multilevel(root, node, parent_relids,
- top_parent_relids);
+ /* Now translate for this child. */
+ appinfos = find_appinfos_by_relids(root, childrel->relids, &nappinfos);
- /* Now translate for this child */
node = adjust_appendrel_attrs(root, node, nappinfos, appinfos);
pfree(appinfos);
@@ -554,56 +549,43 @@ adjust_child_relids(Relids relids, int nappinfos, AppendRelInfo **appinfos)
}
/*
- * Replace any relid present in top_parent_relids with its child in
- * child_relids. Members of child_relids can be multiple levels below top
- * parent in the partition hierarchy.
+ * Substitute child's relids for parent's relids in a Relid set.
+ * The childrel can be multiple inheritance levels below the parent.
*/
Relids
adjust_child_relids_multilevel(PlannerInfo *root, Relids relids,
- Relids child_relids, Relids top_parent_relids)
+ RelOptInfo *childrel,
+ RelOptInfo *parentrel)
{
AppendRelInfo **appinfos;
int nappinfos;
- Relids parent_relids = NULL;
- Relids result;
- Relids tmp_result = NULL;
- int cnt;
/*
- * If the given relids set doesn't contain any of the top parent relids,
- * it will remain unchanged.
+ * If the given relids set doesn't contain any of the parent relids, it
+ * will remain unchanged.
*/
- if (!bms_overlap(relids, top_parent_relids))
+ if (!bms_overlap(relids, parentrel->relids))
return relids;
- appinfos = find_appinfos_by_relids(root, child_relids, &nappinfos);
-
- /* Construct relids set for the immediate parent of the given child. */
- for (cnt = 0; cnt < nappinfos; cnt++)
- {
- AppendRelInfo *appinfo = appinfos[cnt];
-
- parent_relids = bms_add_member(parent_relids, appinfo->parent_relid);
- }
-
/* Recurse if immediate parent is not the top parent. */
- if (!bms_equal(parent_relids, top_parent_relids))
+ if (childrel->parent != parentrel)
{
- tmp_result = adjust_child_relids_multilevel(root, relids,
- parent_relids,
- top_parent_relids);
- relids = tmp_result;
+ if (childrel->parent)
+ relids = adjust_child_relids_multilevel(root, relids,
+ childrel->parent,
+ parentrel);
+ else
+ elog(ERROR, "childrel is not a child of parentrel");
}
- result = adjust_child_relids(relids, nappinfos, appinfos);
+ /* Now translate for this child. */
+ appinfos = find_appinfos_by_relids(root, childrel->relids, &nappinfos);
+
+ relids = adjust_child_relids(relids, nappinfos, appinfos);
- /* Free memory consumed by any intermediate result. */
- if (tmp_result)
- bms_free(tmp_result);
- bms_free(parent_relids);
pfree(appinfos);
- return result;
+ return relids;
}
/*
@@ -694,8 +676,8 @@ get_translated_update_targetlist(PlannerInfo *root, Index relid,
*processed_tlist = (List *)
adjust_appendrel_attrs_multilevel(root,
(Node *) root->processed_tlist,
- bms_make_singleton(relid),
- bms_make_singleton(root->parse->resultRelation));
+ find_base_rel(root, relid),
+ find_base_rel(root, root->parse->resultRelation));
if (update_colnos)
*update_colnos =
adjust_inherited_attnums_multilevel(root, root->update_colnos,
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 483c4f4137..833b440f39 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3994,8 +3994,8 @@ reparameterize_path_by_child(PlannerInfo *root, Path *path,
#define ADJUST_CHILD_ATTRS(node) \
((node) = \
(List *) adjust_appendrel_attrs_multilevel(root, (Node *) (node), \
- child_rel->relids, \
- child_rel->top_parent_relids))
+ child_rel, \
+ child_rel->top_parent))
#define REPARAMETERIZE_CHILD_PATH(path) \
do { \
@@ -4212,8 +4212,8 @@ do { \
old_ppi = new_path->param_info;
required_outer =
adjust_child_relids_multilevel(root, old_ppi->ppi_req_outer,
- child_rel->relids,
- child_rel->top_parent_relids);
+ child_rel,
+ child_rel->top_parent);
/* If we already have a PPI for this parameterization, just return it */
new_ppi = find_param_path_info(new_path->parent, required_outer);
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 520409f4ba..a163853bed 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -265,14 +265,10 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
*/
if (parent)
{
- /*
- * Each direct or indirect child wants to know the relids of its
- * topmost parent.
- */
- if (parent->top_parent_relids)
- rel->top_parent_relids = parent->top_parent_relids;
- else
- rel->top_parent_relids = bms_copy(parent->relids);
+ /* We keep back-links to immediate parent and topmost parent. */
+ rel->parent = parent;
+ rel->top_parent = parent->top_parent ? parent->top_parent : parent;
+ rel->top_parent_relids = rel->top_parent->relids;
/*
* Also propagate lateral-reference information from appendrel parent
@@ -294,6 +290,8 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
}
else
{
+ rel->parent = NULL;
+ rel->top_parent = NULL;
rel->top_parent_relids = NULL;
rel->direct_lateral_relids = NULL;
rel->lateral_relids = NULL;
@@ -663,6 +661,8 @@ build_join_rel(PlannerInfo *root,
joinrel->joininfo = NIL;
joinrel->has_eclass_joins = false;
joinrel->consider_partitionwise_join = false; /* might get changed later */
+ joinrel->parent = NULL;
+ joinrel->top_parent = NULL;
joinrel->top_parent_relids = NULL;
joinrel->part_scheme = NULL;
joinrel->nparts = -1;
@@ -842,7 +842,9 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->joininfo = NIL;
joinrel->has_eclass_joins = false;
joinrel->consider_partitionwise_join = false; /* might get changed later */
- joinrel->top_parent_relids = NULL;
+ joinrel->parent = parent_joinrel;
+ joinrel->top_parent = parent_joinrel->top_parent ? parent_joinrel->top_parent : parent_joinrel;
+ joinrel->top_parent_relids = joinrel->top_parent->relids;
joinrel->part_scheme = NULL;
joinrel->nparts = -1;
joinrel->boundinfo = NULL;
@@ -854,9 +856,6 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->partexprs = NULL;
joinrel->nullable_partexprs = NULL;
- joinrel->top_parent_relids = bms_union(outer_rel->top_parent_relids,
- inner_rel->top_parent_relids);
-
/* Compute information relevant to foreign relations. */
set_foreign_rel_properties(joinrel, outer_rel, inner_rel);
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 9d3c05aed3..aeabc2aca9 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -529,8 +529,8 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
partprunequal = (List *)
adjust_appendrel_attrs_multilevel(root,
(Node *) prunequal,
- subpart->relids,
- targetpart->relids);
+ subpart,
+ targetpart);
}
/*
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 44ffc73f15..ee8bed6452 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -915,7 +915,15 @@ typedef struct RelOptInfo
*/
/* consider partitionwise join paths? (if partitioned rel) */
bool consider_partitionwise_join;
- /* Relids of topmost parents (if "other" rel) */
+
+ /*
+ * inheritance links, if this is an otherrel (otherwise NULL):
+ */
+ /* Immediate parent relation (dumping it would be too verbose) */
+ struct RelOptInfo *parent pg_node_attr(read_write_ignore);
+ /* Topmost parent relation (dumping it would be too verbose) */
+ struct RelOptInfo *top_parent pg_node_attr(read_write_ignore);
+ /* Relids of topmost parent (redundant, but handy) */
Relids top_parent_relids;
/*
diff --git a/src/include/optimizer/appendinfo.h b/src/include/optimizer/appendinfo.h
index fc808dcd27..5e80a741a4 100644
--- a/src/include/optimizer/appendinfo.h
+++ b/src/include/optimizer/appendinfo.h
@@ -23,13 +23,13 @@ extern AppendRelInfo *make_append_rel_info(Relation parentrel,
extern Node *adjust_appendrel_attrs(PlannerInfo *root, Node *node,
int nappinfos, AppendRelInfo **appinfos);
extern Node *adjust_appendrel_attrs_multilevel(PlannerInfo *root, Node *node,
- Relids child_relids,
- Relids top_parent_relids);
+ RelOptInfo *childrel,
+ RelOptInfo *parentrel);
extern Relids adjust_child_relids(Relids relids, int nappinfos,
AppendRelInfo **appinfos);
extern Relids adjust_child_relids_multilevel(PlannerInfo *root, Relids relids,
- Relids child_relids,
- Relids top_parent_relids);
+ RelOptInfo *childrel,
+ RelOptInfo *parentrel);
extern List *adjust_inherited_attnums(List *attnums, AppendRelInfo *context);
extern List *adjust_inherited_attnums_multilevel(PlannerInfo *root,
List *attnums,
commit bd58b68da9d15c0e2cce86a6a837302a8a764598
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sun Jul 10 13:26:16 2022 -0400
Add Var.varnullingrels and PlaceHolderVar.phnullingrels fields.
These fields are always empty as of this commit, so they don't
affect any behavior, even though equal() will compare them.
Update backend/nodes/ and backend/rewrite/ infrastructure as needed.
Also add some rewrite functions we'll need later.
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index 28288dcfc1..19606c495f 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -81,11 +81,13 @@ makeVar(int varno,
var->varlevelsup = varlevelsup;
/*
- * Only a few callers need to make Var nodes with varnosyn/varattnosyn
- * different from varno/varattno. We don't provide separate arguments for
- * them, but just initialize them to the given varno/varattno. This
- * reduces code clutter and chance of error for most callers.
+ * Only a few callers need to make Var nodes with non-null varnullingrels,
+ * or with varnosyn/varattnosyn different from varno/varattno. We don't
+ * provide separate arguments for them, but just initialize them to NULL
+ * and the given varno/varattno. This reduces code clutter and chance of
+ * error for most callers.
*/
+ var->varnullingrels = NULL;
var->varnosyn = (Index) varno;
var->varattnosyn = varattno;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 4cb1744da6..ccf63515fa 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -2847,6 +2847,7 @@ expression_tree_mutator(Node *node,
Var *newnode;
FLATCOPY(newnode, var, Var);
+ /* Assume we need not copy the varnullingrels bitmapset */
return (Node *) newnode;
}
break;
@@ -3442,7 +3443,7 @@ expression_tree_mutator(Node *node,
FLATCOPY(newnode, phv, PlaceHolderVar);
MUTATE(newnode->phexpr, phv->phexpr, Expr *);
- /* Assume we need not copy the relids bitmapset */
+ /* Assume we need not copy the relids bitmapsets */
return (Node *) newnode;
}
break;
diff --git a/src/backend/rewrite/rewriteManip.c b/src/backend/rewrite/rewriteManip.c
index 101c39553a..a0a0026469 100644
--- a/src/backend/rewrite/rewriteManip.c
+++ b/src/backend/rewrite/rewriteManip.c
@@ -40,6 +40,13 @@ typedef struct
int win_location;
} locate_windowfunc_context;
+typedef struct
+{
+ Bitmapset *removable_relids;
+ Bitmapset *except_relids;
+ int sublevels_up;
+} remove_nulling_relids_context;
+
static bool contain_aggs_of_level_walker(Node *node,
contain_aggs_of_level_context *context);
static bool locate_agg_of_level_walker(Node *node,
@@ -50,6 +57,9 @@ static bool locate_windowfunc_walker(Node *node,
static bool checkExprHasSubLink_walker(Node *node, void *context);
static Relids offset_relid_set(Relids relids, int offset);
static Relids adjust_relid_set(Relids relids, int oldrelid, int newrelid);
+static bool get_nulling_relids_walker(Node *node, Bitmapset **context);
+static Node *remove_nulling_relids_mutator(Node *node,
+ remove_nulling_relids_context *context);
/*
@@ -348,6 +358,8 @@ OffsetVarNodes_walker(Node *node, OffsetVarNodes_context *context)
if (var->varlevelsup == context->sublevels_up)
{
var->varno += context->offset;
+ var->varnullingrels = offset_relid_set(var->varnullingrels,
+ context->offset);
if (var->varnosyn > 0)
var->varnosyn += context->offset;
}
@@ -386,6 +398,8 @@ OffsetVarNodes_walker(Node *node, OffsetVarNodes_context *context)
{
phv->phrels = offset_relid_set(phv->phrels,
context->offset);
+ phv->phnullingrels = offset_relid_set(phv->phnullingrels,
+ context->offset);
}
/* fall through to examine children */
}
@@ -510,11 +524,13 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context)
{
Var *var = (Var *) node;
- if (var->varlevelsup == context->sublevels_up &&
- var->varno == context->rt_index)
+ if (var->varlevelsup == context->sublevels_up)
{
- var->varno = context->new_index;
- /* If the syntactic referent is same RTE, fix it too */
+ if (var->varno == context->rt_index)
+ var->varno = context->new_index;
+ var->varnullingrels = adjust_relid_set(var->varnullingrels,
+ context->rt_index,
+ context->new_index);
if (var->varnosyn == context->rt_index)
var->varnosyn = context->new_index;
}
@@ -557,6 +573,9 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context)
phv->phrels = adjust_relid_set(phv->phrels,
context->rt_index,
context->new_index);
+ phv->phnullingrels = adjust_relid_set(phv->phnullingrels,
+ context->rt_index,
+ context->new_index);
}
/* fall through to examine children */
}
@@ -833,7 +852,8 @@ rangeTableEntry_used_walker(Node *node,
Var *var = (Var *) node;
if (var->varlevelsup == context->sublevels_up &&
- var->varno == context->rt_index)
+ (var->varno == context->rt_index ||
+ bms_is_member(context->rt_index, var->varnullingrels)))
return true;
return false;
}
@@ -1061,6 +1081,154 @@ AddInvertedQual(Query *parsetree, Node *qual)
}
+/*
+ * get_nulling_relids collects all the level-zero RT indexes mentioned in
+ * Var.varnullingrels and PlaceHolderVar.phnullingrels fields within the
+ * given expression.
+ */
+Bitmapset *
+get_nulling_relids(Node *node)
+{
+ Bitmapset *result = NULL;
+
+ (void) get_nulling_relids_walker(node, &result);
+ return result;
+}
+
+static bool
+get_nulling_relids_walker(Node *node, Bitmapset **context)
+{
+ if (node == NULL)
+ return false;
+ if (IsA(node, Var))
+ {
+ Var *var = (Var *) node;
+
+ if (var->varlevelsup == 0)
+ *context = bms_add_members(*context, var->varnullingrels);
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ if (phv->phlevelsup == 0)
+ *context = bms_add_members(*context, phv->phnullingrels);
+ }
+
+ /*
+ * Currently, this is only used after the planner has converted SubLinks
+ * to SubPlans, so we don't need to support recursion into sub-Queries; so
+ * no sublevels_up counting is needed.
+ */
+ Assert(!IsA(node, SubLink));
+ Assert(!IsA(node, Query));
+ return expression_tree_walker(node, get_nulling_relids_walker, context);
+}
+
+/*
+ * remove_nulling_relids removes mentions of the specified RT index(es)
+ * in Var.varnullingrels and PlaceHolderVar.phnullingrels fields within
+ * the given expression, except in nodes belonging to rels listed in
+ * except_relids.
+ *
+ * XXX consider making this a destructive walker.
+ */
+Node *
+remove_nulling_relids(Node *node, Bitmapset *removable_relids,
+ Bitmapset *except_relids)
+{
+ remove_nulling_relids_context context;
+
+ context.removable_relids = removable_relids;
+ context.except_relids = except_relids;
+ context.sublevels_up = 0;
+ return query_or_expression_tree_mutator(node,
+ remove_nulling_relids_mutator,
+ &context,
+ 0);
+}
+
+static Node *
+remove_nulling_relids_mutator(Node *node,
+ remove_nulling_relids_context *context)
+{
+ if (node == NULL)
+ return NULL;
+ if (IsA(node, Var))
+ {
+ Var *var = (Var *) node;
+
+ if (var->varlevelsup == context->sublevels_up &&
+ !bms_is_member(var->varno, context->except_relids) &&
+ bms_overlap(var->varnullingrels, context->removable_relids))
+ {
+ Relids newnullingrels = bms_difference(var->varnullingrels,
+ context->removable_relids);
+
+ /* Micro-optimization: ensure nullingrels is NULL if empty */
+ if (bms_is_empty(newnullingrels))
+ newnullingrels = NULL;
+ /* Copy the Var ... */
+ var = copyObject(var);
+ /* ... and replace the copy's varnullingrels field */
+ var->varnullingrels = newnullingrels;
+ return (Node *) var;
+ }
+ /* Otherwise fall through to copy the Var normally */
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ if (phv->phlevelsup == context->sublevels_up &&
+ !bms_overlap(phv->phrels, context->except_relids))
+ {
+ Relids newnullingrels = bms_difference(phv->phnullingrels,
+ context->removable_relids);
+
+ /*
+ * Micro-optimization: ensure nullingrels is NULL if empty.
+ *
+ * Note: it might seem desirable to remove the PHV altogether if
+ * phnullingrels goes to empty. Currently we dare not do that
+ * because we use PHVs in some cases to enforce separate identity
+ * of subexpressions; see wrap_non_vars usages in prepjointree.c.
+ */
+ if (bms_is_empty(newnullingrels))
+ newnullingrels = NULL;
+ /* Copy the PlaceHolderVar and mutate what's below ... */
+ phv = (PlaceHolderVar *)
+ expression_tree_mutator(node,
+ remove_nulling_relids_mutator,
+ (void *) context);
+ /* ... and replace the copy's phnullingrels field */
+ phv->phnullingrels = newnullingrels;
+ /* We must also update phrels, if it contains a removable RTI */
+ phv->phrels = bms_difference(phv->phrels,
+ context->removable_relids);
+ Assert(!bms_is_empty(phv->phrels));
+ return (Node *) phv;
+ }
+ /* Otherwise fall through to copy the PlaceHolderVar normally */
+ }
+ else if (IsA(node, Query))
+ {
+ /* Recurse into RTE or sublink subquery */
+ Query *newnode;
+
+ context->sublevels_up++;
+ newnode = query_tree_mutator((Query *) node,
+ remove_nulling_relids_mutator,
+ (void *) context,
+ 0);
+ context->sublevels_up--;
+ return (Node *) newnode;
+ }
+ return expression_tree_mutator(node, remove_nulling_relids_mutator,
+ (void *) context);
+}
+
+
/*
* replace_rte_variables() finds all Vars in an expression tree
* that reference a particular RTE, and replaces them with substitute
diff --git a/src/backend/utils/misc/queryjumble.c b/src/backend/utils/misc/queryjumble.c
index eeaa0b31fe..e517e0363c 100644
--- a/src/backend/utils/misc/queryjumble.c
+++ b/src/backend/utils/misc/queryjumble.c
@@ -381,6 +381,11 @@ JumbleExpr(JumbleState *jstate, Node *node)
APP_JUMB(var->varno);
APP_JUMB(var->varattno);
APP_JUMB(var->varlevelsup);
+
+ /*
+ * We can omit varnullingrels, because it's fully determined
+ * by varno/varlevelsup plus the Var's query location.
+ */
}
break;
case T_Const:
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index ee8bed6452..369ddfbddc 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2574,10 +2574,15 @@ typedef struct MergeScanSelCache
* of a plan tree. This is used during planning to represent the contained
* expression. At the end of the planning process it is replaced by either
* the contained expression or a Var referring to a lower-level evaluation of
- * the contained expression. Typically the evaluation occurs below an outer
+ * the contained expression. Generally the evaluation occurs below an outer
* join, and Var references above the outer join might thereby yield NULL
* instead of the expression value.
*
+ * phrels and phlevelsup correspond to the varno/varlevelsup fields of a
+ * plain Var, except that phrels has to be a relid set since the evaluation
+ * level of a PlaceHolderVar might be a join rather than a base relation.
+ * Likewise, phnullingrels corresponds to varnullingrels.
+ *
* Although the planner treats this as an expression node type, it is not
* recognized by the parser or executor, so we declare it here rather than
* in primnodes.h.
@@ -2590,8 +2595,10 @@ typedef struct MergeScanSelCache
* PHV. Another way in which it can happen is that initplan sublinks
* could get replaced by differently-numbered Params when sublink folding
* is done. (The end result of such a situation would be some
- * unreferenced initplans, which is annoying but not really a problem.) On
- * the same reasoning, there is no need to examine phrels.
+ * unreferenced initplans, which is annoying but not really a problem.) On
+ * the same reasoning, there is no need to examine phrels. But we do need
+ * to compare phnullingrels, as that represents effects that are external
+ * to the original value of the PHV.
*/
typedef struct PlaceHolderVar
@@ -2601,9 +2608,12 @@ typedef struct PlaceHolderVar
/* the represented expression */
Expr *phexpr pg_node_attr(equal_ignore);
- /* base relids syntactically within expr src */
+ /* base+OJ relids syntactically within expr src */
Relids phrels pg_node_attr(equal_ignore);
+ /* RT indexes of outer joins that can null PHV's value */
+ Relids phnullingrels;
+
/* ID for PHV (unique within planner run) */
Index phid;
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index 1fc2fbffa3..f2a0739e6e 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -189,6 +189,14 @@ typedef struct Expr
* row identity information during UPDATE/DELETE. This value should never
* be seen outside the planner.
*
+ * varnullingrels is the set of RT indexes of outer joins that can force
+ * the Var's value to null (at the point where it appears in the query).
+ * See optimizer/README for discussion of that.
+ *
+ * varlevelsup is greater than zero in Vars that represent outer references.
+ * Note that it affects the meaning of all of varno, varnullingrels, and
+ * varnosyn, all of which refer to the range table of that query level.
+ *
* In the parser, varnosyn and varattnosyn are either identical to
* varno/varattno, or they specify the column's position in an aliased JOIN
* RTE that hides the semantic referent RTE's refname. This is a syntactic
@@ -231,6 +239,8 @@ typedef struct Var
int32 vartypmod;
/* OID of collation, or InvalidOid if none */
Oid varcollid;
+ /* RT indexes of outer joins that can replace the Var's value with null */
+ Bitmapset *varnullingrels;
/*
* for subquery variables referencing outer relations; 0 in a normal var,
diff --git a/src/include/rewrite/rewriteManip.h b/src/include/rewrite/rewriteManip.h
index 98b9b3a288..a3f902c1bb 100644
--- a/src/include/rewrite/rewriteManip.h
+++ b/src/include/rewrite/rewriteManip.h
@@ -63,6 +63,10 @@ extern bool contain_windowfuncs(Node *node);
extern int locate_windowfunc(Node *node);
extern bool checkExprHasSubLink(Node *node);
+extern Bitmapset *get_nulling_relids(Node *node);
+extern Node *remove_nulling_relids(Node *node, Bitmapset *removable_relids,
+ Bitmapset *except_relids);
+
extern Node *replace_rte_variables(Node *node,
int target_varno, int sublevels_up,
replace_rte_variables_callback callback,
commit 2ffacd89273b53d8cc94ae788ad2b6d4d1c3b7fc
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sun Jul 10 13:44:30 2022 -0400
Teach the parser to fill Var.varnullingrels correctly.
Vars emitted by the parser are now marked with RT indexes of outer
joins that can null them. (This is done purely according to the
syntax of the query; we don't consider whether an outer join could
be strength-reduced, for example.)
Although the result of this step compiles, it will fail some
regression tests due to the planner not yet knowing what to do.
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 8ed2c4b8c7..a7fe6f4e66 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -670,6 +670,7 @@ transformInsertStmt(ParseState *pstate, InsertStmt *stmt)
*/
sub_pstate->p_rtable = sub_rtable;
sub_pstate->p_joinexprs = NIL; /* sub_rtable has no joins */
+ sub_pstate->p_nullingrels = NIL;
sub_pstate->p_namespace = sub_namespace;
sub_pstate->p_resolve_unknowns = false;
@@ -851,7 +852,7 @@ transformInsertStmt(ParseState *pstate, InsertStmt *stmt)
/*
* Generate list of Vars referencing the RTE
*/
- exprList = expandNSItemVars(nsitem, 0, -1, NULL);
+ exprList = expandNSItemVars(pstate, nsitem, 0, -1, NULL);
/*
* Re-apply any indirection on the target column specs to the Vars
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index c655d188c7..2abd164380 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -52,7 +52,8 @@
#include "utils/syscache.h"
-static int extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
+static int extractRemainingColumns(ParseState *pstate,
+ ParseNamespaceColumn *src_nscolumns,
List *src_colnames,
List **src_colnos,
List **res_colnames, List **res_colvars,
@@ -75,9 +76,11 @@ static ParseNamespaceItem *getNSItemForSpecialRelationTypes(ParseState *pstate,
static Node *transformFromClauseItem(ParseState *pstate, Node *n,
ParseNamespaceItem **top_nsitem,
List **namespace);
-static Var *buildVarFromNSColumn(ParseNamespaceColumn *nscol);
+static Var *buildVarFromNSColumn(ParseState *pstate,
+ ParseNamespaceColumn *nscol);
static Node *buildMergedJoinVar(ParseState *pstate, JoinType jointype,
Var *l_colvar, Var *r_colvar);
+static void markRelsAsNulledBy(ParseState *pstate, Node *n, int jindex);
static void setNamespaceColumnVisibility(List *namespace, bool cols_visible);
static void setNamespaceLateralState(List *namespace,
bool lateral_only, bool lateral_ok);
@@ -249,7 +252,8 @@ setTargetTable(ParseState *pstate, RangeVar *relation,
* Returns the number of columns added.
*/
static int
-extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
+extractRemainingColumns(ParseState *pstate,
+ ParseNamespaceColumn *src_nscolumns,
List *src_colnames,
List **src_colnos,
List **res_colnames, List **res_colvars,
@@ -285,7 +289,8 @@ extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
*src_colnos = lappend_int(*src_colnos, attnum);
*res_colnames = lappend(*res_colnames, lfirst(lc));
*res_colvars = lappend(*res_colvars,
- buildVarFromNSColumn(src_nscolumns + attnum - 1));
+ buildVarFromNSColumn(pstate,
+ src_nscolumns + attnum - 1));
/* Copy the input relation's nscolumn data for this column */
res_nscolumns[colcount] = src_nscolumns[attnum - 1];
colcount++;
@@ -1295,8 +1300,7 @@ transformFromClauseItem(ParseState *pstate, Node *n,
{
/*
* JOIN/USING (or NATURAL JOIN, as transformed above). Transform
- * the list into an explicit ON-condition, and generate a list of
- * merged result columns.
+ * the list into an explicit ON-condition.
*/
List *ucols = j->usingClause;
List *l_usingvars = NIL;
@@ -1314,8 +1318,6 @@ transformFromClauseItem(ParseState *pstate, Node *n,
int r_index = -1;
Var *l_colvar,
*r_colvar;
- Node *u_colvar;
- ParseNamespaceColumn *res_nscolumn;
Assert(u_colname[0] != '\0');
@@ -1379,17 +1381,109 @@ transformFromClauseItem(ParseState *pstate, Node *n,
u_colname)));
r_colnos = lappend_int(r_colnos, r_index + 1);
- l_colvar = buildVarFromNSColumn(l_nscolumns + l_index);
+ /* Build Vars to use in the generated JOIN ON clause */
+ l_colvar = buildVarFromNSColumn(pstate, l_nscolumns + l_index);
l_usingvars = lappend(l_usingvars, l_colvar);
- r_colvar = buildVarFromNSColumn(r_nscolumns + r_index);
+ r_colvar = buildVarFromNSColumn(pstate, r_nscolumns + r_index);
r_usingvars = lappend(r_usingvars, r_colvar);
+ /*
+ * While we're here, add column names to the res_colnames
+ * list. It's a bit ugly to do this here while the
+ * corresponding res_colvars entries are not made till later,
+ * but doing this later would require an additional traversal
+ * of the usingClause list.
+ */
res_colnames = lappend(res_colnames, lfirst(ucol));
+ }
+
+ /* Construct the generated JOIN ON clause */
+ j->quals = transformJoinUsingClause(pstate,
+ l_usingvars,
+ r_usingvars);
+ }
+ else if (j->quals)
+ {
+ /* User-written ON-condition; transform it */
+ j->quals = transformJoinOnClause(pstate, j, my_namespace);
+ }
+ else
+ {
+ /* CROSS JOIN: no quals */
+ }
+
+ /*
+ * If this is an outer join, now mark the appropriate child RTEs as
+ * being nulled by this join. We have finished processing the child
+ * join expressions as well as the current join's quals, which deal in
+ * non-nulled input columns. All future references to those RTEs will
+ * see possibly-nulled values, and we should mark generated Vars to
+ * account for that. In particular, the join alias Vars that we're
+ * about to build should reflect the nulling effects of this join.
+ *
+ * A difficulty with doing this is that we need the join's RT index,
+ * which we don't officially have yet. However, no other RTE can get
+ * made between here and the addRangeTableEntryForJoin call, so we can
+ * predict what the assignment will be. (Alternatively, we could call
+ * addRangeTableEntryForJoin before we have all the data computed, but
+ * this seems less ugly.)
+ */
+ j->rtindex = list_length(pstate->p_rtable) + 1;
+
+ switch (j->jointype)
+ {
+ case JOIN_INNER:
+ break;
+ case JOIN_LEFT:
+ markRelsAsNulledBy(pstate, j->rarg, j->rtindex);
+ break;
+ case JOIN_FULL:
+ markRelsAsNulledBy(pstate, j->larg, j->rtindex);
+ markRelsAsNulledBy(pstate, j->rarg, j->rtindex);
+ break;
+ case JOIN_RIGHT:
+ markRelsAsNulledBy(pstate, j->larg, j->rtindex);
+ break;
+ default:
+ /* shouldn't see any other types here */
+ elog(ERROR, "unrecognized join type: %d",
+ (int) j->jointype);
+ break;
+ }
+
+ /*
+ * Now we can construct join alias expressions for the USING columns.
+ */
+ if (j->usingClause)
+ {
+ ListCell *lc1,
+ *lc2;
+
+ /* Scan the colnos lists to recover info from the previous loop */
+ forboth(lc1, l_colnos, lc2, r_colnos)
+ {
+ int l_index = lfirst_int(lc1) - 1;
+ int r_index = lfirst_int(lc2) - 1;
+ Var *l_colvar,
+ *r_colvar;
+ Node *u_colvar;
+ ParseNamespaceColumn *res_nscolumn;
+
+ /*
+ * Note we re-build these Vars: they might have different
+ * varnullingrels than the ones made in the previous loop.
+ */
+ l_colvar = buildVarFromNSColumn(pstate, l_nscolumns + l_index);
+ r_colvar = buildVarFromNSColumn(pstate, r_nscolumns + r_index);
+
+ /* Construct the join alias Var for this column */
u_colvar = buildMergedJoinVar(pstate,
j->jointype,
l_colvar,
r_colvar);
res_colvars = lappend(res_colvars, u_colvar);
+
+ /* Construct column's res_nscolumns[] entry */
res_nscolumn = res_nscolumns + res_colindex;
res_colindex++;
if (u_colvar == (Node *) l_colvar)
@@ -1407,47 +1501,45 @@ transformFromClauseItem(ParseState *pstate, Node *n,
/*
* Merged column is not semantically equivalent to either
* input, so it needs to be referenced as the join output
- * column. We don't know the join's varno yet, so we'll
- * replace these zeroes below.
+ * column.
*/
- res_nscolumn->p_varno = 0;
+ res_nscolumn->p_varno = j->rtindex;
res_nscolumn->p_varattno = res_colindex;
res_nscolumn->p_vartype = exprType(u_colvar);
res_nscolumn->p_vartypmod = exprTypmod(u_colvar);
res_nscolumn->p_varcollid = exprCollation(u_colvar);
- res_nscolumn->p_varnosyn = 0;
+ res_nscolumn->p_varnosyn = j->rtindex;
res_nscolumn->p_varattnosyn = res_colindex;
}
}
-
- j->quals = transformJoinUsingClause(pstate,
- l_usingvars,
- r_usingvars);
- }
- else if (j->quals)
- {
- /* User-written ON-condition; transform it */
- j->quals = transformJoinOnClause(pstate, j, my_namespace);
- }
- else
- {
- /* CROSS JOIN: no quals */
}
/* Add remaining columns from each side to the output columns */
res_colindex +=
- extractRemainingColumns(l_nscolumns, l_colnames, &l_colnos,
+ extractRemainingColumns(pstate,
+ l_nscolumns, l_colnames, &l_colnos,
&res_colnames, &res_colvars,
res_nscolumns + res_colindex);
res_colindex +=
- extractRemainingColumns(r_nscolumns, r_colnames, &r_colnos,
+ extractRemainingColumns(pstate,
+ r_nscolumns, r_colnames, &r_colnos,
&res_colnames, &res_colvars,
res_nscolumns + res_colindex);
+ /* If join has an alias, it syntactically hides all inputs */
+ if (j->alias)
+ {
+ for (k = 0; k < res_colindex; k++)
+ {
+ ParseNamespaceColumn *nscol = res_nscolumns + k;
+
+ nscol->p_varnosyn = j->rtindex;
+ nscol->p_varattnosyn = k + 1;
+ }
+ }
+
/*
* Now build an RTE and nsitem for the result of the join.
- * res_nscolumns isn't totally done yet, but that's OK because
- * addRangeTableEntryForJoin doesn't examine it, only store a pointer.
*/
nsitem = addRangeTableEntryForJoin(pstate,
res_colnames,
@@ -1461,31 +1553,16 @@ transformFromClauseItem(ParseState *pstate, Node *n,
j->alias,
true);
- j->rtindex = nsitem->p_rtindex;
+ /* Verify that we correctly predicted the join's RT index */
+ Assert(j->rtindex == nsitem->p_rtindex);
+ /* Cross-check number of columns, too */
+ Assert(res_colindex == list_length(nsitem->p_names->colnames));
/*
- * Now that we know the join RTE's rangetable index, we can fix up the
- * res_nscolumns data in places where it should contain that.
+ * Save a link to the JoinExpr in the proper element of p_joinexprs.
+ * Since we maintain that list lazily, it may be necessary to fill in
+ * empty entries before we can add the JoinExpr in the right place.
*/
- Assert(res_colindex == list_length(nsitem->p_names->colnames));
- for (k = 0; k < res_colindex; k++)
- {
- ParseNamespaceColumn *nscol = res_nscolumns + k;
-
- /* fill in join RTI for merged columns */
- if (nscol->p_varno == 0)
- nscol->p_varno = j->rtindex;
- if (nscol->p_varnosyn == 0)
- nscol->p_varnosyn = j->rtindex;
- /* if join has an alias, it syntactically hides all inputs */
- if (j->alias)
- {
- nscol->p_varnosyn = j->rtindex;
- nscol->p_varattnosyn = k + 1;
- }
- }
-
- /* make a matching link to the JoinExpr for later use */
for (k = list_length(pstate->p_joinexprs) + 1; k < j->rtindex; k++)
pstate->p_joinexprs = lappend(pstate->p_joinexprs, NULL);
pstate->p_joinexprs = lappend(pstate->p_joinexprs, j);
@@ -1554,10 +1631,13 @@ transformFromClauseItem(ParseState *pstate, Node *n,
* buildVarFromNSColumn -
* build a Var node using ParseNamespaceColumn data
*
- * We assume varlevelsup should be 0, and no location is specified
+ * This is used to construct joinaliasvars entries.
+ * We can assume varlevelsup should be 0, and no location is specified.
+ * Note also that no column SELECT privilege is requested here; that would
+ * happen only if the column is actually referenced in the query.
*/
static Var *
-buildVarFromNSColumn(ParseNamespaceColumn *nscol)
+buildVarFromNSColumn(ParseState *pstate, ParseNamespaceColumn *nscol)
{
Var *var;
@@ -1571,6 +1651,10 @@ buildVarFromNSColumn(ParseNamespaceColumn *nscol)
/* makeVar doesn't offer parameters for these, so set by hand: */
var->varnosyn = nscol->p_varnosyn;
var->varattnosyn = nscol->p_varattnosyn;
+
+ /* ... and update varnullingrels */
+ markNullableIfNeeded(pstate, var);
+
return var;
}
@@ -1682,6 +1766,47 @@ buildMergedJoinVar(ParseState *pstate, JoinType jointype,
return res_node;
}
+/*
+ * markRelsAsNulledBy -
+ * Mark the given jointree node and its children as nulled by join jindex
+ */
+static void
+markRelsAsNulledBy(ParseState *pstate, Node *n, int jindex)
+{
+ int varno;
+ ListCell *lc;
+
+ /* Note: we can't see FromExpr here */
+ if (IsA(n, RangeTblRef))
+ {
+ varno = ((RangeTblRef *) n)->rtindex;
+ }
+ else if (IsA(n, JoinExpr))
+ {
+ JoinExpr *j = (JoinExpr *) n;
+
+ /* recurse to children */
+ markRelsAsNulledBy(pstate, j->larg, jindex);
+ markRelsAsNulledBy(pstate, j->rarg, jindex);
+ varno = j->rtindex;
+ }
+ else
+ {
+ elog(ERROR, "unrecognized node type: %d", (int) nodeTag(n));
+ varno = 0; /* keep compiler quiet */
+ }
+
+ /*
+ * Now add jindex to the p_nullingrels set for relation varno. Since we
+ * maintain the p_nullingrels list lazily, we might need to extend it to
+ * make the varno'th entry exist.
+ */
+ while (list_length(pstate->p_nullingrels) < varno)
+ pstate->p_nullingrels = lappend(pstate->p_nullingrels, NULL);
+ lc = list_nth_cell(pstate->p_nullingrels, varno - 1);
+ lfirst(lc) = bms_add_member((Bitmapset *) lfirst(lc), jindex);
+}
+
/*
* setNamespaceColumnVisibility -
* Convenience subroutine to update cols_visible flags in a namespace list.
diff --git a/src/backend/parser/parse_coerce.c b/src/backend/parser/parse_coerce.c
index c4e958e4aa..4ded12e873 100644
--- a/src/backend/parser/parse_coerce.c
+++ b/src/backend/parser/parse_coerce.c
@@ -1042,7 +1042,7 @@ coerce_record_to_complex(ParseState *pstate, Node *node,
ParseNamespaceItem *nsitem;
nsitem = GetNSItemByRangeTablePosn(pstate, rtindex, sublevels_up);
- args = expandNSItemVars(nsitem, sublevels_up, vlocation, NULL);
+ args = expandNSItemVars(pstate, nsitem, sublevels_up, vlocation, NULL);
}
else
ereport(ERROR,
diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c
index efcf1cd5ab..2f6ace0b15 100644
--- a/src/backend/parser/parse_expr.c
+++ b/src/backend/parser/parse_expr.c
@@ -2602,6 +2602,9 @@ transformWholeRowRef(ParseState *pstate, ParseNamespaceItem *nsitem,
/* location is not filled in by makeWholeRowVar */
result->location = location;
+ /* mark Var if it's nulled by any outer joins */
+ markNullableIfNeeded(pstate, result);
+
/* mark relation as requiring whole-row SELECT access */
markVarForSelectPriv(pstate, result);
@@ -2629,6 +2632,8 @@ transformWholeRowRef(ParseState *pstate, ParseNamespaceItem *nsitem,
rowexpr->colnames = copyObject(nsitem->p_names->colnames);
rowexpr->location = location;
+ /* XXX we ought to mark the row as possibly nullable */
+
return (Node *) rowexpr;
}
}
diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c
index 926dcbf30e..deec58e4b1 100644
--- a/src/backend/parser/parse_relation.c
+++ b/src/backend/parser/parse_relation.c
@@ -751,6 +751,9 @@ scanNSItemForColumn(ParseState *pstate, ParseNamespaceItem *nsitem,
}
var->location = location;
+ /* Mark Var if it's nulled by any outer joins */
+ markNullableIfNeeded(pstate, var);
+
/* Require read access to the column */
markVarForSelectPriv(pstate, var);
@@ -1007,6 +1010,35 @@ searchRangeTableForCol(ParseState *pstate, const char *alias, const char *colnam
return fuzzystate;
}
+/*
+ * markNullableIfNeeded
+ * If the RTE referenced by the Var is nullable by outer join(s)
+ * at this point in the query, set var->varnullingrels to show that.
+ */
+void
+markNullableIfNeeded(ParseState *pstate, Var *var)
+{
+ int rtindex = var->varno;
+ Bitmapset *relids;
+
+ /* Find the appropriate pstate */
+ for (int lv = 0; lv < var->varlevelsup; lv++)
+ pstate = pstate->parentParseState;
+
+ /* Find currently-relevant join relids for the Var's rel */
+ if (rtindex > 0 && rtindex <= list_length(pstate->p_nullingrels))
+ relids = (Bitmapset *) list_nth(pstate->p_nullingrels, rtindex - 1);
+ else
+ relids = NULL;
+
+ /*
+ * Merge with any already-declared nulling rels. (Typically there won't
+ * be any, but let's get it right if there are.)
+ */
+ if (relids != NULL)
+ var->varnullingrels = bms_union(var->varnullingrels, relids);
+}
+
/*
* markRTEForSelectPriv
* Mark the specified column of the RTE with index rtindex
@@ -3066,7 +3098,7 @@ expandTupleDesc(TupleDesc tupdesc, Alias *eref, int count, int offset,
* the list elements mustn't be modified.
*/
List *
-expandNSItemVars(ParseNamespaceItem *nsitem,
+expandNSItemVars(ParseState *pstate, ParseNamespaceItem *nsitem,
int sublevels_up, int location,
List **colnames)
{
@@ -3102,6 +3134,10 @@ expandNSItemVars(ParseNamespaceItem *nsitem,
var->varnosyn = nscol->p_varnosyn;
var->varattnosyn = nscol->p_varattnosyn;
var->location = location;
+
+ /* ... and update varnullingrels */
+ markNullableIfNeeded(pstate, var);
+
result = lappend(result, var);
if (colnames)
*colnames = lappend(*colnames, colnameval);
@@ -3136,7 +3172,7 @@ expandNSItemAttrs(ParseState *pstate, ParseNamespaceItem *nsitem,
*var;
List *te_list = NIL;
- vars = expandNSItemVars(nsitem, sublevels_up, location, &names);
+ vars = expandNSItemVars(pstate, nsitem, sublevels_up, location, &names);
/*
* Require read access to the table. This is normally redundant with the
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index 2a1d44b813..22834a5bf3 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -1379,7 +1379,7 @@ ExpandSingleTable(ParseState *pstate, ParseNamespaceItem *nsitem,
List *vars;
ListCell *l;
- vars = expandNSItemVars(nsitem, sublevels_up, location, NULL);
+ vars = expandNSItemVars(pstate, nsitem, sublevels_up, location, NULL);
/*
* Require read access to the table. This is normally redundant with
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 0b6a7bb365..13b6f1a8e0 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1081,6 +1081,14 @@ typedef struct RangeTblEntry
* alias Vars are generated only for merged columns). We keep these
* entries only because they're needed in expandRTE() and similar code.
*
+ * Vars appearing within joinaliasvars are marked with varnullingrels sets
+ * that describe the nulling effects of this join and lower ones. This is
+ * essential for FULL JOIN cases, because the COALESCE expression only
+ * describes the semantics correctly if its inputs have been nulled by the
+ * join. For other cases, it allows expandRTE() to generate a valid
+ * representation of the join's output without consulting additional
+ * parser state.
+ *
* Within a Query loaded from a stored rule, it is possible for non-merged
* joinaliasvars items to be null pointers, which are placeholders for
* (necessarily unreferenced) columns dropped since the rule was made.
diff --git a/src/include/parser/parse_node.h b/src/include/parser/parse_node.h
index cf9c759025..8bef98487d 100644
--- a/src/include/parser/parse_node.h
+++ b/src/include/parser/parse_node.h
@@ -115,6 +115,13 @@ typedef Node *(*CoerceParamHook) (ParseState *pstate, Param *param,
* This is one-for-one with p_rtable, but contains NULLs for non-join
* RTEs, and may be shorter than p_rtable if the last RTE(s) aren't joins.
*
+ * p_nullingrels: list of Bitmapsets associated with p_rtable entries, each
+ * containing the set of outer-join RTE indexes that can null that relation
+ * at the current point in the parse tree. This is one-for-one with p_rtable,
+ * but may be shorter than p_rtable, in which case the missing entries are
+ * implicitly empty (NULL). That rule allows us to save work when the query
+ * contains no outer joins.
+ *
* p_joinlist: list of join items (RangeTblRef and JoinExpr nodes) that
* will become the fromlist of the query's top-level FromExpr node.
*
@@ -182,6 +189,7 @@ struct ParseState
const char *p_sourcetext; /* source text, or NULL if not available */
List *p_rtable; /* range table so far */
List *p_joinexprs; /* JoinExprs for RTE_JOIN p_rtable entries */
+ List *p_nullingrels; /* Bitmapsets showing nulling outer joins */
List *p_joinlist; /* join items so far (will become FromExpr
* node's fromlist) */
List *p_namespace; /* currently-referenceable RTEs (List of
diff --git a/src/include/parser/parse_relation.h b/src/include/parser/parse_relation.h
index de21c3c649..85d96563f3 100644
--- a/src/include/parser/parse_relation.h
+++ b/src/include/parser/parse_relation.h
@@ -41,6 +41,7 @@ extern Node *scanNSItemForColumn(ParseState *pstate, ParseNamespaceItem *nsitem,
int location);
extern Node *colNameToVar(ParseState *pstate, const char *colname, bool localonly,
int location);
+extern void markNullableIfNeeded(ParseState *pstate, Var *var);
extern void markVarForSelectPriv(ParseState *pstate, Var *var);
extern Relation parserOpenTable(ParseState *pstate, const RangeVar *relation,
int lockmode);
@@ -109,7 +110,7 @@ extern void errorMissingColumn(ParseState *pstate,
extern void expandRTE(RangeTblEntry *rte, int rtindex, int sublevels_up,
int location, bool include_dropped,
List **colnames, List **colvars);
-extern List *expandNSItemVars(ParseNamespaceItem *nsitem,
+extern List *expandNSItemVars(ParseState *pstate, ParseNamespaceItem *nsitem,
int sublevels_up, int location,
List **colnames);
extern List *expandNSItemAttrs(ParseState *pstate, ParseNamespaceItem *nsitem,
commit d7a2e3fdc5f502af367ff27bc14bcdebe48ba933
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sun Jul 10 14:31:47 2022 -0400
Teach the planner to cope with Vars bearing nullingrels.
The core idea of this step is to include varnullingrels in the
relid sets that qual clauses are considered to depend on.
So that we can still easily compare quals' relids to RelOptInfos'
relids, that means also adding outer join relids to the identifying
relids of join relations. Much of the bulk of this step is concerned
with fallout from the latter change.
Also, in setrefs.c and some other places, we have to intentionally
ignore varnullingrels when comparing the outputs of a lower plan
node to the Vars required by upper expressions. I'd like to tighten
that up, by accounting for whether a given plan node implements
an outer join and expecting the upper Vars to have that OJ relid
added to their varnullingrels if so. But because of the hackery
involved in implementing outer join identity 3, there are some cases
where the upper Var legitimately won't have that bit set, and it's
unclear how to make a check that doesn't reject such plans. So that
issue is left for later. It would only be a bug-detection aid
anyway, since all the interesting decisions have been made already.
This step removes some low-hanging fruit from the old implementation,
such as the need to track lowest_nulling_outer_join during subquery
pullup. There's much more to do in that line, though.
The result of this step passes most core regression tests, but there
are still some failure cases involving full joins.
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index e9342097e5..97c8cd0711 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -179,6 +179,9 @@ make_one_rel(PlannerInfo *root, List *joinlist)
root->all_baserels = bms_add_member(root->all_baserels, brel->relid);
}
+ /* Now we can form the value of all_query_rels, too */
+ root->all_query_rels = bms_union(root->all_baserels, root->outer_join_rels);
+
/* Mark base rels as to whether we care about fast-start plans */
set_base_rel_consider_startup(root);
@@ -230,9 +233,9 @@ make_one_rel(PlannerInfo *root, List *joinlist)
rel = make_rel_from_joinlist(root, joinlist);
/*
- * The result should join all and only the query's base rels.
+ * The result should join all and only the query's base + outer-join rels.
*/
- Assert(bms_equal(rel->relids, root->all_baserels));
+ Assert(bms_equal(rel->relids, root->all_query_rels));
return rel;
}
@@ -558,7 +561,7 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* (see grouping_planner).
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- bms_membership(root->all_baserels) != BMS_SINGLETON)
+ bms_membership(root->all_query_rels) != BMS_SINGLETON)
generate_useful_gather_paths(root, rel, false);
/* Now find the cheapest of the paths for this rel */
@@ -879,7 +882,7 @@ set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *
* to support an uncommon usage of second-rate sampling methods. Instead,
* if there is a risk that the query might perform an unsafe join, just
* wrap the SampleScan in a Materialize node. We can check for joins by
- * counting the membership of all_baserels (note that this correctly
+ * counting the membership of all_query_rels (note that this correctly
* counts inheritance trees as single rels). If we're inside a subquery,
* we can't easily check whether a join might occur in the outer query, so
* just assume one is possible.
@@ -888,7 +891,7 @@ set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *
* so check repeatable_across_scans last, even though that's a bit odd.
*/
if ((root->query_level > 1 ||
- bms_membership(root->all_baserels) != BMS_SINGLETON) &&
+ bms_membership(root->all_query_rels) != BMS_SINGLETON) &&
!(GetTsmRoutine(rte->tablesample->tsmhandler)->repeatable_across_scans))
{
path = (Path *) create_material_path(rel, path);
diff --git a/src/backend/optimizer/path/clausesel.c b/src/backend/optimizer/path/clausesel.c
index 06f836308d..6a5182c0d7 100644
--- a/src/backend/optimizer/path/clausesel.c
+++ b/src/backend/optimizer/path/clausesel.c
@@ -218,7 +218,7 @@ clauselist_selectivity_ext(PlannerInfo *root,
if (rinfo)
{
- ok = (bms_membership(rinfo->clause_relids) == BMS_SINGLETON) &&
+ ok = (rinfo->num_base_rels == 1) &&
(is_pseudo_constant_clause_relids(lsecond(expr->args),
rinfo->right_relids) ||
(varonleft = false,
@@ -579,30 +579,6 @@ find_single_rel_for_clauses(PlannerInfo *root, List *clauses)
return NULL; /* no clauses */
}
-/*
- * bms_is_subset_singleton
- *
- * Same result as bms_is_subset(s, bms_make_singleton(x)),
- * but a little faster and doesn't leak memory.
- *
- * Is this of use anywhere else? If so move to bitmapset.c ...
- */
-static bool
-bms_is_subset_singleton(const Bitmapset *s, int x)
-{
- switch (bms_membership(s))
- {
- case BMS_EMPTY_SET:
- return true;
- case BMS_SINGLETON:
- return bms_is_member(x, s);
- case BMS_MULTIPLE:
- return false;
- }
- /* can't get here... */
- return false;
-}
-
/*
* treat_as_join_clause -
* Decide whether an operator clause is to be handled by the
@@ -631,17 +607,20 @@ treat_as_join_clause(PlannerInfo *root, Node *clause, RestrictInfo *rinfo,
else
{
/*
- * Otherwise, it's a join if there's more than one relation used. We
- * can optimize this calculation if an rinfo was passed.
+ * Otherwise, it's a join if there's more than one base relation used.
+ * We can optimize this calculation if an rinfo was passed.
*
* XXX Since we know the clause is being evaluated at a join, the
* only way it could be single-relation is if it was delayed by outer
- * joins. Although we can make use of the restriction qual estimators
- * anyway, it seems likely that we ought to account for the
- * probability of injected nulls somehow.
+ * joins. We intentionally count only baserels here, not OJs that
+ * might be present in rinfo->clause_relids, so that we direct such
+ * cases to the restriction qual estimators not join estimators.
+ * Eventually some notice should be taken of the possibility of
+ * injected nulls, but we'll likely want to do that in the restriction
+ * estimators rather than starting to treat such cases as join quals.
*/
if (rinfo)
- return (bms_membership(rinfo->clause_relids) == BMS_MULTIPLE);
+ return (rinfo->num_base_rels > 1);
else
return (NumRelids(root, clause) > 1);
}
@@ -753,8 +732,7 @@ clause_selectivity_ext(PlannerInfo *root,
* considering a unique-ified case, so we only need one cache variable
* for all non-JOIN_INNER cases.
*/
- if (varRelid == 0 ||
- bms_is_subset_singleton(rinfo->clause_relids, varRelid))
+ if (varRelid == 0 || rinfo->num_base_rels <= 1)
{
/* Cacheable --- do we already have the result? */
if (jointype == JOIN_INNER)
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index fcc26b01a4..3ca598830e 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -5083,7 +5083,9 @@ compute_semi_anti_join_factors(PlannerInfo *root,
norm_sjinfo.syn_lefthand = outerrel->relids;
norm_sjinfo.syn_righthand = innerrel->relids;
norm_sjinfo.jointype = JOIN_INNER;
+ norm_sjinfo.ojrelid = 0;
/* we don't bother trying to make the remaining fields valid */
+ norm_sjinfo.strict_relids = NULL;
norm_sjinfo.lhs_strict = false;
norm_sjinfo.delay_upper_joins = false;
norm_sjinfo.semi_can_btree = false;
@@ -5248,7 +5250,9 @@ approx_tuple_count(PlannerInfo *root, JoinPath *path, List *quals)
sjinfo.syn_lefthand = path->outerjoinpath->parent->relids;
sjinfo.syn_righthand = path->innerjoinpath->parent->relids;
sjinfo.jointype = JOIN_INNER;
+ sjinfo.ojrelid = 0;
/* we don't bother trying to make the remaining fields valid */
+ sjinfo.strict_relids = NULL;
sjinfo.lhs_strict = false;
sjinfo.delay_upper_joins = false;
sjinfo.semi_can_btree = false;
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index f8a97622b1..d31ca5e527 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -29,6 +29,7 @@
#include "optimizer/paths.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "rewrite/rewriteManip.h"
#include "utils/lsyscache.h"
@@ -64,7 +65,7 @@ static bool reconsider_outer_join_clause(PlannerInfo *root,
RestrictInfo *rinfo,
bool outer_on_left);
static bool reconsider_full_join_clause(PlannerInfo *root,
- RestrictInfo *rinfo);
+ FullJoinClauseInfo *fjinfo);
static Bitmapset *get_eclass_indexes_for_relids(PlannerInfo *root,
Relids relids);
static Bitmapset *get_common_eclass_indexes(PlannerInfo *root, Relids relids1,
@@ -768,6 +769,9 @@ get_eclass_for_sort_expr(PlannerInfo *root,
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL)
+ continue; /* must be an outer join */
+
Assert(rel->reloptkind == RELOPT_BASEREL ||
rel->reloptkind == RELOPT_DEADREL);
@@ -936,7 +940,36 @@ is_exprlist_member(Expr *node, List *exprs)
if (expr && IsA(expr, TargetEntry))
expr = ((TargetEntry *) expr)->expr;
- if (equal(node, expr))
+ /*
+ * For Vars and PlaceHolderVars, match using the same rules as
+ * setrefs.c will, in particular ignoring nullingrels. XXX when that
+ * gets tightened up, this should too.
+ */
+ if (IsA(node, Var))
+ {
+ if (expr && IsA(expr, Var))
+ {
+ Var *v1 = (Var *) node;
+ Var *v2 = (Var *) expr;
+
+ if (v1->varno == v2->varno &&
+ v1->varattno == v2->varattno &&
+ v1->varlevelsup == v2->varlevelsup)
+ return true;
+ }
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ if (expr && IsA(expr, PlaceHolderVar))
+ {
+ PlaceHolderVar *v1 = (PlaceHolderVar *) node;
+ PlaceHolderVar *v2 = (PlaceHolderVar *) expr;
+
+ if (v1->phid == v2->phid)
+ return true;
+ }
+ }
+ else if (equal(node, expr))
return true;
}
return false;
@@ -1124,6 +1157,9 @@ generate_base_implied_equalities(PlannerInfo *root)
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL)
+ continue; /* must be an outer join */
+
Assert(rel->reloptkind == RELOPT_BASEREL);
rel->eclass_indexes = bms_add_member(rel->eclass_indexes,
@@ -2014,10 +2050,12 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* Process the FULL JOIN clauses */
foreach(cell, root->full_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ FullJoinClauseInfo *fjinfo = (FullJoinClauseInfo *) lfirst(cell);
- if (reconsider_full_join_clause(root, rinfo))
+ if (reconsider_full_join_clause(root, fjinfo))
{
+ RestrictInfo *rinfo = fjinfo->rinfo;
+
found = true;
/* remove it from the list */
root->full_join_clauses =
@@ -2046,9 +2084,9 @@ reconsider_outer_join_clauses(PlannerInfo *root)
}
foreach(cell, root->full_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ FullJoinClauseInfo *fjinfo = (FullJoinClauseInfo *) lfirst(cell);
- distribute_restrictinfo_to_rels(root, rinfo);
+ distribute_restrictinfo_to_rels(root, fjinfo->rinfo);
}
}
@@ -2184,8 +2222,11 @@ reconsider_outer_join_clause(PlannerInfo *root, RestrictInfo *rinfo,
* Returns true if we were able to propagate a constant through the clause.
*/
static bool
-reconsider_full_join_clause(PlannerInfo *root, RestrictInfo *rinfo)
+reconsider_full_join_clause(PlannerInfo *root, FullJoinClauseInfo *fjinfo)
{
+ RestrictInfo *rinfo = fjinfo->rinfo;
+ SpecialJoinInfo *sjinfo = fjinfo->sjinfo;
+ Relids fjrelids = bms_make_singleton(sjinfo->ojrelid);
Expr *leftvar;
Expr *rightvar;
Oid opno,
@@ -2267,6 +2308,18 @@ reconsider_full_join_clause(PlannerInfo *root, RestrictInfo *rinfo)
cfirst = (Node *) linitial(cexpr->args);
csecond = (Node *) lsecond(cexpr->args);
+ /*
+ * The COALESCE arguments will be marked as possibly nulled by
+ * the full join, while we wish to generate clauses that apply
+ * to the join's inputs. So we must strip the join from the
+ * nullingrels fields of cfirst/csecond before comparing them
+ * to leftvar/rightvar. (Perhaps with a less hokey
+ * representation for FULL JOIN USING output columns, this
+ * wouldn't be needed?)
+ */
+ cfirst = remove_nulling_relids(cfirst, fjrelids, NULL);
+ csecond = remove_nulling_relids(csecond, fjrelids, NULL);
+
if (equal(leftvar, cfirst) && equal(rightvar, csecond))
{
coal_idx = foreach_current_index(lc2);
@@ -3203,6 +3256,8 @@ get_eclass_indexes_for_relids(PlannerInfo *root, Relids relids)
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL)
+ continue; /* must be an outer join */
ec_indexes = bms_add_members(ec_indexes, rel->eclass_indexes);
}
return ec_indexes;
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index 0ef70ad7f1..ba451f8952 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -3357,13 +3357,13 @@ check_index_predicates(PlannerInfo *root, RelOptInfo *rel)
* Add on any equivalence-derivable join clauses. Computing the correct
* relid sets for generate_join_implied_equalities is slightly tricky
* because the rel could be a child rel rather than a true baserel, and in
- * that case we must remove its parents' relid(s) from all_baserels.
+ * that case we must subtract its parents' relid(s) from all_query_rels.
*/
if (rel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- otherrels = bms_difference(root->all_baserels,
+ otherrels = bms_difference(root->all_query_rels,
find_childrel_parents(root, rel));
else
- otherrels = bms_difference(root->all_baserels, rel->relids);
+ otherrels = bms_difference(root->all_query_rels, rel->relids);
if (!bms_is_empty(otherrels))
clauselist =
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 2a3f0ab7bf..a1fc72c394 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -250,7 +250,7 @@ add_paths_to_joinrel(PlannerInfo *root,
if (bms_overlap(joinrelids, sjinfo2->min_righthand) &&
!bms_overlap(joinrelids, sjinfo2->min_lefthand))
extra.param_source_rels = bms_join(extra.param_source_rels,
- bms_difference(root->all_baserels,
+ bms_difference(root->all_query_rels,
sjinfo2->min_righthand));
/* full joins constrain both sides symmetrically */
@@ -258,7 +258,7 @@ add_paths_to_joinrel(PlannerInfo *root,
bms_overlap(joinrelids, sjinfo2->min_lefthand) &&
!bms_overlap(joinrelids, sjinfo2->min_righthand))
extra.param_source_rels = bms_join(extra.param_source_rels,
- bms_difference(root->all_baserels,
+ bms_difference(root->all_query_rels,
sjinfo2->min_lefthand));
}
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 9da3ff2f9a..b64c37f089 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -353,7 +353,10 @@ make_rels_by_clauseless_joins(PlannerInfo *root,
*
* Caller must supply not only the two rels, but the union of their relids.
* (We could simplify the API by computing joinrelids locally, but this
- * would be redundant work in the normal path through make_join_rel.)
+ * would be redundant work in the normal path through make_join_rel.
+ * Note that this value does NOT include the RT index of any outer join that
+ * might need to be performed here, so it's not the canonical identifier
+ * of the join relation.)
*
* On success, *sjinfo_p is set to NULL if this is to be a plain inner join,
* else it's set to point to the associated SpecialJoinInfo node. Also,
@@ -695,7 +698,7 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(rel1->relids, rel2->relids));
- /* Construct Relids set that identifies the joinrel. */
+ /* Construct Relids set that identifies the joinrel (without OJ as yet). */
joinrelids = bms_union(rel1->relids, rel2->relids);
/* Check validity and determine join type. */
@@ -707,6 +710,10 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
return NULL;
}
+ /* If we have an outer join, add its RTI to form the canonical relids. */
+ if (sjinfo && sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
+
/* Swap rels if needed to match the join info. */
if (reversed)
{
@@ -730,7 +737,9 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
sjinfo->syn_lefthand = rel1->relids;
sjinfo->syn_righthand = rel2->relids;
sjinfo->jointype = JOIN_INNER;
+ sjinfo->ojrelid = 0;
/* we don't bother trying to make the remaining fields valid */
+ sjinfo->strict_relids = NULL;
sjinfo->lhs_strict = false;
sjinfo->delay_upper_joins = false;
sjinfo->semi_can_btree = false;
@@ -1510,8 +1519,6 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(child_rel1->relids, child_rel2->relids));
- child_joinrelids = bms_union(child_rel1->relids, child_rel2->relids);
- appinfos = find_appinfos_by_relids(root, child_joinrelids, &nappinfos);
/*
* Construct SpecialJoinInfo from parent join relations's
@@ -1521,6 +1528,15 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
child_rel1->relids,
child_rel2->relids);
+ /* Build correct join relids for child join */
+ child_joinrelids = bms_union(child_rel1->relids, child_rel2->relids);
+ if (child_sjinfo->ojrelid != 0)
+ child_joinrelids = bms_add_member(child_joinrelids,
+ child_sjinfo->ojrelid);
+
+ /* Find the AppendRelInfo structures */
+ appinfos = find_appinfos_by_relids(root, child_joinrelids, &nappinfos);
+
/*
* Construct restrictions applicable to the child join from those
* applicable to the parent join.
@@ -1536,8 +1552,7 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
{
child_joinrel = build_child_join_rel(root, child_rel1, child_rel2,
joinrel, child_restrictlist,
- child_sjinfo,
- child_sjinfo->jointype);
+ child_sjinfo);
joinrel->part_rels[cnt_parts] = child_joinrel;
joinrel->live_parts = bms_add_member(joinrel->live_parts, cnt_parts);
joinrel->all_partrels = bms_add_members(joinrel->all_partrels,
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index 337f470d58..bbe31c03fe 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -34,7 +34,7 @@
/* local functions */
static bool join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo);
-static void remove_rel_from_query(PlannerInfo *root, int relid,
+static void remove_rel_from_query(PlannerInfo *root, int relid, int ojrelid,
Relids joinrelids);
static List *remove_rel_from_joinlist(List *joinlist, int relid, int *nremoved);
static bool rel_supports_distinctness(PlannerInfo *root, RelOptInfo *rel);
@@ -70,6 +70,7 @@ restart:
foreach(lc, root->join_info_list)
{
SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+ Relids joinrelids;
int innerrelid;
int nremoved;
@@ -84,9 +85,12 @@ restart:
*/
innerrelid = bms_singleton_member(sjinfo->min_righthand);
- remove_rel_from_query(root, innerrelid,
- bms_union(sjinfo->min_lefthand,
- sjinfo->min_righthand));
+ /* Compute the relid set for the join we are considering */
+ joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ if (sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
+
+ remove_rel_from_query(root, innerrelid, sjinfo->ojrelid, joinrelids);
/* We verify that exactly one reference gets removed from joinlist */
nremoved = 0;
@@ -188,6 +192,8 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
/* Compute the relid set for the join we are considering */
joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ if (sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
/*
* We can't remove the join if any inner-rel attributes are used above the
@@ -306,10 +312,12 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
* no longer treated as a baserel, and that attributes of other baserels
* are no longer marked as being needed at joins involving this rel.
* Also, join quals involving the rel have to be removed from the joininfo
- * lists, but only if they belong to the outer join identified by joinrelids.
+ * lists, but only if they belong to the outer join identified by ojrelid
+ * and joinrelids.
*/
static void
-remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
+remove_rel_from_query(PlannerInfo *root, int relid, int ojrelid,
+ Relids joinrelids)
{
RelOptInfo *rel = find_base_rel(root, relid);
List *joininfos;
@@ -349,6 +357,13 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
}
}
+ /*
+ * The removed outer join has to be dropped from root->outer_join_rels.
+ * (We'd need to update all_baserels and all_query_rels too, but those
+ * haven't been computed yet.)
+ */
+ root->outer_join_rels = bms_del_member(root->outer_join_rels, ojrelid);
+
/*
* Likewise remove references from SpecialJoinInfo data structures.
*
@@ -365,6 +380,10 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
sjinfo->min_righthand = bms_del_member(sjinfo->min_righthand, relid);
sjinfo->syn_lefthand = bms_del_member(sjinfo->syn_lefthand, relid);
sjinfo->syn_righthand = bms_del_member(sjinfo->syn_righthand, relid);
+ sjinfo->min_lefthand = bms_del_member(sjinfo->min_lefthand, ojrelid);
+ sjinfo->min_righthand = bms_del_member(sjinfo->min_righthand, ojrelid);
+ sjinfo->syn_lefthand = bms_del_member(sjinfo->syn_lefthand, ojrelid);
+ sjinfo->syn_righthand = bms_del_member(sjinfo->syn_righthand, ojrelid);
}
/*
@@ -393,8 +412,10 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
else
{
phinfo->ph_eval_at = bms_del_member(phinfo->ph_eval_at, relid);
+ phinfo->ph_eval_at = bms_del_member(phinfo->ph_eval_at, ojrelid);
Assert(!bms_is_empty(phinfo->ph_eval_at));
phinfo->ph_needed = bms_del_member(phinfo->ph_needed, relid);
+ phinfo->ph_needed = bms_del_member(phinfo->ph_needed, ojrelid);
}
}
@@ -431,6 +452,8 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
rinfo->required_relids = bms_copy(rinfo->required_relids);
rinfo->required_relids = bms_del_member(rinfo->required_relids,
relid);
+ rinfo->required_relids = bms_del_member(rinfo->required_relids,
+ ojrelid);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
@@ -545,6 +568,7 @@ reduce_unique_semijoins(PlannerInfo *root)
/* Compute the relid set for the join we are considering */
joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ Assert(sjinfo->ojrelid == 0); /* SEMI joins don't have RT indexes */
/*
* Since we're only considering a single-rel RHS, any join clauses it
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 023efbaf09..b438085af6 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -60,12 +60,15 @@ static void process_security_barrier_quals(PlannerInfo *root,
static SpecialJoinInfo *make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
- JoinType jointype, List *clause);
+ JoinType jointype, Index ojrelid,
+ List *clause);
static void compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo,
List *clause);
+static List *remove_unneeded_nulling_relids(PlannerInfo *root, List *quals,
+ SpecialJoinInfo *sjinfo);
static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool below_outer_join,
- JoinType jointype,
+ SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
Relids ojscope,
@@ -250,10 +253,16 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
attno -= rel->min_attr;
if (rel->attr_needed[attno] == NULL)
{
- /* Variable not yet requested, so add to rel's targetlist */
- /* XXX is copyObject necessary here? */
- rel->reltarget->exprs = lappend(rel->reltarget->exprs,
- copyObject(var));
+ /*
+ * Variable not yet requested, so add to rel's targetlist.
+ *
+ * The value available at the rel's scan level has not been
+ * nulled by any outer join, so drop its varnullingrels.
+ * (We'll put those back as we climb up the join tree.)
+ */
+ var = copyObject(var);
+ var->varnullingrels = NULL;
+ rel->reltarget->exprs = lappend(rel->reltarget->exprs, var);
/* reltarget cost and width will be computed later */
}
rel->attr_needed[attno] = bms_add_members(rel->attr_needed[attno],
@@ -551,8 +560,10 @@ create_lateral_join_info(PlannerInfo *root)
varno = -1;
while ((varno = bms_next_member(eval_at, varno)) >= 0)
{
- RelOptInfo *brel = find_base_rel(root, varno);
+ RelOptInfo *brel = find_base_rel_ignore_join(root, varno);
+ if (brel == NULL)
+ continue; /* ignore outer joins in eval_at */
brel->lateral_relids = bms_add_members(brel->lateral_relids,
phinfo->ph_lateral);
}
@@ -643,7 +654,10 @@ create_lateral_join_info(PlannerInfo *root)
{
RelOptInfo *brel2 = root->simple_rel_array[rti2];
- Assert(brel2 != NULL && brel2->reloptkind == RELOPT_BASEREL);
+ if (brel2 == NULL)
+ continue; /* must be an OJ */
+
+ Assert(brel2->reloptkind == RELOPT_BASEREL);
brel2->lateral_referencers =
bms_add_member(brel2->lateral_referencers, rti);
}
@@ -695,7 +709,8 @@ deconstruct_jointree(PlannerInfo *root)
Assert(root->parse->jointree != NULL &&
IsA(root->parse->jointree, FromExpr));
- /* this is filled as we scan the jointree */
+ /* These are filled as we scan the jointree */
+ root->outer_join_rels = NULL;
root->nullable_baserels = NULL;
result = deconstruct_recurse(root, (Node *) root->parse->jointree, false,
@@ -717,7 +732,7 @@ deconstruct_jointree(PlannerInfo *root)
* below_outer_join is true if this node is within the nullable side of a
* higher-level outer join
* Outputs:
- * *qualscope gets the set of base Relids syntactically included in this
+ * *qualscope gets the set of base+OJ Relids syntactically included in this
* jointree node (do not modify or free this, as it may also be pointed
* to by RestrictInfo and SpecialJoinInfo nodes)
* *inner_join_rels gets the set of base Relids syntactically included in
@@ -802,6 +817,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
* there was exactly one element, we should (and already did) report
* whatever its inner_join_rels were. If there were no elements (is
* that still possible?) the initialization before the loop fixed it.
+ *
+ * XXX now wrong, do we care?
*/
if (list_length(f->fromlist) > 1)
*inner_join_rels = *qualscope;
@@ -816,7 +833,7 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
if (bms_is_subset(pq->relids, *qualscope))
distribute_qual_to_rels(root, pq->qual,
- below_outer_join, JOIN_INNER,
+ below_outer_join, NULL,
root->qual_security_level,
*qualscope, NULL, NULL,
NULL);
@@ -832,7 +849,7 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
Node *qual = (Node *) lfirst(l);
distribute_qual_to_rels(root, qual,
- below_outer_join, JOIN_INNER,
+ below_outer_join, NULL,
root->qual_security_level,
*qualscope, NULL, NULL,
postponed_qual_list);
@@ -896,6 +913,13 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ /* caution: ANTI join derived from SEMI will lack rtindex */
+ if (j->rtindex != 0)
+ {
+ *qualscope = bms_add_member(*qualscope, j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
+ j->rtindex);
+ }
*inner_join_rels = bms_union(left_inners, right_inners);
nonnullable_rels = leftids;
nullable_rels = rightids;
@@ -910,6 +934,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ /* SEMI join never has rtindex, so don't add to qualscope */
+ Assert(j->rtindex == 0);
*inner_join_rels = bms_union(left_inners, right_inners);
/* Semi join adds no restrictions for quals */
nonnullable_rels = NULL;
@@ -931,6 +957,10 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ Assert(j->rtindex != 0);
+ *qualscope = bms_add_member(*qualscope, j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
+ j->rtindex);
*inner_join_rels = bms_union(left_inners, right_inners);
/* each side is both outer and inner */
nonnullable_rels = *qualscope;
@@ -976,32 +1006,44 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
my_quals = list_concat(my_quals, (List *) j->quals);
/*
- * For an OJ, form the SpecialJoinInfo now, because we need the OJ's
- * semantic scope (ojscope) to pass to distribute_qual_to_rels. But
- * we mustn't add it to join_info_list just yet, because we don't want
- * distribute_qual_to_rels to think it is an outer join below us.
- *
- * Semijoins are a bit of a hybrid: we build a SpecialJoinInfo, but we
- * want ojscope = NULL for distribute_qual_to_rels.
+ * For an OJ, form the SpecialJoinInfo now, because we need it for
+ * distribute_qual_to_rels. But we mustn't add it to join_info_list
+ * just yet, because we don't want distribute_qual_to_rels to think it
+ * is an outer join below us.
*/
if (j->jointype != JOIN_INNER)
- {
sjinfo = make_outerjoininfo(root,
leftids, rightids,
*inner_join_rels,
j->jointype,
+ j->rtindex,
my_quals);
- if (j->jointype == JOIN_SEMI)
- ojscope = NULL;
- else
- ojscope = bms_union(sjinfo->min_lefthand,
- sjinfo->min_righthand);
- }
else
- {
sjinfo = NULL;
+
+ /*
+ * If we have a LEFT JOIN whose ON qual is strict for any LHS
+ * relations, we may be able to commute the join with lower outer
+ * joins that null those relations. To do that, we must remove such
+ * lower outer joins from Var.varnullingrels fields within the qual,
+ * else subsequent processing will think that the qual has to be
+ * evaluated above such lower outer joins.
+ */
+ if (j->jointype == JOIN_LEFT && sjinfo->lhs_strict)
+ my_quals = remove_unneeded_nulling_relids(root, my_quals, sjinfo);
+
+ /*
+ * Now we can compute ojscope (we can't do it earlier, because
+ * remove_unneeded_nulling_relids might change the scope).
+ *
+ * Semijoins are a bit of a hybrid: we build a SpecialJoinInfo, but we
+ * want ojscope = NULL for distribute_qual_to_rels.
+ */
+ if (j->jointype == JOIN_INNER || j->jointype == JOIN_SEMI)
ojscope = NULL;
- }
+ else
+ ojscope = bms_union(sjinfo->min_lefthand,
+ sjinfo->min_righthand);
/* Process the JOIN's qual clauses */
foreach(l, my_quals)
@@ -1009,7 +1051,7 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
Node *qual = (Node *) lfirst(l);
distribute_qual_to_rels(root, qual,
- below_outer_join, j->jointype,
+ below_outer_join, sjinfo,
root->qual_security_level,
*qualscope,
ojscope, nonnullable_rels,
@@ -1112,7 +1154,7 @@ process_security_barrier_quals(PlannerInfo *root,
*/
distribute_qual_to_rels(root, qual,
below_outer_join,
- JOIN_INNER,
+ NULL,
security_level,
qualscope,
qualscope,
@@ -1135,6 +1177,7 @@ process_security_barrier_quals(PlannerInfo *root,
* right_rels: the base Relids syntactically on inner side of join
* inner_join_rels: base Relids participating in inner joins below this one
* jointype: what it says (must always be LEFT, FULL, SEMI, or ANTI)
+ * ojrelid: RT index of the join RTE (0 for SEMI, which isn't in the RT list)
* clause: the outer join's join condition (in implicit-AND format)
*
* The node should eventually be appended to root->join_info_list, but we
@@ -1148,7 +1191,8 @@ static SpecialJoinInfo *
make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
- JoinType jointype, List *clause)
+ JoinType jointype, Index ojrelid,
+ List *clause)
{
SpecialJoinInfo *sjinfo = makeNode(SpecialJoinInfo);
Relids clause_relids;
@@ -1196,6 +1240,7 @@ make_outerjoininfo(PlannerInfo *root,
sjinfo->syn_lefthand = left_rels;
sjinfo->syn_righthand = right_rels;
sjinfo->jointype = jointype;
+ sjinfo->ojrelid = ojrelid;
/* this always starts out false */
sjinfo->delay_upper_joins = false;
@@ -1206,6 +1251,7 @@ make_outerjoininfo(PlannerInfo *root,
{
sjinfo->min_lefthand = bms_copy(left_rels);
sjinfo->min_righthand = bms_copy(right_rels);
+ sjinfo->strict_relids = NULL; /* don't care about this */
sjinfo->lhs_strict = false; /* don't care about this */
return sjinfo;
}
@@ -1220,6 +1266,7 @@ make_outerjoininfo(PlannerInfo *root,
* rel's columns are all NULL?
*/
strict_relids = find_nonnullable_rels((Node *) clause);
+ sjinfo->strict_relids = strict_relids;
/* Remember whether the clause is strict for any LHS relations */
sjinfo->lhs_strict = bms_overlap(strict_relids, left_rels);
@@ -1258,6 +1305,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_lefthand = bms_add_member(min_lefthand,
+ otherinfo->ojrelid);
}
if (bms_overlap(right_rels, otherinfo->syn_lefthand) ||
bms_overlap(right_rels, otherinfo->syn_righthand))
@@ -1266,6 +1316,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_righthand = bms_add_member(min_righthand,
+ otherinfo->ojrelid);
}
/* Needn't do anything else with the full join */
continue;
@@ -1295,6 +1348,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_lefthand = bms_add_member(min_lefthand,
+ otherinfo->ojrelid);
}
}
@@ -1337,6 +1393,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_righthand = bms_add_member(min_righthand,
+ otherinfo->ojrelid);
}
}
}
@@ -1561,6 +1620,62 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
sjinfo->semi_rhs_exprs = semi_rhs_exprs;
}
+/*
+ * remove_unneeded_nulling_relids
+ * Remove lower outer joins from Vars (& PHVs) in the quals, if possible
+ *
+ * This paves the way to apply outer join identity 3 to commute the current
+ * LEFT JOIN with lower outer joins. We already know that the quals are
+ * strict for at least one LHS relation.
+ */
+static List *
+remove_unneeded_nulling_relids(PlannerInfo *root, List *quals,
+ SpecialJoinInfo *sjinfo)
+{
+ Relids old_nulling_relids;
+ Relids removable_relids;
+ ListCell *lc;
+
+ /*
+ * Find outer joins mentioned in nullingrel fields in the quals. If there
+ * aren't any (the common case), there's no need to work hard.
+ */
+ old_nulling_relids = get_nulling_relids((Node *) quals);
+ if (bms_is_empty(old_nulling_relids))
+ return quals;
+
+ /*
+ * Thumb through the existing SpecialJoinInfos (which describe all outer
+ * joins below this one, but not yet this one) to find the ones mentioned
+ * in the quals. If the current join's quals are strict for any rel of
+ * one's RHS, we can commute this join with that one, so remove it from
+ * the current join's min_lefthand and from the quals' nullingrel fields.
+ */
+ removable_relids = NULL;
+ foreach(lc, root->join_info_list)
+ {
+ SpecialJoinInfo *sjinfo2 = (SpecialJoinInfo *) lfirst(lc);
+
+ if (sjinfo2->jointype != JOIN_LEFT ||
+ !bms_is_member(sjinfo2->ojrelid, old_nulling_relids))
+ continue; /* it's not relevant */
+ if (bms_is_subset(sjinfo2->syn_righthand, sjinfo->syn_lefthand) &&
+ bms_overlap(sjinfo->strict_relids, sjinfo2->min_righthand))
+ {
+ sjinfo->min_lefthand = bms_del_member(sjinfo->min_lefthand,
+ sjinfo2->ojrelid);
+ removable_relids = bms_add_member(removable_relids,
+ sjinfo2->ojrelid);
+ }
+ }
+
+ if (removable_relids == NULL)
+ return quals; /* no hits, nothing to do */
+
+ return (List *) remove_nulling_relids((Node *) quals,
+ removable_relids, NULL);
+}
+
/*****************************************************************************
*
@@ -1582,7 +1697,7 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
* 'clause': the qual clause to be distributed
* 'below_outer_join': true if the qual is from a JOIN/ON that is below the
* nullable side of a higher-level outer join
- * 'jointype': type of join the qual is from (JOIN_INNER for a WHERE clause)
+ * 'sjinfo': join's SpecialJoinInfo (NULL for an inner join or WHERE clause)
* 'security_level': security_level to assign to the qual
* 'qualscope': set of baserels the qual's syntactic scope covers
* 'ojscope': NULL if not an outer-join qual, else the minimum set of baserels
@@ -1600,12 +1715,13 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
* level, which will be ojscope not necessarily qualscope.
*
* At the time this is called, root->join_info_list must contain entries for
- * all and only those special joins that are syntactically below this qual.
+ * all and only those special joins that are syntactically below this qual;
+ * in particular, the passed-in SpecialJoinInfo isn't yet in that list.
*/
static void
distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool below_outer_join,
- JoinType jointype,
+ SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
Relids ojscope,
@@ -1642,7 +1758,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
PostponedQual *pq = (PostponedQual *) palloc(sizeof(PostponedQual));
Assert(root->hasLateralRTEs); /* shouldn't happen otherwise */
- Assert(jointype == JOIN_INNER); /* mustn't postpone past outer join */
+ Assert(sjinfo == NULL); /* mustn't postpone past outer join */
pq->qual = clause;
pq->relids = relids;
*postponed_qual_list = lappend(*postponed_qual_list, pq);
@@ -1704,7 +1820,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
{
relids =
get_relids_in_jointree((Node *) root->parse->jointree,
- false);
+ true, false);
qualscope = bms_copy(relids);
}
}
@@ -1946,11 +2062,15 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
restrictinfo);
return;
}
- if (jointype == JOIN_FULL)
+ if (sjinfo && sjinfo->jointype == JOIN_FULL)
{
/* FULL JOIN (above tests cannot match in this case) */
+ FullJoinClauseInfo *fjinfo = makeNode(FullJoinClauseInfo);
+
+ fjinfo->rinfo = restrictinfo;
+ fjinfo->sjinfo = sjinfo;
root->full_join_clauses = lappend(root->full_join_clauses,
- restrictinfo);
+ fjinfo);
return;
}
/* nope, so fall through to distribute_restrictinfo_to_rels */
@@ -2344,7 +2464,7 @@ process_implied_equality(PlannerInfo *root,
{
relids =
get_relids_in_jointree((Node *) root->parse->jointree,
- false);
+ true, false);
}
}
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index f6baa2a765..ea077cff4e 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2221,7 +2221,7 @@ preprocess_rowmarks(PlannerInfo *root)
* make a bitmapset of all base rels and then remove the items we don't
* need or have FOR [KEY] UPDATE/SHARE marks for.
*/
- rels = get_relids_in_jointree((Node *) parse->jointree, false);
+ rels = get_relids_in_jointree((Node *) parse->jointree, false, false);
if (parse->resultRelation)
rels = bms_del_member(rels, parse->resultRelation);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 9cef92cab2..14f5c0f897 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -151,6 +151,9 @@ static Var *search_indexed_tlist_for_var(Var *var,
indexed_tlist *itlist,
int newvarno,
int rtoffset);
+static Var *search_indexed_tlist_for_phv(PlaceHolderVar *phv,
+ indexed_tlist *itlist,
+ int newvarno);
static Var *search_indexed_tlist_for_non_var(Expr *node,
indexed_tlist *itlist,
int newvarno);
@@ -2108,6 +2111,7 @@ fix_scan_expr_mutator(Node *node, fix_scan_expr_context *context)
/* At scan level, we should always just evaluate the contained expr */
PlaceHolderVar *phv = (PlaceHolderVar *) node;
+ Assert(phv->phnullingrels == NULL);
return fix_scan_expr_mutator((Node *) phv->phexpr, context);
}
if (IsA(node, AlternativeSubPlan))
@@ -2228,33 +2232,12 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
/*
* Now we need to fix up the targetlist and qpqual, which are logically
* above the join. This means they should not re-use any input expression
- * that was computed in the nullable side of an outer join. Vars and
- * PlaceHolderVars are fine, so we can implement this restriction just by
- * clearing has_non_vars in the indexed_tlist structs.
+ * that was computed in the nullable side of an outer join.
*
- * XXX This is a grotty workaround for the fact that we don't clearly
- * distinguish between a Var appearing below an outer join and the "same"
- * Var appearing above it. If we did, we'd not need to hack the matching
- * rules this way.
+ * XXX we will probably need to pass some flag down to indicate that this
+ * context applies, so that search_indexed_tlist_for_var() and siblings
+ * can correctly check for varnullingrels matches.
*/
- switch (join->jointype)
- {
- case JOIN_LEFT:
- case JOIN_SEMI:
- case JOIN_ANTI:
- inner_itlist->has_non_vars = false;
- break;
- case JOIN_RIGHT:
- outer_itlist->has_non_vars = false;
- break;
- case JOIN_FULL:
- outer_itlist->has_non_vars = false;
- inner_itlist->has_non_vars = false;
- break;
- default:
- break;
- }
-
join->plan.targetlist = fix_join_expr(root,
join->plan.targetlist,
outer_itlist,
@@ -2543,7 +2526,7 @@ set_dummy_tlist_references(Plan *plan, int rtoffset)
* tlist_member() searches.
*
* The result of this function is an indexed_tlist struct to pass to
- * search_indexed_tlist_for_var() or search_indexed_tlist_for_non_var().
+ * search_indexed_tlist_for_var() and siblings.
* When done, the indexed_tlist may be freed with a single pfree().
*/
static indexed_tlist *
@@ -2665,6 +2648,8 @@ search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
/* Found a match */
Var *newvar = copyVar(var);
+ /* XXX we oughta check varnullingrels match here ... */
+
newvar->varno = newvarno;
newvar->varattno = vinfo->resno;
if (newvar->varnosyn > 0)
@@ -2677,15 +2662,55 @@ search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
}
/*
- * search_indexed_tlist_for_non_var --- find a non-Var in an indexed tlist
+ * search_indexed_tlist_for_phv --- find a PlaceHolderVar in an indexed tlist
*
* If a match is found, return a Var constructed to reference the tlist item.
* If no match, return NULL.
*
- * NOTE: it is a waste of time to call this unless itlist->has_ph_vars or
- * itlist->has_non_vars. Furthermore, set_join_references() relies on being
- * able to prevent matching of non-Vars by clearing itlist->has_non_vars,
- * so there's a correctness reason not to call it unless that's set.
+ * NOTE: it is a waste of time to call this unless itlist->has_ph_vars.
+ */
+static Var *
+search_indexed_tlist_for_phv(PlaceHolderVar *phv,
+ indexed_tlist *itlist, int newvarno)
+{
+ ListCell *lc;
+
+ foreach(lc, itlist->tlist)
+ {
+ TargetEntry *tle = (TargetEntry *) lfirst(lc);
+
+ if (tle->expr && IsA(tle->expr, PlaceHolderVar))
+ {
+ PlaceHolderVar *subphv = (PlaceHolderVar *) tle->expr;
+ Var *newvar;
+
+ /*
+ * Analogously to search_indexed_tlist_for_var, we match on phid
+ * only. We don't use equal(), partially for speed but mostly
+ * because phnullingrels might not be exactly equal.
+ *
+ * XXX we really oughta verify phnullingrels.
+ */
+ if (phv->phid != subphv->phid)
+ continue;
+
+ /* Found a matching subplan output expression */
+ newvar = makeVarFromTargetEntry(newvarno, tle);
+ newvar->varnosyn = 0; /* wasn't ever a plain Var */
+ newvar->varattnosyn = 0;
+ return newvar;
+ }
+ }
+ return NULL; /* no match */
+}
+
+/*
+ * search_indexed_tlist_for_non_var --- find a non-Var/PHV in an indexed tlist
+ *
+ * If a match is found, return a Var constructed to reference the tlist item.
+ * If no match, return NULL.
+ *
+ * NOTE: it is a waste of time to call this unless itlist->has_non_vars.
*/
static Var *
search_indexed_tlist_for_non_var(Expr *node,
@@ -2870,22 +2895,23 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* See if the PlaceHolderVar has bubbled up from a lower plan node */
if (context->outer_itlist && context->outer_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->outer_itlist,
- OUTER_VAR);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->outer_itlist,
+ OUTER_VAR);
if (newvar)
return (Node *) newvar;
}
if (context->inner_itlist && context->inner_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->inner_itlist,
- INNER_VAR);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->inner_itlist,
+ INNER_VAR);
if (newvar)
return (Node *) newvar;
}
/* If not supplied by input plans, evaluate the contained expr */
+ /* XXX assert something about phnullingrels */
return fix_join_expr_mutator((Node *) phv->phexpr, context);
}
/* Try matching more complex expressions too, if tlists have any */
@@ -2994,13 +3020,14 @@ fix_upper_expr_mutator(Node *node, fix_upper_expr_context *context)
/* See if the PlaceHolderVar has bubbled up from a lower plan node */
if (context->subplan_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->subplan_itlist,
- context->newvarno);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->subplan_itlist,
+ context->newvarno);
if (newvar)
return (Node *) newvar;
}
/* If not supplied by input plan, evaluate the contained expr */
+ /* XXX assert something about phnullingrels */
return fix_upper_expr_mutator((Node *) phv->phexpr, context);
}
/* Try matching more complex expressions too, if tlist has any */
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 0bd99acf83..389f7d9ce7 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -49,17 +49,28 @@ typedef struct pullup_replace_vars_context
* pullup (set only if target_rte->lateral) */
bool *outer_hasSubLinks; /* -> outer query's hasSubLinks */
int varno; /* varno of subquery */
- bool need_phvs; /* do we need PlaceHolderVars? */
- bool wrap_non_vars; /* do we need 'em on *all* non-Vars? */
+ bool wrap_non_vars; /* do we need all non-Var outputs to be PHVs? */
Node **rv_cache; /* cache for results with PHVs */
} pullup_replace_vars_context;
-typedef struct reduce_outer_joins_state
+typedef struct reduce_outer_joins_pass1_state
{
Relids relids; /* base relids within this subtree */
bool contains_outer; /* does subtree contain outer join(s)? */
List *sub_states; /* List of states for subtree components */
-} reduce_outer_joins_state;
+} reduce_outer_joins_pass1_state;
+
+typedef struct reduce_outer_joins_pass2_state
+{
+ Relids inner_reduced; /* OJ relids reduced to plain inner joins */
+ List *partial_reduced; /* List of partially reduced FULL joins */
+} reduce_outer_joins_pass2_state;
+
+typedef struct reduce_outer_joins_partial_state
+{
+ int full_join_rti; /* RT index of a formerly-FULL join */
+ Relids unreduced_side; /* relids in its still-nullable side */
+} reduce_outer_joins_partial_state;
static Node *pull_up_sublinks_jointree_recurse(PlannerInfo *root, Node *jtnode,
Relids *relids);
@@ -68,12 +79,10 @@ static Node *pull_up_sublinks_qual_recurse(PlannerInfo *root, Node *node,
Node **jtlink2, Relids available_rels2);
static Node *pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static Node *pull_up_simple_subquery(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static Node *pull_up_simple_union_all(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte);
@@ -90,7 +99,6 @@ static Node *pull_up_simple_values(PlannerInfo *root, Node *jtnode,
static bool is_simple_values(PlannerInfo *root, RangeTblEntry *rte);
static Node *pull_up_constant_function(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static bool is_simple_union_all(Query *subquery);
static bool is_simple_union_all_recurse(Node *setOp, Query *setOpQuery,
@@ -101,25 +109,27 @@ static bool jointree_contains_lateral_outer_refs(PlannerInfo *root,
Relids safe_upper_varnos);
static void perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars_context *rvcontext,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static void replace_vars_in_jointree(Node *jtnode,
- pullup_replace_vars_context *context,
- JoinExpr *lowest_nulling_outer_join);
+ pullup_replace_vars_context *context);
static Node *pullup_replace_vars(Node *expr,
pullup_replace_vars_context *context);
static Node *pullup_replace_vars_callback(Var *var,
replace_rte_variables_context *context);
static Query *pullup_replace_vars_subquery(Query *query,
pullup_replace_vars_context *context);
-static reduce_outer_joins_state *reduce_outer_joins_pass1(Node *jtnode);
+static reduce_outer_joins_pass1_state *reduce_outer_joins_pass1(Node *jtnode);
static void reduce_outer_joins_pass2(Node *jtnode,
- reduce_outer_joins_state *state,
+ reduce_outer_joins_pass1_state *state1,
+ reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
List *nonnullable_vars,
List *forced_null_vars);
-static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode);
+static void report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
+ int rtindex, Relids relids);
+static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Relids *dropped_outer_joins);
static int get_result_relid(PlannerInfo *root, Node *jtnode);
static void remove_result_refs(PlannerInfo *root, int varno, Node *newjtloc);
static bool find_dependent_phvs(PlannerInfo *root, int varno);
@@ -764,7 +774,7 @@ pull_up_subqueries(PlannerInfo *root)
/* Recursion starts with no containing join nor appendrel */
root->parse->jointree = (FromExpr *)
pull_up_subqueries_recurse(root, (Node *) root->parse->jointree,
- NULL, NULL, NULL);
+ NULL, NULL);
/* We should still have a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
}
@@ -779,12 +789,6 @@ pull_up_subqueries(PlannerInfo *root)
* lowest_outer_join references the lowest such JoinExpr node; otherwise
* it is NULL. We use this to constrain the effects of LATERAL subqueries.
*
- * If this jointree node is within the nullable side of an outer join, then
- * lowest_nulling_outer_join references the lowest such JoinExpr node;
- * otherwise it is NULL. This forces use of the PlaceHolderVar mechanism for
- * references to non-nullable targetlist items, but only for references above
- * that join.
- *
* If we are looking at a member subquery of an append relation,
* containing_appendrel describes that relation; else it is NULL.
* This forces use of the PlaceHolderVar mechanism for all non-Var targetlist
@@ -801,15 +805,14 @@ pull_up_subqueries(PlannerInfo *root)
* Notice also that we can't turn pullup_replace_vars loose on the whole
* jointree, because it'd return a mutated copy of the tree; we have to
* invoke it just on the quals, instead. This behavior is what makes it
- * reasonable to pass lowest_outer_join and lowest_nulling_outer_join as
- * pointers rather than some more-indirect way of identifying the lowest
- * OJs. Likewise, we don't replace append_rel_list members but only their
- * substructure, so the containing_appendrel reference is safe to use.
+ * reasonable to pass lowest_outer_join as a pointer rather than some
+ * more-indirect way of identifying the lowest OJ. Likewise, we don't
+ * replace append_rel_list members but only their substructure, so the
+ * containing_appendrel reference is safe to use.
*/
static Node *
pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Assert(jtnode != NULL);
@@ -831,7 +834,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
is_safe_append_member(rte->subquery)))
return pull_up_simple_subquery(root, jtnode, rte,
lowest_outer_join,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -864,7 +866,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
*/
if (rte->rtekind == RTE_FUNCTION)
return pull_up_constant_function(root, jtnode, rte,
- lowest_nulling_outer_join,
containing_appendrel);
/* Otherwise, do nothing at this node. */
@@ -880,7 +881,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
{
lfirst(l) = pull_up_subqueries_recurse(root, lfirst(l),
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
}
}
@@ -895,11 +895,9 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
case JOIN_INNER:
j->larg = pull_up_subqueries_recurse(root, j->larg,
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
break;
case JOIN_LEFT:
@@ -907,31 +905,25 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
case JOIN_ANTI:
j->larg = pull_up_subqueries_recurse(root, j->larg,
j,
- lowest_nulling_outer_join,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
- j,
j,
NULL);
break;
case JOIN_FULL:
j->larg = pull_up_subqueries_recurse(root, j->larg,
- j,
j,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
- j,
j,
NULL);
break;
case JOIN_RIGHT:
j->larg = pull_up_subqueries_recurse(root, j->larg,
- j,
j,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
j,
- lowest_nulling_outer_join,
NULL);
break;
default:
@@ -961,7 +953,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
static Node *
pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -1107,31 +1098,25 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* The subquery's targetlist items are now in the appropriate form to
* insert into the top query, except that we may need to wrap them in
* PlaceHolderVars. Set up required context data for pullup_replace_vars.
+ * (Note that we should include the subquery's inner joins in relids,
+ * since it may include join alias vars referencing them.)
*/
rvcontext.root = root;
rvcontext.targetlist = subquery->targetList;
rvcontext.target_rte = rte;
if (rte->lateral)
rvcontext.relids = get_relids_in_jointree((Node *) subquery->jointree,
- true);
+ true, true);
else /* won't need relids */
rvcontext.relids = NULL;
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = varno;
- /* these flags will be set below, if needed */
- rvcontext.need_phvs = false;
+ /* this flag will be set below, if needed */
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(subquery->targetList) + 1) *
sizeof(Node *));
- /*
- * If we are under an outer join then non-nullable items and lateral
- * references may have to be turned into PlaceHolderVars.
- */
- if (lowest_nulling_outer_join != NULL)
- rvcontext.need_phvs = true;
-
/*
* If we are dealing with an appendrel member then anything that's not a
* simple Var has to be turned into a PlaceHolderVar. We force this to
@@ -1140,10 +1125,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* expression actually available from the appendrel.
*/
if (containing_appendrel != NULL)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* If the parent query uses grouping sets, we need a PlaceHolderVar for
@@ -1155,10 +1137,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* that pullup_replace_vars hasn't currently got.)
*/
if (parse->groupingSets)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* Replace all of the top query's references to the subquery's outputs
@@ -1166,7 +1145,6 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* replace any of the jointree structure.
*/
perform_pullup_replace_vars(root, &rvcontext,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -1233,7 +1211,8 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
{
Relids subrelids;
- subrelids = get_relids_in_jointree((Node *) subquery->jointree, false);
+ subrelids = get_relids_in_jointree((Node *) subquery->jointree,
+ true, false);
substitute_phv_relids((Node *) parse, varno, subrelids);
fix_append_rel_relids(root->append_rel_list, varno, subrelids);
}
@@ -1424,7 +1403,7 @@ pull_up_union_leaf_queries(Node *setOp, PlannerInfo *root, int parentRTindex,
rtr = makeNode(RangeTblRef);
rtr->rtindex = childRTindex;
(void) pull_up_subqueries_recurse(root, (Node *) rtr,
- NULL, NULL, appinfo);
+ NULL, appinfo);
}
else if (IsA(setOp, SetOperationStmt))
{
@@ -1561,7 +1540,7 @@ is_simple_subquery(PlannerInfo *root, Query *subquery, RangeTblEntry *rte,
{
restricted = true;
safe_upper_varnos = get_relids_in_jointree((Node *) lowest_outer_join,
- true);
+ true, true);
}
else
{
@@ -1673,7 +1652,6 @@ pull_up_simple_values(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte)
rvcontext.relids = NULL;
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = varno;
- rvcontext.need_phvs = false;
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(tlist) + 1) *
@@ -1685,7 +1663,7 @@ pull_up_simple_values(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte)
* any of the jointree structure. We can assume there's no outer joins or
* appendrels in the dummy Query that surrounds a VALUES RTE.
*/
- perform_pullup_replace_vars(root, &rvcontext, NULL, NULL);
+ perform_pullup_replace_vars(root, &rvcontext, NULL);
/*
* There should be no appendrels to fix, nor any outer joins and hence no
@@ -1784,7 +1762,6 @@ is_simple_values(PlannerInfo *root, RangeTblEntry *rte)
static Node *
pull_up_constant_function(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -1836,40 +1813,26 @@ pull_up_constant_function(PlannerInfo *root, Node *jtnode,
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = ((RangeTblRef *) jtnode)->rtindex;
- /* these flags will be set below, if needed */
- rvcontext.need_phvs = false;
+ /* this flag will be set below, if needed */
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(rvcontext.targetlist) + 1) *
sizeof(Node *));
- /*
- * If we are under an outer join then non-nullable items and lateral
- * references may have to be turned into PlaceHolderVars.
- */
- if (lowest_nulling_outer_join != NULL)
- rvcontext.need_phvs = true;
-
/*
* If we are dealing with an appendrel member then anything that's not a
* simple Var has to be turned into a PlaceHolderVar. (See comments in
* pull_up_simple_subquery().)
*/
if (containing_appendrel != NULL)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* If the parent query uses grouping sets, we need a PlaceHolderVar for
* anything that's not a simple Var.
*/
if (parse->groupingSets)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* Replace all of the top query's references to the RTE's output with
@@ -1877,7 +1840,6 @@ pull_up_constant_function(PlannerInfo *root, Node *jtnode,
* jointree structure.
*/
perform_pullup_replace_vars(root, &rvcontext,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -2099,13 +2061,11 @@ jointree_contains_lateral_outer_refs(PlannerInfo *root, Node *jtnode,
*
* Caller has already filled *rvcontext with data describing what to
* substitute for Vars referencing the target subquery. In addition
- * we need the identity of the lowest outer join that can null the
- * target subquery, and its containing appendrel if any.
+ * we need the identity of the containing appendrel if any.
*/
static void
perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars_context *rvcontext,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -2149,38 +2109,31 @@ perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars((Node *) action->targetList, rvcontext);
}
}
- replace_vars_in_jointree((Node *) parse->jointree, rvcontext,
- lowest_nulling_outer_join);
+ replace_vars_in_jointree((Node *) parse->jointree, rvcontext);
Assert(parse->setOperations == NULL);
parse->havingQual = pullup_replace_vars(parse->havingQual, rvcontext);
/*
* Replace references in the translated_vars lists of appendrels. When
- * pulling up an appendrel member, we do not need PHVs in the list of the
- * parent appendrel --- there isn't any outer join between. Elsewhere,
- * use PHVs for safety. (This analysis could be made tighter but it seems
- * unlikely to be worth much trouble.)
+ * pulling up an appendrel member, we do not want to force PHVs in the
+ * list of the parent appendrel --- there isn't any outer join between.
+ * Elsewhere, use PHVs for safety. (This analysis could be made tighter
+ * but it seems unlikely to be worth much trouble.)
*/
foreach(lc, root->append_rel_list)
{
AppendRelInfo *appinfo = (AppendRelInfo *) lfirst(lc);
- bool save_need_phvs = rvcontext->need_phvs;
+ bool save_wrap_non_vars = rvcontext->wrap_non_vars;
if (appinfo == containing_appendrel)
- rvcontext->need_phvs = false;
+ rvcontext->wrap_non_vars = false;
appinfo->translated_vars = (List *)
pullup_replace_vars((Node *) appinfo->translated_vars, rvcontext);
- rvcontext->need_phvs = save_need_phvs;
+ rvcontext->wrap_non_vars = save_wrap_non_vars;
}
/*
* Replace references in the joinaliasvars lists of join RTEs.
- *
- * You might think that we could avoid using PHVs for alias vars of joins
- * below lowest_nulling_outer_join, but that doesn't work because the
- * alias vars could be referenced above that join; we need the PHVs to be
- * present in such references after the alias vars get flattened. (It
- * might be worth trying to be smarter here, someday.)
*/
foreach(lc, parse->rtable)
{
@@ -2197,14 +2150,10 @@ perform_pullup_replace_vars(PlannerInfo *root,
* Helper routine for perform_pullup_replace_vars: do pullup_replace_vars on
* every expression in the jointree, without changing the jointree structure
* itself. Ugly, but there's no other way...
- *
- * If we are at or below lowest_nulling_outer_join, we can suppress use of
- * PlaceHolderVars wrapped around the replacement expressions.
*/
static void
replace_vars_in_jointree(Node *jtnode,
- pullup_replace_vars_context *context,
- JoinExpr *lowest_nulling_outer_join)
+ pullup_replace_vars_context *context)
{
if (jtnode == NULL)
return;
@@ -2217,7 +2166,7 @@ replace_vars_in_jointree(Node *jtnode,
* jointree scan, rather than a scan of the rtable, for a couple of
* reasons: we can avoid processing no-longer-referenced RTEs, and we
* can use the appropriate setting of need_phvs depending on whether
- * the RTE is above possibly-nulling outer joins or not.
+ * the RTE is above possibly-nulling outer joins or not. XXX fix
*/
int varno = ((RangeTblRef *) jtnode)->rtindex;
@@ -2274,42 +2223,30 @@ replace_vars_in_jointree(Node *jtnode,
ListCell *l;
foreach(l, f->fromlist)
- replace_vars_in_jointree(lfirst(l), context,
- lowest_nulling_outer_join);
+ replace_vars_in_jointree(lfirst(l), context);
f->quals = pullup_replace_vars(f->quals, context);
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- bool save_need_phvs = context->need_phvs;
+ bool save_wrap_non_vars = context->wrap_non_vars;
- if (j == lowest_nulling_outer_join)
- {
- /* no more PHVs in or below this join */
- context->need_phvs = false;
- lowest_nulling_outer_join = NULL;
- }
- replace_vars_in_jointree(j->larg, context, lowest_nulling_outer_join);
- replace_vars_in_jointree(j->rarg, context, lowest_nulling_outer_join);
+ replace_vars_in_jointree(j->larg, context);
+ replace_vars_in_jointree(j->rarg, context);
/*
- * Use PHVs within the join quals of a full join, even when it's the
- * lowest nulling outer join. Otherwise, we cannot identify which
- * side of the join a pulled-up var-free expression came from, which
- * can lead to failure to make a plan at all because none of the quals
- * appear to be mergeable or hashable conditions. For this purpose we
- * don't care about the state of wrap_non_vars, so leave it alone.
+ * Use PHVs within the join quals of a full join. Otherwise, we
+ * cannot identify which side of the join a pulled-up var-free
+ * expression came from, which can lead to failure to make a plan at
+ * all because none of the quals appear to be mergeable or hashable
+ * conditions.
*/
if (j->jointype == JOIN_FULL)
- context->need_phvs = true;
+ context->wrap_non_vars = true;
j->quals = pullup_replace_vars(j->quals, context);
- /*
- * We don't bother to update the colvars list, since it won't be used
- * again ...
- */
- context->need_phvs = save_need_phvs;
+ context->wrap_non_vars = save_wrap_non_vars;
}
else
elog(ERROR, "unrecognized node type: %d",
@@ -2338,8 +2275,18 @@ pullup_replace_vars_callback(Var *var,
{
pullup_replace_vars_context *rcon = (pullup_replace_vars_context *) context->callback_arg;
int varattno = var->varattno;
+ bool need_phv;
Node *newnode;
+ /*
+ * We need a PlaceHolderVar if the Var-to-be-replaced has nonempty
+ * varnullingrels (unless we find below that the replacement expression is
+ * a Var or PlaceHolderVar that we can just add the nullingrels to). We
+ * also need one if the caller has instructed us that all non-Var/PHV
+ * replacements need to be wrapped for identification purposes.
+ */
+ need_phv = (var->varnullingrels != NULL) || rcon->wrap_non_vars;
+
/*
* If PlaceHolderVars are needed, we cache the modified expressions in
* rcon->rv_cache[]. This is not in hopes of any material speed gain
@@ -2348,13 +2295,16 @@ pullup_replace_vars_callback(Var *var,
* and possibly prevent optimizations that rely on recognizing different
* references to the same subquery output as being equal(). So it's worth
* a bit of extra effort to avoid it.
+ *
+ * The cached items have phlevelsup = 0 and phnullingrels = NULL; we'll
+ * copy them and adjust those values for this reference site below.
*/
- if (rcon->need_phvs &&
+ if (need_phv &&
varattno >= InvalidAttrNumber &&
varattno <= list_length(rcon->targetlist) &&
rcon->rv_cache[varattno] != NULL)
{
- /* Just copy the entry and fall through to adjust its varlevelsup */
+ /* Just copy the entry and fall through to adjust phlevelsup etc */
newnode = copyObject(rcon->rv_cache[varattno]);
}
else if (varattno == InvalidAttrNumber)
@@ -2363,7 +2313,7 @@ pullup_replace_vars_callback(Var *var,
RowExpr *rowexpr;
List *colnames;
List *fields;
- bool save_need_phvs = rcon->need_phvs;
+ bool save_wrap_non_vars = rcon->wrap_non_vars;
int save_sublevelsup = context->sublevels_up;
/*
@@ -2374,18 +2324,18 @@ pullup_replace_vars_callback(Var *var,
* the RowExpr for use of the executor and ruleutils.c.
*
* In order to be able to cache the results, we always generate the
- * expansion with varlevelsup = 0, and then adjust if needed.
+ * expansion with varlevelsup = 0, and then adjust below if needed.
*/
expandRTE(rcon->target_rte,
var->varno, 0 /* not varlevelsup */ , var->location,
(var->vartype != RECORDOID),
&colnames, &fields);
- /* Adjust the generated per-field Vars, but don't insert PHVs */
- rcon->need_phvs = false;
+ /* Expand the generated per-field Vars, but don't insert PHVs there */
+ rcon->wrap_non_vars = false;
context->sublevels_up = 0; /* to match the expandRTE output */
fields = (List *) replace_rte_variables_mutator((Node *) fields,
context);
- rcon->need_phvs = save_need_phvs;
+ rcon->wrap_non_vars = save_wrap_non_vars;
context->sublevels_up = save_sublevelsup;
rowexpr = makeNode(RowExpr);
@@ -2403,14 +2353,13 @@ pullup_replace_vars_callback(Var *var,
* expression to yield NULL, not ROW(NULL,NULL,...) when it is forced
* to null by an outer join.
*/
- if (rcon->need_phvs)
+ if (need_phv)
{
- /* RowExpr is certainly not strict, so always need PHV */
newnode = (Node *)
make_placeholder_expr(rcon->root,
(Expr *) newnode,
bms_make_singleton(rcon->varno));
- /* cache it with the PHV, and with varlevelsup still zero */
+ /* cache it with the PHV, and with phlevelsup etc not set yet */
rcon->rv_cache[InvalidAttrNumber] = copyObject(newnode);
}
}
@@ -2427,7 +2376,7 @@ pullup_replace_vars_callback(Var *var,
newnode = (Node *) copyObject(tle->expr);
/* Insert PlaceHolderVar if needed */
- if (rcon->need_phvs)
+ if (need_phv)
{
bool wrap;
@@ -2453,69 +2402,61 @@ pullup_replace_vars_callback(Var *var,
/* No need to wrap a PlaceHolderVar with another one, either */
wrap = false;
}
- else if (rcon->wrap_non_vars)
- {
- /* Wrap all non-Vars in a PlaceHolderVar */
- wrap = true;
- }
else
{
/*
- * If it contains a Var of the subquery being pulled up, and
- * does not contain any non-strict constructs, then it's
- * certainly nullable so we don't need to insert a
- * PlaceHolderVar.
- *
- * This analysis could be tighter: in particular, a non-strict
- * construct hidden within a lower-level PlaceHolderVar is not
- * reason to add another PHV. But for now it doesn't seem
- * worth the code to be more exact.
- *
- * Note: in future maybe we should insert a PlaceHolderVar
- * anyway, if the tlist item is expensive to evaluate?
- *
- * For a LATERAL subquery, we have to check the actual var
- * membership of the node, but if it's non-lateral then any
- * level-zero var must belong to the subquery.
+ * Must wrap, either because we need a place to insert
+ * varnullingrels or because caller told us to wrap
+ * everything.
*/
- if ((rcon->target_rte->lateral ?
- bms_overlap(pull_varnos(rcon->root, (Node *) newnode),
- rcon->relids) :
- contain_vars_of_level((Node *) newnode, 0)) &&
- !contain_nonstrict_functions((Node *) newnode))
- {
- /* No wrap needed */
- wrap = false;
- }
- else
- {
- /* Else wrap it in a PlaceHolderVar */
- wrap = true;
- }
+ wrap = true;
}
if (wrap)
+ {
newnode = (Node *)
make_placeholder_expr(rcon->root,
(Expr *) newnode,
bms_make_singleton(rcon->varno));
- /*
- * Cache it if possible (ie, if the attno is in range, which it
- * probably always should be). We can cache the value even if we
- * decided we didn't need a PHV, since this result will be
- * suitable for any request that has need_phvs.
- */
- if (varattno > InvalidAttrNumber &&
- varattno <= list_length(rcon->targetlist))
- rcon->rv_cache[varattno] = copyObject(newnode);
+ /*
+ * Cache it if possible (ie, if the attno is in range, which
+ * it probably always should be).
+ */
+ if (varattno > InvalidAttrNumber &&
+ varattno <= list_length(rcon->targetlist))
+ rcon->rv_cache[varattno] = copyObject(newnode);
+ }
}
}
- /* Must adjust varlevelsup if tlist item is from higher query */
+ /* Must adjust varlevelsup if replaced Var is within a subquery */
if (var->varlevelsup > 0)
IncrementVarSublevelsUp(newnode, var->varlevelsup, 0);
+ /* Propagate any varnullingrels into the replacement Var or PHV */
+ if (var->varnullingrels != NULL)
+ {
+ if (IsA(newnode, Var))
+ {
+ Var *newvar = (Var *) newnode;
+
+ Assert(newvar->varlevelsup == var->varlevelsup);
+ newvar->varnullingrels = bms_add_members(newvar->varnullingrels,
+ var->varnullingrels);
+ }
+ else if (IsA(newnode, PlaceHolderVar))
+ {
+ PlaceHolderVar *newphv = (PlaceHolderVar *) newnode;
+
+ Assert(newphv->phlevelsup == var->varlevelsup);
+ newphv->phnullingrels = bms_add_members(newphv->phnullingrels,
+ var->varnullingrels);
+ }
+ else
+ elog(ERROR, "failed to wrap a non-Var");
+ }
+
return newnode;
}
@@ -2674,7 +2615,9 @@ flatten_simple_union_all(PlannerInfo *root)
void
reduce_outer_joins(PlannerInfo *root)
{
- reduce_outer_joins_state *state;
+ reduce_outer_joins_pass1_state *state1;
+ reduce_outer_joins_pass2_state state2;
+ ListCell *lc;
/*
* To avoid doing strictness checks on more quals than necessary, we want
@@ -2685,14 +2628,44 @@ reduce_outer_joins(PlannerInfo *root)
* join(s) below each side of each join clause. The second pass examines
* qual clauses and changes join types as it descends the tree.
*/
- state = reduce_outer_joins_pass1((Node *) root->parse->jointree);
+ state1 = reduce_outer_joins_pass1((Node *) root->parse->jointree);
/* planner.c shouldn't have called me if no outer joins */
- if (state == NULL || !state->contains_outer)
+ if (state1 == NULL || !state1->contains_outer)
elog(ERROR, "so where are the outer joins?");
+ state2.inner_reduced = NULL;
+ state2.partial_reduced = NIL;
+
reduce_outer_joins_pass2((Node *) root->parse->jointree,
- state, root, NULL, NIL, NIL);
+ state1, &state2,
+ root, NULL, NIL, NIL);
+
+ /*
+ * If we successfully reduced the strength of any outer joins, we must
+ * remove references to those joins as nulling rels. This is handled as
+ * an additional pass, for simplicity and because we can handle all
+ * fully-reduced joins in a single pass over the parse tree.
+ */
+ if (!bms_is_empty(state2.inner_reduced))
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ state2.inner_reduced,
+ NULL);
+
+ /*
+ * Partially-reduced full joins have to be done one at a time, since
+ * they'll each need a different setting of except_relids.
+ */
+ foreach(lc, state2.partial_reduced)
+ {
+ reduce_outer_joins_partial_state *statep = lfirst(lc);
+
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ bms_make_singleton(statep->full_join_rti),
+ statep->unreduced_side);
+ }
}
/*
@@ -2700,13 +2673,13 @@ reduce_outer_joins(PlannerInfo *root)
*
* Returns a state node describing the given jointree node.
*/
-static reduce_outer_joins_state *
+static reduce_outer_joins_pass1_state *
reduce_outer_joins_pass1(Node *jtnode)
{
- reduce_outer_joins_state *result;
+ reduce_outer_joins_pass1_state *result;
- result = (reduce_outer_joins_state *)
- palloc(sizeof(reduce_outer_joins_state));
+ result = (reduce_outer_joins_pass1_state *)
+ palloc(sizeof(reduce_outer_joins_pass1_state));
result->relids = NULL;
result->contains_outer = false;
result->sub_states = NIL;
@@ -2726,7 +2699,7 @@ reduce_outer_joins_pass1(Node *jtnode)
foreach(l, f->fromlist)
{
- reduce_outer_joins_state *sub_state;
+ reduce_outer_joins_pass1_state *sub_state;
sub_state = reduce_outer_joins_pass1(lfirst(l));
result->relids = bms_add_members(result->relids,
@@ -2738,7 +2711,7 @@ reduce_outer_joins_pass1(Node *jtnode)
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- reduce_outer_joins_state *sub_state;
+ reduce_outer_joins_pass1_state *sub_state;
/* join's own RT index is not wanted in result->relids */
if (IS_OUTER_JOIN(j->jointype))
@@ -2766,15 +2739,23 @@ reduce_outer_joins_pass1(Node *jtnode)
* reduce_outer_joins_pass2 - phase 2 processing
*
* jtnode: current jointree node
- * state: state data collected by phase 1 for this node
+ * state1: state data collected by phase 1 for this node
+ * state2: where to accumulate info about successfully-reduced joins
* root: toplevel planner state
* nonnullable_rels: set of base relids forced non-null by upper quals
* nonnullable_vars: list of Vars forced non-null by upper quals
* forced_null_vars: list of Vars forced null by upper quals
+ *
+ * Returns info in state2 about outer joins that were successfully simplified.
+ * Joins that were fully reduced to inner joins are all added to
+ * state2->inner_reduced. If a full join is reduced to a left join,
+ * it needs its own entry in state2->partial_reduced, since that will
+ * require custom processing to remove only the correct nullingrel markers.
*/
static void
reduce_outer_joins_pass2(Node *jtnode,
- reduce_outer_joins_state *state,
+ reduce_outer_joins_pass1_state *state1,
+ reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
List *nonnullable_vars,
@@ -2808,13 +2789,14 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_forced_null_vars = list_concat(pass_forced_null_vars,
forced_null_vars);
/* And recurse --- but only into interesting subtrees */
- Assert(list_length(f->fromlist) == list_length(state->sub_states));
- forboth(l, f->fromlist, s, state->sub_states)
+ Assert(list_length(f->fromlist) == list_length(state1->sub_states));
+ forboth(l, f->fromlist, s, state1->sub_states)
{
- reduce_outer_joins_state *sub_state = lfirst(s);
+ reduce_outer_joins_pass1_state *sub_state = lfirst(s);
if (sub_state->contains_outer)
- reduce_outer_joins_pass2(lfirst(l), sub_state, root,
+ reduce_outer_joins_pass2(lfirst(l), sub_state,
+ state2, root,
pass_nonnullable_rels,
pass_nonnullable_vars,
pass_forced_null_vars);
@@ -2827,8 +2809,8 @@ reduce_outer_joins_pass2(Node *jtnode,
JoinExpr *j = (JoinExpr *) jtnode;
int rtindex = j->rtindex;
JoinType jointype = j->jointype;
- reduce_outer_joins_state *left_state = linitial(state->sub_states);
- reduce_outer_joins_state *right_state = lsecond(state->sub_states);
+ reduce_outer_joins_pass1_state *left_state = linitial(state1->sub_states);
+ reduce_outer_joins_pass1_state *right_state = lsecond(state1->sub_states);
List *local_nonnullable_vars = NIL;
bool computed_local_nonnullable_vars = false;
@@ -2851,12 +2833,22 @@ reduce_outer_joins_pass2(Node *jtnode,
if (bms_overlap(nonnullable_rels, right_state->relids))
jointype = JOIN_INNER;
else
+ {
jointype = JOIN_LEFT;
+ /* Also report partial reduction in state2 */
+ report_reduced_full_join(state2, rtindex,
+ right_state->relids);
+ }
}
else
{
if (bms_overlap(nonnullable_rels, right_state->relids))
+ {
jointype = JOIN_RIGHT;
+ /* Also report partial reduction in state2 */
+ report_reduced_full_join(state2, rtindex,
+ left_state->relids);
+ }
}
break;
case JOIN_SEMI:
@@ -2889,8 +2881,8 @@ reduce_outer_joins_pass2(Node *jtnode,
j->larg = j->rarg;
j->rarg = tmparg;
jointype = JOIN_LEFT;
- right_state = linitial(state->sub_states);
- left_state = lsecond(state->sub_states);
+ right_state = linitial(state1->sub_states);
+ left_state = lsecond(state1->sub_states);
}
/*
@@ -2923,7 +2915,10 @@ reduce_outer_joins_pass2(Node *jtnode,
jointype = JOIN_ANTI;
}
- /* Apply the jointype change, if any, to both jointree node and RTE */
+ /*
+ * Apply the jointype change, if any, to both jointree node and RTE.
+ * Also, if we changed an RTE to INNER, add its RTI to inner_reduced.
+ */
if (rtindex && jointype != j->jointype)
{
RangeTblEntry *rte = rt_fetch(rtindex, root->parse->rtable);
@@ -2931,6 +2926,9 @@ reduce_outer_joins_pass2(Node *jtnode,
Assert(rte->rtekind == RTE_JOIN);
Assert(rte->jointype == j->jointype);
rte->jointype = jointype;
+ if (jointype == JOIN_INNER)
+ state2->inner_reduced = bms_add_member(state2->inner_reduced,
+ rtindex);
}
j->jointype = jointype;
@@ -3011,7 +3009,8 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_nonnullable_vars = NIL;
pass_forced_null_vars = NIL;
}
- reduce_outer_joins_pass2(j->larg, left_state, root,
+ reduce_outer_joins_pass2(j->larg, left_state,
+ state2, root,
pass_nonnullable_rels,
pass_nonnullable_vars,
pass_forced_null_vars);
@@ -3033,7 +3032,8 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_nonnullable_vars = NIL;
pass_forced_null_vars = NIL;
}
- reduce_outer_joins_pass2(j->rarg, right_state, root,
+ reduce_outer_joins_pass2(j->rarg, right_state,
+ state2, root,
pass_nonnullable_rels,
pass_nonnullable_vars,
pass_forced_null_vars);
@@ -3046,6 +3046,19 @@ reduce_outer_joins_pass2(Node *jtnode,
(int) nodeTag(jtnode));
}
+/* Helper for reduce_outer_joins_pass2 */
+static void
+report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
+ int rtindex, Relids relids)
+{
+ reduce_outer_joins_partial_state *statep;
+
+ statep = palloc(sizeof(reduce_outer_joins_partial_state));
+ statep->full_join_rti = rtindex;
+ statep->unreduced_side = relids;
+ state2->partial_reduced = lappend(state2->partial_reduced, statep);
+}
+
/*
* remove_useless_result_rtes
@@ -3087,16 +3100,34 @@ reduce_outer_joins_pass2(Node *jtnode,
void
remove_useless_result_rtes(PlannerInfo *root)
{
+ Relids dropped_outer_joins = NULL;
ListCell *cell;
/* Top level of jointree must always be a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
/* Recurse ... */
root->parse->jointree = (FromExpr *)
- remove_useless_results_recurse(root, (Node *) root->parse->jointree);
+ remove_useless_results_recurse(root,
+ (Node *) root->parse->jointree,
+ &dropped_outer_joins);
/* We should still have a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
+ /*
+ * If we removed any outer-join nodes from the jointree, run around and
+ * remove references to those joins as nulling rels. (There could be such
+ * references in PHVs that we pulled up out of the original subquery that
+ * the RESULT rel replaced. This is kosher on the grounds that we now
+ * know that such an outer join wouldn't really have nulled anything.) We
+ * don't do this during the main recursion, for simplicity and because we
+ * can handle all such joins in a single pass over the parse tree.
+ */
+ if (!bms_is_empty(dropped_outer_joins))
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ dropped_outer_joins,
+ NULL);
+
/*
* Remove any PlanRowMark referencing an RTE_RESULT RTE. We obviously
* must do that for any RTE_RESULT that we just removed. But one for a
@@ -3122,9 +3153,12 @@ remove_useless_result_rtes(PlannerInfo *root)
* Recursive guts of remove_useless_result_rtes.
*
* This recursively processes the jointree and returns a modified jointree.
+ * In addition, the RT indexes of any removed outer-join nodes are added to
+ * *dropped_outer_joins.
*/
static Node *
-remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
+remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Relids *dropped_outer_joins)
{
Assert(jtnode != NULL);
if (IsA(jtnode, RangeTblRef))
@@ -3152,7 +3186,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
int varno;
/* Recursively transform child ... */
- child = remove_useless_results_recurse(root, child);
+ child = remove_useless_results_recurse(root, child,
+ dropped_outer_joins);
/* ... and stick it back into the tree */
lfirst(cell) = child;
@@ -3201,8 +3236,10 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
int varno;
/* First, recurse */
- j->larg = remove_useless_results_recurse(root, j->larg);
- j->rarg = remove_useless_results_recurse(root, j->rarg);
+ j->larg = remove_useless_results_recurse(root, j->larg,
+ dropped_outer_joins);
+ j->rarg = remove_useless_results_recurse(root, j->rarg,
+ dropped_outer_joins);
/* Apply join-type-specific optimization rules */
switch (j->jointype)
@@ -3270,6 +3307,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
!find_dependent_phvs(root, varno)))
{
remove_result_refs(root, varno, j->larg);
+ *dropped_outer_joins = bms_add_member(*dropped_outer_joins,
+ j->rtindex);
jtnode = j->larg;
}
break;
@@ -3280,6 +3319,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
!find_dependent_phvs(root, varno)))
{
remove_result_refs(root, varno, j->rarg);
+ *dropped_outer_joins = bms_add_member(*dropped_outer_joins,
+ j->rtindex);
jtnode = j->rarg;
}
break;
@@ -3294,11 +3335,14 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
* Unlike the LEFT/RIGHT cases, we just Assert that there are
* no PHVs that need to be evaluated at the semijoin's RHS,
* since the rest of the query couldn't reference any outputs
- * of the semijoin's RHS.
+ * of the semijoin's RHS. Also, we don't need to worry about
+ * removing traces of the join's rtindex, since it hasn't got
+ * one.
*/
if ((varno = get_result_relid(root, j->rarg)) != 0)
{
Assert(!find_dependent_phvs(root, varno));
+ Assert(j->rtindex == 0);
remove_result_refs(root, varno, j->larg);
if (j->quals)
jtnode = (Node *)
@@ -3367,7 +3411,7 @@ remove_result_refs(PlannerInfo *root, int varno, Node *newjtloc)
{
Relids subrelids;
- subrelids = get_relids_in_jointree(newjtloc, false);
+ subrelids = get_relids_in_jointree(newjtloc, true, false);
Assert(!bms_is_empty(subrelids));
substitute_phv_relids((Node *) root->parse, varno, subrelids);
fix_append_rel_relids(root->append_rel_list, varno, subrelids);
@@ -3479,7 +3523,7 @@ find_dependent_phvs_in_jointree(PlannerInfo *root, Node *node, int varno)
* are not marked LATERAL, though, since they couldn't possibly contain
* any cross-references to other RTEs.
*/
- subrelids = get_relids_in_jointree(node, false);
+ subrelids = get_relids_in_jointree(node, false, false);
relid = -1;
while ((relid = bms_next_member(subrelids, relid)) >= 0)
{
@@ -3623,11 +3667,17 @@ fix_append_rel_relids(List *append_rel_list, int varno, Relids subrelids)
/*
* get_relids_in_jointree: get set of RT indexes present in a jointree
*
- * If include_joins is true, join RT indexes are included; if false,
- * only base rels are included.
+ * Base-relation relids are always included in the result.
+ * If include_outer_joins is true, outer-join RT indexes are included.
+ * If include_inner_joins is true, inner-join RT indexes are included.
+ *
+ * Note that for most purposes in the planner, outer joins are included
+ * in standard relid sets. Setting include_inner_joins true is only
+ * appropriate for special purposes during subquery flattening.
*/
Relids
-get_relids_in_jointree(Node *jtnode, bool include_joins)
+get_relids_in_jointree(Node *jtnode, bool include_outer_joins,
+ bool include_inner_joins)
{
Relids result = NULL;
@@ -3648,18 +3698,34 @@ get_relids_in_jointree(Node *jtnode, bool include_joins)
{
result = bms_join(result,
get_relids_in_jointree(lfirst(l),
- include_joins));
+ include_outer_joins,
+ include_inner_joins));
}
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- result = get_relids_in_jointree(j->larg, include_joins);
+ result = get_relids_in_jointree(j->larg,
+ include_outer_joins,
+ include_inner_joins);
result = bms_join(result,
- get_relids_in_jointree(j->rarg, include_joins));
- if (include_joins && j->rtindex)
- result = bms_add_member(result, j->rtindex);
+ get_relids_in_jointree(j->rarg,
+ include_outer_joins,
+ include_inner_joins));
+ if (j->rtindex)
+ {
+ if (j->jointype == JOIN_INNER)
+ {
+ if (include_inner_joins)
+ result = bms_add_member(result, j->rtindex);
+ }
+ else
+ {
+ if (include_outer_joins)
+ result = bms_add_member(result, j->rtindex);
+ }
+ }
}
else
elog(ERROR, "unrecognized node type: %d",
@@ -3668,7 +3734,7 @@ get_relids_in_jointree(Node *jtnode, bool include_joins)
}
/*
- * get_relids_for_join: get set of base RT indexes making up a join
+ * get_relids_for_join: get set of base+OJ RT indexes making up a join
*/
Relids
get_relids_for_join(Query *query, int joinrelid)
@@ -3679,7 +3745,7 @@ get_relids_for_join(Query *query, int joinrelid)
joinrelid);
if (!jtnode)
elog(ERROR, "could not find join node %d", joinrelid);
- return get_relids_in_jointree(jtnode, false);
+ return get_relids_in_jointree(jtnode, true, false);
}
/*
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index 62cccf9d87..e793a4c85b 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -228,6 +228,12 @@ adjust_appendrel_attrs_mutator(Node *node,
if (var->varlevelsup != 0)
return (Node *) var; /* no changes needed */
+ /*
+ * You might think we need to adjust var->varnullingrels, but that
+ * shouldn't need any changes. It will contain outer-join relids,
+ * while the transformation we are making affects only baserels.
+ */
+
for (cnt = 0; cnt < nappinfos; cnt++)
{
if (var->varno == appinfos[cnt]->parent_relid)
@@ -348,6 +354,8 @@ adjust_appendrel_attrs_mutator(Node *node,
var = copyObject(ridinfo->rowidvar);
/* ... but use the correct relid */
var->varno = leaf_relid;
+ /* identity vars shouldn't have nulling rels */
+ Assert(var->varnullingrels == NULL);
/* varnosyn in the RowIdentityVarInfo is probably wrong */
var->varnosyn = 0;
var->varattnosyn = 0;
@@ -392,8 +400,11 @@ adjust_appendrel_attrs_mutator(Node *node,
(void *) context);
/* now fix PlaceHolderVar's relid sets */
if (phv->phlevelsup == 0)
- phv->phrels = adjust_child_relids(phv->phrels, context->nappinfos,
- context->appinfos);
+ {
+ phv->phrels = adjust_child_relids(phv->phrels,
+ nappinfos, appinfos);
+ /* as above, we needn't touch phnullingrels */
+ }
return (Node *) phv;
}
/* Shouldn't need to handle planner auxiliary nodes here */
@@ -688,7 +699,11 @@ get_translated_update_targetlist(PlannerInfo *root, Index relid,
/*
* find_appinfos_by_relids
- * Find AppendRelInfo structures for all relations specified by relids.
+ * Find AppendRelInfo structures for base relations listed in relids.
+ *
+ * The relids argument is typically a join relation's relids, which can
+ * include outer-join RT indexes in addition to baserels. We silently
+ * ignore the outer joins.
*
* The AppendRelInfos are returned in an array, which can be pfree'd by the
* caller. *nappinfos is set to the number of entries in the array.
@@ -700,8 +715,9 @@ find_appinfos_by_relids(PlannerInfo *root, Relids relids, int *nappinfos)
int cnt = 0;
int i;
- *nappinfos = bms_num_members(relids);
- appinfos = (AppendRelInfo **) palloc(sizeof(AppendRelInfo *) * *nappinfos);
+ /* Allocate an array that's certainly big enough */
+ appinfos = (AppendRelInfo **)
+ palloc(sizeof(AppendRelInfo *) * bms_num_members(relids));
i = -1;
while ((i = bms_next_member(relids, i)) >= 0)
@@ -709,10 +725,17 @@ find_appinfos_by_relids(PlannerInfo *root, Relids relids, int *nappinfos)
AppendRelInfo *appinfo = root->append_rel_array[i];
if (!appinfo)
+ {
+ /* Probably i is an OJ index, but let's check */
+ if (find_base_rel_ignore_join(root, i) == NULL)
+ continue;
+ /* It's a base rel, but we lack an append_rel_array entry */
elog(ERROR, "child rel %d not found in append_rel_array", i);
+ }
appinfos[cnt++] = appinfo;
}
+ *nappinfos = cnt;
return appinfos;
}
@@ -754,6 +777,7 @@ add_row_identity_var(PlannerInfo *root, Var *orig_var,
Assert(IsA(orig_var, Var));
Assert(orig_var->varno == rtindex);
Assert(orig_var->varlevelsup == 0);
+ Assert(orig_var->varnullingrels == NULL);
/*
* If we're doing non-inherited UPDATE/DELETE/MERGE, there's little need
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 533df86ff7..c82fd451b2 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -2022,14 +2022,16 @@ is_pseudo_constant_clause_relids(Node *clause, Relids relids)
* NumRelids
* (formerly clause_relids)
*
- * Returns the number of different relations referenced in 'clause'.
+ * Returns the number of different base relations referenced in 'clause'.
*/
int
NumRelids(PlannerInfo *root, Node *clause)
{
+ int result;
Relids varnos = pull_varnos(root, clause);
- int result = bms_num_members(varnos);
+ varnos = bms_del_members(varnos, root->outer_join_rels);
+ result = bms_num_members(varnos);
bms_free(varnos);
return result;
}
diff --git a/src/backend/optimizer/util/joininfo.c b/src/backend/optimizer/util/joininfo.c
index d4cffdb198..afd243f5d8 100644
--- a/src/backend/optimizer/util/joininfo.c
+++ b/src/backend/optimizer/util/joininfo.c
@@ -88,8 +88,8 @@ have_relevant_joinclause(PlannerInfo *root,
* not depend on context).
*
* 'restrictinfo' describes the join clause
- * 'join_relids' is the list of relations participating in the join clause
- * (there must be more than one)
+ * 'join_relids' is the set of relations participating in the join clause
+ * (some of these could be outer joins)
*/
void
add_join_clause_to_rels(PlannerInfo *root,
@@ -101,8 +101,11 @@ add_join_clause_to_rels(PlannerInfo *root,
cur_relid = -1;
while ((cur_relid = bms_next_member(join_relids, cur_relid)) >= 0)
{
- RelOptInfo *rel = find_base_rel(root, cur_relid);
+ RelOptInfo *rel = find_base_rel_ignore_join(root, cur_relid);
+ /* We only need to add the clause to baserels */
+ if (rel == NULL)
+ continue;
rel->joininfo = lappend(rel->joininfo, restrictinfo);
}
}
@@ -115,8 +118,8 @@ add_join_clause_to_rels(PlannerInfo *root,
* discover that a relation need not be joined at all.
*
* 'restrictinfo' describes the join clause
- * 'join_relids' is the list of relations participating in the join clause
- * (there must be more than one)
+ * 'join_relids' is the set of relations participating in the join clause
+ * (some of these could be outer joins)
*/
void
remove_join_clause_from_rels(PlannerInfo *root,
@@ -128,7 +131,11 @@ remove_join_clause_from_rels(PlannerInfo *root,
cur_relid = -1;
while ((cur_relid = bms_next_member(join_relids, cur_relid)) >= 0)
{
- RelOptInfo *rel = find_base_rel(root, cur_relid);
+ RelOptInfo *rel = find_base_rel_ignore_join(root, cur_relid);
+
+ /* We would only have added the clause to baserels */
+ if (rel == NULL)
+ continue;
/*
* Remove the restrictinfo from the list. Pointer comparison is
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index b1363df065..a62d4587ea 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -338,7 +338,9 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
sjinfo.syn_lefthand = sjinfo.min_lefthand;
sjinfo.syn_righthand = sjinfo.min_righthand;
sjinfo.jointype = JOIN_INNER;
+ sjinfo.ojrelid = 0;
/* we don't bother trying to make the remaining fields valid */
+ sjinfo.strict_relids = NULL;
sjinfo.lhs_strict = false;
sjinfo.delay_upper_joins = false;
sjinfo.semi_can_btree = false;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 833b440f39..23e3e430f4 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -1307,7 +1307,7 @@ create_append_path(PlannerInfo *root,
* Apply query-wide LIMIT if known and path is for sole base relation.
* (Handling this at this low level is a bit klugy.)
*/
- if (root != NULL && bms_equal(rel->relids, root->all_baserels))
+ if (root != NULL && bms_equal(rel->relids, root->all_query_rels))
pathnode->limit_tuples = root->limit_tuples;
else
pathnode->limit_tuples = -1.0;
@@ -1427,7 +1427,7 @@ create_merge_append_path(PlannerInfo *root,
* Apply query-wide LIMIT if known and path is for sole base relation.
* (Handling this at this low level is a bit klugy.)
*/
- if (bms_equal(rel->relids, root->all_baserels))
+ if (bms_equal(rel->relids, root->all_query_rels))
pathnode->limit_tuples = root->limit_tuples;
else
pathnode->limit_tuples = -1.0;
diff --git a/src/backend/optimizer/util/placeholder.c b/src/backend/optimizer/util/placeholder.c
index 3b0f0584f0..4166e55d9a 100644
--- a/src/backend/optimizer/util/placeholder.c
+++ b/src/backend/optimizer/util/placeholder.c
@@ -32,8 +32,14 @@ static void find_placeholders_in_expr(PlannerInfo *root, Node *expr);
* make_placeholder_expr
* Make a PlaceHolderVar for the given expression.
*
- * phrels is the syntactic location (as a set of baserels) to attribute
+ * phrels is the syntactic location (as a set of relids) to attribute
* to the expression.
+ *
+ * The caller is responsible for adjusting phlevelsup and phnullingrels
+ * as needed. Because we do not know here which query level the PHV
+ * will be associated with, it's important that this function touches
+ * only root->glob; messing with other parts of PlannerInfo would be
+ * likely to do the wrong thing.
*/
PlaceHolderVar *
make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
@@ -42,8 +48,9 @@ make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
phv->phexpr = expr;
phv->phrels = phrels;
+ phv->phnullingrels = NULL; /* caller may change this later */
phv->phid = ++(root->glob->lastPHId);
- phv->phlevelsup = 0;
+ phv->phlevelsup = 0; /* caller may change this later */
return phv;
}
@@ -317,6 +324,8 @@ update_placeholder_eval_levels(PlannerInfo *root, SpecialJoinInfo *new_sjinfo)
sjinfo->min_lefthand);
eval_at = bms_add_members(eval_at,
sjinfo->min_righthand);
+ if (sjinfo->ojrelid)
+ eval_at = bms_add_member(eval_at, sjinfo->ojrelid);
/* we'll need another iteration */
found_some = true;
}
@@ -390,9 +399,16 @@ add_placeholders_to_base_rels(PlannerInfo *root)
bms_nonempty_difference(phinfo->ph_needed, eval_at))
{
RelOptInfo *rel = find_base_rel(root, varno);
+ PlaceHolderVar *phv;
- rel->reltarget->exprs = lappend(rel->reltarget->exprs,
- copyObject(phinfo->ph_var));
+ /*
+ * As in add_vars_to_targetlist(), a value computed at scan level
+ * has not yet been nulled by any outer join, so set its
+ * phnullingrels to empty.
+ */
+ phv = copyObject(phinfo->ph_var);
+ phv->phnullingrels = NULL;
+ rel->reltarget->exprs = lappend(rel->reltarget->exprs, phv);
/* reltarget's cost and width fields will be updated later */
}
}
@@ -411,7 +427,8 @@ add_placeholders_to_base_rels(PlannerInfo *root)
*/
void
add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *outer_rel, RelOptInfo *inner_rel)
+ RelOptInfo *outer_rel, RelOptInfo *inner_rel,
+ SpecialJoinInfo *sjinfo)
{
Relids relids = joinrel->relids;
ListCell *lc;
@@ -426,10 +443,11 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
/* Is it still needed above this joinrel? */
if (bms_nonempty_difference(phinfo->ph_needed, relids))
{
- /* Yup, add it to the output */
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- phinfo->ph_var);
- joinrel->reltarget->width += phinfo->ph_width;
+ /*
+ * Yup, we must add it to the output. Make a copy so we can
+ * adjust phnullingrels if needed.
+ */
+ PlaceHolderVar *phv = copyObject(phinfo->ph_var);
/*
* Charge the cost of evaluating the contained expression if
@@ -442,16 +460,42 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
* with that; but we might want to improve it later by
* refiguring the reltarget costs for each pair of inputs.
*/
- if (!bms_is_subset(phinfo->ph_eval_at, outer_rel->relids) &&
- !bms_is_subset(phinfo->ph_eval_at, inner_rel->relids))
+ if (bms_is_subset(phinfo->ph_eval_at, outer_rel->relids))
+ {
+ if (sjinfo->jointype == JOIN_FULL && sjinfo->ojrelid != 0)
+ {
+ /* PHV's value can be nulled at this join */
+ phv->phnullingrels = bms_add_member(phv->phnullingrels,
+ sjinfo->ojrelid);
+ }
+ }
+ else if (bms_is_subset(phinfo->ph_eval_at, inner_rel->relids))
{
+ if (sjinfo->jointype != JOIN_INNER && sjinfo->ojrelid != 0)
+ {
+ /* PHV's value can be nulled at this join */
+ phv->phnullingrels = bms_add_member(phv->phnullingrels,
+ sjinfo->ojrelid);
+ }
+ }
+ else
+ {
+ /* It must be computed here. */
QualCost cost;
+ /* It'll start out not nulled by anything */
+ phv->phnullingrels = NULL;
+ /* Add the appropriate cost */
cost_qual_eval_node(&cost, (Node *) phinfo->ph_var->phexpr,
root);
joinrel->reltarget->cost.startup += cost.startup;
joinrel->reltarget->cost.per_tuple += cost.per_tuple;
}
+
+ joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
+ phv);
+ /* Update width estimate, too */
+ joinrel->reltarget->width += phinfo->ph_width;
}
/*
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index a163853bed..9544ced296 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -39,7 +39,7 @@ typedef struct JoinHashEntry
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *input_rel);
+ RelOptInfo *input_rel, int ojrelid);
static List *build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outer_rel,
@@ -58,7 +58,8 @@ static void set_foreign_rel_properties(RelOptInfo *joinrel,
static void add_join_rel(PlannerInfo *root, RelOptInfo *joinrel);
static void build_joinrel_partition_info(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
- List *restrictlist, JoinType jointype);
+ SpecialJoinInfo *sjinfo,
+ List *restrictlist);
static bool have_partkey_equi_join(RelOptInfo *joinrel,
RelOptInfo *rel1, RelOptInfo *rel2,
JoinType jointype, List *restrictlist);
@@ -66,7 +67,8 @@ static int match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel,
bool strict_op);
static void set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
- JoinType jointype);
+ SpecialJoinInfo *sjinfo);
+static Node *add_nullingrel_to(Node *node, int relid);
static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *parentrel,
RelOptInfo *childrel,
@@ -367,7 +369,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
/*
* find_base_rel
- * Find a base or other relation entry, which must already exist.
+ * Find a base or otherrel relation entry, which must already exist.
*/
RelOptInfo *
find_base_rel(PlannerInfo *root, int relid)
@@ -388,6 +390,44 @@ find_base_rel(PlannerInfo *root, int relid)
return NULL; /* keep compiler quiet */
}
+/*
+ * find_base_rel_ignore_join
+ * Find a base or otherrel relation entry, which must already exist.
+ *
+ * Unlike find_base_rel, if relid references an outer join then this
+ * will return NULL rather than raising an error. This is convenient
+ * for callers that must deal with relid sets including both base and
+ * outer joins.
+ */
+RelOptInfo *
+find_base_rel_ignore_join(PlannerInfo *root, int relid)
+{
+ Assert(relid > 0);
+
+ if (relid < root->simple_rel_array_size)
+ {
+ RelOptInfo *rel;
+ RangeTblEntry *rte;
+
+ rel = root->simple_rel_array[relid];
+ if (rel)
+ return rel;
+
+ /*
+ * We could just return NULL here, but for debugging purposes it seems
+ * best to actually verify that the relid is an outer join and not
+ * something weird.
+ */
+ rte = root->simple_rte_array[relid];
+ if (rte && rte->rtekind == RTE_JOIN && rte->jointype != JOIN_INNER)
+ return NULL;
+ }
+
+ elog(ERROR, "no relation entry for relid %d", relid);
+
+ return NULL; /* keep compiler quiet */
+}
+
/*
* build_join_rel_hash
* Construct the auxiliary hash table for join relations.
@@ -686,9 +726,11 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ build_joinrel_tlist(root, joinrel, outer_rel,
+ (sjinfo->jointype == JOIN_FULL) ? sjinfo->ojrelid : 0);
+ build_joinrel_tlist(root, joinrel, inner_rel,
+ (sjinfo->jointype != JOIN_INNER) ? sjinfo->ojrelid : 0);
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel, sjinfo);
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -720,8 +762,8 @@ build_join_rel(PlannerInfo *root,
joinrel->has_eclass_joins = has_relevant_eclass_joinclause(root, joinrel);
/* Store the partition information. */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- sjinfo->jointype);
+ build_joinrel_partition_info(joinrel, outer_rel, inner_rel, sjinfo,
+ restrictlist);
/*
* Set estimates of the joinrel's size.
@@ -777,16 +819,14 @@ build_join_rel(PlannerInfo *root,
* 'parent_joinrel' is the RelOptInfo representing the join between parent
* relations. Some of the members of new RelOptInfo are produced by
* translating corresponding members of this RelOptInfo
- * 'sjinfo': child-join context info
* 'restrictlist': list of RestrictInfo nodes that apply to this particular
* pair of joinable relations
- * 'jointype' is the join type (inner, left, full, etc)
+ * 'sjinfo': child join's join-type details
*/
RelOptInfo *
build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
RelOptInfo *inner_rel, RelOptInfo *parent_joinrel,
- List *restrictlist, SpecialJoinInfo *sjinfo,
- JoinType jointype)
+ List *restrictlist, SpecialJoinInfo *sjinfo)
{
RelOptInfo *joinrel = makeNode(RelOptInfo);
AppendRelInfo **appinfos;
@@ -800,6 +840,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->reloptkind = RELOPT_OTHER_JOINREL;
joinrel->relids = bms_union(outer_rel->relids, inner_rel->relids);
+ if (sjinfo->ojrelid != 0)
+ joinrel->relids = bms_add_member(joinrel->relids, sjinfo->ojrelid);
joinrel->rows = 0;
/* cheap startup cost is interesting iff not all tuples to be retrieved */
joinrel->consider_startup = (root->tuple_fraction > 0);
@@ -886,8 +928,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->has_eclass_joins = parent_joinrel->has_eclass_joins;
/* Is the join between partitions itself partitioned? */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- jointype);
+ build_joinrel_partition_info(joinrel, outer_rel, inner_rel, sjinfo,
+ restrictlist);
/* Child joinrel is parallel safe if parent is parallel safe. */
joinrel->consider_parallel = parent_joinrel->consider_parallel;
@@ -966,12 +1008,15 @@ min_join_parameterization(PlannerInfo *root,
* will still be needed above the join. This subroutine adds all such
* Vars from the specified input rel's tlist to the join rel's tlist.
*
+ * If the join can null Vars from this input relation, pass its RT index
+ * (if any) as ojrelid; if not, pass zero.
+ *
* We also compute the expected width of the join's output, making use
* of data that was cached at the baserel level by set_rel_width().
*/
static void
build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *input_rel)
+ RelOptInfo *input_rel, int ojrelid)
{
Relids relids = joinrel->relids;
ListCell *vars;
@@ -1002,9 +1047,7 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
RowIdentityVarInfo *ridinfo = (RowIdentityVarInfo *)
list_nth(root->row_identity_vars, var->varattno - 1);
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- var);
- /* Vars have cost zero, so no need to adjust reltarget->cost */
+ /* Update reltarget width estimate from RowIdentityVarInfo */
joinrel->reltarget->width += ridinfo->rowidwidth;
}
else
@@ -1017,15 +1060,28 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
/* Is it still needed above this joinrel? */
ndx = var->varattno - baserel->min_attr;
- if (bms_nonempty_difference(baserel->attr_needed[ndx], relids))
- {
- /* Yup, add it to the output */
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- var);
- /* Vars have cost zero, so no need to adjust reltarget->cost */
- joinrel->reltarget->width += baserel->attr_widths[ndx];
- }
+ if (!bms_nonempty_difference(baserel->attr_needed[ndx], relids))
+ continue; /* nope, skip it */
+
+ /* Update reltarget width estimate from baserel's attr_widths */
+ joinrel->reltarget->width += baserel->attr_widths[ndx];
+ }
+
+ /*
+ * Add the Var to the output. If this join potentially nulls this
+ * input, we have to update the Var's varnullingrels, which means
+ * making a copy.
+ */
+ if (ojrelid != 0)
+ {
+ var = copyObject(var);
+ var->varnullingrels = bms_add_member(var->varnullingrels, ojrelid);
}
+
+ joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
+ var);
+
+ /* Vars have cost zero, so no need to adjust reltarget->cost */
}
}
@@ -1044,7 +1100,7 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
* is not handled in the sub-relations, so it depends on which
* sub-relations are considered.
*
- * If a join clause from an input relation refers to base rels still not
+ * If a join clause from an input relation refers to base+OJ rels still not
* present in the joinrel, then it is still a join clause for the joinrel;
* we put it into the joininfo list for the joinrel. Otherwise,
* the clause is now a restrict clause for the joined relation, and we
@@ -1645,8 +1701,8 @@ find_param_path_info(RelOptInfo *rel, Relids required_outer)
*/
static void
build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
- RelOptInfo *inner_rel, List *restrictlist,
- JoinType jointype)
+ RelOptInfo *inner_rel, SpecialJoinInfo *sjinfo,
+ List *restrictlist)
{
PartitionScheme part_scheme;
@@ -1673,7 +1729,7 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
!inner_rel->consider_partitionwise_join ||
outer_rel->part_scheme != inner_rel->part_scheme ||
!have_partkey_equi_join(joinrel, outer_rel, inner_rel,
- jointype, restrictlist))
+ sjinfo->jointype, restrictlist))
{
Assert(!IS_PARTITIONED_REL(joinrel));
return;
@@ -1697,7 +1753,7 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* child-join relations of the join relation in try_partitionwise_join().
*/
joinrel->part_scheme = part_scheme;
- set_joinrel_partition_key_exprs(joinrel, outer_rel, inner_rel, jointype);
+ set_joinrel_partition_key_exprs(joinrel, outer_rel, inner_rel, sjinfo);
/*
* Set the consider_partitionwise_join flag.
@@ -1877,6 +1933,23 @@ match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel, bool strict_op)
{
if (equal(lfirst(lc), expr))
return cnt;
+
+ /*
+ * XXX For the moment, also allow a match if we have Vars that
+ * match except for varnullingrels. This may be indicative of a
+ * bug, although given the restriction to strict join operators,
+ * it could be okay.
+ */
+ if (IsA(expr, Var) && IsA(lfirst(lc), Var))
+ {
+ Var *v1 = (Var *) expr;
+ Var *v2 = (Var *) lfirst(lc);
+
+ if (v1->varno == v2->varno &&
+ v1->varattno == v2->varattno &&
+ v1->varlevelsup == v2->varlevelsup)
+ return cnt;
+ }
}
}
@@ -1890,7 +1963,7 @@ match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel, bool strict_op)
static void
set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
- JoinType jointype)
+ SpecialJoinInfo *sjinfo)
{
PartitionScheme part_scheme = joinrel->part_scheme;
int partnatts = part_scheme->partnatts;
@@ -1916,7 +1989,7 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
List *nullable_partexpr = NIL;
ListCell *lc;
- switch (jointype)
+ switch (sjinfo->jointype)
{
/*
* A join relation resulting from an INNER join may be
@@ -1992,18 +2065,37 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
* partitionwise nesting of any outer join.) We assume no
* type coercions are needed to make the coalesce expressions,
* since columns of different types won't have gotten
- * classified as the same PartitionScheme.
+ * classified as the same PartitionScheme. However, we do
+ * have to worry about marking the COALESCE inputs as nullable
+ * by the full join, else these won't match the real thing.
*/
foreach(lc, list_concat_copy(outer_expr, outer_null_expr))
{
Node *larg = (Node *) lfirst(lc);
ListCell *lc2;
+ /* Insert nullingrel, or skip it if we can't */
+ larg = add_nullingrel_to(larg, sjinfo->ojrelid);
+ if (larg == NULL)
+ continue;
+
foreach(lc2, list_concat_copy(inner_expr, inner_null_expr))
{
Node *rarg = (Node *) lfirst(lc2);
- CoalesceExpr *c = makeNode(CoalesceExpr);
+ CoalesceExpr *c;
+
+ /* Forget it if coercions would be needed */
+ if (exprType(larg) != exprType(rarg) ||
+ exprCollation(larg) != exprCollation(rarg))
+ continue;
+ /* Insert nullingrel, or skip it if we can't */
+ rarg = add_nullingrel_to(rarg, sjinfo->ojrelid);
+ if (rarg == NULL)
+ continue;
+
+ /* Now we can build a valid merged join variable */
+ c = makeNode(CoalesceExpr);
c->coalescetype = exprType(larg);
c->coalescecollid = exprCollation(larg);
c->args = list_make2(larg, rarg);
@@ -2014,7 +2106,8 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
break;
default:
- elog(ERROR, "unrecognized join type: %d", (int) jointype);
+ elog(ERROR, "unrecognized join type: %d",
+ (int) sjinfo->jointype);
}
joinrel->partexprs[cnt] = partexpr;
@@ -2022,6 +2115,54 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
}
}
+/*
+ * Attempt to add relid to nullingrels of a FULL JOIN USING variable.
+ * Returns the modified expression if successful, or NULL if we failed.
+ *
+ * We currently don't support any cases where type coercion is involved,
+ * so only plain Vars and COALESCE nodes need be handled. However, we
+ * do need to support nested COALESCEs, so recursion is required.
+ */
+static Node *
+add_nullingrel_to(Node *node, int relid)
+{
+ if (IsA(node, Var))
+ {
+ /* Copy so we can modify it... */
+ Var *var = (Var *) copyObject(node);
+
+ /* ... and insert the correct nullingrel marker */
+ var->varnullingrels = bms_add_member(var->varnullingrels,
+ relid);
+ return (Node *) var;
+ }
+ if (IsA(node, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) node;
+ CoalesceExpr *newcexpr;
+ List *newargs = NIL;
+ ListCell *lc;
+
+ /* Try to modify each argument ... */
+ foreach(lc, cexpr->args)
+ {
+ Node *newarg = add_nullingrel_to((Node *) lfirst(lc), relid);
+
+ if (newarg == NULL)
+ return NULL;
+ newargs = lappend(newargs, newarg);
+ }
+ /* Success, so make the result node */
+ newcexpr = makeNode(CoalesceExpr);
+ newcexpr->coalescetype = cexpr->coalescetype;
+ newcexpr->coalescecollid = cexpr->coalescecollid;
+ newcexpr->args = newargs;
+ newcexpr->location = cexpr->location;
+ return (Node *) newcexpr;
+ }
+ return NULL;
+}
+
/*
* build_child_join_reltarget
* Set up a child-join relation's reltarget from a parent-join relation.
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index ef8df3d098..6902c2a9d7 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -116,6 +116,7 @@ make_restrictinfo_internal(PlannerInfo *root,
Relids nullable_relids)
{
RestrictInfo *restrictinfo = makeNode(RestrictInfo);
+ Relids baserels;
restrictinfo->clause = clause;
restrictinfo->orclause = orclause;
@@ -187,6 +188,20 @@ make_restrictinfo_internal(PlannerInfo *root,
else
restrictinfo->required_relids = restrictinfo->clause_relids;
+ /*
+ * Count the number of base rels appearing in clause_relids. To do this,
+ * we just delete rels mentioned in root->outer_join_rels and count the
+ * survivors. Because we are called during deconstruct_jointree which is
+ * the same tree walk that populates outer_join_rels, this is a little bit
+ * unsafe-looking; but it should be fine because the recursion in
+ * deconstruct_jointree should already have visited any outer join that
+ * could be mentioned in this clause.
+ */
+ baserels = bms_difference(restrictinfo->clause_relids,
+ root->outer_join_rels);
+ restrictinfo->num_base_rels = bms_num_members(baserels);
+ bms_free(baserels);
+
/*
* Fill in all the cacheable fields with "not yet set" markers. None of
* these will be computed until/unless needed. Note in particular that we
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
index ebc6ce84b0..99126958db 100644
--- a/src/backend/optimizer/util/var.c
+++ b/src/backend/optimizer/util/var.c
@@ -88,6 +88,9 @@ static Relids alias_relid_set(Query *query, Relids relids);
* Create a set of all the distinct varnos present in a parsetree.
* Only varnos that reference level-zero rtable entries are considered.
*
+ * The result includes outer-join relids mentioned in Var.varnullingrels and
+ * PlaceHolderVar.phnullingrels fields in the parsetree.
+ *
* "root" can be passed as NULL if it is not necessary to process
* PlaceHolderVars.
*
@@ -153,7 +156,11 @@ pull_varnos_walker(Node *node, pull_varnos_context *context)
Var *var = (Var *) node;
if (var->varlevelsup == context->sublevels_up)
+ {
context->varnos = bms_add_member(context->varnos, var->varno);
+ context->varnos = bms_add_members(context->varnos,
+ var->varnullingrels);
+ }
return false;
}
if (IsA(node, CurrentOfExpr))
@@ -251,6 +258,14 @@ pull_varnos_walker(Node *node, pull_varnos_context *context)
context->varnos = bms_join(context->varnos,
newevalat);
}
+
+ /*
+ * In all three cases, include phnullingrels in the result. We
+ * don't worry about possibly needing to translate it, because
+ * appendrels only translate varnos of baserels, not outer joins.
+ */
+ context->varnos = bms_add_members(context->varnos,
+ phv->phnullingrels);
return false; /* don't recurse into expression */
}
}
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index fa1f589fad..e4033b1572 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -2204,7 +2204,7 @@ rowcomparesel(PlannerInfo *root,
else
{
/*
- * Otherwise, it's a join if there's more than one relation used.
+ * Otherwise, it's a join if there's more than one base relation used.
*/
is_join_clause = (NumRelids(root, (Node *) opargs) > 1);
}
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 369ddfbddc..522c432841 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -240,13 +240,26 @@ struct PlannerInfo
struct AppendRelInfo **append_rel_array pg_node_attr(read_write_ignore);
/*
- * all_baserels is a Relids set of all base relids (but not "other"
- * relids) in the query; that is, the Relids identifier of the final join
- * we need to form. This is computed in make_one_rel, just before we
- * start making Paths.
+ * all_baserels is a Relids set of all base relids (but not joins or
+ * "other" relids) in the query. This is computed in make_one_rel, just
+ * before we start making Paths.
*/
Relids all_baserels;
+ /*
+ * outer_join_rels is a Relids set of all outer-join relids in the query.
+ * This is computed in deconstruct_jointree.
+ */
+ Relids outer_join_rels;
+
+ /*
+ * all_query_rels is a Relids set of all base relids and outer join relids
+ * (but not "other" relids) in the query. This is the Relids identifier
+ * of the final join we need to form. This is computed in make_one_rel,
+ * just before we start making Paths.
+ */
+ Relids all_query_rels;
+
/*
* nullable_baserels is a Relids set of base relids that are nullable by
* some outer join in the jointree; these are rels that are potentially
@@ -313,7 +326,7 @@ struct PlannerInfo
List *right_join_clauses;
/*
- * list of RestrictInfos for mergejoinable full join clauses
+ * list of FullJoinClauseInfos for mergejoinable full join clauses
*/
List *full_join_clauses;
@@ -532,9 +545,10 @@ typedef struct PartitionSchemeData *PartitionScheme;
* or the output of a sub-SELECT or function that appears in the range table.
* In either case it is uniquely identified by an RT index. A "joinrel"
* is the joining of two or more base rels. A joinrel is identified by
- * the set of RT indexes for its component baserels. We create RelOptInfo
- * nodes for each baserel and joinrel, and store them in the PlannerInfo's
- * simple_rel_array and join_rel_list respectively.
+ * the set of RT indexes for its component baserels, along with RT indexes
+ * for any outer joins it has computed. We create RelOptInfo nodes for each
+ * baserel and joinrel, and store them in the PlannerInfo's simple_rel_array
+ * and join_rel_list respectively.
*
* Note that there is only one joinrel for any given set of component
* baserels, no matter what order we assemble them in; so an unordered
@@ -573,8 +587,10 @@ typedef struct PartitionSchemeData *PartitionScheme;
* Parts of this data structure are specific to various scan and join
* mechanisms. It didn't seem worth creating new node types for them.
*
- * relids - Set of base-relation identifiers; it is a base relation
- * if there is just one, a join relation if more than one
+ * relids - Set of relation identifiers (RT indexes). This is a base
+ * relation if there is just one, a join relation if more;
+ * in the join case, RT indexes of any outer joins formed
+ * at or below this join are included along with baserels
* rows - estimated number of tuples in the relation after restriction
* clauses have been applied (ie, output rows of a plan for it)
* consider_startup - true if there is any value in keeping plain paths for
@@ -786,7 +802,7 @@ typedef struct RelOptInfo
RelOptKind reloptkind;
/*
- * all relations included in this RelOptInfo; set of base relids
+ * all relations included in this RelOptInfo; set of base + OJ relids
* (rangetable indexes)
*/
Relids relids;
@@ -2269,17 +2285,17 @@ typedef struct LimitPath
* If a restriction clause references a single base relation, it will appear
* in the baserestrictinfo list of the RelOptInfo for that base rel.
*
- * If a restriction clause references more than one base rel, it will
+ * If a restriction clause references more than one base+OJ relation, it will
* appear in the joininfo list of every RelOptInfo that describes a strict
- * subset of the base rels mentioned in the clause. The joininfo lists are
+ * subset of the relations mentioned in the clause. The joininfo lists are
* used to drive join tree building by selecting plausible join candidates.
* The clause cannot actually be applied until we have built a join rel
- * containing all the base rels it references, however.
+ * containing all the relations it references, however.
*
- * When we construct a join rel that includes all the base rels referenced
+ * When we construct a join rel that includes all the relations referenced
* in a multi-relation restriction clause, we place that clause into the
* joinrestrictinfo lists of paths for the join rel, if neither left nor
- * right sub-path includes all base rels referenced in the clause. The clause
+ * right sub-path includes all relations referenced in the clause. The clause
* will be applied at that join level, and will not propagate any further up
* the join tree. (Note: the "predicate migration" code was once intended to
* push restriction clauses up and down the plan tree based on evaluation
@@ -2300,12 +2316,15 @@ typedef struct LimitPath
* or join to enforce that all members of each EquivalenceClass are in fact
* equal in all rows emitted by the scan or join.
*
- * When dealing with outer joins we have to be very careful about pushing qual
- * clauses up and down the tree. An outer join's own JOIN/ON conditions must
- * be evaluated exactly at that join node, unless they are "degenerate"
- * conditions that reference only Vars from the nullable side of the join.
- * Quals appearing in WHERE or in a JOIN above the outer join cannot be pushed
- * down below the outer join, if they reference any nullable Vars.
+ * The clause_relids field lists the base plus outer-join RT indexes that
+ * actually appear in the clause. required_relids lists the minimum set of
+ * relids needed to evaluate the clause; while this is often equal to
+ * clause_relids, it can be more. We will add relids to required_relids when
+ * we need to force an outer join ON clause to be evaluated exactly at the
+ * level of the outer join, which is true except when it is a "degenerate"
+ * condition that references only Vars from the nullable side of the join.
+ *
+ * XXX rewrite or remove me:
* RestrictInfo nodes contain a flag to indicate whether a qual has been
* pushed down to a lower level than its original syntactic placement in the
* join tree would suggest. If an outer join prevents us from pushing a qual
@@ -2427,13 +2446,16 @@ typedef struct RestrictInfo
/* true if known to contain no leaked Vars */
bool leakproof pg_node_attr(equal_ignore);
- /* to indicate if clause contains any volatile functions. */
+ /* indicates if clause contains any volatile functions */
VolatileFunctionStatus has_volatile pg_node_attr(equal_ignore);
/* see comment above */
Index security_level;
- /* The set of relids (varnos) actually referenced in the clause: */
+ /* number of base rels in clause_relids */
+ int num_base_rels pg_node_attr(equal_ignore);
+
+ /* The relids (varnos+varnullingrels) actually referenced in the clause: */
Relids clause_relids pg_node_attr(equal_ignore);
/* The set of relids required to evaluate the clause: */
@@ -2535,6 +2557,7 @@ typedef struct RestrictInfo
} RestrictInfo;
/*
+ * XXX this will need work:
* This macro embodies the correct way to test whether a RestrictInfo is
* "pushed down" to a given outer join, that is, should be treated as a filter
* clause rather than a join clause at that outer join. This is certainly so
@@ -2637,17 +2660,20 @@ typedef struct PlaceHolderVar
* We make SpecialJoinInfos for FULL JOINs even though there is no flexibility
* of planning for them, because this simplifies make_join_rel()'s API.
*
- * min_lefthand and min_righthand are the sets of base relids that must be
- * available on each side when performing the special join. lhs_strict is
- * true if the special join's condition cannot succeed when the LHS variables
- * are all NULL (this means that an outer join can commute with upper-level
+ * min_lefthand and min_righthand are the sets of base+OJ relids that must be
+ * available on each side when performing the special join.
+ *
+ * strict_relids is the set of base+OJ relids for which the special join's
+ * condition is strict, ie it cannot succeed if any of those rels produce
+ * an all-NULL row. lhs_strict reports whether any LHS rels appear in
+ * strict_relids (this means that an outer join can commute with upper-level
* outer joins even if it appears in their RHS). We don't bother to set
- * lhs_strict for FULL JOINs, however.
+ * strict_relids or lhs_strict for FULL JOINs, however.
*
* It is not valid for either min_lefthand or min_righthand to be empty sets;
* if they were, this would break the logic that enforces join order.
*
- * syn_lefthand and syn_righthand are the sets of base relids that are
+ * syn_lefthand and syn_righthand are the sets of base+OJ relids that are
* syntactically below this special join. (These are needed to help compute
* min_lefthand and min_righthand for higher joins.)
*
@@ -2669,14 +2695,18 @@ typedef struct PlaceHolderVar
* the inputs to make it a LEFT JOIN. So the allowed values of jointype
* in a join_info_list member are only LEFT, FULL, SEMI, or ANTI.
*
+ * ojrelid is the RT index of the join RTE representing this outer join,
+ * if there is one. It is zero when jointype is INNER or SEMI.
+ *
* For purposes of join selectivity estimation, we create transient
* SpecialJoinInfo structures for regular inner joins; so it is possible
* to have jointype == JOIN_INNER in such a structure, even though this is
* not allowed within join_info_list. We also create transient
* SpecialJoinInfos with jointype == JOIN_INNER for outer joins, since for
* cost estimation purposes it is sometimes useful to know the join size under
- * plain innerjoin semantics. Note that lhs_strict, delay_upper_joins, and
- * of course the semi_xxx fields are not set meaningfully within such structs.
+ * plain innerjoin semantics. Note that strict_relids, lhs_strict,
+ * delay_upper_joins, and of course the semi_xxx fields are not set
+ * meaningfully within such structs.
*/
#ifndef HAVE_SPECIALJOININFO_TYPEDEF
typedef struct SpecialJoinInfo SpecialJoinInfo;
@@ -2688,11 +2718,13 @@ struct SpecialJoinInfo
pg_node_attr(no_read)
NodeTag type;
- Relids min_lefthand; /* base relids in minimum LHS for join */
- Relids min_righthand; /* base relids in minimum RHS for join */
- Relids syn_lefthand; /* base relids syntactically within LHS */
- Relids syn_righthand; /* base relids syntactically within RHS */
+ Relids min_lefthand; /* base+OJ relids in minimum LHS for join */
+ Relids min_righthand; /* base+OJ relids in minimum RHS for join */
+ Relids syn_lefthand; /* base+OJ relids syntactically within LHS */
+ Relids syn_righthand; /* base+OJ relids syntactically within RHS */
JoinType jointype; /* always INNER, LEFT, FULL, SEMI, or ANTI */
+ Index ojrelid; /* outer join's RT index; 0 if none */
+ Relids strict_relids; /* joinclause is strict for these relids */
bool lhs_strict; /* joinclause is strict for some LHS rel */
bool delay_upper_joins; /* can't commute with upper RHS */
/* Remaining fields are set only for JOIN_SEMI jointype: */
@@ -2702,6 +2734,21 @@ struct SpecialJoinInfo
List *semi_rhs_exprs; /* righthand-side expressions of these ops */
};
+/*
+ * FULL JOIN clause info.
+ *
+ * We set aside every FULL JOIN ON clause that looks mergejoinable, and
+ * process it specially at the end of qual distribution.
+ */
+typedef struct FullJoinClauseInfo
+{
+ pg_node_attr(no_copy_equal, no_read)
+
+ NodeTag type;
+ RestrictInfo *rinfo; /* a mergejoinable FULL JOIN clause */
+ SpecialJoinInfo *sjinfo; /* the FULL JOIN's SpecialJoinInfo */
+} FullJoinClauseInfo;
+
/*
* Append-relation info.
*
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 635cc0a0a6..1255641176 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -301,6 +301,7 @@ extern void expand_planner_arrays(PlannerInfo *root, int add_size);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
+extern RelOptInfo *find_base_rel_ignore_join(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
@@ -332,6 +333,6 @@ extern ParamPathInfo *find_param_path_info(RelOptInfo *rel,
extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
- SpecialJoinInfo *sjinfo, JoinType jointype);
+ SpecialJoinInfo *sjinfo);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/placeholder.h b/src/include/optimizer/placeholder.h
index 39803ea41f..34b118a5c9 100644
--- a/src/include/optimizer/placeholder.h
+++ b/src/include/optimizer/placeholder.h
@@ -27,6 +27,7 @@ extern void update_placeholder_eval_levels(PlannerInfo *root,
extern void fix_placeholder_input_needed_levels(PlannerInfo *root);
extern void add_placeholders_to_base_rels(PlannerInfo *root);
extern void add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *outer_rel, RelOptInfo *inner_rel);
+ RelOptInfo *outer_rel, RelOptInfo *inner_rel,
+ SpecialJoinInfo *sjinfo);
#endif /* PLACEHOLDER_H */
diff --git a/src/include/optimizer/prep.h b/src/include/optimizer/prep.h
index 2b11ff1d1f..ca03f32174 100644
--- a/src/include/optimizer/prep.h
+++ b/src/include/optimizer/prep.h
@@ -29,7 +29,8 @@ extern void pull_up_subqueries(PlannerInfo *root);
extern void flatten_simple_union_all(PlannerInfo *root);
extern void reduce_outer_joins(PlannerInfo *root);
extern void remove_useless_result_rtes(PlannerInfo *root);
-extern Relids get_relids_in_jointree(Node *jtnode, bool include_joins);
+extern Relids get_relids_in_jointree(Node *jtnode, bool include_outer_joins,
+ bool include_inner_joins);
extern Relids get_relids_for_join(Query *query, int joinrelid);
/*
commit b6c973fd890a5d7df7ffcc043e6a5defc269a303
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sun Jul 10 15:02:41 2022 -0400
Fix flatten_join_alias_vars() to handle varnullingrels correctly.
The remaining core regression test failures occur because
flatten_join_alias_vars() isn't doing the right thing. The
alias Var it needs to replace may have acquired varnullingrels
bits signifying the effect of upper outer joins, and if so we
must preserve that information in the replacement expression.
The simplest way to do that is to wrap the replacement expression
in a PlaceHolderVar, and that's what we have to do in the general
case where subquery pullup has mutated the replacement joinaliasvars
entry into an arbitrary expression. But in simpler cases, such as
where the joinaliasvars entry is just a Var, we'd prefer to do it
by merging the alias Var's varnullingrels into the replacement Var.
In that way the flattened alias will compare equal() to semantically
equivalent references that didn't use the alias name.
Moreover, the parser also uses this code while checking certain
semantic constraints, and in that context we *must not* generate
PlaceHolderVars. PHVs shouldn't appear in parse-time expressions,
and adding one would certainly cause the parser to decide the
query is invalid (because the result wouldn't compare equal() to
what it needs to). Fortunately, during parsing the set of possible
contents of a joinaliasvars entry is quite constrained, so we can
guarantee to apply the nullingrels info to the Vars therein.
The result of this step passes all core regression tests, but there
are still loose ends for FDWs (so that contrib/postgres_fdw will fail).
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index ea077cff4e..127f06fb04 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -898,7 +898,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
*/
if (rte->lateral && root->hasJoinRTEs)
rte->subquery = (Query *)
- flatten_join_alias_vars(root->parse,
+ flatten_join_alias_vars(root, root->parse,
(Node *) rte->subquery);
}
else if (rte->rtekind == RTE_FUNCTION)
@@ -1099,7 +1099,7 @@ preprocess_expression(PlannerInfo *root, Node *expr, int kind)
kind == EXPRKIND_VALUES ||
kind == EXPRKIND_TABLESAMPLE ||
kind == EXPRKIND_TABLEFUNC))
- expr = flatten_join_alias_vars(root->parse, expr);
+ expr = flatten_join_alias_vars(root, root->parse, expr);
/*
* Simplify constant expressions. For function RTEs, this was already
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 389f7d9ce7..7b22254173 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -1076,7 +1076,8 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* maybe even in the rewriter; but for now let's just fix this case here.)
*/
subquery->targetList = (List *)
- flatten_join_alias_vars(subroot->parse, (Node *) subquery->targetList);
+ flatten_join_alias_vars(subroot, subroot->parse,
+ (Node *) subquery->targetList);
/*
* Adjust level-0 varnos in subquery so that we can append its rangetable
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
index 99126958db..81fc3002fe 100644
--- a/src/backend/optimizer/util/var.c
+++ b/src/backend/optimizer/util/var.c
@@ -62,6 +62,7 @@ typedef struct
typedef struct
{
+ PlannerInfo *root; /* could be NULL! */
Query *query; /* outer Query */
int sublevels_up;
bool possible_sublink; /* could aliases include a SubLink? */
@@ -80,6 +81,10 @@ static bool pull_var_clause_walker(Node *node,
pull_var_clause_context *context);
static Node *flatten_join_alias_vars_mutator(Node *node,
flatten_join_alias_vars_context *context);
+static Node *add_nullingrels_if_needed(PlannerInfo *root, Node *newnode,
+ Var *oldvar);
+static bool is_standard_join_alias_expression(Node *newnode, Var *oldvar);
+static void adjust_standard_join_alias_expression(Node *newnode, Var *oldvar);
static Relids alias_relid_set(Query *query, Relids relids);
@@ -729,26 +734,42 @@ pull_var_clause_walker(Node *node, pull_var_clause_context *context)
* is the only way that the executor can directly handle whole-row Vars.
*
* This also adjusts relid sets found in some expression node types to
- * substitute the contained base rels for any join relid.
+ * substitute the contained base+OJ rels for any join relid.
*
* If a JOIN contains sub-selects that have been flattened, its join alias
* entries might now be arbitrary expressions, not just Vars. This affects
- * this function in one important way: we might find ourselves inserting
- * SubLink expressions into subqueries, and we must make sure that their
- * Query.hasSubLinks fields get set to true if so. If there are any
+ * this function in two important ways. First, we might find ourselves
+ * inserting SubLink expressions into subqueries, and we must make sure that
+ * their Query.hasSubLinks fields get set to true if so. If there are any
* SubLinks in the join alias lists, the outer Query should already have
* hasSubLinks = true, so this is only relevant to un-flattened subqueries.
+ * Second, we have to preserve any varnullingrels info attached to the
+ * alias Vars we're replacing. If the replacement expression is a Var or
+ * PlaceHolderVar or constructed from those, we can just add the
+ * varnullingrels bits to the existing nullingrels field(s); otherwise
+ * we have to add a PlaceHolderVar wrapper.
*
- * NOTE: this is used on not-yet-planned expressions. We do not expect it
- * to be applied directly to the whole Query, so if we see a Query to start
- * with, we do want to increment sublevels_up (this occurs for LATERAL
- * subqueries).
+ * NOTE: this is also used by the parser, to expand join alias Vars before
+ * checking GROUP BY validity. For that use-case, root will be NULL, which
+ * is why we have to pass the Query separately. We need the root itself only
+ * for making PlaceHolderVars. We can avoid making PlaceHolderVars in the
+ * parser's usage because it won't be dealing with arbitrary expressions:
+ * so long as adjust_standard_join_alias_expression can handle everything
+ * the parser would make as a join alias expression, we're OK.
*/
Node *
-flatten_join_alias_vars(Query *query, Node *node)
+flatten_join_alias_vars(PlannerInfo *root, Query *query, Node *node)
{
flatten_join_alias_vars_context context;
+ /*
+ * We do not expect this to be applied to the whole Query, only to
+ * expressions or LATERAL subqueries. Hence, if the top node is a Query,
+ * it's okay to immediately increment sublevels_up.
+ */
+ Assert(node != (Node *) query);
+
+ context.root = root;
context.query = query;
context.sublevels_up = 0;
/* flag whether join aliases could possibly contain SubLinks */
@@ -819,7 +840,9 @@ flatten_join_alias_vars_mutator(Node *node,
rowexpr->colnames = colnames;
rowexpr->location = var->location;
- return (Node *) rowexpr;
+ /* Lastly, add any varnullingrels to the replacement expression */
+ return add_nullingrels_if_needed(context->root, (Node *) rowexpr,
+ var);
}
/* Expand join alias reference */
@@ -846,7 +869,8 @@ flatten_join_alias_vars_mutator(Node *node,
if (context->possible_sublink && !context->inserted_sublink)
context->inserted_sublink = checkExprHasSubLink(newvar);
- return newvar;
+ /* Lastly, add any varnullingrels to the replacement expression */
+ return add_nullingrels_if_needed(context->root, newvar, var);
}
if (IsA(node, PlaceHolderVar))
{
@@ -861,6 +885,7 @@ flatten_join_alias_vars_mutator(Node *node,
{
phv->phrels = alias_relid_set(context->query,
phv->phrels);
+ /* we *don't* change phnullingrels */
}
return (Node *) phv;
}
@@ -894,9 +919,145 @@ flatten_join_alias_vars_mutator(Node *node,
(void *) context);
}
+/*
+ * Add oldvar's varnullingrels, if any, to a flattened join alias expression.
+ * The newnode has been copied, so we can modify it freely.
+ */
+static Node *
+add_nullingrels_if_needed(PlannerInfo *root, Node *newnode, Var *oldvar)
+{
+ if (oldvar->varnullingrels == NULL)
+ return newnode; /* nothing to do */
+ /* If possible, do it by adding to existing nullingrel fields */
+ if (is_standard_join_alias_expression(newnode, oldvar))
+ adjust_standard_join_alias_expression(newnode, oldvar);
+ else if (root)
+ {
+ /* We can insert a PlaceHolderVar to carry the nullingrels */
+ PlaceHolderVar *newphv;
+ Relids phrels = pull_varnos(root, newnode);
+
+ /* XXX what if phrels is empty? */
+ Assert(!bms_is_empty(phrels)); /* probably wrong */
+ newphv = make_placeholder_expr(root, (Expr *) newnode, phrels);
+ /* newphv has zero phlevelsup and NULL phnullingrels; fix it */
+ newphv->phlevelsup = oldvar->varlevelsup;
+ newphv->phnullingrels = bms_copy(oldvar->varnullingrels);
+ newnode = (Node *) newphv;
+ }
+ else
+ {
+ /* ooops, we're missing support for something the parser can make */
+ elog(ERROR, "unsupported join alias expression");
+ }
+ return newnode;
+}
+
+/*
+ * Check to see if we can insert nullingrels into this join alias expression
+ * without use of a separate PlaceHolderVar.
+ *
+ * This will handle Vars, PlaceHolderVars, and implicit-coercion and COALESCE
+ * expressions built from those. This coverage needs to handle anything
+ * that the parser would put into joinaliasvars.
+ * XXX it's probably incomplete at the moment.
+ */
+static bool
+is_standard_join_alias_expression(Node *newnode, Var *oldvar)
+{
+ if (newnode == NULL)
+ return false;
+ if (IsA(newnode, Var) &&
+ ((Var *) newnode)->varlevelsup == oldvar->varlevelsup)
+ return true;
+ else if (IsA(newnode, PlaceHolderVar) &&
+ ((PlaceHolderVar *) newnode)->phlevelsup == oldvar->varlevelsup)
+ return true;
+ else if (IsA(newnode, FuncExpr))
+ {
+ FuncExpr *fexpr = (FuncExpr *) newnode;
+
+ /*
+ * We need to assume that the function wouldn't produce non-NULL from
+ * NULL, which is reasonable for implicit coercions but otherwise not
+ * so much. (Looking at its strictness is likely overkill, and anyway
+ * it would cause us to fail if someone forgot to mark an implicit
+ * coercion as strict.)
+ */
+ if (fexpr->funcformat != COERCE_IMPLICIT_CAST ||
+ fexpr->args == NIL)
+ return false;
+
+ /*
+ * Examine only the first argument --- coercions might have additional
+ * arguments that are constants.
+ */
+ return is_standard_join_alias_expression(linitial(fexpr->args), oldvar);
+ }
+ else if (IsA(newnode, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) newnode;
+ ListCell *lc;
+
+ Assert(cexpr->args != NIL);
+ foreach(lc, cexpr->args)
+ {
+ if (!is_standard_join_alias_expression(lfirst(lc), oldvar))
+ return false;
+ }
+ return true;
+ }
+ else
+ return false;
+}
+
+/*
+ * Insert nullingrels into an expression accepted by
+ * is_standard_join_alias_expression.
+ */
+static void
+adjust_standard_join_alias_expression(Node *newnode, Var *oldvar)
+{
+ if (IsA(newnode, Var) &&
+ ((Var *) newnode)->varlevelsup == oldvar->varlevelsup)
+ {
+ Var *newvar = (Var *) newnode;
+
+ newvar->varnullingrels = bms_add_members(newvar->varnullingrels,
+ oldvar->varnullingrels);
+ }
+ else if (IsA(newnode, PlaceHolderVar) &&
+ ((PlaceHolderVar *) newnode)->phlevelsup == oldvar->varlevelsup)
+ {
+ PlaceHolderVar *newphv = (PlaceHolderVar *) newnode;
+
+ newphv->phnullingrels = bms_add_members(newphv->phnullingrels,
+ oldvar->varnullingrels);
+ }
+ else if (IsA(newnode, FuncExpr))
+ {
+ FuncExpr *fexpr = (FuncExpr *) newnode;
+
+ adjust_standard_join_alias_expression(linitial(fexpr->args), oldvar);
+ }
+ else if (IsA(newnode, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) newnode;
+ ListCell *lc;
+
+ Assert(cexpr->args != NIL);
+ foreach(lc, cexpr->args)
+ {
+ adjust_standard_join_alias_expression(lfirst(lc), oldvar);
+ }
+ }
+ else
+ Assert(false);
+}
+
/*
* alias_relid_set: in a set of RT indexes, replace joins by their
- * underlying base relids
+ * underlying base+OJ relids
*/
static Relids
alias_relid_set(Query *query, Relids relids)
diff --git a/src/backend/parser/parse_agg.c b/src/backend/parser/parse_agg.c
index 3ef9e8ee5e..c15fab0f68 100644
--- a/src/backend/parser/parse_agg.c
+++ b/src/backend/parser/parse_agg.c
@@ -1162,7 +1162,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
* entries are RTE_JOIN kind.
*/
if (hasJoinRTEs)
- groupClauses = (List *) flatten_join_alias_vars(qry,
+ groupClauses = (List *) flatten_join_alias_vars(NULL, qry,
(Node *) groupClauses);
/*
@@ -1206,7 +1206,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
groupClauses, hasJoinRTEs,
have_non_var_grouping);
if (hasJoinRTEs)
- clause = flatten_join_alias_vars(qry, clause);
+ clause = flatten_join_alias_vars(NULL, qry, clause);
check_ungrouped_columns(clause, pstate, qry,
groupClauses, groupClauseCommonVars,
have_non_var_grouping,
@@ -1217,7 +1217,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
groupClauses, hasJoinRTEs,
have_non_var_grouping);
if (hasJoinRTEs)
- clause = flatten_join_alias_vars(qry, clause);
+ clause = flatten_join_alias_vars(NULL, qry, clause);
check_ungrouped_columns(clause, pstate, qry,
groupClauses, groupClauseCommonVars,
have_non_var_grouping,
@@ -1546,7 +1546,7 @@ finalize_grouping_exprs_walker(Node *node,
Index ref = 0;
if (context->hasJoinRTEs)
- expr = flatten_join_alias_vars(context->qry, expr);
+ expr = flatten_join_alias_vars(NULL, context->qry, expr);
/*
* Each expression must match a grouping entry at the current
diff --git a/src/include/optimizer/optimizer.h b/src/include/optimizer/optimizer.h
index 7be1e5906b..1f5e0b24ca 100644
--- a/src/include/optimizer/optimizer.h
+++ b/src/include/optimizer/optimizer.h
@@ -202,6 +202,6 @@ extern bool contain_var_clause(Node *node);
extern bool contain_vars_of_level(Node *node, int levelsup);
extern int locate_var_of_level(Node *node, int levelsup);
extern List *pull_var_clause(Node *node, int flags);
-extern Node *flatten_join_alias_vars(Query *query, Node *node);
+extern Node *flatten_join_alias_vars(PlannerInfo *root, Query *query, Node *node);
#endif /* OPTIMIZER_H */
commit cac202978c90883ee1a8139d919c3e1ece9264fa
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sun Jul 10 15:27:35 2022 -0400
Teach FDWs about base-plus-outer-join relids.
Conversion of the planner to include OJ relids in join relids
affects FDWs that want to plan foreign joins. They *must* follow
suit when labeling foreign joins in order to match with the core
planner, but for many purposes (if postgres_fdw is any guide)
they'd prefer to consider only base relations within the join.
To support both requirements, redefine ForeignScan.fs_relids as
base+OJ relids, and add a new field fs_base_relids that's set up
by the core planner.
Another way we could do this is to keep fs_relids as just base
relids and make the new field be the one with OJ relids added.
While that would be more backwards-compatible in some sense,
it would be inconsistent with the naming used in the core planner,
and I think that it might allow some types of bugs to escape
immediate detection.
postgres_fdw also has one place where it needs to ignore varnullingrels
while matching Vars, similarly to the unfinished work in setrefs.c.
(That requirement will only affect join-planning FDWs, too, since
Vars seen at a base relation scan should never have any varnullingrels.)
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
index 8f4d8a5022..cfac1ea57b 100644
--- a/contrib/postgres_fdw/deparse.c
+++ b/contrib/postgres_fdw/deparse.c
@@ -3880,7 +3880,17 @@ get_relation_column_alias_ids(Var *node, RelOptInfo *foreignrel,
i = 1;
foreach(lc, foreignrel->reltarget->exprs)
{
- if (equal(lfirst(lc), (Node *) node))
+ Var *tlvar = (Var *) lfirst(lc);
+
+ /*
+ * As in setrefs.c, we match only on varno/varattno. Ideally there
+ * would be some cross-check on varnullingrels, but it's unclear what
+ * to do exactly; we don't have enough context to know what that value
+ * should be.
+ */
+ if (IsA(tlvar, Var) &&
+ tlvar->varno == node->varno &&
+ tlvar->varattno == node->varattno)
{
*colno = i;
return;
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 955a428e3d..d91c3622df 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -1513,13 +1513,13 @@ postgresBeginForeignScan(ForeignScanState *node, int eflags)
/*
* Identify which user to do the remote access as. This should match what
* ExecCheckRTEPerms() does. In case of a join or aggregate, use the
- * lowest-numbered member RTE as a representative; we would get the same
- * result from any.
+ * lowest-numbered member base RTE as a representative; we would get the
+ * same result from any.
*/
if (fsplan->scan.scanrelid > 0)
rtindex = fsplan->scan.scanrelid;
else
- rtindex = bms_next_member(fsplan->fs_relids, -1);
+ rtindex = bms_next_member(fsplan->fs_base_relids, -1);
rte = exec_rt_fetch(rtindex, estate);
userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
@@ -2405,7 +2405,7 @@ find_modifytable_subplan(PlannerInfo *root,
{
ForeignScan *fscan = (ForeignScan *) subplan;
- if (bms_is_member(rtindex, fscan->fs_relids))
+ if (bms_is_member(rtindex, fscan->fs_base_relids))
return fscan;
}
@@ -2831,8 +2831,8 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
* that setrefs.c won't update the string when flattening the
* rangetable. To find out what rtoffset was applied, identify the
* minimum RT index appearing in the string and compare it to the
- * minimum member of plan->fs_relids. (We expect all the relids in
- * the join will have been offset by the same amount; the Asserts
+ * minimum member of plan->fs_base_relids. (We expect all the relids
+ * in the join will have been offset by the same amount; the Asserts
* below should catch it if that ever changes.)
*/
minrti = INT_MAX;
@@ -2849,7 +2849,7 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
else
ptr++;
}
- rtoffset = bms_next_member(plan->fs_relids, -1) - minrti;
+ rtoffset = bms_next_member(plan->fs_base_relids, -1) - minrti;
/* Now we can translate the string */
relations = makeStringInfo();
@@ -2864,7 +2864,7 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
char *refname;
rti += rtoffset;
- Assert(bms_is_member(rti, plan->fs_relids));
+ Assert(bms_is_member(rti, plan->fs_base_relids));
rte = rt_fetch(rti, es->rtable);
Assert(rte->rtekind == RTE_RELATION);
/* This logic should agree with explain.c's ExplainTargetRel */
diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml
index d0b5951019..329affa30b 100644
--- a/doc/src/sgml/fdwhandler.sgml
+++ b/doc/src/sgml/fdwhandler.sgml
@@ -351,6 +351,17 @@ GetForeignJoinPaths(PlannerInfo *root,
it will supply at run time in the tuples it returns.
</para>
+ <note>
+ <para>
+ Beginning with <productname>PostgreSQL</productname> 16,
+ <structfield>fs_relids</structfield> includes the rangetable indexes
+ of outer joins, if any were involved in this join. The new field
+ <structfield>fs_base_relids</structfield> includes only base
+ relation indexes, and thus
+ mimics <structfield>fs_relids</structfield>'s old semantics.
+ </para>
+ </note>
+
<para>
See <xref linkend="fdw-planning"/> for additional information.
</para>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index e29c2ae206..ded37cb2e9 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1114,7 +1114,7 @@ ExplainPreScanNode(PlanState *planstate, Bitmapset **rels_used)
break;
case T_ForeignScan:
*rels_used = bms_add_members(*rels_used,
- ((ForeignScan *) plan)->fs_relids);
+ ((ForeignScan *) plan)->fs_base_relids);
break;
case T_CustomScan:
*rels_used = bms_add_members(*rels_used,
diff --git a/src/backend/executor/execScan.c b/src/backend/executor/execScan.c
index 043bb83f55..2b37266b6a 100644
--- a/src/backend/executor/execScan.c
+++ b/src/backend/executor/execScan.c
@@ -325,7 +325,7 @@ ExecScanReScan(ScanState *node)
* all of them.
*/
if (IsA(node->ps.plan, ForeignScan))
- relids = ((ForeignScan *) node->ps.plan)->fs_relids;
+ relids = ((ForeignScan *) node->ps.plan)->fs_base_relids;
else if (IsA(node->ps.plan, CustomScan))
relids = ((CustomScan *) node->ps.plan)->custom_relids;
else
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 76606faa3e..16247dd9ce 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -29,6 +29,7 @@
#include "optimizer/cost.h"
#include "optimizer/optimizer.h"
#include "optimizer/paramassign.h"
+#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/placeholder.h"
#include "optimizer/plancat.h"
@@ -4106,6 +4107,8 @@ create_foreignscan_plan(PlannerInfo *root, ForeignPath *best_path,
Index scan_relid = rel->relid;
Oid rel_oid = InvalidOid;
Plan *outer_plan = NULL;
+ Relids fs_base_relids;
+ int rtindex;
Assert(rel->fdwroutine != NULL);
@@ -4154,14 +4157,28 @@ create_foreignscan_plan(PlannerInfo *root, ForeignPath *best_path,
/*
* Likewise, copy the relids that are represented by this foreign scan. An
- * upper rel doesn't have relids set, but it covers all the base relations
- * participating in the underlying scan, so use root's all_baserels.
+ * upper rel doesn't have relids set, but it covers all the relations
+ * participating in the underlying scan/join, so use root->all_query_rels.
*/
if (rel->reloptkind == RELOPT_UPPER_REL)
- scan_plan->fs_relids = root->all_baserels;
+ scan_plan->fs_relids = root->all_query_rels;
else
scan_plan->fs_relids = best_path->path.parent->relids;
+ /*
+ * Join relid sets include relevant outer joins, but FDWs may need to know
+ * which are the included base rels. That's a bit tedious to get without
+ * access to the plan-time data structures, so compute it here.
+ */
+ fs_base_relids = NULL;
+ rtindex = -1;
+ while ((rtindex = bms_next_member(scan_plan->fs_relids, rtindex)) >= 0)
+ {
+ if (find_base_rel_ignore_join(root, rtindex) != NULL)
+ fs_base_relids = bms_add_member(fs_base_relids, rtindex);
+ }
+ scan_plan->fs_base_relids = fs_base_relids;
+
/*
* If this is a foreign join, and to make it valid to push down we had to
* assume that the current user is the same as some user explicitly named
@@ -5806,8 +5823,9 @@ make_foreignscan(List *qptlist,
node->fdw_private = fdw_private;
node->fdw_scan_tlist = fdw_scan_tlist;
node->fdw_recheck_quals = fdw_recheck_quals;
- /* fs_relids will be filled in by create_foreignscan_plan */
+ /* fs_relids, fs_base_relids will be filled by create_foreignscan_plan */
node->fs_relids = NULL;
+ node->fs_base_relids = NULL;
/* fsSystemCol will be filled in by create_foreignscan_plan */
node->fsSystemCol = false;
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 14f5c0f897..3b720ec5d6 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1533,6 +1533,7 @@ set_foreignscan_references(PlannerInfo *root,
}
fscan->fs_relids = offset_relid_set(fscan->fs_relids, rtoffset);
+ fscan->fs_base_relids = offset_relid_set(fscan->fs_base_relids, rtoffset);
/* Adjust resultRelation if it's valid */
if (fscan->resultRelation > 0)
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index dca2a21e7a..7e98d0b7a3 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -688,6 +688,7 @@ typedef struct WorkTableScan
* When the plan node represents a foreign join, scan.scanrelid is zero and
* fs_relids must be consulted to identify the join relation. (fs_relids
* is valid for simple scans as well, but will always match scan.scanrelid.)
+ * fs_relids includes outer joins; fs_base_relids does not.
*
* If the FDW's PlanDirectModify() callback decides to repurpose a ForeignScan
* node to perform the UPDATE or DELETE operation directly in the remote
@@ -707,7 +708,8 @@ typedef struct ForeignScan
List *fdw_private; /* private data for FDW */
List *fdw_scan_tlist; /* optional tlist describing scan tuple */
List *fdw_recheck_quals; /* original quals not in scan.plan.qual */
- Bitmapset *fs_relids; /* RTIs generated by this scan */
+ Bitmapset *fs_relids; /* base+OJ RTIs generated by this scan */
+ Bitmapset *fs_base_relids; /* base RTIs generated by this scan */
bool fsSystemCol; /* true if any "system column" is needed */
} ForeignScan;
On Sun, Jul 10, 2022 at 12:39 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Here's v2 of this patch series. It's functionally identical to v1,
but I've rebased it over the recent auto-node-support-generation
changes, and also extracted a few separable bits in hopes of making
the main planner patch smaller. (It's still pretty durn large,
unfortunately.) Unlike the original submission, each step will
compile on its own, though the intermediate states mostly don't
pass all regression tests.
regards, tom lane
Hi,
For v2-0004-cope-with-nullability-in-planner.patch.
In remove_unneeded_nulling_relids():
+ if (removable_relids == NULL)
Why is bms_is_empty() not used in the above check ?
Earlier there is `if (bms_is_empty(old_nulling_relids))`
+typedef struct reduce_outer_joins_partial_state
Since there are already reduce_outer_joins_pass1_state and reduce_outer_joins_pass2_state, a comment above reduce_outer_joins_partial_state would help other people follow its purpose.
+ if (j->rtindex)
+ {
+ if (j->jointype == JOIN_INNER)
+ {
+ if (include_inner_joins)
+ result = bms_add_member(result, j->rtindex);
+ }
+ else
+ {
+ if (include_outer_joins)
+ {
+ if (j->jointype == JOIN_INNER)
+ {
+ if (include_inner_joins)
+ result = bms_add_member(result, j->rtindex);
+ }
+ else
+ {
+ if (include_outer_joins)
Since there are other join types beside JOIN_INNER, should there be an assertion in the else block ? e.g. jointype wouldn't be JOIN_UNIQUE_INNER.
Cheers
Zhihong Yu <zyu@yugabyte.com> writes:
> In remove_unneeded_nulling_relids():
> + if (removable_relids == NULL)
> Why is bms_is_empty() not used in the above check ?
We initialized that to NULL just a few lines above, and then did
nothing to it other than perhaps bms_add_member, so it's impossible
for it to be empty-and-yet-not-NULL.
> +typedef struct reduce_outer_joins_partial_state
> Since there are already reduce_outer_joins_pass1_state
> and reduce_outer_joins_pass2_state, a comment
> above reduce_outer_joins_partial_state would help other people follow its
> purpose.
We generally document these sorts of structs in the using code,
not at the struct declaration.
> + if (j->rtindex)
> + {
> + if (j->jointype == JOIN_INNER)
> + {
> + if (include_inner_joins)
> + result = bms_add_member(result, j->rtindex);
> + }
> + else
> + {
> + if (include_outer_joins)
> Since there are other join types beside JOIN_INNER, should there be an
> assertion in the else block?
Like what? We don't particularly care what the join type is here,
as long as it's not INNER. In any case there is plenty of nearby
code checking for unsupported join types.
regards, tom lane
On Mon, Jul 11, 2022 at 3:38 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Here's v2 of this patch series. It's functionally identical to v1,
but I've rebased it over the recent auto-node-support-generation
changes, and also extracted a few separable bits in hopes of making
the main planner patch smaller. (It's still pretty durn large,
unfortunately.) Unlike the original submission, each step will
compile on its own, though the intermediate states mostly don't
pass all regression tests.
Take the query below as an example:
select a.i, ss.jj from a left join (select b.i, b.j + 1 as jj from b) ss
on a.i = ss.i;
Previously the expression 'b.j + 1' would not be wrapped in a
PlaceHolderVar, since it contains a Var of the subquery and meanwhile it
does not contain any non-strict constructs. And now in the patch, we
would insert a PlaceHolderVar for it, in order to have a place to store
varnullingrels. So the plan for the above query now becomes:
# explain (verbose, costs off) select a.i, ss.jj from a left join
(select b.i, b.j + 1 as jj from b) ss on a.i = ss.i;
QUERY PLAN
----------------------------------
Hash Right Join
Output: a.i, ((b.j + 1))
Hash Cond: (b.i = a.i)
-> Seq Scan on public.b
Output: b.i, (b.j + 1)
-> Hash
Output: a.i
-> Seq Scan on public.a
Output: a.i
(9 rows)
Note that the evaluation of expression 'b.j + 1' now occurs below the
outer join. Is this something we need to be concerned about?
Thanks
Richard
Richard Guo <guofenglinux@gmail.com> writes:
> Note that the evaluation of expression 'b.j + 1' now occurs below the
> outer join. Is this something we need to be concerned about?
It seems more formally correct to me, but perhaps somebody would
complain about possibly-useless expression evals. We could likely
re-complicate the logic that inserts PHVs during pullup so that it
looks for Vars it can apply the nullingrels to. Maybe there's an
opportunity to share code with flatten_join_alias_vars?
regards, tom lane
On Tue, Jul 12, 2022 at 9:37 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Richard Guo <guofenglinux@gmail.com> writes:
> Note that the evaluation of expression 'b.j + 1' now occurs below the
> outer join. Is this something we need to be concerned about?
It seems more formally correct to me, but perhaps somebody would
complain about possibly-useless expression evals. We could likely
re-complicate the logic that inserts PHVs during pullup so that it
looks for Vars it can apply the nullingrels to. Maybe there's an
opportunity to share code with flatten_join_alias_vars?
adjust_standard_join_alias_expression() to do that?
But I'm not sure which is better, to evaluate the expression below or
above the outer join. It seems to me that if the size of base rel is
large and somehow the size of the joinrel is small, evaluation above the
outer join would win. And in the opposite case evaluation below the
outer join would be better.
Thanks
Richard
Richard Guo <guofenglinux@gmail.com> writes:
> But I'm not sure which is better, to evaluate the expression below or
> above the outer join. It seems to me that if the size of base rel is
> large and somehow the size of the joinrel is small, evaluation above the
> outer join would win. And in the opposite case evaluation below the
> outer join would be better.
Reasonable question. But I think for the purposes of this patch,
it's better to keep the old behavior as much as we can. People
have probably relied on it while tuning queries. (I'm not saying
it has to be *exactly* bug-compatible, but simple cases like your
example probably should work the same.)
regards, tom lane
Here's a rebase up to HEAD, mostly to placate the cfbot.
I accounted for d8e34fa7a (s/all_baserels/all_query_rels/
in those places) and made one tiny bug-fix change.
Nothing substantive as yet.
regards, tom lane
commit 37d5ef90d226b0b170a755e221794acb4ff2771b
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Aug 1 14:26:48 2022 -0400
Add overview documentation.
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index 41c120e0cd..2b30d22aed 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -295,6 +295,191 @@ Therefore, we don't merge FROM-lists if the result would have too many
FROM-items in one list.
+Vars and PlaceHolderVars
+------------------------
+
+A Var node is simply the parse-tree representation of a table column
+reference. However, in the presence of outer joins, that concept is
+more subtle than it might seem. We need to distinguish the values of
+a Var "above" and "below" any outer join that could force the Var to
+null. As an example, consider
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE foo(t2.z)
+
+(Assume foo() is not strict, so that we can't reduce the left join to
+a plain join.) A naive implementation might try to push the foo(t2.z)
+call down to the scan of t2, but that is not correct because
+(a) what foo() should actually see for a null-extended join row is NULL,
+and (b) if foo() returns false, we should suppress the t1 row from the
+join altogether, not emit it with a null-extended t2 row. On the other
+hand, it *would* be correct (and desirable) to push the call down to
+the scan level if the query were
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y AND foo(t2.z))
+
+This motivates considering "t2.z" within the left join's ON clause
+to be a different value from "t2.z" outside the JOIN clause. The
+former can be identified with t2.z as seen at the relation scan level,
+but the latter can't.
+
+Another example occurs in connection with EquivalenceClasses (discussed
+below). Given
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE t1.x = 42
+
+we would like to put t1.x and t2.y and 42 into the same EquivalenceClass
+and then derive "t2.y = 42" to use as a restriction clause for the scan
+of t2. However, it'd be wrong to conclude that t2.y will always have
+the value 42, or that it's equal to t1.x in every joined row. We can
+solve this problem by deeming that "t2.y" in the ON clause refers to
+the relation-scan-level value of t2.y, but not to the value that y will
+have in joined rows, where it might be NULL rather than equal to t1.x.
+
+Therefore, Var nodes are decorated with "varnullingrels", which are sets
+of the rangetable indexes of outer joins that potentially null this Var
+at the point where it appears in the query. (Using a set, not an
+ordered list, is fine since it doesn't matter which join forced the
+value to null; and that avoids having to change the representation when
+we consider different outer-join orders.) In the examples above, all
+occurrences of t1.x would have empty varnullingrels, since the left join
+doesn't null t1. The t2 references within the JOIN ON clauses would
+also have empty varnullingrels, but other references to t2 columns would
+be labeled with the index of the JOIN's rangetable entry (RTE), so that
+they'd be understood as potentially different from the t2 values seen at
+scan level. Labeling t2.z in the WHERE clause with the JOIN's RT index
+lets us recognize that that occurrence of foo(t2.z) cannot be pushed
+down to the t2 scan level: we cannot evaluate that value at the scan
+level, but only after the join has been done.
+
+For LEFT and RIGHT outer joins, only Vars coming from the nullable side
+of the join are marked with that join's RT index. For FULL joins, all
+Vars are marked. (Such marking doesn't let us tell which side of the
+full join a Var came from; but that information can be found elsewhere
+at need.)
+
+Notionally, a Var having nonempty varnullingrels can be thought of as
+ CASE WHEN any-of-these-outer-joins-produced-a-null-extended-row
+ THEN NULL
+ ELSE the-scan-level-value-of-the-column
+ END
+It's only notional, because no such calculation is ever done explicitly.
+In a finished plan, Vars occurring in scan-level plan nodes represent
+the actual table column values, but upper-level Vars are always
+references to outputs of lower-level plan nodes. When a join node emits
+a null-extended row, it just returns nulls for the relevant output
+columns rather than copying up values from its input. Because we don't
+ever have to do this calculation explicitly, it's not necessary to
+distinguish which side of an outer join got null-extended, which'd
+otherwise be essential information for FULL JOIN cases.
+
+Outer join identity 3 (discussed above) complicates this picture
+a bit. In the form
+ A leftjoin (B leftjoin C on (Pbc)) on (Pab)
+all of the Vars in clauses Pbc and Pab will have empty varnullingrels,
+but if we start with
+ (A leftjoin B on (Pab)) leftjoin C on (Pbc)
+then the parser will have marked Pbc's B Vars with the A/B join's
+RT index, making this form artificially different from the first.
+We resolve this by, after noting that Pbc is strict, running
+through that clause and removing any varnullingrels references to
+left joins in the lefthand side. That makes the clause equivalent
+to what it would have looked like if the first form were presented,
+so that we can freely consider both join orders. However, because
+we have done this, if we do construct a plan based on the second
+join order then we cannot cross-check that B Vars appearing above
+the A/B join are all marked with that join's RT index. That would
+be a useful cross-check to have to catch planner bugs, but it
+doesn't seem useful enough to justify the extra complication of
+devising a representation that would support it.
+
+Outer joins also complicate handling of subquery pull-up. Consider
+
+ SELECT ..., ss.x FROM tab1
+ LEFT JOIN (SELECT *, 42 AS x FROM tab2) ss ON ...
+
+We want to be able to pull up the subquery as discussed previously,
+but we can't just replace the "ss.x" Var in the top-level SELECT list
+with the constant 42. That'd result in always emitting 42, rather
+than emitting NULL in null-extended join rows.
+
+To solve this, we introduce the concept of PlaceHolderVars.
+A PlaceHolderVar is somewhat like a Var, in that its value originates
+at a relation scan level and can then be forced to null by higher-level
+outer joins; hence PlaceHolderVars carry a set of nulling rel IDs just
+like Vars. Unlike a Var, whose original value comes from a table,
+a PlaceHolderVar's original value is defined by a query-determined
+expression ("42" in this example); so we represent the PlaceHolderVar
+as a node with that expression as child. We insert a PlaceHolderVar
+whenever subquery pullup needs to replace a subquery-referencing Var
+that has nonempty varnullingrels with an expression that is not simply a
+Var. (When the replacement expression is a pulled-up Var, we can just
+add the replaced Var's varnullingrels to its set. Also, if the replaced
+Var has empty varnullingrels, we don't need a PlaceHolderVar: there is
+nothing that'd force the value to null, so the pulled-up expression is
+fine to use as-is.) In a finished plan, a PlaceHolderVar becomes just
+the contained expression at whatever plan level it's supposed to be
+evaluated at, and then upper-level occurrences are replaced by
+references to that output column of the lower plan level. That causes
+the value to go to null when appropriate at an outer join, in the same
+way as for Vars. Thus, PlaceHolderVars are never seen outside the
+planner.
+
+PlaceHolderVars (PHVs) are more complicated than Vars in another way:
+their original value might need to be calculated at a join, not a
+base-level relation scan. This can happen if a pulled-up subquery
+contains a join. Because of this, a PHV can create a join order
+constraint that wouldn't otherwise exist, to ensure that it can
+be calculated before it is used. A PHV's expression can also contain
+LATERAL references, adding complications that are discussed below.
+
+
+Relation Identification and Qual Clause Placement
+-------------------------------------------------
+
+A qual clause obtained from WHERE or JOIN/ON can be enforced at the lowest
+scan or join level that includes all relations used in the clause. For
+this purpose we consider that outer joins listed in varnullingrels or
+phnullingrels are used in the clause, since we can't compute the qual's
+result correctly until we know whether such Vars have gone to null.
+
+The one exception to this general rule is that a non-degenerate outer
+JOIN/ON qual (one that references the non-nullable side of the join)
+cannot be enforced below that join, even if it doesn't reference the
+nullable side. Pushing it down into the non-nullable side would result
+in rows disappearing from the join's result, rather than appearing as
+null-extended rows. To handle that, when we identify such a qual we
+artificially add the join's minimum input relid set to the set of
+relations it is considered to use, forcing it to be evaluated exactly at
+that join level. The same happens for outer-join quals that mention no
+relations at all.
+
+When attaching a qual clause to a join plan node that is performing
+an outer join, the qual clause is considered a "join clause" (that
+is, it is applied before the join) if it does not use that specific
+outer join, or a "filter clause" (applied after the join) if it does
+use that outer join.
+
+These things lead us to identify join relations within the planner
+by the sets of base relation RT indexes plus outer join RT indexes
+that they include. In that way, the sets of relations used by qual
+clauses can be directly compared to join relations' relid sets to
+see where to place the clauses. These identifying sets are unique
+because, for any given collection of base relations, there is only
+one valid set of outer joins to have performed along the way to
+joining that set of base relations (although the order of applying
+them could vary, as discussed above).
+
+SEMI joins do not have RT indexes, because they are artifacts made by
+the planner rather than the parser. (We could create rangetable
+entries for them, but there seems no need at present.) This does not
+cause a problem for qual placement, because the nullable side of a
+semijoin is not referenceable from above the join, so there is never a
+need to cite it in varnullingrels or phnullingrels. It does not cause
+a problem for join relation identification either, since again whether
+a semijoin has been completed is implicit in the set of base relations
+included in the join.
+
+
Optimizer Functions
-------------------
@@ -437,11 +622,10 @@ inputs.
EquivalenceClasses
------------------
-During the deconstruct_jointree() scan of the query's qual clauses, we look
-for mergejoinable equality clauses A = B whose applicability is not delayed
-by an outer join; these are called "equivalence clauses". When we find
-one, we create an EquivalenceClass containing the expressions A and B to
-record this knowledge. If we later find another equivalence clause B = C,
+During the deconstruct_jointree() scan of the query's qual clauses, we
+look for mergejoinable equality clauses A = B. When we find one, we
+create an EquivalenceClass containing the expressions A and B to record
+that they are equal. If we later find another equivalence clause B = C,
we add C to the existing EquivalenceClass for {A B}; this may require
merging two existing EquivalenceClasses. At the end of the scan, we have
sets of values that are known all transitively equal to each other. We can
@@ -473,15 +657,26 @@ asserts that at any plan node where more than one of its member values
can be computed, output rows in which the values are not all equal may
be discarded without affecting the query result. (We require all levels
of the plan to enforce EquivalenceClasses, hence a join need not recheck
-equality of values that were computable by one of its children.) For an
-ordinary EquivalenceClass that is "valid everywhere", we can further infer
-that the values are all non-null, because all mergejoinable operators are
-strict. However, we also allow equivalence clauses that appear below the
-nullable side of an outer join to form EquivalenceClasses; for these
-classes, the interpretation is that either all the values are equal, or
-all (except pseudo-constants) have gone to null. (This requires a
-limitation that non-constant members be strict, else they might not go
-to null when the other members do.) Consider for example
+equality of values that were computable by one of its children.)
+
+We can further infer that the values are all non-null, because all
+mergejoinable operators are strict. This is a little tricky in the
+presence of outer joins. Consider
+
+ SELECT *
+ FROM a LEFT JOIN
+ (SELECT * FROM b LEFT JOIN c ON b.y = c.z WHERE b.y = 10) ss
+ ON a.x = ss.y
+ WHERE a.x = 42;
+
+We can form the EquivalenceClass {b.y c.z 10} and thereby apply c.z = 10
+while scanning c. However it would be incorrect to conclude that a.x
+is also a member of that EquivalenceClass. Instead, we form a second
+EquivalenceClass {a.x ss.y 42}, where (as discussed earlier) ss.y
+references the same table column as b.y but has a different
+varnullingrels label and is therefore considered a distinct Var.
+
+If the lower join were INNER:
SELECT *
FROM a LEFT JOIN
@@ -489,40 +684,23 @@ to null when the other members do.) Consider for example
ON a.x = ss.y
WHERE a.x = 42;
-We can form the below-outer-join EquivalenceClass {b.y c.z 10} and thereby
-apply c.z = 10 while scanning c. (The reason we disallow outerjoin-delayed
-clauses from forming EquivalenceClasses is exactly that we want to be able
-to push any derived clauses as far down as possible.) But once above the
-outer join it's no longer necessarily the case that b.y = 10, and thus we
-cannot use such EquivalenceClasses to conclude that sorting is unnecessary
-(see discussion of PathKeys below).
-
-In this example, notice also that a.x = ss.y (really a.x = b.y) is not an
-equivalence clause because its applicability to b is delayed by the outer
-join; thus we do not try to insert b.y into the equivalence class {a.x 42}.
-But since we see that a.x has been equated to 42 above the outer join, we
-are able to form a below-outer-join class {b.y 42}; this restriction can be
-added because no b/c row not having b.y = 42 can contribute to the result
-of the outer join, and so we need not compute such rows. Now this class
-will get merged with {b.y c.z 10}, leading to the contradiction 10 = 42,
-which lets the planner deduce that the b/c join need not be computed at all
-because none of its rows can contribute to the outer join. (This gets
-implemented as a gating Result filter, since more usually the potential
-contradiction involves Param values rather than just Consts, and thus has
-to be checked at runtime.)
+then ss.y is not any different from b.y and we'd end up with the
+EquivalenceClass {a.x b.y c.z 10 42}. This leads to the contradiction
+10 = 42, which lets the planner deduce that the b/c join need not be
+computed at all because none of its rows can contribute to the outer
+join. (This gets implemented as a gating Result filter, since more
+usually the potential contradiction involves Param values rather than
+just Consts, and thus has to be checked at runtime.)
To aid in determining the sort ordering(s) that can work with a mergejoin,
we mark each mergejoinable clause with the EquivalenceClasses of its left
-and right inputs. For an equivalence clause, these are of course the same
-EquivalenceClass. For a non-equivalence mergejoinable clause (such as an
-outer-join qualification), we generate two separate EquivalenceClasses for
-the left and right inputs. This may result in creating single-item
-equivalence "classes", though of course these are still subject to merging
-if other equivalence clauses are later found to bear on the same
-expressions.
-
-Another way that we may form a single-item EquivalenceClass is in creation
-of a PathKey to represent a desired sort order (see below). This is a bit
+and right inputs. (These are in fact always the same EquivalenceClass.)
+
+In some cases we will form single-item EquivalenceClasses. This happens
+if an ORDER BY or GROUP BY key is not mentioned in any equivalence
+clause. We need to reason about sort orders in such queries, and our
+representation of sort ordering is a PathKey (see below) which uses an
+EquivalenceClass, so we have to make an EquivalenceClass. This is a bit
different from the above cases because such an EquivalenceClass might
contain an aggregate function or volatile expression. (A clause containing
a volatile function will never be considered mergejoinable, even if its top
@@ -579,7 +757,7 @@ Index scans have Path.pathkeys that represent the chosen index's ordering,
if any. A single-key index would create a single-PathKey list, while a
multi-column index generates a list with one element per key index column.
Non-key columns specified in the INCLUDE clause of covering indexes don't
-have corresponding PathKeys in the list, because the have no influence on
+have corresponding PathKeys in the list, because they have no influence on
index ordering. (Actually, since an index can be scanned either forward or
backward, there are two possible sort orders and two possible PathKey lists
it can generate.)
@@ -655,14 +833,9 @@ redundancy, we save time and improve planning, since the planner will more
easily recognize equivalent orderings as being equivalent.
Another interesting property is that if the underlying EquivalenceClass
-contains a constant and is not below an outer join, then the pathkey is
-completely redundant and need not be sorted by at all! Every row must
-contain the same constant value, so there's no need to sort. (If the EC is
-below an outer join, we still have to sort, since some of the rows might
-have gone to null and others not. In this case we must be careful to pick
-a non-const member to sort by. The assumption that all the non-const
-members go to null at the same plan level is critical here, else they might
-not produce the same sort order.) This might seem pointless because users
+contains a constant, then the pathkey is completely redundant and need
+not be sorted by at all! Every row must contain the same value, so
+there's no need to sort. This might seem pointless because users
are unlikely to write "... WHERE x = 42 ORDER BY x", but it allows us to
recognize when particular index columns are irrelevant to the sort order:
if we have "... WHERE x = 42 ORDER BY y", scanning an index on (x,y)
@@ -670,15 +843,6 @@ produces correctly ordered data without a sort step. We used to have very
ugly ad-hoc code to recognize that in limited contexts, but discarding
constant ECs from pathkeys makes it happen cleanly and automatically.
-You might object that a below-outer-join EquivalenceClass doesn't always
-represent the same values at every level of the join tree, and so using
-it to uniquely identify a sort order is dubious. This is true, but we
-can avoid dealing with the fact explicitly because we always consider that
-an outer join destroys any ordering of its nullable inputs. Thus, even
-if a path was sorted by {a.x} below an outer join, we'll re-sort if that
-sort ordering was important; and so using the same PathKey for both sort
-orderings doesn't create any real problem.
-
Order of processing for EquivalenceClasses and PathKeys
-------------------------------------------------------
commit 7d986729a5946b5ee368ae22888e8cc1c9015c80
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Aug 1 14:32:44 2022 -0400
Improve performance of adjust_appendrel_attrs_multilevel.
The present implementations of adjust_appendrel_attrs_multilevel and
its sibling adjust_child_relids_multilevel are very messy, because
they work by reconstructing the relids of the child's immediate
parent and then seeing if that's bms_equal to the relids of the
target parent. Aside from being quite inefficient, this will not
work for joinrels whose relids contain outer-join relids in addition
to baserels.
The whole thing can be solved at a stroke by adding explicit parent
and top_parent links to child RelOptInfos, and making these functions
work with RelOptInfo pointers instead of relids. Doing that is
simpler for most callers, too.
In my original version of this patch, I got rid of
RelOptInfo.top_parent_relids on the grounds that it was now redundant.
However, that adds a lot of code churn in places that otherwise would
not need changing, and arguably the extra indirection needed to fetch
top_parent->relids in those places costs something. So this version
leaves that field in place.
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 60c0e3f108..f8a97622b1 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -1760,8 +1760,8 @@ generate_join_implied_equalities_broken(PlannerInfo *root,
if (IS_OTHER_REL(inner_rel) && result != NIL)
result = (List *) adjust_appendrel_attrs_multilevel(root,
(Node *) result,
- inner_rel->relids,
- inner_rel->top_parent_relids);
+ inner_rel,
+ inner_rel->top_parent);
return result;
}
@@ -2626,8 +2626,8 @@ add_child_rel_equivalences(PlannerInfo *root,
child_expr = (Expr *)
adjust_appendrel_attrs_multilevel(root,
(Node *) cur_em->em_expr,
- child_relids,
- top_parent_relids);
+ child_rel,
+ child_rel->top_parent);
}
/*
@@ -2768,8 +2768,8 @@ add_child_join_rel_equivalences(PlannerInfo *root,
child_expr = (Expr *)
adjust_appendrel_attrs_multilevel(root,
(Node *) cur_em->em_expr,
- child_relids,
- top_parent_relids);
+ child_joinrel,
+ child_joinrel->top_parent);
}
/*
@@ -2791,8 +2791,8 @@ add_child_join_rel_equivalences(PlannerInfo *root,
new_nullable_relids =
adjust_child_relids_multilevel(root,
new_nullable_relids,
- child_relids,
- top_parent_relids);
+ child_joinrel,
+ child_joinrel->top_parent);
(void) add_eq_member(cur_ec, child_expr,
new_relids, new_nullable_relids,
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 06ad856eac..f6baa2a765 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -1791,8 +1791,8 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
withCheckOptions = (List *)
adjust_appendrel_attrs_multilevel(root,
(Node *) withCheckOptions,
- this_result_rel->relids,
- top_result_rel->relids);
+ this_result_rel,
+ top_result_rel);
withCheckOptionLists = lappend(withCheckOptionLists,
withCheckOptions);
}
@@ -1804,8 +1804,8 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
returningList = (List *)
adjust_appendrel_attrs_multilevel(root,
(Node *) returningList,
- this_result_rel->relids,
- top_result_rel->relids);
+ this_result_rel,
+ top_result_rel);
returningLists = lappend(returningLists,
returningList);
}
@@ -1826,13 +1826,13 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
leaf_action->qual =
adjust_appendrel_attrs_multilevel(root,
(Node *) action->qual,
- this_result_rel->relids,
- top_result_rel->relids);
+ this_result_rel,
+ top_result_rel);
leaf_action->targetList = (List *)
adjust_appendrel_attrs_multilevel(root,
(Node *) action->targetList,
- this_result_rel->relids,
- top_result_rel->relids);
+ this_result_rel,
+ top_result_rel);
if (leaf_action->commandType == CMD_UPDATE)
leaf_action->updateColnos =
adjust_inherited_attnums_multilevel(root,
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index 9d4bb47027..62cccf9d87 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -479,39 +479,34 @@ adjust_appendrel_attrs_mutator(Node *node,
/*
* adjust_appendrel_attrs_multilevel
- * Apply Var translations from a toplevel appendrel parent down to a child.
+ * Apply Var translations from an appendrel parent down to a child.
*
- * In some cases we need to translate expressions referencing a parent relation
- * to reference an appendrel child that's multiple levels removed from it.
+ * Replace Vars in the "node" expression that reference "parentrel" with
+ * the appropriate Vars for "childrel". childrel can be more than one
+ * inheritance level removed from parentrel.
*/
Node *
adjust_appendrel_attrs_multilevel(PlannerInfo *root, Node *node,
- Relids child_relids,
- Relids top_parent_relids)
+ RelOptInfo *childrel,
+ RelOptInfo *parentrel)
{
AppendRelInfo **appinfos;
- Bitmapset *parent_relids = NULL;
int nappinfos;
- int cnt;
-
- Assert(bms_num_members(child_relids) == bms_num_members(top_parent_relids));
-
- appinfos = find_appinfos_by_relids(root, child_relids, &nappinfos);
- /* Construct relids set for the immediate parent of given child. */
- for (cnt = 0; cnt < nappinfos; cnt++)
+ /* Recurse if immediate parent is not the top parent. */
+ if (childrel->parent != parentrel)
{
- AppendRelInfo *appinfo = appinfos[cnt];
-
- parent_relids = bms_add_member(parent_relids, appinfo->parent_relid);
+ if (childrel->parent)
+ node = adjust_appendrel_attrs_multilevel(root, node,
+ childrel->parent,
+ parentrel);
+ else
+ elog(ERROR, "childrel is not a child of parentrel");
}
- /* Recurse if immediate parent is not the top parent. */
- if (!bms_equal(parent_relids, top_parent_relids))
- node = adjust_appendrel_attrs_multilevel(root, node, parent_relids,
- top_parent_relids);
+ /* Now translate for this child. */
+ appinfos = find_appinfos_by_relids(root, childrel->relids, &nappinfos);
- /* Now translate for this child */
node = adjust_appendrel_attrs(root, node, nappinfos, appinfos);
pfree(appinfos);
@@ -554,56 +549,43 @@ adjust_child_relids(Relids relids, int nappinfos, AppendRelInfo **appinfos)
}
/*
- * Replace any relid present in top_parent_relids with its child in
- * child_relids. Members of child_relids can be multiple levels below top
- * parent in the partition hierarchy.
+ * Substitute child's relids for parent's relids in a Relid set.
+ * The childrel can be multiple inheritance levels below the parent.
*/
Relids
adjust_child_relids_multilevel(PlannerInfo *root, Relids relids,
- Relids child_relids, Relids top_parent_relids)
+ RelOptInfo *childrel,
+ RelOptInfo *parentrel)
{
AppendRelInfo **appinfos;
int nappinfos;
- Relids parent_relids = NULL;
- Relids result;
- Relids tmp_result = NULL;
- int cnt;
/*
- * If the given relids set doesn't contain any of the top parent relids,
- * it will remain unchanged.
+ * If the given relids set doesn't contain any of the parent relids, it
+ * will remain unchanged.
*/
- if (!bms_overlap(relids, top_parent_relids))
+ if (!bms_overlap(relids, parentrel->relids))
return relids;
- appinfos = find_appinfos_by_relids(root, child_relids, &nappinfos);
-
- /* Construct relids set for the immediate parent of the given child. */
- for (cnt = 0; cnt < nappinfos; cnt++)
- {
- AppendRelInfo *appinfo = appinfos[cnt];
-
- parent_relids = bms_add_member(parent_relids, appinfo->parent_relid);
- }
-
/* Recurse if immediate parent is not the top parent. */
- if (!bms_equal(parent_relids, top_parent_relids))
+ if (childrel->parent != parentrel)
{
- tmp_result = adjust_child_relids_multilevel(root, relids,
- parent_relids,
- top_parent_relids);
- relids = tmp_result;
+ if (childrel->parent)
+ relids = adjust_child_relids_multilevel(root, relids,
+ childrel->parent,
+ parentrel);
+ else
+ elog(ERROR, "childrel is not a child of parentrel");
}
- result = adjust_child_relids(relids, nappinfos, appinfos);
+ /* Now translate for this child. */
+ appinfos = find_appinfos_by_relids(root, childrel->relids, &nappinfos);
+
+ relids = adjust_child_relids(relids, nappinfos, appinfos);
- /* Free memory consumed by any intermediate result. */
- if (tmp_result)
- bms_free(tmp_result);
- bms_free(parent_relids);
pfree(appinfos);
- return result;
+ return relids;
}
/*
@@ -694,8 +676,8 @@ get_translated_update_targetlist(PlannerInfo *root, Index relid,
*processed_tlist = (List *)
adjust_appendrel_attrs_multilevel(root,
(Node *) root->processed_tlist,
- bms_make_singleton(relid),
- bms_make_singleton(root->parse->resultRelation));
+ find_base_rel(root, relid),
+ find_base_rel(root, root->parse->resultRelation));
if (update_colnos)
*update_colnos =
adjust_inherited_attnums_multilevel(root, root->update_colnos,
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index dd64b46086..e10561d843 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -4026,8 +4026,8 @@ reparameterize_path_by_child(PlannerInfo *root, Path *path,
#define ADJUST_CHILD_ATTRS(node) \
((node) = \
(List *) adjust_appendrel_attrs_multilevel(root, (Node *) (node), \
- child_rel->relids, \
- child_rel->top_parent_relids))
+ child_rel, \
+ child_rel->top_parent))
#define REPARAMETERIZE_CHILD_PATH(path) \
do { \
@@ -4244,8 +4244,8 @@ do { \
old_ppi = new_path->param_info;
required_outer =
adjust_child_relids_multilevel(root, old_ppi->ppi_req_outer,
- child_rel->relids,
- child_rel->top_parent_relids);
+ child_rel,
+ child_rel->top_parent);
/* If we already have a PPI for this parameterization, just return it */
new_ppi = find_param_path_info(new_path->parent, required_outer);
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 520409f4ba..a163853bed 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -265,14 +265,10 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
*/
if (parent)
{
- /*
- * Each direct or indirect child wants to know the relids of its
- * topmost parent.
- */
- if (parent->top_parent_relids)
- rel->top_parent_relids = parent->top_parent_relids;
- else
- rel->top_parent_relids = bms_copy(parent->relids);
+ /* We keep back-links to immediate parent and topmost parent. */
+ rel->parent = parent;
+ rel->top_parent = parent->top_parent ? parent->top_parent : parent;
+ rel->top_parent_relids = rel->top_parent->relids;
/*
* Also propagate lateral-reference information from appendrel parent
@@ -294,6 +290,8 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
}
else
{
+ rel->parent = NULL;
+ rel->top_parent = NULL;
rel->top_parent_relids = NULL;
rel->direct_lateral_relids = NULL;
rel->lateral_relids = NULL;
@@ -663,6 +661,8 @@ build_join_rel(PlannerInfo *root,
joinrel->joininfo = NIL;
joinrel->has_eclass_joins = false;
joinrel->consider_partitionwise_join = false; /* might get changed later */
+ joinrel->parent = NULL;
+ joinrel->top_parent = NULL;
joinrel->top_parent_relids = NULL;
joinrel->part_scheme = NULL;
joinrel->nparts = -1;
@@ -842,7 +842,9 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->joininfo = NIL;
joinrel->has_eclass_joins = false;
joinrel->consider_partitionwise_join = false; /* might get changed later */
- joinrel->top_parent_relids = NULL;
+ joinrel->parent = parent_joinrel;
+ joinrel->top_parent = parent_joinrel->top_parent ? parent_joinrel->top_parent : parent_joinrel;
+ joinrel->top_parent_relids = joinrel->top_parent->relids;
joinrel->part_scheme = NULL;
joinrel->nparts = -1;
joinrel->boundinfo = NULL;
@@ -854,9 +856,6 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->partexprs = NULL;
joinrel->nullable_partexprs = NULL;
- joinrel->top_parent_relids = bms_union(outer_rel->top_parent_relids,
- inner_rel->top_parent_relids);
-
/* Compute information relevant to foreign relations. */
set_foreign_rel_properties(joinrel, outer_rel, inner_rel);
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 9d3c05aed3..aeabc2aca9 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -529,8 +529,8 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
partprunequal = (List *)
adjust_appendrel_attrs_multilevel(root,
(Node *) prunequal,
- subpart->relids,
- targetpart->relids);
+ subpart,
+ targetpart);
}
/*
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index e2081db4ed..1f06c86323 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -923,7 +923,15 @@ typedef struct RelOptInfo
*/
/* consider partitionwise join paths? (if partitioned rel) */
bool consider_partitionwise_join;
- /* Relids of topmost parents (if "other" rel) */
+
+ /*
+ * inheritance links, if this is an otherrel (otherwise NULL):
+ */
+ /* Immediate parent relation (dumping it would be too verbose) */
+ struct RelOptInfo *parent pg_node_attr(read_write_ignore);
+ /* Topmost parent relation (dumping it would be too verbose) */
+ struct RelOptInfo *top_parent pg_node_attr(read_write_ignore);
+ /* Relids of topmost parent (redundant, but handy) */
Relids top_parent_relids;
/*
diff --git a/src/include/optimizer/appendinfo.h b/src/include/optimizer/appendinfo.h
index fc808dcd27..5e80a741a4 100644
--- a/src/include/optimizer/appendinfo.h
+++ b/src/include/optimizer/appendinfo.h
@@ -23,13 +23,13 @@ extern AppendRelInfo *make_append_rel_info(Relation parentrel,
extern Node *adjust_appendrel_attrs(PlannerInfo *root, Node *node,
int nappinfos, AppendRelInfo **appinfos);
extern Node *adjust_appendrel_attrs_multilevel(PlannerInfo *root, Node *node,
- Relids child_relids,
- Relids top_parent_relids);
+ RelOptInfo *childrel,
+ RelOptInfo *parentrel);
extern Relids adjust_child_relids(Relids relids, int nappinfos,
AppendRelInfo **appinfos);
extern Relids adjust_child_relids_multilevel(PlannerInfo *root, Relids relids,
- Relids child_relids,
- Relids top_parent_relids);
+ RelOptInfo *childrel,
+ RelOptInfo *parentrel);
extern List *adjust_inherited_attnums(List *attnums, AppendRelInfo *context);
extern List *adjust_inherited_attnums_multilevel(PlannerInfo *root,
List *attnums,
commit 4a170a5a77e04fe415e14a3f468a72d24b2d9f9e
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Aug 1 14:40:35 2022 -0400
Add Var.varnullingrels and PlaceHolderVar.phnullingrels fields.
These fields are always empty as of this commit, so they don't
affect any behavior, even though equal() will compare them.
Update backend/nodes/ and backend/rewrite/ infrastructure as needed.
Also add some rewrite functions we'll need later.
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index 28288dcfc1..19606c495f 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -81,11 +81,13 @@ makeVar(int varno,
var->varlevelsup = varlevelsup;
/*
- * Only a few callers need to make Var nodes with varnosyn/varattnosyn
- * different from varno/varattno. We don't provide separate arguments for
- * them, but just initialize them to the given varno/varattno. This
- * reduces code clutter and chance of error for most callers.
+ * Only a few callers need to make Var nodes with non-null varnullingrels,
+ * or with varnosyn/varattnosyn different from varno/varattno. We don't
+ * provide separate arguments for them, but just initialize them to NULL
+ * and the given varno/varattno. This reduces code clutter and chance of
+ * error for most callers.
*/
+ var->varnullingrels = NULL;
var->varnosyn = (Index) varno;
var->varattnosyn = varattno;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 4cb1744da6..ccf63515fa 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -2847,6 +2847,7 @@ expression_tree_mutator(Node *node,
Var *newnode;
FLATCOPY(newnode, var, Var);
+ /* Assume we need not copy the varnullingrels bitmapset */
return (Node *) newnode;
}
break;
@@ -3442,7 +3443,7 @@ expression_tree_mutator(Node *node,
FLATCOPY(newnode, phv, PlaceHolderVar);
MUTATE(newnode->phexpr, phv->phexpr, Expr *);
- /* Assume we need not copy the relids bitmapset */
+ /* Assume we need not copy the relids bitmapsets */
return (Node *) newnode;
}
break;
diff --git a/src/backend/rewrite/rewriteManip.c b/src/backend/rewrite/rewriteManip.c
index 101c39553a..a0a0026469 100644
--- a/src/backend/rewrite/rewriteManip.c
+++ b/src/backend/rewrite/rewriteManip.c
@@ -40,6 +40,13 @@ typedef struct
int win_location;
} locate_windowfunc_context;
+typedef struct
+{
+ Bitmapset *removable_relids;
+ Bitmapset *except_relids;
+ int sublevels_up;
+} remove_nulling_relids_context;
+
static bool contain_aggs_of_level_walker(Node *node,
contain_aggs_of_level_context *context);
static bool locate_agg_of_level_walker(Node *node,
@@ -50,6 +57,9 @@ static bool locate_windowfunc_walker(Node *node,
static bool checkExprHasSubLink_walker(Node *node, void *context);
static Relids offset_relid_set(Relids relids, int offset);
static Relids adjust_relid_set(Relids relids, int oldrelid, int newrelid);
+static bool get_nulling_relids_walker(Node *node, Bitmapset **context);
+static Node *remove_nulling_relids_mutator(Node *node,
+ remove_nulling_relids_context *context);
/*
@@ -348,6 +358,8 @@ OffsetVarNodes_walker(Node *node, OffsetVarNodes_context *context)
if (var->varlevelsup == context->sublevels_up)
{
var->varno += context->offset;
+ var->varnullingrels = offset_relid_set(var->varnullingrels,
+ context->offset);
if (var->varnosyn > 0)
var->varnosyn += context->offset;
}
@@ -386,6 +398,8 @@ OffsetVarNodes_walker(Node *node, OffsetVarNodes_context *context)
{
phv->phrels = offset_relid_set(phv->phrels,
context->offset);
+ phv->phnullingrels = offset_relid_set(phv->phnullingrels,
+ context->offset);
}
/* fall through to examine children */
}
@@ -510,11 +524,13 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context)
{
Var *var = (Var *) node;
- if (var->varlevelsup == context->sublevels_up &&
- var->varno == context->rt_index)
+ if (var->varlevelsup == context->sublevels_up)
{
- var->varno = context->new_index;
- /* If the syntactic referent is same RTE, fix it too */
+ if (var->varno == context->rt_index)
+ var->varno = context->new_index;
+ var->varnullingrels = adjust_relid_set(var->varnullingrels,
+ context->rt_index,
+ context->new_index);
if (var->varnosyn == context->rt_index)
var->varnosyn = context->new_index;
}
@@ -557,6 +573,9 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context)
phv->phrels = adjust_relid_set(phv->phrels,
context->rt_index,
context->new_index);
+ phv->phnullingrels = adjust_relid_set(phv->phnullingrels,
+ context->rt_index,
+ context->new_index);
}
/* fall through to examine children */
}
@@ -833,7 +852,8 @@ rangeTableEntry_used_walker(Node *node,
Var *var = (Var *) node;
if (var->varlevelsup == context->sublevels_up &&
- var->varno == context->rt_index)
+ (var->varno == context->rt_index ||
+ bms_is_member(context->rt_index, var->varnullingrels)))
return true;
return false;
}
@@ -1061,6 +1081,154 @@ AddInvertedQual(Query *parsetree, Node *qual)
}
+/*
+ * get_nulling_relids collects all the level-zero RT indexes mentioned in
+ * Var.varnullingrels and PlaceHolderVar.phnullingrels fields within the
+ * given expression.
+ */
+Bitmapset *
+get_nulling_relids(Node *node)
+{
+ Bitmapset *result = NULL;
+
+ (void) get_nulling_relids_walker(node, &result);
+ return result;
+}
+
+static bool
+get_nulling_relids_walker(Node *node, Bitmapset **context)
+{
+ if (node == NULL)
+ return false;
+ if (IsA(node, Var))
+ {
+ Var *var = (Var *) node;
+
+ if (var->varlevelsup == 0)
+ *context = bms_add_members(*context, var->varnullingrels);
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ if (phv->phlevelsup == 0)
+ *context = bms_add_members(*context, phv->phnullingrels);
+ }
+
+ /*
+ * Currently, this is only used after the planner has converted SubLinks
+ * to SubPlans, so we don't need to support recursion into sub-Queries; so
+ * no sublevels_up counting is needed.
+ */
+ Assert(!IsA(node, SubLink));
+ Assert(!IsA(node, Query));
+ return expression_tree_walker(node, get_nulling_relids_walker, context);
+}
+
+/*
+ * remove_nulling_relids removes mentions of the specified RT index(es)
+ * in Var.varnullingrels and PlaceHolderVar.phnullingrels fields within
+ * the given expression, except in nodes belonging to rels listed in
+ * except_relids.
+ *
+ * XXX consider making this a destructive walker.
+ */
+Node *
+remove_nulling_relids(Node *node, Bitmapset *removable_relids,
+ Bitmapset *except_relids)
+{
+ remove_nulling_relids_context context;
+
+ context.removable_relids = removable_relids;
+ context.except_relids = except_relids;
+ context.sublevels_up = 0;
+ return query_or_expression_tree_mutator(node,
+ remove_nulling_relids_mutator,
+ &context,
+ 0);
+}
+
+static Node *
+remove_nulling_relids_mutator(Node *node,
+ remove_nulling_relids_context *context)
+{
+ if (node == NULL)
+ return NULL;
+ if (IsA(node, Var))
+ {
+ Var *var = (Var *) node;
+
+ if (var->varlevelsup == context->sublevels_up &&
+ !bms_is_member(var->varno, context->except_relids) &&
+ bms_overlap(var->varnullingrels, context->removable_relids))
+ {
+ Relids newnullingrels = bms_difference(var->varnullingrels,
+ context->removable_relids);
+
+ /* Micro-optimization: ensure nullingrels is NULL if empty */
+ if (bms_is_empty(newnullingrels))
+ newnullingrels = NULL;
+ /* Copy the Var ... */
+ var = copyObject(var);
+ /* ... and replace the copy's varnullingrels field */
+ var->varnullingrels = newnullingrels;
+ return (Node *) var;
+ }
+ /* Otherwise fall through to copy the Var normally */
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ if (phv->phlevelsup == context->sublevels_up &&
+ !bms_overlap(phv->phrels, context->except_relids))
+ {
+ Relids newnullingrels = bms_difference(phv->phnullingrels,
+ context->removable_relids);
+
+ /*
+ * Micro-optimization: ensure nullingrels is NULL if empty.
+ *
+ * Note: it might seem desirable to remove the PHV altogether if
+ * phnullingrels goes to empty. Currently we dare not do that
+ * because we use PHVs in some cases to enforce separate identity
+ * of subexpressions; see wrap_non_vars usages in prepjointree.c.
+ */
+ if (bms_is_empty(newnullingrels))
+ newnullingrels = NULL;
+ /* Copy the PlaceHolderVar and mutate what's below ... */
+ phv = (PlaceHolderVar *)
+ expression_tree_mutator(node,
+ remove_nulling_relids_mutator,
+ (void *) context);
+ /* ... and replace the copy's phnullingrels field */
+ phv->phnullingrels = newnullingrels;
+ /* We must also update phrels, if it contains a removable RTI */
+ phv->phrels = bms_difference(phv->phrels,
+ context->removable_relids);
+ Assert(!bms_is_empty(phv->phrels));
+ return (Node *) phv;
+ }
+ /* Otherwise fall through to copy the PlaceHolderVar normally */
+ }
+ else if (IsA(node, Query))
+ {
+ /* Recurse into RTE or sublink subquery */
+ Query *newnode;
+
+ context->sublevels_up++;
+ newnode = query_tree_mutator((Query *) node,
+ remove_nulling_relids_mutator,
+ (void *) context,
+ 0);
+ context->sublevels_up--;
+ return (Node *) newnode;
+ }
+ return expression_tree_mutator(node, remove_nulling_relids_mutator,
+ (void *) context);
+}
+
+
/*
* replace_rte_variables() finds all Vars in an expression tree
* that reference a particular RTE, and replaces them with substitute
diff --git a/src/backend/utils/misc/queryjumble.c b/src/backend/utils/misc/queryjumble.c
index eeaa0b31fe..e517e0363c 100644
--- a/src/backend/utils/misc/queryjumble.c
+++ b/src/backend/utils/misc/queryjumble.c
@@ -381,6 +381,11 @@ JumbleExpr(JumbleState *jstate, Node *node)
APP_JUMB(var->varno);
APP_JUMB(var->varattno);
APP_JUMB(var->varlevelsup);
+
+ /*
+ * We can omit varnullingrels, because it's fully determined
+ * by varno/varlevelsup plus the Var's query location.
+ */
}
break;
case T_Const:
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 1f06c86323..98302ff393 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2583,10 +2583,15 @@ typedef struct MergeScanSelCache
* of a plan tree. This is used during planning to represent the contained
* expression. At the end of the planning process it is replaced by either
* the contained expression or a Var referring to a lower-level evaluation of
- * the contained expression. Typically the evaluation occurs below an outer
+ * the contained expression. Generally the evaluation occurs below an outer
* join, and Var references above the outer join might thereby yield NULL
* instead of the expression value.
*
+ * phrels and phlevelsup correspond to the varno/varlevelsup fields of a
+ * plain Var, except that phrels has to be a relid set since the evaluation
+ * level of a PlaceHolderVar might be a join rather than a base relation.
+ * Likewise, phnullingrels corresponds to varnullingrels.
+ *
* Although the planner treats this as an expression node type, it is not
* recognized by the parser or executor, so we declare it here rather than
* in primnodes.h.
@@ -2599,8 +2604,10 @@ typedef struct MergeScanSelCache
* PHV. Another way in which it can happen is that initplan sublinks
* could get replaced by differently-numbered Params when sublink folding
* is done. (The end result of such a situation would be some
- * unreferenced initplans, which is annoying but not really a problem.) On
- * the same reasoning, there is no need to examine phrels.
+ * unreferenced initplans, which is annoying but not really a problem.) On
+ * the same reasoning, there is no need to examine phrels. But we do need
+ * to compare phnullingrels, as that represents effects that are external
+ * to the original value of the PHV.
*/
typedef struct PlaceHolderVar
@@ -2610,9 +2617,12 @@ typedef struct PlaceHolderVar
/* the represented expression */
Expr *phexpr pg_node_attr(equal_ignore);
- /* base relids syntactically within expr src */
+ /* base+OJ relids syntactically within expr src */
Relids phrels pg_node_attr(equal_ignore);
+ /* RT indexes of outer joins that can null PHV's value */
+ Relids phnullingrels;
+
/* ID for PHV (unique within planner run) */
Index phid;
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index 1fc2fbffa3..f2a0739e6e 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -189,6 +189,14 @@ typedef struct Expr
* row identity information during UPDATE/DELETE. This value should never
* be seen outside the planner.
*
+ * varnullingrels is the set of RT indexes of outer joins that can force
+ * the Var's value to null (at the point where it appears in the query).
+ * See optimizer/README for discussion of that.
+ *
+ * varlevelsup is greater than zero in Vars that represent outer references.
+ * Note that it affects the meaning of all of varno, varnullingrels, and
+ * varnosyn, all of which refer to the range table of that query level.
+ *
* In the parser, varnosyn and varattnosyn are either identical to
* varno/varattno, or they specify the column's position in an aliased JOIN
* RTE that hides the semantic referent RTE's refname. This is a syntactic
@@ -231,6 +239,8 @@ typedef struct Var
int32 vartypmod;
/* OID of collation, or InvalidOid if none */
Oid varcollid;
+ /* RT indexes of outer joins that can replace the Var's value with null */
+ Bitmapset *varnullingrels;
/*
* for subquery variables referencing outer relations; 0 in a normal var,
diff --git a/src/include/rewrite/rewriteManip.h b/src/include/rewrite/rewriteManip.h
index 98b9b3a288..a3f902c1bb 100644
--- a/src/include/rewrite/rewriteManip.h
+++ b/src/include/rewrite/rewriteManip.h
@@ -63,6 +63,10 @@ extern bool contain_windowfuncs(Node *node);
extern int locate_windowfunc(Node *node);
extern bool checkExprHasSubLink(Node *node);
+extern Bitmapset *get_nulling_relids(Node *node);
+extern Node *remove_nulling_relids(Node *node, Bitmapset *removable_relids,
+ Bitmapset *except_relids);
+
extern Node *replace_rte_variables(Node *node,
int target_varno, int sublevels_up,
replace_rte_variables_callback callback,
commit 353bab92af64ce4532bdaf9e1e314e11c29bdcb8
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Aug 1 14:44:47 2022 -0400
Teach the parser to fill Var.varnullingrels correctly.
Vars emitted by the parser are now marked with RT indexes of outer
joins that can null them. (This is done purely according to the
syntax of the query; we don't consider whether an outer join could
be strength-reduced, for example.)
Although the result of this step compiles, it will fail some
regression tests due to the planner not yet knowing what to do.
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 6688c2a865..dff3b1e349 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -670,6 +670,7 @@ transformInsertStmt(ParseState *pstate, InsertStmt *stmt)
*/
sub_pstate->p_rtable = sub_rtable;
sub_pstate->p_joinexprs = NIL; /* sub_rtable has no joins */
+ sub_pstate->p_nullingrels = NIL;
sub_pstate->p_namespace = sub_namespace;
sub_pstate->p_resolve_unknowns = false;
@@ -851,7 +852,7 @@ transformInsertStmt(ParseState *pstate, InsertStmt *stmt)
/*
* Generate list of Vars referencing the RTE
*/
- exprList = expandNSItemVars(nsitem, 0, -1, NULL);
+ exprList = expandNSItemVars(pstate, nsitem, 0, -1, NULL);
/*
* Re-apply any indirection on the target column specs to the Vars
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index 5a18107e79..b2e2b1d6e5 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -52,7 +52,8 @@
#include "utils/syscache.h"
-static int extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
+static int extractRemainingColumns(ParseState *pstate,
+ ParseNamespaceColumn *src_nscolumns,
List *src_colnames,
List **src_colnos,
List **res_colnames, List **res_colvars,
@@ -75,9 +76,11 @@ static ParseNamespaceItem *getNSItemForSpecialRelationTypes(ParseState *pstate,
static Node *transformFromClauseItem(ParseState *pstate, Node *n,
ParseNamespaceItem **top_nsitem,
List **namespace);
-static Var *buildVarFromNSColumn(ParseNamespaceColumn *nscol);
+static Var *buildVarFromNSColumn(ParseState *pstate,
+ ParseNamespaceColumn *nscol);
static Node *buildMergedJoinVar(ParseState *pstate, JoinType jointype,
Var *l_colvar, Var *r_colvar);
+static void markRelsAsNulledBy(ParseState *pstate, Node *n, int jindex);
static void setNamespaceColumnVisibility(List *namespace, bool cols_visible);
static void setNamespaceLateralState(List *namespace,
bool lateral_only, bool lateral_ok);
@@ -249,7 +252,8 @@ setTargetTable(ParseState *pstate, RangeVar *relation,
* Returns the number of columns added.
*/
static int
-extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
+extractRemainingColumns(ParseState *pstate,
+ ParseNamespaceColumn *src_nscolumns,
List *src_colnames,
List **src_colnos,
List **res_colnames, List **res_colvars,
@@ -285,7 +289,8 @@ extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
*src_colnos = lappend_int(*src_colnos, attnum);
*res_colnames = lappend(*res_colnames, lfirst(lc));
*res_colvars = lappend(*res_colvars,
- buildVarFromNSColumn(src_nscolumns + attnum - 1));
+ buildVarFromNSColumn(pstate,
+ src_nscolumns + attnum - 1));
/* Copy the input relation's nscolumn data for this column */
res_nscolumns[colcount] = src_nscolumns[attnum - 1];
colcount++;
@@ -1289,8 +1294,7 @@ transformFromClauseItem(ParseState *pstate, Node *n,
{
/*
* JOIN/USING (or NATURAL JOIN, as transformed above). Transform
- * the list into an explicit ON-condition, and generate a list of
- * merged result columns.
+ * the list into an explicit ON-condition.
*/
List *ucols = j->usingClause;
List *l_usingvars = NIL;
@@ -1308,8 +1312,6 @@ transformFromClauseItem(ParseState *pstate, Node *n,
int r_index = -1;
Var *l_colvar,
*r_colvar;
- Node *u_colvar;
- ParseNamespaceColumn *res_nscolumn;
Assert(u_colname[0] != '\0');
@@ -1373,17 +1375,109 @@ transformFromClauseItem(ParseState *pstate, Node *n,
u_colname)));
r_colnos = lappend_int(r_colnos, r_index + 1);
- l_colvar = buildVarFromNSColumn(l_nscolumns + l_index);
+ /* Build Vars to use in the generated JOIN ON clause */
+ l_colvar = buildVarFromNSColumn(pstate, l_nscolumns + l_index);
l_usingvars = lappend(l_usingvars, l_colvar);
- r_colvar = buildVarFromNSColumn(r_nscolumns + r_index);
+ r_colvar = buildVarFromNSColumn(pstate, r_nscolumns + r_index);
r_usingvars = lappend(r_usingvars, r_colvar);
+ /*
+ * While we're here, add column names to the res_colnames
+ * list. It's a bit ugly to do this here while the
+ * corresponding res_colvars entries are not made till later,
+ * but doing this later would require an additional traversal
+ * of the usingClause list.
+ */
res_colnames = lappend(res_colnames, lfirst(ucol));
+ }
+
+ /* Construct the generated JOIN ON clause */
+ j->quals = transformJoinUsingClause(pstate,
+ l_usingvars,
+ r_usingvars);
+ }
+ else if (j->quals)
+ {
+ /* User-written ON-condition; transform it */
+ j->quals = transformJoinOnClause(pstate, j, my_namespace);
+ }
+ else
+ {
+ /* CROSS JOIN: no quals */
+ }
+
+ /*
+ * If this is an outer join, now mark the appropriate child RTEs as
+ * being nulled by this join. We have finished processing the child
+ * join expressions as well as the current join's quals, which deal in
+ * non-nulled input columns. All future references to those RTEs will
+ * see possibly-nulled values, and we should mark generated Vars to
+ * account for that. In particular, the join alias Vars that we're
+ * about to build should reflect the nulling effects of this join.
+ *
+ * A difficulty with doing this is that we need the join's RT index,
+ * which we don't officially have yet. However, no other RTE can get
+ * made between here and the addRangeTableEntryForJoin call, so we can
+ * predict what the assignment will be. (Alternatively, we could call
+ * addRangeTableEntryForJoin before we have all the data computed, but
+ * this seems less ugly.)
+ */
+ j->rtindex = list_length(pstate->p_rtable) + 1;
+
+ switch (j->jointype)
+ {
+ case JOIN_INNER:
+ break;
+ case JOIN_LEFT:
+ markRelsAsNulledBy(pstate, j->rarg, j->rtindex);
+ break;
+ case JOIN_FULL:
+ markRelsAsNulledBy(pstate, j->larg, j->rtindex);
+ markRelsAsNulledBy(pstate, j->rarg, j->rtindex);
+ break;
+ case JOIN_RIGHT:
+ markRelsAsNulledBy(pstate, j->larg, j->rtindex);
+ break;
+ default:
+ /* shouldn't see any other types here */
+ elog(ERROR, "unrecognized join type: %d",
+ (int) j->jointype);
+ break;
+ }
+
+ /*
+ * Now we can construct join alias expressions for the USING columns.
+ */
+ if (j->usingClause)
+ {
+ ListCell *lc1,
+ *lc2;
+
+ /* Scan the colnos lists to recover info from the previous loop */
+ forboth(lc1, l_colnos, lc2, r_colnos)
+ {
+ int l_index = lfirst_int(lc1) - 1;
+ int r_index = lfirst_int(lc2) - 1;
+ Var *l_colvar,
+ *r_colvar;
+ Node *u_colvar;
+ ParseNamespaceColumn *res_nscolumn;
+
+ /*
+ * Note we re-build these Vars: they might have different
+ * varnullingrels than the ones made in the previous loop.
+ */
+ l_colvar = buildVarFromNSColumn(pstate, l_nscolumns + l_index);
+ r_colvar = buildVarFromNSColumn(pstate, r_nscolumns + r_index);
+
+ /* Construct the join alias Var for this column */
u_colvar = buildMergedJoinVar(pstate,
j->jointype,
l_colvar,
r_colvar);
res_colvars = lappend(res_colvars, u_colvar);
+
+ /* Construct column's res_nscolumns[] entry */
res_nscolumn = res_nscolumns + res_colindex;
res_colindex++;
if (u_colvar == (Node *) l_colvar)
@@ -1401,47 +1495,45 @@ transformFromClauseItem(ParseState *pstate, Node *n,
/*
* Merged column is not semantically equivalent to either
* input, so it needs to be referenced as the join output
- * column. We don't know the join's varno yet, so we'll
- * replace these zeroes below.
+ * column.
*/
- res_nscolumn->p_varno = 0;
+ res_nscolumn->p_varno = j->rtindex;
res_nscolumn->p_varattno = res_colindex;
res_nscolumn->p_vartype = exprType(u_colvar);
res_nscolumn->p_vartypmod = exprTypmod(u_colvar);
res_nscolumn->p_varcollid = exprCollation(u_colvar);
- res_nscolumn->p_varnosyn = 0;
+ res_nscolumn->p_varnosyn = j->rtindex;
res_nscolumn->p_varattnosyn = res_colindex;
}
}
-
- j->quals = transformJoinUsingClause(pstate,
- l_usingvars,
- r_usingvars);
- }
- else if (j->quals)
- {
- /* User-written ON-condition; transform it */
- j->quals = transformJoinOnClause(pstate, j, my_namespace);
- }
- else
- {
- /* CROSS JOIN: no quals */
}
/* Add remaining columns from each side to the output columns */
res_colindex +=
- extractRemainingColumns(l_nscolumns, l_colnames, &l_colnos,
+ extractRemainingColumns(pstate,
+ l_nscolumns, l_colnames, &l_colnos,
&res_colnames, &res_colvars,
res_nscolumns + res_colindex);
res_colindex +=
- extractRemainingColumns(r_nscolumns, r_colnames, &r_colnos,
+ extractRemainingColumns(pstate,
+ r_nscolumns, r_colnames, &r_colnos,
&res_colnames, &res_colvars,
res_nscolumns + res_colindex);
+ /* If join has an alias, it syntactically hides all inputs */
+ if (j->alias)
+ {
+ for (k = 0; k < res_colindex; k++)
+ {
+ ParseNamespaceColumn *nscol = res_nscolumns + k;
+
+ nscol->p_varnosyn = j->rtindex;
+ nscol->p_varattnosyn = k + 1;
+ }
+ }
+
/*
* Now build an RTE and nsitem for the result of the join.
- * res_nscolumns isn't totally done yet, but that's OK because
- * addRangeTableEntryForJoin doesn't examine it, only store a pointer.
*/
nsitem = addRangeTableEntryForJoin(pstate,
res_colnames,
@@ -1455,31 +1547,16 @@ transformFromClauseItem(ParseState *pstate, Node *n,
j->alias,
true);
- j->rtindex = nsitem->p_rtindex;
+ /* Verify that we correctly predicted the join's RT index */
+ Assert(j->rtindex == nsitem->p_rtindex);
+ /* Cross-check number of columns, too */
+ Assert(res_colindex == list_length(nsitem->p_names->colnames));
/*
- * Now that we know the join RTE's rangetable index, we can fix up the
- * res_nscolumns data in places where it should contain that.
+ * Save a link to the JoinExpr in the proper element of p_joinexprs.
+ * Since we maintain that list lazily, it may be necessary to fill in
+ * empty entries before we can add the JoinExpr in the right place.
*/
- Assert(res_colindex == list_length(nsitem->p_names->colnames));
- for (k = 0; k < res_colindex; k++)
- {
- ParseNamespaceColumn *nscol = res_nscolumns + k;
-
- /* fill in join RTI for merged columns */
- if (nscol->p_varno == 0)
- nscol->p_varno = j->rtindex;
- if (nscol->p_varnosyn == 0)
- nscol->p_varnosyn = j->rtindex;
- /* if join has an alias, it syntactically hides all inputs */
- if (j->alias)
- {
- nscol->p_varnosyn = j->rtindex;
- nscol->p_varattnosyn = k + 1;
- }
- }
-
- /* make a matching link to the JoinExpr for later use */
for (k = list_length(pstate->p_joinexprs) + 1; k < j->rtindex; k++)
pstate->p_joinexprs = lappend(pstate->p_joinexprs, NULL);
pstate->p_joinexprs = lappend(pstate->p_joinexprs, j);
@@ -1548,10 +1625,13 @@ transformFromClauseItem(ParseState *pstate, Node *n,
* buildVarFromNSColumn -
* build a Var node using ParseNamespaceColumn data
*
- * We assume varlevelsup should be 0, and no location is specified
+ * This is used to construct joinaliasvars entries.
+ * We can assume varlevelsup should be 0, and no location is specified.
+ * Note also that no column SELECT privilege is requested here; that would
+ * happen only if the column is actually referenced in the query.
*/
static Var *
-buildVarFromNSColumn(ParseNamespaceColumn *nscol)
+buildVarFromNSColumn(ParseState *pstate, ParseNamespaceColumn *nscol)
{
Var *var;
@@ -1565,6 +1645,10 @@ buildVarFromNSColumn(ParseNamespaceColumn *nscol)
/* makeVar doesn't offer parameters for these, so set by hand: */
var->varnosyn = nscol->p_varnosyn;
var->varattnosyn = nscol->p_varattnosyn;
+
+ /* ... and update varnullingrels */
+ markNullableIfNeeded(pstate, var);
+
return var;
}
@@ -1676,6 +1760,47 @@ buildMergedJoinVar(ParseState *pstate, JoinType jointype,
return res_node;
}
+/*
+ * markRelsAsNulledBy -
+ * Mark the given jointree node and its children as nulled by join jindex
+ */
+static void
+markRelsAsNulledBy(ParseState *pstate, Node *n, int jindex)
+{
+ int varno;
+ ListCell *lc;
+
+ /* Note: we can't see FromExpr here */
+ if (IsA(n, RangeTblRef))
+ {
+ varno = ((RangeTblRef *) n)->rtindex;
+ }
+ else if (IsA(n, JoinExpr))
+ {
+ JoinExpr *j = (JoinExpr *) n;
+
+ /* recurse to children */
+ markRelsAsNulledBy(pstate, j->larg, jindex);
+ markRelsAsNulledBy(pstate, j->rarg, jindex);
+ varno = j->rtindex;
+ }
+ else
+ {
+ elog(ERROR, "unrecognized node type: %d", (int) nodeTag(n));
+ varno = 0; /* keep compiler quiet */
+ }
+
+ /*
+ * Now add jindex to the p_nullingrels set for relation varno. Since we
+ * maintain the p_nullingrels list lazily, we might need to extend it to
+ * make the varno'th entry exist.
+ */
+ while (list_length(pstate->p_nullingrels) < varno)
+ pstate->p_nullingrels = lappend(pstate->p_nullingrels, NULL);
+ lc = list_nth_cell(pstate->p_nullingrels, varno - 1);
+ lfirst(lc) = bms_add_member((Bitmapset *) lfirst(lc), jindex);
+}
+
/*
* setNamespaceColumnVisibility -
* Convenience subroutine to update cols_visible flags in a namespace list.
diff --git a/src/backend/parser/parse_coerce.c b/src/backend/parser/parse_coerce.c
index c4e958e4aa..4ded12e873 100644
--- a/src/backend/parser/parse_coerce.c
+++ b/src/backend/parser/parse_coerce.c
@@ -1042,7 +1042,7 @@ coerce_record_to_complex(ParseState *pstate, Node *node,
ParseNamespaceItem *nsitem;
nsitem = GetNSItemByRangeTablePosn(pstate, rtindex, sublevels_up);
- args = expandNSItemVars(nsitem, sublevels_up, vlocation, NULL);
+ args = expandNSItemVars(pstate, nsitem, sublevels_up, vlocation, NULL);
}
else
ereport(ERROR,
diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c
index 059cb7097c..f6cc41c5df 100644
--- a/src/backend/parser/parse_expr.c
+++ b/src/backend/parser/parse_expr.c
@@ -2600,6 +2600,9 @@ transformWholeRowRef(ParseState *pstate, ParseNamespaceItem *nsitem,
/* location is not filled in by makeWholeRowVar */
result->location = location;
+ /* mark Var if it's nulled by any outer joins */
+ markNullableIfNeeded(pstate, result);
+
/* mark relation as requiring whole-row SELECT access */
markVarForSelectPriv(pstate, result);
@@ -2627,6 +2630,8 @@ transformWholeRowRef(ParseState *pstate, ParseNamespaceItem *nsitem,
rowexpr->colnames = copyObject(nsitem->p_names->colnames);
rowexpr->location = location;
+ /* XXX we ought to mark the row as possibly nullable */
+
return (Node *) rowexpr;
}
}
diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c
index f6b740df0a..d956b24f73 100644
--- a/src/backend/parser/parse_relation.c
+++ b/src/backend/parser/parse_relation.c
@@ -751,6 +751,9 @@ scanNSItemForColumn(ParseState *pstate, ParseNamespaceItem *nsitem,
}
var->location = location;
+ /* Mark Var if it's nulled by any outer joins */
+ markNullableIfNeeded(pstate, var);
+
/* Require read access to the column */
markVarForSelectPriv(pstate, var);
@@ -1007,6 +1010,35 @@ searchRangeTableForCol(ParseState *pstate, const char *alias, const char *colnam
return fuzzystate;
}
+/*
+ * markNullableIfNeeded
+ * If the RTE referenced by the Var is nullable by outer join(s)
+ * at this point in the query, set var->varnullingrels to show that.
+ */
+void
+markNullableIfNeeded(ParseState *pstate, Var *var)
+{
+ int rtindex = var->varno;
+ Bitmapset *relids;
+
+ /* Find the appropriate pstate */
+ for (int lv = 0; lv < var->varlevelsup; lv++)
+ pstate = pstate->parentParseState;
+
+ /* Find currently-relevant join relids for the Var's rel */
+ if (rtindex > 0 && rtindex <= list_length(pstate->p_nullingrels))
+ relids = (Bitmapset *) list_nth(pstate->p_nullingrels, rtindex - 1);
+ else
+ relids = NULL;
+
+ /*
+ * Merge with any already-declared nulling rels. (Typically there won't
+ * be any, but let's get it right if there are.)
+ */
+ if (relids != NULL)
+ var->varnullingrels = bms_union(var->varnullingrels, relids);
+}
+
/*
* markRTEForSelectPriv
* Mark the specified column of the RTE with index rtindex
@@ -3110,7 +3142,7 @@ expandTupleDesc(TupleDesc tupdesc, Alias *eref, int count, int offset,
* the list elements mustn't be modified.
*/
List *
-expandNSItemVars(ParseNamespaceItem *nsitem,
+expandNSItemVars(ParseState *pstate, ParseNamespaceItem *nsitem,
int sublevels_up, int location,
List **colnames)
{
@@ -3146,6 +3178,10 @@ expandNSItemVars(ParseNamespaceItem *nsitem,
var->varnosyn = nscol->p_varnosyn;
var->varattnosyn = nscol->p_varattnosyn;
var->location = location;
+
+ /* ... and update varnullingrels */
+ markNullableIfNeeded(pstate, var);
+
result = lappend(result, var);
if (colnames)
*colnames = lappend(*colnames, colnameval);
@@ -3180,7 +3216,7 @@ expandNSItemAttrs(ParseState *pstate, ParseNamespaceItem *nsitem,
*var;
List *te_list = NIL;
- vars = expandNSItemVars(nsitem, sublevels_up, location, &names);
+ vars = expandNSItemVars(pstate, nsitem, sublevels_up, location, &names);
/*
* Require read access to the table. This is normally redundant with the
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index 16a0fe59e2..f81548441a 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -1379,7 +1379,7 @@ ExpandSingleTable(ParseState *pstate, ParseNamespaceItem *nsitem,
List *vars;
ListCell *l;
- vars = expandNSItemVars(nsitem, sublevels_up, location, NULL);
+ vars = expandNSItemVars(pstate, nsitem, sublevels_up, location, NULL);
/*
* Require read access to the table. This is normally redundant with
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 98fe1abaa2..43d987640e 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1078,6 +1078,14 @@ typedef struct RangeTblEntry
* alias Vars are generated only for merged columns). We keep these
* entries only because they're needed in expandRTE() and similar code.
*
+ * Vars appearing within joinaliasvars are marked with varnullingrels sets
+ * that describe the nulling effects of this join and lower ones. This is
+ * essential for FULL JOIN cases, because the COALESCE expression only
+ * describes the semantics correctly if its inputs have been nulled by the
+ * join. For other cases, it allows expandRTE() to generate a valid
+ * representation of the join's output without consulting additional
+ * parser state.
+ *
* Within a Query loaded from a stored rule, it is possible for non-merged
* joinaliasvars items to be null pointers, which are placeholders for
* (necessarily unreferenced) columns dropped since the rule was made.
diff --git a/src/include/parser/parse_node.h b/src/include/parser/parse_node.h
index 962ebf65de..636d3231cd 100644
--- a/src/include/parser/parse_node.h
+++ b/src/include/parser/parse_node.h
@@ -115,6 +115,13 @@ typedef Node *(*CoerceParamHook) (ParseState *pstate, Param *param,
* This is one-for-one with p_rtable, but contains NULLs for non-join
* RTEs, and may be shorter than p_rtable if the last RTE(s) aren't joins.
*
+ * p_nullingrels: list of Bitmapsets associated with p_rtable entries, each
+ * containing the set of outer-join RTE indexes that can null that relation
+ * at the current point in the parse tree. This is one-for-one with p_rtable,
+ * but may be shorter than p_rtable, in which case the missing entries are
+ * implicitly empty (NULL). That rule allows us to save work when the query
+ * contains no outer joins.
+ *
* p_joinlist: list of join items (RangeTblRef and JoinExpr nodes) that
* will become the fromlist of the query's top-level FromExpr node.
*
@@ -182,6 +189,7 @@ struct ParseState
const char *p_sourcetext; /* source text, or NULL if not available */
List *p_rtable; /* range table so far */
List *p_joinexprs; /* JoinExprs for RTE_JOIN p_rtable entries */
+ List *p_nullingrels; /* Bitmapsets showing nulling outer joins */
List *p_joinlist; /* join items so far (will become FromExpr
* node's fromlist) */
List *p_namespace; /* currently-referenceable RTEs (List of
diff --git a/src/include/parser/parse_relation.h b/src/include/parser/parse_relation.h
index de21c3c649..85d96563f3 100644
--- a/src/include/parser/parse_relation.h
+++ b/src/include/parser/parse_relation.h
@@ -41,6 +41,7 @@ extern Node *scanNSItemForColumn(ParseState *pstate, ParseNamespaceItem *nsitem,
int location);
extern Node *colNameToVar(ParseState *pstate, const char *colname, bool localonly,
int location);
+extern void markNullableIfNeeded(ParseState *pstate, Var *var);
extern void markVarForSelectPriv(ParseState *pstate, Var *var);
extern Relation parserOpenTable(ParseState *pstate, const RangeVar *relation,
int lockmode);
@@ -109,7 +110,7 @@ extern void errorMissingColumn(ParseState *pstate,
extern void expandRTE(RangeTblEntry *rte, int rtindex, int sublevels_up,
int location, bool include_dropped,
List **colnames, List **colvars);
-extern List *expandNSItemVars(ParseNamespaceItem *nsitem,
+extern List *expandNSItemVars(ParseState *pstate, ParseNamespaceItem *nsitem,
int sublevels_up, int location,
List **colnames);
extern List *expandNSItemAttrs(ParseState *pstate, ParseNamespaceItem *nsitem,
commit fa3d148ee448f05d82524074ab4175c2a15b46d7
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Aug 1 15:25:14 2022 -0400
Teach the planner to cope with Vars bearing nullingrels.
The core idea of this step is to include varnullingrels in the
relid sets that qual clauses are considered to depend on.
So that we can still easily compare quals' relids to RelOptInfos'
relids, that means also adding outer join relids to the identifying
relids of join relations. Much of the bulk of this step is concerned
with fallout from the latter change.
Also, in setrefs.c and some other places, we have to intentionally
ignore varnullingrels when comparing the outputs of a lower plan
node to the Vars required by upper expressions. I'd like to tighten
that up, by accounting for whether a given plan node implements
an outer join and expecting the upper Vars to have that OJ relid
added to their varnullingrels if so. But because of the hackery
involved in implementing outer join identity 3, there are some cases
where the upper Var legitimately won't have that bit set, and it's
unclear how to make a check that doesn't reject such plans. So that
issue is left for later. It would only be a bug-detection aid
anyway, since all the interesting decisions have been made already.
This step removes some low-hanging fruit from the old implementation,
such as the need to track lowest_nulling_outer_join during subquery
pullup. There's much more to do in that line, though.
The result of this step passes most core regression tests, but there
are still some failure cases involving full joins.
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index 004481d608..1c921879a9 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -273,7 +273,7 @@ merge_clump(PlannerInfo *root, List *clumps, Clump *new_clump, int num_gene,
* rel once we know the final targetlist (see
* grouping_planner).
*/
- if (!bms_equal(joinrel->relids, root->all_baserels))
+ if (!bms_equal(joinrel->relids, root->all_query_rels))
generate_useful_gather_paths(root, joinrel, false);
/* Find and save the cheapest paths for this joinrel */
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index b23cc8ee82..e14d3b136e 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -180,6 +180,9 @@ make_one_rel(PlannerInfo *root, List *joinlist)
root->all_baserels = bms_add_member(root->all_baserels, brel->relid);
}
+ /* Now we can form the value of all_query_rels, too */
+ root->all_query_rels = bms_union(root->all_baserels, root->outer_join_rels);
+
/* Mark base rels as to whether we care about fast-start plans */
set_base_rel_consider_startup(root);
@@ -231,9 +234,9 @@ make_one_rel(PlannerInfo *root, List *joinlist)
rel = make_rel_from_joinlist(root, joinlist);
/*
- * The result should join all and only the query's base rels.
+ * The result should join all and only the query's base + outer-join rels.
*/
- Assert(bms_equal(rel->relids, root->all_baserels));
+ Assert(bms_equal(rel->relids, root->all_query_rels));
return rel;
}
@@ -558,7 +561,7 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* the final scan/join targetlist is available (see grouping_planner).
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- !bms_equal(rel->relids, root->all_baserels))
+ !bms_equal(rel->relids, root->all_query_rels))
generate_useful_gather_paths(root, rel, false);
/* Now find the cheapest of the paths for this rel */
@@ -879,7 +882,7 @@ set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *
* to support an uncommon usage of second-rate sampling methods. Instead,
* if there is a risk that the query might perform an unsafe join, just
* wrap the SampleScan in a Materialize node. We can check for joins by
- * counting the membership of all_baserels (note that this correctly
+ * counting the membership of all_query_rels (note that this correctly
* counts inheritance trees as single rels). If we're inside a subquery,
* we can't easily check whether a join might occur in the outer query, so
* just assume one is possible.
@@ -888,7 +891,7 @@ set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *
* so check repeatable_across_scans last, even though that's a bit odd.
*/
if ((root->query_level > 1 ||
- bms_membership(root->all_baserels) != BMS_SINGLETON) &&
+ bms_membership(root->all_query_rels) != BMS_SINGLETON) &&
!(GetTsmRoutine(rte->tablesample->tsmhandler)->repeatable_across_scans))
{
path = (Path *) create_material_path(rel, path);
@@ -3434,7 +3437,7 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
* partial paths. We'll do the same for the topmost scan/join rel
* once we know the final targetlist (see grouping_planner).
*/
- if (!bms_equal(rel->relids, root->all_baserels))
+ if (!bms_equal(rel->relids, root->all_query_rels))
generate_useful_gather_paths(root, rel, false);
/* Find and save the cheapest paths for this rel */
diff --git a/src/backend/optimizer/path/clausesel.c b/src/backend/optimizer/path/clausesel.c
index 06f836308d..c08eb2b1c5 100644
--- a/src/backend/optimizer/path/clausesel.c
+++ b/src/backend/optimizer/path/clausesel.c
@@ -218,7 +218,7 @@ clauselist_selectivity_ext(PlannerInfo *root,
if (rinfo)
{
- ok = (bms_membership(rinfo->clause_relids) == BMS_SINGLETON) &&
+ ok = (rinfo->num_base_rels == 1) &&
(is_pseudo_constant_clause_relids(lsecond(expr->args),
rinfo->right_relids) ||
(varonleft = false,
@@ -579,30 +579,6 @@ find_single_rel_for_clauses(PlannerInfo *root, List *clauses)
return NULL; /* no clauses */
}
-/*
- * bms_is_subset_singleton
- *
- * Same result as bms_is_subset(s, bms_make_singleton(x)),
- * but a little faster and doesn't leak memory.
- *
- * Is this of use anywhere else? If so move to bitmapset.c ...
- */
-static bool
-bms_is_subset_singleton(const Bitmapset *s, int x)
-{
- switch (bms_membership(s))
- {
- case BMS_EMPTY_SET:
- return true;
- case BMS_SINGLETON:
- return bms_is_member(x, s);
- case BMS_MULTIPLE:
- return false;
- }
- /* can't get here... */
- return false;
-}
-
/*
* treat_as_join_clause -
* Decide whether an operator clause is to be handled by the
@@ -631,17 +607,20 @@ treat_as_join_clause(PlannerInfo *root, Node *clause, RestrictInfo *rinfo,
else
{
/*
- * Otherwise, it's a join if there's more than one relation used. We
- * can optimize this calculation if an rinfo was passed.
+ * Otherwise, it's a join if there's more than one base relation used.
+ * We can optimize this calculation if an rinfo was passed.
*
* XXX Since we know the clause is being evaluated at a join, the
* only way it could be single-relation is if it was delayed by outer
- * joins. Although we can make use of the restriction qual estimators
- * anyway, it seems likely that we ought to account for the
- * probability of injected nulls somehow.
+ * joins. We intentionally count only baserels here, not OJs that
+ * might be present in rinfo->clause_relids, so that we direct such
+ * cases to the restriction qual estimators not join estimators.
+ * Eventually some notice should be taken of the possibility of
+ * injected nulls, but we'll likely want to do that in the restriction
+ * estimators rather than starting to treat such cases as join quals.
*/
if (rinfo)
- return (bms_membership(rinfo->clause_relids) == BMS_MULTIPLE);
+ return (rinfo->num_base_rels > 1);
else
return (NumRelids(root, clause) > 1);
}
@@ -754,7 +733,9 @@ clause_selectivity_ext(PlannerInfo *root,
* for all non-JOIN_INNER cases.
*/
if (varRelid == 0 ||
- bms_is_subset_singleton(rinfo->clause_relids, varRelid))
+ rinfo->num_base_rels == 0 ||
+ (rinfo->num_base_rels == 1 &&
+ bms_is_member(varRelid, rinfo->clause_relids)))
{
/* Cacheable --- do we already have the result? */
if (jointype == JOIN_INNER)
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index fb28e6411a..24cf498de7 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -5101,7 +5101,9 @@ compute_semi_anti_join_factors(PlannerInfo *root,
norm_sjinfo.syn_lefthand = outerrel->relids;
norm_sjinfo.syn_righthand = innerrel->relids;
norm_sjinfo.jointype = JOIN_INNER;
+ norm_sjinfo.ojrelid = 0;
/* we don't bother trying to make the remaining fields valid */
+ norm_sjinfo.strict_relids = NULL;
norm_sjinfo.lhs_strict = false;
norm_sjinfo.delay_upper_joins = false;
norm_sjinfo.semi_can_btree = false;
@@ -5266,7 +5268,9 @@ approx_tuple_count(PlannerInfo *root, JoinPath *path, List *quals)
sjinfo.syn_lefthand = path->outerjoinpath->parent->relids;
sjinfo.syn_righthand = path->innerjoinpath->parent->relids;
sjinfo.jointype = JOIN_INNER;
+ sjinfo.ojrelid = 0;
/* we don't bother trying to make the remaining fields valid */
+ sjinfo.strict_relids = NULL;
sjinfo.lhs_strict = false;
sjinfo.delay_upper_joins = false;
sjinfo.semi_can_btree = false;
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index f8a97622b1..d31ca5e527 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -29,6 +29,7 @@
#include "optimizer/paths.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "rewrite/rewriteManip.h"
#include "utils/lsyscache.h"
@@ -64,7 +65,7 @@ static bool reconsider_outer_join_clause(PlannerInfo *root,
RestrictInfo *rinfo,
bool outer_on_left);
static bool reconsider_full_join_clause(PlannerInfo *root,
- RestrictInfo *rinfo);
+ FullJoinClauseInfo *fjinfo);
static Bitmapset *get_eclass_indexes_for_relids(PlannerInfo *root,
Relids relids);
static Bitmapset *get_common_eclass_indexes(PlannerInfo *root, Relids relids1,
@@ -768,6 +769,9 @@ get_eclass_for_sort_expr(PlannerInfo *root,
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL)
+ continue; /* must be an outer join */
+
Assert(rel->reloptkind == RELOPT_BASEREL ||
rel->reloptkind == RELOPT_DEADREL);
@@ -936,7 +940,36 @@ is_exprlist_member(Expr *node, List *exprs)
if (expr && IsA(expr, TargetEntry))
expr = ((TargetEntry *) expr)->expr;
- if (equal(node, expr))
+ /*
+ * For Vars and PlaceHolderVars, match using the same rules as
+ * setrefs.c will, in particular ignoring nullingrels. XXX when that
+ * gets tightened up, this should too.
+ */
+ if (IsA(node, Var))
+ {
+ if (expr && IsA(expr, Var))
+ {
+ Var *v1 = (Var *) node;
+ Var *v2 = (Var *) expr;
+
+ if (v1->varno == v2->varno &&
+ v1->varattno == v2->varattno &&
+ v1->varlevelsup == v2->varlevelsup)
+ return true;
+ }
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ if (expr && IsA(expr, PlaceHolderVar))
+ {
+ PlaceHolderVar *v1 = (PlaceHolderVar *) node;
+ PlaceHolderVar *v2 = (PlaceHolderVar *) expr;
+
+ if (v1->phid == v2->phid)
+ return true;
+ }
+ }
+ else if (equal(node, expr))
return true;
}
return false;
@@ -1124,6 +1157,9 @@ generate_base_implied_equalities(PlannerInfo *root)
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL)
+ continue; /* must be an outer join */
+
Assert(rel->reloptkind == RELOPT_BASEREL);
rel->eclass_indexes = bms_add_member(rel->eclass_indexes,
@@ -2014,10 +2050,12 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* Process the FULL JOIN clauses */
foreach(cell, root->full_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ FullJoinClauseInfo *fjinfo = (FullJoinClauseInfo *) lfirst(cell);
- if (reconsider_full_join_clause(root, rinfo))
+ if (reconsider_full_join_clause(root, fjinfo))
{
+ RestrictInfo *rinfo = fjinfo->rinfo;
+
found = true;
/* remove it from the list */
root->full_join_clauses =
@@ -2046,9 +2084,9 @@ reconsider_outer_join_clauses(PlannerInfo *root)
}
foreach(cell, root->full_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ FullJoinClauseInfo *fjinfo = (FullJoinClauseInfo *) lfirst(cell);
- distribute_restrictinfo_to_rels(root, rinfo);
+ distribute_restrictinfo_to_rels(root, fjinfo->rinfo);
}
}
@@ -2184,8 +2222,11 @@ reconsider_outer_join_clause(PlannerInfo *root, RestrictInfo *rinfo,
* Returns true if we were able to propagate a constant through the clause.
*/
static bool
-reconsider_full_join_clause(PlannerInfo *root, RestrictInfo *rinfo)
+reconsider_full_join_clause(PlannerInfo *root, FullJoinClauseInfo *fjinfo)
{
+ RestrictInfo *rinfo = fjinfo->rinfo;
+ SpecialJoinInfo *sjinfo = fjinfo->sjinfo;
+ Relids fjrelids = bms_make_singleton(sjinfo->ojrelid);
Expr *leftvar;
Expr *rightvar;
Oid opno,
@@ -2267,6 +2308,18 @@ reconsider_full_join_clause(PlannerInfo *root, RestrictInfo *rinfo)
cfirst = (Node *) linitial(cexpr->args);
csecond = (Node *) lsecond(cexpr->args);
+ /*
+ * The COALESCE arguments will be marked as possibly nulled by
+ * the full join, while we wish to generate clauses that apply
+ * to the join's inputs. So we must strip the join from the
+ * nullingrels fields of cfirst/csecond before comparing them
+ * to leftvar/rightvar. (Perhaps with a less hokey
+ * representation for FULL JOIN USING output columns, this
+ * wouldn't be needed?)
+ */
+ cfirst = remove_nulling_relids(cfirst, fjrelids, NULL);
+ csecond = remove_nulling_relids(csecond, fjrelids, NULL);
+
if (equal(leftvar, cfirst) && equal(rightvar, csecond))
{
coal_idx = foreach_current_index(lc2);
@@ -3203,6 +3256,8 @@ get_eclass_indexes_for_relids(PlannerInfo *root, Relids relids)
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL)
+ continue; /* must be an outer join */
ec_indexes = bms_add_members(ec_indexes, rel->eclass_indexes);
}
return ec_indexes;
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index 7d176e7b00..78d817bf1b 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -3355,13 +3355,13 @@ check_index_predicates(PlannerInfo *root, RelOptInfo *rel)
* Add on any equivalence-derivable join clauses. Computing the correct
* relid sets for generate_join_implied_equalities is slightly tricky
* because the rel could be a child rel rather than a true baserel, and in
- * that case we must remove its parents' relid(s) from all_baserels.
+ * that case we must subtract its parents' relid(s) from all_query_rels.
*/
if (rel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- otherrels = bms_difference(root->all_baserels,
+ otherrels = bms_difference(root->all_query_rels,
find_childrel_parents(root, rel));
else
- otherrels = bms_difference(root->all_baserels, rel->relids);
+ otherrels = bms_difference(root->all_query_rels, rel->relids);
if (!bms_is_empty(otherrels))
clauselist =
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 2a3f0ab7bf..a1fc72c394 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -250,7 +250,7 @@ add_paths_to_joinrel(PlannerInfo *root,
if (bms_overlap(joinrelids, sjinfo2->min_righthand) &&
!bms_overlap(joinrelids, sjinfo2->min_lefthand))
extra.param_source_rels = bms_join(extra.param_source_rels,
- bms_difference(root->all_baserels,
+ bms_difference(root->all_query_rels,
sjinfo2->min_righthand));
/* full joins constrain both sides symmetrically */
@@ -258,7 +258,7 @@ add_paths_to_joinrel(PlannerInfo *root,
bms_overlap(joinrelids, sjinfo2->min_lefthand) &&
!bms_overlap(joinrelids, sjinfo2->min_righthand))
extra.param_source_rels = bms_join(extra.param_source_rels,
- bms_difference(root->all_baserels,
+ bms_difference(root->all_query_rels,
sjinfo2->min_lefthand));
}
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 9da3ff2f9a..b64c37f089 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -353,7 +353,10 @@ make_rels_by_clauseless_joins(PlannerInfo *root,
*
* Caller must supply not only the two rels, but the union of their relids.
* (We could simplify the API by computing joinrelids locally, but this
- * would be redundant work in the normal path through make_join_rel.)
+ * would be redundant work in the normal path through make_join_rel.
+ * Note that this value does NOT include the RT index of any outer join that
+ * might need to be performed here, so it's not the canonical identifier
+ * of the join relation.)
*
* On success, *sjinfo_p is set to NULL if this is to be a plain inner join,
* else it's set to point to the associated SpecialJoinInfo node. Also,
@@ -695,7 +698,7 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(rel1->relids, rel2->relids));
- /* Construct Relids set that identifies the joinrel. */
+ /* Construct Relids set that identifies the joinrel (without OJ as yet). */
joinrelids = bms_union(rel1->relids, rel2->relids);
/* Check validity and determine join type. */
@@ -707,6 +710,10 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
return NULL;
}
+ /* If we have an outer join, add its RTI to form the canonical relids. */
+ if (sjinfo && sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
+
/* Swap rels if needed to match the join info. */
if (reversed)
{
@@ -730,7 +737,9 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
sjinfo->syn_lefthand = rel1->relids;
sjinfo->syn_righthand = rel2->relids;
sjinfo->jointype = JOIN_INNER;
+ sjinfo->ojrelid = 0;
/* we don't bother trying to make the remaining fields valid */
+ sjinfo->strict_relids = NULL;
sjinfo->lhs_strict = false;
sjinfo->delay_upper_joins = false;
sjinfo->semi_can_btree = false;
@@ -1510,8 +1519,6 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(child_rel1->relids, child_rel2->relids));
- child_joinrelids = bms_union(child_rel1->relids, child_rel2->relids);
- appinfos = find_appinfos_by_relids(root, child_joinrelids, &nappinfos);
/*
* Construct SpecialJoinInfo from parent join relations's
@@ -1521,6 +1528,15 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
child_rel1->relids,
child_rel2->relids);
+ /* Build correct join relids for child join */
+ child_joinrelids = bms_union(child_rel1->relids, child_rel2->relids);
+ if (child_sjinfo->ojrelid != 0)
+ child_joinrelids = bms_add_member(child_joinrelids,
+ child_sjinfo->ojrelid);
+
+ /* Find the AppendRelInfo structures */
+ appinfos = find_appinfos_by_relids(root, child_joinrelids, &nappinfos);
+
/*
* Construct restrictions applicable to the child join from those
* applicable to the parent join.
@@ -1536,8 +1552,7 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
{
child_joinrel = build_child_join_rel(root, child_rel1, child_rel2,
joinrel, child_restrictlist,
- child_sjinfo,
- child_sjinfo->jointype);
+ child_sjinfo);
joinrel->part_rels[cnt_parts] = child_joinrel;
joinrel->live_parts = bms_add_member(joinrel->live_parts, cnt_parts);
joinrel->all_partrels = bms_add_members(joinrel->all_partrels,
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index 337f470d58..bbe31c03fe 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -34,7 +34,7 @@
/* local functions */
static bool join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo);
-static void remove_rel_from_query(PlannerInfo *root, int relid,
+static void remove_rel_from_query(PlannerInfo *root, int relid, int ojrelid,
Relids joinrelids);
static List *remove_rel_from_joinlist(List *joinlist, int relid, int *nremoved);
static bool rel_supports_distinctness(PlannerInfo *root, RelOptInfo *rel);
@@ -70,6 +70,7 @@ restart:
foreach(lc, root->join_info_list)
{
SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+ Relids joinrelids;
int innerrelid;
int nremoved;
@@ -84,9 +85,12 @@ restart:
*/
innerrelid = bms_singleton_member(sjinfo->min_righthand);
- remove_rel_from_query(root, innerrelid,
- bms_union(sjinfo->min_lefthand,
- sjinfo->min_righthand));
+ /* Compute the relid set for the join we are considering */
+ joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ if (sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
+
+ remove_rel_from_query(root, innerrelid, sjinfo->ojrelid, joinrelids);
/* We verify that exactly one reference gets removed from joinlist */
nremoved = 0;
@@ -188,6 +192,8 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
/* Compute the relid set for the join we are considering */
joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ if (sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
/*
* We can't remove the join if any inner-rel attributes are used above the
@@ -306,10 +312,12 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
* no longer treated as a baserel, and that attributes of other baserels
* are no longer marked as being needed at joins involving this rel.
* Also, join quals involving the rel have to be removed from the joininfo
- * lists, but only if they belong to the outer join identified by joinrelids.
+ * lists, but only if they belong to the outer join identified by ojrelid
+ * and joinrelids.
*/
static void
-remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
+remove_rel_from_query(PlannerInfo *root, int relid, int ojrelid,
+ Relids joinrelids)
{
RelOptInfo *rel = find_base_rel(root, relid);
List *joininfos;
@@ -349,6 +357,13 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
}
}
+ /*
+ * The removed outer join has to be dropped from root->outer_join_rels.
+ * (We'd need to update all_baserels and all_query_rels too, but those
+ * haven't been computed yet.)
+ */
+ root->outer_join_rels = bms_del_member(root->outer_join_rels, ojrelid);
+
/*
* Likewise remove references from SpecialJoinInfo data structures.
*
@@ -365,6 +380,10 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
sjinfo->min_righthand = bms_del_member(sjinfo->min_righthand, relid);
sjinfo->syn_lefthand = bms_del_member(sjinfo->syn_lefthand, relid);
sjinfo->syn_righthand = bms_del_member(sjinfo->syn_righthand, relid);
+ sjinfo->min_lefthand = bms_del_member(sjinfo->min_lefthand, ojrelid);
+ sjinfo->min_righthand = bms_del_member(sjinfo->min_righthand, ojrelid);
+ sjinfo->syn_lefthand = bms_del_member(sjinfo->syn_lefthand, ojrelid);
+ sjinfo->syn_righthand = bms_del_member(sjinfo->syn_righthand, ojrelid);
}
/*
@@ -393,8 +412,10 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
else
{
phinfo->ph_eval_at = bms_del_member(phinfo->ph_eval_at, relid);
+ phinfo->ph_eval_at = bms_del_member(phinfo->ph_eval_at, ojrelid);
Assert(!bms_is_empty(phinfo->ph_eval_at));
phinfo->ph_needed = bms_del_member(phinfo->ph_needed, relid);
+ phinfo->ph_needed = bms_del_member(phinfo->ph_needed, ojrelid);
}
}
@@ -431,6 +452,8 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
rinfo->required_relids = bms_copy(rinfo->required_relids);
rinfo->required_relids = bms_del_member(rinfo->required_relids,
relid);
+ rinfo->required_relids = bms_del_member(rinfo->required_relids,
+ ojrelid);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
@@ -545,6 +568,7 @@ reduce_unique_semijoins(PlannerInfo *root)
/* Compute the relid set for the join we are considering */
joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ Assert(sjinfo->ojrelid == 0); /* SEMI joins don't have RT indexes */
/*
* Since we're only considering a single-rel RHS, any join clauses it
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 023efbaf09..b438085af6 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -60,12 +60,15 @@ static void process_security_barrier_quals(PlannerInfo *root,
static SpecialJoinInfo *make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
- JoinType jointype, List *clause);
+ JoinType jointype, Index ojrelid,
+ List *clause);
static void compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo,
List *clause);
+static List *remove_unneeded_nulling_relids(PlannerInfo *root, List *quals,
+ SpecialJoinInfo *sjinfo);
static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool below_outer_join,
- JoinType jointype,
+ SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
Relids ojscope,
@@ -250,10 +253,16 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
attno -= rel->min_attr;
if (rel->attr_needed[attno] == NULL)
{
- /* Variable not yet requested, so add to rel's targetlist */
- /* XXX is copyObject necessary here? */
- rel->reltarget->exprs = lappend(rel->reltarget->exprs,
- copyObject(var));
+ /*
+ * Variable not yet requested, so add to rel's targetlist.
+ *
+ * The value available at the rel's scan level has not been
+ * nulled by any outer join, so drop its varnullingrels.
+ * (We'll put those back as we climb up the join tree.)
+ */
+ var = copyObject(var);
+ var->varnullingrels = NULL;
+ rel->reltarget->exprs = lappend(rel->reltarget->exprs, var);
/* reltarget cost and width will be computed later */
}
rel->attr_needed[attno] = bms_add_members(rel->attr_needed[attno],
@@ -551,8 +560,10 @@ create_lateral_join_info(PlannerInfo *root)
varno = -1;
while ((varno = bms_next_member(eval_at, varno)) >= 0)
{
- RelOptInfo *brel = find_base_rel(root, varno);
+ RelOptInfo *brel = find_base_rel_ignore_join(root, varno);
+ if (brel == NULL)
+ continue; /* ignore outer joins in eval_at */
brel->lateral_relids = bms_add_members(brel->lateral_relids,
phinfo->ph_lateral);
}
@@ -643,7 +654,10 @@ create_lateral_join_info(PlannerInfo *root)
{
RelOptInfo *brel2 = root->simple_rel_array[rti2];
- Assert(brel2 != NULL && brel2->reloptkind == RELOPT_BASEREL);
+ if (brel2 == NULL)
+ continue; /* must be an OJ */
+
+ Assert(brel2->reloptkind == RELOPT_BASEREL);
brel2->lateral_referencers =
bms_add_member(brel2->lateral_referencers, rti);
}
@@ -695,7 +709,8 @@ deconstruct_jointree(PlannerInfo *root)
Assert(root->parse->jointree != NULL &&
IsA(root->parse->jointree, FromExpr));
- /* this is filled as we scan the jointree */
+ /* These are filled as we scan the jointree */
+ root->outer_join_rels = NULL;
root->nullable_baserels = NULL;
result = deconstruct_recurse(root, (Node *) root->parse->jointree, false,
@@ -717,7 +732,7 @@ deconstruct_jointree(PlannerInfo *root)
* below_outer_join is true if this node is within the nullable side of a
* higher-level outer join
* Outputs:
- * *qualscope gets the set of base Relids syntactically included in this
+ * *qualscope gets the set of base+OJ Relids syntactically included in this
* jointree node (do not modify or free this, as it may also be pointed
* to by RestrictInfo and SpecialJoinInfo nodes)
* *inner_join_rels gets the set of base Relids syntactically included in
@@ -802,6 +817,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
* there was exactly one element, we should (and already did) report
* whatever its inner_join_rels were. If there were no elements (is
* that still possible?) the initialization before the loop fixed it.
+ *
+ * XXX now wrong, do we care?
*/
if (list_length(f->fromlist) > 1)
*inner_join_rels = *qualscope;
@@ -816,7 +833,7 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
if (bms_is_subset(pq->relids, *qualscope))
distribute_qual_to_rels(root, pq->qual,
- below_outer_join, JOIN_INNER,
+ below_outer_join, NULL,
root->qual_security_level,
*qualscope, NULL, NULL,
NULL);
@@ -832,7 +849,7 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
Node *qual = (Node *) lfirst(l);
distribute_qual_to_rels(root, qual,
- below_outer_join, JOIN_INNER,
+ below_outer_join, NULL,
root->qual_security_level,
*qualscope, NULL, NULL,
postponed_qual_list);
@@ -896,6 +913,13 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ /* caution: ANTI join derived from SEMI will lack rtindex */
+ if (j->rtindex != 0)
+ {
+ *qualscope = bms_add_member(*qualscope, j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
+ j->rtindex);
+ }
*inner_join_rels = bms_union(left_inners, right_inners);
nonnullable_rels = leftids;
nullable_rels = rightids;
@@ -910,6 +934,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ /* SEMI join never has rtindex, so don't add to qualscope */
+ Assert(j->rtindex == 0);
*inner_join_rels = bms_union(left_inners, right_inners);
/* Semi join adds no restrictions for quals */
nonnullable_rels = NULL;
@@ -931,6 +957,10 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ Assert(j->rtindex != 0);
+ *qualscope = bms_add_member(*qualscope, j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
+ j->rtindex);
*inner_join_rels = bms_union(left_inners, right_inners);
/* each side is both outer and inner */
nonnullable_rels = *qualscope;
@@ -976,32 +1006,44 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
my_quals = list_concat(my_quals, (List *) j->quals);
/*
- * For an OJ, form the SpecialJoinInfo now, because we need the OJ's
- * semantic scope (ojscope) to pass to distribute_qual_to_rels. But
- * we mustn't add it to join_info_list just yet, because we don't want
- * distribute_qual_to_rels to think it is an outer join below us.
- *
- * Semijoins are a bit of a hybrid: we build a SpecialJoinInfo, but we
- * want ojscope = NULL for distribute_qual_to_rels.
+ * For an OJ, form the SpecialJoinInfo now, because we need it for
+ * distribute_qual_to_rels. But we mustn't add it to join_info_list
+ * just yet, because we don't want distribute_qual_to_rels to think it
+ * is an outer join below us.
*/
if (j->jointype != JOIN_INNER)
- {
sjinfo = make_outerjoininfo(root,
leftids, rightids,
*inner_join_rels,
j->jointype,
+ j->rtindex,
my_quals);
- if (j->jointype == JOIN_SEMI)
- ojscope = NULL;
- else
- ojscope = bms_union(sjinfo->min_lefthand,
- sjinfo->min_righthand);
- }
else
- {
sjinfo = NULL;
+
+ /*
+ * If we have a LEFT JOIN whose ON qual is strict for any LHS
+ * relations, we may be able to commute the join with lower outer
+ * joins that null those relations. To do that, we must remove such
+ * lower outer joins from Var.varnullingrels fields within the qual,
+ * else subsequent processing will think that the qual has to be
+ * evaluated above such lower outer joins.
+ */
+ if (j->jointype == JOIN_LEFT && sjinfo->lhs_strict)
+ my_quals = remove_unneeded_nulling_relids(root, my_quals, sjinfo);
+
+ /*
+ * Now we can compute ojscope (we can't do it earlier, because
+ * remove_unneeded_nulling_relids might change the scope).
+ *
+ * Semijoins are a bit of a hybrid: we build a SpecialJoinInfo, but we
+ * want ojscope = NULL for distribute_qual_to_rels.
+ */
+ if (j->jointype == JOIN_INNER || j->jointype == JOIN_SEMI)
ojscope = NULL;
- }
+ else
+ ojscope = bms_union(sjinfo->min_lefthand,
+ sjinfo->min_righthand);
/* Process the JOIN's qual clauses */
foreach(l, my_quals)
@@ -1009,7 +1051,7 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
Node *qual = (Node *) lfirst(l);
distribute_qual_to_rels(root, qual,
- below_outer_join, j->jointype,
+ below_outer_join, sjinfo,
root->qual_security_level,
*qualscope,
ojscope, nonnullable_rels,
@@ -1112,7 +1154,7 @@ process_security_barrier_quals(PlannerInfo *root,
*/
distribute_qual_to_rels(root, qual,
below_outer_join,
- JOIN_INNER,
+ NULL,
security_level,
qualscope,
qualscope,
@@ -1135,6 +1177,7 @@ process_security_barrier_quals(PlannerInfo *root,
* right_rels: the base Relids syntactically on inner side of join
* inner_join_rels: base Relids participating in inner joins below this one
* jointype: what it says (must always be LEFT, FULL, SEMI, or ANTI)
+ * ojrelid: RT index of the join RTE (0 for SEMI, which isn't in the RT list)
* clause: the outer join's join condition (in implicit-AND format)
*
* The node should eventually be appended to root->join_info_list, but we
@@ -1148,7 +1191,8 @@ static SpecialJoinInfo *
make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
- JoinType jointype, List *clause)
+ JoinType jointype, Index ojrelid,
+ List *clause)
{
SpecialJoinInfo *sjinfo = makeNode(SpecialJoinInfo);
Relids clause_relids;
@@ -1196,6 +1240,7 @@ make_outerjoininfo(PlannerInfo *root,
sjinfo->syn_lefthand = left_rels;
sjinfo->syn_righthand = right_rels;
sjinfo->jointype = jointype;
+ sjinfo->ojrelid = ojrelid;
/* this always starts out false */
sjinfo->delay_upper_joins = false;
@@ -1206,6 +1251,7 @@ make_outerjoininfo(PlannerInfo *root,
{
sjinfo->min_lefthand = bms_copy(left_rels);
sjinfo->min_righthand = bms_copy(right_rels);
+ sjinfo->strict_relids = NULL; /* don't care about this */
sjinfo->lhs_strict = false; /* don't care about this */
return sjinfo;
}
@@ -1220,6 +1266,7 @@ make_outerjoininfo(PlannerInfo *root,
* rel's columns are all NULL?
*/
strict_relids = find_nonnullable_rels((Node *) clause);
+ sjinfo->strict_relids = strict_relids;
/* Remember whether the clause is strict for any LHS relations */
sjinfo->lhs_strict = bms_overlap(strict_relids, left_rels);
@@ -1258,6 +1305,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_lefthand = bms_add_member(min_lefthand,
+ otherinfo->ojrelid);
}
if (bms_overlap(right_rels, otherinfo->syn_lefthand) ||
bms_overlap(right_rels, otherinfo->syn_righthand))
@@ -1266,6 +1316,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_righthand = bms_add_member(min_righthand,
+ otherinfo->ojrelid);
}
/* Needn't do anything else with the full join */
continue;
@@ -1295,6 +1348,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_lefthand = bms_add_member(min_lefthand,
+ otherinfo->ojrelid);
}
}
@@ -1337,6 +1393,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_righthand = bms_add_member(min_righthand,
+ otherinfo->ojrelid);
}
}
}
@@ -1561,6 +1620,62 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
sjinfo->semi_rhs_exprs = semi_rhs_exprs;
}
+/*
+ * remove_unneeded_nulling_relids
+ * Remove lower outer joins from Vars (& PHVs) in the quals, if possible
+ *
+ * This paves the way to apply outer join identity 3 to commute the current
+ * LEFT JOIN with lower outer joins. We already know that the quals are
+ * strict for at least one LHS relation.
+ */
+static List *
+remove_unneeded_nulling_relids(PlannerInfo *root, List *quals,
+ SpecialJoinInfo *sjinfo)
+{
+ Relids old_nulling_relids;
+ Relids removable_relids;
+ ListCell *lc;
+
+ /*
+ * Find outer joins mentioned in nullingrel fields in the quals. If there
+ * aren't any (the common case), there's no need to work hard.
+ */
+ old_nulling_relids = get_nulling_relids((Node *) quals);
+ if (bms_is_empty(old_nulling_relids))
+ return quals;
+
+ /*
+ * Thumb through the existing SpecialJoinInfos (which describe all outer
+ * joins below this one, but not yet this one) to find the ones mentioned
+ * in the quals. If the current join's quals are strict for any rel of
+ * one's RHS, we can commute this join with that one, so remove it from
+ * the current join's min_lefthand and from the quals' nullingrel fields.
+ */
+ removable_relids = NULL;
+ foreach(lc, root->join_info_list)
+ {
+ SpecialJoinInfo *sjinfo2 = (SpecialJoinInfo *) lfirst(lc);
+
+ if (sjinfo2->jointype != JOIN_LEFT ||
+ !bms_is_member(sjinfo2->ojrelid, old_nulling_relids))
+ continue; /* it's not relevant */
+ if (bms_is_subset(sjinfo2->syn_righthand, sjinfo->syn_lefthand) &&
+ bms_overlap(sjinfo->strict_relids, sjinfo2->min_righthand))
+ {
+ sjinfo->min_lefthand = bms_del_member(sjinfo->min_lefthand,
+ sjinfo2->ojrelid);
+ removable_relids = bms_add_member(removable_relids,
+ sjinfo2->ojrelid);
+ }
+ }
+
+ if (removable_relids == NULL)
+ return quals; /* no hits, nothing to do */
+
+ return (List *) remove_nulling_relids((Node *) quals,
+ removable_relids, NULL);
+}
+
/*****************************************************************************
*
@@ -1582,7 +1697,7 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
* 'clause': the qual clause to be distributed
* 'below_outer_join': true if the qual is from a JOIN/ON that is below the
* nullable side of a higher-level outer join
- * 'jointype': type of join the qual is from (JOIN_INNER for a WHERE clause)
+ * 'sjinfo': join's SpecialJoinInfo (NULL for an inner join or WHERE clause)
* 'security_level': security_level to assign to the qual
* 'qualscope': set of baserels the qual's syntactic scope covers
* 'ojscope': NULL if not an outer-join qual, else the minimum set of baserels
@@ -1600,12 +1715,13 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
* level, which will be ojscope not necessarily qualscope.
*
* At the time this is called, root->join_info_list must contain entries for
- * all and only those special joins that are syntactically below this qual.
+ * all and only those special joins that are syntactically below this qual;
+ * in particular, the passed-in SpecialJoinInfo isn't yet in that list.
*/
static void
distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool below_outer_join,
- JoinType jointype,
+ SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
Relids ojscope,
@@ -1642,7 +1758,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
PostponedQual *pq = (PostponedQual *) palloc(sizeof(PostponedQual));
Assert(root->hasLateralRTEs); /* shouldn't happen otherwise */
- Assert(jointype == JOIN_INNER); /* mustn't postpone past outer join */
+ Assert(sjinfo == NULL); /* mustn't postpone past outer join */
pq->qual = clause;
pq->relids = relids;
*postponed_qual_list = lappend(*postponed_qual_list, pq);
@@ -1704,7 +1820,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
{
relids =
get_relids_in_jointree((Node *) root->parse->jointree,
- false);
+ true, false);
qualscope = bms_copy(relids);
}
}
@@ -1946,11 +2062,15 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
restrictinfo);
return;
}
- if (jointype == JOIN_FULL)
+ if (sjinfo && sjinfo->jointype == JOIN_FULL)
{
/* FULL JOIN (above tests cannot match in this case) */
+ FullJoinClauseInfo *fjinfo = makeNode(FullJoinClauseInfo);
+
+ fjinfo->rinfo = restrictinfo;
+ fjinfo->sjinfo = sjinfo;
root->full_join_clauses = lappend(root->full_join_clauses,
- restrictinfo);
+ fjinfo);
return;
}
/* nope, so fall through to distribute_restrictinfo_to_rels */
@@ -2344,7 +2464,7 @@ process_implied_equality(PlannerInfo *root,
{
relids =
get_relids_in_jointree((Node *) root->parse->jointree,
- false);
+ true, false);
}
}
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index f6baa2a765..ea077cff4e 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2221,7 +2221,7 @@ preprocess_rowmarks(PlannerInfo *root)
* make a bitmapset of all base rels and then remove the items we don't
* need or have FOR [KEY] UPDATE/SHARE marks for.
*/
- rels = get_relids_in_jointree((Node *) parse->jointree, false);
+ rels = get_relids_in_jointree((Node *) parse->jointree, false, false);
if (parse->resultRelation)
rels = bms_del_member(rels, parse->resultRelation);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 1cb0abdbc1..a1827d113d 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -151,6 +151,9 @@ static Var *search_indexed_tlist_for_var(Var *var,
indexed_tlist *itlist,
int newvarno,
int rtoffset);
+static Var *search_indexed_tlist_for_phv(PlaceHolderVar *phv,
+ indexed_tlist *itlist,
+ int newvarno);
static Var *search_indexed_tlist_for_non_var(Expr *node,
indexed_tlist *itlist,
int newvarno);
@@ -2115,6 +2118,7 @@ fix_scan_expr_mutator(Node *node, fix_scan_expr_context *context)
/* At scan level, we should always just evaluate the contained expr */
PlaceHolderVar *phv = (PlaceHolderVar *) node;
+ Assert(phv->phnullingrels == NULL);
return fix_scan_expr_mutator((Node *) phv->phexpr, context);
}
if (IsA(node, AlternativeSubPlan))
@@ -2235,33 +2239,12 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
/*
* Now we need to fix up the targetlist and qpqual, which are logically
* above the join. This means they should not re-use any input expression
- * that was computed in the nullable side of an outer join. Vars and
- * PlaceHolderVars are fine, so we can implement this restriction just by
- * clearing has_non_vars in the indexed_tlist structs.
+ * that was computed in the nullable side of an outer join.
*
- * XXX This is a grotty workaround for the fact that we don't clearly
- * distinguish between a Var appearing below an outer join and the "same"
- * Var appearing above it. If we did, we'd not need to hack the matching
- * rules this way.
+ * XXX we will probably need to pass some flag down to indicate that this
+ * context applies, so that search_indexed_tlist_for_var() and siblings
+ * can correctly check for varnullingrels matches.
*/
- switch (join->jointype)
- {
- case JOIN_LEFT:
- case JOIN_SEMI:
- case JOIN_ANTI:
- inner_itlist->has_non_vars = false;
- break;
- case JOIN_RIGHT:
- outer_itlist->has_non_vars = false;
- break;
- case JOIN_FULL:
- outer_itlist->has_non_vars = false;
- inner_itlist->has_non_vars = false;
- break;
- default:
- break;
- }
-
join->plan.targetlist = fix_join_expr(root,
join->plan.targetlist,
outer_itlist,
@@ -2550,7 +2533,7 @@ set_dummy_tlist_references(Plan *plan, int rtoffset)
* tlist_member() searches.
*
* The result of this function is an indexed_tlist struct to pass to
- * search_indexed_tlist_for_var() or search_indexed_tlist_for_non_var().
+ * search_indexed_tlist_for_var() and siblings.
* When done, the indexed_tlist may be freed with a single pfree().
*/
static indexed_tlist *
@@ -2672,6 +2655,8 @@ search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
/* Found a match */
Var *newvar = copyVar(var);
+ /* XXX we oughta check varnullingrels match here ... */
+
newvar->varno = newvarno;
newvar->varattno = vinfo->resno;
if (newvar->varnosyn > 0)
@@ -2684,15 +2669,55 @@ search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
}
/*
- * search_indexed_tlist_for_non_var --- find a non-Var in an indexed tlist
+ * search_indexed_tlist_for_phv --- find a PlaceHolderVar in an indexed tlist
*
* If a match is found, return a Var constructed to reference the tlist item.
* If no match, return NULL.
*
- * NOTE: it is a waste of time to call this unless itlist->has_ph_vars or
- * itlist->has_non_vars. Furthermore, set_join_references() relies on being
- * able to prevent matching of non-Vars by clearing itlist->has_non_vars,
- * so there's a correctness reason not to call it unless that's set.
+ * NOTE: it is a waste of time to call this unless itlist->has_ph_vars.
+ */
+static Var *
+search_indexed_tlist_for_phv(PlaceHolderVar *phv,
+ indexed_tlist *itlist, int newvarno)
+{
+ ListCell *lc;
+
+ foreach(lc, itlist->tlist)
+ {
+ TargetEntry *tle = (TargetEntry *) lfirst(lc);
+
+ if (tle->expr && IsA(tle->expr, PlaceHolderVar))
+ {
+ PlaceHolderVar *subphv = (PlaceHolderVar *) tle->expr;
+ Var *newvar;
+
+ /*
+ * Analogously to search_indexed_tlist_for_var, we match on phid
+ * only. We don't use equal(), partially for speed but mostly
+ * because phnullingrels might not be exactly equal.
+ *
+ * XXX we really oughta verify phnullingrels.
+ */
+ if (phv->phid != subphv->phid)
+ continue;
+
+ /* Found a matching subplan output expression */
+ newvar = makeVarFromTargetEntry(newvarno, tle);
+ newvar->varnosyn = 0; /* wasn't ever a plain Var */
+ newvar->varattnosyn = 0;
+ return newvar;
+ }
+ }
+ return NULL; /* no match */
+}
+
+/*
+ * search_indexed_tlist_for_non_var --- find a non-Var/PHV in an indexed tlist
+ *
+ * If a match is found, return a Var constructed to reference the tlist item.
+ * If no match, return NULL.
+ *
+ * NOTE: it is a waste of time to call this unless itlist->has_non_vars.
*/
static Var *
search_indexed_tlist_for_non_var(Expr *node,
@@ -2877,22 +2902,23 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* See if the PlaceHolderVar has bubbled up from a lower plan node */
if (context->outer_itlist && context->outer_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->outer_itlist,
- OUTER_VAR);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->outer_itlist,
+ OUTER_VAR);
if (newvar)
return (Node *) newvar;
}
if (context->inner_itlist && context->inner_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->inner_itlist,
- INNER_VAR);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->inner_itlist,
+ INNER_VAR);
if (newvar)
return (Node *) newvar;
}
/* If not supplied by input plans, evaluate the contained expr */
+ /* XXX assert something about phnullingrels */
return fix_join_expr_mutator((Node *) phv->phexpr, context);
}
/* Try matching more complex expressions too, if tlists have any */
@@ -3001,13 +3027,14 @@ fix_upper_expr_mutator(Node *node, fix_upper_expr_context *context)
/* See if the PlaceHolderVar has bubbled up from a lower plan node */
if (context->subplan_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->subplan_itlist,
- context->newvarno);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->subplan_itlist,
+ context->newvarno);
if (newvar)
return (Node *) newvar;
}
/* If not supplied by input plan, evaluate the contained expr */
+ /* XXX assert something about phnullingrels */
return fix_upper_expr_mutator((Node *) phv->phexpr, context);
}
/* Try matching more complex expressions too, if tlist has any */
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 0bd99acf83..389f7d9ce7 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -49,17 +49,28 @@ typedef struct pullup_replace_vars_context
* pullup (set only if target_rte->lateral) */
bool *outer_hasSubLinks; /* -> outer query's hasSubLinks */
int varno; /* varno of subquery */
- bool need_phvs; /* do we need PlaceHolderVars? */
- bool wrap_non_vars; /* do we need 'em on *all* non-Vars? */
+ bool wrap_non_vars; /* do we need all non-Var outputs to be PHVs? */
Node **rv_cache; /* cache for results with PHVs */
} pullup_replace_vars_context;
-typedef struct reduce_outer_joins_state
+typedef struct reduce_outer_joins_pass1_state
{
Relids relids; /* base relids within this subtree */
bool contains_outer; /* does subtree contain outer join(s)? */
List *sub_states; /* List of states for subtree components */
-} reduce_outer_joins_state;
+} reduce_outer_joins_pass1_state;
+
+typedef struct reduce_outer_joins_pass2_state
+{
+ Relids inner_reduced; /* OJ relids reduced to plain inner joins */
+ List *partial_reduced; /* List of partially reduced FULL joins */
+} reduce_outer_joins_pass2_state;
+
+typedef struct reduce_outer_joins_partial_state
+{
+ int full_join_rti; /* RT index of a formerly-FULL join */
+ Relids unreduced_side; /* relids in its still-nullable side */
+} reduce_outer_joins_partial_state;
static Node *pull_up_sublinks_jointree_recurse(PlannerInfo *root, Node *jtnode,
Relids *relids);
@@ -68,12 +79,10 @@ static Node *pull_up_sublinks_qual_recurse(PlannerInfo *root, Node *node,
Node **jtlink2, Relids available_rels2);
static Node *pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static Node *pull_up_simple_subquery(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static Node *pull_up_simple_union_all(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte);
@@ -90,7 +99,6 @@ static Node *pull_up_simple_values(PlannerInfo *root, Node *jtnode,
static bool is_simple_values(PlannerInfo *root, RangeTblEntry *rte);
static Node *pull_up_constant_function(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static bool is_simple_union_all(Query *subquery);
static bool is_simple_union_all_recurse(Node *setOp, Query *setOpQuery,
@@ -101,25 +109,27 @@ static bool jointree_contains_lateral_outer_refs(PlannerInfo *root,
Relids safe_upper_varnos);
static void perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars_context *rvcontext,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static void replace_vars_in_jointree(Node *jtnode,
- pullup_replace_vars_context *context,
- JoinExpr *lowest_nulling_outer_join);
+ pullup_replace_vars_context *context);
static Node *pullup_replace_vars(Node *expr,
pullup_replace_vars_context *context);
static Node *pullup_replace_vars_callback(Var *var,
replace_rte_variables_context *context);
static Query *pullup_replace_vars_subquery(Query *query,
pullup_replace_vars_context *context);
-static reduce_outer_joins_state *reduce_outer_joins_pass1(Node *jtnode);
+static reduce_outer_joins_pass1_state *reduce_outer_joins_pass1(Node *jtnode);
static void reduce_outer_joins_pass2(Node *jtnode,
- reduce_outer_joins_state *state,
+ reduce_outer_joins_pass1_state *state1,
+ reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
List *nonnullable_vars,
List *forced_null_vars);
-static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode);
+static void report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
+ int rtindex, Relids relids);
+static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Relids *dropped_outer_joins);
static int get_result_relid(PlannerInfo *root, Node *jtnode);
static void remove_result_refs(PlannerInfo *root, int varno, Node *newjtloc);
static bool find_dependent_phvs(PlannerInfo *root, int varno);
@@ -764,7 +774,7 @@ pull_up_subqueries(PlannerInfo *root)
/* Recursion starts with no containing join nor appendrel */
root->parse->jointree = (FromExpr *)
pull_up_subqueries_recurse(root, (Node *) root->parse->jointree,
- NULL, NULL, NULL);
+ NULL, NULL);
/* We should still have a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
}
@@ -779,12 +789,6 @@ pull_up_subqueries(PlannerInfo *root)
* lowest_outer_join references the lowest such JoinExpr node; otherwise
* it is NULL. We use this to constrain the effects of LATERAL subqueries.
*
- * If this jointree node is within the nullable side of an outer join, then
- * lowest_nulling_outer_join references the lowest such JoinExpr node;
- * otherwise it is NULL. This forces use of the PlaceHolderVar mechanism for
- * references to non-nullable targetlist items, but only for references above
- * that join.
- *
* If we are looking at a member subquery of an append relation,
* containing_appendrel describes that relation; else it is NULL.
* This forces use of the PlaceHolderVar mechanism for all non-Var targetlist
@@ -801,15 +805,14 @@ pull_up_subqueries(PlannerInfo *root)
* Notice also that we can't turn pullup_replace_vars loose on the whole
* jointree, because it'd return a mutated copy of the tree; we have to
* invoke it just on the quals, instead. This behavior is what makes it
- * reasonable to pass lowest_outer_join and lowest_nulling_outer_join as
- * pointers rather than some more-indirect way of identifying the lowest
- * OJs. Likewise, we don't replace append_rel_list members but only their
- * substructure, so the containing_appendrel reference is safe to use.
+ * reasonable to pass lowest_outer_join as a pointer rather than some
+ * more-indirect way of identifying the lowest OJ. Likewise, we don't
+ * replace append_rel_list members but only their substructure, so the
+ * containing_appendrel reference is safe to use.
*/
static Node *
pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Assert(jtnode != NULL);
@@ -831,7 +834,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
is_safe_append_member(rte->subquery)))
return pull_up_simple_subquery(root, jtnode, rte,
lowest_outer_join,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -864,7 +866,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
*/
if (rte->rtekind == RTE_FUNCTION)
return pull_up_constant_function(root, jtnode, rte,
- lowest_nulling_outer_join,
containing_appendrel);
/* Otherwise, do nothing at this node. */
@@ -880,7 +881,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
{
lfirst(l) = pull_up_subqueries_recurse(root, lfirst(l),
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
}
}
@@ -895,11 +895,9 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
case JOIN_INNER:
j->larg = pull_up_subqueries_recurse(root, j->larg,
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
break;
case JOIN_LEFT:
@@ -907,31 +905,25 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
case JOIN_ANTI:
j->larg = pull_up_subqueries_recurse(root, j->larg,
j,
- lowest_nulling_outer_join,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
- j,
j,
NULL);
break;
case JOIN_FULL:
j->larg = pull_up_subqueries_recurse(root, j->larg,
- j,
j,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
- j,
j,
NULL);
break;
case JOIN_RIGHT:
j->larg = pull_up_subqueries_recurse(root, j->larg,
- j,
j,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
j,
- lowest_nulling_outer_join,
NULL);
break;
default:
@@ -961,7 +953,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
static Node *
pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -1107,31 +1098,25 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* The subquery's targetlist items are now in the appropriate form to
* insert into the top query, except that we may need to wrap them in
* PlaceHolderVars. Set up required context data for pullup_replace_vars.
+ * (Note that we should include the subquery's inner joins in relids,
+ * since it may include join alias vars referencing them.)
*/
rvcontext.root = root;
rvcontext.targetlist = subquery->targetList;
rvcontext.target_rte = rte;
if (rte->lateral)
rvcontext.relids = get_relids_in_jointree((Node *) subquery->jointree,
- true);
+ true, true);
else /* won't need relids */
rvcontext.relids = NULL;
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = varno;
- /* these flags will be set below, if needed */
- rvcontext.need_phvs = false;
+ /* this flag will be set below, if needed */
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(subquery->targetList) + 1) *
sizeof(Node *));
- /*
- * If we are under an outer join then non-nullable items and lateral
- * references may have to be turned into PlaceHolderVars.
- */
- if (lowest_nulling_outer_join != NULL)
- rvcontext.need_phvs = true;
-
/*
* If we are dealing with an appendrel member then anything that's not a
* simple Var has to be turned into a PlaceHolderVar. We force this to
@@ -1140,10 +1125,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* expression actually available from the appendrel.
*/
if (containing_appendrel != NULL)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* If the parent query uses grouping sets, we need a PlaceHolderVar for
@@ -1155,10 +1137,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* that pullup_replace_vars hasn't currently got.)
*/
if (parse->groupingSets)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* Replace all of the top query's references to the subquery's outputs
@@ -1166,7 +1145,6 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* replace any of the jointree structure.
*/
perform_pullup_replace_vars(root, &rvcontext,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -1233,7 +1211,8 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
{
Relids subrelids;
- subrelids = get_relids_in_jointree((Node *) subquery->jointree, false);
+ subrelids = get_relids_in_jointree((Node *) subquery->jointree,
+ true, false);
substitute_phv_relids((Node *) parse, varno, subrelids);
fix_append_rel_relids(root->append_rel_list, varno, subrelids);
}
@@ -1424,7 +1403,7 @@ pull_up_union_leaf_queries(Node *setOp, PlannerInfo *root, int parentRTindex,
rtr = makeNode(RangeTblRef);
rtr->rtindex = childRTindex;
(void) pull_up_subqueries_recurse(root, (Node *) rtr,
- NULL, NULL, appinfo);
+ NULL, appinfo);
}
else if (IsA(setOp, SetOperationStmt))
{
@@ -1561,7 +1540,7 @@ is_simple_subquery(PlannerInfo *root, Query *subquery, RangeTblEntry *rte,
{
restricted = true;
safe_upper_varnos = get_relids_in_jointree((Node *) lowest_outer_join,
- true);
+ true, true);
}
else
{
@@ -1673,7 +1652,6 @@ pull_up_simple_values(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte)
rvcontext.relids = NULL;
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = varno;
- rvcontext.need_phvs = false;
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(tlist) + 1) *
@@ -1685,7 +1663,7 @@ pull_up_simple_values(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte)
* any of the jointree structure. We can assume there's no outer joins or
* appendrels in the dummy Query that surrounds a VALUES RTE.
*/
- perform_pullup_replace_vars(root, &rvcontext, NULL, NULL);
+ perform_pullup_replace_vars(root, &rvcontext, NULL);
/*
* There should be no appendrels to fix, nor any outer joins and hence no
@@ -1784,7 +1762,6 @@ is_simple_values(PlannerInfo *root, RangeTblEntry *rte)
static Node *
pull_up_constant_function(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -1836,40 +1813,26 @@ pull_up_constant_function(PlannerInfo *root, Node *jtnode,
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = ((RangeTblRef *) jtnode)->rtindex;
- /* these flags will be set below, if needed */
- rvcontext.need_phvs = false;
+ /* this flag will be set below, if needed */
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(rvcontext.targetlist) + 1) *
sizeof(Node *));
- /*
- * If we are under an outer join then non-nullable items and lateral
- * references may have to be turned into PlaceHolderVars.
- */
- if (lowest_nulling_outer_join != NULL)
- rvcontext.need_phvs = true;
-
/*
* If we are dealing with an appendrel member then anything that's not a
* simple Var has to be turned into a PlaceHolderVar. (See comments in
* pull_up_simple_subquery().)
*/
if (containing_appendrel != NULL)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* If the parent query uses grouping sets, we need a PlaceHolderVar for
* anything that's not a simple Var.
*/
if (parse->groupingSets)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* Replace all of the top query's references to the RTE's output with
@@ -1877,7 +1840,6 @@ pull_up_constant_function(PlannerInfo *root, Node *jtnode,
* jointree structure.
*/
perform_pullup_replace_vars(root, &rvcontext,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -2099,13 +2061,11 @@ jointree_contains_lateral_outer_refs(PlannerInfo *root, Node *jtnode,
*
* Caller has already filled *rvcontext with data describing what to
* substitute for Vars referencing the target subquery. In addition
- * we need the identity of the lowest outer join that can null the
- * target subquery, and its containing appendrel if any.
+ * we need the identity of the containing appendrel if any.
*/
static void
perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars_context *rvcontext,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -2149,38 +2109,31 @@ perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars((Node *) action->targetList, rvcontext);
}
}
- replace_vars_in_jointree((Node *) parse->jointree, rvcontext,
- lowest_nulling_outer_join);
+ replace_vars_in_jointree((Node *) parse->jointree, rvcontext);
Assert(parse->setOperations == NULL);
parse->havingQual = pullup_replace_vars(parse->havingQual, rvcontext);
/*
* Replace references in the translated_vars lists of appendrels. When
- * pulling up an appendrel member, we do not need PHVs in the list of the
- * parent appendrel --- there isn't any outer join between. Elsewhere,
- * use PHVs for safety. (This analysis could be made tighter but it seems
- * unlikely to be worth much trouble.)
+ * pulling up an appendrel member, we do not want to force PHVs in the
+ * list of the parent appendrel --- there isn't any outer join between.
+ * Elsewhere, use PHVs for safety. (This analysis could be made tighter
+ * but it seems unlikely to be worth much trouble.)
*/
foreach(lc, root->append_rel_list)
{
AppendRelInfo *appinfo = (AppendRelInfo *) lfirst(lc);
- bool save_need_phvs = rvcontext->need_phvs;
+ bool save_wrap_non_vars = rvcontext->wrap_non_vars;
if (appinfo == containing_appendrel)
- rvcontext->need_phvs = false;
+ rvcontext->wrap_non_vars = false;
appinfo->translated_vars = (List *)
pullup_replace_vars((Node *) appinfo->translated_vars, rvcontext);
- rvcontext->need_phvs = save_need_phvs;
+ rvcontext->wrap_non_vars = save_wrap_non_vars;
}
/*
* Replace references in the joinaliasvars lists of join RTEs.
- *
- * You might think that we could avoid using PHVs for alias vars of joins
- * below lowest_nulling_outer_join, but that doesn't work because the
- * alias vars could be referenced above that join; we need the PHVs to be
- * present in such references after the alias vars get flattened. (It
- * might be worth trying to be smarter here, someday.)
*/
foreach(lc, parse->rtable)
{
@@ -2197,14 +2150,10 @@ perform_pullup_replace_vars(PlannerInfo *root,
* Helper routine for perform_pullup_replace_vars: do pullup_replace_vars on
* every expression in the jointree, without changing the jointree structure
* itself. Ugly, but there's no other way...
- *
- * If we are at or below lowest_nulling_outer_join, we can suppress use of
- * PlaceHolderVars wrapped around the replacement expressions.
*/
static void
replace_vars_in_jointree(Node *jtnode,
- pullup_replace_vars_context *context,
- JoinExpr *lowest_nulling_outer_join)
+ pullup_replace_vars_context *context)
{
if (jtnode == NULL)
return;
@@ -2217,7 +2166,7 @@ replace_vars_in_jointree(Node *jtnode,
* jointree scan, rather than a scan of the rtable, for a couple of
* reasons: we can avoid processing no-longer-referenced RTEs, and we
* can use the appropriate setting of need_phvs depending on whether
- * the RTE is above possibly-nulling outer joins or not.
+ * the RTE is above possibly-nulling outer joins or not. XXX fix
*/
int varno = ((RangeTblRef *) jtnode)->rtindex;
@@ -2274,42 +2223,30 @@ replace_vars_in_jointree(Node *jtnode,
ListCell *l;
foreach(l, f->fromlist)
- replace_vars_in_jointree(lfirst(l), context,
- lowest_nulling_outer_join);
+ replace_vars_in_jointree(lfirst(l), context);
f->quals = pullup_replace_vars(f->quals, context);
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- bool save_need_phvs = context->need_phvs;
+ bool save_wrap_non_vars = context->wrap_non_vars;
- if (j == lowest_nulling_outer_join)
- {
- /* no more PHVs in or below this join */
- context->need_phvs = false;
- lowest_nulling_outer_join = NULL;
- }
- replace_vars_in_jointree(j->larg, context, lowest_nulling_outer_join);
- replace_vars_in_jointree(j->rarg, context, lowest_nulling_outer_join);
+ replace_vars_in_jointree(j->larg, context);
+ replace_vars_in_jointree(j->rarg, context);
/*
- * Use PHVs within the join quals of a full join, even when it's the
- * lowest nulling outer join. Otherwise, we cannot identify which
- * side of the join a pulled-up var-free expression came from, which
- * can lead to failure to make a plan at all because none of the quals
- * appear to be mergeable or hashable conditions. For this purpose we
- * don't care about the state of wrap_non_vars, so leave it alone.
+ * Use PHVs within the join quals of a full join. Otherwise, we
+ * cannot identify which side of the join a pulled-up var-free
+ * expression came from, which can lead to failure to make a plan at
+ * all because none of the quals appear to be mergeable or hashable
+ * conditions.
*/
if (j->jointype == JOIN_FULL)
- context->need_phvs = true;
+ context->wrap_non_vars = true;
j->quals = pullup_replace_vars(j->quals, context);
- /*
- * We don't bother to update the colvars list, since it won't be used
- * again ...
- */
- context->need_phvs = save_need_phvs;
+ context->wrap_non_vars = save_wrap_non_vars;
}
else
elog(ERROR, "unrecognized node type: %d",
@@ -2338,8 +2275,18 @@ pullup_replace_vars_callback(Var *var,
{
pullup_replace_vars_context *rcon = (pullup_replace_vars_context *) context->callback_arg;
int varattno = var->varattno;
+ bool need_phv;
Node *newnode;
+ /*
+ * We need a PlaceHolderVar if the Var-to-be-replaced has nonempty
+ * varnullingrels (unless we find below that the replacement expression is
+ * a Var or PlaceHolderVar that we can just add the nullingrels to). We
+ * also need one if the caller has instructed us that all non-Var/PHV
+ * replacements need to be wrapped for identification purposes.
+ */
+ need_phv = (var->varnullingrels != NULL) || rcon->wrap_non_vars;
+
/*
* If PlaceHolderVars are needed, we cache the modified expressions in
* rcon->rv_cache[]. This is not in hopes of any material speed gain
@@ -2348,13 +2295,16 @@ pullup_replace_vars_callback(Var *var,
* and possibly prevent optimizations that rely on recognizing different
* references to the same subquery output as being equal(). So it's worth
* a bit of extra effort to avoid it.
+ *
+ * The cached items have phlevelsup = 0 and phnullingrels = NULL; we'll
+ * copy them and adjust those values for this reference site below.
*/
- if (rcon->need_phvs &&
+ if (need_phv &&
varattno >= InvalidAttrNumber &&
varattno <= list_length(rcon->targetlist) &&
rcon->rv_cache[varattno] != NULL)
{
- /* Just copy the entry and fall through to adjust its varlevelsup */
+ /* Just copy the entry and fall through to adjust phlevelsup etc */
newnode = copyObject(rcon->rv_cache[varattno]);
}
else if (varattno == InvalidAttrNumber)
@@ -2363,7 +2313,7 @@ pullup_replace_vars_callback(Var *var,
RowExpr *rowexpr;
List *colnames;
List *fields;
- bool save_need_phvs = rcon->need_phvs;
+ bool save_wrap_non_vars = rcon->wrap_non_vars;
int save_sublevelsup = context->sublevels_up;
/*
@@ -2374,18 +2324,18 @@ pullup_replace_vars_callback(Var *var,
* the RowExpr for use of the executor and ruleutils.c.
*
* In order to be able to cache the results, we always generate the
- * expansion with varlevelsup = 0, and then adjust if needed.
+ * expansion with varlevelsup = 0, and then adjust below if needed.
*/
expandRTE(rcon->target_rte,
var->varno, 0 /* not varlevelsup */ , var->location,
(var->vartype != RECORDOID),
&colnames, &fields);
- /* Adjust the generated per-field Vars, but don't insert PHVs */
- rcon->need_phvs = false;
+ /* Expand the generated per-field Vars, but don't insert PHVs there */
+ rcon->wrap_non_vars = false;
context->sublevels_up = 0; /* to match the expandRTE output */
fields = (List *) replace_rte_variables_mutator((Node *) fields,
context);
- rcon->need_phvs = save_need_phvs;
+ rcon->wrap_non_vars = save_wrap_non_vars;
context->sublevels_up = save_sublevelsup;
rowexpr = makeNode(RowExpr);
@@ -2403,14 +2353,13 @@ pullup_replace_vars_callback(Var *var,
* expression to yield NULL, not ROW(NULL,NULL,...) when it is forced
* to null by an outer join.
*/
- if (rcon->need_phvs)
+ if (need_phv)
{
- /* RowExpr is certainly not strict, so always need PHV */
newnode = (Node *)
make_placeholder_expr(rcon->root,
(Expr *) newnode,
bms_make_singleton(rcon->varno));
- /* cache it with the PHV, and with varlevelsup still zero */
+ /* cache it with the PHV, and with phlevelsup etc not set yet */
rcon->rv_cache[InvalidAttrNumber] = copyObject(newnode);
}
}
@@ -2427,7 +2376,7 @@ pullup_replace_vars_callback(Var *var,
newnode = (Node *) copyObject(tle->expr);
/* Insert PlaceHolderVar if needed */
- if (rcon->need_phvs)
+ if (need_phv)
{
bool wrap;
@@ -2453,69 +2402,61 @@ pullup_replace_vars_callback(Var *var,
/* No need to wrap a PlaceHolderVar with another one, either */
wrap = false;
}
- else if (rcon->wrap_non_vars)
- {
- /* Wrap all non-Vars in a PlaceHolderVar */
- wrap = true;
- }
else
{
/*
- * If it contains a Var of the subquery being pulled up, and
- * does not contain any non-strict constructs, then it's
- * certainly nullable so we don't need to insert a
- * PlaceHolderVar.
- *
- * This analysis could be tighter: in particular, a non-strict
- * construct hidden within a lower-level PlaceHolderVar is not
- * reason to add another PHV. But for now it doesn't seem
- * worth the code to be more exact.
- *
- * Note: in future maybe we should insert a PlaceHolderVar
- * anyway, if the tlist item is expensive to evaluate?
- *
- * For a LATERAL subquery, we have to check the actual var
- * membership of the node, but if it's non-lateral then any
- * level-zero var must belong to the subquery.
+ * Must wrap, either because we need a place to insert
+ * varnullingrels or because caller told us to wrap
+ * everything.
*/
- if ((rcon->target_rte->lateral ?
- bms_overlap(pull_varnos(rcon->root, (Node *) newnode),
- rcon->relids) :
- contain_vars_of_level((Node *) newnode, 0)) &&
- !contain_nonstrict_functions((Node *) newnode))
- {
- /* No wrap needed */
- wrap = false;
- }
- else
- {
- /* Else wrap it in a PlaceHolderVar */
- wrap = true;
- }
+ wrap = true;
}
if (wrap)
+ {
newnode = (Node *)
make_placeholder_expr(rcon->root,
(Expr *) newnode,
bms_make_singleton(rcon->varno));
- /*
- * Cache it if possible (ie, if the attno is in range, which it
- * probably always should be). We can cache the value even if we
- * decided we didn't need a PHV, since this result will be
- * suitable for any request that has need_phvs.
- */
- if (varattno > InvalidAttrNumber &&
- varattno <= list_length(rcon->targetlist))
- rcon->rv_cache[varattno] = copyObject(newnode);
+ /*
+ * Cache it if possible (ie, if the attno is in range, which
+ * it probably always should be).
+ */
+ if (varattno > InvalidAttrNumber &&
+ varattno <= list_length(rcon->targetlist))
+ rcon->rv_cache[varattno] = copyObject(newnode);
+ }
}
}
- /* Must adjust varlevelsup if tlist item is from higher query */
+ /* Must adjust varlevelsup if replaced Var is within a subquery */
if (var->varlevelsup > 0)
IncrementVarSublevelsUp(newnode, var->varlevelsup, 0);
+ /* Propagate any varnullingrels into the replacement Var or PHV */
+ if (var->varnullingrels != NULL)
+ {
+ if (IsA(newnode, Var))
+ {
+ Var *newvar = (Var *) newnode;
+
+ Assert(newvar->varlevelsup == var->varlevelsup);
+ newvar->varnullingrels = bms_add_members(newvar->varnullingrels,
+ var->varnullingrels);
+ }
+ else if (IsA(newnode, PlaceHolderVar))
+ {
+ PlaceHolderVar *newphv = (PlaceHolderVar *) newnode;
+
+ Assert(newphv->phlevelsup == var->varlevelsup);
+ newphv->phnullingrels = bms_add_members(newphv->phnullingrels,
+ var->varnullingrels);
+ }
+ else
+ elog(ERROR, "failed to wrap a non-Var");
+ }
+
return newnode;
}
@@ -2674,7 +2615,9 @@ flatten_simple_union_all(PlannerInfo *root)
void
reduce_outer_joins(PlannerInfo *root)
{
- reduce_outer_joins_state *state;
+ reduce_outer_joins_pass1_state *state1;
+ reduce_outer_joins_pass2_state state2;
+ ListCell *lc;
/*
* To avoid doing strictness checks on more quals than necessary, we want
@@ -2685,14 +2628,44 @@ reduce_outer_joins(PlannerInfo *root)
* join(s) below each side of each join clause. The second pass examines
* qual clauses and changes join types as it descends the tree.
*/
- state = reduce_outer_joins_pass1((Node *) root->parse->jointree);
+ state1 = reduce_outer_joins_pass1((Node *) root->parse->jointree);
/* planner.c shouldn't have called me if no outer joins */
- if (state == NULL || !state->contains_outer)
+ if (state1 == NULL || !state1->contains_outer)
elog(ERROR, "so where are the outer joins?");
+ state2.inner_reduced = NULL;
+ state2.partial_reduced = NIL;
+
reduce_outer_joins_pass2((Node *) root->parse->jointree,
- state, root, NULL, NIL, NIL);
+ state1, &state2,
+ root, NULL, NIL, NIL);
+
+ /*
+ * If we successfully reduced the strength of any outer joins, we must
+ * remove references to those joins as nulling rels. This is handled as
+ * an additional pass, for simplicity and because we can handle all
+ * fully-reduced joins in a single pass over the parse tree.
+ */
+ if (!bms_is_empty(state2.inner_reduced))
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ state2.inner_reduced,
+ NULL);
+
+ /*
+ * Partially-reduced full joins have to be done one at a time, since
+ * they'll each need a different setting of except_relids.
+ */
+ foreach(lc, state2.partial_reduced)
+ {
+ reduce_outer_joins_partial_state *statep = lfirst(lc);
+
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ bms_make_singleton(statep->full_join_rti),
+ statep->unreduced_side);
+ }
}
/*
@@ -2700,13 +2673,13 @@ reduce_outer_joins(PlannerInfo *root)
*
* Returns a state node describing the given jointree node.
*/
-static reduce_outer_joins_state *
+static reduce_outer_joins_pass1_state *
reduce_outer_joins_pass1(Node *jtnode)
{
- reduce_outer_joins_state *result;
+ reduce_outer_joins_pass1_state *result;
- result = (reduce_outer_joins_state *)
- palloc(sizeof(reduce_outer_joins_state));
+ result = (reduce_outer_joins_pass1_state *)
+ palloc(sizeof(reduce_outer_joins_pass1_state));
result->relids = NULL;
result->contains_outer = false;
result->sub_states = NIL;
@@ -2726,7 +2699,7 @@ reduce_outer_joins_pass1(Node *jtnode)
foreach(l, f->fromlist)
{
- reduce_outer_joins_state *sub_state;
+ reduce_outer_joins_pass1_state *sub_state;
sub_state = reduce_outer_joins_pass1(lfirst(l));
result->relids = bms_add_members(result->relids,
@@ -2738,7 +2711,7 @@ reduce_outer_joins_pass1(Node *jtnode)
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- reduce_outer_joins_state *sub_state;
+ reduce_outer_joins_pass1_state *sub_state;
/* join's own RT index is not wanted in result->relids */
if (IS_OUTER_JOIN(j->jointype))
@@ -2766,15 +2739,23 @@ reduce_outer_joins_pass1(Node *jtnode)
* reduce_outer_joins_pass2 - phase 2 processing
*
* jtnode: current jointree node
- * state: state data collected by phase 1 for this node
+ * state1: state data collected by phase 1 for this node
+ * state2: where to accumulate info about successfully-reduced joins
* root: toplevel planner state
* nonnullable_rels: set of base relids forced non-null by upper quals
* nonnullable_vars: list of Vars forced non-null by upper quals
* forced_null_vars: list of Vars forced null by upper quals
+ *
+ * Returns info in state2 about outer joins that were successfully simplified.
+ * Joins that were fully reduced to inner joins are all added to
+ * state2->inner_reduced. If a full join is reduced to a left join,
+ * it needs its own entry in state2->partial_reduced, since that will
+ * require custom processing to remove only the correct nullingrel markers.
*/
static void
reduce_outer_joins_pass2(Node *jtnode,
- reduce_outer_joins_state *state,
+ reduce_outer_joins_pass1_state *state1,
+ reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
List *nonnullable_vars,
@@ -2808,13 +2789,14 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_forced_null_vars = list_concat(pass_forced_null_vars,
forced_null_vars);
/* And recurse --- but only into interesting subtrees */
- Assert(list_length(f->fromlist) == list_length(state->sub_states));
- forboth(l, f->fromlist, s, state->sub_states)
+ Assert(list_length(f->fromlist) == list_length(state1->sub_states));
+ forboth(l, f->fromlist, s, state1->sub_states)
{
- reduce_outer_joins_state *sub_state = lfirst(s);
+ reduce_outer_joins_pass1_state *sub_state = lfirst(s);
if (sub_state->contains_outer)
- reduce_outer_joins_pass2(lfirst(l), sub_state, root,
+ reduce_outer_joins_pass2(lfirst(l), sub_state,
+ state2, root,
pass_nonnullable_rels,
pass_nonnullable_vars,
pass_forced_null_vars);
@@ -2827,8 +2809,8 @@ reduce_outer_joins_pass2(Node *jtnode,
JoinExpr *j = (JoinExpr *) jtnode;
int rtindex = j->rtindex;
JoinType jointype = j->jointype;
- reduce_outer_joins_state *left_state = linitial(state->sub_states);
- reduce_outer_joins_state *right_state = lsecond(state->sub_states);
+ reduce_outer_joins_pass1_state *left_state = linitial(state1->sub_states);
+ reduce_outer_joins_pass1_state *right_state = lsecond(state1->sub_states);
List *local_nonnullable_vars = NIL;
bool computed_local_nonnullable_vars = false;
@@ -2851,12 +2833,22 @@ reduce_outer_joins_pass2(Node *jtnode,
if (bms_overlap(nonnullable_rels, right_state->relids))
jointype = JOIN_INNER;
else
+ {
jointype = JOIN_LEFT;
+ /* Also report partial reduction in state2 */
+ report_reduced_full_join(state2, rtindex,
+ right_state->relids);
+ }
}
else
{
if (bms_overlap(nonnullable_rels, right_state->relids))
+ {
jointype = JOIN_RIGHT;
+ /* Also report partial reduction in state2 */
+ report_reduced_full_join(state2, rtindex,
+ left_state->relids);
+ }
}
break;
case JOIN_SEMI:
@@ -2889,8 +2881,8 @@ reduce_outer_joins_pass2(Node *jtnode,
j->larg = j->rarg;
j->rarg = tmparg;
jointype = JOIN_LEFT;
- right_state = linitial(state->sub_states);
- left_state = lsecond(state->sub_states);
+ right_state = linitial(state1->sub_states);
+ left_state = lsecond(state1->sub_states);
}
/*
@@ -2923,7 +2915,10 @@ reduce_outer_joins_pass2(Node *jtnode,
jointype = JOIN_ANTI;
}
- /* Apply the jointype change, if any, to both jointree node and RTE */
+ /*
+ * Apply the jointype change, if any, to both jointree node and RTE.
+ * Also, if we changed an RTE to INNER, add its RTI to inner_reduced.
+ */
if (rtindex && jointype != j->jointype)
{
RangeTblEntry *rte = rt_fetch(rtindex, root->parse->rtable);
@@ -2931,6 +2926,9 @@ reduce_outer_joins_pass2(Node *jtnode,
Assert(rte->rtekind == RTE_JOIN);
Assert(rte->jointype == j->jointype);
rte->jointype = jointype;
+ if (jointype == JOIN_INNER)
+ state2->inner_reduced = bms_add_member(state2->inner_reduced,
+ rtindex);
}
j->jointype = jointype;
@@ -3011,7 +3009,8 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_nonnullable_vars = NIL;
pass_forced_null_vars = NIL;
}
- reduce_outer_joins_pass2(j->larg, left_state, root,
+ reduce_outer_joins_pass2(j->larg, left_state,
+ state2, root,
pass_nonnullable_rels,
pass_nonnullable_vars,
pass_forced_null_vars);
@@ -3033,7 +3032,8 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_nonnullable_vars = NIL;
pass_forced_null_vars = NIL;
}
- reduce_outer_joins_pass2(j->rarg, right_state, root,
+ reduce_outer_joins_pass2(j->rarg, right_state,
+ state2, root,
pass_nonnullable_rels,
pass_nonnullable_vars,
pass_forced_null_vars);
@@ -3046,6 +3046,19 @@ reduce_outer_joins_pass2(Node *jtnode,
(int) nodeTag(jtnode));
}
+/* Helper for reduce_outer_joins_pass2 */
+static void
+report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
+ int rtindex, Relids relids)
+{
+ reduce_outer_joins_partial_state *statep;
+
+ statep = palloc(sizeof(reduce_outer_joins_partial_state));
+ statep->full_join_rti = rtindex;
+ statep->unreduced_side = relids;
+ state2->partial_reduced = lappend(state2->partial_reduced, statep);
+}
+
/*
* remove_useless_result_rtes
@@ -3087,16 +3100,34 @@ reduce_outer_joins_pass2(Node *jtnode,
void
remove_useless_result_rtes(PlannerInfo *root)
{
+ Relids dropped_outer_joins = NULL;
ListCell *cell;
/* Top level of jointree must always be a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
/* Recurse ... */
root->parse->jointree = (FromExpr *)
- remove_useless_results_recurse(root, (Node *) root->parse->jointree);
+ remove_useless_results_recurse(root,
+ (Node *) root->parse->jointree,
+ &dropped_outer_joins);
/* We should still have a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
+ /*
+ * If we removed any outer-join nodes from the jointree, run around and
+ * remove references to those joins as nulling rels. (There could be such
+ * references in PHVs that we pulled up out of the original subquery that
+ * the RESULT rel replaced. This is kosher on the grounds that we now
+ * know that such an outer join wouldn't really have nulled anything.) We
+ * don't do this during the main recursion, for simplicity and because we
+ * can handle all such joins in a single pass over the parse tree.
+ */
+ if (!bms_is_empty(dropped_outer_joins))
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ dropped_outer_joins,
+ NULL);
+
/*
* Remove any PlanRowMark referencing an RTE_RESULT RTE. We obviously
* must do that for any RTE_RESULT that we just removed. But one for a
@@ -3122,9 +3153,12 @@ remove_useless_result_rtes(PlannerInfo *root)
* Recursive guts of remove_useless_result_rtes.
*
* This recursively processes the jointree and returns a modified jointree.
+ * In addition, the RT indexes of any removed outer-join nodes are added to
+ * *dropped_outer_joins.
*/
static Node *
-remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
+remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Relids *dropped_outer_joins)
{
Assert(jtnode != NULL);
if (IsA(jtnode, RangeTblRef))
@@ -3152,7 +3186,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
int varno;
/* Recursively transform child ... */
- child = remove_useless_results_recurse(root, child);
+ child = remove_useless_results_recurse(root, child,
+ dropped_outer_joins);
/* ... and stick it back into the tree */
lfirst(cell) = child;
@@ -3201,8 +3236,10 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
int varno;
/* First, recurse */
- j->larg = remove_useless_results_recurse(root, j->larg);
- j->rarg = remove_useless_results_recurse(root, j->rarg);
+ j->larg = remove_useless_results_recurse(root, j->larg,
+ dropped_outer_joins);
+ j->rarg = remove_useless_results_recurse(root, j->rarg,
+ dropped_outer_joins);
/* Apply join-type-specific optimization rules */
switch (j->jointype)
@@ -3270,6 +3307,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
!find_dependent_phvs(root, varno)))
{
remove_result_refs(root, varno, j->larg);
+ *dropped_outer_joins = bms_add_member(*dropped_outer_joins,
+ j->rtindex);
jtnode = j->larg;
}
break;
@@ -3280,6 +3319,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
!find_dependent_phvs(root, varno)))
{
remove_result_refs(root, varno, j->rarg);
+ *dropped_outer_joins = bms_add_member(*dropped_outer_joins,
+ j->rtindex);
jtnode = j->rarg;
}
break;
@@ -3294,11 +3335,14 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
* Unlike the LEFT/RIGHT cases, we just Assert that there are
* no PHVs that need to be evaluated at the semijoin's RHS,
* since the rest of the query couldn't reference any outputs
- * of the semijoin's RHS.
+ * of the semijoin's RHS. Also, we don't need to worry about
+ * removing traces of the join's rtindex, since it hasn't got
+ * one.
*/
if ((varno = get_result_relid(root, j->rarg)) != 0)
{
Assert(!find_dependent_phvs(root, varno));
+ Assert(j->rtindex == 0);
remove_result_refs(root, varno, j->larg);
if (j->quals)
jtnode = (Node *)
@@ -3367,7 +3411,7 @@ remove_result_refs(PlannerInfo *root, int varno, Node *newjtloc)
{
Relids subrelids;
- subrelids = get_relids_in_jointree(newjtloc, false);
+ subrelids = get_relids_in_jointree(newjtloc, true, false);
Assert(!bms_is_empty(subrelids));
substitute_phv_relids((Node *) root->parse, varno, subrelids);
fix_append_rel_relids(root->append_rel_list, varno, subrelids);
@@ -3479,7 +3523,7 @@ find_dependent_phvs_in_jointree(PlannerInfo *root, Node *node, int varno)
* are not marked LATERAL, though, since they couldn't possibly contain
* any cross-references to other RTEs.
*/
- subrelids = get_relids_in_jointree(node, false);
+ subrelids = get_relids_in_jointree(node, false, false);
relid = -1;
while ((relid = bms_next_member(subrelids, relid)) >= 0)
{
@@ -3623,11 +3667,17 @@ fix_append_rel_relids(List *append_rel_list, int varno, Relids subrelids)
/*
* get_relids_in_jointree: get set of RT indexes present in a jointree
*
- * If include_joins is true, join RT indexes are included; if false,
- * only base rels are included.
+ * Base-relation relids are always included in the result.
+ * If include_outer_joins is true, outer-join RT indexes are included.
+ * If include_inner_joins is true, inner-join RT indexes are included.
+ *
+ * Note that for most purposes in the planner, outer joins are included
+ * in standard relid sets. Setting include_inner_joins true is only
+ * appropriate for special purposes during subquery flattening.
*/
Relids
-get_relids_in_jointree(Node *jtnode, bool include_joins)
+get_relids_in_jointree(Node *jtnode, bool include_outer_joins,
+ bool include_inner_joins)
{
Relids result = NULL;
@@ -3648,18 +3698,34 @@ get_relids_in_jointree(Node *jtnode, bool include_joins)
{
result = bms_join(result,
get_relids_in_jointree(lfirst(l),
- include_joins));
+ include_outer_joins,
+ include_inner_joins));
}
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- result = get_relids_in_jointree(j->larg, include_joins);
+ result = get_relids_in_jointree(j->larg,
+ include_outer_joins,
+ include_inner_joins);
result = bms_join(result,
- get_relids_in_jointree(j->rarg, include_joins));
- if (include_joins && j->rtindex)
- result = bms_add_member(result, j->rtindex);
+ get_relids_in_jointree(j->rarg,
+ include_outer_joins,
+ include_inner_joins));
+ if (j->rtindex)
+ {
+ if (j->jointype == JOIN_INNER)
+ {
+ if (include_inner_joins)
+ result = bms_add_member(result, j->rtindex);
+ }
+ else
+ {
+ if (include_outer_joins)
+ result = bms_add_member(result, j->rtindex);
+ }
+ }
}
else
elog(ERROR, "unrecognized node type: %d",
@@ -3668,7 +3734,7 @@ get_relids_in_jointree(Node *jtnode, bool include_joins)
}
/*
- * get_relids_for_join: get set of base RT indexes making up a join
+ * get_relids_for_join: get set of base+OJ RT indexes making up a join
*/
Relids
get_relids_for_join(Query *query, int joinrelid)
@@ -3679,7 +3745,7 @@ get_relids_for_join(Query *query, int joinrelid)
joinrelid);
if (!jtnode)
elog(ERROR, "could not find join node %d", joinrelid);
- return get_relids_in_jointree(jtnode, false);
+ return get_relids_in_jointree(jtnode, true, false);
}
/*
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index 62cccf9d87..e793a4c85b 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -228,6 +228,12 @@ adjust_appendrel_attrs_mutator(Node *node,
if (var->varlevelsup != 0)
return (Node *) var; /* no changes needed */
+ /*
+ * You might think we need to adjust var->varnullingrels, but that
+ * shouldn't need any changes. It will contain outer-join relids,
+ * while the transformation we are making affects only baserels.
+ */
+
for (cnt = 0; cnt < nappinfos; cnt++)
{
if (var->varno == appinfos[cnt]->parent_relid)
@@ -348,6 +354,8 @@ adjust_appendrel_attrs_mutator(Node *node,
var = copyObject(ridinfo->rowidvar);
/* ... but use the correct relid */
var->varno = leaf_relid;
+ /* identity vars shouldn't have nulling rels */
+ Assert(var->varnullingrels == NULL);
/* varnosyn in the RowIdentityVarInfo is probably wrong */
var->varnosyn = 0;
var->varattnosyn = 0;
@@ -392,8 +400,11 @@ adjust_appendrel_attrs_mutator(Node *node,
(void *) context);
/* now fix PlaceHolderVar's relid sets */
if (phv->phlevelsup == 0)
- phv->phrels = adjust_child_relids(phv->phrels, context->nappinfos,
- context->appinfos);
+ {
+ phv->phrels = adjust_child_relids(phv->phrels,
+ nappinfos, appinfos);
+ /* as above, we needn't touch phnullingrels */
+ }
return (Node *) phv;
}
/* Shouldn't need to handle planner auxiliary nodes here */
@@ -688,7 +699,11 @@ get_translated_update_targetlist(PlannerInfo *root, Index relid,
/*
* find_appinfos_by_relids
- * Find AppendRelInfo structures for all relations specified by relids.
+ * Find AppendRelInfo structures for base relations listed in relids.
+ *
+ * The relids argument is typically a join relation's relids, which can
+ * include outer-join RT indexes in addition to baserels. We silently
+ * ignore the outer joins.
*
* The AppendRelInfos are returned in an array, which can be pfree'd by the
* caller. *nappinfos is set to the number of entries in the array.
@@ -700,8 +715,9 @@ find_appinfos_by_relids(PlannerInfo *root, Relids relids, int *nappinfos)
int cnt = 0;
int i;
- *nappinfos = bms_num_members(relids);
- appinfos = (AppendRelInfo **) palloc(sizeof(AppendRelInfo *) * *nappinfos);
+ /* Allocate an array that's certainly big enough */
+ appinfos = (AppendRelInfo **)
+ palloc(sizeof(AppendRelInfo *) * bms_num_members(relids));
i = -1;
while ((i = bms_next_member(relids, i)) >= 0)
@@ -709,10 +725,17 @@ find_appinfos_by_relids(PlannerInfo *root, Relids relids, int *nappinfos)
AppendRelInfo *appinfo = root->append_rel_array[i];
if (!appinfo)
+ {
+ /* Probably i is an OJ index, but let's check */
+ if (find_base_rel_ignore_join(root, i) == NULL)
+ continue;
+ /* It's a base rel, but we lack an append_rel_array entry */
elog(ERROR, "child rel %d not found in append_rel_array", i);
+ }
appinfos[cnt++] = appinfo;
}
+ *nappinfos = cnt;
return appinfos;
}
@@ -754,6 +777,7 @@ add_row_identity_var(PlannerInfo *root, Var *orig_var,
Assert(IsA(orig_var, Var));
Assert(orig_var->varno == rtindex);
Assert(orig_var->varlevelsup == 0);
+ Assert(orig_var->varnullingrels == NULL);
/*
* If we're doing non-inherited UPDATE/DELETE/MERGE, there's little need
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 533df86ff7..c82fd451b2 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -2022,14 +2022,16 @@ is_pseudo_constant_clause_relids(Node *clause, Relids relids)
* NumRelids
* (formerly clause_relids)
*
- * Returns the number of different relations referenced in 'clause'.
+ * Returns the number of different base relations referenced in 'clause'.
*/
int
NumRelids(PlannerInfo *root, Node *clause)
{
+ int result;
Relids varnos = pull_varnos(root, clause);
- int result = bms_num_members(varnos);
+ varnos = bms_del_members(varnos, root->outer_join_rels);
+ result = bms_num_members(varnos);
bms_free(varnos);
return result;
}
diff --git a/src/backend/optimizer/util/joininfo.c b/src/backend/optimizer/util/joininfo.c
index d4cffdb198..afd243f5d8 100644
--- a/src/backend/optimizer/util/joininfo.c
+++ b/src/backend/optimizer/util/joininfo.c
@@ -88,8 +88,8 @@ have_relevant_joinclause(PlannerInfo *root,
* not depend on context).
*
* 'restrictinfo' describes the join clause
- * 'join_relids' is the list of relations participating in the join clause
- * (there must be more than one)
+ * 'join_relids' is the set of relations participating in the join clause
+ * (some of these could be outer joins)
*/
void
add_join_clause_to_rels(PlannerInfo *root,
@@ -101,8 +101,11 @@ add_join_clause_to_rels(PlannerInfo *root,
cur_relid = -1;
while ((cur_relid = bms_next_member(join_relids, cur_relid)) >= 0)
{
- RelOptInfo *rel = find_base_rel(root, cur_relid);
+ RelOptInfo *rel = find_base_rel_ignore_join(root, cur_relid);
+ /* We only need to add the clause to baserels */
+ if (rel == NULL)
+ continue;
rel->joininfo = lappend(rel->joininfo, restrictinfo);
}
}
@@ -115,8 +118,8 @@ add_join_clause_to_rels(PlannerInfo *root,
* discover that a relation need not be joined at all.
*
* 'restrictinfo' describes the join clause
- * 'join_relids' is the list of relations participating in the join clause
- * (there must be more than one)
+ * 'join_relids' is the set of relations participating in the join clause
+ * (some of these could be outer joins)
*/
void
remove_join_clause_from_rels(PlannerInfo *root,
@@ -128,7 +131,11 @@ remove_join_clause_from_rels(PlannerInfo *root,
cur_relid = -1;
while ((cur_relid = bms_next_member(join_relids, cur_relid)) >= 0)
{
- RelOptInfo *rel = find_base_rel(root, cur_relid);
+ RelOptInfo *rel = find_base_rel_ignore_join(root, cur_relid);
+
+ /* We would only have added the clause to baserels */
+ if (rel == NULL)
+ continue;
/*
* Remove the restrictinfo from the list. Pointer comparison is
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index b1363df065..a62d4587ea 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -338,7 +338,9 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
sjinfo.syn_lefthand = sjinfo.min_lefthand;
sjinfo.syn_righthand = sjinfo.min_righthand;
sjinfo.jointype = JOIN_INNER;
+ sjinfo.ojrelid = 0;
/* we don't bother trying to make the remaining fields valid */
+ sjinfo.strict_relids = NULL;
sjinfo.lhs_strict = false;
sjinfo.delay_upper_joins = false;
sjinfo.semi_can_btree = false;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index e10561d843..b11d50bbe9 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -1307,7 +1307,7 @@ create_append_path(PlannerInfo *root,
* Apply query-wide LIMIT if known and path is for sole base relation.
* (Handling this at this low level is a bit klugy.)
*/
- if (root != NULL && bms_equal(rel->relids, root->all_baserels))
+ if (root != NULL && bms_equal(rel->relids, root->all_query_rels))
pathnode->limit_tuples = root->limit_tuples;
else
pathnode->limit_tuples = -1.0;
@@ -1436,7 +1436,7 @@ create_merge_append_path(PlannerInfo *root,
* Apply query-wide LIMIT if known and path is for sole base relation.
* (Handling this at this low level is a bit klugy.)
*/
- if (bms_equal(rel->relids, root->all_baserels))
+ if (bms_equal(rel->relids, root->all_query_rels))
pathnode->limit_tuples = root->limit_tuples;
else
pathnode->limit_tuples = -1.0;
diff --git a/src/backend/optimizer/util/placeholder.c b/src/backend/optimizer/util/placeholder.c
index 3b0f0584f0..4166e55d9a 100644
--- a/src/backend/optimizer/util/placeholder.c
+++ b/src/backend/optimizer/util/placeholder.c
@@ -32,8 +32,14 @@ static void find_placeholders_in_expr(PlannerInfo *root, Node *expr);
* make_placeholder_expr
* Make a PlaceHolderVar for the given expression.
*
- * phrels is the syntactic location (as a set of baserels) to attribute
+ * phrels is the syntactic location (as a set of relids) to attribute
* to the expression.
+ *
+ * The caller is responsible for adjusting phlevelsup and phnullingrels
+ * as needed. Because we do not know here which query level the PHV
+ * will be associated with, it's important that this function touches
+ * only root->glob; messing with other parts of PlannerInfo would be
+ * likely to do the wrong thing.
*/
PlaceHolderVar *
make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
@@ -42,8 +48,9 @@ make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
phv->phexpr = expr;
phv->phrels = phrels;
+ phv->phnullingrels = NULL; /* caller may change this later */
phv->phid = ++(root->glob->lastPHId);
- phv->phlevelsup = 0;
+ phv->phlevelsup = 0; /* caller may change this later */
return phv;
}
@@ -317,6 +324,8 @@ update_placeholder_eval_levels(PlannerInfo *root, SpecialJoinInfo *new_sjinfo)
sjinfo->min_lefthand);
eval_at = bms_add_members(eval_at,
sjinfo->min_righthand);
+ if (sjinfo->ojrelid)
+ eval_at = bms_add_member(eval_at, sjinfo->ojrelid);
/* we'll need another iteration */
found_some = true;
}
@@ -390,9 +399,16 @@ add_placeholders_to_base_rels(PlannerInfo *root)
bms_nonempty_difference(phinfo->ph_needed, eval_at))
{
RelOptInfo *rel = find_base_rel(root, varno);
+ PlaceHolderVar *phv;
- rel->reltarget->exprs = lappend(rel->reltarget->exprs,
- copyObject(phinfo->ph_var));
+ /*
+ * As in add_vars_to_targetlist(), a value computed at scan level
+ * has not yet been nulled by any outer join, so set its
+ * phnullingrels to empty.
+ */
+ phv = copyObject(phinfo->ph_var);
+ phv->phnullingrels = NULL;
+ rel->reltarget->exprs = lappend(rel->reltarget->exprs, phv);
/* reltarget's cost and width fields will be updated later */
}
}
@@ -411,7 +427,8 @@ add_placeholders_to_base_rels(PlannerInfo *root)
*/
void
add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *outer_rel, RelOptInfo *inner_rel)
+ RelOptInfo *outer_rel, RelOptInfo *inner_rel,
+ SpecialJoinInfo *sjinfo)
{
Relids relids = joinrel->relids;
ListCell *lc;
@@ -426,10 +443,11 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
/* Is it still needed above this joinrel? */
if (bms_nonempty_difference(phinfo->ph_needed, relids))
{
- /* Yup, add it to the output */
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- phinfo->ph_var);
- joinrel->reltarget->width += phinfo->ph_width;
+ /*
+ * Yup, we must add it to the output. Make a copy so we can
+ * adjust phnullingrels if needed.
+ */
+ PlaceHolderVar *phv = copyObject(phinfo->ph_var);
/*
* Charge the cost of evaluating the contained expression if
@@ -442,16 +460,42 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
* with that; but we might want to improve it later by
* refiguring the reltarget costs for each pair of inputs.
*/
- if (!bms_is_subset(phinfo->ph_eval_at, outer_rel->relids) &&
- !bms_is_subset(phinfo->ph_eval_at, inner_rel->relids))
+ if (bms_is_subset(phinfo->ph_eval_at, outer_rel->relids))
+ {
+ if (sjinfo->jointype == JOIN_FULL && sjinfo->ojrelid != 0)
+ {
+ /* PHV's value can be nulled at this join */
+ phv->phnullingrels = bms_add_member(phv->phnullingrels,
+ sjinfo->ojrelid);
+ }
+ }
+ else if (bms_is_subset(phinfo->ph_eval_at, inner_rel->relids))
{
+ if (sjinfo->jointype != JOIN_INNER && sjinfo->ojrelid != 0)
+ {
+ /* PHV's value can be nulled at this join */
+ phv->phnullingrels = bms_add_member(phv->phnullingrels,
+ sjinfo->ojrelid);
+ }
+ }
+ else
+ {
+ /* It must be computed here. */
QualCost cost;
+ /* It'll start out not nulled by anything */
+ phv->phnullingrels = NULL;
+ /* Add the appropriate cost */
cost_qual_eval_node(&cost, (Node *) phinfo->ph_var->phexpr,
root);
joinrel->reltarget->cost.startup += cost.startup;
joinrel->reltarget->cost.per_tuple += cost.per_tuple;
}
+
+ joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
+ phv);
+ /* Update width estimate, too */
+ joinrel->reltarget->width += phinfo->ph_width;
}
/*
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index a163853bed..9544ced296 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -39,7 +39,7 @@ typedef struct JoinHashEntry
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *input_rel);
+ RelOptInfo *input_rel, int ojrelid);
static List *build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outer_rel,
@@ -58,7 +58,8 @@ static void set_foreign_rel_properties(RelOptInfo *joinrel,
static void add_join_rel(PlannerInfo *root, RelOptInfo *joinrel);
static void build_joinrel_partition_info(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
- List *restrictlist, JoinType jointype);
+ SpecialJoinInfo *sjinfo,
+ List *restrictlist);
static bool have_partkey_equi_join(RelOptInfo *joinrel,
RelOptInfo *rel1, RelOptInfo *rel2,
JoinType jointype, List *restrictlist);
@@ -66,7 +67,8 @@ static int match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel,
bool strict_op);
static void set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
- JoinType jointype);
+ SpecialJoinInfo *sjinfo);
+static Node *add_nullingrel_to(Node *node, int relid);
static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *parentrel,
RelOptInfo *childrel,
@@ -367,7 +369,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
/*
* find_base_rel
- * Find a base or other relation entry, which must already exist.
+ * Find a base or otherrel relation entry, which must already exist.
*/
RelOptInfo *
find_base_rel(PlannerInfo *root, int relid)
@@ -388,6 +390,44 @@ find_base_rel(PlannerInfo *root, int relid)
return NULL; /* keep compiler quiet */
}
+/*
+ * find_base_rel_ignore_join
+ * Find a base or otherrel relation entry, which must already exist.
+ *
+ * Unlike find_base_rel, if relid references an outer join then this
+ * will return NULL rather than raising an error. This is convenient
+ * for callers that must deal with relid sets including both base and
+ * outer joins.
+ */
+RelOptInfo *
+find_base_rel_ignore_join(PlannerInfo *root, int relid)
+{
+ Assert(relid > 0);
+
+ if (relid < root->simple_rel_array_size)
+ {
+ RelOptInfo *rel;
+ RangeTblEntry *rte;
+
+ rel = root->simple_rel_array[relid];
+ if (rel)
+ return rel;
+
+ /*
+ * We could just return NULL here, but for debugging purposes it seems
+ * best to actually verify that the relid is an outer join and not
+ * something weird.
+ */
+ rte = root->simple_rte_array[relid];
+ if (rte && rte->rtekind == RTE_JOIN && rte->jointype != JOIN_INNER)
+ return NULL;
+ }
+
+ elog(ERROR, "no relation entry for relid %d", relid);
+
+ return NULL; /* keep compiler quiet */
+}
+
/*
* build_join_rel_hash
* Construct the auxiliary hash table for join relations.
@@ -686,9 +726,11 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ build_joinrel_tlist(root, joinrel, outer_rel,
+ (sjinfo->jointype == JOIN_FULL) ? sjinfo->ojrelid : 0);
+ build_joinrel_tlist(root, joinrel, inner_rel,
+ (sjinfo->jointype != JOIN_INNER) ? sjinfo->ojrelid : 0);
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel, sjinfo);
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -720,8 +762,8 @@ build_join_rel(PlannerInfo *root,
joinrel->has_eclass_joins = has_relevant_eclass_joinclause(root, joinrel);
/* Store the partition information. */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- sjinfo->jointype);
+ build_joinrel_partition_info(joinrel, outer_rel, inner_rel, sjinfo,
+ restrictlist);
/*
* Set estimates of the joinrel's size.
@@ -777,16 +819,14 @@ build_join_rel(PlannerInfo *root,
* 'parent_joinrel' is the RelOptInfo representing the join between parent
* relations. Some of the members of new RelOptInfo are produced by
* translating corresponding members of this RelOptInfo
- * 'sjinfo': child-join context info
* 'restrictlist': list of RestrictInfo nodes that apply to this particular
* pair of joinable relations
- * 'jointype' is the join type (inner, left, full, etc)
+ * 'sjinfo': child join's join-type details
*/
RelOptInfo *
build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
RelOptInfo *inner_rel, RelOptInfo *parent_joinrel,
- List *restrictlist, SpecialJoinInfo *sjinfo,
- JoinType jointype)
+ List *restrictlist, SpecialJoinInfo *sjinfo)
{
RelOptInfo *joinrel = makeNode(RelOptInfo);
AppendRelInfo **appinfos;
@@ -800,6 +840,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->reloptkind = RELOPT_OTHER_JOINREL;
joinrel->relids = bms_union(outer_rel->relids, inner_rel->relids);
+ if (sjinfo->ojrelid != 0)
+ joinrel->relids = bms_add_member(joinrel->relids, sjinfo->ojrelid);
joinrel->rows = 0;
/* cheap startup cost is interesting iff not all tuples to be retrieved */
joinrel->consider_startup = (root->tuple_fraction > 0);
@@ -886,8 +928,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->has_eclass_joins = parent_joinrel->has_eclass_joins;
/* Is the join between partitions itself partitioned? */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- jointype);
+ build_joinrel_partition_info(joinrel, outer_rel, inner_rel, sjinfo,
+ restrictlist);
/* Child joinrel is parallel safe if parent is parallel safe. */
joinrel->consider_parallel = parent_joinrel->consider_parallel;
@@ -966,12 +1008,15 @@ min_join_parameterization(PlannerInfo *root,
* will still be needed above the join. This subroutine adds all such
* Vars from the specified input rel's tlist to the join rel's tlist.
*
+ * If the join can null Vars from this input relation, pass its RT index
+ * (if any) as ojrelid; if not, pass zero.
+ *
* We also compute the expected width of the join's output, making use
* of data that was cached at the baserel level by set_rel_width().
*/
static void
build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *input_rel)
+ RelOptInfo *input_rel, int ojrelid)
{
Relids relids = joinrel->relids;
ListCell *vars;
@@ -1002,9 +1047,7 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
RowIdentityVarInfo *ridinfo = (RowIdentityVarInfo *)
list_nth(root->row_identity_vars, var->varattno - 1);
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- var);
- /* Vars have cost zero, so no need to adjust reltarget->cost */
+ /* Update reltarget width estimate from RowIdentityVarInfo */
joinrel->reltarget->width += ridinfo->rowidwidth;
}
else
@@ -1017,15 +1060,28 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
/* Is it still needed above this joinrel? */
ndx = var->varattno - baserel->min_attr;
- if (bms_nonempty_difference(baserel->attr_needed[ndx], relids))
- {
- /* Yup, add it to the output */
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- var);
- /* Vars have cost zero, so no need to adjust reltarget->cost */
- joinrel->reltarget->width += baserel->attr_widths[ndx];
- }
+ if (!bms_nonempty_difference(baserel->attr_needed[ndx], relids))
+ continue; /* nope, skip it */
+
+ /* Update reltarget width estimate from baserel's attr_widths */
+ joinrel->reltarget->width += baserel->attr_widths[ndx];
+ }
+
+ /*
+ * Add the Var to the output. If this join potentially nulls this
+ * input, we have to update the Var's varnullingrels, which means
+ * making a copy.
+ */
+ if (ojrelid != 0)
+ {
+ var = copyObject(var);
+ var->varnullingrels = bms_add_member(var->varnullingrels, ojrelid);
}
+
+ joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
+ var);
+
+ /* Vars have cost zero, so no need to adjust reltarget->cost */
}
}
@@ -1044,7 +1100,7 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
* is not handled in the sub-relations, so it depends on which
* sub-relations are considered.
*
- * If a join clause from an input relation refers to base rels still not
+ * If a join clause from an input relation refers to base+OJ rels still not
* present in the joinrel, then it is still a join clause for the joinrel;
* we put it into the joininfo list for the joinrel. Otherwise,
* the clause is now a restrict clause for the joined relation, and we
@@ -1645,8 +1701,8 @@ find_param_path_info(RelOptInfo *rel, Relids required_outer)
*/
static void
build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
- RelOptInfo *inner_rel, List *restrictlist,
- JoinType jointype)
+ RelOptInfo *inner_rel, SpecialJoinInfo *sjinfo,
+ List *restrictlist)
{
PartitionScheme part_scheme;
@@ -1673,7 +1729,7 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
!inner_rel->consider_partitionwise_join ||
outer_rel->part_scheme != inner_rel->part_scheme ||
!have_partkey_equi_join(joinrel, outer_rel, inner_rel,
- jointype, restrictlist))
+ sjinfo->jointype, restrictlist))
{
Assert(!IS_PARTITIONED_REL(joinrel));
return;
@@ -1697,7 +1753,7 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* child-join relations of the join relation in try_partitionwise_join().
*/
joinrel->part_scheme = part_scheme;
- set_joinrel_partition_key_exprs(joinrel, outer_rel, inner_rel, jointype);
+ set_joinrel_partition_key_exprs(joinrel, outer_rel, inner_rel, sjinfo);
/*
* Set the consider_partitionwise_join flag.
@@ -1877,6 +1933,23 @@ match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel, bool strict_op)
{
if (equal(lfirst(lc), expr))
return cnt;
+
+ /*
+ * XXX For the moment, also allow a match if we have Vars that
+ * match except for varnullingrels. This may be indicative of a
+ * bug, although given the restriction to strict join operators,
+ * it could be okay.
+ */
+ if (IsA(expr, Var) && IsA(lfirst(lc), Var))
+ {
+ Var *v1 = (Var *) expr;
+ Var *v2 = (Var *) lfirst(lc);
+
+ if (v1->varno == v2->varno &&
+ v1->varattno == v2->varattno &&
+ v1->varlevelsup == v2->varlevelsup)
+ return cnt;
+ }
}
}
@@ -1890,7 +1963,7 @@ match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel, bool strict_op)
static void
set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
- JoinType jointype)
+ SpecialJoinInfo *sjinfo)
{
PartitionScheme part_scheme = joinrel->part_scheme;
int partnatts = part_scheme->partnatts;
@@ -1916,7 +1989,7 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
List *nullable_partexpr = NIL;
ListCell *lc;
- switch (jointype)
+ switch (sjinfo->jointype)
{
/*
* A join relation resulting from an INNER join may be
@@ -1992,18 +2065,37 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
* partitionwise nesting of any outer join.) We assume no
* type coercions are needed to make the coalesce expressions,
* since columns of different types won't have gotten
- * classified as the same PartitionScheme.
+ * classified as the same PartitionScheme. However, we do
+ * have to worry about marking the COALESCE inputs as nullable
+ * by the full join, else these won't match the real thing.
*/
foreach(lc, list_concat_copy(outer_expr, outer_null_expr))
{
Node *larg = (Node *) lfirst(lc);
ListCell *lc2;
+ /* Insert nullingrel, or skip it if we can't */
+ larg = add_nullingrel_to(larg, sjinfo->ojrelid);
+ if (larg == NULL)
+ continue;
+
foreach(lc2, list_concat_copy(inner_expr, inner_null_expr))
{
Node *rarg = (Node *) lfirst(lc2);
- CoalesceExpr *c = makeNode(CoalesceExpr);
+ CoalesceExpr *c;
+
+ /* Forget it if coercions would be needed */
+ if (exprType(larg) != exprType(rarg) ||
+ exprCollation(larg) != exprCollation(rarg))
+ continue;
+ /* Insert nullingrel, or skip it if we can't */
+ rarg = add_nullingrel_to(rarg, sjinfo->ojrelid);
+ if (rarg == NULL)
+ continue;
+
+ /* Now we can build a valid merged join variable */
+ c = makeNode(CoalesceExpr);
c->coalescetype = exprType(larg);
c->coalescecollid = exprCollation(larg);
c->args = list_make2(larg, rarg);
@@ -2014,7 +2106,8 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
break;
default:
- elog(ERROR, "unrecognized join type: %d", (int) jointype);
+ elog(ERROR, "unrecognized join type: %d",
+ (int) sjinfo->jointype);
}
joinrel->partexprs[cnt] = partexpr;
@@ -2022,6 +2115,54 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
}
}
+/*
+ * Attempt to add relid to nullingrels of a FULL JOIN USING variable.
+ * Returns the modified expression if successful, or NULL if we failed.
+ *
+ * We currently don't support any cases where type coercion is involved,
+ * so only plain Vars and COALESCE nodes need be handled. However, we
+ * do need to support nested COALESCEs, so recursion is required.
+ */
+static Node *
+add_nullingrel_to(Node *node, int relid)
+{
+ if (IsA(node, Var))
+ {
+ /* Copy so we can modify it... */
+ Var *var = (Var *) copyObject(node);
+
+ /* ... and insert the correct nullingrel marker */
+ var->varnullingrels = bms_add_member(var->varnullingrels,
+ relid);
+ return (Node *) var;
+ }
+ if (IsA(node, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) node;
+ CoalesceExpr *newcexpr;
+ List *newargs = NIL;
+ ListCell *lc;
+
+ /* Try to modify each argument ... */
+ foreach(lc, cexpr->args)
+ {
+ Node *newarg = add_nullingrel_to((Node *) lfirst(lc), relid);
+
+ if (newarg == NULL)
+ return NULL;
+ newargs = lappend(newargs, newarg);
+ }
+ /* Success, so make the result node */
+ newcexpr = makeNode(CoalesceExpr);
+ newcexpr->coalescetype = cexpr->coalescetype;
+ newcexpr->coalescecollid = cexpr->coalescecollid;
+ newcexpr->args = newargs;
+ newcexpr->location = cexpr->location;
+ return (Node *) newcexpr;
+ }
+ return NULL;
+}
+
/*
* build_child_join_reltarget
* Set up a child-join relation's reltarget from a parent-join relation.
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index ef8df3d098..6902c2a9d7 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -116,6 +116,7 @@ make_restrictinfo_internal(PlannerInfo *root,
Relids nullable_relids)
{
RestrictInfo *restrictinfo = makeNode(RestrictInfo);
+ Relids baserels;
restrictinfo->clause = clause;
restrictinfo->orclause = orclause;
@@ -187,6 +188,20 @@ make_restrictinfo_internal(PlannerInfo *root,
else
restrictinfo->required_relids = restrictinfo->clause_relids;
+ /*
+ * Count the number of base rels appearing in clause_relids. To do this,
+ * we just delete rels mentioned in root->outer_join_rels and count the
+ * survivors. Because we are called during deconstruct_jointree which is
+ * the same tree walk that populates outer_join_rels, this is a little bit
+ * unsafe-looking; but it should be fine because the recursion in
+ * deconstruct_jointree should already have visited any outer join that
+ * could be mentioned in this clause.
+ */
+ baserels = bms_difference(restrictinfo->clause_relids,
+ root->outer_join_rels);
+ restrictinfo->num_base_rels = bms_num_members(baserels);
+ bms_free(baserels);
+
/*
* Fill in all the cacheable fields with "not yet set" markers. None of
* these will be computed until/unless needed. Note in particular that we
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
index ebc6ce84b0..99126958db 100644
--- a/src/backend/optimizer/util/var.c
+++ b/src/backend/optimizer/util/var.c
@@ -88,6 +88,9 @@ static Relids alias_relid_set(Query *query, Relids relids);
* Create a set of all the distinct varnos present in a parsetree.
* Only varnos that reference level-zero rtable entries are considered.
*
+ * The result includes outer-join relids mentioned in Var.varnullingrels and
+ * PlaceHolderVar.phnullingrels fields in the parsetree.
+ *
* "root" can be passed as NULL if it is not necessary to process
* PlaceHolderVars.
*
@@ -153,7 +156,11 @@ pull_varnos_walker(Node *node, pull_varnos_context *context)
Var *var = (Var *) node;
if (var->varlevelsup == context->sublevels_up)
+ {
context->varnos = bms_add_member(context->varnos, var->varno);
+ context->varnos = bms_add_members(context->varnos,
+ var->varnullingrels);
+ }
return false;
}
if (IsA(node, CurrentOfExpr))
@@ -251,6 +258,14 @@ pull_varnos_walker(Node *node, pull_varnos_context *context)
context->varnos = bms_join(context->varnos,
newevalat);
}
+
+ /*
+ * In all three cases, include phnullingrels in the result. We
+ * don't worry about possibly needing to translate it, because
+ * appendrels only translate varnos of baserels, not outer joins.
+ */
+ context->varnos = bms_add_members(context->varnos,
+ phv->phnullingrels);
return false; /* don't recurse into expression */
}
}
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index d35e5605de..62178ba3b4 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -2204,7 +2204,7 @@ rowcomparesel(PlannerInfo *root,
else
{
/*
- * Otherwise, it's a join if there's more than one relation used.
+ * Otherwise, it's a join if there's more than one base relation used.
*/
is_join_clause = (NumRelids(root, (Node *) opargs) > 1);
}
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 98302ff393..10b89df6fe 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -243,13 +243,26 @@ struct PlannerInfo
struct AppendRelInfo **append_rel_array pg_node_attr(read_write_ignore);
/*
- * all_baserels is a Relids set of all base relids (but not "other"
- * relids) in the query; that is, the Relids identifier of the final join
- * we need to form. This is computed in make_one_rel, just before we
- * start making Paths.
+ * all_baserels is a Relids set of all base relids (but not joins or
+ * "other" relids) in the query. This is computed in make_one_rel, just
+ * before we start making Paths.
*/
Relids all_baserels;
+ /*
+ * outer_join_rels is a Relids set of all outer-join relids in the query.
+ * This is computed in deconstruct_jointree.
+ */
+ Relids outer_join_rels;
+
+ /*
+ * all_query_rels is a Relids set of all base relids and outer join relids
+ * (but not "other" relids) in the query. This is the Relids identifier
+ * of the final join we need to form. This is computed in make_one_rel,
+ * just before we start making Paths.
+ */
+ Relids all_query_rels;
+
/*
* nullable_baserels is a Relids set of base relids that are nullable by
* some outer join in the jointree; these are rels that are potentially
@@ -319,7 +332,7 @@ struct PlannerInfo
List *right_join_clauses;
/*
- * list of RestrictInfos for mergejoinable full join clauses
+ * list of FullJoinClauseInfos for mergejoinable full join clauses
*/
List *full_join_clauses;
@@ -540,9 +553,10 @@ typedef struct PartitionSchemeData *PartitionScheme;
* or the output of a sub-SELECT or function that appears in the range table.
* In either case it is uniquely identified by an RT index. A "joinrel"
* is the joining of two or more base rels. A joinrel is identified by
- * the set of RT indexes for its component baserels. We create RelOptInfo
- * nodes for each baserel and joinrel, and store them in the PlannerInfo's
- * simple_rel_array and join_rel_list respectively.
+ * the set of RT indexes for its component baserels, along with RT indexes
+ * for any outer joins it has computed. We create RelOptInfo nodes for each
+ * baserel and joinrel, and store them in the PlannerInfo's simple_rel_array
+ * and join_rel_list respectively.
*
* Note that there is only one joinrel for any given set of component
* baserels, no matter what order we assemble them in; so an unordered
@@ -581,8 +595,10 @@ typedef struct PartitionSchemeData *PartitionScheme;
* Parts of this data structure are specific to various scan and join
* mechanisms. It didn't seem worth creating new node types for them.
*
- * relids - Set of base-relation identifiers; it is a base relation
- * if there is just one, a join relation if more than one
+ * relids - Set of relation identifiers (RT indexes). This is a base
+ * relation if there is just one, a join relation if more;
+ * in the join case, RT indexes of any outer joins formed
+ * at or below this join are included along with baserels
* rows - estimated number of tuples in the relation after restriction
* clauses have been applied (ie, output rows of a plan for it)
* consider_startup - true if there is any value in keeping plain paths for
@@ -794,7 +810,7 @@ typedef struct RelOptInfo
RelOptKind reloptkind;
/*
- * all relations included in this RelOptInfo; set of base relids
+ * all relations included in this RelOptInfo; set of base + OJ relids
* (rangetable indexes)
*/
Relids relids;
@@ -2278,17 +2294,17 @@ typedef struct LimitPath
* If a restriction clause references a single base relation, it will appear
* in the baserestrictinfo list of the RelOptInfo for that base rel.
*
- * If a restriction clause references more than one base rel, it will
+ * If a restriction clause references more than one base+OJ relation, it will
* appear in the joininfo list of every RelOptInfo that describes a strict
- * subset of the base rels mentioned in the clause. The joininfo lists are
+ * subset of the relations mentioned in the clause. The joininfo lists are
* used to drive join tree building by selecting plausible join candidates.
* The clause cannot actually be applied until we have built a join rel
- * containing all the base rels it references, however.
+ * containing all the relations it references, however.
*
- * When we construct a join rel that includes all the base rels referenced
+ * When we construct a join rel that includes all the relations referenced
* in a multi-relation restriction clause, we place that clause into the
* joinrestrictinfo lists of paths for the join rel, if neither left nor
- * right sub-path includes all base rels referenced in the clause. The clause
+ * right sub-path includes all relations referenced in the clause. The clause
* will be applied at that join level, and will not propagate any further up
* the join tree. (Note: the "predicate migration" code was once intended to
* push restriction clauses up and down the plan tree based on evaluation
@@ -2309,12 +2325,15 @@ typedef struct LimitPath
* or join to enforce that all members of each EquivalenceClass are in fact
* equal in all rows emitted by the scan or join.
*
- * When dealing with outer joins we have to be very careful about pushing qual
- * clauses up and down the tree. An outer join's own JOIN/ON conditions must
- * be evaluated exactly at that join node, unless they are "degenerate"
- * conditions that reference only Vars from the nullable side of the join.
- * Quals appearing in WHERE or in a JOIN above the outer join cannot be pushed
- * down below the outer join, if they reference any nullable Vars.
+ * The clause_relids field lists the base plus outer-join RT indexes that
+ * actually appear in the clause. required_relids lists the minimum set of
+ * relids needed to evaluate the clause; while this is often equal to
+ * clause_relids, it can be more. We will add relids to required_relids when
+ * we need to force an outer join ON clause to be evaluated exactly at the
+ * level of the outer join, which is true except when it is a "degenerate"
+ * condition that references only Vars from the nullable side of the join.
+ *
+ * XXX rewrite or remove me:
* RestrictInfo nodes contain a flag to indicate whether a qual has been
* pushed down to a lower level than its original syntactic placement in the
* join tree would suggest. If an outer join prevents us from pushing a qual
@@ -2436,13 +2455,16 @@ typedef struct RestrictInfo
/* true if known to contain no leaked Vars */
bool leakproof pg_node_attr(equal_ignore);
- /* to indicate if clause contains any volatile functions. */
+ /* indicates if clause contains any volatile functions */
VolatileFunctionStatus has_volatile pg_node_attr(equal_ignore);
/* see comment above */
Index security_level;
- /* The set of relids (varnos) actually referenced in the clause: */
+ /* number of base rels in clause_relids */
+ int num_base_rels pg_node_attr(equal_ignore);
+
+ /* The relids (varnos+varnullingrels) actually referenced in the clause: */
Relids clause_relids pg_node_attr(equal_ignore);
/* The set of relids required to evaluate the clause: */
@@ -2544,6 +2566,7 @@ typedef struct RestrictInfo
} RestrictInfo;
/*
+ * XXX this will need work:
* This macro embodies the correct way to test whether a RestrictInfo is
* "pushed down" to a given outer join, that is, should be treated as a filter
* clause rather than a join clause at that outer join. This is certainly so
@@ -2646,17 +2669,20 @@ typedef struct PlaceHolderVar
* We make SpecialJoinInfos for FULL JOINs even though there is no flexibility
* of planning for them, because this simplifies make_join_rel()'s API.
*
- * min_lefthand and min_righthand are the sets of base relids that must be
- * available on each side when performing the special join. lhs_strict is
- * true if the special join's condition cannot succeed when the LHS variables
- * are all NULL (this means that an outer join can commute with upper-level
+ * min_lefthand and min_righthand are the sets of base+OJ relids that must be
+ * available on each side when performing the special join.
+ *
+ * strict_relids is the set of base+OJ relids for which the special join's
+ * condition is strict, ie it cannot succeed if any of those rels produce
+ * an all-NULL row. lhs_strict reports whether any LHS rels appear in
+ * strict_relids (this means that an outer join can commute with upper-level
* outer joins even if it appears in their RHS). We don't bother to set
- * lhs_strict for FULL JOINs, however.
+ * strict_relids or lhs_strict for FULL JOINs, however.
*
* It is not valid for either min_lefthand or min_righthand to be empty sets;
* if they were, this would break the logic that enforces join order.
*
- * syn_lefthand and syn_righthand are the sets of base relids that are
+ * syn_lefthand and syn_righthand are the sets of base+OJ relids that are
* syntactically below this special join. (These are needed to help compute
* min_lefthand and min_righthand for higher joins.)
*
@@ -2678,14 +2704,18 @@ typedef struct PlaceHolderVar
* the inputs to make it a LEFT JOIN. So the allowed values of jointype
* in a join_info_list member are only LEFT, FULL, SEMI, or ANTI.
*
+ * ojrelid is the RT index of the join RTE representing this outer join,
+ * if there is one. It is zero when jointype is INNER or SEMI.
+ *
* For purposes of join selectivity estimation, we create transient
* SpecialJoinInfo structures for regular inner joins; so it is possible
* to have jointype == JOIN_INNER in such a structure, even though this is
* not allowed within join_info_list. We also create transient
* SpecialJoinInfos with jointype == JOIN_INNER for outer joins, since for
* cost estimation purposes it is sometimes useful to know the join size under
- * plain innerjoin semantics. Note that lhs_strict, delay_upper_joins, and
- * of course the semi_xxx fields are not set meaningfully within such structs.
+ * plain innerjoin semantics. Note that strict_relids, lhs_strict,
+ * delay_upper_joins, and of course the semi_xxx fields are not set
+ * meaningfully within such structs.
*/
#ifndef HAVE_SPECIALJOININFO_TYPEDEF
typedef struct SpecialJoinInfo SpecialJoinInfo;
@@ -2697,11 +2727,13 @@ struct SpecialJoinInfo
pg_node_attr(no_read)
NodeTag type;
- Relids min_lefthand; /* base relids in minimum LHS for join */
- Relids min_righthand; /* base relids in minimum RHS for join */
- Relids syn_lefthand; /* base relids syntactically within LHS */
- Relids syn_righthand; /* base relids syntactically within RHS */
+ Relids min_lefthand; /* base+OJ relids in minimum LHS for join */
+ Relids min_righthand; /* base+OJ relids in minimum RHS for join */
+ Relids syn_lefthand; /* base+OJ relids syntactically within LHS */
+ Relids syn_righthand; /* base+OJ relids syntactically within RHS */
JoinType jointype; /* always INNER, LEFT, FULL, SEMI, or ANTI */
+ Index ojrelid; /* outer join's RT index; 0 if none */
+ Relids strict_relids; /* joinclause is strict for these relids */
bool lhs_strict; /* joinclause is strict for some LHS rel */
bool delay_upper_joins; /* can't commute with upper RHS */
/* Remaining fields are set only for JOIN_SEMI jointype: */
@@ -2711,6 +2743,21 @@ struct SpecialJoinInfo
List *semi_rhs_exprs; /* righthand-side expressions of these ops */
};
+/*
+ * FULL JOIN clause info.
+ *
+ * We set aside every FULL JOIN ON clause that looks mergejoinable, and
+ * process it specially at the end of qual distribution.
+ */
+typedef struct FullJoinClauseInfo
+{
+ pg_node_attr(no_copy_equal, no_read)
+
+ NodeTag type;
+ RestrictInfo *rinfo; /* a mergejoinable FULL JOIN clause */
+ SpecialJoinInfo *sjinfo; /* the FULL JOIN's SpecialJoinInfo */
+} FullJoinClauseInfo;
+
/*
* Append-relation info.
*
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 050f00e79a..197234d44c 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -304,6 +304,7 @@ extern void expand_planner_arrays(PlannerInfo *root, int add_size);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
+extern RelOptInfo *find_base_rel_ignore_join(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
@@ -335,6 +336,6 @@ extern ParamPathInfo *find_param_path_info(RelOptInfo *rel,
extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
- SpecialJoinInfo *sjinfo, JoinType jointype);
+ SpecialJoinInfo *sjinfo);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/placeholder.h b/src/include/optimizer/placeholder.h
index 39803ea41f..34b118a5c9 100644
--- a/src/include/optimizer/placeholder.h
+++ b/src/include/optimizer/placeholder.h
@@ -27,6 +27,7 @@ extern void update_placeholder_eval_levels(PlannerInfo *root,
extern void fix_placeholder_input_needed_levels(PlannerInfo *root);
extern void add_placeholders_to_base_rels(PlannerInfo *root);
extern void add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *outer_rel, RelOptInfo *inner_rel);
+ RelOptInfo *outer_rel, RelOptInfo *inner_rel,
+ SpecialJoinInfo *sjinfo);
#endif /* PLACEHOLDER_H */
diff --git a/src/include/optimizer/prep.h b/src/include/optimizer/prep.h
index 2b11ff1d1f..ca03f32174 100644
--- a/src/include/optimizer/prep.h
+++ b/src/include/optimizer/prep.h
@@ -29,7 +29,8 @@ extern void pull_up_subqueries(PlannerInfo *root);
extern void flatten_simple_union_all(PlannerInfo *root);
extern void reduce_outer_joins(PlannerInfo *root);
extern void remove_useless_result_rtes(PlannerInfo *root);
-extern Relids get_relids_in_jointree(Node *jtnode, bool include_joins);
+extern Relids get_relids_in_jointree(Node *jtnode, bool include_outer_joins,
+ bool include_inner_joins);
extern Relids get_relids_for_join(Query *query, int joinrelid);
/*
commit db3cdf0c4034dd153534bf0b63cfbd8978b00954
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Aug 1 15:35:07 2022 -0400
Fix flatten_join_alias_vars() to handle varnullingrels correctly.
The remaining core regression test failures occur because
flatten_join_alias_vars() isn't doing the right thing. The
alias Var it needs to replace may have acquired varnullingrels
bits signifying the effect of upper outer joins, and if so we
must preserve that information in the replacement expression.
The simplest way to do that is to wrap the replacement expression
in a PlaceHolderVar, and that's what we have to do in the general
case where subquery pullup has mutated the replacement joinaliasvars
entry into an arbitrary expression. But in simpler cases, such as
where the joinaliasvars entry is just a Var, we'd prefer to do it
by merging the alias Var's varnullingrels into the replacement Var.
In that way the flattened alias will compare equal() to semantically
equivalent references that didn't use the alias name.
Moreover, the parser also uses this code while checking certain
semantic constraints, and in that context we *must not* generate
PlaceHolderVars. PHVs shouldn't appear in parse-time expressions,
and adding one would certainly cause the parser to decide the
query is invalid (because the result wouldn't compare equal() to
what it needs to). Fortunately, during parsing the set of possible
contents of a joinaliasvars entry is quite constrained, so we can
guarantee to apply the nullingrels info to the Vars therein.
The result of this step passes all core regression tests, but there
are still loose ends for FDWs (so that contrib/postgres_fdw will fail).
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index ea077cff4e..127f06fb04 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -898,7 +898,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
*/
if (rte->lateral && root->hasJoinRTEs)
rte->subquery = (Query *)
- flatten_join_alias_vars(root->parse,
+ flatten_join_alias_vars(root, root->parse,
(Node *) rte->subquery);
}
else if (rte->rtekind == RTE_FUNCTION)
@@ -1099,7 +1099,7 @@ preprocess_expression(PlannerInfo *root, Node *expr, int kind)
kind == EXPRKIND_VALUES ||
kind == EXPRKIND_TABLESAMPLE ||
kind == EXPRKIND_TABLEFUNC))
- expr = flatten_join_alias_vars(root->parse, expr);
+ expr = flatten_join_alias_vars(root, root->parse, expr);
/*
* Simplify constant expressions. For function RTEs, this was already
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 389f7d9ce7..7b22254173 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -1076,7 +1076,8 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* maybe even in the rewriter; but for now let's just fix this case here.)
*/
subquery->targetList = (List *)
- flatten_join_alias_vars(subroot->parse, (Node *) subquery->targetList);
+ flatten_join_alias_vars(subroot, subroot->parse,
+ (Node *) subquery->targetList);
/*
* Adjust level-0 varnos in subquery so that we can append its rangetable
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
index 99126958db..81fc3002fe 100644
--- a/src/backend/optimizer/util/var.c
+++ b/src/backend/optimizer/util/var.c
@@ -62,6 +62,7 @@ typedef struct
typedef struct
{
+ PlannerInfo *root; /* could be NULL! */
Query *query; /* outer Query */
int sublevels_up;
bool possible_sublink; /* could aliases include a SubLink? */
@@ -80,6 +81,10 @@ static bool pull_var_clause_walker(Node *node,
pull_var_clause_context *context);
static Node *flatten_join_alias_vars_mutator(Node *node,
flatten_join_alias_vars_context *context);
+static Node *add_nullingrels_if_needed(PlannerInfo *root, Node *newnode,
+ Var *oldvar);
+static bool is_standard_join_alias_expression(Node *newnode, Var *oldvar);
+static void adjust_standard_join_alias_expression(Node *newnode, Var *oldvar);
static Relids alias_relid_set(Query *query, Relids relids);
@@ -729,26 +734,42 @@ pull_var_clause_walker(Node *node, pull_var_clause_context *context)
* is the only way that the executor can directly handle whole-row Vars.
*
* This also adjusts relid sets found in some expression node types to
- * substitute the contained base rels for any join relid.
+ * substitute the contained base+OJ rels for any join relid.
*
* If a JOIN contains sub-selects that have been flattened, its join alias
* entries might now be arbitrary expressions, not just Vars. This affects
- * this function in one important way: we might find ourselves inserting
- * SubLink expressions into subqueries, and we must make sure that their
- * Query.hasSubLinks fields get set to true if so. If there are any
+ * this function in two important ways. First, we might find ourselves
+ * inserting SubLink expressions into subqueries, and we must make sure that
+ * their Query.hasSubLinks fields get set to true if so. If there are any
* SubLinks in the join alias lists, the outer Query should already have
* hasSubLinks = true, so this is only relevant to un-flattened subqueries.
+ * Second, we have to preserve any varnullingrels info attached to the
+ * alias Vars we're replacing. If the replacement expression is a Var or
+ * PlaceHolderVar or constructed from those, we can just add the
+ * varnullingrels bits to the existing nullingrels field(s); otherwise
+ * we have to add a PlaceHolderVar wrapper.
*
- * NOTE: this is used on not-yet-planned expressions. We do not expect it
- * to be applied directly to the whole Query, so if we see a Query to start
- * with, we do want to increment sublevels_up (this occurs for LATERAL
- * subqueries).
+ * NOTE: this is also used by the parser, to expand join alias Vars before
+ * checking GROUP BY validity. For that use-case, root will be NULL, which
+ * is why we have to pass the Query separately. We need the root itself only
+ * for making PlaceHolderVars. We can avoid making PlaceHolderVars in the
+ * parser's usage because it won't be dealing with arbitrary expressions:
+ * so long as adjust_standard_join_alias_expression can handle everything
+ * the parser would make as a join alias expression, we're OK.
*/
Node *
-flatten_join_alias_vars(Query *query, Node *node)
+flatten_join_alias_vars(PlannerInfo *root, Query *query, Node *node)
{
flatten_join_alias_vars_context context;
+ /*
+ * We do not expect this to be applied to the whole Query, only to
+ * expressions or LATERAL subqueries. Hence, if the top node is a Query,
+ * it's okay to immediately increment sublevels_up.
+ */
+ Assert(node != (Node *) query);
+
+ context.root = root;
context.query = query;
context.sublevels_up = 0;
/* flag whether join aliases could possibly contain SubLinks */
@@ -819,7 +840,9 @@ flatten_join_alias_vars_mutator(Node *node,
rowexpr->colnames = colnames;
rowexpr->location = var->location;
- return (Node *) rowexpr;
+ /* Lastly, add any varnullingrels to the replacement expression */
+ return add_nullingrels_if_needed(context->root, (Node *) rowexpr,
+ var);
}
/* Expand join alias reference */
@@ -846,7 +869,8 @@ flatten_join_alias_vars_mutator(Node *node,
if (context->possible_sublink && !context->inserted_sublink)
context->inserted_sublink = checkExprHasSubLink(newvar);
- return newvar;
+ /* Lastly, add any varnullingrels to the replacement expression */
+ return add_nullingrels_if_needed(context->root, newvar, var);
}
if (IsA(node, PlaceHolderVar))
{
@@ -861,6 +885,7 @@ flatten_join_alias_vars_mutator(Node *node,
{
phv->phrels = alias_relid_set(context->query,
phv->phrels);
+ /* we *don't* change phnullingrels */
}
return (Node *) phv;
}
@@ -894,9 +919,145 @@ flatten_join_alias_vars_mutator(Node *node,
(void *) context);
}
+/*
+ * Add oldvar's varnullingrels, if any, to a flattened join alias expression.
+ * The newnode has been copied, so we can modify it freely.
+ */
+static Node *
+add_nullingrels_if_needed(PlannerInfo *root, Node *newnode, Var *oldvar)
+{
+ if (oldvar->varnullingrels == NULL)
+ return newnode; /* nothing to do */
+ /* If possible, do it by adding to existing nullingrel fields */
+ if (is_standard_join_alias_expression(newnode, oldvar))
+ adjust_standard_join_alias_expression(newnode, oldvar);
+ else if (root)
+ {
+ /* We can insert a PlaceHolderVar to carry the nullingrels */
+ PlaceHolderVar *newphv;
+ Relids phrels = pull_varnos(root, newnode);
+
+ /* XXX what if phrels is empty? */
+ Assert(!bms_is_empty(phrels)); /* probably wrong */
+ newphv = make_placeholder_expr(root, (Expr *) newnode, phrels);
+ /* newphv has zero phlevelsup and NULL phnullingrels; fix it */
+ newphv->phlevelsup = oldvar->varlevelsup;
+ newphv->phnullingrels = bms_copy(oldvar->varnullingrels);
+ newnode = (Node *) newphv;
+ }
+ else
+ {
+ /* ooops, we're missing support for something the parser can make */
+ elog(ERROR, "unsupported join alias expression");
+ }
+ return newnode;
+}
+
+/*
+ * Check to see if we can insert nullingrels into this join alias expression
+ * without use of a separate PlaceHolderVar.
+ *
+ * This will handle Vars, PlaceHolderVars, and implicit-coercion and COALESCE
+ * expressions built from those. This coverage needs to handle anything
+ * that the parser would put into joinaliasvars.
+ * XXX it's probably incomplete at the moment.
+ */
+static bool
+is_standard_join_alias_expression(Node *newnode, Var *oldvar)
+{
+ if (newnode == NULL)
+ return false;
+ if (IsA(newnode, Var) &&
+ ((Var *) newnode)->varlevelsup == oldvar->varlevelsup)
+ return true;
+ else if (IsA(newnode, PlaceHolderVar) &&
+ ((PlaceHolderVar *) newnode)->phlevelsup == oldvar->varlevelsup)
+ return true;
+ else if (IsA(newnode, FuncExpr))
+ {
+ FuncExpr *fexpr = (FuncExpr *) newnode;
+
+ /*
+ * We need to assume that the function wouldn't produce non-NULL from
+ * NULL, which is reasonable for implicit coercions but otherwise not
+ * so much. (Looking at its strictness is likely overkill, and anyway
+ * it would cause us to fail if someone forgot to mark an implicit
+ * coercion as strict.)
+ */
+ if (fexpr->funcformat != COERCE_IMPLICIT_CAST ||
+ fexpr->args == NIL)
+ return false;
+
+ /*
+ * Examine only the first argument --- coercions might have additional
+ * arguments that are constants.
+ */
+ return is_standard_join_alias_expression(linitial(fexpr->args), oldvar);
+ }
+ else if (IsA(newnode, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) newnode;
+ ListCell *lc;
+
+ Assert(cexpr->args != NIL);
+ foreach(lc, cexpr->args)
+ {
+ if (!is_standard_join_alias_expression(lfirst(lc), oldvar))
+ return false;
+ }
+ return true;
+ }
+ else
+ return false;
+}
+
+/*
+ * Insert nullingrels into an expression accepted by
+ * is_standard_join_alias_expression.
+ */
+static void
+adjust_standard_join_alias_expression(Node *newnode, Var *oldvar)
+{
+ if (IsA(newnode, Var) &&
+ ((Var *) newnode)->varlevelsup == oldvar->varlevelsup)
+ {
+ Var *newvar = (Var *) newnode;
+
+ newvar->varnullingrels = bms_add_members(newvar->varnullingrels,
+ oldvar->varnullingrels);
+ }
+ else if (IsA(newnode, PlaceHolderVar) &&
+ ((PlaceHolderVar *) newnode)->phlevelsup == oldvar->varlevelsup)
+ {
+ PlaceHolderVar *newphv = (PlaceHolderVar *) newnode;
+
+ newphv->phnullingrels = bms_add_members(newphv->phnullingrels,
+ oldvar->varnullingrels);
+ }
+ else if (IsA(newnode, FuncExpr))
+ {
+ FuncExpr *fexpr = (FuncExpr *) newnode;
+
+ adjust_standard_join_alias_expression(linitial(fexpr->args), oldvar);
+ }
+ else if (IsA(newnode, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) newnode;
+ ListCell *lc;
+
+ Assert(cexpr->args != NIL);
+ foreach(lc, cexpr->args)
+ {
+ adjust_standard_join_alias_expression(lfirst(lc), oldvar);
+ }
+ }
+ else
+ Assert(false);
+}
+
/*
* alias_relid_set: in a set of RT indexes, replace joins by their
- * underlying base relids
+ * underlying base+OJ relids
*/
static Relids
alias_relid_set(Query *query, Relids relids)
diff --git a/src/backend/parser/parse_agg.c b/src/backend/parser/parse_agg.c
index 3ef9e8ee5e..c15fab0f68 100644
--- a/src/backend/parser/parse_agg.c
+++ b/src/backend/parser/parse_agg.c
@@ -1162,7 +1162,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
* entries are RTE_JOIN kind.
*/
if (hasJoinRTEs)
- groupClauses = (List *) flatten_join_alias_vars(qry,
+ groupClauses = (List *) flatten_join_alias_vars(NULL, qry,
(Node *) groupClauses);
/*
@@ -1206,7 +1206,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
groupClauses, hasJoinRTEs,
have_non_var_grouping);
if (hasJoinRTEs)
- clause = flatten_join_alias_vars(qry, clause);
+ clause = flatten_join_alias_vars(NULL, qry, clause);
check_ungrouped_columns(clause, pstate, qry,
groupClauses, groupClauseCommonVars,
have_non_var_grouping,
@@ -1217,7 +1217,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
groupClauses, hasJoinRTEs,
have_non_var_grouping);
if (hasJoinRTEs)
- clause = flatten_join_alias_vars(qry, clause);
+ clause = flatten_join_alias_vars(NULL, qry, clause);
check_ungrouped_columns(clause, pstate, qry,
groupClauses, groupClauseCommonVars,
have_non_var_grouping,
@@ -1546,7 +1546,7 @@ finalize_grouping_exprs_walker(Node *node,
Index ref = 0;
if (context->hasJoinRTEs)
- expr = flatten_join_alias_vars(context->qry, expr);
+ expr = flatten_join_alias_vars(NULL, context->qry, expr);
/*
* Each expression must match a grouping entry at the current
diff --git a/src/include/optimizer/optimizer.h b/src/include/optimizer/optimizer.h
index 7be1e5906b..1f5e0b24ca 100644
--- a/src/include/optimizer/optimizer.h
+++ b/src/include/optimizer/optimizer.h
@@ -202,6 +202,6 @@ extern bool contain_var_clause(Node *node);
extern bool contain_vars_of_level(Node *node, int levelsup);
extern int locate_var_of_level(Node *node, int levelsup);
extern List *pull_var_clause(Node *node, int flags);
-extern Node *flatten_join_alias_vars(Query *query, Node *node);
+extern Node *flatten_join_alias_vars(PlannerInfo *root, Query *query, Node *node);
#endif /* OPTIMIZER_H */
commit 9741326c32bebe41fcb1febbcc9289328eec2dbb
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Aug 1 15:41:56 2022 -0400
Teach FDWs about base-plus-outer-join relids.
Conversion of the planner to include OJ relids in join relids
affects FDWs that want to plan foreign joins. They *must* follow
suit when labeling foreign joins in order to match with the core
planner, but for many purposes (if postgres_fdw is any guide)
they'd prefer to consider only base relations within the join.
To support both requirements, redefine ForeignScan.fs_relids as
base+OJ relids, and add a new field fs_base_relids that's set up
by the core planner.
Another way we could do this is to keep fs_relids as just base
relids and make the new field be the one with OJ relids added.
While that would be more backwards-compatible in some sense,
it would be inconsistent with the naming used in the core planner,
and I think that it might allow some types of bugs to escape
immediate detection.
postgres_fdw also has one place where it needs to ignore varnullingrels
while matching Vars, similarly to the unfinished work in setrefs.c.
(That requirement will only affect join-planning FDWs, too, since
Vars seen at a base relation scan should never have any varnullingrels.)
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
index a9766f9734..a187cd08fa 100644
--- a/contrib/postgres_fdw/deparse.c
+++ b/contrib/postgres_fdw/deparse.c
@@ -3952,7 +3952,17 @@ get_relation_column_alias_ids(Var *node, RelOptInfo *foreignrel,
i = 1;
foreach(lc, foreignrel->reltarget->exprs)
{
- if (equal(lfirst(lc), (Node *) node))
+ Var *tlvar = (Var *) lfirst(lc);
+
+ /*
+ * As in setrefs.c, we match only on varno/varattno. Ideally there
+ * would be some cross-check on varnullingrels, but it's unclear what
+ * to do exactly; we don't have enough context to know what that value
+ * should be.
+ */
+ if (IsA(tlvar, Var) &&
+ tlvar->varno == node->varno &&
+ tlvar->varattno == node->varattno)
{
*colno = i;
return;
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 048db542d3..08d5042ffc 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -1513,13 +1513,13 @@ postgresBeginForeignScan(ForeignScanState *node, int eflags)
/*
* Identify which user to do the remote access as. This should match what
* ExecCheckRTEPerms() does. In case of a join or aggregate, use the
- * lowest-numbered member RTE as a representative; we would get the same
- * result from any.
+ * lowest-numbered member base RTE as a representative; we would get the
+ * same result from any.
*/
if (fsplan->scan.scanrelid > 0)
rtindex = fsplan->scan.scanrelid;
else
- rtindex = bms_next_member(fsplan->fs_relids, -1);
+ rtindex = bms_next_member(fsplan->fs_base_relids, -1);
rte = exec_rt_fetch(rtindex, estate);
userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
@@ -2405,7 +2405,7 @@ find_modifytable_subplan(PlannerInfo *root,
{
ForeignScan *fscan = (ForeignScan *) subplan;
- if (bms_is_member(rtindex, fscan->fs_relids))
+ if (bms_is_member(rtindex, fscan->fs_base_relids))
return fscan;
}
@@ -2831,8 +2831,8 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
* that setrefs.c won't update the string when flattening the
* rangetable. To find out what rtoffset was applied, identify the
* minimum RT index appearing in the string and compare it to the
- * minimum member of plan->fs_relids. (We expect all the relids in
- * the join will have been offset by the same amount; the Asserts
+ * minimum member of plan->fs_base_relids. (We expect all the relids
+ * in the join will have been offset by the same amount; the Asserts
* below should catch it if that ever changes.)
*/
minrti = INT_MAX;
@@ -2849,7 +2849,7 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
else
ptr++;
}
- rtoffset = bms_next_member(plan->fs_relids, -1) - minrti;
+ rtoffset = bms_next_member(plan->fs_base_relids, -1) - minrti;
/* Now we can translate the string */
relations = makeStringInfo();
@@ -2864,7 +2864,7 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
char *refname;
rti += rtoffset;
- Assert(bms_is_member(rti, plan->fs_relids));
+ Assert(bms_is_member(rti, plan->fs_base_relids));
rte = rt_fetch(rti, es->rtable);
Assert(rte->rtekind == RTE_RELATION);
/* This logic should agree with explain.c's ExplainTargetRel */
diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml
index d0b5951019..329affa30b 100644
--- a/doc/src/sgml/fdwhandler.sgml
+++ b/doc/src/sgml/fdwhandler.sgml
@@ -351,6 +351,17 @@ GetForeignJoinPaths(PlannerInfo *root,
it will supply at run time in the tuples it returns.
</para>
+ <note>
+ <para>
+ Beginning with <productname>PostgreSQL</productname> 16,
+ <structfield>fs_relids</structfield> includes the rangetable indexes
+ of outer joins, if any were involved in this join. The new field
+ <structfield>fs_base_relids</structfield> includes only base
+ relation indexes, and thus
+ mimics <structfield>fs_relids</structfield>'s old semantics.
+ </para>
+ </note>
+
<para>
See <xref linkend="fdw-planning"/> for additional information.
</para>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index e29c2ae206..ded37cb2e9 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1114,7 +1114,7 @@ ExplainPreScanNode(PlanState *planstate, Bitmapset **rels_used)
break;
case T_ForeignScan:
*rels_used = bms_add_members(*rels_used,
- ((ForeignScan *) plan)->fs_relids);
+ ((ForeignScan *) plan)->fs_base_relids);
break;
case T_CustomScan:
*rels_used = bms_add_members(*rels_used,
diff --git a/src/backend/executor/execScan.c b/src/backend/executor/execScan.c
index 043bb83f55..2b37266b6a 100644
--- a/src/backend/executor/execScan.c
+++ b/src/backend/executor/execScan.c
@@ -325,7 +325,7 @@ ExecScanReScan(ScanState *node)
* all of them.
*/
if (IsA(node->ps.plan, ForeignScan))
- relids = ((ForeignScan *) node->ps.plan)->fs_relids;
+ relids = ((ForeignScan *) node->ps.plan)->fs_base_relids;
else if (IsA(node->ps.plan, CustomScan))
relids = ((CustomScan *) node->ps.plan)->custom_relids;
else
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index e37f2933eb..bd9af88d55 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -29,6 +29,7 @@
#include "optimizer/cost.h"
#include "optimizer/optimizer.h"
#include "optimizer/paramassign.h"
+#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/placeholder.h"
#include "optimizer/plancat.h"
@@ -4105,6 +4106,8 @@ create_foreignscan_plan(PlannerInfo *root, ForeignPath *best_path,
Index scan_relid = rel->relid;
Oid rel_oid = InvalidOid;
Plan *outer_plan = NULL;
+ Relids fs_base_relids;
+ int rtindex;
Assert(rel->fdwroutine != NULL);
@@ -4153,14 +4156,28 @@ create_foreignscan_plan(PlannerInfo *root, ForeignPath *best_path,
/*
* Likewise, copy the relids that are represented by this foreign scan. An
- * upper rel doesn't have relids set, but it covers all the base relations
- * participating in the underlying scan, so use root's all_baserels.
+ * upper rel doesn't have relids set, but it covers all the relations
+ * participating in the underlying scan/join, so use root->all_query_rels.
*/
if (rel->reloptkind == RELOPT_UPPER_REL)
- scan_plan->fs_relids = root->all_baserels;
+ scan_plan->fs_relids = root->all_query_rels;
else
scan_plan->fs_relids = best_path->path.parent->relids;
+ /*
+ * Join relid sets include relevant outer joins, but FDWs may need to know
+ * which are the included base rels. That's a bit tedious to get without
+ * access to the plan-time data structures, so compute it here.
+ */
+ fs_base_relids = NULL;
+ rtindex = -1;
+ while ((rtindex = bms_next_member(scan_plan->fs_relids, rtindex)) >= 0)
+ {
+ if (find_base_rel_ignore_join(root, rtindex) != NULL)
+ fs_base_relids = bms_add_member(fs_base_relids, rtindex);
+ }
+ scan_plan->fs_base_relids = fs_base_relids;
+
/*
* If this is a foreign join, and to make it valid to push down we had to
* assume that the current user is the same as some user explicitly named
@@ -5805,8 +5822,9 @@ make_foreignscan(List *qptlist,
node->fdw_private = fdw_private;
node->fdw_scan_tlist = fdw_scan_tlist;
node->fdw_recheck_quals = fdw_recheck_quals;
- /* fs_relids will be filled in by create_foreignscan_plan */
+ /* fs_relids, fs_base_relids will be filled by create_foreignscan_plan */
node->fs_relids = NULL;
+ node->fs_base_relids = NULL;
/* fsSystemCol will be filled in by create_foreignscan_plan */
node->fsSystemCol = false;
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index a1827d113d..1a047ddda2 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1533,6 +1533,7 @@ set_foreignscan_references(PlannerInfo *root,
}
fscan->fs_relids = offset_relid_set(fscan->fs_relids, rtoffset);
+ fscan->fs_base_relids = offset_relid_set(fscan->fs_base_relids, rtoffset);
/* Adjust resultRelation if it's valid */
if (fscan->resultRelation > 0)
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index dca2a21e7a..7e98d0b7a3 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -688,6 +688,7 @@ typedef struct WorkTableScan
* When the plan node represents a foreign join, scan.scanrelid is zero and
* fs_relids must be consulted to identify the join relation. (fs_relids
* is valid for simple scans as well, but will always match scan.scanrelid.)
+ * fs_relids includes outer joins; fs_base_relids does not.
*
* If the FDW's PlanDirectModify() callback decides to repurpose a ForeignScan
* node to perform the UPDATE or DELETE operation directly in the remote
@@ -707,7 +708,8 @@ typedef struct ForeignScan
List *fdw_private; /* private data for FDW */
List *fdw_scan_tlist; /* optional tlist describing scan tuple */
List *fdw_recheck_quals; /* original quals not in scan.plan.qual */
- Bitmapset *fs_relids; /* RTIs generated by this scan */
+ Bitmapset *fs_relids; /* base+OJ RTIs generated by this scan */
+ Bitmapset *fs_base_relids; /* base RTIs generated by this scan */
bool fsSystemCol; /* true if any "system column" is needed */
} ForeignScan;
Zhihong Yu <zyu@yugabyte.com> writes:
> For v3-0003-label-Var-nullability-in-parser.patch :
> + if (rtindex > 0 && rtindex <= list_length(pstate->p_nullingrels))
> + relids = (Bitmapset *) list_nth(pstate->p_nullingrels, rtindex - 1);
> + else
> + relids = NULL;
> +
> + /*
> + * Merge with any already-declared nulling rels. (Typically there won't
> + * be any, but let's get it right if there are.)
> + */
> + if (relids != NULL)
> It seems the last if block can be merged into the previous if block. That
> way `relids = NULL` can be omitted.
No, because the list entry we fetch could be NULL.
regards, tom lane
On Mon, Aug 1, 2022 at 12:51 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Here's a rebase up to HEAD, mostly to placate the cfbot.
I accounted for d8e34fa7a (s/all_baserels/all_query_rels/
in those places) and made one tiny bug-fix change.
Nothing substantive as yet.
regards, tom lane
Hi,
For v3-0003-label-Var-nullability-in-parser.patch :
+ relids = (Bitmapset *) list_nth(pstate->p_nullingrels, rtindex - 1);
+ else
+ relids = NULL;
+
+ /*
+ * Merge with any already-declared nulling rels. (Typically there won't
+ * be any, but let's get it right if there are.)
+ */
+ if (relids != NULL)
It seems the last if block can be merged into the previous if block. That way `relids = NULL` can be omitted.
Cheers
On Tue, Aug 2, 2022 at 3:51 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Here's a rebase up to HEAD, mostly to placate the cfbot.
I accounted for d8e34fa7a (s/all_baserels/all_query_rels/
in those places) and made one tiny bug-fix change.
Nothing substantive as yet.
When we add required PlaceHolderVars to a join rel's targetlist, if the
PHV can be computed in the nullable-side of the join, we would add the
join's RT index to phnullingrels. This is correct as we know the PHV's
value can be nulled at this join. But I'm wondering if it is necessary
since we have already propagated any varnullingrels into the PHV when we
apply pullup variable replacement in perform_pullup_replace_vars().
On the other hand, the phnullingrels may contain RT indexes of outer
joins above this join level. It seems not good to add such a PHV to the
joinrel's targetlist. Below is an example:
# explain (verbose, costs off) select a.i, ss.jj from a left join (b left join (select c.i, coalesce(c.j, 1) as jj from c) ss on b.i = ss.i) on true;
QUERY PLAN
---------------------------------------------------------
Nested Loop Left Join
Output: a.i, (COALESCE(c.j, 1))
-> Seq Scan on public.a
Output: a.i, a.j
-> Materialize
Output: (COALESCE(c.j, 1))
-> Hash Left Join
Output: (COALESCE(c.j, 1))
Hash Cond: (b.i = c.i)
-> Seq Scan on public.b
Output: b.i, b.j
-> Hash
Output: c.i, (COALESCE(c.j, 1))
-> Seq Scan on public.c
Output: c.i, COALESCE(c.j, 1)
(15 rows)
In this query, for the joinrel {B, C}, the PHV in its targetlist has a
phnullingrels that contains the join of {A} and {BC}, which is confusing
because we have not reached that join level.
I tried the changes below to illustrate the two issues. The assertion
holds true during regression tests and the error pops up for the query
above.
PHV can be computed in the nullable-side of the join, we would add the
join's RT index to phnullingrels. This is correct as we know the PHV's
value can be nulled at this join. But I'm wondering if it is necessary
since we have already propagated any varnullingrels into the PHV when we
apply pullup variable replacement in perform_pullup_replace_vars().
On the other hand, the phnullingrels may contain RT indexes of outer
joins above this join level. It seems not good to add such a PHV to the
joinrel's targetlist. Below is an example:
# explain (verbose, costs off) select a.i, ss.jj from a left join (b left join (select c.i, coalesce(c.j, 1) as jj from c) ss on b.i = ss.i) on true;
QUERY PLAN
---------------------------------------------------------
Nested Loop Left Join
Output: a.i, (COALESCE(c.j, 1))
-> Seq Scan on public.a
Output: a.i, a.j
-> Materialize
Output: (COALESCE(c.j, 1))
-> Hash Left Join
Output: (COALESCE(c.j, 1))
Hash Cond: (b.i = c.i)
-> Seq Scan on public.b
Output: b.i, b.j
-> Hash
Output: c.i, (COALESCE(c.j, 1))
-> Seq Scan on public.c
Output: c.i, COALESCE(c.j, 1)
(15 rows)
In this query, for the joinrel {B, C}, the PHV in its targetlist has a
phnullingrels that contains the join of {A} and {BC}, which is confusing
because we have not reached that join level.
I tried the changes below to illustrate the two issues. The assertion
holds true during regression tests and the error pops up for the query
above.
--- a/src/backend/optimizer/util/placeholder.c
+++ b/src/backend/optimizer/util/placeholder.c
@@ -464,18 +464,20 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
{
if (sjinfo->jointype == JOIN_FULL && sjinfo->ojrelid != 0)
{
- /* PHV's value can be nulled at this join */
- phv->phnullingrels = bms_add_member(phv->phnullingrels,
- sjinfo->ojrelid);
+ Assert(bms_is_member(sjinfo->ojrelid, phv->phnullingrels));
+
+ if (!bms_is_subset(phv->phnullingrels, joinrel->relids))
+ elog(ERROR, "phnullingrels is not subset of joinrel's relids");
}
}
else if (bms_is_subset(phinfo->ph_eval_at, inner_rel->relids))
{
if (sjinfo->jointype != JOIN_INNER && sjinfo->ojrelid != 0)
{
- /* PHV's value can be nulled at this join */
- phv->phnullingrels = bms_add_member(phv->phnullingrels,
- sjinfo->ojrelid);
+ Assert(bms_is_member(sjinfo->ojrelid, phv->phnullingrels));
+
+ if (!bms_is_subset(phv->phnullingrels, joinrel->relids))
+ elog(ERROR, "phnullingrels is not subset of joinrel's relids");
}
}
If the two issues are indeed something we need to fix, maybe we can
change add_placeholders_to_joinrel() to search the PHVs from
outer_rel/inner_rel's targetlist, and add the ojrelid to phnullingrels
if needed, just like what we do in build_joinrel_tlist(). The PHVs there
should have the correct phnullingrels (at least the PHVs in base rels'
targetlists have correctly set phnullingrels to NULL).
Thanks
Richard
+++ b/src/backend/optimizer/util/placeholder.c
@@ -464,18 +464,20 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
{
if (sjinfo->jointype == JOIN_FULL && sjinfo->ojrelid != 0)
{
- /* PHV's value can be nulled at this join */
- phv->phnullingrels = bms_add_member(phv->phnullingrels,
- sjinfo->ojrelid);
+ Assert(bms_is_member(sjinfo->ojrelid, phv->phnullingrels));
+
+ if (!bms_is_subset(phv->phnullingrels, joinrel->relids))
+ elog(ERROR, "phnullingrels is not subset of joinrel's relids");
}
}
else if (bms_is_subset(phinfo->ph_eval_at, inner_rel->relids))
{
if (sjinfo->jointype != JOIN_INNER && sjinfo->ojrelid != 0)
{
- /* PHV's value can be nulled at this join */
- phv->phnullingrels = bms_add_member(phv->phnullingrels,
- sjinfo->ojrelid);
+ Assert(bms_is_member(sjinfo->ojrelid, phv->phnullingrels));
+
+ if (!bms_is_subset(phv->phnullingrels, joinrel->relids))
+ elog(ERROR, "phnullingrels is not subset of joinrel's relids");
}
}
If the two issues are indeed something we need to fix, maybe we can
change add_placeholders_to_joinrel() to search the PHVs from
outer_rel/inner_rel's targetlist, and add the ojrelid to phnullingrels
if needed, just like what we do in build_joinrel_tlist(). The PHVs there
should have the correct phnullingrels (at least the PHVs in base rels'
targetlists have correctly set phnullingrels to NULL).
Thanks
Richard
Richard Guo <guofenglinux@gmail.com> writes:
> When we add required PlaceHolderVars to a join rel's targetlist, if the
> PHV can be computed in the nullable-side of the join, we would add the
> join's RT index to phnullingrels. This is correct as we know the PHV's
> value can be nulled at this join. But I'm wondering if it is necessary
> since we have already propagated any varnullingrels into the PHV when we
> apply pullup variable replacement in perform_pullup_replace_vars().
I'm not seeing the connection there? Any nullingrels that are set
during perform_pullup_replace_vars would refer to outer joins within the
pulled-up subquery, whereas what you are talking about here is what
happens when the PHV's value propagates up through outer joins of the
parent query. There's no overlap between those relid sets, or if there
is we have some fault in the logic that constrains join order to ensure
that there's a valid place to compute each PHV.
> On the other hand, the phnullingrels may contain RT indexes of outer
> joins above this join level. It seems not good to add such a PHV to the
> joinrel's targetlist.
Hmm, yeah, add_placeholders_to_joinrel is doing this wrong. The
intent was to match what build_joinrel_tlist does with plain Vars,
but in that case we're adding the join's relid to what we find in
varnullingrels in the input tlist. Using the phnullingrels from
the placeholder_list entry is wrong. (I wonder whether a
placeholder_list entry's phnullingrels is meaningful at all?
Maybe it isn't.) I think it might work to take the intersection
of the join's relids with root->outer_join_rels.
> If the two issues are indeed something we need to fix, maybe we can
> change add_placeholders_to_joinrel() to search the PHVs from
> outer_rel/inner_rel's targetlist, and add the ojrelid to phnullingrels
> if needed, just like what we do in build_joinrel_tlist(). The PHVs there
> should have the correct phnullingrels (at least the PHVs in base rels'
> targetlists have correctly set phnullingrels to NULL).
Yeah, maybe we should do something more invasive and make use of the
input targetlists rather than doing this from scratch. Not sure.
I'm worried that doing it that way would increase the risk of getting
different join tlist contents depending on which pair of input rels
we happen to consider first.
regards, tom lane
I wrote:
> Richard Guo <guofenglinux@gmail.com> writes:
>> If the two issues are indeed something we need to fix, maybe we can
>> change add_placeholders_to_joinrel() to search the PHVs from
>> outer_rel/inner_rel's targetlist, and add the ojrelid to phnullingrels
>> if needed, just like what we do in build_joinrel_tlist(). The PHVs there
>> should have the correct phnullingrels (at least the PHVs in base rels'
>> targetlists have correctly set phnullingrels to NULL).
> Yeah, maybe we should do something more invasive and make use of the
> input targetlists rather than doing this from scratch. Not sure.
> I'm worried that doing it that way would increase the risk of getting
> different join tlist contents depending on which pair of input rels
> we happen to consider first.
After chewing on that for awhile, I've concluded that that is the way
to go. 0001 attached is a standalone patch to rearrange the way that
PHVs are added to joinrel targetlists. It results in one cosmetic
change in the regression tests, where the targetlist order for an
intermediate join node changes. I think that's fine; if anything,
the new order is more sensible than the old because it matches the
inputs' targetlist orders better.
I believe the reason I didn't do it like this to start with is that
I was concerned about the cost of searching the placeholder_list
repeatedly. With a lot of PHVs, build_joinrel_tlist becomes O(N^2)
just from the repeated find_placeholder_info lookups. We can fix
that by adding an index array to go straight from phid to the
PlaceHolderInfo. While thinking about where to construct such
an index array, I decided it'd be a good idea to have an explicit
step to "freeze" the set of PlaceHolderInfos, at the start of
deconstruct_jointree. This allows getting rid of the create_new_ph
flags for find_placeholder_info and add_vars_to_targetlist, which
I've always feared were bugs waiting to happen: they require callers
to have a very clear understanding of when they're invoked. There
might be some speed gain over existing code too, but I've not really
tried to measure it. I did drop a couple of hacks that were only
meant to short-circuit find_placeholder_info calls; that function
should now be cheap enough to not matter.
Barring objections, I'd like to push these soon and then rebase
the main outer-join-vars patch set over them.
regards, tom lane
diff --git a/src/backend/optimizer/util/placeholder.c b/src/backend/optimizer/util/placeholder.c
index 3b0f0584f0..f0e8cd9965 100644
--- a/src/backend/optimizer/util/placeholder.c
+++ b/src/backend/optimizer/util/placeholder.c
@@ -400,14 +400,16 @@ add_placeholders_to_base_rels(PlannerInfo *root)
/*
* add_placeholders_to_joinrel
- * Add any required PlaceHolderVars to a join rel's targetlist;
- * and if they contain lateral references, add those references to the
- * joinrel's direct_lateral_relids.
+ * Add any newly-computable PlaceHolderVars to a join rel's targetlist;
+ * and if computable PHVs contain lateral references, add those
+ * references to the joinrel's direct_lateral_relids.
*
* A join rel should emit a PlaceHolderVar if (a) the PHV can be computed
* at or below this join level and (b) the PHV is needed above this level.
- * However, condition (a) is sufficient to add to direct_lateral_relids,
- * as explained below.
+ * Our caller build_join_rel() has already added any PHVs that were computed
+ * in either join input rel, so we need add only newly-computable ones to
+ * the targetlist. However, direct_lateral_relids must be updated for every
+ * PHV computable at or below this join, as explained below.
*/
void
add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
@@ -426,13 +428,10 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
/* Is it still needed above this joinrel? */
if (bms_nonempty_difference(phinfo->ph_needed, relids))
{
- /* Yup, add it to the output */
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- phinfo->ph_var);
- joinrel->reltarget->width += phinfo->ph_width;
-
/*
- * Charge the cost of evaluating the contained expression if
+ * Yes, but only add to tlist if it wasn't computed in either
+ * input; otherwise it should be there already. Also, we
+ * charge the cost of evaluating the contained expression if
* the PHV can be computed here but not in either input. This
* is a bit bogus because we make the decision based on the
* first pair of possible input relations considered for the
@@ -445,12 +444,15 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
if (!bms_is_subset(phinfo->ph_eval_at, outer_rel->relids) &&
!bms_is_subset(phinfo->ph_eval_at, inner_rel->relids))
{
+ PlaceHolderVar *phv = phinfo->ph_var;
QualCost cost;
- cost_qual_eval_node(&cost, (Node *) phinfo->ph_var->phexpr,
- root);
+ joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
+ phv);
+ cost_qual_eval_node(&cost, (Node *) phv->phexpr, root);
joinrel->reltarget->cost.startup += cost.startup;
joinrel->reltarget->cost.per_tuple += cost.per_tuple;
+ joinrel->reltarget->width += phinfo->ph_width;
}
}
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 520409f4ba..442c12acef 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -688,6 +688,8 @@ build_join_rel(PlannerInfo *root,
*/
build_joinrel_tlist(root, joinrel, outer_rel);
build_joinrel_tlist(root, joinrel, inner_rel);
+
+ /* Add any newly-computable PlaceHolderVars, too */
add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
/*
@@ -966,6 +968,7 @@ min_join_parameterization(PlannerInfo *root,
* The join's targetlist includes all Vars of its member relations that
* will still be needed above the join. This subroutine adds all such
* Vars from the specified input rel's tlist to the join rel's tlist.
+ * Likewise for any PlaceHolderVars emitted by the input rel.
*
* We also compute the expected width of the join's output, making use
* of data that was cached at the baserel level by set_rel_width().
@@ -982,11 +985,24 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
Var *var = (Var *) lfirst(vars);
/*
- * Ignore PlaceHolderVars in the input tlists; we'll make our own
- * decisions about whether to copy them.
+ * For a PlaceHolderVar, we have to look up the PlaceHolderInfo.
*/
if (IsA(var, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) var;
+ PlaceHolderInfo *phinfo = find_placeholder_info(root, phv, false);
+
+ /* Is it still needed above this joinrel? */
+ if (bms_nonempty_difference(phinfo->ph_needed, relids))
+ {
+ /* Yup, add it to the output */
+ joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
+ phv);
+ /* Bubbling up the precomputed result has cost zero */
+ joinrel->reltarget->width += phinfo->ph_width;
+ }
continue;
+ }
/*
* Otherwise, anything in a baserel or joinrel targetlist ought to be
diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out
index b9853af2dc..2ed2e542a4 100644
--- a/src/test/regress/expected/join.out
+++ b/src/test/regress/expected/join.out
@@ -5793,10 +5793,10 @@ select * from
Nested Loop
Output: c.q1, c.q2, a.q1, a.q2, b.q1, (COALESCE(b.q2, '42'::bigint)), d.q1, (COALESCE((COALESCE(b.q2,
'42'::bigint)),d.q2)), ((COALESCE((COALESCE(b.q2, '42'::bigint)), d.q2)))
-> Hash Right Join
- Output: c.q1, c.q2, a.q1, a.q2, b.q1, d.q1, (COALESCE(b.q2, '42'::bigint)), (COALESCE((COALESCE(b.q2,
'42'::bigint)),d.q2))
+ Output: c.q1, c.q2, a.q1, a.q2, b.q1, (COALESCE(b.q2, '42'::bigint)), d.q1, (COALESCE((COALESCE(b.q2,
'42'::bigint)),d.q2))
Hash Cond: (d.q1 = c.q2)
-> Nested Loop
- Output: a.q1, a.q2, b.q1, d.q1, (COALESCE(b.q2, '42'::bigint)), (COALESCE((COALESCE(b.q2,
'42'::bigint)),d.q2))
+ Output: a.q1, a.q2, b.q1, (COALESCE(b.q2, '42'::bigint)), d.q1, (COALESCE((COALESCE(b.q2,
'42'::bigint)),d.q2))
-> Hash Left Join
Output: a.q1, a.q2, b.q1, (COALESCE(b.q2, '42'::bigint))
Hash Cond: (a.q2 = b.q1)
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index fb28e6411a..bf2586a341 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -6245,7 +6245,7 @@ set_rel_width(PlannerInfo *root, RelOptInfo *rel)
* scanning this rel, so be sure to include it in reltarget->cost.
*/
PlaceHolderVar *phv = (PlaceHolderVar *) node;
- PlaceHolderInfo *phinfo = find_placeholder_info(root, phv, false);
+ PlaceHolderInfo *phinfo = find_placeholder_info(root, phv);
QualCost cost;
tuple_width += phinfo->ph_width;
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 7991295548..2c900e6b12 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -1323,7 +1323,7 @@ generate_base_implied_equalities_no_const(PlannerInfo *root,
PVC_RECURSE_WINDOWFUNCS |
PVC_INCLUDE_PLACEHOLDERS);
- add_vars_to_targetlist(root, vars, ec->ec_relids, false);
+ add_vars_to_targetlist(root, vars, ec->ec_relids);
list_free(vars);
}
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index e37f2933eb..f138f93509 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -4915,13 +4915,8 @@ replace_nestloop_params_mutator(Node *node, PlannerInfo *root)
/* Upper-level PlaceHolderVars should be long gone at this point */
Assert(phv->phlevelsup == 0);
- /*
- * Check whether we need to replace the PHV. We use bms_overlap as a
- * cheap/quick test to see if the PHV might be evaluated in the outer
- * rels, and then grab its PlaceHolderInfo to tell for sure.
- */
- if (!bms_overlap(phv->phrels, root->curOuterRels) ||
- !bms_is_subset(find_placeholder_info(root, phv, false)->ph_eval_at,
+ /* Check whether we need to replace the PHV */
+ if (!bms_is_subset(find_placeholder_info(root, phv)->ph_eval_at,
root->curOuterRels))
{
/*
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 023efbaf09..1867a097fd 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -189,7 +189,7 @@ build_base_rel_tlists(PlannerInfo *root, List *final_tlist)
if (tlist_vars != NIL)
{
- add_vars_to_targetlist(root, tlist_vars, bms_make_singleton(0), true);
+ add_vars_to_targetlist(root, tlist_vars, bms_make_singleton(0));
list_free(tlist_vars);
}
@@ -206,7 +206,7 @@ build_base_rel_tlists(PlannerInfo *root, List *final_tlist)
if (having_vars != NIL)
{
add_vars_to_targetlist(root, having_vars,
- bms_make_singleton(0), true);
+ bms_make_singleton(0));
list_free(having_vars);
}
}
@@ -221,14 +221,12 @@ build_base_rel_tlists(PlannerInfo *root, List *final_tlist)
*
* The list may also contain PlaceHolderVars. These don't necessarily
* have a single owning relation; we keep their attr_needed info in
- * root->placeholder_list instead. If create_new_ph is true, it's OK
- * to create new PlaceHolderInfos; otherwise, the PlaceHolderInfos must
- * already exist, and we should only update their ph_needed. (This should
- * be true before deconstruct_jointree begins, and false after that.)
+ * root->placeholder_list instead. Find or create the associated
+ * PlaceHolderInfo entry, and update its ph_needed.
*/
void
add_vars_to_targetlist(PlannerInfo *root, List *vars,
- Relids where_needed, bool create_new_ph)
+ Relids where_needed)
{
ListCell *temp;
@@ -262,8 +260,7 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
else if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
- PlaceHolderInfo *phinfo = find_placeholder_info(root, phv,
- create_new_ph);
+ PlaceHolderInfo *phinfo = find_placeholder_info(root, phv);
phinfo->ph_needed = bms_add_members(phinfo->ph_needed,
where_needed);
@@ -432,7 +429,7 @@ extract_lateral_references(PlannerInfo *root, RelOptInfo *brel, Index rtindex)
* Push Vars into their source relations' targetlists, and PHVs into
* root->placeholder_list.
*/
- add_vars_to_targetlist(root, newvars, where_needed, true);
+ add_vars_to_targetlist(root, newvars, where_needed);
/* Remember the lateral references for create_lateral_join_info */
brel->lateral_vars = newvars;
@@ -493,8 +490,7 @@ create_lateral_join_info(PlannerInfo *root)
else if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
- PlaceHolderInfo *phinfo = find_placeholder_info(root, phv,
- false);
+ PlaceHolderInfo *phinfo = find_placeholder_info(root, phv);
found_laterals = true;
lateral_relids = bms_add_members(lateral_relids,
@@ -691,6 +687,14 @@ deconstruct_jointree(PlannerInfo *root)
Relids inner_join_rels;
List *postponed_qual_list = NIL;
+ /*
+ * After this point, no more PlaceHolderInfos may be made, because
+ * make_outerjoininfo and update_placeholder_eval_levels require all
+ * active placeholders to be present in root->placeholder_list while we
+ * crawl up the join tree.
+ */
+ freeze_placeholder_set(root);
+
/* Start recursion at top of jointree */
Assert(root->parse->jointree != NULL &&
IsA(root->parse->jointree, FromExpr));
@@ -1866,7 +1870,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
PVC_RECURSE_WINDOWFUNCS |
PVC_INCLUDE_PLACEHOLDERS);
- add_vars_to_targetlist(root, vars, relids, false);
+ add_vars_to_targetlist(root, vars, relids);
list_free(vars);
}
@@ -2380,7 +2384,7 @@ process_implied_equality(PlannerInfo *root,
PVC_RECURSE_WINDOWFUNCS |
PVC_INCLUDE_PLACEHOLDERS);
- add_vars_to_targetlist(root, vars, relids, false);
+ add_vars_to_targetlist(root, vars, relids);
list_free(vars);
}
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index c92ddd27ed..248cde4d9b 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -75,6 +75,8 @@ query_planner(PlannerInfo *root,
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->placeholder_array = NULL;
+ root->placeholder_array_size = 0;
root->fkey_list = NIL;
root->initial_rels = NIL;
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 7e134822f3..cf7691a474 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -291,7 +291,7 @@ expand_inherited_rtentry(PlannerInfo *root, RelOptInfo *rel,
* Add the newly added Vars to parent's reltarget. We needn't worry
* about the children's reltargets, they'll be made later.
*/
- add_vars_to_targetlist(root, newvars, bms_make_singleton(0), false);
+ add_vars_to_targetlist(root, newvars, bms_make_singleton(0));
}
table_close(oldrelation, NoLock);
diff --git a/src/backend/optimizer/util/paramassign.c b/src/backend/optimizer/util/paramassign.c
index 12486cb067..8e2d4bf515 100644
--- a/src/backend/optimizer/util/paramassign.c
+++ b/src/backend/optimizer/util/paramassign.c
@@ -470,7 +470,7 @@ process_subquery_nestloop_params(PlannerInfo *root, List *subplan_params)
ListCell *lc;
/* If not from a nestloop outer rel, complain */
- if (!bms_is_subset(find_placeholder_info(root, phv, false)->ph_eval_at,
+ if (!bms_is_subset(find_placeholder_info(root, phv)->ph_eval_at,
root->curOuterRels))
elog(ERROR, "non-LATERAL parameter required by subquery");
@@ -517,8 +517,7 @@ identify_current_nestloop_params(PlannerInfo *root, Relids leftrelids)
/*
* We are looking for Vars and PHVs that can be supplied by the
- * lefthand rels. The "bms_overlap" test is just an optimization to
- * allow skipping find_placeholder_info() if the PHV couldn't match.
+ * lefthand rels.
*/
if (IsA(nlp->paramval, Var) &&
bms_is_member(nlp->paramval->varno, leftrelids))
@@ -528,11 +527,8 @@ identify_current_nestloop_params(PlannerInfo *root, Relids leftrelids)
result = lappend(result, nlp);
}
else if (IsA(nlp->paramval, PlaceHolderVar) &&
- bms_overlap(((PlaceHolderVar *) nlp->paramval)->phrels,
- leftrelids) &&
bms_is_subset(find_placeholder_info(root,
- (PlaceHolderVar *) nlp->paramval,
- false)->ph_eval_at,
+ (PlaceHolderVar *) nlp->paramval)->ph_eval_at,
leftrelids))
{
root->curOuterParams = foreach_delete_current(root->curOuterParams,
diff --git a/src/backend/optimizer/util/placeholder.c b/src/backend/optimizer/util/placeholder.c
index f0e8cd9965..e34402523d 100644
--- a/src/backend/optimizer/util/placeholder.c
+++ b/src/backend/optimizer/util/placeholder.c
@@ -52,8 +52,8 @@ make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
* find_placeholder_info
* Fetch the PlaceHolderInfo for the given PHV
*
- * If the PlaceHolderInfo doesn't exist yet, create it if create_new_ph is
- * true, else throw an error.
+ * If the PlaceHolderInfo doesn't exist yet, create it if we haven't yet
+ * frozen the set of PlaceHolderInfos for the query; else throw an error.
*
* This is separate from make_placeholder_expr because subquery pullup has
* to make PlaceHolderVars for expressions that might not be used at all in
@@ -61,13 +61,10 @@ make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
* We build PlaceHolderInfos only for PHVs that are still present in the
* simplified query passed to query_planner().
*
- * Note: this should only be called after query_planner() has started. Also,
- * create_new_ph must not be true after deconstruct_jointree begins, because
- * make_outerjoininfo assumes that we already know about all placeholders.
+ * Note: this should only be called after query_planner() has started.
*/
PlaceHolderInfo *
-find_placeholder_info(PlannerInfo *root, PlaceHolderVar *phv,
- bool create_new_ph)
+find_placeholder_info(PlannerInfo *root, PlaceHolderVar *phv)
{
PlaceHolderInfo *phinfo;
Relids rels_used;
@@ -76,6 +73,23 @@ find_placeholder_info(PlannerInfo *root, PlaceHolderVar *phv,
/* if this ever isn't true, we'd need to be able to look in parent lists */
Assert(phv->phlevelsup == 0);
+ /* Use the index array if it exists */
+ if (root->placeholder_array != NULL)
+ {
+ if (phv->phid < root->placeholder_array_size)
+ phinfo = root->placeholder_array[phv->phid];
+ else
+ phinfo = NULL;
+ if (phinfo != NULL)
+ {
+ Assert(phinfo->phid == phv->phid);
+ return phinfo;
+ }
+ /* Existence of the array means we've frozen the PHI set */
+ elog(ERROR, "too late to create a new PlaceHolderInfo");
+ }
+
+ /* Otherwise, find it the hard way in placeholder_list */
foreach(lc, root->placeholder_list)
{
phinfo = (PlaceHolderInfo *) lfirst(lc);
@@ -84,9 +98,6 @@ find_placeholder_info(PlannerInfo *root, PlaceHolderVar *phv,
}
/* Not found, so create it */
- if (!create_new_ph)
- elog(ERROR, "too late to create a new PlaceHolderInfo");
-
phinfo = makeNode(PlaceHolderInfo);
phinfo->phid = phv->phid;
@@ -133,16 +144,13 @@ find_placeholder_info(PlannerInfo *root, PlaceHolderVar *phv,
*
* We don't need to look at the targetlist because build_base_rel_tlists()
* will already have made entries for any PHVs in the tlist.
- *
- * This is called before we begin deconstruct_jointree. Once we begin
- * deconstruct_jointree, all active placeholders must be present in
- * root->placeholder_list, because make_outerjoininfo and
- * update_placeholder_eval_levels require this info to be available
- * while we crawl up the join tree.
*/
void
find_placeholders_in_jointree(PlannerInfo *root)
{
+ /* This must be done before freezing the set of PHIs */
+ Assert(root->placeholder_array == NULL);
+
/* We need do nothing if the query contains no PlaceHolderVars */
if (root->glob->lastPHId != 0)
{
@@ -232,11 +240,37 @@ find_placeholders_in_expr(PlannerInfo *root, Node *expr)
continue;
/* Create a PlaceHolderInfo entry if there's not one already */
- (void) find_placeholder_info(root, phv, true);
+ (void) find_placeholder_info(root, phv);
}
list_free(vars);
}
+/*
+ * freeze_placeholder_set
+ * Mark that no more PlaceHolderInfos may be created for this query
+ *
+ * We do this by creating an index array root->placeholder_array[],
+ * which also serves to speed up future find_placeholder_info() lookups.
+ */
+void
+freeze_placeholder_set(PlannerInfo *root)
+{
+ ListCell *lc;
+
+ /* The global lastPHId may be an overestimate, but it's safe */
+ root->placeholder_array_size = root->glob->lastPHId + 1;
+ root->placeholder_array = (PlaceHolderInfo **)
+ palloc0(root->placeholder_array_size * sizeof(PlaceHolderInfo *));
+
+ foreach(lc, root->placeholder_list)
+ {
+ PlaceHolderInfo *phinfo = (PlaceHolderInfo *) lfirst(lc);
+
+ Assert(phinfo->phid < root->placeholder_array_size);
+ root->placeholder_array[phinfo->phid] = phinfo;
+ }
+}
+
/*
* update_placeholder_eval_levels
* Adjust the target evaluation levels for placeholders
@@ -359,7 +393,7 @@ fix_placeholder_input_needed_levels(PlannerInfo *root)
PVC_RECURSE_WINDOWFUNCS |
PVC_INCLUDE_PLACEHOLDERS);
- add_vars_to_targetlist(root, vars, phinfo->ph_eval_at, false);
+ add_vars_to_targetlist(root, vars, phinfo->ph_eval_at);
list_free(vars);
}
}
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 442c12acef..13e36dd0e1 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -990,7 +990,7 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
if (IsA(var, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) var;
- PlaceHolderInfo *phinfo = find_placeholder_info(root, phv, false);
+ PlaceHolderInfo *phinfo = find_placeholder_info(root, phv);
/* Is it still needed above this joinrel? */
if (bms_nonempty_difference(phinfo->ph_needed, relids))
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
index ebc6ce84b0..ee915546ac 100644
--- a/src/backend/optimizer/util/var.c
+++ b/src/backend/optimizer/util/var.c
@@ -212,6 +212,7 @@ pull_varnos_walker(Node *node, pull_varnos_context *context)
{
ListCell *lc;
+ /* can't rely on placeholder_array[] yet */
foreach(lc, context->root->placeholder_list)
{
phinfo = (PlaceHolderInfo *) lfirst(lc);
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 3b065139e6..4c8fe28bfc 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -357,6 +357,15 @@ struct PlannerInfo
/* list of PlaceHolderInfos */
List *placeholder_list;
+ /*
+ * array of PlaceHolderInfos indexed by phid. We create this in
+ * freeze_placeholder_set(); it serves both to speed up later lookups by
+ * phid, and to signal that no more PlaceHolderInfos can be made.
+ */
+ struct PlaceHolderInfo **placeholder_array pg_node_attr(read_write_ignore, array_size(placeholder_array_size));
+ /* allocated size of array */
+ int placeholder_array_size pg_node_attr(read_write_ignore);
+
/* list of ForeignKeyOptInfos */
List *fkey_list;
diff --git a/src/include/optimizer/placeholder.h b/src/include/optimizer/placeholder.h
index 39803ea41f..a5153c707b 100644
--- a/src/include/optimizer/placeholder.h
+++ b/src/include/optimizer/placeholder.h
@@ -20,8 +20,9 @@
extern PlaceHolderVar *make_placeholder_expr(PlannerInfo *root, Expr *expr,
Relids phrels);
extern PlaceHolderInfo *find_placeholder_info(PlannerInfo *root,
- PlaceHolderVar *phv, bool create_new_ph);
+ PlaceHolderVar *phv);
extern void find_placeholders_in_jointree(PlannerInfo *root);
+extern void freeze_placeholder_set(PlannerInfo *root);
extern void update_placeholder_eval_levels(PlannerInfo *root,
SpecialJoinInfo *new_sjinfo);
extern void fix_placeholder_input_needed_levels(PlannerInfo *root);
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index c4f61c1a09..1566f435b3 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -71,7 +71,7 @@ extern void add_base_rels_to_query(PlannerInfo *root, Node *jtnode);
extern void add_other_rels_to_query(PlannerInfo *root);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
- Relids where_needed, bool create_new_ph);
+ Relids where_needed);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
I wrote:
> ... We can fix
> that by adding an index array to go straight from phid to the
> PlaceHolderInfo. While thinking about where to construct such
> an index array, I decided it'd be a good idea to have an explicit
> step to "freeze" the set of PlaceHolderInfos, at the start of
> deconstruct_jointree.
On further thought, it seems better to maintain the index array
from the start, allowing complete replacement of the linear list
searches. We can add a separate bool flag to denote frozen-ness.
This does have minor downsides:
* Some fiddly code is needed to enlarge the index array at need.
But it's not different from that for, say, simple_rel_array.
* If we ever have a need to mutate the placeholder_list as a whole,
we'd have to reconstruct the index array to point to the new
objects. We don't do that at present, except in one place in
analyzejoins.c, which is easily fixed. While the same argument
could be raised against the v1 patch, it's not very likely that
we'd be doing such mutation beyond the start of deconstruct_jointree.
Hence, see v2 attached.
regards, tom lane
diff --git a/src/backend/optimizer/util/placeholder.c b/src/backend/optimizer/util/placeholder.c
index 3b0f0584f0..f0e8cd9965 100644
--- a/src/backend/optimizer/util/placeholder.c
+++ b/src/backend/optimizer/util/placeholder.c
@@ -400,14 +400,16 @@ add_placeholders_to_base_rels(PlannerInfo *root)
/*
* add_placeholders_to_joinrel
- * Add any required PlaceHolderVars to a join rel's targetlist;
- * and if they contain lateral references, add those references to the
- * joinrel's direct_lateral_relids.
+ * Add any newly-computable PlaceHolderVars to a join rel's targetlist;
+ * and if computable PHVs contain lateral references, add those
+ * references to the joinrel's direct_lateral_relids.
*
* A join rel should emit a PlaceHolderVar if (a) the PHV can be computed
* at or below this join level and (b) the PHV is needed above this level.
- * However, condition (a) is sufficient to add to direct_lateral_relids,
- * as explained below.
+ * Our caller build_join_rel() has already added any PHVs that were computed
+ * in either join input rel, so we need add only newly-computable ones to
+ * the targetlist. However, direct_lateral_relids must be updated for every
+ * PHV computable at or below this join, as explained below.
*/
void
add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
@@ -426,13 +428,10 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
/* Is it still needed above this joinrel? */
if (bms_nonempty_difference(phinfo->ph_needed, relids))
{
- /* Yup, add it to the output */
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- phinfo->ph_var);
- joinrel->reltarget->width += phinfo->ph_width;
-
/*
- * Charge the cost of evaluating the contained expression if
+ * Yes, but only add to tlist if it wasn't computed in either
+ * input; otherwise it should be there already. Also, we
+ * charge the cost of evaluating the contained expression if
* the PHV can be computed here but not in either input. This
* is a bit bogus because we make the decision based on the
* first pair of possible input relations considered for the
@@ -445,12 +444,15 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
if (!bms_is_subset(phinfo->ph_eval_at, outer_rel->relids) &&
!bms_is_subset(phinfo->ph_eval_at, inner_rel->relids))
{
+ PlaceHolderVar *phv = phinfo->ph_var;
QualCost cost;
- cost_qual_eval_node(&cost, (Node *) phinfo->ph_var->phexpr,
- root);
+ joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
+ phv);
+ cost_qual_eval_node(&cost, (Node *) phv->phexpr, root);
joinrel->reltarget->cost.startup += cost.startup;
joinrel->reltarget->cost.per_tuple += cost.per_tuple;
+ joinrel->reltarget->width += phinfo->ph_width;
}
}
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 520409f4ba..442c12acef 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -688,6 +688,8 @@ build_join_rel(PlannerInfo *root,
*/
build_joinrel_tlist(root, joinrel, outer_rel);
build_joinrel_tlist(root, joinrel, inner_rel);
+
+ /* Add any newly-computable PlaceHolderVars, too */
add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
/*
@@ -966,6 +968,7 @@ min_join_parameterization(PlannerInfo *root,
* The join's targetlist includes all Vars of its member relations that
* will still be needed above the join. This subroutine adds all such
* Vars from the specified input rel's tlist to the join rel's tlist.
+ * Likewise for any PlaceHolderVars emitted by the input rel.
*
* We also compute the expected width of the join's output, making use
* of data that was cached at the baserel level by set_rel_width().
@@ -982,11 +985,24 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
Var *var = (Var *) lfirst(vars);
/*
- * Ignore PlaceHolderVars in the input tlists; we'll make our own
- * decisions about whether to copy them.
+ * For a PlaceHolderVar, we have to look up the PlaceHolderInfo.
*/
if (IsA(var, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) var;
+ PlaceHolderInfo *phinfo = find_placeholder_info(root, phv, false);
+
+ /* Is it still needed above this joinrel? */
+ if (bms_nonempty_difference(phinfo->ph_needed, relids))
+ {
+ /* Yup, add it to the output */
+ joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
+ phv);
+ /* Bubbling up the precomputed result has cost zero */
+ joinrel->reltarget->width += phinfo->ph_width;
+ }
continue;
+ }
/*
* Otherwise, anything in a baserel or joinrel targetlist ought to be
diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out
index b9853af2dc..2ed2e542a4 100644
--- a/src/test/regress/expected/join.out
+++ b/src/test/regress/expected/join.out
@@ -5793,10 +5793,10 @@ select * from
Nested Loop
Output: c.q1, c.q2, a.q1, a.q2, b.q1, (COALESCE(b.q2, '42'::bigint)), d.q1, (COALESCE((COALESCE(b.q2,
'42'::bigint)),d.q2)), ((COALESCE((COALESCE(b.q2, '42'::bigint)), d.q2)))
-> Hash Right Join
- Output: c.q1, c.q2, a.q1, a.q2, b.q1, d.q1, (COALESCE(b.q2, '42'::bigint)), (COALESCE((COALESCE(b.q2,
'42'::bigint)),d.q2))
+ Output: c.q1, c.q2, a.q1, a.q2, b.q1, (COALESCE(b.q2, '42'::bigint)), d.q1, (COALESCE((COALESCE(b.q2,
'42'::bigint)),d.q2))
Hash Cond: (d.q1 = c.q2)
-> Nested Loop
- Output: a.q1, a.q2, b.q1, d.q1, (COALESCE(b.q2, '42'::bigint)), (COALESCE((COALESCE(b.q2,
'42'::bigint)),d.q2))
+ Output: a.q1, a.q2, b.q1, (COALESCE(b.q2, '42'::bigint)), d.q1, (COALESCE((COALESCE(b.q2,
'42'::bigint)),d.q2))
-> Hash Left Join
Output: a.q1, a.q2, b.q1, (COALESCE(b.q2, '42'::bigint))
Hash Cond: (a.q2 = b.q1)
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index fb28e6411a..bf2586a341 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -6245,7 +6245,7 @@ set_rel_width(PlannerInfo *root, RelOptInfo *rel)
* scanning this rel, so be sure to include it in reltarget->cost.
*/
PlaceHolderVar *phv = (PlaceHolderVar *) node;
- PlaceHolderInfo *phinfo = find_placeholder_info(root, phv, false);
+ PlaceHolderInfo *phinfo = find_placeholder_info(root, phv);
QualCost cost;
tuple_width += phinfo->ph_width;
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 7991295548..2c900e6b12 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -1323,7 +1323,7 @@ generate_base_implied_equalities_no_const(PlannerInfo *root,
PVC_RECURSE_WINDOWFUNCS |
PVC_INCLUDE_PLACEHOLDERS);
- add_vars_to_targetlist(root, vars, ec->ec_relids, false);
+ add_vars_to_targetlist(root, vars, ec->ec_relids);
list_free(vars);
}
}
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index 337f470d58..bbeca9a9ab 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -388,8 +388,11 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
Assert(!bms_is_member(relid, phinfo->ph_lateral));
if (bms_is_subset(phinfo->ph_needed, joinrelids) &&
bms_is_member(relid, phinfo->ph_eval_at))
+ {
root->placeholder_list = foreach_delete_current(root->placeholder_list,
l);
+ root->placeholder_array[phinfo->phid] = NULL;
+ }
else
{
phinfo->ph_eval_at = bms_del_member(phinfo->ph_eval_at, relid);
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index e37f2933eb..f138f93509 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -4915,13 +4915,8 @@ replace_nestloop_params_mutator(Node *node, PlannerInfo *root)
/* Upper-level PlaceHolderVars should be long gone at this point */
Assert(phv->phlevelsup == 0);
- /*
- * Check whether we need to replace the PHV. We use bms_overlap as a
- * cheap/quick test to see if the PHV might be evaluated in the outer
- * rels, and then grab its PlaceHolderInfo to tell for sure.
- */
- if (!bms_overlap(phv->phrels, root->curOuterRels) ||
- !bms_is_subset(find_placeholder_info(root, phv, false)->ph_eval_at,
+ /* Check whether we need to replace the PHV */
+ if (!bms_is_subset(find_placeholder_info(root, phv)->ph_eval_at,
root->curOuterRels))
{
/*
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 023efbaf09..fd8cbb1dc7 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -189,7 +189,7 @@ build_base_rel_tlists(PlannerInfo *root, List *final_tlist)
if (tlist_vars != NIL)
{
- add_vars_to_targetlist(root, tlist_vars, bms_make_singleton(0), true);
+ add_vars_to_targetlist(root, tlist_vars, bms_make_singleton(0));
list_free(tlist_vars);
}
@@ -206,7 +206,7 @@ build_base_rel_tlists(PlannerInfo *root, List *final_tlist)
if (having_vars != NIL)
{
add_vars_to_targetlist(root, having_vars,
- bms_make_singleton(0), true);
+ bms_make_singleton(0));
list_free(having_vars);
}
}
@@ -221,14 +221,12 @@ build_base_rel_tlists(PlannerInfo *root, List *final_tlist)
*
* The list may also contain PlaceHolderVars. These don't necessarily
* have a single owning relation; we keep their attr_needed info in
- * root->placeholder_list instead. If create_new_ph is true, it's OK
- * to create new PlaceHolderInfos; otherwise, the PlaceHolderInfos must
- * already exist, and we should only update their ph_needed. (This should
- * be true before deconstruct_jointree begins, and false after that.)
+ * root->placeholder_list instead. Find or create the associated
+ * PlaceHolderInfo entry, and update its ph_needed.
*/
void
add_vars_to_targetlist(PlannerInfo *root, List *vars,
- Relids where_needed, bool create_new_ph)
+ Relids where_needed)
{
ListCell *temp;
@@ -262,8 +260,7 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
else if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
- PlaceHolderInfo *phinfo = find_placeholder_info(root, phv,
- create_new_ph);
+ PlaceHolderInfo *phinfo = find_placeholder_info(root, phv);
phinfo->ph_needed = bms_add_members(phinfo->ph_needed,
where_needed);
@@ -432,7 +429,7 @@ extract_lateral_references(PlannerInfo *root, RelOptInfo *brel, Index rtindex)
* Push Vars into their source relations' targetlists, and PHVs into
* root->placeholder_list.
*/
- add_vars_to_targetlist(root, newvars, where_needed, true);
+ add_vars_to_targetlist(root, newvars, where_needed);
/* Remember the lateral references for create_lateral_join_info */
brel->lateral_vars = newvars;
@@ -493,8 +490,7 @@ create_lateral_join_info(PlannerInfo *root)
else if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
- PlaceHolderInfo *phinfo = find_placeholder_info(root, phv,
- false);
+ PlaceHolderInfo *phinfo = find_placeholder_info(root, phv);
found_laterals = true;
lateral_relids = bms_add_members(lateral_relids,
@@ -691,6 +687,14 @@ deconstruct_jointree(PlannerInfo *root)
Relids inner_join_rels;
List *postponed_qual_list = NIL;
+ /*
+ * After this point, no more PlaceHolderInfos may be made, because
+ * make_outerjoininfo and update_placeholder_eval_levels require all
+ * active placeholders to be present in root->placeholder_list while we
+ * crawl up the join tree.
+ */
+ root->placeholdersFrozen = true;
+
/* Start recursion at top of jointree */
Assert(root->parse->jointree != NULL &&
IsA(root->parse->jointree, FromExpr));
@@ -1866,7 +1870,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
PVC_RECURSE_WINDOWFUNCS |
PVC_INCLUDE_PLACEHOLDERS);
- add_vars_to_targetlist(root, vars, relids, false);
+ add_vars_to_targetlist(root, vars, relids);
list_free(vars);
}
@@ -2380,7 +2384,7 @@ process_implied_equality(PlannerInfo *root,
PVC_RECURSE_WINDOWFUNCS |
PVC_INCLUDE_PLACEHOLDERS);
- add_vars_to_targetlist(root, vars, relids, false);
+ add_vars_to_targetlist(root, vars, relids);
list_free(vars);
}
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index c92ddd27ed..248cde4d9b 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -75,6 +75,8 @@ query_planner(PlannerInfo *root,
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->placeholder_array = NULL;
+ root->placeholder_array_size = 0;
root->fkey_list = NIL;
root->initial_rels = NIL;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 64632db73c..b27cd28338 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -635,6 +635,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
root->qual_security_level = 0;
root->hasPseudoConstantQuals = false;
root->hasAlternativeSubPlans = false;
+ root->placeholdersFrozen = false;
root->hasRecursion = hasRecursion;
if (hasRecursion)
root->wt_param_id = assign_special_exec_param(root);
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 0bd99acf83..41c7066d90 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -1011,6 +1011,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
subroot->grouping_map = NULL;
subroot->minmax_aggs = NIL;
subroot->qual_security_level = 0;
+ subroot->placeholdersFrozen = false;
subroot->hasRecursion = false;
subroot->wt_param_id = -1;
subroot->non_recursive_path = NULL;
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 7e134822f3..cf7691a474 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -291,7 +291,7 @@ expand_inherited_rtentry(PlannerInfo *root, RelOptInfo *rel,
* Add the newly added Vars to parent's reltarget. We needn't worry
* about the children's reltargets, they'll be made later.
*/
- add_vars_to_targetlist(root, newvars, bms_make_singleton(0), false);
+ add_vars_to_targetlist(root, newvars, bms_make_singleton(0));
}
table_close(oldrelation, NoLock);
diff --git a/src/backend/optimizer/util/paramassign.c b/src/backend/optimizer/util/paramassign.c
index 12486cb067..8e2d4bf515 100644
--- a/src/backend/optimizer/util/paramassign.c
+++ b/src/backend/optimizer/util/paramassign.c
@@ -470,7 +470,7 @@ process_subquery_nestloop_params(PlannerInfo *root, List *subplan_params)
ListCell *lc;
/* If not from a nestloop outer rel, complain */
- if (!bms_is_subset(find_placeholder_info(root, phv, false)->ph_eval_at,
+ if (!bms_is_subset(find_placeholder_info(root, phv)->ph_eval_at,
root->curOuterRels))
elog(ERROR, "non-LATERAL parameter required by subquery");
@@ -517,8 +517,7 @@ identify_current_nestloop_params(PlannerInfo *root, Relids leftrelids)
/*
* We are looking for Vars and PHVs that can be supplied by the
- * lefthand rels. The "bms_overlap" test is just an optimization to
- * allow skipping find_placeholder_info() if the PHV couldn't match.
+ * lefthand rels.
*/
if (IsA(nlp->paramval, Var) &&
bms_is_member(nlp->paramval->varno, leftrelids))
@@ -528,11 +527,8 @@ identify_current_nestloop_params(PlannerInfo *root, Relids leftrelids)
result = lappend(result, nlp);
}
else if (IsA(nlp->paramval, PlaceHolderVar) &&
- bms_overlap(((PlaceHolderVar *) nlp->paramval)->phrels,
- leftrelids) &&
bms_is_subset(find_placeholder_info(root,
- (PlaceHolderVar *) nlp->paramval,
- false)->ph_eval_at,
+ (PlaceHolderVar *) nlp->paramval)->ph_eval_at,
leftrelids))
{
root->curOuterParams = foreach_delete_current(root->curOuterParams,
diff --git a/src/backend/optimizer/util/placeholder.c b/src/backend/optimizer/util/placeholder.c
index f0e8cd9965..44284b42a9 100644
--- a/src/backend/optimizer/util/placeholder.c
+++ b/src/backend/optimizer/util/placeholder.c
@@ -52,8 +52,8 @@ make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
* find_placeholder_info
* Fetch the PlaceHolderInfo for the given PHV
*
- * If the PlaceHolderInfo doesn't exist yet, create it if create_new_ph is
- * true, else throw an error.
+ * If the PlaceHolderInfo doesn't exist yet, create it if we haven't yet
+ * frozen the set of PlaceHolderInfos for the query; else throw an error.
*
* This is separate from make_placeholder_expr because subquery pullup has
* to make PlaceHolderVars for expressions that might not be used at all in
@@ -61,30 +61,30 @@ make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
* We build PlaceHolderInfos only for PHVs that are still present in the
* simplified query passed to query_planner().
*
- * Note: this should only be called after query_planner() has started. Also,
- * create_new_ph must not be true after deconstruct_jointree begins, because
- * make_outerjoininfo assumes that we already know about all placeholders.
+ * Note: this should only be called after query_planner() has started.
*/
PlaceHolderInfo *
-find_placeholder_info(PlannerInfo *root, PlaceHolderVar *phv,
- bool create_new_ph)
+find_placeholder_info(PlannerInfo *root, PlaceHolderVar *phv)
{
PlaceHolderInfo *phinfo;
Relids rels_used;
- ListCell *lc;
/* if this ever isn't true, we'd need to be able to look in parent lists */
Assert(phv->phlevelsup == 0);
- foreach(lc, root->placeholder_list)
+ /* Use placeholder_array to look up existing PlaceHolderInfo quickly */
+ if (phv->phid < root->placeholder_array_size)
+ phinfo = root->placeholder_array[phv->phid];
+ else
+ phinfo = NULL;
+ if (phinfo != NULL)
{
- phinfo = (PlaceHolderInfo *) lfirst(lc);
- if (phinfo->phid == phv->phid)
- return phinfo;
+ Assert(phinfo->phid == phv->phid);
+ return phinfo;
}
/* Not found, so create it */
- if (!create_new_ph)
+ if (root->placeholdersFrozen)
elog(ERROR, "too late to create a new PlaceHolderInfo");
phinfo = makeNode(PlaceHolderInfo);
@@ -115,8 +115,32 @@ find_placeholder_info(PlannerInfo *root, PlaceHolderVar *phv,
phinfo->ph_width = get_typavgwidth(exprType((Node *) phv->phexpr),
exprTypmod((Node *) phv->phexpr));
+ /* Add to placeholder_list and placeholder_array */
root->placeholder_list = lappend(root->placeholder_list, phinfo);
+ if (phinfo->phid >= root->placeholder_array_size)
+ {
+ /* Must allocate or enlarge placeholder_array */
+ int new_size;
+
+ new_size = root->placeholder_array_size ? root->placeholder_array_size * 2 : 8;
+ while (phinfo->phid >= new_size)
+ new_size *= 2;
+ if (root->placeholder_array)
+ {
+ root->placeholder_array = (PlaceHolderInfo **)
+ repalloc(root->placeholder_array,
+ sizeof(PlaceHolderInfo *) * new_size);
+ MemSet(root->placeholder_array + root->placeholder_array_size, 0,
+ sizeof(PlaceHolderInfo *) * (new_size - root->placeholder_array_size));
+ }
+ else
+ root->placeholder_array = (PlaceHolderInfo **)
+ palloc0(new_size * sizeof(PlaceHolderInfo *));
+ root->placeholder_array_size = new_size;
+ }
+ root->placeholder_array[phinfo->phid] = phinfo;
+
/*
* The PHV's contained expression may contain other, lower-level PHVs. We
* now know we need to get those into the PlaceHolderInfo list, too, so we
@@ -133,16 +157,13 @@ find_placeholder_info(PlannerInfo *root, PlaceHolderVar *phv,
*
* We don't need to look at the targetlist because build_base_rel_tlists()
* will already have made entries for any PHVs in the tlist.
- *
- * This is called before we begin deconstruct_jointree. Once we begin
- * deconstruct_jointree, all active placeholders must be present in
- * root->placeholder_list, because make_outerjoininfo and
- * update_placeholder_eval_levels require this info to be available
- * while we crawl up the join tree.
*/
void
find_placeholders_in_jointree(PlannerInfo *root)
{
+ /* This must be done before freezing the set of PHIs */
+ Assert(!root->placeholdersFrozen);
+
/* We need do nothing if the query contains no PlaceHolderVars */
if (root->glob->lastPHId != 0)
{
@@ -232,7 +253,7 @@ find_placeholders_in_expr(PlannerInfo *root, Node *expr)
continue;
/* Create a PlaceHolderInfo entry if there's not one already */
- (void) find_placeholder_info(root, phv, true);
+ (void) find_placeholder_info(root, phv);
}
list_free(vars);
}
@@ -359,7 +380,7 @@ fix_placeholder_input_needed_levels(PlannerInfo *root)
PVC_RECURSE_WINDOWFUNCS |
PVC_INCLUDE_PLACEHOLDERS);
- add_vars_to_targetlist(root, vars, phinfo->ph_eval_at, false);
+ add_vars_to_targetlist(root, vars, phinfo->ph_eval_at);
list_free(vars);
}
}
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 442c12acef..13e36dd0e1 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -990,7 +990,7 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
if (IsA(var, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) var;
- PlaceHolderInfo *phinfo = find_placeholder_info(root, phv, false);
+ PlaceHolderInfo *phinfo = find_placeholder_info(root, phv);
/* Is it still needed above this joinrel? */
if (bms_nonempty_difference(phinfo->ph_needed, relids))
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
index ebc6ce84b0..7db86c39ef 100644
--- a/src/backend/optimizer/util/var.c
+++ b/src/backend/optimizer/util/var.c
@@ -210,15 +210,8 @@ pull_varnos_walker(Node *node, pull_varnos_context *context)
if (phv->phlevelsup == 0)
{
- ListCell *lc;
-
- foreach(lc, context->root->placeholder_list)
- {
- phinfo = (PlaceHolderInfo *) lfirst(lc);
- if (phinfo->phid == phv->phid)
- break;
- phinfo = NULL;
- }
+ if (phv->phid < context->root->placeholder_array_size)
+ phinfo = context->root->placeholder_array[phv->phid];
}
if (phinfo == NULL)
{
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 3b065139e6..bdc7b50db9 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -357,6 +357,11 @@ struct PlannerInfo
/* list of PlaceHolderInfos */
List *placeholder_list;
+ /* array of PlaceHolderInfos indexed by phid */
+ struct PlaceHolderInfo **placeholder_array pg_node_attr(read_write_ignore, array_size(placeholder_array_size));
+ /* allocated size of array */
+ int placeholder_array_size pg_node_attr(read_write_ignore);
+
/* list of ForeignKeyOptInfos */
List *fkey_list;
@@ -449,6 +454,8 @@ struct PlannerInfo
bool hasPseudoConstantQuals;
/* true if we've made any of those */
bool hasAlternativeSubPlans;
+ /* true once we're no longer allowed to add PlaceHolderInfos */
+ bool placeholdersFrozen;
/* true if planning a recursive WITH item */
bool hasRecursion;
diff --git a/src/include/optimizer/placeholder.h b/src/include/optimizer/placeholder.h
index 39803ea41f..507dbc6175 100644
--- a/src/include/optimizer/placeholder.h
+++ b/src/include/optimizer/placeholder.h
@@ -20,7 +20,7 @@
extern PlaceHolderVar *make_placeholder_expr(PlannerInfo *root, Expr *expr,
Relids phrels);
extern PlaceHolderInfo *find_placeholder_info(PlannerInfo *root,
- PlaceHolderVar *phv, bool create_new_ph);
+ PlaceHolderVar *phv);
extern void find_placeholders_in_jointree(PlannerInfo *root);
extern void update_placeholder_eval_levels(PlannerInfo *root,
SpecialJoinInfo *new_sjinfo);
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index c4f61c1a09..1566f435b3 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -71,7 +71,7 @@ extern void add_base_rels_to_query(PlannerInfo *root, Node *jtnode);
extern void add_other_rels_to_query(PlannerInfo *root);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
- Relids where_needed, bool create_new_ph);
+ Relids where_needed);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
On Wed, Aug 17, 2022 at 4:57 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
On further thought, it seems better to maintain the index array
from the start, allowing complete replacement of the linear list
searches. We can add a separate bool flag to denote frozen-ness.
+1 for 0001 patch. Now we process plain Vars and PlaceHolderVars in a
more consistent way when building joinrel's tlist. And this change would
make it easier to build up phnullingrels for PHVs as we climb up the
join tree.
BTW, the comment just above the two calls to build_joinrel_tlist says:
* Create a new tlist containing just the vars that need to be output from
Here by 'vars' it means both plain Vars and PlaceHolderVars, right? If
not we may need to adjust this comment to also include PlaceHolderVars.
0002 patch looks good to me. Glad we can get rid of create_new_ph flag.
A minor comment is that seems we can get rid of phid inside
PlaceHolderInfo, since we do not do linear list searches any more. It's
some duplicate to the phid inside PlaceHolderVar. Currently there are
two places referencing PlaceHolderInfo->phid, remove_rel_from_query and
find_placeholder_info. We can use PlaceHolderVar->phid instead in both
the two places.
Thanks
Richard
more consistent way when building joinrel's tlist. And this change would
make it easier to build up phnullingrels for PHVs as we climb up the
join tree.
BTW, the comment just above the two calls to build_joinrel_tlist says:
* Create a new tlist containing just the vars that need to be output from
Here by 'vars' it means both plain Vars and PlaceHolderVars, right? If
not we may need to adjust this comment to also include PlaceHolderVars.
0002 patch looks good to me. Glad we can get rid of create_new_ph flag.
A minor comment is that seems we can get rid of phid inside
PlaceHolderInfo, since we do not do linear list searches any more. It's
some duplicate to the phid inside PlaceHolderVar. Currently there are
two places referencing PlaceHolderInfo->phid, remove_rel_from_query and
find_placeholder_info. We can use PlaceHolderVar->phid instead in both
the two places.
Thanks
Richard
Richard Guo <guofenglinux@gmail.com> writes:
> BTW, the comment just above the two calls to build_joinrel_tlist says:
> * Create a new tlist containing just the vars that need to be output from
> Here by 'vars' it means both plain Vars and PlaceHolderVars, right? If
> not we may need to adjust this comment to also include PlaceHolderVars.
I think it did intend just Vars because that's all that
build_joinrel_tlist did; but we really should have updated it when we
invented PlaceHolderVars, and even more so now that build_joinrel_tlist
adds PHVs too. I changed the wording.
> A minor comment is that seems we can get rid of phid inside
> PlaceHolderInfo, since we do not do linear list searches any more. It's
> some duplicate to the phid inside PlaceHolderVar. Currently there are
> two places referencing PlaceHolderInfo->phid, remove_rel_from_query and
> find_placeholder_info. We can use PlaceHolderVar->phid instead in both
> the two places.
Meh, I'm not excited about that. I don't think that the phid field
is only there to make the search loops faster; it's the basic
identity of the PlaceHolderInfo.
regards, tom lane
Here's a rebase up to HEAD, mainly to get the cfbot back in sync
as to what's the live patch.
I went ahead and pushed improve-adjust_appendrel_attrs_multilevel.patch,
as that seemed uncontroversial and independently useful. So that's
gone from this patchset. I also cleaned up the mess with phnullingrels
in PHVs created for join tlists, as we discussed. No other interesting
changes.
regards, tom lane
commit 0977c58a9c582a76eb5da070b2c99c41a268f828
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu Aug 18 12:49:18 2022 -0400
Add overview documentation.
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index 41c120e0cd..2b30d22aed 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -295,6 +295,191 @@ Therefore, we don't merge FROM-lists if the result would have too many
FROM-items in one list.
+Vars and PlaceHolderVars
+------------------------
+
+A Var node is simply the parse-tree representation of a table column
+reference. However, in the presence of outer joins, that concept is
+more subtle than it might seem. We need to distinguish the values of
+a Var "above" and "below" any outer join that could force the Var to
+null. As an example, consider
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE foo(t2.z)
+
+(Assume foo() is not strict, so that we can't reduce the left join to
+a plain join.) A naive implementation might try to push the foo(t2.z)
+call down to the scan of t2, but that is not correct because
+(a) what foo() should actually see for a null-extended join row is NULL,
+and (b) if foo() returns false, we should suppress the t1 row from the
+join altogether, not emit it with a null-extended t2 row. On the other
+hand, it *would* be correct (and desirable) to push the call down to
+the scan level if the query were
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y AND foo(t2.z))
+
+This motivates considering "t2.z" within the left join's ON clause
+to be a different value from "t2.z" outside the JOIN clause. The
+former can be identified with t2.z as seen at the relation scan level,
+but the latter can't.
+
+Another example occurs in connection with EquivalenceClasses (discussed
+below). Given
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE t1.x = 42
+
+we would like to put t1.x and t2.y and 42 into the same EquivalenceClass
+and then derive "t2.y = 42" to use as a restriction clause for the scan
+of t2. However, it'd be wrong to conclude that t2.y will always have
+the value 42, or that it's equal to t1.x in every joined row. We can
+solve this problem by deeming that "t2.y" in the ON clause refers to
+the relation-scan-level value of t2.y, but not to the value that y will
+have in joined rows, where it might be NULL rather than equal to t1.x.
+
+Therefore, Var nodes are decorated with "varnullingrels", which are sets
+of the rangetable indexes of outer joins that potentially null this Var
+at the point where it appears in the query. (Using a set, not an
+ordered list, is fine since it doesn't matter which join forced the
+value to null; and that avoids having to change the representation when
+we consider different outer-join orders.) In the examples above, all
+occurrences of t1.x would have empty varnullingrels, since the left join
+doesn't null t1. The t2 references within the JOIN ON clauses would
+also have empty varnullingrels, but other references to t2 columns would
+be labeled with the index of the JOIN's rangetable entry (RTE), so that
+they'd be understood as potentially different from the t2 values seen at
+scan level. Labeling t2.z in the WHERE clause with the JOIN's RT index
+lets us recognize that that occurrence of foo(t2.z) cannot be pushed
+down to the t2 scan level: we cannot evaluate that value at the scan
+level, but only after the join has been done.
+
+For LEFT and RIGHT outer joins, only Vars coming from the nullable side
+of the join are marked with that join's RT index. For FULL joins, all
+Vars are marked. (Such marking doesn't let us tell which side of the
+full join a Var came from; but that information can be found elsewhere
+at need.)
+
+Notionally, a Var having nonempty varnullingrels can be thought of as
+ CASE WHEN any-of-these-outer-joins-produced-a-null-extended-row
+ THEN NULL
+ ELSE the-scan-level-value-of-the-column
+ END
+It's only notional, because no such calculation is ever done explicitly.
+In a finished plan, Vars occurring in scan-level plan nodes represent
+the actual table column values, but upper-level Vars are always
+references to outputs of lower-level plan nodes. When a join node emits
+a null-extended row, it just returns nulls for the relevant output
+columns rather than copying up values from its input. Because we don't
+ever have to do this calculation explicitly, it's not necessary to
+distinguish which side of an outer join got null-extended, which'd
+otherwise be essential information for FULL JOIN cases.
+
+Outer join identity 3 (discussed above) complicates this picture
+a bit. In the form
+ A leftjoin (B leftjoin C on (Pbc)) on (Pab)
+all of the Vars in clauses Pbc and Pab will have empty varnullingrels,
+but if we start with
+ (A leftjoin B on (Pab)) leftjoin C on (Pbc)
+then the parser will have marked Pbc's B Vars with the A/B join's
+RT index, making this form artificially different from the first.
+We resolve this by, after noting that Pbc is strict, running
+through that clause and removing any varnullingrels references to
+left joins in the lefthand side. That makes the clause equivalent
+to what it would have looked like if the first form were presented,
+so that we can freely consider both join orders. However, because
+we have done this, if we do construct a plan based on the second
+join order then we cannot cross-check that B Vars appearing above
+the A/B join are all marked with that join's RT index. That would
+be a useful cross-check to have to catch planner bugs, but it
+doesn't seem useful enough to justify the extra complication of
+devising a representation that would support it.
+
+Outer joins also complicate handling of subquery pull-up. Consider
+
+ SELECT ..., ss.x FROM tab1
+ LEFT JOIN (SELECT *, 42 AS x FROM tab2) ss ON ...
+
+We want to be able to pull up the subquery as discussed previously,
+but we can't just replace the "ss.x" Var in the top-level SELECT list
+with the constant 42. That'd result in always emitting 42, rather
+than emitting NULL in null-extended join rows.
+
+To solve this, we introduce the concept of PlaceHolderVars.
+A PlaceHolderVar is somewhat like a Var, in that its value originates
+at a relation scan level and can then be forced to null by higher-level
+outer joins; hence PlaceHolderVars carry a set of nulling rel IDs just
+like Vars. Unlike a Var, whose original value comes from a table,
+a PlaceHolderVar's original value is defined by a query-determined
+expression ("42" in this example); so we represent the PlaceHolderVar
+as a node with that expression as child. We insert a PlaceHolderVar
+whenever subquery pullup needs to replace a subquery-referencing Var
+that has nonempty varnullingrels with an expression that is not simply a
+Var. (When the replacement expression is a pulled-up Var, we can just
+add the replaced Var's varnullingrels to its set. Also, if the replaced
+Var has empty varnullingrels, we don't need a PlaceHolderVar: there is
+nothing that'd force the value to null, so the pulled-up expression is
+fine to use as-is.) In a finished plan, a PlaceHolderVar becomes just
+the contained expression at whatever plan level it's supposed to be
+evaluated at, and then upper-level occurrences are replaced by
+references to that output column of the lower plan level. That causes
+the value to go to null when appropriate at an outer join, in the same
+way as for Vars. Thus, PlaceHolderVars are never seen outside the
+planner.
+
+PlaceHolderVars (PHVs) are more complicated than Vars in another way:
+their original value might need to be calculated at a join, not a
+base-level relation scan. This can happen if a pulled-up subquery
+contains a join. Because of this, a PHV can create a join order
+constraint that wouldn't otherwise exist, to ensure that it can
+be calculated before it is used. A PHV's expression can also contain
+LATERAL references, adding complications that are discussed below.
+
+
+Relation Identification and Qual Clause Placement
+-------------------------------------------------
+
+A qual clause obtained from WHERE or JOIN/ON can be enforced at the lowest
+scan or join level that includes all relations used in the clause. For
+this purpose we consider that outer joins listed in varnullingrels or
+phnullingrels are used in the clause, since we can't compute the qual's
+result correctly until we know whether such Vars have gone to null.
+
+The one exception to this general rule is that a non-degenerate outer
+JOIN/ON qual (one that references the non-nullable side of the join)
+cannot be enforced below that join, even if it doesn't reference the
+nullable side. Pushing it down into the non-nullable side would result
+in rows disappearing from the join's result, rather than appearing as
+null-extended rows. To handle that, when we identify such a qual we
+artificially add the join's minimum input relid set to the set of
+relations it is considered to use, forcing it to be evaluated exactly at
+that join level. The same happens for outer-join quals that mention no
+relations at all.
+
+When attaching a qual clause to a join plan node that is performing
+an outer join, the qual clause is considered a "join clause" (that
+is, it is applied before the join) if it does not use that specific
+outer join, or a "filter clause" (applied after the join) if it does
+use that outer join.
+
+These things lead us to identify join relations within the planner
+by the sets of base relation RT indexes plus outer join RT indexes
+that they include. In that way, the sets of relations used by qual
+clauses can be directly compared to join relations' relid sets to
+see where to place the clauses. These identifying sets are unique
+because, for any given collection of base relations, there is only
+one valid set of outer joins to have performed along the way to
+joining that set of base relations (although the order of applying
+them could vary, as discussed above).
+
+SEMI joins do not have RT indexes, because they are artifacts made by
+the planner rather than the parser. (We could create rangetable
+entries for them, but there seems no need at present.) This does not
+cause a problem for qual placement, because the nullable side of a
+semijoin is not referenceable from above the join, so there is never a
+need to cite it in varnullingrels or phnullingrels. It does not cause
+a problem for join relation identification either, since again whether
+a semijoin has been completed is implicit in the set of base relations
+included in the join.
+
+
Optimizer Functions
-------------------
@@ -437,11 +622,10 @@ inputs.
EquivalenceClasses
------------------
-During the deconstruct_jointree() scan of the query's qual clauses, we look
-for mergejoinable equality clauses A = B whose applicability is not delayed
-by an outer join; these are called "equivalence clauses". When we find
-one, we create an EquivalenceClass containing the expressions A and B to
-record this knowledge. If we later find another equivalence clause B = C,
+During the deconstruct_jointree() scan of the query's qual clauses, we
+look for mergejoinable equality clauses A = B. When we find one, we
+create an EquivalenceClass containing the expressions A and B to record
+that they are equal. If we later find another equivalence clause B = C,
we add C to the existing EquivalenceClass for {A B}; this may require
merging two existing EquivalenceClasses. At the end of the scan, we have
sets of values that are known all transitively equal to each other. We can
@@ -473,15 +657,26 @@ asserts that at any plan node where more than one of its member values
can be computed, output rows in which the values are not all equal may
be discarded without affecting the query result. (We require all levels
of the plan to enforce EquivalenceClasses, hence a join need not recheck
-equality of values that were computable by one of its children.) For an
-ordinary EquivalenceClass that is "valid everywhere", we can further infer
-that the values are all non-null, because all mergejoinable operators are
-strict. However, we also allow equivalence clauses that appear below the
-nullable side of an outer join to form EquivalenceClasses; for these
-classes, the interpretation is that either all the values are equal, or
-all (except pseudo-constants) have gone to null. (This requires a
-limitation that non-constant members be strict, else they might not go
-to null when the other members do.) Consider for example
+equality of values that were computable by one of its children.)
+
+We can further infer that the values are all non-null, because all
+mergejoinable operators are strict. This is a little tricky in the
+presence of outer joins. Consider
+
+ SELECT *
+ FROM a LEFT JOIN
+ (SELECT * FROM b LEFT JOIN c ON b.y = c.z WHERE b.y = 10) ss
+ ON a.x = ss.y
+ WHERE a.x = 42;
+
+We can form the EquivalenceClass {b.y c.z 10} and thereby apply c.z = 10
+while scanning c. However it would be incorrect to conclude that a.x
+is also a member of that EquivalenceClass. Instead, we form a second
+EquivalenceClass {a.x ss.y 42}, where (as discussed earlier) ss.y
+references the same table column as b.y but has a different
+varnullingrels label and is therefore considered a distinct Var.
+
+If the lower join were INNER:
SELECT *
FROM a LEFT JOIN
@@ -489,40 +684,23 @@ to null when the other members do.) Consider for example
ON a.x = ss.y
WHERE a.x = 42;
-We can form the below-outer-join EquivalenceClass {b.y c.z 10} and thereby
-apply c.z = 10 while scanning c. (The reason we disallow outerjoin-delayed
-clauses from forming EquivalenceClasses is exactly that we want to be able
-to push any derived clauses as far down as possible.) But once above the
-outer join it's no longer necessarily the case that b.y = 10, and thus we
-cannot use such EquivalenceClasses to conclude that sorting is unnecessary
-(see discussion of PathKeys below).
-
-In this example, notice also that a.x = ss.y (really a.x = b.y) is not an
-equivalence clause because its applicability to b is delayed by the outer
-join; thus we do not try to insert b.y into the equivalence class {a.x 42}.
-But since we see that a.x has been equated to 42 above the outer join, we
-are able to form a below-outer-join class {b.y 42}; this restriction can be
-added because no b/c row not having b.y = 42 can contribute to the result
-of the outer join, and so we need not compute such rows. Now this class
-will get merged with {b.y c.z 10}, leading to the contradiction 10 = 42,
-which lets the planner deduce that the b/c join need not be computed at all
-because none of its rows can contribute to the outer join. (This gets
-implemented as a gating Result filter, since more usually the potential
-contradiction involves Param values rather than just Consts, and thus has
-to be checked at runtime.)
+then ss.y is not any different from b.y and we'd end up with the
+EquivalenceClass {a.x b.y c.z 10 42}. This leads to the contradiction
+10 = 42, which lets the planner deduce that the b/c join need not be
+computed at all because none of its rows can contribute to the outer
+join. (This gets implemented as a gating Result filter, since more
+usually the potential contradiction involves Param values rather than
+just Consts, and thus has to be checked at runtime.)
To aid in determining the sort ordering(s) that can work with a mergejoin,
we mark each mergejoinable clause with the EquivalenceClasses of its left
-and right inputs. For an equivalence clause, these are of course the same
-EquivalenceClass. For a non-equivalence mergejoinable clause (such as an
-outer-join qualification), we generate two separate EquivalenceClasses for
-the left and right inputs. This may result in creating single-item
-equivalence "classes", though of course these are still subject to merging
-if other equivalence clauses are later found to bear on the same
-expressions.
-
-Another way that we may form a single-item EquivalenceClass is in creation
-of a PathKey to represent a desired sort order (see below). This is a bit
+and right inputs. (These are in fact always the same EquivalenceClass.)
+
+In some cases we will form single-item EquivalenceClasses. This happens
+if an ORDER BY or GROUP BY key is not mentioned in any equivalence
+clause. We need to reason about sort orders in such queries, and our
+representation of sort ordering is a PathKey (see below) which uses an
+EquivalenceClass, so we have to make an EquivalenceClass. This is a bit
different from the above cases because such an EquivalenceClass might
contain an aggregate function or volatile expression. (A clause containing
a volatile function will never be considered mergejoinable, even if its top
@@ -579,7 +757,7 @@ Index scans have Path.pathkeys that represent the chosen index's ordering,
if any. A single-key index would create a single-PathKey list, while a
multi-column index generates a list with one element per key index column.
Non-key columns specified in the INCLUDE clause of covering indexes don't
-have corresponding PathKeys in the list, because the have no influence on
+have corresponding PathKeys in the list, because they have no influence on
index ordering. (Actually, since an index can be scanned either forward or
backward, there are two possible sort orders and two possible PathKey lists
it can generate.)
@@ -655,14 +833,9 @@ redundancy, we save time and improve planning, since the planner will more
easily recognize equivalent orderings as being equivalent.
Another interesting property is that if the underlying EquivalenceClass
-contains a constant and is not below an outer join, then the pathkey is
-completely redundant and need not be sorted by at all! Every row must
-contain the same constant value, so there's no need to sort. (If the EC is
-below an outer join, we still have to sort, since some of the rows might
-have gone to null and others not. In this case we must be careful to pick
-a non-const member to sort by. The assumption that all the non-const
-members go to null at the same plan level is critical here, else they might
-not produce the same sort order.) This might seem pointless because users
+contains a constant, then the pathkey is completely redundant and need
+not be sorted by at all! Every row must contain the same value, so
+there's no need to sort. This might seem pointless because users
are unlikely to write "... WHERE x = 42 ORDER BY x", but it allows us to
recognize when particular index columns are irrelevant to the sort order:
if we have "... WHERE x = 42 ORDER BY y", scanning an index on (x,y)
@@ -670,15 +843,6 @@ produces correctly ordered data without a sort step. We used to have very
ugly ad-hoc code to recognize that in limited contexts, but discarding
constant ECs from pathkeys makes it happen cleanly and automatically.
-You might object that a below-outer-join EquivalenceClass doesn't always
-represent the same values at every level of the join tree, and so using
-it to uniquely identify a sort order is dubious. This is true, but we
-can avoid dealing with the fact explicitly because we always consider that
-an outer join destroys any ordering of its nullable inputs. Thus, even
-if a path was sorted by {a.x} below an outer join, we'll re-sort if that
-sort ordering was important; and so using the same PathKey for both sort
-orderings doesn't create any real problem.
-
Order of processing for EquivalenceClasses and PathKeys
-------------------------------------------------------
commit b2c019a596dc988bd8291ee95f17118397121839
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu Aug 18 13:05:54 2022 -0400
Add Var.varnullingrels and PlaceHolderVar.phnullingrels fields.
These fields are always empty as of this commit, so they don't
affect any behavior, even though equal() will compare them.
Update backend/nodes/ and backend/rewrite/ infrastructure as needed.
Also add some rewrite functions we'll need later.
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index 28288dcfc1..19606c495f 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -81,11 +81,13 @@ makeVar(int varno,
var->varlevelsup = varlevelsup;
/*
- * Only a few callers need to make Var nodes with varnosyn/varattnosyn
- * different from varno/varattno. We don't provide separate arguments for
- * them, but just initialize them to the given varno/varattno. This
- * reduces code clutter and chance of error for most callers.
+ * Only a few callers need to make Var nodes with non-null varnullingrels,
+ * or with varnosyn/varattnosyn different from varno/varattno. We don't
+ * provide separate arguments for them, but just initialize them to NULL
+ * and the given varno/varattno. This reduces code clutter and chance of
+ * error for most callers.
*/
+ var->varnullingrels = NULL;
var->varnosyn = (Index) varno;
var->varattnosyn = varattno;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index c334daae39..0b0f5c9752 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -2853,6 +2853,7 @@ expression_tree_mutator(Node *node,
Var *newnode;
FLATCOPY(newnode, var, Var);
+ /* Assume we need not copy the varnullingrels bitmapset */
return (Node *) newnode;
}
break;
@@ -3448,7 +3449,7 @@ expression_tree_mutator(Node *node,
FLATCOPY(newnode, phv, PlaceHolderVar);
MUTATE(newnode->phexpr, phv->phexpr, Expr *);
- /* Assume we need not copy the relids bitmapset */
+ /* Assume we need not copy the relids bitmapsets */
return (Node *) newnode;
}
break;
diff --git a/src/backend/rewrite/rewriteManip.c b/src/backend/rewrite/rewriteManip.c
index 101c39553a..a0a0026469 100644
--- a/src/backend/rewrite/rewriteManip.c
+++ b/src/backend/rewrite/rewriteManip.c
@@ -40,6 +40,13 @@ typedef struct
int win_location;
} locate_windowfunc_context;
+typedef struct
+{
+ Bitmapset *removable_relids;
+ Bitmapset *except_relids;
+ int sublevels_up;
+} remove_nulling_relids_context;
+
static bool contain_aggs_of_level_walker(Node *node,
contain_aggs_of_level_context *context);
static bool locate_agg_of_level_walker(Node *node,
@@ -50,6 +57,9 @@ static bool locate_windowfunc_walker(Node *node,
static bool checkExprHasSubLink_walker(Node *node, void *context);
static Relids offset_relid_set(Relids relids, int offset);
static Relids adjust_relid_set(Relids relids, int oldrelid, int newrelid);
+static bool get_nulling_relids_walker(Node *node, Bitmapset **context);
+static Node *remove_nulling_relids_mutator(Node *node,
+ remove_nulling_relids_context *context);
/*
@@ -348,6 +358,8 @@ OffsetVarNodes_walker(Node *node, OffsetVarNodes_context *context)
if (var->varlevelsup == context->sublevels_up)
{
var->varno += context->offset;
+ var->varnullingrels = offset_relid_set(var->varnullingrels,
+ context->offset);
if (var->varnosyn > 0)
var->varnosyn += context->offset;
}
@@ -386,6 +398,8 @@ OffsetVarNodes_walker(Node *node, OffsetVarNodes_context *context)
{
phv->phrels = offset_relid_set(phv->phrels,
context->offset);
+ phv->phnullingrels = offset_relid_set(phv->phnullingrels,
+ context->offset);
}
/* fall through to examine children */
}
@@ -510,11 +524,13 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context)
{
Var *var = (Var *) node;
- if (var->varlevelsup == context->sublevels_up &&
- var->varno == context->rt_index)
+ if (var->varlevelsup == context->sublevels_up)
{
- var->varno = context->new_index;
- /* If the syntactic referent is same RTE, fix it too */
+ if (var->varno == context->rt_index)
+ var->varno = context->new_index;
+ var->varnullingrels = adjust_relid_set(var->varnullingrels,
+ context->rt_index,
+ context->new_index);
if (var->varnosyn == context->rt_index)
var->varnosyn = context->new_index;
}
@@ -557,6 +573,9 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context)
phv->phrels = adjust_relid_set(phv->phrels,
context->rt_index,
context->new_index);
+ phv->phnullingrels = adjust_relid_set(phv->phnullingrels,
+ context->rt_index,
+ context->new_index);
}
/* fall through to examine children */
}
@@ -833,7 +852,8 @@ rangeTableEntry_used_walker(Node *node,
Var *var = (Var *) node;
if (var->varlevelsup == context->sublevels_up &&
- var->varno == context->rt_index)
+ (var->varno == context->rt_index ||
+ bms_is_member(context->rt_index, var->varnullingrels)))
return true;
return false;
}
@@ -1061,6 +1081,154 @@ AddInvertedQual(Query *parsetree, Node *qual)
}
+/*
+ * get_nulling_relids collects all the level-zero RT indexes mentioned in
+ * Var.varnullingrels and PlaceHolderVar.phnullingrels fields within the
+ * given expression.
+ */
+Bitmapset *
+get_nulling_relids(Node *node)
+{
+ Bitmapset *result = NULL;
+
+ (void) get_nulling_relids_walker(node, &result);
+ return result;
+}
+
+static bool
+get_nulling_relids_walker(Node *node, Bitmapset **context)
+{
+ if (node == NULL)
+ return false;
+ if (IsA(node, Var))
+ {
+ Var *var = (Var *) node;
+
+ if (var->varlevelsup == 0)
+ *context = bms_add_members(*context, var->varnullingrels);
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ if (phv->phlevelsup == 0)
+ *context = bms_add_members(*context, phv->phnullingrels);
+ }
+
+ /*
+ * Currently, this is only used after the planner has converted SubLinks
+ * to SubPlans, so we don't need to support recursion into sub-Queries; so
+ * no sublevels_up counting is needed.
+ */
+ Assert(!IsA(node, SubLink));
+ Assert(!IsA(node, Query));
+ return expression_tree_walker(node, get_nulling_relids_walker, context);
+}
+
+/*
+ * remove_nulling_relids removes mentions of the specified RT index(es)
+ * in Var.varnullingrels and PlaceHolderVar.phnullingrels fields within
+ * the given expression, except in nodes belonging to rels listed in
+ * except_relids.
+ *
+ * XXX consider making this a destructive walker.
+ */
+Node *
+remove_nulling_relids(Node *node, Bitmapset *removable_relids,
+ Bitmapset *except_relids)
+{
+ remove_nulling_relids_context context;
+
+ context.removable_relids = removable_relids;
+ context.except_relids = except_relids;
+ context.sublevels_up = 0;
+ return query_or_expression_tree_mutator(node,
+ remove_nulling_relids_mutator,
+ &context,
+ 0);
+}
+
+static Node *
+remove_nulling_relids_mutator(Node *node,
+ remove_nulling_relids_context *context)
+{
+ if (node == NULL)
+ return NULL;
+ if (IsA(node, Var))
+ {
+ Var *var = (Var *) node;
+
+ if (var->varlevelsup == context->sublevels_up &&
+ !bms_is_member(var->varno, context->except_relids) &&
+ bms_overlap(var->varnullingrels, context->removable_relids))
+ {
+ Relids newnullingrels = bms_difference(var->varnullingrels,
+ context->removable_relids);
+
+ /* Micro-optimization: ensure nullingrels is NULL if empty */
+ if (bms_is_empty(newnullingrels))
+ newnullingrels = NULL;
+ /* Copy the Var ... */
+ var = copyObject(var);
+ /* ... and replace the copy's varnullingrels field */
+ var->varnullingrels = newnullingrels;
+ return (Node *) var;
+ }
+ /* Otherwise fall through to copy the Var normally */
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ if (phv->phlevelsup == context->sublevels_up &&
+ !bms_overlap(phv->phrels, context->except_relids))
+ {
+ Relids newnullingrels = bms_difference(phv->phnullingrels,
+ context->removable_relids);
+
+ /*
+ * Micro-optimization: ensure nullingrels is NULL if empty.
+ *
+ * Note: it might seem desirable to remove the PHV altogether if
+ * phnullingrels goes to empty. Currently we dare not do that
+ * because we use PHVs in some cases to enforce separate identity
+ * of subexpressions; see wrap_non_vars usages in prepjointree.c.
+ */
+ if (bms_is_empty(newnullingrels))
+ newnullingrels = NULL;
+ /* Copy the PlaceHolderVar and mutate what's below ... */
+ phv = (PlaceHolderVar *)
+ expression_tree_mutator(node,
+ remove_nulling_relids_mutator,
+ (void *) context);
+ /* ... and replace the copy's phnullingrels field */
+ phv->phnullingrels = newnullingrels;
+ /* We must also update phrels, if it contains a removable RTI */
+ phv->phrels = bms_difference(phv->phrels,
+ context->removable_relids);
+ Assert(!bms_is_empty(phv->phrels));
+ return (Node *) phv;
+ }
+ /* Otherwise fall through to copy the PlaceHolderVar normally */
+ }
+ else if (IsA(node, Query))
+ {
+ /* Recurse into RTE or sublink subquery */
+ Query *newnode;
+
+ context->sublevels_up++;
+ newnode = query_tree_mutator((Query *) node,
+ remove_nulling_relids_mutator,
+ (void *) context,
+ 0);
+ context->sublevels_up--;
+ return (Node *) newnode;
+ }
+ return expression_tree_mutator(node, remove_nulling_relids_mutator,
+ (void *) context);
+}
+
+
/*
* replace_rte_variables() finds all Vars in an expression tree
* that reference a particular RTE, and replaces them with substitute
diff --git a/src/backend/utils/misc/queryjumble.c b/src/backend/utils/misc/queryjumble.c
index eeaa0b31fe..e517e0363c 100644
--- a/src/backend/utils/misc/queryjumble.c
+++ b/src/backend/utils/misc/queryjumble.c
@@ -381,6 +381,11 @@ JumbleExpr(JumbleState *jstate, Node *node)
APP_JUMB(var->varno);
APP_JUMB(var->varattno);
APP_JUMB(var->varlevelsup);
+
+ /*
+ * We can omit varnullingrels, because it's fully determined
+ * by varno/varlevelsup plus the Var's query location.
+ */
}
break;
case T_Const:
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 294cfe9c47..024ba376b4 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2598,10 +2598,15 @@ typedef struct MergeScanSelCache
* of a plan tree. This is used during planning to represent the contained
* expression. At the end of the planning process it is replaced by either
* the contained expression or a Var referring to a lower-level evaluation of
- * the contained expression. Typically the evaluation occurs below an outer
+ * the contained expression. Generally the evaluation occurs below an outer
* join, and Var references above the outer join might thereby yield NULL
* instead of the expression value.
*
+ * phrels and phlevelsup correspond to the varno/varlevelsup fields of a
+ * plain Var, except that phrels has to be a relid set since the evaluation
+ * level of a PlaceHolderVar might be a join rather than a base relation.
+ * Likewise, phnullingrels corresponds to varnullingrels.
+ *
* Although the planner treats this as an expression node type, it is not
* recognized by the parser or executor, so we declare it here rather than
* in primnodes.h.
@@ -2614,8 +2619,10 @@ typedef struct MergeScanSelCache
* PHV. Another way in which it can happen is that initplan sublinks
* could get replaced by differently-numbered Params when sublink folding
* is done. (The end result of such a situation would be some
- * unreferenced initplans, which is annoying but not really a problem.) On
- * the same reasoning, there is no need to examine phrels.
+ * unreferenced initplans, which is annoying but not really a problem.) On
+ * the same reasoning, there is no need to examine phrels. But we do need
+ * to compare phnullingrels, as that represents effects that are external
+ * to the original value of the PHV.
*/
typedef struct PlaceHolderVar
@@ -2625,9 +2632,12 @@ typedef struct PlaceHolderVar
/* the represented expression */
Expr *phexpr pg_node_attr(equal_ignore);
- /* base relids syntactically within expr src */
+ /* base+OJ relids syntactically within expr src */
Relids phrels pg_node_attr(equal_ignore);
+ /* RT indexes of outer joins that can null PHV's value */
+ Relids phnullingrels;
+
/* ID for PHV (unique within planner run) */
Index phid;
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index 3aa96bb685..3e42df7464 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -189,6 +189,14 @@ typedef struct Expr
* row identity information during UPDATE/DELETE. This value should never
* be seen outside the planner.
*
+ * varnullingrels is the set of RT indexes of outer joins that can force
+ * the Var's value to null (at the point where it appears in the query).
+ * See optimizer/README for discussion of that.
+ *
+ * varlevelsup is greater than zero in Vars that represent outer references.
+ * Note that it affects the meaning of all of varno, varnullingrels, and
+ * varnosyn, all of which refer to the range table of that query level.
+ *
* In the parser, varnosyn and varattnosyn are either identical to
* varno/varattno, or they specify the column's position in an aliased JOIN
* RTE that hides the semantic referent RTE's refname. This is a syntactic
@@ -231,6 +239,8 @@ typedef struct Var
int32 vartypmod;
/* OID of collation, or InvalidOid if none */
Oid varcollid;
+ /* RT indexes of outer joins that can replace the Var's value with null */
+ Bitmapset *varnullingrels;
/*
* for subquery variables referencing outer relations; 0 in a normal var,
diff --git a/src/include/rewrite/rewriteManip.h b/src/include/rewrite/rewriteManip.h
index 98b9b3a288..a3f902c1bb 100644
--- a/src/include/rewrite/rewriteManip.h
+++ b/src/include/rewrite/rewriteManip.h
@@ -63,6 +63,10 @@ extern bool contain_windowfuncs(Node *node);
extern int locate_windowfunc(Node *node);
extern bool checkExprHasSubLink(Node *node);
+extern Bitmapset *get_nulling_relids(Node *node);
+extern Node *remove_nulling_relids(Node *node, Bitmapset *removable_relids,
+ Bitmapset *except_relids);
+
extern Node *replace_rte_variables(Node *node,
int target_varno, int sublevels_up,
replace_rte_variables_callback callback,
commit 222182ec2852564d096336ebed0706f71fadd911
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu Aug 18 13:13:38 2022 -0400
Teach the parser to fill Var.varnullingrels correctly.
Vars emitted by the parser are now marked with RT indexes of outer
joins that can null them. (This is done purely according to the
syntax of the query; we don't consider whether an outer join could
be strength-reduced, for example.)
Although the result of this step compiles, it will fail some
regression tests due to the planner not yet knowing what to do.
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 6688c2a865..dff3b1e349 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -670,6 +670,7 @@ transformInsertStmt(ParseState *pstate, InsertStmt *stmt)
*/
sub_pstate->p_rtable = sub_rtable;
sub_pstate->p_joinexprs = NIL; /* sub_rtable has no joins */
+ sub_pstate->p_nullingrels = NIL;
sub_pstate->p_namespace = sub_namespace;
sub_pstate->p_resolve_unknowns = false;
@@ -851,7 +852,7 @@ transformInsertStmt(ParseState *pstate, InsertStmt *stmt)
/*
* Generate list of Vars referencing the RTE
*/
- exprList = expandNSItemVars(nsitem, 0, -1, NULL);
+ exprList = expandNSItemVars(pstate, nsitem, 0, -1, NULL);
/*
* Re-apply any indirection on the target column specs to the Vars
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index b85fbebd00..2eb577cacc 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -52,7 +52,8 @@
#include "utils/syscache.h"
-static int extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
+static int extractRemainingColumns(ParseState *pstate,
+ ParseNamespaceColumn *src_nscolumns,
List *src_colnames,
List **src_colnos,
List **res_colnames, List **res_colvars,
@@ -75,9 +76,11 @@ static ParseNamespaceItem *getNSItemForSpecialRelationTypes(ParseState *pstate,
static Node *transformFromClauseItem(ParseState *pstate, Node *n,
ParseNamespaceItem **top_nsitem,
List **namespace);
-static Var *buildVarFromNSColumn(ParseNamespaceColumn *nscol);
+static Var *buildVarFromNSColumn(ParseState *pstate,
+ ParseNamespaceColumn *nscol);
static Node *buildMergedJoinVar(ParseState *pstate, JoinType jointype,
Var *l_colvar, Var *r_colvar);
+static void markRelsAsNulledBy(ParseState *pstate, Node *n, int jindex);
static void setNamespaceColumnVisibility(List *namespace, bool cols_visible);
static void setNamespaceLateralState(List *namespace,
bool lateral_only, bool lateral_ok);
@@ -249,7 +252,8 @@ setTargetTable(ParseState *pstate, RangeVar *relation,
* Returns the number of columns added.
*/
static int
-extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
+extractRemainingColumns(ParseState *pstate,
+ ParseNamespaceColumn *src_nscolumns,
List *src_colnames,
List **src_colnos,
List **res_colnames, List **res_colvars,
@@ -285,7 +289,8 @@ extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
*src_colnos = lappend_int(*src_colnos, attnum);
*res_colnames = lappend(*res_colnames, lfirst(lc));
*res_colvars = lappend(*res_colvars,
- buildVarFromNSColumn(src_nscolumns + attnum - 1));
+ buildVarFromNSColumn(pstate,
+ src_nscolumns + attnum - 1));
/* Copy the input relation's nscolumn data for this column */
res_nscolumns[colcount] = src_nscolumns[attnum - 1];
colcount++;
@@ -1292,8 +1297,7 @@ transformFromClauseItem(ParseState *pstate, Node *n,
{
/*
* JOIN/USING (or NATURAL JOIN, as transformed above). Transform
- * the list into an explicit ON-condition, and generate a list of
- * merged result columns.
+ * the list into an explicit ON-condition.
*/
List *ucols = j->usingClause;
List *l_usingvars = NIL;
@@ -1311,8 +1315,6 @@ transformFromClauseItem(ParseState *pstate, Node *n,
int r_index = -1;
Var *l_colvar,
*r_colvar;
- Node *u_colvar;
- ParseNamespaceColumn *res_nscolumn;
Assert(u_colname[0] != '\0');
@@ -1376,17 +1378,109 @@ transformFromClauseItem(ParseState *pstate, Node *n,
u_colname)));
r_colnos = lappend_int(r_colnos, r_index + 1);
- l_colvar = buildVarFromNSColumn(l_nscolumns + l_index);
+ /* Build Vars to use in the generated JOIN ON clause */
+ l_colvar = buildVarFromNSColumn(pstate, l_nscolumns + l_index);
l_usingvars = lappend(l_usingvars, l_colvar);
- r_colvar = buildVarFromNSColumn(r_nscolumns + r_index);
+ r_colvar = buildVarFromNSColumn(pstate, r_nscolumns + r_index);
r_usingvars = lappend(r_usingvars, r_colvar);
+ /*
+ * While we're here, add column names to the res_colnames
+ * list. It's a bit ugly to do this here while the
+ * corresponding res_colvars entries are not made till later,
+ * but doing this later would require an additional traversal
+ * of the usingClause list.
+ */
res_colnames = lappend(res_colnames, lfirst(ucol));
+ }
+
+ /* Construct the generated JOIN ON clause */
+ j->quals = transformJoinUsingClause(pstate,
+ l_usingvars,
+ r_usingvars);
+ }
+ else if (j->quals)
+ {
+ /* User-written ON-condition; transform it */
+ j->quals = transformJoinOnClause(pstate, j, my_namespace);
+ }
+ else
+ {
+ /* CROSS JOIN: no quals */
+ }
+
+ /*
+ * If this is an outer join, now mark the appropriate child RTEs as
+ * being nulled by this join. We have finished processing the child
+ * join expressions as well as the current join's quals, which deal in
+ * non-nulled input columns. All future references to those RTEs will
+ * see possibly-nulled values, and we should mark generated Vars to
+ * account for that. In particular, the join alias Vars that we're
+ * about to build should reflect the nulling effects of this join.
+ *
+ * A difficulty with doing this is that we need the join's RT index,
+ * which we don't officially have yet. However, no other RTE can get
+ * made between here and the addRangeTableEntryForJoin call, so we can
+ * predict what the assignment will be. (Alternatively, we could call
+ * addRangeTableEntryForJoin before we have all the data computed, but
+ * this seems less ugly.)
+ */
+ j->rtindex = list_length(pstate->p_rtable) + 1;
+
+ switch (j->jointype)
+ {
+ case JOIN_INNER:
+ break;
+ case JOIN_LEFT:
+ markRelsAsNulledBy(pstate, j->rarg, j->rtindex);
+ break;
+ case JOIN_FULL:
+ markRelsAsNulledBy(pstate, j->larg, j->rtindex);
+ markRelsAsNulledBy(pstate, j->rarg, j->rtindex);
+ break;
+ case JOIN_RIGHT:
+ markRelsAsNulledBy(pstate, j->larg, j->rtindex);
+ break;
+ default:
+ /* shouldn't see any other types here */
+ elog(ERROR, "unrecognized join type: %d",
+ (int) j->jointype);
+ break;
+ }
+
+ /*
+ * Now we can construct join alias expressions for the USING columns.
+ */
+ if (j->usingClause)
+ {
+ ListCell *lc1,
+ *lc2;
+
+ /* Scan the colnos lists to recover info from the previous loop */
+ forboth(lc1, l_colnos, lc2, r_colnos)
+ {
+ int l_index = lfirst_int(lc1) - 1;
+ int r_index = lfirst_int(lc2) - 1;
+ Var *l_colvar,
+ *r_colvar;
+ Node *u_colvar;
+ ParseNamespaceColumn *res_nscolumn;
+
+ /*
+ * Note we re-build these Vars: they might have different
+ * varnullingrels than the ones made in the previous loop.
+ */
+ l_colvar = buildVarFromNSColumn(pstate, l_nscolumns + l_index);
+ r_colvar = buildVarFromNSColumn(pstate, r_nscolumns + r_index);
+
+ /* Construct the join alias Var for this column */
u_colvar = buildMergedJoinVar(pstate,
j->jointype,
l_colvar,
r_colvar);
res_colvars = lappend(res_colvars, u_colvar);
+
+ /* Construct column's res_nscolumns[] entry */
res_nscolumn = res_nscolumns + res_colindex;
res_colindex++;
if (u_colvar == (Node *) l_colvar)
@@ -1404,47 +1498,45 @@ transformFromClauseItem(ParseState *pstate, Node *n,
/*
* Merged column is not semantically equivalent to either
* input, so it needs to be referenced as the join output
- * column. We don't know the join's varno yet, so we'll
- * replace these zeroes below.
+ * column.
*/
- res_nscolumn->p_varno = 0;
+ res_nscolumn->p_varno = j->rtindex;
res_nscolumn->p_varattno = res_colindex;
res_nscolumn->p_vartype = exprType(u_colvar);
res_nscolumn->p_vartypmod = exprTypmod(u_colvar);
res_nscolumn->p_varcollid = exprCollation(u_colvar);
- res_nscolumn->p_varnosyn = 0;
+ res_nscolumn->p_varnosyn = j->rtindex;
res_nscolumn->p_varattnosyn = res_colindex;
}
}
-
- j->quals = transformJoinUsingClause(pstate,
- l_usingvars,
- r_usingvars);
- }
- else if (j->quals)
- {
- /* User-written ON-condition; transform it */
- j->quals = transformJoinOnClause(pstate, j, my_namespace);
- }
- else
- {
- /* CROSS JOIN: no quals */
}
/* Add remaining columns from each side to the output columns */
res_colindex +=
- extractRemainingColumns(l_nscolumns, l_colnames, &l_colnos,
+ extractRemainingColumns(pstate,
+ l_nscolumns, l_colnames, &l_colnos,
&res_colnames, &res_colvars,
res_nscolumns + res_colindex);
res_colindex +=
- extractRemainingColumns(r_nscolumns, r_colnames, &r_colnos,
+ extractRemainingColumns(pstate,
+ r_nscolumns, r_colnames, &r_colnos,
&res_colnames, &res_colvars,
res_nscolumns + res_colindex);
+ /* If join has an alias, it syntactically hides all inputs */
+ if (j->alias)
+ {
+ for (k = 0; k < res_colindex; k++)
+ {
+ ParseNamespaceColumn *nscol = res_nscolumns + k;
+
+ nscol->p_varnosyn = j->rtindex;
+ nscol->p_varattnosyn = k + 1;
+ }
+ }
+
/*
* Now build an RTE and nsitem for the result of the join.
- * res_nscolumns isn't totally done yet, but that's OK because
- * addRangeTableEntryForJoin doesn't examine it, only store a pointer.
*/
nsitem = addRangeTableEntryForJoin(pstate,
res_colnames,
@@ -1458,31 +1550,16 @@ transformFromClauseItem(ParseState *pstate, Node *n,
j->alias,
true);
- j->rtindex = nsitem->p_rtindex;
+ /* Verify that we correctly predicted the join's RT index */
+ Assert(j->rtindex == nsitem->p_rtindex);
+ /* Cross-check number of columns, too */
+ Assert(res_colindex == list_length(nsitem->p_names->colnames));
/*
- * Now that we know the join RTE's rangetable index, we can fix up the
- * res_nscolumns data in places where it should contain that.
+ * Save a link to the JoinExpr in the proper element of p_joinexprs.
+ * Since we maintain that list lazily, it may be necessary to fill in
+ * empty entries before we can add the JoinExpr in the right place.
*/
- Assert(res_colindex == list_length(nsitem->p_names->colnames));
- for (k = 0; k < res_colindex; k++)
- {
- ParseNamespaceColumn *nscol = res_nscolumns + k;
-
- /* fill in join RTI for merged columns */
- if (nscol->p_varno == 0)
- nscol->p_varno = j->rtindex;
- if (nscol->p_varnosyn == 0)
- nscol->p_varnosyn = j->rtindex;
- /* if join has an alias, it syntactically hides all inputs */
- if (j->alias)
- {
- nscol->p_varnosyn = j->rtindex;
- nscol->p_varattnosyn = k + 1;
- }
- }
-
- /* make a matching link to the JoinExpr for later use */
for (k = list_length(pstate->p_joinexprs) + 1; k < j->rtindex; k++)
pstate->p_joinexprs = lappend(pstate->p_joinexprs, NULL);
pstate->p_joinexprs = lappend(pstate->p_joinexprs, j);
@@ -1551,10 +1628,13 @@ transformFromClauseItem(ParseState *pstate, Node *n,
* buildVarFromNSColumn -
* build a Var node using ParseNamespaceColumn data
*
- * We assume varlevelsup should be 0, and no location is specified
+ * This is used to construct joinaliasvars entries.
+ * We can assume varlevelsup should be 0, and no location is specified.
+ * Note also that no column SELECT privilege is requested here; that would
+ * happen only if the column is actually referenced in the query.
*/
static Var *
-buildVarFromNSColumn(ParseNamespaceColumn *nscol)
+buildVarFromNSColumn(ParseState *pstate, ParseNamespaceColumn *nscol)
{
Var *var;
@@ -1568,6 +1648,10 @@ buildVarFromNSColumn(ParseNamespaceColumn *nscol)
/* makeVar doesn't offer parameters for these, so set by hand: */
var->varnosyn = nscol->p_varnosyn;
var->varattnosyn = nscol->p_varattnosyn;
+
+ /* ... and update varnullingrels */
+ markNullableIfNeeded(pstate, var);
+
return var;
}
@@ -1679,6 +1763,47 @@ buildMergedJoinVar(ParseState *pstate, JoinType jointype,
return res_node;
}
+/*
+ * markRelsAsNulledBy -
+ * Mark the given jointree node and its children as nulled by join jindex
+ */
+static void
+markRelsAsNulledBy(ParseState *pstate, Node *n, int jindex)
+{
+ int varno;
+ ListCell *lc;
+
+ /* Note: we can't see FromExpr here */
+ if (IsA(n, RangeTblRef))
+ {
+ varno = ((RangeTblRef *) n)->rtindex;
+ }
+ else if (IsA(n, JoinExpr))
+ {
+ JoinExpr *j = (JoinExpr *) n;
+
+ /* recurse to children */
+ markRelsAsNulledBy(pstate, j->larg, jindex);
+ markRelsAsNulledBy(pstate, j->rarg, jindex);
+ varno = j->rtindex;
+ }
+ else
+ {
+ elog(ERROR, "unrecognized node type: %d", (int) nodeTag(n));
+ varno = 0; /* keep compiler quiet */
+ }
+
+ /*
+ * Now add jindex to the p_nullingrels set for relation varno. Since we
+ * maintain the p_nullingrels list lazily, we might need to extend it to
+ * make the varno'th entry exist.
+ */
+ while (list_length(pstate->p_nullingrels) < varno)
+ pstate->p_nullingrels = lappend(pstate->p_nullingrels, NULL);
+ lc = list_nth_cell(pstate->p_nullingrels, varno - 1);
+ lfirst(lc) = bms_add_member((Bitmapset *) lfirst(lc), jindex);
+}
+
/*
* setNamespaceColumnVisibility -
* Convenience subroutine to update cols_visible flags in a namespace list.
diff --git a/src/backend/parser/parse_coerce.c b/src/backend/parser/parse_coerce.c
index c4e958e4aa..4ded12e873 100644
--- a/src/backend/parser/parse_coerce.c
+++ b/src/backend/parser/parse_coerce.c
@@ -1042,7 +1042,7 @@ coerce_record_to_complex(ParseState *pstate, Node *node,
ParseNamespaceItem *nsitem;
nsitem = GetNSItemByRangeTablePosn(pstate, rtindex, sublevels_up);
- args = expandNSItemVars(nsitem, sublevels_up, vlocation, NULL);
+ args = expandNSItemVars(pstate, nsitem, sublevels_up, vlocation, NULL);
}
else
ereport(ERROR,
diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c
index fabb5f7207..ff7fadfc47 100644
--- a/src/backend/parser/parse_expr.c
+++ b/src/backend/parser/parse_expr.c
@@ -2600,6 +2600,9 @@ transformWholeRowRef(ParseState *pstate, ParseNamespaceItem *nsitem,
/* location is not filled in by makeWholeRowVar */
result->location = location;
+ /* mark Var if it's nulled by any outer joins */
+ markNullableIfNeeded(pstate, result);
+
/* mark relation as requiring whole-row SELECT access */
markVarForSelectPriv(pstate, result);
@@ -2627,6 +2630,8 @@ transformWholeRowRef(ParseState *pstate, ParseNamespaceItem *nsitem,
rowexpr->colnames = copyObject(nsitem->p_names->colnames);
rowexpr->location = location;
+ /* XXX we ought to mark the row as possibly nullable */
+
return (Node *) rowexpr;
}
}
diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c
index f6b740df0a..d956b24f73 100644
--- a/src/backend/parser/parse_relation.c
+++ b/src/backend/parser/parse_relation.c
@@ -751,6 +751,9 @@ scanNSItemForColumn(ParseState *pstate, ParseNamespaceItem *nsitem,
}
var->location = location;
+ /* Mark Var if it's nulled by any outer joins */
+ markNullableIfNeeded(pstate, var);
+
/* Require read access to the column */
markVarForSelectPriv(pstate, var);
@@ -1007,6 +1010,35 @@ searchRangeTableForCol(ParseState *pstate, const char *alias, const char *colnam
return fuzzystate;
}
+/*
+ * markNullableIfNeeded
+ * If the RTE referenced by the Var is nullable by outer join(s)
+ * at this point in the query, set var->varnullingrels to show that.
+ */
+void
+markNullableIfNeeded(ParseState *pstate, Var *var)
+{
+ int rtindex = var->varno;
+ Bitmapset *relids;
+
+ /* Find the appropriate pstate */
+ for (int lv = 0; lv < var->varlevelsup; lv++)
+ pstate = pstate->parentParseState;
+
+ /* Find currently-relevant join relids for the Var's rel */
+ if (rtindex > 0 && rtindex <= list_length(pstate->p_nullingrels))
+ relids = (Bitmapset *) list_nth(pstate->p_nullingrels, rtindex - 1);
+ else
+ relids = NULL;
+
+ /*
+ * Merge with any already-declared nulling rels. (Typically there won't
+ * be any, but let's get it right if there are.)
+ */
+ if (relids != NULL)
+ var->varnullingrels = bms_union(var->varnullingrels, relids);
+}
+
/*
* markRTEForSelectPriv
* Mark the specified column of the RTE with index rtindex
@@ -3110,7 +3142,7 @@ expandTupleDesc(TupleDesc tupdesc, Alias *eref, int count, int offset,
* the list elements mustn't be modified.
*/
List *
-expandNSItemVars(ParseNamespaceItem *nsitem,
+expandNSItemVars(ParseState *pstate, ParseNamespaceItem *nsitem,
int sublevels_up, int location,
List **colnames)
{
@@ -3146,6 +3178,10 @@ expandNSItemVars(ParseNamespaceItem *nsitem,
var->varnosyn = nscol->p_varnosyn;
var->varattnosyn = nscol->p_varattnosyn;
var->location = location;
+
+ /* ... and update varnullingrels */
+ markNullableIfNeeded(pstate, var);
+
result = lappend(result, var);
if (colnames)
*colnames = lappend(*colnames, colnameval);
@@ -3180,7 +3216,7 @@ expandNSItemAttrs(ParseState *pstate, ParseNamespaceItem *nsitem,
*var;
List *te_list = NIL;
- vars = expandNSItemVars(nsitem, sublevels_up, location, &names);
+ vars = expandNSItemVars(pstate, nsitem, sublevels_up, location, &names);
/*
* Require read access to the table. This is normally redundant with the
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index 16a0fe59e2..f81548441a 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -1379,7 +1379,7 @@ ExpandSingleTable(ParseState *pstate, ParseNamespaceItem *nsitem,
List *vars;
ListCell *l;
- vars = expandNSItemVars(nsitem, sublevels_up, location, NULL);
+ vars = expandNSItemVars(pstate, nsitem, sublevels_up, location, NULL);
/*
* Require read access to the table. This is normally redundant with
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index b376031856..e5f81329e5 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1078,6 +1078,14 @@ typedef struct RangeTblEntry
* alias Vars are generated only for merged columns). We keep these
* entries only because they're needed in expandRTE() and similar code.
*
+ * Vars appearing within joinaliasvars are marked with varnullingrels sets
+ * that describe the nulling effects of this join and lower ones. This is
+ * essential for FULL JOIN cases, because the COALESCE expression only
+ * describes the semantics correctly if its inputs have been nulled by the
+ * join. For other cases, it allows expandRTE() to generate a valid
+ * representation of the join's output without consulting additional
+ * parser state.
+ *
* Within a Query loaded from a stored rule, it is possible for non-merged
* joinaliasvars items to be null pointers, which are placeholders for
* (necessarily unreferenced) columns dropped since the rule was made.
diff --git a/src/include/parser/parse_node.h b/src/include/parser/parse_node.h
index 962ebf65de..636d3231cd 100644
--- a/src/include/parser/parse_node.h
+++ b/src/include/parser/parse_node.h
@@ -115,6 +115,13 @@ typedef Node *(*CoerceParamHook) (ParseState *pstate, Param *param,
* This is one-for-one with p_rtable, but contains NULLs for non-join
* RTEs, and may be shorter than p_rtable if the last RTE(s) aren't joins.
*
+ * p_nullingrels: list of Bitmapsets associated with p_rtable entries, each
+ * containing the set of outer-join RTE indexes that can null that relation
+ * at the current point in the parse tree. This is one-for-one with p_rtable,
+ * but may be shorter than p_rtable, in which case the missing entries are
+ * implicitly empty (NULL). That rule allows us to save work when the query
+ * contains no outer joins.
+ *
* p_joinlist: list of join items (RangeTblRef and JoinExpr nodes) that
* will become the fromlist of the query's top-level FromExpr node.
*
@@ -182,6 +189,7 @@ struct ParseState
const char *p_sourcetext; /* source text, or NULL if not available */
List *p_rtable; /* range table so far */
List *p_joinexprs; /* JoinExprs for RTE_JOIN p_rtable entries */
+ List *p_nullingrels; /* Bitmapsets showing nulling outer joins */
List *p_joinlist; /* join items so far (will become FromExpr
* node's fromlist) */
List *p_namespace; /* currently-referenceable RTEs (List of
diff --git a/src/include/parser/parse_relation.h b/src/include/parser/parse_relation.h
index de21c3c649..85d96563f3 100644
--- a/src/include/parser/parse_relation.h
+++ b/src/include/parser/parse_relation.h
@@ -41,6 +41,7 @@ extern Node *scanNSItemForColumn(ParseState *pstate, ParseNamespaceItem *nsitem,
int location);
extern Node *colNameToVar(ParseState *pstate, const char *colname, bool localonly,
int location);
+extern void markNullableIfNeeded(ParseState *pstate, Var *var);
extern void markVarForSelectPriv(ParseState *pstate, Var *var);
extern Relation parserOpenTable(ParseState *pstate, const RangeVar *relation,
int lockmode);
@@ -109,7 +110,7 @@ extern void errorMissingColumn(ParseState *pstate,
extern void expandRTE(RangeTblEntry *rte, int rtindex, int sublevels_up,
int location, bool include_dropped,
List **colnames, List **colvars);
-extern List *expandNSItemVars(ParseNamespaceItem *nsitem,
+extern List *expandNSItemVars(ParseState *pstate, ParseNamespaceItem *nsitem,
int sublevels_up, int location,
List **colnames);
extern List *expandNSItemAttrs(ParseState *pstate, ParseNamespaceItem *nsitem,
commit e554c9e00a14d01fa5e29367d6820d59fee7b69b
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu Aug 18 14:14:58 2022 -0400
Teach the planner to cope with Vars bearing nullingrels.
The core idea of this step is to include varnullingrels in the
relid sets that qual clauses are considered to depend on.
So that we can still easily compare quals' relids to RelOptInfos'
relids, that means also adding outer join relids to the identifying
relids of join relations. Much of the bulk of this step is concerned
with fallout from the latter change.
Also, in setrefs.c and some other places, we have to intentionally
ignore varnullingrels when comparing the outputs of a lower plan
node to the Vars required by upper expressions. I'd like to tighten
that up, by accounting for whether a given plan node implements
an outer join and expecting the upper Vars to have that OJ relid
added to their varnullingrels if so. But because of the hackery
involved in implementing outer join identity 3, there are some cases
where the upper Var legitimately won't have that bit set, and it's
unclear how to make a check that doesn't reject such plans. So that
issue is left for later. It would only be a bug-detection aid
anyway, since all the interesting decisions have been made already.
This step removes some low-hanging fruit from the old implementation,
such as the need to track lowest_nulling_outer_join during subquery
pullup. There's much more to do in that line, though.
The result of this step passes most core regression tests, but there
are still some failure cases involving full joins.
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index 004481d608..1c921879a9 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -273,7 +273,7 @@ merge_clump(PlannerInfo *root, List *clumps, Clump *new_clump, int num_gene,
* rel once we know the final targetlist (see
* grouping_planner).
*/
- if (!bms_equal(joinrel->relids, root->all_baserels))
+ if (!bms_equal(joinrel->relids, root->all_query_rels))
generate_useful_gather_paths(root, joinrel, false);
/* Find and save the cheapest paths for this joinrel */
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 8fc28007f5..251ac77a78 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -180,6 +180,9 @@ make_one_rel(PlannerInfo *root, List *joinlist)
root->all_baserels = bms_add_member(root->all_baserels, brel->relid);
}
+ /* Now we can form the value of all_query_rels, too */
+ root->all_query_rels = bms_union(root->all_baserels, root->outer_join_rels);
+
/* Mark base rels as to whether we care about fast-start plans */
set_base_rel_consider_startup(root);
@@ -231,9 +234,9 @@ make_one_rel(PlannerInfo *root, List *joinlist)
rel = make_rel_from_joinlist(root, joinlist);
/*
- * The result should join all and only the query's base rels.
+ * The result should join all and only the query's base + outer-join rels.
*/
- Assert(bms_equal(rel->relids, root->all_baserels));
+ Assert(bms_equal(rel->relids, root->all_query_rels));
return rel;
}
@@ -558,7 +561,7 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* the final scan/join targetlist is available (see grouping_planner).
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- !bms_equal(rel->relids, root->all_baserels))
+ !bms_equal(rel->relids, root->all_query_rels))
generate_useful_gather_paths(root, rel, false);
/* Now find the cheapest of the paths for this rel */
@@ -879,7 +882,7 @@ set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *
* to support an uncommon usage of second-rate sampling methods. Instead,
* if there is a risk that the query might perform an unsafe join, just
* wrap the SampleScan in a Materialize node. We can check for joins by
- * counting the membership of all_baserels (note that this correctly
+ * counting the membership of all_query_rels (note that this correctly
* counts inheritance trees as single rels). If we're inside a subquery,
* we can't easily check whether a join might occur in the outer query, so
* just assume one is possible.
@@ -888,7 +891,7 @@ set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *
* so check repeatable_across_scans last, even though that's a bit odd.
*/
if ((root->query_level > 1 ||
- bms_membership(root->all_baserels) != BMS_SINGLETON) &&
+ bms_membership(root->all_query_rels) != BMS_SINGLETON) &&
!(GetTsmRoutine(rte->tablesample->tsmhandler)->repeatable_across_scans))
{
path = (Path *) create_material_path(rel, path);
@@ -3435,7 +3438,7 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
* partial paths. We'll do the same for the topmost scan/join rel
* once we know the final targetlist (see grouping_planner).
*/
- if (!bms_equal(rel->relids, root->all_baserels))
+ if (!bms_equal(rel->relids, root->all_query_rels))
generate_useful_gather_paths(root, rel, false);
/* Find and save the cheapest paths for this rel */
diff --git a/src/backend/optimizer/path/clausesel.c b/src/backend/optimizer/path/clausesel.c
index 06f836308d..c08eb2b1c5 100644
--- a/src/backend/optimizer/path/clausesel.c
+++ b/src/backend/optimizer/path/clausesel.c
@@ -218,7 +218,7 @@ clauselist_selectivity_ext(PlannerInfo *root,
if (rinfo)
{
- ok = (bms_membership(rinfo->clause_relids) == BMS_SINGLETON) &&
+ ok = (rinfo->num_base_rels == 1) &&
(is_pseudo_constant_clause_relids(lsecond(expr->args),
rinfo->right_relids) ||
(varonleft = false,
@@ -579,30 +579,6 @@ find_single_rel_for_clauses(PlannerInfo *root, List *clauses)
return NULL; /* no clauses */
}
-/*
- * bms_is_subset_singleton
- *
- * Same result as bms_is_subset(s, bms_make_singleton(x)),
- * but a little faster and doesn't leak memory.
- *
- * Is this of use anywhere else? If so move to bitmapset.c ...
- */
-static bool
-bms_is_subset_singleton(const Bitmapset *s, int x)
-{
- switch (bms_membership(s))
- {
- case BMS_EMPTY_SET:
- return true;
- case BMS_SINGLETON:
- return bms_is_member(x, s);
- case BMS_MULTIPLE:
- return false;
- }
- /* can't get here... */
- return false;
-}
-
/*
* treat_as_join_clause -
* Decide whether an operator clause is to be handled by the
@@ -631,17 +607,20 @@ treat_as_join_clause(PlannerInfo *root, Node *clause, RestrictInfo *rinfo,
else
{
/*
- * Otherwise, it's a join if there's more than one relation used. We
- * can optimize this calculation if an rinfo was passed.
+ * Otherwise, it's a join if there's more than one base relation used.
+ * We can optimize this calculation if an rinfo was passed.
*
* XXX Since we know the clause is being evaluated at a join, the
* only way it could be single-relation is if it was delayed by outer
- * joins. Although we can make use of the restriction qual estimators
- * anyway, it seems likely that we ought to account for the
- * probability of injected nulls somehow.
+ * joins. We intentionally count only baserels here, not OJs that
+ * might be present in rinfo->clause_relids, so that we direct such
+ * cases to the restriction qual estimators not join estimators.
+ * Eventually some notice should be taken of the possibility of
+ * injected nulls, but we'll likely want to do that in the restriction
+ * estimators rather than starting to treat such cases as join quals.
*/
if (rinfo)
- return (bms_membership(rinfo->clause_relids) == BMS_MULTIPLE);
+ return (rinfo->num_base_rels > 1);
else
return (NumRelids(root, clause) > 1);
}
@@ -754,7 +733,9 @@ clause_selectivity_ext(PlannerInfo *root,
* for all non-JOIN_INNER cases.
*/
if (varRelid == 0 ||
- bms_is_subset_singleton(rinfo->clause_relids, varRelid))
+ rinfo->num_base_rels == 0 ||
+ (rinfo->num_base_rels == 1 &&
+ bms_is_member(varRelid, rinfo->clause_relids)))
{
/* Cacheable --- do we already have the result? */
if (jointype == JOIN_INNER)
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 1e94c5aa7c..09b18eac56 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -5101,7 +5101,9 @@ compute_semi_anti_join_factors(PlannerInfo *root,
norm_sjinfo.syn_lefthand = outerrel->relids;
norm_sjinfo.syn_righthand = innerrel->relids;
norm_sjinfo.jointype = JOIN_INNER;
+ norm_sjinfo.ojrelid = 0;
/* we don't bother trying to make the remaining fields valid */
+ norm_sjinfo.strict_relids = NULL;
norm_sjinfo.lhs_strict = false;
norm_sjinfo.delay_upper_joins = false;
norm_sjinfo.semi_can_btree = false;
@@ -5266,7 +5268,9 @@ approx_tuple_count(PlannerInfo *root, JoinPath *path, List *quals)
sjinfo.syn_lefthand = path->outerjoinpath->parent->relids;
sjinfo.syn_righthand = path->innerjoinpath->parent->relids;
sjinfo.jointype = JOIN_INNER;
+ sjinfo.ojrelid = 0;
/* we don't bother trying to make the remaining fields valid */
+ sjinfo.strict_relids = NULL;
sjinfo.lhs_strict = false;
sjinfo.delay_upper_joins = false;
sjinfo.semi_can_btree = false;
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 799bdc91d0..da7ad75207 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -29,6 +29,7 @@
#include "optimizer/paths.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "rewrite/rewriteManip.h"
#include "utils/lsyscache.h"
@@ -64,7 +65,7 @@ static bool reconsider_outer_join_clause(PlannerInfo *root,
RestrictInfo *rinfo,
bool outer_on_left);
static bool reconsider_full_join_clause(PlannerInfo *root,
- RestrictInfo *rinfo);
+ FullJoinClauseInfo *fjinfo);
static Bitmapset *get_eclass_indexes_for_relids(PlannerInfo *root,
Relids relids);
static Bitmapset *get_common_eclass_indexes(PlannerInfo *root, Relids relids1,
@@ -768,6 +769,9 @@ get_eclass_for_sort_expr(PlannerInfo *root,
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL)
+ continue; /* must be an outer join */
+
Assert(rel->reloptkind == RELOPT_BASEREL ||
rel->reloptkind == RELOPT_DEADREL);
@@ -936,7 +940,36 @@ is_exprlist_member(Expr *node, List *exprs)
if (expr && IsA(expr, TargetEntry))
expr = ((TargetEntry *) expr)->expr;
- if (equal(node, expr))
+ /*
+ * For Vars and PlaceHolderVars, match using the same rules as
+ * setrefs.c will, in particular ignoring nullingrels. XXX when that
+ * gets tightened up, this should too.
+ */
+ if (IsA(node, Var))
+ {
+ if (expr && IsA(expr, Var))
+ {
+ Var *v1 = (Var *) node;
+ Var *v2 = (Var *) expr;
+
+ if (v1->varno == v2->varno &&
+ v1->varattno == v2->varattno &&
+ v1->varlevelsup == v2->varlevelsup)
+ return true;
+ }
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ if (expr && IsA(expr, PlaceHolderVar))
+ {
+ PlaceHolderVar *v1 = (PlaceHolderVar *) node;
+ PlaceHolderVar *v2 = (PlaceHolderVar *) expr;
+
+ if (v1->phid == v2->phid)
+ return true;
+ }
+ }
+ else if (equal(node, expr))
return true;
}
return false;
@@ -1124,6 +1157,9 @@ generate_base_implied_equalities(PlannerInfo *root)
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL)
+ continue; /* must be an outer join */
+
Assert(rel->reloptkind == RELOPT_BASEREL);
rel->eclass_indexes = bms_add_member(rel->eclass_indexes,
@@ -2014,10 +2050,12 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* Process the FULL JOIN clauses */
foreach(cell, root->full_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ FullJoinClauseInfo *fjinfo = (FullJoinClauseInfo *) lfirst(cell);
- if (reconsider_full_join_clause(root, rinfo))
+ if (reconsider_full_join_clause(root, fjinfo))
{
+ RestrictInfo *rinfo = fjinfo->rinfo;
+
found = true;
/* remove it from the list */
root->full_join_clauses =
@@ -2046,9 +2084,9 @@ reconsider_outer_join_clauses(PlannerInfo *root)
}
foreach(cell, root->full_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ FullJoinClauseInfo *fjinfo = (FullJoinClauseInfo *) lfirst(cell);
- distribute_restrictinfo_to_rels(root, rinfo);
+ distribute_restrictinfo_to_rels(root, fjinfo->rinfo);
}
}
@@ -2184,8 +2222,11 @@ reconsider_outer_join_clause(PlannerInfo *root, RestrictInfo *rinfo,
* Returns true if we were able to propagate a constant through the clause.
*/
static bool
-reconsider_full_join_clause(PlannerInfo *root, RestrictInfo *rinfo)
+reconsider_full_join_clause(PlannerInfo *root, FullJoinClauseInfo *fjinfo)
{
+ RestrictInfo *rinfo = fjinfo->rinfo;
+ SpecialJoinInfo *sjinfo = fjinfo->sjinfo;
+ Relids fjrelids = bms_make_singleton(sjinfo->ojrelid);
Expr *leftvar;
Expr *rightvar;
Oid opno,
@@ -2267,6 +2308,18 @@ reconsider_full_join_clause(PlannerInfo *root, RestrictInfo *rinfo)
cfirst = (Node *) linitial(cexpr->args);
csecond = (Node *) lsecond(cexpr->args);
+ /*
+ * The COALESCE arguments will be marked as possibly nulled by
+ * the full join, while we wish to generate clauses that apply
+ * to the join's inputs. So we must strip the join from the
+ * nullingrels fields of cfirst/csecond before comparing them
+ * to leftvar/rightvar. (Perhaps with a less hokey
+ * representation for FULL JOIN USING output columns, this
+ * wouldn't be needed?)
+ */
+ cfirst = remove_nulling_relids(cfirst, fjrelids, NULL);
+ csecond = remove_nulling_relids(csecond, fjrelids, NULL);
+
if (equal(leftvar, cfirst) && equal(rightvar, csecond))
{
coal_idx = foreach_current_index(lc2);
@@ -3203,6 +3256,8 @@ get_eclass_indexes_for_relids(PlannerInfo *root, Relids relids)
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL)
+ continue; /* must be an outer join */
ec_indexes = bms_add_members(ec_indexes, rel->eclass_indexes);
}
return ec_indexes;
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index 7d176e7b00..78d817bf1b 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -3355,13 +3355,13 @@ check_index_predicates(PlannerInfo *root, RelOptInfo *rel)
* Add on any equivalence-derivable join clauses. Computing the correct
* relid sets for generate_join_implied_equalities is slightly tricky
* because the rel could be a child rel rather than a true baserel, and in
- * that case we must remove its parents' relid(s) from all_baserels.
+ * that case we must subtract its parents' relid(s) from all_query_rels.
*/
if (rel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- otherrels = bms_difference(root->all_baserels,
+ otherrels = bms_difference(root->all_query_rels,
find_childrel_parents(root, rel));
else
- otherrels = bms_difference(root->all_baserels, rel->relids);
+ otherrels = bms_difference(root->all_query_rels, rel->relids);
if (!bms_is_empty(otherrels))
clauselist =
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 2a3f0ab7bf..a1fc72c394 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -250,7 +250,7 @@ add_paths_to_joinrel(PlannerInfo *root,
if (bms_overlap(joinrelids, sjinfo2->min_righthand) &&
!bms_overlap(joinrelids, sjinfo2->min_lefthand))
extra.param_source_rels = bms_join(extra.param_source_rels,
- bms_difference(root->all_baserels,
+ bms_difference(root->all_query_rels,
sjinfo2->min_righthand));
/* full joins constrain both sides symmetrically */
@@ -258,7 +258,7 @@ add_paths_to_joinrel(PlannerInfo *root,
bms_overlap(joinrelids, sjinfo2->min_lefthand) &&
!bms_overlap(joinrelids, sjinfo2->min_righthand))
extra.param_source_rels = bms_join(extra.param_source_rels,
- bms_difference(root->all_baserels,
+ bms_difference(root->all_query_rels,
sjinfo2->min_lefthand));
}
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 9da3ff2f9a..b64c37f089 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -353,7 +353,10 @@ make_rels_by_clauseless_joins(PlannerInfo *root,
*
* Caller must supply not only the two rels, but the union of their relids.
* (We could simplify the API by computing joinrelids locally, but this
- * would be redundant work in the normal path through make_join_rel.)
+ * would be redundant work in the normal path through make_join_rel.
+ * Note that this value does NOT include the RT index of any outer join that
+ * might need to be performed here, so it's not the canonical identifier
+ * of the join relation.)
*
* On success, *sjinfo_p is set to NULL if this is to be a plain inner join,
* else it's set to point to the associated SpecialJoinInfo node. Also,
@@ -695,7 +698,7 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(rel1->relids, rel2->relids));
- /* Construct Relids set that identifies the joinrel. */
+ /* Construct Relids set that identifies the joinrel (without OJ as yet). */
joinrelids = bms_union(rel1->relids, rel2->relids);
/* Check validity and determine join type. */
@@ -707,6 +710,10 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
return NULL;
}
+ /* If we have an outer join, add its RTI to form the canonical relids. */
+ if (sjinfo && sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
+
/* Swap rels if needed to match the join info. */
if (reversed)
{
@@ -730,7 +737,9 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
sjinfo->syn_lefthand = rel1->relids;
sjinfo->syn_righthand = rel2->relids;
sjinfo->jointype = JOIN_INNER;
+ sjinfo->ojrelid = 0;
/* we don't bother trying to make the remaining fields valid */
+ sjinfo->strict_relids = NULL;
sjinfo->lhs_strict = false;
sjinfo->delay_upper_joins = false;
sjinfo->semi_can_btree = false;
@@ -1510,8 +1519,6 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(child_rel1->relids, child_rel2->relids));
- child_joinrelids = bms_union(child_rel1->relids, child_rel2->relids);
- appinfos = find_appinfos_by_relids(root, child_joinrelids, &nappinfos);
/*
* Construct SpecialJoinInfo from parent join relations's
@@ -1521,6 +1528,15 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
child_rel1->relids,
child_rel2->relids);
+ /* Build correct join relids for child join */
+ child_joinrelids = bms_union(child_rel1->relids, child_rel2->relids);
+ if (child_sjinfo->ojrelid != 0)
+ child_joinrelids = bms_add_member(child_joinrelids,
+ child_sjinfo->ojrelid);
+
+ /* Find the AppendRelInfo structures */
+ appinfos = find_appinfos_by_relids(root, child_joinrelids, &nappinfos);
+
/*
* Construct restrictions applicable to the child join from those
* applicable to the parent join.
@@ -1536,8 +1552,7 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
{
child_joinrel = build_child_join_rel(root, child_rel1, child_rel2,
joinrel, child_restrictlist,
- child_sjinfo,
- child_sjinfo->jointype);
+ child_sjinfo);
joinrel->part_rels[cnt_parts] = child_joinrel;
joinrel->live_parts = bms_add_member(joinrel->live_parts, cnt_parts);
joinrel->all_partrels = bms_add_members(joinrel->all_partrels,
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index bbeca9a9ab..72d9da5187 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -34,7 +34,7 @@
/* local functions */
static bool join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo);
-static void remove_rel_from_query(PlannerInfo *root, int relid,
+static void remove_rel_from_query(PlannerInfo *root, int relid, int ojrelid,
Relids joinrelids);
static List *remove_rel_from_joinlist(List *joinlist, int relid, int *nremoved);
static bool rel_supports_distinctness(PlannerInfo *root, RelOptInfo *rel);
@@ -70,6 +70,7 @@ restart:
foreach(lc, root->join_info_list)
{
SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+ Relids joinrelids;
int innerrelid;
int nremoved;
@@ -84,9 +85,12 @@ restart:
*/
innerrelid = bms_singleton_member(sjinfo->min_righthand);
- remove_rel_from_query(root, innerrelid,
- bms_union(sjinfo->min_lefthand,
- sjinfo->min_righthand));
+ /* Compute the relid set for the join we are considering */
+ joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ if (sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
+
+ remove_rel_from_query(root, innerrelid, sjinfo->ojrelid, joinrelids);
/* We verify that exactly one reference gets removed from joinlist */
nremoved = 0;
@@ -188,6 +192,8 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
/* Compute the relid set for the join we are considering */
joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ if (sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
/*
* We can't remove the join if any inner-rel attributes are used above the
@@ -306,10 +312,12 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
* no longer treated as a baserel, and that attributes of other baserels
* are no longer marked as being needed at joins involving this rel.
* Also, join quals involving the rel have to be removed from the joininfo
- * lists, but only if they belong to the outer join identified by joinrelids.
+ * lists, but only if they belong to the outer join identified by ojrelid
+ * and joinrelids.
*/
static void
-remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
+remove_rel_from_query(PlannerInfo *root, int relid, int ojrelid,
+ Relids joinrelids)
{
RelOptInfo *rel = find_base_rel(root, relid);
List *joininfos;
@@ -349,6 +357,13 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
}
}
+ /*
+ * The removed outer join has to be dropped from root->outer_join_rels.
+ * (We'd need to update all_baserels and all_query_rels too, but those
+ * haven't been computed yet.)
+ */
+ root->outer_join_rels = bms_del_member(root->outer_join_rels, ojrelid);
+
/*
* Likewise remove references from SpecialJoinInfo data structures.
*
@@ -365,6 +380,10 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
sjinfo->min_righthand = bms_del_member(sjinfo->min_righthand, relid);
sjinfo->syn_lefthand = bms_del_member(sjinfo->syn_lefthand, relid);
sjinfo->syn_righthand = bms_del_member(sjinfo->syn_righthand, relid);
+ sjinfo->min_lefthand = bms_del_member(sjinfo->min_lefthand, ojrelid);
+ sjinfo->min_righthand = bms_del_member(sjinfo->min_righthand, ojrelid);
+ sjinfo->syn_lefthand = bms_del_member(sjinfo->syn_lefthand, ojrelid);
+ sjinfo->syn_righthand = bms_del_member(sjinfo->syn_righthand, ojrelid);
}
/*
@@ -396,8 +415,10 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
else
{
phinfo->ph_eval_at = bms_del_member(phinfo->ph_eval_at, relid);
+ phinfo->ph_eval_at = bms_del_member(phinfo->ph_eval_at, ojrelid);
Assert(!bms_is_empty(phinfo->ph_eval_at));
phinfo->ph_needed = bms_del_member(phinfo->ph_needed, relid);
+ phinfo->ph_needed = bms_del_member(phinfo->ph_needed, ojrelid);
}
}
@@ -434,6 +455,8 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
rinfo->required_relids = bms_copy(rinfo->required_relids);
rinfo->required_relids = bms_del_member(rinfo->required_relids,
relid);
+ rinfo->required_relids = bms_del_member(rinfo->required_relids,
+ ojrelid);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
@@ -548,6 +571,7 @@ reduce_unique_semijoins(PlannerInfo *root)
/* Compute the relid set for the join we are considering */
joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ Assert(sjinfo->ojrelid == 0); /* SEMI joins don't have RT indexes */
/*
* Since we're only considering a single-rel RHS, any join clauses it
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index fd8cbb1dc7..4fd1db3e3d 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -60,12 +60,15 @@ static void process_security_barrier_quals(PlannerInfo *root,
static SpecialJoinInfo *make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
- JoinType jointype, List *clause);
+ JoinType jointype, Index ojrelid,
+ List *clause);
static void compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo,
List *clause);
+static List *remove_unneeded_nulling_relids(PlannerInfo *root, List *quals,
+ SpecialJoinInfo *sjinfo);
static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool below_outer_join,
- JoinType jointype,
+ SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
Relids ojscope,
@@ -248,10 +251,16 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
attno -= rel->min_attr;
if (rel->attr_needed[attno] == NULL)
{
- /* Variable not yet requested, so add to rel's targetlist */
- /* XXX is copyObject necessary here? */
- rel->reltarget->exprs = lappend(rel->reltarget->exprs,
- copyObject(var));
+ /*
+ * Variable not yet requested, so add to rel's targetlist.
+ *
+ * The value available at the rel's scan level has not been
+ * nulled by any outer join, so drop its varnullingrels.
+ * (We'll put those back as we climb up the join tree.)
+ */
+ var = copyObject(var);
+ var->varnullingrels = NULL;
+ rel->reltarget->exprs = lappend(rel->reltarget->exprs, var);
/* reltarget cost and width will be computed later */
}
rel->attr_needed[attno] = bms_add_members(rel->attr_needed[attno],
@@ -547,8 +556,10 @@ create_lateral_join_info(PlannerInfo *root)
varno = -1;
while ((varno = bms_next_member(eval_at, varno)) >= 0)
{
- RelOptInfo *brel = find_base_rel(root, varno);
+ RelOptInfo *brel = find_base_rel_ignore_join(root, varno);
+ if (brel == NULL)
+ continue; /* ignore outer joins in eval_at */
brel->lateral_relids = bms_add_members(brel->lateral_relids,
phinfo->ph_lateral);
}
@@ -639,7 +650,10 @@ create_lateral_join_info(PlannerInfo *root)
{
RelOptInfo *brel2 = root->simple_rel_array[rti2];
- Assert(brel2 != NULL && brel2->reloptkind == RELOPT_BASEREL);
+ if (brel2 == NULL)
+ continue; /* must be an OJ */
+
+ Assert(brel2->reloptkind == RELOPT_BASEREL);
brel2->lateral_referencers =
bms_add_member(brel2->lateral_referencers, rti);
}
@@ -699,7 +713,8 @@ deconstruct_jointree(PlannerInfo *root)
Assert(root->parse->jointree != NULL &&
IsA(root->parse->jointree, FromExpr));
- /* this is filled as we scan the jointree */
+ /* These are filled as we scan the jointree */
+ root->outer_join_rels = NULL;
root->nullable_baserels = NULL;
result = deconstruct_recurse(root, (Node *) root->parse->jointree, false,
@@ -721,7 +736,7 @@ deconstruct_jointree(PlannerInfo *root)
* below_outer_join is true if this node is within the nullable side of a
* higher-level outer join
* Outputs:
- * *qualscope gets the set of base Relids syntactically included in this
+ * *qualscope gets the set of base+OJ Relids syntactically included in this
* jointree node (do not modify or free this, as it may also be pointed
* to by RestrictInfo and SpecialJoinInfo nodes)
* *inner_join_rels gets the set of base Relids syntactically included in
@@ -806,6 +821,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
* there was exactly one element, we should (and already did) report
* whatever its inner_join_rels were. If there were no elements (is
* that still possible?) the initialization before the loop fixed it.
+ *
+ * XXX now wrong, do we care?
*/
if (list_length(f->fromlist) > 1)
*inner_join_rels = *qualscope;
@@ -820,7 +837,7 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
if (bms_is_subset(pq->relids, *qualscope))
distribute_qual_to_rels(root, pq->qual,
- below_outer_join, JOIN_INNER,
+ below_outer_join, NULL,
root->qual_security_level,
*qualscope, NULL, NULL,
NULL);
@@ -836,7 +853,7 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
Node *qual = (Node *) lfirst(l);
distribute_qual_to_rels(root, qual,
- below_outer_join, JOIN_INNER,
+ below_outer_join, NULL,
root->qual_security_level,
*qualscope, NULL, NULL,
postponed_qual_list);
@@ -900,6 +917,13 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ /* caution: ANTI join derived from SEMI will lack rtindex */
+ if (j->rtindex != 0)
+ {
+ *qualscope = bms_add_member(*qualscope, j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
+ j->rtindex);
+ }
*inner_join_rels = bms_union(left_inners, right_inners);
nonnullable_rels = leftids;
nullable_rels = rightids;
@@ -914,6 +938,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ /* SEMI join never has rtindex, so don't add to qualscope */
+ Assert(j->rtindex == 0);
*inner_join_rels = bms_union(left_inners, right_inners);
/* Semi join adds no restrictions for quals */
nonnullable_rels = NULL;
@@ -935,6 +961,10 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ Assert(j->rtindex != 0);
+ *qualscope = bms_add_member(*qualscope, j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
+ j->rtindex);
*inner_join_rels = bms_union(left_inners, right_inners);
/* each side is both outer and inner */
nonnullable_rels = *qualscope;
@@ -980,32 +1010,44 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
my_quals = list_concat(my_quals, (List *) j->quals);
/*
- * For an OJ, form the SpecialJoinInfo now, because we need the OJ's
- * semantic scope (ojscope) to pass to distribute_qual_to_rels. But
- * we mustn't add it to join_info_list just yet, because we don't want
- * distribute_qual_to_rels to think it is an outer join below us.
- *
- * Semijoins are a bit of a hybrid: we build a SpecialJoinInfo, but we
- * want ojscope = NULL for distribute_qual_to_rels.
+ * For an OJ, form the SpecialJoinInfo now, because we need it for
+ * distribute_qual_to_rels. But we mustn't add it to join_info_list
+ * just yet, because we don't want distribute_qual_to_rels to think it
+ * is an outer join below us.
*/
if (j->jointype != JOIN_INNER)
- {
sjinfo = make_outerjoininfo(root,
leftids, rightids,
*inner_join_rels,
j->jointype,
+ j->rtindex,
my_quals);
- if (j->jointype == JOIN_SEMI)
- ojscope = NULL;
- else
- ojscope = bms_union(sjinfo->min_lefthand,
- sjinfo->min_righthand);
- }
else
- {
sjinfo = NULL;
+
+ /*
+ * If we have a LEFT JOIN whose ON qual is strict for any LHS
+ * relations, we may be able to commute the join with lower outer
+ * joins that null those relations. To do that, we must remove such
+ * lower outer joins from Var.varnullingrels fields within the qual,
+ * else subsequent processing will think that the qual has to be
+ * evaluated above such lower outer joins.
+ */
+ if (j->jointype == JOIN_LEFT && sjinfo->lhs_strict)
+ my_quals = remove_unneeded_nulling_relids(root, my_quals, sjinfo);
+
+ /*
+ * Now we can compute ojscope (we can't do it earlier, because
+ * remove_unneeded_nulling_relids might change the scope).
+ *
+ * Semijoins are a bit of a hybrid: we build a SpecialJoinInfo, but we
+ * want ojscope = NULL for distribute_qual_to_rels.
+ */
+ if (j->jointype == JOIN_INNER || j->jointype == JOIN_SEMI)
ojscope = NULL;
- }
+ else
+ ojscope = bms_union(sjinfo->min_lefthand,
+ sjinfo->min_righthand);
/* Process the JOIN's qual clauses */
foreach(l, my_quals)
@@ -1013,7 +1055,7 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
Node *qual = (Node *) lfirst(l);
distribute_qual_to_rels(root, qual,
- below_outer_join, j->jointype,
+ below_outer_join, sjinfo,
root->qual_security_level,
*qualscope,
ojscope, nonnullable_rels,
@@ -1116,7 +1158,7 @@ process_security_barrier_quals(PlannerInfo *root,
*/
distribute_qual_to_rels(root, qual,
below_outer_join,
- JOIN_INNER,
+ NULL,
security_level,
qualscope,
qualscope,
@@ -1139,6 +1181,7 @@ process_security_barrier_quals(PlannerInfo *root,
* right_rels: the base Relids syntactically on inner side of join
* inner_join_rels: base Relids participating in inner joins below this one
* jointype: what it says (must always be LEFT, FULL, SEMI, or ANTI)
+ * ojrelid: RT index of the join RTE (0 for SEMI, which isn't in the RT list)
* clause: the outer join's join condition (in implicit-AND format)
*
* The node should eventually be appended to root->join_info_list, but we
@@ -1152,7 +1195,8 @@ static SpecialJoinInfo *
make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
- JoinType jointype, List *clause)
+ JoinType jointype, Index ojrelid,
+ List *clause)
{
SpecialJoinInfo *sjinfo = makeNode(SpecialJoinInfo);
Relids clause_relids;
@@ -1200,6 +1244,7 @@ make_outerjoininfo(PlannerInfo *root,
sjinfo->syn_lefthand = left_rels;
sjinfo->syn_righthand = right_rels;
sjinfo->jointype = jointype;
+ sjinfo->ojrelid = ojrelid;
/* this always starts out false */
sjinfo->delay_upper_joins = false;
@@ -1210,6 +1255,7 @@ make_outerjoininfo(PlannerInfo *root,
{
sjinfo->min_lefthand = bms_copy(left_rels);
sjinfo->min_righthand = bms_copy(right_rels);
+ sjinfo->strict_relids = NULL; /* don't care about this */
sjinfo->lhs_strict = false; /* don't care about this */
return sjinfo;
}
@@ -1224,6 +1270,7 @@ make_outerjoininfo(PlannerInfo *root,
* rel's columns are all NULL?
*/
strict_relids = find_nonnullable_rels((Node *) clause);
+ sjinfo->strict_relids = strict_relids;
/* Remember whether the clause is strict for any LHS relations */
sjinfo->lhs_strict = bms_overlap(strict_relids, left_rels);
@@ -1262,6 +1309,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_lefthand = bms_add_member(min_lefthand,
+ otherinfo->ojrelid);
}
if (bms_overlap(right_rels, otherinfo->syn_lefthand) ||
bms_overlap(right_rels, otherinfo->syn_righthand))
@@ -1270,6 +1320,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_righthand = bms_add_member(min_righthand,
+ otherinfo->ojrelid);
}
/* Needn't do anything else with the full join */
continue;
@@ -1299,6 +1352,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_lefthand = bms_add_member(min_lefthand,
+ otherinfo->ojrelid);
}
}
@@ -1341,6 +1397,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_righthand = bms_add_member(min_righthand,
+ otherinfo->ojrelid);
}
}
}
@@ -1565,6 +1624,62 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
sjinfo->semi_rhs_exprs = semi_rhs_exprs;
}
+/*
+ * remove_unneeded_nulling_relids
+ * Remove lower outer joins from Vars (& PHVs) in the quals, if possible
+ *
+ * This paves the way to apply outer join identity 3 to commute the current
+ * LEFT JOIN with lower outer joins. We already know that the quals are
+ * strict for at least one LHS relation.
+ */
+static List *
+remove_unneeded_nulling_relids(PlannerInfo *root, List *quals,
+ SpecialJoinInfo *sjinfo)
+{
+ Relids old_nulling_relids;
+ Relids removable_relids;
+ ListCell *lc;
+
+ /*
+ * Find outer joins mentioned in nullingrel fields in the quals. If there
+ * aren't any (the common case), there's no need to work hard.
+ */
+ old_nulling_relids = get_nulling_relids((Node *) quals);
+ if (bms_is_empty(old_nulling_relids))
+ return quals;
+
+ /*
+ * Thumb through the existing SpecialJoinInfos (which describe all outer
+ * joins below this one, but not yet this one) to find the ones mentioned
+ * in the quals. If the current join's quals are strict for any rel of
+ * one's RHS, we can commute this join with that one, so remove it from
+ * the current join's min_lefthand and from the quals' nullingrel fields.
+ */
+ removable_relids = NULL;
+ foreach(lc, root->join_info_list)
+ {
+ SpecialJoinInfo *sjinfo2 = (SpecialJoinInfo *) lfirst(lc);
+
+ if (sjinfo2->jointype != JOIN_LEFT ||
+ !bms_is_member(sjinfo2->ojrelid, old_nulling_relids))
+ continue; /* it's not relevant */
+ if (bms_is_subset(sjinfo2->syn_righthand, sjinfo->syn_lefthand) &&
+ bms_overlap(sjinfo->strict_relids, sjinfo2->min_righthand))
+ {
+ sjinfo->min_lefthand = bms_del_member(sjinfo->min_lefthand,
+ sjinfo2->ojrelid);
+ removable_relids = bms_add_member(removable_relids,
+ sjinfo2->ojrelid);
+ }
+ }
+
+ if (removable_relids == NULL)
+ return quals; /* no hits, nothing to do */
+
+ return (List *) remove_nulling_relids((Node *) quals,
+ removable_relids, NULL);
+}
+
/*****************************************************************************
*
@@ -1586,7 +1701,7 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
* 'clause': the qual clause to be distributed
* 'below_outer_join': true if the qual is from a JOIN/ON that is below the
* nullable side of a higher-level outer join
- * 'jointype': type of join the qual is from (JOIN_INNER for a WHERE clause)
+ * 'sjinfo': join's SpecialJoinInfo (NULL for an inner join or WHERE clause)
* 'security_level': security_level to assign to the qual
* 'qualscope': set of baserels the qual's syntactic scope covers
* 'ojscope': NULL if not an outer-join qual, else the minimum set of baserels
@@ -1604,12 +1719,13 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
* level, which will be ojscope not necessarily qualscope.
*
* At the time this is called, root->join_info_list must contain entries for
- * all and only those special joins that are syntactically below this qual.
+ * all and only those special joins that are syntactically below this qual;
+ * in particular, the passed-in SpecialJoinInfo isn't yet in that list.
*/
static void
distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool below_outer_join,
- JoinType jointype,
+ SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
Relids ojscope,
@@ -1646,7 +1762,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
PostponedQual *pq = (PostponedQual *) palloc(sizeof(PostponedQual));
Assert(root->hasLateralRTEs); /* shouldn't happen otherwise */
- Assert(jointype == JOIN_INNER); /* mustn't postpone past outer join */
+ Assert(sjinfo == NULL); /* mustn't postpone past outer join */
pq->qual = clause;
pq->relids = relids;
*postponed_qual_list = lappend(*postponed_qual_list, pq);
@@ -1708,7 +1824,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
{
relids =
get_relids_in_jointree((Node *) root->parse->jointree,
- false);
+ true, false);
qualscope = bms_copy(relids);
}
}
@@ -1950,11 +2066,15 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
restrictinfo);
return;
}
- if (jointype == JOIN_FULL)
+ if (sjinfo && sjinfo->jointype == JOIN_FULL)
{
/* FULL JOIN (above tests cannot match in this case) */
+ FullJoinClauseInfo *fjinfo = makeNode(FullJoinClauseInfo);
+
+ fjinfo->rinfo = restrictinfo;
+ fjinfo->sjinfo = sjinfo;
root->full_join_clauses = lappend(root->full_join_clauses,
- restrictinfo);
+ fjinfo);
return;
}
/* nope, so fall through to distribute_restrictinfo_to_rels */
@@ -2348,7 +2468,7 @@ process_implied_equality(PlannerInfo *root,
{
relids =
get_relids_in_jointree((Node *) root->parse->jointree,
- false);
+ true, false);
}
}
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index cf9e0a74db..556acd2bd6 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2223,7 +2223,7 @@ preprocess_rowmarks(PlannerInfo *root)
* make a bitmapset of all base rels and then remove the items we don't
* need or have FOR [KEY] UPDATE/SHARE marks for.
*/
- rels = get_relids_in_jointree((Node *) parse->jointree, false);
+ rels = get_relids_in_jointree((Node *) parse->jointree, false, false);
if (parse->resultRelation)
rels = bms_del_member(rels, parse->resultRelation);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 1cb0abdbc1..a1827d113d 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -151,6 +151,9 @@ static Var *search_indexed_tlist_for_var(Var *var,
indexed_tlist *itlist,
int newvarno,
int rtoffset);
+static Var *search_indexed_tlist_for_phv(PlaceHolderVar *phv,
+ indexed_tlist *itlist,
+ int newvarno);
static Var *search_indexed_tlist_for_non_var(Expr *node,
indexed_tlist *itlist,
int newvarno);
@@ -2115,6 +2118,7 @@ fix_scan_expr_mutator(Node *node, fix_scan_expr_context *context)
/* At scan level, we should always just evaluate the contained expr */
PlaceHolderVar *phv = (PlaceHolderVar *) node;
+ Assert(phv->phnullingrels == NULL);
return fix_scan_expr_mutator((Node *) phv->phexpr, context);
}
if (IsA(node, AlternativeSubPlan))
@@ -2235,33 +2239,12 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
/*
* Now we need to fix up the targetlist and qpqual, which are logically
* above the join. This means they should not re-use any input expression
- * that was computed in the nullable side of an outer join. Vars and
- * PlaceHolderVars are fine, so we can implement this restriction just by
- * clearing has_non_vars in the indexed_tlist structs.
+ * that was computed in the nullable side of an outer join.
*
- * XXX This is a grotty workaround for the fact that we don't clearly
- * distinguish between a Var appearing below an outer join and the "same"
- * Var appearing above it. If we did, we'd not need to hack the matching
- * rules this way.
+ * XXX we will probably need to pass some flag down to indicate that this
+ * context applies, so that search_indexed_tlist_for_var() and siblings
+ * can correctly check for varnullingrels matches.
*/
- switch (join->jointype)
- {
- case JOIN_LEFT:
- case JOIN_SEMI:
- case JOIN_ANTI:
- inner_itlist->has_non_vars = false;
- break;
- case JOIN_RIGHT:
- outer_itlist->has_non_vars = false;
- break;
- case JOIN_FULL:
- outer_itlist->has_non_vars = false;
- inner_itlist->has_non_vars = false;
- break;
- default:
- break;
- }
-
join->plan.targetlist = fix_join_expr(root,
join->plan.targetlist,
outer_itlist,
@@ -2550,7 +2533,7 @@ set_dummy_tlist_references(Plan *plan, int rtoffset)
* tlist_member() searches.
*
* The result of this function is an indexed_tlist struct to pass to
- * search_indexed_tlist_for_var() or search_indexed_tlist_for_non_var().
+ * search_indexed_tlist_for_var() and siblings.
* When done, the indexed_tlist may be freed with a single pfree().
*/
static indexed_tlist *
@@ -2672,6 +2655,8 @@ search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
/* Found a match */
Var *newvar = copyVar(var);
+ /* XXX we oughta check varnullingrels match here ... */
+
newvar->varno = newvarno;
newvar->varattno = vinfo->resno;
if (newvar->varnosyn > 0)
@@ -2684,15 +2669,55 @@ search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
}
/*
- * search_indexed_tlist_for_non_var --- find a non-Var in an indexed tlist
+ * search_indexed_tlist_for_phv --- find a PlaceHolderVar in an indexed tlist
*
* If a match is found, return a Var constructed to reference the tlist item.
* If no match, return NULL.
*
- * NOTE: it is a waste of time to call this unless itlist->has_ph_vars or
- * itlist->has_non_vars. Furthermore, set_join_references() relies on being
- * able to prevent matching of non-Vars by clearing itlist->has_non_vars,
- * so there's a correctness reason not to call it unless that's set.
+ * NOTE: it is a waste of time to call this unless itlist->has_ph_vars.
+ */
+static Var *
+search_indexed_tlist_for_phv(PlaceHolderVar *phv,
+ indexed_tlist *itlist, int newvarno)
+{
+ ListCell *lc;
+
+ foreach(lc, itlist->tlist)
+ {
+ TargetEntry *tle = (TargetEntry *) lfirst(lc);
+
+ if (tle->expr && IsA(tle->expr, PlaceHolderVar))
+ {
+ PlaceHolderVar *subphv = (PlaceHolderVar *) tle->expr;
+ Var *newvar;
+
+ /*
+ * Analogously to search_indexed_tlist_for_var, we match on phid
+ * only. We don't use equal(), partially for speed but mostly
+ * because phnullingrels might not be exactly equal.
+ *
+ * XXX we really oughta verify phnullingrels.
+ */
+ if (phv->phid != subphv->phid)
+ continue;
+
+ /* Found a matching subplan output expression */
+ newvar = makeVarFromTargetEntry(newvarno, tle);
+ newvar->varnosyn = 0; /* wasn't ever a plain Var */
+ newvar->varattnosyn = 0;
+ return newvar;
+ }
+ }
+ return NULL; /* no match */
+}
+
+/*
+ * search_indexed_tlist_for_non_var --- find a non-Var/PHV in an indexed tlist
+ *
+ * If a match is found, return a Var constructed to reference the tlist item.
+ * If no match, return NULL.
+ *
+ * NOTE: it is a waste of time to call this unless itlist->has_non_vars.
*/
static Var *
search_indexed_tlist_for_non_var(Expr *node,
@@ -2877,22 +2902,23 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* See if the PlaceHolderVar has bubbled up from a lower plan node */
if (context->outer_itlist && context->outer_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->outer_itlist,
- OUTER_VAR);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->outer_itlist,
+ OUTER_VAR);
if (newvar)
return (Node *) newvar;
}
if (context->inner_itlist && context->inner_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->inner_itlist,
- INNER_VAR);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->inner_itlist,
+ INNER_VAR);
if (newvar)
return (Node *) newvar;
}
/* If not supplied by input plans, evaluate the contained expr */
+ /* XXX assert something about phnullingrels */
return fix_join_expr_mutator((Node *) phv->phexpr, context);
}
/* Try matching more complex expressions too, if tlists have any */
@@ -3001,13 +3027,14 @@ fix_upper_expr_mutator(Node *node, fix_upper_expr_context *context)
/* See if the PlaceHolderVar has bubbled up from a lower plan node */
if (context->subplan_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->subplan_itlist,
- context->newvarno);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->subplan_itlist,
+ context->newvarno);
if (newvar)
return (Node *) newvar;
}
/* If not supplied by input plan, evaluate the contained expr */
+ /* XXX assert something about phnullingrels */
return fix_upper_expr_mutator((Node *) phv->phexpr, context);
}
/* Try matching more complex expressions too, if tlist has any */
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 41c7066d90..7c001cdf35 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -49,17 +49,28 @@ typedef struct pullup_replace_vars_context
* pullup (set only if target_rte->lateral) */
bool *outer_hasSubLinks; /* -> outer query's hasSubLinks */
int varno; /* varno of subquery */
- bool need_phvs; /* do we need PlaceHolderVars? */
- bool wrap_non_vars; /* do we need 'em on *all* non-Vars? */
+ bool wrap_non_vars; /* do we need all non-Var outputs to be PHVs? */
Node **rv_cache; /* cache for results with PHVs */
} pullup_replace_vars_context;
-typedef struct reduce_outer_joins_state
+typedef struct reduce_outer_joins_pass1_state
{
Relids relids; /* base relids within this subtree */
bool contains_outer; /* does subtree contain outer join(s)? */
List *sub_states; /* List of states for subtree components */
-} reduce_outer_joins_state;
+} reduce_outer_joins_pass1_state;
+
+typedef struct reduce_outer_joins_pass2_state
+{
+ Relids inner_reduced; /* OJ relids reduced to plain inner joins */
+ List *partial_reduced; /* List of partially reduced FULL joins */
+} reduce_outer_joins_pass2_state;
+
+typedef struct reduce_outer_joins_partial_state
+{
+ int full_join_rti; /* RT index of a formerly-FULL join */
+ Relids unreduced_side; /* relids in its still-nullable side */
+} reduce_outer_joins_partial_state;
static Node *pull_up_sublinks_jointree_recurse(PlannerInfo *root, Node *jtnode,
Relids *relids);
@@ -68,12 +79,10 @@ static Node *pull_up_sublinks_qual_recurse(PlannerInfo *root, Node *node,
Node **jtlink2, Relids available_rels2);
static Node *pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static Node *pull_up_simple_subquery(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static Node *pull_up_simple_union_all(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte);
@@ -90,7 +99,6 @@ static Node *pull_up_simple_values(PlannerInfo *root, Node *jtnode,
static bool is_simple_values(PlannerInfo *root, RangeTblEntry *rte);
static Node *pull_up_constant_function(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static bool is_simple_union_all(Query *subquery);
static bool is_simple_union_all_recurse(Node *setOp, Query *setOpQuery,
@@ -101,25 +109,27 @@ static bool jointree_contains_lateral_outer_refs(PlannerInfo *root,
Relids safe_upper_varnos);
static void perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars_context *rvcontext,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static void replace_vars_in_jointree(Node *jtnode,
- pullup_replace_vars_context *context,
- JoinExpr *lowest_nulling_outer_join);
+ pullup_replace_vars_context *context);
static Node *pullup_replace_vars(Node *expr,
pullup_replace_vars_context *context);
static Node *pullup_replace_vars_callback(Var *var,
replace_rte_variables_context *context);
static Query *pullup_replace_vars_subquery(Query *query,
pullup_replace_vars_context *context);
-static reduce_outer_joins_state *reduce_outer_joins_pass1(Node *jtnode);
+static reduce_outer_joins_pass1_state *reduce_outer_joins_pass1(Node *jtnode);
static void reduce_outer_joins_pass2(Node *jtnode,
- reduce_outer_joins_state *state,
+ reduce_outer_joins_pass1_state *state1,
+ reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
List *nonnullable_vars,
List *forced_null_vars);
-static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode);
+static void report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
+ int rtindex, Relids relids);
+static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Relids *dropped_outer_joins);
static int get_result_relid(PlannerInfo *root, Node *jtnode);
static void remove_result_refs(PlannerInfo *root, int varno, Node *newjtloc);
static bool find_dependent_phvs(PlannerInfo *root, int varno);
@@ -764,7 +774,7 @@ pull_up_subqueries(PlannerInfo *root)
/* Recursion starts with no containing join nor appendrel */
root->parse->jointree = (FromExpr *)
pull_up_subqueries_recurse(root, (Node *) root->parse->jointree,
- NULL, NULL, NULL);
+ NULL, NULL);
/* We should still have a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
}
@@ -779,12 +789,6 @@ pull_up_subqueries(PlannerInfo *root)
* lowest_outer_join references the lowest such JoinExpr node; otherwise
* it is NULL. We use this to constrain the effects of LATERAL subqueries.
*
- * If this jointree node is within the nullable side of an outer join, then
- * lowest_nulling_outer_join references the lowest such JoinExpr node;
- * otherwise it is NULL. This forces use of the PlaceHolderVar mechanism for
- * references to non-nullable targetlist items, but only for references above
- * that join.
- *
* If we are looking at a member subquery of an append relation,
* containing_appendrel describes that relation; else it is NULL.
* This forces use of the PlaceHolderVar mechanism for all non-Var targetlist
@@ -801,15 +805,14 @@ pull_up_subqueries(PlannerInfo *root)
* Notice also that we can't turn pullup_replace_vars loose on the whole
* jointree, because it'd return a mutated copy of the tree; we have to
* invoke it just on the quals, instead. This behavior is what makes it
- * reasonable to pass lowest_outer_join and lowest_nulling_outer_join as
- * pointers rather than some more-indirect way of identifying the lowest
- * OJs. Likewise, we don't replace append_rel_list members but only their
- * substructure, so the containing_appendrel reference is safe to use.
+ * reasonable to pass lowest_outer_join as a pointer rather than some
+ * more-indirect way of identifying the lowest OJ. Likewise, we don't
+ * replace append_rel_list members but only their substructure, so the
+ * containing_appendrel reference is safe to use.
*/
static Node *
pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Assert(jtnode != NULL);
@@ -831,7 +834,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
is_safe_append_member(rte->subquery)))
return pull_up_simple_subquery(root, jtnode, rte,
lowest_outer_join,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -864,7 +866,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
*/
if (rte->rtekind == RTE_FUNCTION)
return pull_up_constant_function(root, jtnode, rte,
- lowest_nulling_outer_join,
containing_appendrel);
/* Otherwise, do nothing at this node. */
@@ -880,7 +881,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
{
lfirst(l) = pull_up_subqueries_recurse(root, lfirst(l),
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
}
}
@@ -895,11 +895,9 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
case JOIN_INNER:
j->larg = pull_up_subqueries_recurse(root, j->larg,
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
break;
case JOIN_LEFT:
@@ -907,31 +905,25 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
case JOIN_ANTI:
j->larg = pull_up_subqueries_recurse(root, j->larg,
j,
- lowest_nulling_outer_join,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
- j,
j,
NULL);
break;
case JOIN_FULL:
j->larg = pull_up_subqueries_recurse(root, j->larg,
- j,
j,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
- j,
j,
NULL);
break;
case JOIN_RIGHT:
j->larg = pull_up_subqueries_recurse(root, j->larg,
- j,
j,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
j,
- lowest_nulling_outer_join,
NULL);
break;
default:
@@ -961,7 +953,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
static Node *
pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -1108,31 +1099,25 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* The subquery's targetlist items are now in the appropriate form to
* insert into the top query, except that we may need to wrap them in
* PlaceHolderVars. Set up required context data for pullup_replace_vars.
+ * (Note that we should include the subquery's inner joins in relids,
+ * since it may include join alias vars referencing them.)
*/
rvcontext.root = root;
rvcontext.targetlist = subquery->targetList;
rvcontext.target_rte = rte;
if (rte->lateral)
rvcontext.relids = get_relids_in_jointree((Node *) subquery->jointree,
- true);
+ true, true);
else /* won't need relids */
rvcontext.relids = NULL;
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = varno;
- /* these flags will be set below, if needed */
- rvcontext.need_phvs = false;
+ /* this flag will be set below, if needed */
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(subquery->targetList) + 1) *
sizeof(Node *));
- /*
- * If we are under an outer join then non-nullable items and lateral
- * references may have to be turned into PlaceHolderVars.
- */
- if (lowest_nulling_outer_join != NULL)
- rvcontext.need_phvs = true;
-
/*
* If we are dealing with an appendrel member then anything that's not a
* simple Var has to be turned into a PlaceHolderVar. We force this to
@@ -1141,10 +1126,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* expression actually available from the appendrel.
*/
if (containing_appendrel != NULL)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* If the parent query uses grouping sets, we need a PlaceHolderVar for
@@ -1156,10 +1138,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* that pullup_replace_vars hasn't currently got.)
*/
if (parse->groupingSets)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* Replace all of the top query's references to the subquery's outputs
@@ -1167,7 +1146,6 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* replace any of the jointree structure.
*/
perform_pullup_replace_vars(root, &rvcontext,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -1234,7 +1212,8 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
{
Relids subrelids;
- subrelids = get_relids_in_jointree((Node *) subquery->jointree, false);
+ subrelids = get_relids_in_jointree((Node *) subquery->jointree,
+ true, false);
substitute_phv_relids((Node *) parse, varno, subrelids);
fix_append_rel_relids(root->append_rel_list, varno, subrelids);
}
@@ -1425,7 +1404,7 @@ pull_up_union_leaf_queries(Node *setOp, PlannerInfo *root, int parentRTindex,
rtr = makeNode(RangeTblRef);
rtr->rtindex = childRTindex;
(void) pull_up_subqueries_recurse(root, (Node *) rtr,
- NULL, NULL, appinfo);
+ NULL, appinfo);
}
else if (IsA(setOp, SetOperationStmt))
{
@@ -1562,7 +1541,7 @@ is_simple_subquery(PlannerInfo *root, Query *subquery, RangeTblEntry *rte,
{
restricted = true;
safe_upper_varnos = get_relids_in_jointree((Node *) lowest_outer_join,
- true);
+ true, true);
}
else
{
@@ -1674,7 +1653,6 @@ pull_up_simple_values(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte)
rvcontext.relids = NULL;
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = varno;
- rvcontext.need_phvs = false;
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(tlist) + 1) *
@@ -1686,7 +1664,7 @@ pull_up_simple_values(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte)
* any of the jointree structure. We can assume there's no outer joins or
* appendrels in the dummy Query that surrounds a VALUES RTE.
*/
- perform_pullup_replace_vars(root, &rvcontext, NULL, NULL);
+ perform_pullup_replace_vars(root, &rvcontext, NULL);
/*
* There should be no appendrels to fix, nor any outer joins and hence no
@@ -1785,7 +1763,6 @@ is_simple_values(PlannerInfo *root, RangeTblEntry *rte)
static Node *
pull_up_constant_function(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -1837,40 +1814,26 @@ pull_up_constant_function(PlannerInfo *root, Node *jtnode,
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = ((RangeTblRef *) jtnode)->rtindex;
- /* these flags will be set below, if needed */
- rvcontext.need_phvs = false;
+ /* this flag will be set below, if needed */
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(rvcontext.targetlist) + 1) *
sizeof(Node *));
- /*
- * If we are under an outer join then non-nullable items and lateral
- * references may have to be turned into PlaceHolderVars.
- */
- if (lowest_nulling_outer_join != NULL)
- rvcontext.need_phvs = true;
-
/*
* If we are dealing with an appendrel member then anything that's not a
* simple Var has to be turned into a PlaceHolderVar. (See comments in
* pull_up_simple_subquery().)
*/
if (containing_appendrel != NULL)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* If the parent query uses grouping sets, we need a PlaceHolderVar for
* anything that's not a simple Var.
*/
if (parse->groupingSets)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* Replace all of the top query's references to the RTE's output with
@@ -1878,7 +1841,6 @@ pull_up_constant_function(PlannerInfo *root, Node *jtnode,
* jointree structure.
*/
perform_pullup_replace_vars(root, &rvcontext,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -2100,13 +2062,11 @@ jointree_contains_lateral_outer_refs(PlannerInfo *root, Node *jtnode,
*
* Caller has already filled *rvcontext with data describing what to
* substitute for Vars referencing the target subquery. In addition
- * we need the identity of the lowest outer join that can null the
- * target subquery, and its containing appendrel if any.
+ * we need the identity of the containing appendrel if any.
*/
static void
perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars_context *rvcontext,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -2150,38 +2110,31 @@ perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars((Node *) action->targetList, rvcontext);
}
}
- replace_vars_in_jointree((Node *) parse->jointree, rvcontext,
- lowest_nulling_outer_join);
+ replace_vars_in_jointree((Node *) parse->jointree, rvcontext);
Assert(parse->setOperations == NULL);
parse->havingQual = pullup_replace_vars(parse->havingQual, rvcontext);
/*
* Replace references in the translated_vars lists of appendrels. When
- * pulling up an appendrel member, we do not need PHVs in the list of the
- * parent appendrel --- there isn't any outer join between. Elsewhere,
- * use PHVs for safety. (This analysis could be made tighter but it seems
- * unlikely to be worth much trouble.)
+ * pulling up an appendrel member, we do not want to force PHVs in the
+ * list of the parent appendrel --- there isn't any outer join between.
+ * Elsewhere, use PHVs for safety. (This analysis could be made tighter
+ * but it seems unlikely to be worth much trouble.)
*/
foreach(lc, root->append_rel_list)
{
AppendRelInfo *appinfo = (AppendRelInfo *) lfirst(lc);
- bool save_need_phvs = rvcontext->need_phvs;
+ bool save_wrap_non_vars = rvcontext->wrap_non_vars;
if (appinfo == containing_appendrel)
- rvcontext->need_phvs = false;
+ rvcontext->wrap_non_vars = false;
appinfo->translated_vars = (List *)
pullup_replace_vars((Node *) appinfo->translated_vars, rvcontext);
- rvcontext->need_phvs = save_need_phvs;
+ rvcontext->wrap_non_vars = save_wrap_non_vars;
}
/*
* Replace references in the joinaliasvars lists of join RTEs.
- *
- * You might think that we could avoid using PHVs for alias vars of joins
- * below lowest_nulling_outer_join, but that doesn't work because the
- * alias vars could be referenced above that join; we need the PHVs to be
- * present in such references after the alias vars get flattened. (It
- * might be worth trying to be smarter here, someday.)
*/
foreach(lc, parse->rtable)
{
@@ -2198,14 +2151,10 @@ perform_pullup_replace_vars(PlannerInfo *root,
* Helper routine for perform_pullup_replace_vars: do pullup_replace_vars on
* every expression in the jointree, without changing the jointree structure
* itself. Ugly, but there's no other way...
- *
- * If we are at or below lowest_nulling_outer_join, we can suppress use of
- * PlaceHolderVars wrapped around the replacement expressions.
*/
static void
replace_vars_in_jointree(Node *jtnode,
- pullup_replace_vars_context *context,
- JoinExpr *lowest_nulling_outer_join)
+ pullup_replace_vars_context *context)
{
if (jtnode == NULL)
return;
@@ -2218,7 +2167,7 @@ replace_vars_in_jointree(Node *jtnode,
* jointree scan, rather than a scan of the rtable, for a couple of
* reasons: we can avoid processing no-longer-referenced RTEs, and we
* can use the appropriate setting of need_phvs depending on whether
- * the RTE is above possibly-nulling outer joins or not.
+ * the RTE is above possibly-nulling outer joins or not. XXX fix
*/
int varno = ((RangeTblRef *) jtnode)->rtindex;
@@ -2275,42 +2224,30 @@ replace_vars_in_jointree(Node *jtnode,
ListCell *l;
foreach(l, f->fromlist)
- replace_vars_in_jointree(lfirst(l), context,
- lowest_nulling_outer_join);
+ replace_vars_in_jointree(lfirst(l), context);
f->quals = pullup_replace_vars(f->quals, context);
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- bool save_need_phvs = context->need_phvs;
+ bool save_wrap_non_vars = context->wrap_non_vars;
- if (j == lowest_nulling_outer_join)
- {
- /* no more PHVs in or below this join */
- context->need_phvs = false;
- lowest_nulling_outer_join = NULL;
- }
- replace_vars_in_jointree(j->larg, context, lowest_nulling_outer_join);
- replace_vars_in_jointree(j->rarg, context, lowest_nulling_outer_join);
+ replace_vars_in_jointree(j->larg, context);
+ replace_vars_in_jointree(j->rarg, context);
/*
- * Use PHVs within the join quals of a full join, even when it's the
- * lowest nulling outer join. Otherwise, we cannot identify which
- * side of the join a pulled-up var-free expression came from, which
- * can lead to failure to make a plan at all because none of the quals
- * appear to be mergeable or hashable conditions. For this purpose we
- * don't care about the state of wrap_non_vars, so leave it alone.
+ * Use PHVs within the join quals of a full join. Otherwise, we
+ * cannot identify which side of the join a pulled-up var-free
+ * expression came from, which can lead to failure to make a plan at
+ * all because none of the quals appear to be mergeable or hashable
+ * conditions.
*/
if (j->jointype == JOIN_FULL)
- context->need_phvs = true;
+ context->wrap_non_vars = true;
j->quals = pullup_replace_vars(j->quals, context);
- /*
- * We don't bother to update the colvars list, since it won't be used
- * again ...
- */
- context->need_phvs = save_need_phvs;
+ context->wrap_non_vars = save_wrap_non_vars;
}
else
elog(ERROR, "unrecognized node type: %d",
@@ -2339,8 +2276,18 @@ pullup_replace_vars_callback(Var *var,
{
pullup_replace_vars_context *rcon = (pullup_replace_vars_context *) context->callback_arg;
int varattno = var->varattno;
+ bool need_phv;
Node *newnode;
+ /*
+ * We need a PlaceHolderVar if the Var-to-be-replaced has nonempty
+ * varnullingrels (unless we find below that the replacement expression is
+ * a Var or PlaceHolderVar that we can just add the nullingrels to). We
+ * also need one if the caller has instructed us that all non-Var/PHV
+ * replacements need to be wrapped for identification purposes.
+ */
+ need_phv = (var->varnullingrels != NULL) || rcon->wrap_non_vars;
+
/*
* If PlaceHolderVars are needed, we cache the modified expressions in
* rcon->rv_cache[]. This is not in hopes of any material speed gain
@@ -2349,13 +2296,16 @@ pullup_replace_vars_callback(Var *var,
* and possibly prevent optimizations that rely on recognizing different
* references to the same subquery output as being equal(). So it's worth
* a bit of extra effort to avoid it.
+ *
+ * The cached items have phlevelsup = 0 and phnullingrels = NULL; we'll
+ * copy them and adjust those values for this reference site below.
*/
- if (rcon->need_phvs &&
+ if (need_phv &&
varattno >= InvalidAttrNumber &&
varattno <= list_length(rcon->targetlist) &&
rcon->rv_cache[varattno] != NULL)
{
- /* Just copy the entry and fall through to adjust its varlevelsup */
+ /* Just copy the entry and fall through to adjust phlevelsup etc */
newnode = copyObject(rcon->rv_cache[varattno]);
}
else if (varattno == InvalidAttrNumber)
@@ -2364,7 +2314,7 @@ pullup_replace_vars_callback(Var *var,
RowExpr *rowexpr;
List *colnames;
List *fields;
- bool save_need_phvs = rcon->need_phvs;
+ bool save_wrap_non_vars = rcon->wrap_non_vars;
int save_sublevelsup = context->sublevels_up;
/*
@@ -2375,18 +2325,18 @@ pullup_replace_vars_callback(Var *var,
* the RowExpr for use of the executor and ruleutils.c.
*
* In order to be able to cache the results, we always generate the
- * expansion with varlevelsup = 0, and then adjust if needed.
+ * expansion with varlevelsup = 0, and then adjust below if needed.
*/
expandRTE(rcon->target_rte,
var->varno, 0 /* not varlevelsup */ , var->location,
(var->vartype != RECORDOID),
&colnames, &fields);
- /* Adjust the generated per-field Vars, but don't insert PHVs */
- rcon->need_phvs = false;
+ /* Expand the generated per-field Vars, but don't insert PHVs there */
+ rcon->wrap_non_vars = false;
context->sublevels_up = 0; /* to match the expandRTE output */
fields = (List *) replace_rte_variables_mutator((Node *) fields,
context);
- rcon->need_phvs = save_need_phvs;
+ rcon->wrap_non_vars = save_wrap_non_vars;
context->sublevels_up = save_sublevelsup;
rowexpr = makeNode(RowExpr);
@@ -2404,14 +2354,13 @@ pullup_replace_vars_callback(Var *var,
* expression to yield NULL, not ROW(NULL,NULL,...) when it is forced
* to null by an outer join.
*/
- if (rcon->need_phvs)
+ if (need_phv)
{
- /* RowExpr is certainly not strict, so always need PHV */
newnode = (Node *)
make_placeholder_expr(rcon->root,
(Expr *) newnode,
bms_make_singleton(rcon->varno));
- /* cache it with the PHV, and with varlevelsup still zero */
+ /* cache it with the PHV, and with phlevelsup etc not set yet */
rcon->rv_cache[InvalidAttrNumber] = copyObject(newnode);
}
}
@@ -2428,7 +2377,7 @@ pullup_replace_vars_callback(Var *var,
newnode = (Node *) copyObject(tle->expr);
/* Insert PlaceHolderVar if needed */
- if (rcon->need_phvs)
+ if (need_phv)
{
bool wrap;
@@ -2454,69 +2403,61 @@ pullup_replace_vars_callback(Var *var,
/* No need to wrap a PlaceHolderVar with another one, either */
wrap = false;
}
- else if (rcon->wrap_non_vars)
- {
- /* Wrap all non-Vars in a PlaceHolderVar */
- wrap = true;
- }
else
{
/*
- * If it contains a Var of the subquery being pulled up, and
- * does not contain any non-strict constructs, then it's
- * certainly nullable so we don't need to insert a
- * PlaceHolderVar.
- *
- * This analysis could be tighter: in particular, a non-strict
- * construct hidden within a lower-level PlaceHolderVar is not
- * reason to add another PHV. But for now it doesn't seem
- * worth the code to be more exact.
- *
- * Note: in future maybe we should insert a PlaceHolderVar
- * anyway, if the tlist item is expensive to evaluate?
- *
- * For a LATERAL subquery, we have to check the actual var
- * membership of the node, but if it's non-lateral then any
- * level-zero var must belong to the subquery.
+ * Must wrap, either because we need a place to insert
+ * varnullingrels or because caller told us to wrap
+ * everything.
*/
- if ((rcon->target_rte->lateral ?
- bms_overlap(pull_varnos(rcon->root, (Node *) newnode),
- rcon->relids) :
- contain_vars_of_level((Node *) newnode, 0)) &&
- !contain_nonstrict_functions((Node *) newnode))
- {
- /* No wrap needed */
- wrap = false;
- }
- else
- {
- /* Else wrap it in a PlaceHolderVar */
- wrap = true;
- }
+ wrap = true;
}
if (wrap)
+ {
newnode = (Node *)
make_placeholder_expr(rcon->root,
(Expr *) newnode,
bms_make_singleton(rcon->varno));
- /*
- * Cache it if possible (ie, if the attno is in range, which it
- * probably always should be). We can cache the value even if we
- * decided we didn't need a PHV, since this result will be
- * suitable for any request that has need_phvs.
- */
- if (varattno > InvalidAttrNumber &&
- varattno <= list_length(rcon->targetlist))
- rcon->rv_cache[varattno] = copyObject(newnode);
+ /*
+ * Cache it if possible (ie, if the attno is in range, which
+ * it probably always should be).
+ */
+ if (varattno > InvalidAttrNumber &&
+ varattno <= list_length(rcon->targetlist))
+ rcon->rv_cache[varattno] = copyObject(newnode);
+ }
}
}
- /* Must adjust varlevelsup if tlist item is from higher query */
+ /* Must adjust varlevelsup if replaced Var is within a subquery */
if (var->varlevelsup > 0)
IncrementVarSublevelsUp(newnode, var->varlevelsup, 0);
+ /* Propagate any varnullingrels into the replacement Var or PHV */
+ if (var->varnullingrels != NULL)
+ {
+ if (IsA(newnode, Var))
+ {
+ Var *newvar = (Var *) newnode;
+
+ Assert(newvar->varlevelsup == var->varlevelsup);
+ newvar->varnullingrels = bms_add_members(newvar->varnullingrels,
+ var->varnullingrels);
+ }
+ else if (IsA(newnode, PlaceHolderVar))
+ {
+ PlaceHolderVar *newphv = (PlaceHolderVar *) newnode;
+
+ Assert(newphv->phlevelsup == var->varlevelsup);
+ newphv->phnullingrels = bms_add_members(newphv->phnullingrels,
+ var->varnullingrels);
+ }
+ else
+ elog(ERROR, "failed to wrap a non-Var");
+ }
+
return newnode;
}
@@ -2675,7 +2616,9 @@ flatten_simple_union_all(PlannerInfo *root)
void
reduce_outer_joins(PlannerInfo *root)
{
- reduce_outer_joins_state *state;
+ reduce_outer_joins_pass1_state *state1;
+ reduce_outer_joins_pass2_state state2;
+ ListCell *lc;
/*
* To avoid doing strictness checks on more quals than necessary, we want
@@ -2686,14 +2629,44 @@ reduce_outer_joins(PlannerInfo *root)
* join(s) below each side of each join clause. The second pass examines
* qual clauses and changes join types as it descends the tree.
*/
- state = reduce_outer_joins_pass1((Node *) root->parse->jointree);
+ state1 = reduce_outer_joins_pass1((Node *) root->parse->jointree);
/* planner.c shouldn't have called me if no outer joins */
- if (state == NULL || !state->contains_outer)
+ if (state1 == NULL || !state1->contains_outer)
elog(ERROR, "so where are the outer joins?");
+ state2.inner_reduced = NULL;
+ state2.partial_reduced = NIL;
+
reduce_outer_joins_pass2((Node *) root->parse->jointree,
- state, root, NULL, NIL, NIL);
+ state1, &state2,
+ root, NULL, NIL, NIL);
+
+ /*
+ * If we successfully reduced the strength of any outer joins, we must
+ * remove references to those joins as nulling rels. This is handled as
+ * an additional pass, for simplicity and because we can handle all
+ * fully-reduced joins in a single pass over the parse tree.
+ */
+ if (!bms_is_empty(state2.inner_reduced))
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ state2.inner_reduced,
+ NULL);
+
+ /*
+ * Partially-reduced full joins have to be done one at a time, since
+ * they'll each need a different setting of except_relids.
+ */
+ foreach(lc, state2.partial_reduced)
+ {
+ reduce_outer_joins_partial_state *statep = lfirst(lc);
+
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ bms_make_singleton(statep->full_join_rti),
+ statep->unreduced_side);
+ }
}
/*
@@ -2701,13 +2674,13 @@ reduce_outer_joins(PlannerInfo *root)
*
* Returns a state node describing the given jointree node.
*/
-static reduce_outer_joins_state *
+static reduce_outer_joins_pass1_state *
reduce_outer_joins_pass1(Node *jtnode)
{
- reduce_outer_joins_state *result;
+ reduce_outer_joins_pass1_state *result;
- result = (reduce_outer_joins_state *)
- palloc(sizeof(reduce_outer_joins_state));
+ result = (reduce_outer_joins_pass1_state *)
+ palloc(sizeof(reduce_outer_joins_pass1_state));
result->relids = NULL;
result->contains_outer = false;
result->sub_states = NIL;
@@ -2727,7 +2700,7 @@ reduce_outer_joins_pass1(Node *jtnode)
foreach(l, f->fromlist)
{
- reduce_outer_joins_state *sub_state;
+ reduce_outer_joins_pass1_state *sub_state;
sub_state = reduce_outer_joins_pass1(lfirst(l));
result->relids = bms_add_members(result->relids,
@@ -2739,7 +2712,7 @@ reduce_outer_joins_pass1(Node *jtnode)
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- reduce_outer_joins_state *sub_state;
+ reduce_outer_joins_pass1_state *sub_state;
/* join's own RT index is not wanted in result->relids */
if (IS_OUTER_JOIN(j->jointype))
@@ -2767,15 +2740,23 @@ reduce_outer_joins_pass1(Node *jtnode)
* reduce_outer_joins_pass2 - phase 2 processing
*
* jtnode: current jointree node
- * state: state data collected by phase 1 for this node
+ * state1: state data collected by phase 1 for this node
+ * state2: where to accumulate info about successfully-reduced joins
* root: toplevel planner state
* nonnullable_rels: set of base relids forced non-null by upper quals
* nonnullable_vars: list of Vars forced non-null by upper quals
* forced_null_vars: list of Vars forced null by upper quals
+ *
+ * Returns info in state2 about outer joins that were successfully simplified.
+ * Joins that were fully reduced to inner joins are all added to
+ * state2->inner_reduced. If a full join is reduced to a left join,
+ * it needs its own entry in state2->partial_reduced, since that will
+ * require custom processing to remove only the correct nullingrel markers.
*/
static void
reduce_outer_joins_pass2(Node *jtnode,
- reduce_outer_joins_state *state,
+ reduce_outer_joins_pass1_state *state1,
+ reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
List *nonnullable_vars,
@@ -2809,13 +2790,14 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_forced_null_vars = list_concat(pass_forced_null_vars,
forced_null_vars);
/* And recurse --- but only into interesting subtrees */
- Assert(list_length(f->fromlist) == list_length(state->sub_states));
- forboth(l, f->fromlist, s, state->sub_states)
+ Assert(list_length(f->fromlist) == list_length(state1->sub_states));
+ forboth(l, f->fromlist, s, state1->sub_states)
{
- reduce_outer_joins_state *sub_state = lfirst(s);
+ reduce_outer_joins_pass1_state *sub_state = lfirst(s);
if (sub_state->contains_outer)
- reduce_outer_joins_pass2(lfirst(l), sub_state, root,
+ reduce_outer_joins_pass2(lfirst(l), sub_state,
+ state2, root,
pass_nonnullable_rels,
pass_nonnullable_vars,
pass_forced_null_vars);
@@ -2828,8 +2810,8 @@ reduce_outer_joins_pass2(Node *jtnode,
JoinExpr *j = (JoinExpr *) jtnode;
int rtindex = j->rtindex;
JoinType jointype = j->jointype;
- reduce_outer_joins_state *left_state = linitial(state->sub_states);
- reduce_outer_joins_state *right_state = lsecond(state->sub_states);
+ reduce_outer_joins_pass1_state *left_state = linitial(state1->sub_states);
+ reduce_outer_joins_pass1_state *right_state = lsecond(state1->sub_states);
List *local_nonnullable_vars = NIL;
bool computed_local_nonnullable_vars = false;
@@ -2852,12 +2834,22 @@ reduce_outer_joins_pass2(Node *jtnode,
if (bms_overlap(nonnullable_rels, right_state->relids))
jointype = JOIN_INNER;
else
+ {
jointype = JOIN_LEFT;
+ /* Also report partial reduction in state2 */
+ report_reduced_full_join(state2, rtindex,
+ right_state->relids);
+ }
}
else
{
if (bms_overlap(nonnullable_rels, right_state->relids))
+ {
jointype = JOIN_RIGHT;
+ /* Also report partial reduction in state2 */
+ report_reduced_full_join(state2, rtindex,
+ left_state->relids);
+ }
}
break;
case JOIN_SEMI:
@@ -2890,8 +2882,8 @@ reduce_outer_joins_pass2(Node *jtnode,
j->larg = j->rarg;
j->rarg = tmparg;
jointype = JOIN_LEFT;
- right_state = linitial(state->sub_states);
- left_state = lsecond(state->sub_states);
+ right_state = linitial(state1->sub_states);
+ left_state = lsecond(state1->sub_states);
}
/*
@@ -2924,7 +2916,10 @@ reduce_outer_joins_pass2(Node *jtnode,
jointype = JOIN_ANTI;
}
- /* Apply the jointype change, if any, to both jointree node and RTE */
+ /*
+ * Apply the jointype change, if any, to both jointree node and RTE.
+ * Also, if we changed an RTE to INNER, add its RTI to inner_reduced.
+ */
if (rtindex && jointype != j->jointype)
{
RangeTblEntry *rte = rt_fetch(rtindex, root->parse->rtable);
@@ -2932,6 +2927,9 @@ reduce_outer_joins_pass2(Node *jtnode,
Assert(rte->rtekind == RTE_JOIN);
Assert(rte->jointype == j->jointype);
rte->jointype = jointype;
+ if (jointype == JOIN_INNER)
+ state2->inner_reduced = bms_add_member(state2->inner_reduced,
+ rtindex);
}
j->jointype = jointype;
@@ -3012,7 +3010,8 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_nonnullable_vars = NIL;
pass_forced_null_vars = NIL;
}
- reduce_outer_joins_pass2(j->larg, left_state, root,
+ reduce_outer_joins_pass2(j->larg, left_state,
+ state2, root,
pass_nonnullable_rels,
pass_nonnullable_vars,
pass_forced_null_vars);
@@ -3034,7 +3033,8 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_nonnullable_vars = NIL;
pass_forced_null_vars = NIL;
}
- reduce_outer_joins_pass2(j->rarg, right_state, root,
+ reduce_outer_joins_pass2(j->rarg, right_state,
+ state2, root,
pass_nonnullable_rels,
pass_nonnullable_vars,
pass_forced_null_vars);
@@ -3047,6 +3047,19 @@ reduce_outer_joins_pass2(Node *jtnode,
(int) nodeTag(jtnode));
}
+/* Helper for reduce_outer_joins_pass2 */
+static void
+report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
+ int rtindex, Relids relids)
+{
+ reduce_outer_joins_partial_state *statep;
+
+ statep = palloc(sizeof(reduce_outer_joins_partial_state));
+ statep->full_join_rti = rtindex;
+ statep->unreduced_side = relids;
+ state2->partial_reduced = lappend(state2->partial_reduced, statep);
+}
+
/*
* remove_useless_result_rtes
@@ -3088,16 +3101,34 @@ reduce_outer_joins_pass2(Node *jtnode,
void
remove_useless_result_rtes(PlannerInfo *root)
{
+ Relids dropped_outer_joins = NULL;
ListCell *cell;
/* Top level of jointree must always be a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
/* Recurse ... */
root->parse->jointree = (FromExpr *)
- remove_useless_results_recurse(root, (Node *) root->parse->jointree);
+ remove_useless_results_recurse(root,
+ (Node *) root->parse->jointree,
+ &dropped_outer_joins);
/* We should still have a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
+ /*
+ * If we removed any outer-join nodes from the jointree, run around and
+ * remove references to those joins as nulling rels. (There could be such
+ * references in PHVs that we pulled up out of the original subquery that
+ * the RESULT rel replaced. This is kosher on the grounds that we now
+ * know that such an outer join wouldn't really have nulled anything.) We
+ * don't do this during the main recursion, for simplicity and because we
+ * can handle all such joins in a single pass over the parse tree.
+ */
+ if (!bms_is_empty(dropped_outer_joins))
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ dropped_outer_joins,
+ NULL);
+
/*
* Remove any PlanRowMark referencing an RTE_RESULT RTE. We obviously
* must do that for any RTE_RESULT that we just removed. But one for a
@@ -3123,9 +3154,12 @@ remove_useless_result_rtes(PlannerInfo *root)
* Recursive guts of remove_useless_result_rtes.
*
* This recursively processes the jointree and returns a modified jointree.
+ * In addition, the RT indexes of any removed outer-join nodes are added to
+ * *dropped_outer_joins.
*/
static Node *
-remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
+remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Relids *dropped_outer_joins)
{
Assert(jtnode != NULL);
if (IsA(jtnode, RangeTblRef))
@@ -3153,7 +3187,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
int varno;
/* Recursively transform child ... */
- child = remove_useless_results_recurse(root, child);
+ child = remove_useless_results_recurse(root, child,
+ dropped_outer_joins);
/* ... and stick it back into the tree */
lfirst(cell) = child;
@@ -3202,8 +3237,10 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
int varno;
/* First, recurse */
- j->larg = remove_useless_results_recurse(root, j->larg);
- j->rarg = remove_useless_results_recurse(root, j->rarg);
+ j->larg = remove_useless_results_recurse(root, j->larg,
+ dropped_outer_joins);
+ j->rarg = remove_useless_results_recurse(root, j->rarg,
+ dropped_outer_joins);
/* Apply join-type-specific optimization rules */
switch (j->jointype)
@@ -3271,6 +3308,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
!find_dependent_phvs(root, varno)))
{
remove_result_refs(root, varno, j->larg);
+ *dropped_outer_joins = bms_add_member(*dropped_outer_joins,
+ j->rtindex);
jtnode = j->larg;
}
break;
@@ -3281,6 +3320,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
!find_dependent_phvs(root, varno)))
{
remove_result_refs(root, varno, j->rarg);
+ *dropped_outer_joins = bms_add_member(*dropped_outer_joins,
+ j->rtindex);
jtnode = j->rarg;
}
break;
@@ -3295,11 +3336,14 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
* Unlike the LEFT/RIGHT cases, we just Assert that there are
* no PHVs that need to be evaluated at the semijoin's RHS,
* since the rest of the query couldn't reference any outputs
- * of the semijoin's RHS.
+ * of the semijoin's RHS. Also, we don't need to worry about
+ * removing traces of the join's rtindex, since it hasn't got
+ * one.
*/
if ((varno = get_result_relid(root, j->rarg)) != 0)
{
Assert(!find_dependent_phvs(root, varno));
+ Assert(j->rtindex == 0);
remove_result_refs(root, varno, j->larg);
if (j->quals)
jtnode = (Node *)
@@ -3368,7 +3412,7 @@ remove_result_refs(PlannerInfo *root, int varno, Node *newjtloc)
{
Relids subrelids;
- subrelids = get_relids_in_jointree(newjtloc, false);
+ subrelids = get_relids_in_jointree(newjtloc, true, false);
Assert(!bms_is_empty(subrelids));
substitute_phv_relids((Node *) root->parse, varno, subrelids);
fix_append_rel_relids(root->append_rel_list, varno, subrelids);
@@ -3480,7 +3524,7 @@ find_dependent_phvs_in_jointree(PlannerInfo *root, Node *node, int varno)
* are not marked LATERAL, though, since they couldn't possibly contain
* any cross-references to other RTEs.
*/
- subrelids = get_relids_in_jointree(node, false);
+ subrelids = get_relids_in_jointree(node, false, false);
relid = -1;
while ((relid = bms_next_member(subrelids, relid)) >= 0)
{
@@ -3624,11 +3668,17 @@ fix_append_rel_relids(List *append_rel_list, int varno, Relids subrelids)
/*
* get_relids_in_jointree: get set of RT indexes present in a jointree
*
- * If include_joins is true, join RT indexes are included; if false,
- * only base rels are included.
+ * Base-relation relids are always included in the result.
+ * If include_outer_joins is true, outer-join RT indexes are included.
+ * If include_inner_joins is true, inner-join RT indexes are included.
+ *
+ * Note that for most purposes in the planner, outer joins are included
+ * in standard relid sets. Setting include_inner_joins true is only
+ * appropriate for special purposes during subquery flattening.
*/
Relids
-get_relids_in_jointree(Node *jtnode, bool include_joins)
+get_relids_in_jointree(Node *jtnode, bool include_outer_joins,
+ bool include_inner_joins)
{
Relids result = NULL;
@@ -3649,18 +3699,34 @@ get_relids_in_jointree(Node *jtnode, bool include_joins)
{
result = bms_join(result,
get_relids_in_jointree(lfirst(l),
- include_joins));
+ include_outer_joins,
+ include_inner_joins));
}
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- result = get_relids_in_jointree(j->larg, include_joins);
+ result = get_relids_in_jointree(j->larg,
+ include_outer_joins,
+ include_inner_joins);
result = bms_join(result,
- get_relids_in_jointree(j->rarg, include_joins));
- if (include_joins && j->rtindex)
- result = bms_add_member(result, j->rtindex);
+ get_relids_in_jointree(j->rarg,
+ include_outer_joins,
+ include_inner_joins));
+ if (j->rtindex)
+ {
+ if (j->jointype == JOIN_INNER)
+ {
+ if (include_inner_joins)
+ result = bms_add_member(result, j->rtindex);
+ }
+ else
+ {
+ if (include_outer_joins)
+ result = bms_add_member(result, j->rtindex);
+ }
+ }
}
else
elog(ERROR, "unrecognized node type: %d",
@@ -3669,7 +3735,7 @@ get_relids_in_jointree(Node *jtnode, bool include_joins)
}
/*
- * get_relids_for_join: get set of base RT indexes making up a join
+ * get_relids_for_join: get set of base+OJ RT indexes making up a join
*/
Relids
get_relids_for_join(Query *query, int joinrelid)
@@ -3680,7 +3746,7 @@ get_relids_for_join(Query *query, int joinrelid)
joinrelid);
if (!jtnode)
elog(ERROR, "could not find join node %d", joinrelid);
- return get_relids_in_jointree(jtnode, false);
+ return get_relids_in_jointree(jtnode, true, false);
}
/*
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index 62cccf9d87..e793a4c85b 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -228,6 +228,12 @@ adjust_appendrel_attrs_mutator(Node *node,
if (var->varlevelsup != 0)
return (Node *) var; /* no changes needed */
+ /*
+ * You might think we need to adjust var->varnullingrels, but that
+ * shouldn't need any changes. It will contain outer-join relids,
+ * while the transformation we are making affects only baserels.
+ */
+
for (cnt = 0; cnt < nappinfos; cnt++)
{
if (var->varno == appinfos[cnt]->parent_relid)
@@ -348,6 +354,8 @@ adjust_appendrel_attrs_mutator(Node *node,
var = copyObject(ridinfo->rowidvar);
/* ... but use the correct relid */
var->varno = leaf_relid;
+ /* identity vars shouldn't have nulling rels */
+ Assert(var->varnullingrels == NULL);
/* varnosyn in the RowIdentityVarInfo is probably wrong */
var->varnosyn = 0;
var->varattnosyn = 0;
@@ -392,8 +400,11 @@ adjust_appendrel_attrs_mutator(Node *node,
(void *) context);
/* now fix PlaceHolderVar's relid sets */
if (phv->phlevelsup == 0)
- phv->phrels = adjust_child_relids(phv->phrels, context->nappinfos,
- context->appinfos);
+ {
+ phv->phrels = adjust_child_relids(phv->phrels,
+ nappinfos, appinfos);
+ /* as above, we needn't touch phnullingrels */
+ }
return (Node *) phv;
}
/* Shouldn't need to handle planner auxiliary nodes here */
@@ -688,7 +699,11 @@ get_translated_update_targetlist(PlannerInfo *root, Index relid,
/*
* find_appinfos_by_relids
- * Find AppendRelInfo structures for all relations specified by relids.
+ * Find AppendRelInfo structures for base relations listed in relids.
+ *
+ * The relids argument is typically a join relation's relids, which can
+ * include outer-join RT indexes in addition to baserels. We silently
+ * ignore the outer joins.
*
* The AppendRelInfos are returned in an array, which can be pfree'd by the
* caller. *nappinfos is set to the number of entries in the array.
@@ -700,8 +715,9 @@ find_appinfos_by_relids(PlannerInfo *root, Relids relids, int *nappinfos)
int cnt = 0;
int i;
- *nappinfos = bms_num_members(relids);
- appinfos = (AppendRelInfo **) palloc(sizeof(AppendRelInfo *) * *nappinfos);
+ /* Allocate an array that's certainly big enough */
+ appinfos = (AppendRelInfo **)
+ palloc(sizeof(AppendRelInfo *) * bms_num_members(relids));
i = -1;
while ((i = bms_next_member(relids, i)) >= 0)
@@ -709,10 +725,17 @@ find_appinfos_by_relids(PlannerInfo *root, Relids relids, int *nappinfos)
AppendRelInfo *appinfo = root->append_rel_array[i];
if (!appinfo)
+ {
+ /* Probably i is an OJ index, but let's check */
+ if (find_base_rel_ignore_join(root, i) == NULL)
+ continue;
+ /* It's a base rel, but we lack an append_rel_array entry */
elog(ERROR, "child rel %d not found in append_rel_array", i);
+ }
appinfos[cnt++] = appinfo;
}
+ *nappinfos = cnt;
return appinfos;
}
@@ -754,6 +777,7 @@ add_row_identity_var(PlannerInfo *root, Var *orig_var,
Assert(IsA(orig_var, Var));
Assert(orig_var->varno == rtindex);
Assert(orig_var->varlevelsup == 0);
+ Assert(orig_var->varnullingrels == NULL);
/*
* If we're doing non-inherited UPDATE/DELETE/MERGE, there's little need
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 533df86ff7..c82fd451b2 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -2022,14 +2022,16 @@ is_pseudo_constant_clause_relids(Node *clause, Relids relids)
* NumRelids
* (formerly clause_relids)
*
- * Returns the number of different relations referenced in 'clause'.
+ * Returns the number of different base relations referenced in 'clause'.
*/
int
NumRelids(PlannerInfo *root, Node *clause)
{
+ int result;
Relids varnos = pull_varnos(root, clause);
- int result = bms_num_members(varnos);
+ varnos = bms_del_members(varnos, root->outer_join_rels);
+ result = bms_num_members(varnos);
bms_free(varnos);
return result;
}
diff --git a/src/backend/optimizer/util/joininfo.c b/src/backend/optimizer/util/joininfo.c
index d4cffdb198..afd243f5d8 100644
--- a/src/backend/optimizer/util/joininfo.c
+++ b/src/backend/optimizer/util/joininfo.c
@@ -88,8 +88,8 @@ have_relevant_joinclause(PlannerInfo *root,
* not depend on context).
*
* 'restrictinfo' describes the join clause
- * 'join_relids' is the list of relations participating in the join clause
- * (there must be more than one)
+ * 'join_relids' is the set of relations participating in the join clause
+ * (some of these could be outer joins)
*/
void
add_join_clause_to_rels(PlannerInfo *root,
@@ -101,8 +101,11 @@ add_join_clause_to_rels(PlannerInfo *root,
cur_relid = -1;
while ((cur_relid = bms_next_member(join_relids, cur_relid)) >= 0)
{
- RelOptInfo *rel = find_base_rel(root, cur_relid);
+ RelOptInfo *rel = find_base_rel_ignore_join(root, cur_relid);
+ /* We only need to add the clause to baserels */
+ if (rel == NULL)
+ continue;
rel->joininfo = lappend(rel->joininfo, restrictinfo);
}
}
@@ -115,8 +118,8 @@ add_join_clause_to_rels(PlannerInfo *root,
* discover that a relation need not be joined at all.
*
* 'restrictinfo' describes the join clause
- * 'join_relids' is the list of relations participating in the join clause
- * (there must be more than one)
+ * 'join_relids' is the set of relations participating in the join clause
+ * (some of these could be outer joins)
*/
void
remove_join_clause_from_rels(PlannerInfo *root,
@@ -128,7 +131,11 @@ remove_join_clause_from_rels(PlannerInfo *root,
cur_relid = -1;
while ((cur_relid = bms_next_member(join_relids, cur_relid)) >= 0)
{
- RelOptInfo *rel = find_base_rel(root, cur_relid);
+ RelOptInfo *rel = find_base_rel_ignore_join(root, cur_relid);
+
+ /* We would only have added the clause to baserels */
+ if (rel == NULL)
+ continue;
/*
* Remove the restrictinfo from the list. Pointer comparison is
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index b1363df065..a62d4587ea 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -338,7 +338,9 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
sjinfo.syn_lefthand = sjinfo.min_lefthand;
sjinfo.syn_righthand = sjinfo.min_righthand;
sjinfo.jointype = JOIN_INNER;
+ sjinfo.ojrelid = 0;
/* we don't bother trying to make the remaining fields valid */
+ sjinfo.strict_relids = NULL;
sjinfo.lhs_strict = false;
sjinfo.delay_upper_joins = false;
sjinfo.semi_can_btree = false;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index e10561d843..b11d50bbe9 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -1307,7 +1307,7 @@ create_append_path(PlannerInfo *root,
* Apply query-wide LIMIT if known and path is for sole base relation.
* (Handling this at this low level is a bit klugy.)
*/
- if (root != NULL && bms_equal(rel->relids, root->all_baserels))
+ if (root != NULL && bms_equal(rel->relids, root->all_query_rels))
pathnode->limit_tuples = root->limit_tuples;
else
pathnode->limit_tuples = -1.0;
@@ -1436,7 +1436,7 @@ create_merge_append_path(PlannerInfo *root,
* Apply query-wide LIMIT if known and path is for sole base relation.
* (Handling this at this low level is a bit klugy.)
*/
- if (bms_equal(rel->relids, root->all_baserels))
+ if (bms_equal(rel->relids, root->all_query_rels))
pathnode->limit_tuples = root->limit_tuples;
else
pathnode->limit_tuples = -1.0;
diff --git a/src/backend/optimizer/util/placeholder.c b/src/backend/optimizer/util/placeholder.c
index c7bfa293c9..26c2455a64 100644
--- a/src/backend/optimizer/util/placeholder.c
+++ b/src/backend/optimizer/util/placeholder.c
@@ -32,8 +32,14 @@ static void find_placeholders_in_expr(PlannerInfo *root, Node *expr);
* make_placeholder_expr
* Make a PlaceHolderVar for the given expression.
*
- * phrels is the syntactic location (as a set of baserels) to attribute
+ * phrels is the syntactic location (as a set of relids) to attribute
* to the expression.
+ *
+ * The caller is responsible for adjusting phlevelsup and phnullingrels
+ * as needed. Because we do not know here which query level the PHV
+ * will be associated with, it's important that this function touches
+ * only root->glob; messing with other parts of PlannerInfo would be
+ * likely to do the wrong thing.
*/
PlaceHolderVar *
make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
@@ -42,8 +48,9 @@ make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
phv->phexpr = expr;
phv->phrels = phrels;
+ phv->phnullingrels = NULL; /* caller may change this later */
phv->phid = ++(root->glob->lastPHId);
- phv->phlevelsup = 0;
+ phv->phlevelsup = 0; /* caller may change this later */
return phv;
}
@@ -92,6 +99,15 @@ find_placeholder_info(PlannerInfo *root, PlaceHolderVar *phv)
phinfo->phid = phv->phid;
phinfo->ph_var = copyObject(phv);
+ /*
+ * By convention, phinfo->ph_var->phnullingrels is always empty, since the
+ * PlaceHolderInfo represents the initially-calculated state of the
+ * PlaceHolderVar. PlaceHolderVars appearing in the query tree might have
+ * varying values of phnullingrels, reflecting outer joins applied above
+ * the calculation level.
+ */
+ phinfo->ph_var->phnullingrels = NULL;
+
/*
* Any referenced rels that are outside the PHV's syntactic scope are
* LATERAL references, which should be included in ph_lateral but not in
@@ -344,6 +360,8 @@ update_placeholder_eval_levels(PlannerInfo *root, SpecialJoinInfo *new_sjinfo)
sjinfo->min_lefthand);
eval_at = bms_add_members(eval_at,
sjinfo->min_righthand);
+ if (sjinfo->ojrelid)
+ eval_at = bms_add_member(eval_at, sjinfo->ojrelid);
/* we'll need another iteration */
found_some = true;
}
@@ -418,6 +436,14 @@ add_placeholders_to_base_rels(PlannerInfo *root)
{
RelOptInfo *rel = find_base_rel(root, varno);
+ /*
+ * As in add_vars_to_targetlist(), a value computed at scan level
+ * has not yet been nulled by any outer join, so its phnullingrels
+ * should be empty.
+ */
+ Assert(phinfo->ph_var->phnullingrels == NULL);
+
+ /* Copying the PHV might be unnecessary here, but be safe */
rel->reltarget->exprs = lappend(rel->reltarget->exprs,
copyObject(phinfo->ph_var));
/* reltarget's cost and width fields will be updated later */
@@ -440,7 +466,8 @@ add_placeholders_to_base_rels(PlannerInfo *root)
*/
void
add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *outer_rel, RelOptInfo *inner_rel)
+ RelOptInfo *outer_rel, RelOptInfo *inner_rel,
+ SpecialJoinInfo *sjinfo)
{
Relids relids = joinrel->relids;
ListCell *lc;
@@ -471,9 +498,17 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
if (!bms_is_subset(phinfo->ph_eval_at, outer_rel->relids) &&
!bms_is_subset(phinfo->ph_eval_at, inner_rel->relids))
{
- PlaceHolderVar *phv = phinfo->ph_var;
+ /* Copying might be unnecessary here, but be safe */
+ PlaceHolderVar *phv = copyObject(phinfo->ph_var);
QualCost cost;
+ /*
+ * It'll start out not nulled by anything. Joins above
+ * this one might add to its phnullingrels later, in much
+ * the same way as for Vars.
+ */
+ Assert(phv->phnullingrels == NULL);
+
joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
phv);
cost_qual_eval_node(&cost, (Node *) phv->phexpr, root);
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index edcdd0a360..b088ed0415 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -39,7 +39,7 @@ typedef struct JoinHashEntry
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *input_rel);
+ RelOptInfo *input_rel, int ojrelid);
static List *build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outer_rel,
@@ -58,7 +58,8 @@ static void set_foreign_rel_properties(RelOptInfo *joinrel,
static void add_join_rel(PlannerInfo *root, RelOptInfo *joinrel);
static void build_joinrel_partition_info(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
- List *restrictlist, JoinType jointype);
+ SpecialJoinInfo *sjinfo,
+ List *restrictlist);
static bool have_partkey_equi_join(RelOptInfo *joinrel,
RelOptInfo *rel1, RelOptInfo *rel2,
JoinType jointype, List *restrictlist);
@@ -66,7 +67,8 @@ static int match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel,
bool strict_op);
static void set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
- JoinType jointype);
+ SpecialJoinInfo *sjinfo);
+static Node *add_nullingrel_to(Node *node, int relid);
static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *parentrel,
RelOptInfo *childrel,
@@ -367,7 +369,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
/*
* find_base_rel
- * Find a base or other relation entry, which must already exist.
+ * Find a base or otherrel relation entry, which must already exist.
*/
RelOptInfo *
find_base_rel(PlannerInfo *root, int relid)
@@ -388,6 +390,44 @@ find_base_rel(PlannerInfo *root, int relid)
return NULL; /* keep compiler quiet */
}
+/*
+ * find_base_rel_ignore_join
+ * Find a base or otherrel relation entry, which must already exist.
+ *
+ * Unlike find_base_rel, if relid references an outer join then this
+ * will return NULL rather than raising an error. This is convenient
+ * for callers that must deal with relid sets including both base and
+ * outer joins.
+ */
+RelOptInfo *
+find_base_rel_ignore_join(PlannerInfo *root, int relid)
+{
+ Assert(relid > 0);
+
+ if (relid < root->simple_rel_array_size)
+ {
+ RelOptInfo *rel;
+ RangeTblEntry *rte;
+
+ rel = root->simple_rel_array[relid];
+ if (rel)
+ return rel;
+
+ /*
+ * We could just return NULL here, but for debugging purposes it seems
+ * best to actually verify that the relid is an outer join and not
+ * something weird.
+ */
+ rte = root->simple_rte_array[relid];
+ if (rte && rte->rtekind == RTE_JOIN && rte->jointype != JOIN_INNER)
+ return NULL;
+ }
+
+ elog(ERROR, "no relation entry for relid %d", relid);
+
+ return NULL; /* keep compiler quiet */
+}
+
/*
* build_join_rel_hash
* Construct the auxiliary hash table for join relations.
@@ -687,9 +727,11 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ build_joinrel_tlist(root, joinrel, outer_rel,
+ (sjinfo->jointype == JOIN_FULL) ? sjinfo->ojrelid : 0);
+ build_joinrel_tlist(root, joinrel, inner_rel,
+ (sjinfo->jointype != JOIN_INNER) ? sjinfo->ojrelid : 0);
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel, sjinfo);
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -721,8 +763,8 @@ build_join_rel(PlannerInfo *root,
joinrel->has_eclass_joins = has_relevant_eclass_joinclause(root, joinrel);
/* Store the partition information. */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- sjinfo->jointype);
+ build_joinrel_partition_info(joinrel, outer_rel, inner_rel, sjinfo,
+ restrictlist);
/*
* Set estimates of the joinrel's size.
@@ -778,16 +820,14 @@ build_join_rel(PlannerInfo *root,
* 'parent_joinrel' is the RelOptInfo representing the join between parent
* relations. Some of the members of new RelOptInfo are produced by
* translating corresponding members of this RelOptInfo
- * 'sjinfo': child-join context info
* 'restrictlist': list of RestrictInfo nodes that apply to this particular
* pair of joinable relations
- * 'jointype' is the join type (inner, left, full, etc)
+ * 'sjinfo': child join's join-type details
*/
RelOptInfo *
build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
RelOptInfo *inner_rel, RelOptInfo *parent_joinrel,
- List *restrictlist, SpecialJoinInfo *sjinfo,
- JoinType jointype)
+ List *restrictlist, SpecialJoinInfo *sjinfo)
{
RelOptInfo *joinrel = makeNode(RelOptInfo);
AppendRelInfo **appinfos;
@@ -801,6 +841,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->reloptkind = RELOPT_OTHER_JOINREL;
joinrel->relids = bms_union(outer_rel->relids, inner_rel->relids);
+ if (sjinfo->ojrelid != 0)
+ joinrel->relids = bms_add_member(joinrel->relids, sjinfo->ojrelid);
joinrel->rows = 0;
/* cheap startup cost is interesting iff not all tuples to be retrieved */
joinrel->consider_startup = (root->tuple_fraction > 0);
@@ -887,8 +929,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->has_eclass_joins = parent_joinrel->has_eclass_joins;
/* Is the join between partitions itself partitioned? */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- jointype);
+ build_joinrel_partition_info(joinrel, outer_rel, inner_rel, sjinfo,
+ restrictlist);
/* Child joinrel is parallel safe if parent is parallel safe. */
joinrel->consider_parallel = parent_joinrel->consider_parallel;
@@ -968,12 +1010,15 @@ min_join_parameterization(PlannerInfo *root,
* Vars from the specified input rel's tlist to the join rel's tlist.
* Likewise for any PlaceHolderVars emitted by the input rel.
*
+ * If the join can null Vars from this input relation, pass its RT index
+ * (if any) as ojrelid; if not, pass zero.
+ *
* We also compute the expected width of the join's output, making use
* of data that was cached at the baserel level by set_rel_width().
*/
static void
build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *input_rel)
+ RelOptInfo *input_rel, int ojrelid)
{
Relids relids = joinrel->relids;
ListCell *vars;
@@ -993,7 +1038,17 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
/* Is it still needed above this joinrel? */
if (bms_nonempty_difference(phinfo->ph_needed, relids))
{
- /* Yup, add it to the output */
+ /*
+ * Yup, add it to the output. If this join potentially nulls
+ * this input, we have to update the PHV's phnullingrels,
+ * which means making a copy.
+ */
+ if (ojrelid != 0)
+ {
+ phv = copyObject(phv);
+ phv->phnullingrels = bms_add_member(phv->phnullingrels, ojrelid);
+ }
+
joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
phv);
/* Bubbling up the precomputed result has cost zero */
@@ -1017,9 +1072,7 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
RowIdentityVarInfo *ridinfo = (RowIdentityVarInfo *)
list_nth(root->row_identity_vars, var->varattno - 1);
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- var);
- /* Vars have cost zero, so no need to adjust reltarget->cost */
+ /* Update reltarget width estimate from RowIdentityVarInfo */
joinrel->reltarget->width += ridinfo->rowidwidth;
}
else
@@ -1032,15 +1085,28 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
/* Is it still needed above this joinrel? */
ndx = var->varattno - baserel->min_attr;
- if (bms_nonempty_difference(baserel->attr_needed[ndx], relids))
- {
- /* Yup, add it to the output */
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- var);
- /* Vars have cost zero, so no need to adjust reltarget->cost */
- joinrel->reltarget->width += baserel->attr_widths[ndx];
- }
+ if (!bms_nonempty_difference(baserel->attr_needed[ndx], relids))
+ continue; /* nope, skip it */
+
+ /* Update reltarget width estimate from baserel's attr_widths */
+ joinrel->reltarget->width += baserel->attr_widths[ndx];
}
+
+ /*
+ * Add the Var to the output. If this join potentially nulls this
+ * input, we have to update the Var's varnullingrels, which means
+ * making a copy.
+ */
+ if (ojrelid != 0)
+ {
+ var = copyObject(var);
+ var->varnullingrels = bms_add_member(var->varnullingrels, ojrelid);
+ }
+
+ joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
+ var);
+
+ /* Vars have cost zero, so no need to adjust reltarget->cost */
}
}
@@ -1059,7 +1125,7 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
* is not handled in the sub-relations, so it depends on which
* sub-relations are considered.
*
- * If a join clause from an input relation refers to base rels still not
+ * If a join clause from an input relation refers to base+OJ rels still not
* present in the joinrel, then it is still a join clause for the joinrel;
* we put it into the joininfo list for the joinrel. Otherwise,
* the clause is now a restrict clause for the joined relation, and we
@@ -1660,8 +1726,8 @@ find_param_path_info(RelOptInfo *rel, Relids required_outer)
*/
static void
build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
- RelOptInfo *inner_rel, List *restrictlist,
- JoinType jointype)
+ RelOptInfo *inner_rel, SpecialJoinInfo *sjinfo,
+ List *restrictlist)
{
PartitionScheme part_scheme;
@@ -1688,7 +1754,7 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
!inner_rel->consider_partitionwise_join ||
outer_rel->part_scheme != inner_rel->part_scheme ||
!have_partkey_equi_join(joinrel, outer_rel, inner_rel,
- jointype, restrictlist))
+ sjinfo->jointype, restrictlist))
{
Assert(!IS_PARTITIONED_REL(joinrel));
return;
@@ -1712,7 +1778,7 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* child-join relations of the join relation in try_partitionwise_join().
*/
joinrel->part_scheme = part_scheme;
- set_joinrel_partition_key_exprs(joinrel, outer_rel, inner_rel, jointype);
+ set_joinrel_partition_key_exprs(joinrel, outer_rel, inner_rel, sjinfo);
/*
* Set the consider_partitionwise_join flag.
@@ -1892,6 +1958,23 @@ match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel, bool strict_op)
{
if (equal(lfirst(lc), expr))
return cnt;
+
+ /*
+ * XXX For the moment, also allow a match if we have Vars that
+ * match except for varnullingrels. This may be indicative of a
+ * bug, although given the restriction to strict join operators,
+ * it could be okay.
+ */
+ if (IsA(expr, Var) && IsA(lfirst(lc), Var))
+ {
+ Var *v1 = (Var *) expr;
+ Var *v2 = (Var *) lfirst(lc);
+
+ if (v1->varno == v2->varno &&
+ v1->varattno == v2->varattno &&
+ v1->varlevelsup == v2->varlevelsup)
+ return cnt;
+ }
}
}
@@ -1905,7 +1988,7 @@ match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel, bool strict_op)
static void
set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
- JoinType jointype)
+ SpecialJoinInfo *sjinfo)
{
PartitionScheme part_scheme = joinrel->part_scheme;
int partnatts = part_scheme->partnatts;
@@ -1931,7 +2014,7 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
List *nullable_partexpr = NIL;
ListCell *lc;
- switch (jointype)
+ switch (sjinfo->jointype)
{
/*
* A join relation resulting from an INNER join may be
@@ -2007,18 +2090,37 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
* partitionwise nesting of any outer join.) We assume no
* type coercions are needed to make the coalesce expressions,
* since columns of different types won't have gotten
- * classified as the same PartitionScheme.
+ * classified as the same PartitionScheme. However, we do
+ * have to worry about marking the COALESCE inputs as nullable
+ * by the full join, else these won't match the real thing.
*/
foreach(lc, list_concat_copy(outer_expr, outer_null_expr))
{
Node *larg = (Node *) lfirst(lc);
ListCell *lc2;
+ /* Insert nullingrel, or skip it if we can't */
+ larg = add_nullingrel_to(larg, sjinfo->ojrelid);
+ if (larg == NULL)
+ continue;
+
foreach(lc2, list_concat_copy(inner_expr, inner_null_expr))
{
Node *rarg = (Node *) lfirst(lc2);
- CoalesceExpr *c = makeNode(CoalesceExpr);
+ CoalesceExpr *c;
+
+ /* Forget it if coercions would be needed */
+ if (exprType(larg) != exprType(rarg) ||
+ exprCollation(larg) != exprCollation(rarg))
+ continue;
+ /* Insert nullingrel, or skip it if we can't */
+ rarg = add_nullingrel_to(rarg, sjinfo->ojrelid);
+ if (rarg == NULL)
+ continue;
+
+ /* Now we can build a valid merged join variable */
+ c = makeNode(CoalesceExpr);
c->coalescetype = exprType(larg);
c->coalescecollid = exprCollation(larg);
c->args = list_make2(larg, rarg);
@@ -2029,7 +2131,8 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
break;
default:
- elog(ERROR, "unrecognized join type: %d", (int) jointype);
+ elog(ERROR, "unrecognized join type: %d",
+ (int) sjinfo->jointype);
}
joinrel->partexprs[cnt] = partexpr;
@@ -2037,6 +2140,54 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
}
}
+/*
+ * Attempt to add relid to nullingrels of a FULL JOIN USING variable.
+ * Returns the modified expression if successful, or NULL if we failed.
+ *
+ * We currently don't support any cases where type coercion is involved,
+ * so only plain Vars and COALESCE nodes need be handled. However, we
+ * do need to support nested COALESCEs, so recursion is required.
+ */
+static Node *
+add_nullingrel_to(Node *node, int relid)
+{
+ if (IsA(node, Var))
+ {
+ /* Copy so we can modify it... */
+ Var *var = (Var *) copyObject(node);
+
+ /* ... and insert the correct nullingrel marker */
+ var->varnullingrels = bms_add_member(var->varnullingrels,
+ relid);
+ return (Node *) var;
+ }
+ if (IsA(node, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) node;
+ CoalesceExpr *newcexpr;
+ List *newargs = NIL;
+ ListCell *lc;
+
+ /* Try to modify each argument ... */
+ foreach(lc, cexpr->args)
+ {
+ Node *newarg = add_nullingrel_to((Node *) lfirst(lc), relid);
+
+ if (newarg == NULL)
+ return NULL;
+ newargs = lappend(newargs, newarg);
+ }
+ /* Success, so make the result node */
+ newcexpr = makeNode(CoalesceExpr);
+ newcexpr->coalescetype = cexpr->coalescetype;
+ newcexpr->coalescecollid = cexpr->coalescecollid;
+ newcexpr->args = newargs;
+ newcexpr->location = cexpr->location;
+ return (Node *) newcexpr;
+ }
+ return NULL;
+}
+
/*
* build_child_join_reltarget
* Set up a child-join relation's reltarget from a parent-join relation.
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index ef8df3d098..6902c2a9d7 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -116,6 +116,7 @@ make_restrictinfo_internal(PlannerInfo *root,
Relids nullable_relids)
{
RestrictInfo *restrictinfo = makeNode(RestrictInfo);
+ Relids baserels;
restrictinfo->clause = clause;
restrictinfo->orclause = orclause;
@@ -187,6 +188,20 @@ make_restrictinfo_internal(PlannerInfo *root,
else
restrictinfo->required_relids = restrictinfo->clause_relids;
+ /*
+ * Count the number of base rels appearing in clause_relids. To do this,
+ * we just delete rels mentioned in root->outer_join_rels and count the
+ * survivors. Because we are called during deconstruct_jointree which is
+ * the same tree walk that populates outer_join_rels, this is a little bit
+ * unsafe-looking; but it should be fine because the recursion in
+ * deconstruct_jointree should already have visited any outer join that
+ * could be mentioned in this clause.
+ */
+ baserels = bms_difference(restrictinfo->clause_relids,
+ root->outer_join_rels);
+ restrictinfo->num_base_rels = bms_num_members(baserels);
+ bms_free(baserels);
+
/*
* Fill in all the cacheable fields with "not yet set" markers. None of
* these will be computed until/unless needed. Note in particular that we
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
index 7db86c39ef..8d8c9136f8 100644
--- a/src/backend/optimizer/util/var.c
+++ b/src/backend/optimizer/util/var.c
@@ -88,6 +88,9 @@ static Relids alias_relid_set(Query *query, Relids relids);
* Create a set of all the distinct varnos present in a parsetree.
* Only varnos that reference level-zero rtable entries are considered.
*
+ * The result includes outer-join relids mentioned in Var.varnullingrels and
+ * PlaceHolderVar.phnullingrels fields in the parsetree.
+ *
* "root" can be passed as NULL if it is not necessary to process
* PlaceHolderVars.
*
@@ -153,7 +156,11 @@ pull_varnos_walker(Node *node, pull_varnos_context *context)
Var *var = (Var *) node;
if (var->varlevelsup == context->sublevels_up)
+ {
context->varnos = bms_add_member(context->varnos, var->varno);
+ context->varnos = bms_add_members(context->varnos,
+ var->varnullingrels);
+ }
return false;
}
if (IsA(node, CurrentOfExpr))
@@ -244,6 +251,14 @@ pull_varnos_walker(Node *node, pull_varnos_context *context)
context->varnos = bms_join(context->varnos,
newevalat);
}
+
+ /*
+ * In all three cases, include phnullingrels in the result. We
+ * don't worry about possibly needing to translate it, because
+ * appendrels only translate varnos of baserels, not outer joins.
+ */
+ context->varnos = bms_add_members(context->varnos,
+ phv->phnullingrels);
return false; /* don't recurse into expression */
}
}
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index 50b588e3d0..bd41987f04 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -2204,7 +2204,7 @@ rowcomparesel(PlannerInfo *root,
else
{
/*
- * Otherwise, it's a join if there's more than one relation used.
+ * Otherwise, it's a join if there's more than one base relation used.
*/
is_join_clause = (NumRelids(root, (Node *) opargs) > 1);
}
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 024ba376b4..0bad27f07e 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -243,13 +243,26 @@ struct PlannerInfo
struct AppendRelInfo **append_rel_array pg_node_attr(read_write_ignore);
/*
- * all_baserels is a Relids set of all base relids (but not "other"
- * relids) in the query; that is, the Relids identifier of the final join
- * we need to form. This is computed in make_one_rel, just before we
- * start making Paths.
+ * all_baserels is a Relids set of all base relids (but not joins or
+ * "other" relids) in the query. This is computed in make_one_rel, just
+ * before we start making Paths.
*/
Relids all_baserels;
+ /*
+ * outer_join_rels is a Relids set of all outer-join relids in the query.
+ * This is computed in deconstruct_jointree.
+ */
+ Relids outer_join_rels;
+
+ /*
+ * all_query_rels is a Relids set of all base relids and outer join relids
+ * (but not "other" relids) in the query. This is the Relids identifier
+ * of the final join we need to form. This is computed in make_one_rel,
+ * just before we start making Paths.
+ */
+ Relids all_query_rels;
+
/*
* nullable_baserels is a Relids set of base relids that are nullable by
* some outer join in the jointree; these are rels that are potentially
@@ -319,7 +332,7 @@ struct PlannerInfo
List *right_join_clauses;
/*
- * list of RestrictInfos for mergejoinable full join clauses
+ * list of FullJoinClauseInfos for mergejoinable full join clauses
*/
List *full_join_clauses;
@@ -555,9 +568,10 @@ typedef struct PartitionSchemeData *PartitionScheme;
* or the output of a sub-SELECT or function that appears in the range table.
* In either case it is uniquely identified by an RT index. A "joinrel"
* is the joining of two or more base rels. A joinrel is identified by
- * the set of RT indexes for its component baserels. We create RelOptInfo
- * nodes for each baserel and joinrel, and store them in the PlannerInfo's
- * simple_rel_array and join_rel_list respectively.
+ * the set of RT indexes for its component baserels, along with RT indexes
+ * for any outer joins it has computed. We create RelOptInfo nodes for each
+ * baserel and joinrel, and store them in the PlannerInfo's simple_rel_array
+ * and join_rel_list respectively.
*
* Note that there is only one joinrel for any given set of component
* baserels, no matter what order we assemble them in; so an unordered
@@ -596,8 +610,10 @@ typedef struct PartitionSchemeData *PartitionScheme;
* Parts of this data structure are specific to various scan and join
* mechanisms. It didn't seem worth creating new node types for them.
*
- * relids - Set of base-relation identifiers; it is a base relation
- * if there is just one, a join relation if more than one
+ * relids - Set of relation identifiers (RT indexes). This is a base
+ * relation if there is just one, a join relation if more;
+ * in the join case, RT indexes of any outer joins formed
+ * at or below this join are included along with baserels
* rows - estimated number of tuples in the relation after restriction
* clauses have been applied (ie, output rows of a plan for it)
* consider_startup - true if there is any value in keeping plain paths for
@@ -809,7 +825,7 @@ typedef struct RelOptInfo
RelOptKind reloptkind;
/*
- * all relations included in this RelOptInfo; set of base relids
+ * all relations included in this RelOptInfo; set of base + OJ relids
* (rangetable indexes)
*/
Relids relids;
@@ -2293,17 +2309,17 @@ typedef struct LimitPath
* If a restriction clause references a single base relation, it will appear
* in the baserestrictinfo list of the RelOptInfo for that base rel.
*
- * If a restriction clause references more than one base rel, it will
+ * If a restriction clause references more than one base+OJ relation, it will
* appear in the joininfo list of every RelOptInfo that describes a strict
- * subset of the base rels mentioned in the clause. The joininfo lists are
+ * subset of the relations mentioned in the clause. The joininfo lists are
* used to drive join tree building by selecting plausible join candidates.
* The clause cannot actually be applied until we have built a join rel
- * containing all the base rels it references, however.
+ * containing all the relations it references, however.
*
- * When we construct a join rel that includes all the base rels referenced
+ * When we construct a join rel that includes all the relations referenced
* in a multi-relation restriction clause, we place that clause into the
* joinrestrictinfo lists of paths for the join rel, if neither left nor
- * right sub-path includes all base rels referenced in the clause. The clause
+ * right sub-path includes all relations referenced in the clause. The clause
* will be applied at that join level, and will not propagate any further up
* the join tree. (Note: the "predicate migration" code was once intended to
* push restriction clauses up and down the plan tree based on evaluation
@@ -2324,12 +2340,15 @@ typedef struct LimitPath
* or join to enforce that all members of each EquivalenceClass are in fact
* equal in all rows emitted by the scan or join.
*
- * When dealing with outer joins we have to be very careful about pushing qual
- * clauses up and down the tree. An outer join's own JOIN/ON conditions must
- * be evaluated exactly at that join node, unless they are "degenerate"
- * conditions that reference only Vars from the nullable side of the join.
- * Quals appearing in WHERE or in a JOIN above the outer join cannot be pushed
- * down below the outer join, if they reference any nullable Vars.
+ * The clause_relids field lists the base plus outer-join RT indexes that
+ * actually appear in the clause. required_relids lists the minimum set of
+ * relids needed to evaluate the clause; while this is often equal to
+ * clause_relids, it can be more. We will add relids to required_relids when
+ * we need to force an outer join ON clause to be evaluated exactly at the
+ * level of the outer join, which is true except when it is a "degenerate"
+ * condition that references only Vars from the nullable side of the join.
+ *
+ * XXX rewrite or remove me:
* RestrictInfo nodes contain a flag to indicate whether a qual has been
* pushed down to a lower level than its original syntactic placement in the
* join tree would suggest. If an outer join prevents us from pushing a qual
@@ -2451,13 +2470,16 @@ typedef struct RestrictInfo
/* true if known to contain no leaked Vars */
bool leakproof pg_node_attr(equal_ignore);
- /* to indicate if clause contains any volatile functions. */
+ /* indicates if clause contains any volatile functions */
VolatileFunctionStatus has_volatile pg_node_attr(equal_ignore);
/* see comment above */
Index security_level;
- /* The set of relids (varnos) actually referenced in the clause: */
+ /* number of base rels in clause_relids */
+ int num_base_rels pg_node_attr(equal_ignore);
+
+ /* The relids (varnos+varnullingrels) actually referenced in the clause: */
Relids clause_relids pg_node_attr(equal_ignore);
/* The set of relids required to evaluate the clause: */
@@ -2559,6 +2581,7 @@ typedef struct RestrictInfo
} RestrictInfo;
/*
+ * XXX this will need work:
* This macro embodies the correct way to test whether a RestrictInfo is
* "pushed down" to a given outer join, that is, should be treated as a filter
* clause rather than a join clause at that outer join. This is certainly so
@@ -2661,17 +2684,20 @@ typedef struct PlaceHolderVar
* We make SpecialJoinInfos for FULL JOINs even though there is no flexibility
* of planning for them, because this simplifies make_join_rel()'s API.
*
- * min_lefthand and min_righthand are the sets of base relids that must be
- * available on each side when performing the special join. lhs_strict is
- * true if the special join's condition cannot succeed when the LHS variables
- * are all NULL (this means that an outer join can commute with upper-level
+ * min_lefthand and min_righthand are the sets of base+OJ relids that must be
+ * available on each side when performing the special join.
+ *
+ * strict_relids is the set of base+OJ relids for which the special join's
+ * condition is strict, ie it cannot succeed if any of those rels produce
+ * an all-NULL row. lhs_strict reports whether any LHS rels appear in
+ * strict_relids (this means that an outer join can commute with upper-level
* outer joins even if it appears in their RHS). We don't bother to set
- * lhs_strict for FULL JOINs, however.
+ * strict_relids or lhs_strict for FULL JOINs, however.
*
* It is not valid for either min_lefthand or min_righthand to be empty sets;
* if they were, this would break the logic that enforces join order.
*
- * syn_lefthand and syn_righthand are the sets of base relids that are
+ * syn_lefthand and syn_righthand are the sets of base+OJ relids that are
* syntactically below this special join. (These are needed to help compute
* min_lefthand and min_righthand for higher joins.)
*
@@ -2693,14 +2719,18 @@ typedef struct PlaceHolderVar
* the inputs to make it a LEFT JOIN. So the allowed values of jointype
* in a join_info_list member are only LEFT, FULL, SEMI, or ANTI.
*
+ * ojrelid is the RT index of the join RTE representing this outer join,
+ * if there is one. It is zero when jointype is INNER or SEMI.
+ *
* For purposes of join selectivity estimation, we create transient
* SpecialJoinInfo structures for regular inner joins; so it is possible
* to have jointype == JOIN_INNER in such a structure, even though this is
* not allowed within join_info_list. We also create transient
* SpecialJoinInfos with jointype == JOIN_INNER for outer joins, since for
* cost estimation purposes it is sometimes useful to know the join size under
- * plain innerjoin semantics. Note that lhs_strict, delay_upper_joins, and
- * of course the semi_xxx fields are not set meaningfully within such structs.
+ * plain innerjoin semantics. Note that strict_relids, lhs_strict,
+ * delay_upper_joins, and of course the semi_xxx fields are not set
+ * meaningfully within such structs.
*/
#ifndef HAVE_SPECIALJOININFO_TYPEDEF
typedef struct SpecialJoinInfo SpecialJoinInfo;
@@ -2712,11 +2742,13 @@ struct SpecialJoinInfo
pg_node_attr(no_read)
NodeTag type;
- Relids min_lefthand; /* base relids in minimum LHS for join */
- Relids min_righthand; /* base relids in minimum RHS for join */
- Relids syn_lefthand; /* base relids syntactically within LHS */
- Relids syn_righthand; /* base relids syntactically within RHS */
+ Relids min_lefthand; /* base+OJ relids in minimum LHS for join */
+ Relids min_righthand; /* base+OJ relids in minimum RHS for join */
+ Relids syn_lefthand; /* base+OJ relids syntactically within LHS */
+ Relids syn_righthand; /* base+OJ relids syntactically within RHS */
JoinType jointype; /* always INNER, LEFT, FULL, SEMI, or ANTI */
+ Index ojrelid; /* outer join's RT index; 0 if none */
+ Relids strict_relids; /* joinclause is strict for these relids */
bool lhs_strict; /* joinclause is strict for some LHS rel */
bool delay_upper_joins; /* can't commute with upper RHS */
/* Remaining fields are set only for JOIN_SEMI jointype: */
@@ -2726,6 +2758,21 @@ struct SpecialJoinInfo
List *semi_rhs_exprs; /* righthand-side expressions of these ops */
};
+/*
+ * FULL JOIN clause info.
+ *
+ * We set aside every FULL JOIN ON clause that looks mergejoinable, and
+ * process it specially at the end of qual distribution.
+ */
+typedef struct FullJoinClauseInfo
+{
+ pg_node_attr(no_copy_equal, no_read)
+
+ NodeTag type;
+ RestrictInfo *rinfo; /* a mergejoinable FULL JOIN clause */
+ SpecialJoinInfo *sjinfo; /* the FULL JOIN's SpecialJoinInfo */
+} FullJoinClauseInfo;
+
/*
* Append-relation info.
*
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 050f00e79a..197234d44c 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -304,6 +304,7 @@ extern void expand_planner_arrays(PlannerInfo *root, int add_size);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
+extern RelOptInfo *find_base_rel_ignore_join(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
@@ -335,6 +336,6 @@ extern ParamPathInfo *find_param_path_info(RelOptInfo *rel,
extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
- SpecialJoinInfo *sjinfo, JoinType jointype);
+ SpecialJoinInfo *sjinfo);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/placeholder.h b/src/include/optimizer/placeholder.h
index 507dbc6175..030c1a03bb 100644
--- a/src/include/optimizer/placeholder.h
+++ b/src/include/optimizer/placeholder.h
@@ -27,6 +27,7 @@ extern void update_placeholder_eval_levels(PlannerInfo *root,
extern void fix_placeholder_input_needed_levels(PlannerInfo *root);
extern void add_placeholders_to_base_rels(PlannerInfo *root);
extern void add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *outer_rel, RelOptInfo *inner_rel);
+ RelOptInfo *outer_rel, RelOptInfo *inner_rel,
+ SpecialJoinInfo *sjinfo);
#endif /* PLACEHOLDER_H */
diff --git a/src/include/optimizer/prep.h b/src/include/optimizer/prep.h
index 2b11ff1d1f..ca03f32174 100644
--- a/src/include/optimizer/prep.h
+++ b/src/include/optimizer/prep.h
@@ -29,7 +29,8 @@ extern void pull_up_subqueries(PlannerInfo *root);
extern void flatten_simple_union_all(PlannerInfo *root);
extern void reduce_outer_joins(PlannerInfo *root);
extern void remove_useless_result_rtes(PlannerInfo *root);
-extern Relids get_relids_in_jointree(Node *jtnode, bool include_joins);
+extern Relids get_relids_in_jointree(Node *jtnode, bool include_outer_joins,
+ bool include_inner_joins);
extern Relids get_relids_for_join(Query *query, int joinrelid);
/*
commit 1d8bd7369430a7a7f2ff2f2ea48638e1e6301f6a
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu Aug 18 14:25:15 2022 -0400
Fix flatten_join_alias_vars() to handle varnullingrels correctly.
The remaining core regression test failures occur because
flatten_join_alias_vars() isn't doing the right thing. The
alias Var it needs to replace may have acquired varnullingrels
bits signifying the effect of upper outer joins, and if so we
must preserve that information in the replacement expression.
The simplest way to do that is to wrap the replacement expression
in a PlaceHolderVar, and that's what we have to do in the general
case where subquery pullup has mutated the replacement joinaliasvars
entry into an arbitrary expression. But in simpler cases, such as
where the joinaliasvars entry is just a Var, we'd prefer to do it
by merging the alias Var's varnullingrels into the replacement Var.
In that way the flattened alias will compare equal() to semantically
equivalent references that didn't use the alias name.
Moreover, the parser also uses this code while checking certain
semantic constraints, and in that context we *must not* generate
PlaceHolderVars. PHVs shouldn't appear in parse-time expressions,
and adding one would certainly cause the parser to decide the
query is invalid (because the result wouldn't compare equal() to
what it needs to). Fortunately, during parsing the set of possible
contents of a joinaliasvars entry is quite constrained, so we can
guarantee to apply the nullingrels info to the Vars therein.
The result of this step passes all core regression tests, but there
are still loose ends for FDWs (so that contrib/postgres_fdw will fail).
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 556acd2bd6..f8f33bb77c 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -900,7 +900,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
*/
if (rte->lateral && root->hasJoinRTEs)
rte->subquery = (Query *)
- flatten_join_alias_vars(root->parse,
+ flatten_join_alias_vars(root, root->parse,
(Node *) rte->subquery);
}
else if (rte->rtekind == RTE_FUNCTION)
@@ -1101,7 +1101,7 @@ preprocess_expression(PlannerInfo *root, Node *expr, int kind)
kind == EXPRKIND_VALUES ||
kind == EXPRKIND_TABLESAMPLE ||
kind == EXPRKIND_TABLEFUNC))
- expr = flatten_join_alias_vars(root->parse, expr);
+ expr = flatten_join_alias_vars(root, root->parse, expr);
/*
* Simplify constant expressions. For function RTEs, this was already
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 7c001cdf35..3d829865ea 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -1077,7 +1077,8 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* maybe even in the rewriter; but for now let's just fix this case here.)
*/
subquery->targetList = (List *)
- flatten_join_alias_vars(subroot->parse, (Node *) subquery->targetList);
+ flatten_join_alias_vars(subroot, subroot->parse,
+ (Node *) subquery->targetList);
/*
* Adjust level-0 varnos in subquery so that we can append its rangetable
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
index 8d8c9136f8..69c2019553 100644
--- a/src/backend/optimizer/util/var.c
+++ b/src/backend/optimizer/util/var.c
@@ -62,6 +62,7 @@ typedef struct
typedef struct
{
+ PlannerInfo *root; /* could be NULL! */
Query *query; /* outer Query */
int sublevels_up;
bool possible_sublink; /* could aliases include a SubLink? */
@@ -80,6 +81,10 @@ static bool pull_var_clause_walker(Node *node,
pull_var_clause_context *context);
static Node *flatten_join_alias_vars_mutator(Node *node,
flatten_join_alias_vars_context *context);
+static Node *add_nullingrels_if_needed(PlannerInfo *root, Node *newnode,
+ Var *oldvar);
+static bool is_standard_join_alias_expression(Node *newnode, Var *oldvar);
+static void adjust_standard_join_alias_expression(Node *newnode, Var *oldvar);
static Relids alias_relid_set(Query *query, Relids relids);
@@ -722,26 +727,42 @@ pull_var_clause_walker(Node *node, pull_var_clause_context *context)
* is the only way that the executor can directly handle whole-row Vars.
*
* This also adjusts relid sets found in some expression node types to
- * substitute the contained base rels for any join relid.
+ * substitute the contained base+OJ rels for any join relid.
*
* If a JOIN contains sub-selects that have been flattened, its join alias
* entries might now be arbitrary expressions, not just Vars. This affects
- * this function in one important way: we might find ourselves inserting
- * SubLink expressions into subqueries, and we must make sure that their
- * Query.hasSubLinks fields get set to true if so. If there are any
+ * this function in two important ways. First, we might find ourselves
+ * inserting SubLink expressions into subqueries, and we must make sure that
+ * their Query.hasSubLinks fields get set to true if so. If there are any
* SubLinks in the join alias lists, the outer Query should already have
* hasSubLinks = true, so this is only relevant to un-flattened subqueries.
+ * Second, we have to preserve any varnullingrels info attached to the
+ * alias Vars we're replacing. If the replacement expression is a Var or
+ * PlaceHolderVar or constructed from those, we can just add the
+ * varnullingrels bits to the existing nullingrels field(s); otherwise
+ * we have to add a PlaceHolderVar wrapper.
*
- * NOTE: this is used on not-yet-planned expressions. We do not expect it
- * to be applied directly to the whole Query, so if we see a Query to start
- * with, we do want to increment sublevels_up (this occurs for LATERAL
- * subqueries).
+ * NOTE: this is also used by the parser, to expand join alias Vars before
+ * checking GROUP BY validity. For that use-case, root will be NULL, which
+ * is why we have to pass the Query separately. We need the root itself only
+ * for making PlaceHolderVars. We can avoid making PlaceHolderVars in the
+ * parser's usage because it won't be dealing with arbitrary expressions:
+ * so long as adjust_standard_join_alias_expression can handle everything
+ * the parser would make as a join alias expression, we're OK.
*/
Node *
-flatten_join_alias_vars(Query *query, Node *node)
+flatten_join_alias_vars(PlannerInfo *root, Query *query, Node *node)
{
flatten_join_alias_vars_context context;
+ /*
+ * We do not expect this to be applied to the whole Query, only to
+ * expressions or LATERAL subqueries. Hence, if the top node is a Query,
+ * it's okay to immediately increment sublevels_up.
+ */
+ Assert(node != (Node *) query);
+
+ context.root = root;
context.query = query;
context.sublevels_up = 0;
/* flag whether join aliases could possibly contain SubLinks */
@@ -812,7 +833,9 @@ flatten_join_alias_vars_mutator(Node *node,
rowexpr->colnames = colnames;
rowexpr->location = var->location;
- return (Node *) rowexpr;
+ /* Lastly, add any varnullingrels to the replacement expression */
+ return add_nullingrels_if_needed(context->root, (Node *) rowexpr,
+ var);
}
/* Expand join alias reference */
@@ -839,7 +862,8 @@ flatten_join_alias_vars_mutator(Node *node,
if (context->possible_sublink && !context->inserted_sublink)
context->inserted_sublink = checkExprHasSubLink(newvar);
- return newvar;
+ /* Lastly, add any varnullingrels to the replacement expression */
+ return add_nullingrels_if_needed(context->root, newvar, var);
}
if (IsA(node, PlaceHolderVar))
{
@@ -854,6 +878,7 @@ flatten_join_alias_vars_mutator(Node *node,
{
phv->phrels = alias_relid_set(context->query,
phv->phrels);
+ /* we *don't* change phnullingrels */
}
return (Node *) phv;
}
@@ -887,9 +912,145 @@ flatten_join_alias_vars_mutator(Node *node,
(void *) context);
}
+/*
+ * Add oldvar's varnullingrels, if any, to a flattened join alias expression.
+ * The newnode has been copied, so we can modify it freely.
+ */
+static Node *
+add_nullingrels_if_needed(PlannerInfo *root, Node *newnode, Var *oldvar)
+{
+ if (oldvar->varnullingrels == NULL)
+ return newnode; /* nothing to do */
+ /* If possible, do it by adding to existing nullingrel fields */
+ if (is_standard_join_alias_expression(newnode, oldvar))
+ adjust_standard_join_alias_expression(newnode, oldvar);
+ else if (root)
+ {
+ /* We can insert a PlaceHolderVar to carry the nullingrels */
+ PlaceHolderVar *newphv;
+ Relids phrels = pull_varnos(root, newnode);
+
+ /* XXX what if phrels is empty? */
+ Assert(!bms_is_empty(phrels)); /* probably wrong */
+ newphv = make_placeholder_expr(root, (Expr *) newnode, phrels);
+ /* newphv has zero phlevelsup and NULL phnullingrels; fix it */
+ newphv->phlevelsup = oldvar->varlevelsup;
+ newphv->phnullingrels = bms_copy(oldvar->varnullingrels);
+ newnode = (Node *) newphv;
+ }
+ else
+ {
+ /* ooops, we're missing support for something the parser can make */
+ elog(ERROR, "unsupported join alias expression");
+ }
+ return newnode;
+}
+
+/*
+ * Check to see if we can insert nullingrels into this join alias expression
+ * without use of a separate PlaceHolderVar.
+ *
+ * This will handle Vars, PlaceHolderVars, and implicit-coercion and COALESCE
+ * expressions built from those. This coverage needs to handle anything
+ * that the parser would put into joinaliasvars.
+ * XXX it's probably incomplete at the moment.
+ */
+static bool
+is_standard_join_alias_expression(Node *newnode, Var *oldvar)
+{
+ if (newnode == NULL)
+ return false;
+ if (IsA(newnode, Var) &&
+ ((Var *) newnode)->varlevelsup == oldvar->varlevelsup)
+ return true;
+ else if (IsA(newnode, PlaceHolderVar) &&
+ ((PlaceHolderVar *) newnode)->phlevelsup == oldvar->varlevelsup)
+ return true;
+ else if (IsA(newnode, FuncExpr))
+ {
+ FuncExpr *fexpr = (FuncExpr *) newnode;
+
+ /*
+ * We need to assume that the function wouldn't produce non-NULL from
+ * NULL, which is reasonable for implicit coercions but otherwise not
+ * so much. (Looking at its strictness is likely overkill, and anyway
+ * it would cause us to fail if someone forgot to mark an implicit
+ * coercion as strict.)
+ */
+ if (fexpr->funcformat != COERCE_IMPLICIT_CAST ||
+ fexpr->args == NIL)
+ return false;
+
+ /*
+ * Examine only the first argument --- coercions might have additional
+ * arguments that are constants.
+ */
+ return is_standard_join_alias_expression(linitial(fexpr->args), oldvar);
+ }
+ else if (IsA(newnode, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) newnode;
+ ListCell *lc;
+
+ Assert(cexpr->args != NIL);
+ foreach(lc, cexpr->args)
+ {
+ if (!is_standard_join_alias_expression(lfirst(lc), oldvar))
+ return false;
+ }
+ return true;
+ }
+ else
+ return false;
+}
+
+/*
+ * Insert nullingrels into an expression accepted by
+ * is_standard_join_alias_expression.
+ */
+static void
+adjust_standard_join_alias_expression(Node *newnode, Var *oldvar)
+{
+ if (IsA(newnode, Var) &&
+ ((Var *) newnode)->varlevelsup == oldvar->varlevelsup)
+ {
+ Var *newvar = (Var *) newnode;
+
+ newvar->varnullingrels = bms_add_members(newvar->varnullingrels,
+ oldvar->varnullingrels);
+ }
+ else if (IsA(newnode, PlaceHolderVar) &&
+ ((PlaceHolderVar *) newnode)->phlevelsup == oldvar->varlevelsup)
+ {
+ PlaceHolderVar *newphv = (PlaceHolderVar *) newnode;
+
+ newphv->phnullingrels = bms_add_members(newphv->phnullingrels,
+ oldvar->varnullingrels);
+ }
+ else if (IsA(newnode, FuncExpr))
+ {
+ FuncExpr *fexpr = (FuncExpr *) newnode;
+
+ adjust_standard_join_alias_expression(linitial(fexpr->args), oldvar);
+ }
+ else if (IsA(newnode, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) newnode;
+ ListCell *lc;
+
+ Assert(cexpr->args != NIL);
+ foreach(lc, cexpr->args)
+ {
+ adjust_standard_join_alias_expression(lfirst(lc), oldvar);
+ }
+ }
+ else
+ Assert(false);
+}
+
/*
* alias_relid_set: in a set of RT indexes, replace joins by their
- * underlying base relids
+ * underlying base+OJ relids
*/
static Relids
alias_relid_set(Query *query, Relids relids)
diff --git a/src/backend/parser/parse_agg.c b/src/backend/parser/parse_agg.c
index 3ef9e8ee5e..c15fab0f68 100644
--- a/src/backend/parser/parse_agg.c
+++ b/src/backend/parser/parse_agg.c
@@ -1162,7 +1162,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
* entries are RTE_JOIN kind.
*/
if (hasJoinRTEs)
- groupClauses = (List *) flatten_join_alias_vars(qry,
+ groupClauses = (List *) flatten_join_alias_vars(NULL, qry,
(Node *) groupClauses);
/*
@@ -1206,7 +1206,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
groupClauses, hasJoinRTEs,
have_non_var_grouping);
if (hasJoinRTEs)
- clause = flatten_join_alias_vars(qry, clause);
+ clause = flatten_join_alias_vars(NULL, qry, clause);
check_ungrouped_columns(clause, pstate, qry,
groupClauses, groupClauseCommonVars,
have_non_var_grouping,
@@ -1217,7 +1217,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
groupClauses, hasJoinRTEs,
have_non_var_grouping);
if (hasJoinRTEs)
- clause = flatten_join_alias_vars(qry, clause);
+ clause = flatten_join_alias_vars(NULL, qry, clause);
check_ungrouped_columns(clause, pstate, qry,
groupClauses, groupClauseCommonVars,
have_non_var_grouping,
@@ -1546,7 +1546,7 @@ finalize_grouping_exprs_walker(Node *node,
Index ref = 0;
if (context->hasJoinRTEs)
- expr = flatten_join_alias_vars(context->qry, expr);
+ expr = flatten_join_alias_vars(NULL, context->qry, expr);
/*
* Each expression must match a grouping entry at the current
diff --git a/src/include/optimizer/optimizer.h b/src/include/optimizer/optimizer.h
index 409005bae9..95f3461a3d 100644
--- a/src/include/optimizer/optimizer.h
+++ b/src/include/optimizer/optimizer.h
@@ -197,6 +197,6 @@ extern bool contain_var_clause(Node *node);
extern bool contain_vars_of_level(Node *node, int levelsup);
extern int locate_var_of_level(Node *node, int levelsup);
extern List *pull_var_clause(Node *node, int flags);
-extern Node *flatten_join_alias_vars(Query *query, Node *node);
+extern Node *flatten_join_alias_vars(PlannerInfo *root, Query *query, Node *node);
#endif /* OPTIMIZER_H */
commit e0f35326c479ab14809bd4e0da9ef7e53f840d3a
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu Aug 18 14:32:53 2022 -0400
Teach FDWs about base-plus-outer-join relids.
Conversion of the planner to include OJ relids in join relids
affects FDWs that want to plan foreign joins. They *must* follow
suit when labeling foreign joins in order to match with the core
planner, but for many purposes (if postgres_fdw is any guide)
they'd prefer to consider only base relations within the join.
To support both requirements, redefine ForeignScan.fs_relids as
base+OJ relids, and add a new field fs_base_relids that's set up
by the core planner.
Another way we could do this is to keep fs_relids as just base
relids and make the new field be the one with OJ relids added.
While that would be more backwards-compatible in some sense,
it would be inconsistent with the naming used in the core planner,
and I think that it might allow some types of bugs to escape
immediate detection.
postgres_fdw also has one place where it needs to ignore varnullingrels
while matching Vars, similarly to the unfinished work in setrefs.c.
(That requirement will only affect join-planning FDWs, too, since
Vars seen at a base relation scan should never have any varnullingrels.)
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
index a9766f9734..a187cd08fa 100644
--- a/contrib/postgres_fdw/deparse.c
+++ b/contrib/postgres_fdw/deparse.c
@@ -3952,7 +3952,17 @@ get_relation_column_alias_ids(Var *node, RelOptInfo *foreignrel,
i = 1;
foreach(lc, foreignrel->reltarget->exprs)
{
- if (equal(lfirst(lc), (Node *) node))
+ Var *tlvar = (Var *) lfirst(lc);
+
+ /*
+ * As in setrefs.c, we match only on varno/varattno. Ideally there
+ * would be some cross-check on varnullingrels, but it's unclear what
+ * to do exactly; we don't have enough context to know what that value
+ * should be.
+ */
+ if (IsA(tlvar, Var) &&
+ tlvar->varno == node->varno &&
+ tlvar->varattno == node->varattno)
{
*colno = i;
return;
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 16320170ce..85712cdce6 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -1513,13 +1513,13 @@ postgresBeginForeignScan(ForeignScanState *node, int eflags)
/*
* Identify which user to do the remote access as. This should match what
* ExecCheckRTEPerms() does. In case of a join or aggregate, use the
- * lowest-numbered member RTE as a representative; we would get the same
- * result from any.
+ * lowest-numbered member base RTE as a representative; we would get the
+ * same result from any.
*/
if (fsplan->scan.scanrelid > 0)
rtindex = fsplan->scan.scanrelid;
else
- rtindex = bms_next_member(fsplan->fs_relids, -1);
+ rtindex = bms_next_member(fsplan->fs_base_relids, -1);
rte = exec_rt_fetch(rtindex, estate);
userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
@@ -2407,7 +2407,7 @@ find_modifytable_subplan(PlannerInfo *root,
{
ForeignScan *fscan = (ForeignScan *) subplan;
- if (bms_is_member(rtindex, fscan->fs_relids))
+ if (bms_is_member(rtindex, fscan->fs_base_relids))
return fscan;
}
@@ -2833,8 +2833,8 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
* that setrefs.c won't update the string when flattening the
* rangetable. To find out what rtoffset was applied, identify the
* minimum RT index appearing in the string and compare it to the
- * minimum member of plan->fs_relids. (We expect all the relids in
- * the join will have been offset by the same amount; the Asserts
+ * minimum member of plan->fs_base_relids. (We expect all the relids
+ * in the join will have been offset by the same amount; the Asserts
* below should catch it if that ever changes.)
*/
minrti = INT_MAX;
@@ -2851,7 +2851,7 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
else
ptr++;
}
- rtoffset = bms_next_member(plan->fs_relids, -1) - minrti;
+ rtoffset = bms_next_member(plan->fs_base_relids, -1) - minrti;
/* Now we can translate the string */
relations = makeStringInfo();
@@ -2866,7 +2866,7 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
char *refname;
rti += rtoffset;
- Assert(bms_is_member(rti, plan->fs_relids));
+ Assert(bms_is_member(rti, plan->fs_base_relids));
rte = rt_fetch(rti, es->rtable);
Assert(rte->rtekind == RTE_RELATION);
/* This logic should agree with explain.c's ExplainTargetRel */
diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml
index d0b5951019..329affa30b 100644
--- a/doc/src/sgml/fdwhandler.sgml
+++ b/doc/src/sgml/fdwhandler.sgml
@@ -351,6 +351,17 @@ GetForeignJoinPaths(PlannerInfo *root,
it will supply at run time in the tuples it returns.
</para>
+ <note>
+ <para>
+ Beginning with <productname>PostgreSQL</productname> 16,
+ <structfield>fs_relids</structfield> includes the rangetable indexes
+ of outer joins, if any were involved in this join. The new field
+ <structfield>fs_base_relids</structfield> includes only base
+ relation indexes, and thus
+ mimics <structfield>fs_relids</structfield>'s old semantics.
+ </para>
+ </note>
+
<para>
See <xref linkend="fdw-planning"/> for additional information.
</para>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index e078456b19..36dcfc1fab 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1114,7 +1114,7 @@ ExplainPreScanNode(PlanState *planstate, Bitmapset **rels_used)
break;
case T_ForeignScan:
*rels_used = bms_add_members(*rels_used,
- ((ForeignScan *) plan)->fs_relids);
+ ((ForeignScan *) plan)->fs_base_relids);
break;
case T_CustomScan:
*rels_used = bms_add_members(*rels_used,
diff --git a/src/backend/executor/execScan.c b/src/backend/executor/execScan.c
index 043bb83f55..2b37266b6a 100644
--- a/src/backend/executor/execScan.c
+++ b/src/backend/executor/execScan.c
@@ -325,7 +325,7 @@ ExecScanReScan(ScanState *node)
* all of them.
*/
if (IsA(node->ps.plan, ForeignScan))
- relids = ((ForeignScan *) node->ps.plan)->fs_relids;
+ relids = ((ForeignScan *) node->ps.plan)->fs_base_relids;
else if (IsA(node->ps.plan, CustomScan))
relids = ((CustomScan *) node->ps.plan)->custom_relids;
else
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index cd8a3ef7cb..880cbfeb5e 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -29,6 +29,7 @@
#include "optimizer/cost.h"
#include "optimizer/optimizer.h"
#include "optimizer/paramassign.h"
+#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/placeholder.h"
#include "optimizer/plancat.h"
@@ -4105,6 +4106,8 @@ create_foreignscan_plan(PlannerInfo *root, ForeignPath *best_path,
Index scan_relid = rel->relid;
Oid rel_oid = InvalidOid;
Plan *outer_plan = NULL;
+ Relids fs_base_relids;
+ int rtindex;
Assert(rel->fdwroutine != NULL);
@@ -4153,14 +4156,28 @@ create_foreignscan_plan(PlannerInfo *root, ForeignPath *best_path,
/*
* Likewise, copy the relids that are represented by this foreign scan. An
- * upper rel doesn't have relids set, but it covers all the base relations
- * participating in the underlying scan, so use root's all_baserels.
+ * upper rel doesn't have relids set, but it covers all the relations
+ * participating in the underlying scan/join, so use root->all_query_rels.
*/
if (rel->reloptkind == RELOPT_UPPER_REL)
- scan_plan->fs_relids = root->all_baserels;
+ scan_plan->fs_relids = root->all_query_rels;
else
scan_plan->fs_relids = best_path->path.parent->relids;
+ /*
+ * Join relid sets include relevant outer joins, but FDWs may need to know
+ * which are the included base rels. That's a bit tedious to get without
+ * access to the plan-time data structures, so compute it here.
+ */
+ fs_base_relids = NULL;
+ rtindex = -1;
+ while ((rtindex = bms_next_member(scan_plan->fs_relids, rtindex)) >= 0)
+ {
+ if (find_base_rel_ignore_join(root, rtindex) != NULL)
+ fs_base_relids = bms_add_member(fs_base_relids, rtindex);
+ }
+ scan_plan->fs_base_relids = fs_base_relids;
+
/*
* If this is a foreign join, and to make it valid to push down we had to
* assume that the current user is the same as some user explicitly named
@@ -5800,8 +5817,9 @@ make_foreignscan(List *qptlist,
node->fdw_private = fdw_private;
node->fdw_scan_tlist = fdw_scan_tlist;
node->fdw_recheck_quals = fdw_recheck_quals;
- /* fs_relids will be filled in by create_foreignscan_plan */
+ /* fs_relids, fs_base_relids will be filled by create_foreignscan_plan */
node->fs_relids = NULL;
+ node->fs_base_relids = NULL;
/* fsSystemCol will be filled in by create_foreignscan_plan */
node->fsSystemCol = false;
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index a1827d113d..1a047ddda2 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1533,6 +1533,7 @@ set_foreignscan_references(PlannerInfo *root,
}
fscan->fs_relids = offset_relid_set(fscan->fs_relids, rtoffset);
+ fscan->fs_base_relids = offset_relid_set(fscan->fs_base_relids, rtoffset);
/* Adjust resultRelation if it's valid */
if (fscan->resultRelation > 0)
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index dca2a21e7a..7e98d0b7a3 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -688,6 +688,7 @@ typedef struct WorkTableScan
* When the plan node represents a foreign join, scan.scanrelid is zero and
* fs_relids must be consulted to identify the join relation. (fs_relids
* is valid for simple scans as well, but will always match scan.scanrelid.)
+ * fs_relids includes outer joins; fs_base_relids does not.
*
* If the FDW's PlanDirectModify() callback decides to repurpose a ForeignScan
* node to perform the UPDATE or DELETE operation directly in the remote
@@ -707,7 +708,8 @@ typedef struct ForeignScan
List *fdw_private; /* private data for FDW */
List *fdw_scan_tlist; /* optional tlist describing scan tuple */
List *fdw_recheck_quals; /* original quals not in scan.plan.qual */
- Bitmapset *fs_relids; /* RTIs generated by this scan */
+ Bitmapset *fs_relids; /* base+OJ RTIs generated by this scan */
+ Bitmapset *fs_base_relids; /* base RTIs generated by this scan */
bool fsSystemCol; /* true if any "system column" is needed */
} ForeignScan;
Progress report on this ...
I've been trying to remove some of the cruftier aspects of
EquivalenceClasses (which really is the main point of this whole
effort), and gotten repeatedly blocked by the fact that the semantics
are still a bit crufty thanks to the hacking associated with outer
join identity 3. I think I see a path forward though.
To recap, the thing about identity 3 is that it says you can get
equivalent results from
(A leftjoin B on (Pab)) leftjoin C on (Pb*c)
A leftjoin (B leftjoin C on (Pbc)) on (Pab)
if Pbc is strict for B. Unlike what it says in optimizer/README,
I've written the first form as "Pb*c" to indicate that any B Vars
appearing in that clause will be marked as possibly nulled by
the A/B join. This makes the problem apparent: we cannot use
the same representation of Pbc for both join orders, because
in the second variant B's Vars are not nulled by anything.
We've been trying to get away with writing Pbc just one way,
and that leads directly to the semantic squishiness I've been
fighting.
What I'm thinking we should do about this, once we detect that
this identity is applicable, is to generate *both* forms of Pbc,
either adding or removing the varnullingrels bits depending on
which form we got from the parser. Then, when we come to forming
join paths, use the appropriate variant depending on which join
order we're considering. build_join_rel() already has the concept
that the join restriction list varies depending on which input
relations we're trying to join, so this doesn't require any
fundamental restructuring -- only a way to identify which
RestrictInfos to use or ignore for a particular join. That will
probably require some new field in RestrictInfo, but I'm not
fussed about that because there are other fields we'll be able
to remove as this work progresses.
Similarly, generate_join_implied_equalities() will need to generate
EquivalenceClass-derived join clauses with or without varnullingrels
marks, as appropriate. I'm not quite sure how to do that, but it
feels like just a small matter of programming, not a fundamental
problem with the model which is where things are right now.
We'll only need this for ECs that include source clauses coming
from a movable outer join clause (i.e., Pbc in identity 3).
An interesting point is that I think we want to force movable
outer joins into the second format for the purpose of generating
ECs, that is we want to use Pbc not Pb*c as the EC source form.
The reason for that is to allow generation of relation-scan-level
clauses from an EC, particularly an EC that includes a constant.
As an example, given
A leftjoin (B leftjoin C on (B.b = C.c)) on (A.a = B.b)
where A.a = constant
we can decide unconditionally that A.a, B.b, C.c, and the constant all
belong to the same equivalence class, and thereby generate relation
scan restrictions A.a = constant, B.b = constant, and C.c = constant.
If we start with the other join order, which will include "B.b* = C.c"
(ie Pb*c) then we'd have two separate ECs: {A.a, B.b, constant} and
{B.b*, C.c}. So we'll fail to produce any scan restriction for C, or
at least we can't do so in any principled way.
Furthermore, if the joins are done in the second order then we don't
need any additional join clauses -- both joins can act like "LEFT JOIN
ON TRUE". (Right now, we'll emit redundant B.b = C.c and A.a = B.b
join clauses in addition to the scan-level clauses, which is
inefficient.) However, if we make use of identity 3 to do the
joins in the other order, then we do need an extra join clause, like
(A leftjoin B on (true)) leftjoin C on (B.b* = C.c)
(or maybe we could just emit "B.b* IS NOT NULL" for Pb*c?)
Without any Pb*c join constraint we get wrong answers because
nulling of B fails to propagate to C.
So while there are lots of details to work out, it feels like
this line of thought can lead to something where setrefs.c
doesn't have to ignore varnullingrels mismatches (yay) and
there is no squishiness in EquivalenceClass semantics.
regards, tom lane
On Sun, Aug 21, 2022 at 6:52 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
What I'm thinking we should do about this, once we detect that
this identity is applicable, is to generate *both* forms of Pbc,
either adding or removing the varnullingrels bits depending on
which form we got from the parser. Then, when we come to forming
join paths, use the appropriate variant depending on which join
order we're considering. build_join_rel() already has the concept
that the join restriction list varies depending on which input
relations we're trying to join, so this doesn't require any
fundamental restructuring -- only a way to identify which
RestrictInfos to use or ignore for a particular join. That will
probably require some new field in RestrictInfo, but I'm not
fussed about that because there are other fields we'll be able
to remove as this work progresses.
Do you mean we generate two RestrictInfos for Pbc in the case of
identity 3, one with varnullingrels and one without varnullingrels, and
choose the appropriate one when forming join paths? Do we need to also
generate two SpecialJoinInfos for the B/C join in the first order, with
and without the A/B join in its min_lefthand?
Thanks
Richard
identity 3, one with varnullingrels and one without varnullingrels, and
choose the appropriate one when forming join paths? Do we need to also
generate two SpecialJoinInfos for the B/C join in the first order, with
and without the A/B join in its min_lefthand?
Thanks
Richard
Richard Guo <guofenglinux@gmail.com> writes:
> On Sun, Aug 21, 2022 at 6:52 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> What I'm thinking we should do about this, once we detect that
>> this identity is applicable, is to generate *both* forms of Pbc,
>> either adding or removing the varnullingrels bits depending on
>> which form we got from the parser.
> Do you mean we generate two RestrictInfos for Pbc in the case of
> identity 3, one with varnullingrels and one without varnullingrels, and
> choose the appropriate one when forming join paths?
Right.
> Do we need to also
> generate two SpecialJoinInfos for the B/C join in the first order, with
> and without the A/B join in its min_lefthand?
No, the SpecialJoinInfos would stay as they are now. It's already the
case that the first join's min_righthand would contain only B, and
the second one's min_righthand would contain only C.
regards, tom lane
On Thu, Aug 25, 2022 at 5:18 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Richard Guo <guofenglinux@gmail.com> writes:
> Do we need to also
> generate two SpecialJoinInfos for the B/C join in the first order, with
> and without the A/B join in its min_lefthand?
No, the SpecialJoinInfos would stay as they are now. It's already the
case that the first join's min_righthand would contain only B, and
the second one's min_righthand would contain only C.
I'm not sure if I understand it correctly. If we are given the first
order from the parser, the SpecialJoinInfo for the B/C join would have
min_lefthand as containing both B and the A/B join. And this
SpecialJoinInfo would make the B/C join be invalid, which is not what we
want. Currently the patch resolves this by explicitly running
remove_unneeded_nulling_relids, and the A/B join would be removed from
B/C join's min_lefthand, if Pbc is strict for B.
Do we still need this kind of fixup if we are to keep just one form of
SpecialJoinInfo and two forms of RestrictInfos?
Thanks
Richard
order from the parser, the SpecialJoinInfo for the B/C join would have
min_lefthand as containing both B and the A/B join. And this
SpecialJoinInfo would make the B/C join be invalid, which is not what we
want. Currently the patch resolves this by explicitly running
remove_unneeded_nulling_relids, and the A/B join would be removed from
B/C join's min_lefthand, if Pbc is strict for B.
Do we still need this kind of fixup if we are to keep just one form of
SpecialJoinInfo and two forms of RestrictInfos?
Thanks
Richard
On Fri, Aug 19, 2022 at 2:45 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Here's a rebase up to HEAD, mainly to get the cfbot back in sync
as to what's the live patch.
Noticed another different behavior from previous. When we try to reduce
JOIN_LEFT to JOIN_ANTI, we want to know if the join's own quals are
strict for any Var that was forced null by higher qual levels. We do
that by checking whether local_nonnullable_vars and forced_null_vars
overlap. However, the same Var from local_nonnullable_vars and
forced_null_vars may be labeled with different varnullingrels. If that
is the case, currently we would fail to tell they actually overlap. As
an example, consider 'b.i' in the query below
# explain (costs off) select * from a left join b on a.i = b.i where b.i is null;
QUERY PLAN
---------------------------
Hash Left Join
Hash Cond: (a.i = b.i)
Filter: (b.i IS NULL)
-> Seq Scan on a
-> Hash
-> Seq Scan on b
(6 rows)
Thanks
Richard
JOIN_LEFT to JOIN_ANTI, we want to know if the join's own quals are
strict for any Var that was forced null by higher qual levels. We do
that by checking whether local_nonnullable_vars and forced_null_vars
overlap. However, the same Var from local_nonnullable_vars and
forced_null_vars may be labeled with different varnullingrels. If that
is the case, currently we would fail to tell they actually overlap. As
an example, consider 'b.i' in the query below
# explain (costs off) select * from a left join b on a.i = b.i where b.i is null;
QUERY PLAN
---------------------------
Hash Left Join
Hash Cond: (a.i = b.i)
Filter: (b.i IS NULL)
-> Seq Scan on a
-> Hash
-> Seq Scan on b
(6 rows)
Thanks
Richard
On Mon, Aug 29, 2022 at 2:30 PM Richard Guo <guofenglinux@gmail.com> wrote:
On Fri, Aug 19, 2022 at 2:45 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:Here's a rebase up to HEAD, mainly to get the cfbot back in sync
as to what's the live patch.Noticed another different behavior from previous. When we try to reduce
JOIN_LEFT to JOIN_ANTI, we want to know if the join's own quals are
strict for any Var that was forced null by higher qual levels. We do
that by checking whether local_nonnullable_vars and forced_null_vars
overlap. However, the same Var from local_nonnullable_vars and
forced_null_vars may be labeled with different varnullingrels. If that
is the case, currently we would fail to tell they actually overlap.
I wonder why this is not noticed by regression tests. So I did some
search and it seems we do not have any test cases covering the
transformation we apply to reduce outer joins. I think maybe we should
add such cases in regression tests.
Thanks
Richard
search and it seems we do not have any test cases covering the
transformation we apply to reduce outer joins. I think maybe we should
add such cases in regression tests.
Thanks
Richard
Richard Guo <guofenglinux@gmail.com> writes:
> I wonder why this is not noticed by regression tests. So I did some
> search and it seems we do not have any test cases covering the
> transformation we apply to reduce outer joins. I think maybe we should
> add such cases in regression tests.
Right, done at 0043aa6b8. The actual fix is in 0010 below (it would
have been earlier, except I'd forgotten about this issue).
I've been working away at this patch series, and here is an up-to-date
version. I've mostly fixed the inability to check in setrefs.c that
varnullingrels match up at different join levels, and I've found a
solution that I feel reasonably happy about for variant join quals
depending on application of outer-join identity 3. There's certainly
bits of this that could be done in other ways, but overall I'm pleased
with the state of these patches.
I think that the next step is to change things so that the "push
a constant through outer-join quals" hacks are replaced by
EquivalenceClass-based logic. That turns out to be harder than
I'd supposed initially, because labeling Vars with nullingrels
fixes only part of the problem there. The other part is that
deductions we make from an outer-join qual can only be applied
below the nullable side of that join --- we can't use them to
remove rows from the non-nullable side. I have an idea how to
make that work, but it's not passing regression tests yet :-(.
Anyway, there's much more to do, but here's what I've got today.
regards, tom lane
commit d7afb222b0e3896103c5b7704bcb738547d58d4d
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sun Oct 30 15:39:58 2022 -0400
Add overview documentation.
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index 41c120e0cd..360d37bcaa 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -295,6 +295,239 @@ Therefore, we don't merge FROM-lists if the result would have too many
FROM-items in one list.
+Vars and PlaceHolderVars
+------------------------
+
+A Var node is simply the parse-tree representation of a table column
+reference. However, in the presence of outer joins, that concept is
+more subtle than it might seem. We need to distinguish the values of
+a Var "above" and "below" any outer join that could force the Var to
+null. As an example, consider
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE foo(t2.z)
+
+(Assume foo() is not strict, so that we can't reduce the left join to
+a plain join.) A naive implementation might try to push the foo(t2.z)
+call down to the scan of t2, but that is not correct because
+(a) what foo() should actually see for a null-extended join row is NULL,
+and (b) if foo() returns false, we should suppress the t1 row from the
+join altogether, not emit it with a null-extended t2 row. On the other
+hand, it *would* be correct (and desirable) to push the call down to
+the scan level if the query were
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y AND foo(t2.z))
+
+This motivates considering "t2.z" within the left join's ON clause
+to be a different value from "t2.z" outside the JOIN clause. The
+former can be identified with t2.z as seen at the relation scan level,
+but the latter can't.
+
+Another example occurs in connection with EquivalenceClasses (discussed
+below). Given
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE t1.x = 42
+
+we would like to use the EquivalenceClass mechanisms to derive "t2.y = 42"
+to use as a restriction clause for the scan of t2. (That works, because t2
+rows having y different from 42 cannot affect the query result.) However,
+it'd be wrong to conclude that t2.y will be equal to t1.x in every joined
+row. Part of the solution to this problem is to deem that "t2.y" in the
+ON clause refers to the relation-scan-level value of t2.y, but not to the
+value that y will have in joined rows, where it might be NULL rather than
+equal to t1.x.
+
+Therefore, Var nodes are decorated with "varnullingrels", which are sets
+of the rangetable indexes of outer joins that potentially null the Var
+at the point where it appears in the query. (Using a set, not an ordered
+list, is fine since it doesn't matter which join forced the value to null;
+and that avoids having to change the representation when we consider
+different outer-join orders.) In the examples above, all occurrences of
+t1.x would have empty varnullingrels, since the left join doesn't null t1.
+The t2 references within the JOIN ON clauses would also have empty
+varnullingrels. But outside the JOIN clauses, any Vars referencing t2
+would have varnullingrels containing the index of the JOIN's rangetable
+entry (RTE), so that they'd be understood as potentially different from
+the t2 values seen at scan level. Labeling t2.z in the WHERE clause with
+the JOIN's RT index lets us recognize that that occurrence of foo(t2.z)
+cannot be pushed down to the t2 scan level: we cannot evaluate that value
+at the scan level, but only after the join has been done.
+
+For LEFT and RIGHT outer joins, only Vars coming from the nullable side
+of the join are marked with that join's RT index. For FULL joins, Vars
+from both inputs are marked. (Such marking doesn't let us tell which
+side of the full join a Var came from; but that information can be found
+elsewhere at need.)
+
+Notionally, a Var having nonempty varnullingrels can be thought of as
+ CASE WHEN any-of-these-outer-joins-produced-a-null-extended-row
+ THEN NULL
+ ELSE the-scan-level-value-of-the-column
+ END
+It's only notional, because no such calculation is ever done explicitly.
+In a finished plan, Vars occurring in scan-level plan nodes represent
+the actual table column values, but upper-level Vars are always
+references to outputs of lower-level plan nodes. When a join node emits
+a null-extended row, it just returns nulls for the relevant output
+columns rather than copying up values from its input. Because we don't
+ever have to do this calculation explicitly, it's not necessary to
+distinguish which side of an outer join got null-extended, which'd
+otherwise be essential information for FULL JOIN cases.
+
+Outer join identity 3 (discussed above) complicates this picture
+a bit. In the form
+ A leftjoin (B leftjoin C on (Pbc)) on (Pab)
+all of the Vars in clauses Pbc and Pab will have empty varnullingrels,
+but if we start with
+ (A leftjoin B on (Pab)) leftjoin C on (Pbc)
+then the parser will have marked Pbc's B Vars with the A/B join's
+RT index, making this form artificially different from the first.
+For discussion's sake, let's denote this marking with a star:
+ (A leftjoin B on (Pab)) leftjoin C on (Pb*c)
+To cope with this, once we have detected that commuting these joins
+is legal, we generate both the Pbc and Pb*c forms of that ON clause,
+by either removing or adding the first join's RT index in the B Vars
+that the parser created. While generating paths for a plan step that
+joins B and C, we include as a relevant join qual only the form that
+is appropriate depending on whether A has already been joined to B.
+
+It's also worth noting that identity 3 makes "the left join's RT index"
+itself a bit of a fuzzy concept, since the syntactic scope of each join
+RTE will depend on which form was produced by the parser. We resolve
+this by considering that a left join's identity is determined by its
+minimum set of right-hand-side input relations. In both forms allowed
+by identity 3, we can identify the first join as having minimum RHS B
+and the second join as having minimum RHS C.
+
+Another thing to notice is that C Vars appearing outside the nested
+JOIN clauses will be marked as nulled by both left joins if the
+original parser input was in the first form of identity 3, but if the
+parser input was in the second form, such Vars will only be marked as
+nulled by the second join. This is not really a semantic problem:
+such Vars will be marked the same way throughout the upper part of the
+query, so they will all look equal() which is correct; and they will not
+look equal() to any C Var appearing in the JOIN ON clause or below these
+joins. However, when building Vars representing the outputs of join
+relations, we need to ensure that their varnullingrels are set to
+values consistent with the syntactic join order, so that they will
+appear equal() to pre-existing Vars in the upper part of the query.
+
+Outer joins also complicate handling of subquery pull-up. Consider
+
+ SELECT ..., ss.x FROM tab1
+ LEFT JOIN (SELECT *, 42 AS x FROM tab2) ss ON ...
+
+We want to be able to pull up the subquery as discussed previously,
+but we can't just replace the "ss.x" Var in the top-level SELECT list
+with the constant 42. That'd result in always emitting 42, rather
+than emitting NULL in null-extended join rows.
+
+To solve this, we introduce the concept of PlaceHolderVars.
+A PlaceHolderVar is somewhat like a Var, in that its value originates
+at a relation scan level and can then be forced to null by higher-level
+outer joins; hence PlaceHolderVars carry a set of nulling rel IDs just
+like Vars. Unlike a Var, whose original value comes from a table,
+a PlaceHolderVar's original value is defined by a query-determined
+expression ("42" in this example); so we represent the PlaceHolderVar
+as a node with that expression as child. We insert a PlaceHolderVar
+whenever subquery pullup needs to replace a subquery-referencing Var
+that has nonempty varnullingrels with an expression that is not simply a
+Var. (When the replacement expression is a pulled-up Var, we can just
+add the replaced Var's varnullingrels to its set. Also, if the replaced
+Var has empty varnullingrels, we don't need a PlaceHolderVar: there is
+nothing that'd force the value to null, so the pulled-up expression is
+fine to use as-is.) In a finished plan, a PlaceHolderVar becomes just
+the contained expression at whatever plan level it's supposed to be
+evaluated at, and then upper-level occurrences are replaced by
+references to that output column of the lower plan level. That causes
+the value to go to null when appropriate at an outer join, in the same
+way as for Vars. Thus, PlaceHolderVars are never seen outside the
+planner.
+
+PlaceHolderVars (PHVs) are more complicated than Vars in another way:
+their original value might need to be calculated at a join, not a
+base-level relation scan. This can happen if a pulled-up subquery
+contains a join. Because of this, a PHV can create a join order
+constraint that wouldn't otherwise exist, to ensure that it can
+be calculated before it is used. A PHV's expression can also contain
+LATERAL references, adding complications that are discussed below.
+
+
+Relation Identification and Qual Clause Placement
+-------------------------------------------------
+
+A qual clause obtained from WHERE or JOIN/ON can be enforced at the lowest
+scan or join level that includes all relations used in the clause. For
+this purpose we consider that outer joins listed in varnullingrels or
+phnullingrels are used in the clause, since we can't compute the qual's
+result correctly until we know whether such Vars have gone to null.
+
+The one exception to this general rule is that a non-degenerate outer
+JOIN/ON qual (one that references the non-nullable side of the join)
+cannot be enforced below that join, even if it doesn't reference the
+nullable side. Pushing it down into the non-nullable side would result
+in rows disappearing from the join's result, rather than appearing as
+null-extended rows. To handle that, when we identify such a qual we
+artificially add the join's minimum input relid set to the set of
+relations it is considered to use, forcing it to be evaluated exactly at
+that join level. The same happens for outer-join quals that mention no
+relations at all.
+
+When attaching a qual clause to a join plan node that is performing an
+outer join, the qual clause is considered a "join clause" (that is, it is
+applied before the join performs null-extension) if it does not reference
+that outer join in any varnullingrels or phnullingrels set, or a "filter
+clause" (applied after null-extension) if it does reference that outer
+join. A qual clause that originally appeared in that outer join's JOIN/ON
+will fall into the first category, since the parser would not have marked
+any of its Vars as referencing the outer join. A qual clause that
+originally came from some upper ON clause or WHERE clause will be seen as
+referencing the outer join if it references any of the nullable side's
+Vars, since those Vars will be so marked by the parser. But, if such a
+qual does not reference any nullable-side Vars, it's okay to push it down
+into the non-nullable side, so it won't get attached to the join node in
+the first place.
+
+These things lead us to identify join relations within the planner
+by the sets of base relation RT indexes plus outer join RT indexes
+that they include. In that way, the sets of relations used by qual
+clauses can be directly compared to join relations' relid sets to
+see where to place the clauses. These identifying sets are unique
+because, for any given collection of base relations, there is only
+one valid set of outer joins to have performed along the way to
+joining that set of base relations (although the order of applying
+them could vary, as discussed above).
+
+SEMI joins do not have RT indexes, because they are artifacts made by
+the planner rather than the parser. (We could create rangetable
+entries for them, but there seems no need at present.) This does not
+cause a problem for qual placement, because the nullable side of a
+semijoin is not referenceable from above the join, so there is never a
+need to cite it in varnullingrels or phnullingrels. It does not cause a
+problem for join relation identification either, since whether a semijoin
+has been completed is again implicit in the set of base relations
+included in the join.
+
+There is one additional complication for qual clause placement, which
+occurs when we have made multiple versions of an outer-join clause as
+described previously (that is, we have both "Pbc" and "Pb*c" forms of
+the same clause seen in outer join identity 3). When forming an outer
+join we only want to apply one of the redundant versions of the clause.
+If we are forming the B/C join without having yet computed the A/B
+join, it's easy to reject the "Pb*c" form since its required relid
+set includes the A/B join relid which is not in the input. However,
+if we form B/C after A/B, then both forms of the clause are applicable
+so far as that test can tell. We have to look more closely to notice
+that the "Pbc" clause form refers to relation B which is no longer
+directly accessible. While this check is straightforward, it's not
+especially cheap (see clause_is_computable_at()). To avoid doing it
+unnecessarily, we mark the variant versions of a redundant clause as
+either "has_clone" or "is_clone". A production build of Postgres
+checks restriction_is_computable_at() to disentangle which clone copy
+to apply at a given join level. In debug builds, we also Assert that
+non-clone clauses are validly computable, but that seems too expensive
+for production usage.
+
+
Optimizer Functions
-------------------
@@ -437,11 +670,10 @@ inputs.
EquivalenceClasses
------------------
-During the deconstruct_jointree() scan of the query's qual clauses, we look
-for mergejoinable equality clauses A = B whose applicability is not delayed
-by an outer join; these are called "equivalence clauses". When we find
-one, we create an EquivalenceClass containing the expressions A and B to
-record this knowledge. If we later find another equivalence clause B = C,
+During the deconstruct_jointree() scan of the query's qual clauses, we
+look for mergejoinable equality clauses A = B. When we find one, we
+create an EquivalenceClass containing the expressions A and B to record
+that they are equal. If we later find another equivalence clause B = C,
we add C to the existing EquivalenceClass for {A B}; this may require
merging two existing EquivalenceClasses. At the end of the scan, we have
sets of values that are known all transitively equal to each other. We can
@@ -473,15 +705,54 @@ asserts that at any plan node where more than one of its member values
can be computed, output rows in which the values are not all equal may
be discarded without affecting the query result. (We require all levels
of the plan to enforce EquivalenceClasses, hence a join need not recheck
-equality of values that were computable by one of its children.) For an
-ordinary EquivalenceClass that is "valid everywhere", we can further infer
-that the values are all non-null, because all mergejoinable operators are
-strict. However, we also allow equivalence clauses that appear below the
-nullable side of an outer join to form EquivalenceClasses; for these
-classes, the interpretation is that either all the values are equal, or
-all (except pseudo-constants) have gone to null. (This requires a
-limitation that non-constant members be strict, else they might not go
-to null when the other members do.) Consider for example
+equality of values that were computable by one of its children.)
+
+It's tempting to include equality clauses appearing in outer-join
+conditions as sources of EquivalenceClasses, but there's a serious
+difficulty: the resulting deductions are not valid everywhere.
+For example, given
+
+ SELECT * FROM a LEFT JOIN b ON a.x = b.y WHERE a.x = 42;
+
+we could safely derive b.y = 42 and use that in the scan of B,
+because B rows not having b.y = 42 will not contribute to the
+join result. Likewise, given
+
+ SELECT * FROM a LEFT JOIN b ON a.x = b.y AND a.x = b.z;
+
+it's all right to apply b.y = b.z while scanning B, and then only
+one of the two equality conditions need be tested at the join.
+However, if we have
+
+ SELECT * FROM a LEFT JOIN b ON a.x1 = b.y AND a.x2 = b.y;
+
+it'd be completely incorrect to push "a.x1 = a.x2" down to the scan
+of A. Rows where they are different should not be eliminated from
+the join result, but instead produce null-extended join rows.
+
+In general, therefore, we can treat outer-join equalities somewhat like
+real equivalences, but we can only produce derived clauses at that
+outer join and at scans and joins contained within its nullable side.
+(FULL JOIN conditions can't be optimized at all this way, since derived
+clauses couldn't be enforced on either side.)
+
+Another instructive example is:
+
+ SELECT *
+ FROM a LEFT JOIN
+ (SELECT * FROM b JOIN c ON b.y = c.z WHERE b.y = 10) ss
+ ON a.x = ss.y
+ ORDER BY ss.y;
+
+We can form the EquivalenceClass {b.y c.z 10} and thereby apply c.z = 10
+while scanning c. However, this does not tell us anything about the
+ss.y reference appearing in ORDER BY (which is another name for b.y*,
+that is the possibly-nulled form of b.y), so we don't get to conclude
+that sorting for the ORDER BY is unnecessary, as it would be if we could
+prove that b.y* is equal to a constant (see discussion of PathKeys
+below).
+
+Also consider this variant:
SELECT *
FROM a LEFT JOIN
@@ -489,40 +760,60 @@ to null when the other members do.) Consider for example
ON a.x = ss.y
WHERE a.x = 42;
-We can form the below-outer-join EquivalenceClass {b.y c.z 10} and thereby
-apply c.z = 10 while scanning c. (The reason we disallow outerjoin-delayed
-clauses from forming EquivalenceClasses is exactly that we want to be able
-to push any derived clauses as far down as possible.) But once above the
-outer join it's no longer necessarily the case that b.y = 10, and thus we
-cannot use such EquivalenceClasses to conclude that sorting is unnecessary
-(see discussion of PathKeys below).
-
-In this example, notice also that a.x = ss.y (really a.x = b.y) is not an
-equivalence clause because its applicability to b is delayed by the outer
-join; thus we do not try to insert b.y into the equivalence class {a.x 42}.
-But since we see that a.x has been equated to 42 above the outer join, we
-are able to form a below-outer-join class {b.y 42}; this restriction can be
-added because no b/c row not having b.y = 42 can contribute to the result
-of the outer join, and so we need not compute such rows. Now this class
-will get merged with {b.y c.z 10}, leading to the contradiction 10 = 42,
+Here, we have an EquivalenceClass {a.x 42} in addition to {b.y c.z 10},
+and we have an outer-join condition a.x = b.y (not b.y*). That lets us
+derive b.y = 42, but we can only constrain scans/joins below the left join
+that way. Nonetheless, we can still produce the contradiction 10 = 42,
which lets the planner deduce that the b/c join need not be computed at all
because none of its rows can contribute to the outer join. (This gets
implemented as a gating Result filter, since more usually the potential
contradiction involves Param values rather than just Consts, and thus has
to be checked at runtime.)
+To handle outer-join conditions this way, we put their left and right
+operands into EquivalenceClasses in the usual way. (This may result in
+creating single-item equivalence "classes", though of course these are
+still subject to merging if other equivalence clauses are found that
+mention the same Vars.) We do not merge those two EquivalenceClasses
+as would happen with an ordinary equivalence condition. Instead, the
+outer-join condition is recorded in a separate "ConstrainedEquivalence"
+data structure, showing the EquivalenceClasses it connects and the scope
+of the outer join that it is valid within. We can make deductions as
+if the two classes were one, but only when considering a scan or join
+within the scope of the constrained equivalence.
+
To aid in determining the sort ordering(s) that can work with a mergejoin,
we mark each mergejoinable clause with the EquivalenceClasses of its left
-and right inputs. For an equivalence clause, these are of course the same
-EquivalenceClass. For a non-equivalence mergejoinable clause (such as an
-outer-join qualification), we generate two separate EquivalenceClasses for
-the left and right inputs. This may result in creating single-item
-equivalence "classes", though of course these are still subject to merging
-if other equivalence clauses are later found to bear on the same
-expressions.
+and right inputs. For an ordinary equivalence clause these will be the
+same EquivalenceClass, since processing of the clause itself causes its
+inputs to be put into the same EquivalenceClass. But as described above,
+mergejoinable outer-join clauses will end up with different
+EquivalenceClasses for left and right sides.
+
+There is an additional complication when re-ordering outer joins according
+to identity 3. Recall that the two choices we consider for such joins are
+ A leftjoin (B leftjoin C on (Pbc)) on (Pab)
+ (A leftjoin B on (Pab)) leftjoin C on (Pb*c)
+where the star denotes varnullingrels markers on B's Vars. When Pbc
+is (or includes) a mergejoinable clause, we have something like
+ A leftjoin (B leftjoin C on (b.b = c.c)) on (Pab)
+ (A leftjoin B on (Pab)) leftjoin C on (b.b* = c.c)
+We could generate a ConstrainedEquivalence linking b.b and c.c, and
+another one linking b.b* and c.c. (b.b and b.b* are necessarily in
+different EquivalenceClasses: there is no mechanism whereby they
+could be found to be equal.) However, these would generate largely
+duplicative conditions. Conditions involving b.b* can't be computed
+below this join nest, and any that can be computed would be duplicative
+of what we'd get from the b.b/c.c ConstrainedEquivalence. Therefore,
+we choose to generate a ConstrainedEquivalence for b.b and c.c, but
+"b.b* = c.c" is handled as just an ordinary clause.
Another way that we may form a single-item EquivalenceClass is in creation
-of a PathKey to represent a desired sort order (see below). This is a bit
+of a PathKey to represent a desired sort order (see below). This happens
+if an ORDER BY or GROUP BY key is not mentioned in any equivalence
+clause. We need to reason about sort orders in such queries, and our
+representation of sort ordering is a PathKey (see below) which uses an
+EquivalenceClass, so we have to make an EquivalenceClass. This is a bit
different from the above cases because such an EquivalenceClass might
contain an aggregate function or volatile expression. (A clause containing
a volatile function will never be considered mergejoinable, even if its top
@@ -579,7 +870,7 @@ Index scans have Path.pathkeys that represent the chosen index's ordering,
if any. A single-key index would create a single-PathKey list, while a
multi-column index generates a list with one element per key index column.
Non-key columns specified in the INCLUDE clause of covering indexes don't
-have corresponding PathKeys in the list, because the have no influence on
+have corresponding PathKeys in the list, because they have no influence on
index ordering. (Actually, since an index can be scanned either forward or
backward, there are two possible sort orders and two possible PathKey lists
it can generate.)
@@ -655,14 +946,9 @@ redundancy, we save time and improve planning, since the planner will more
easily recognize equivalent orderings as being equivalent.
Another interesting property is that if the underlying EquivalenceClass
-contains a constant and is not below an outer join, then the pathkey is
-completely redundant and need not be sorted by at all! Every row must
-contain the same constant value, so there's no need to sort. (If the EC is
-below an outer join, we still have to sort, since some of the rows might
-have gone to null and others not. In this case we must be careful to pick
-a non-const member to sort by. The assumption that all the non-const
-members go to null at the same plan level is critical here, else they might
-not produce the same sort order.) This might seem pointless because users
+contains a constant, then the pathkey is completely redundant and need not
+be sorted by at all! Every interesting row must contain the same value,
+so there's no need to sort. This might seem pointless because users
are unlikely to write "... WHERE x = 42 ORDER BY x", but it allows us to
recognize when particular index columns are irrelevant to the sort order:
if we have "... WHERE x = 42 ORDER BY y", scanning an index on (x,y)
@@ -670,15 +956,6 @@ produces correctly ordered data without a sort step. We used to have very
ugly ad-hoc code to recognize that in limited contexts, but discarding
constant ECs from pathkeys makes it happen cleanly and automatically.
-You might object that a below-outer-join EquivalenceClass doesn't always
-represent the same values at every level of the join tree, and so using
-it to uniquely identify a sort order is dubious. This is true, but we
-can avoid dealing with the fact explicitly because we always consider that
-an outer join destroys any ordering of its nullable inputs. Thus, even
-if a path was sorted by {a.x} below an outer join, we'll re-sort if that
-sort ordering was important; and so using the same PathKey for both sort
-orderings doesn't create any real problem.
-
Order of processing for EquivalenceClasses and PathKeys
-------------------------------------------------------
commit b98559d7c9a576a306b3367eb967ddda6ae18a85
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu Oct 27 14:56:08 2022 -0400
Add Var.varnullingrels and PlaceHolderVar.phnullingrels fields.
These fields are always empty as of this commit, so they don't
affect any behavior, even though equal() will compare them.
Update backend/nodes/ and backend/rewrite/ infrastructure as needed.
Also add some rewrite functions we'll need later.
Note this will require a catversion bump when committed.
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index c85d8fe975..cced668f58 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -80,11 +80,13 @@ makeVar(int varno,
var->varlevelsup = varlevelsup;
/*
- * Only a few callers need to make Var nodes with varnosyn/varattnosyn
- * different from varno/varattno. We don't provide separate arguments for
- * them, but just initialize them to the given varno/varattno. This
- * reduces code clutter and chance of error for most callers.
+ * Only a few callers need to make Var nodes with non-null varnullingrels,
+ * or with varnosyn/varattnosyn different from varno/varattno. We don't
+ * provide separate arguments for them, but just initialize them to NULL
+ * and the given varno/varattno. This reduces code clutter and chance of
+ * error for most callers.
*/
+ var->varnullingrels = NULL;
var->varnosyn = (Index) varno;
var->varattnosyn = varattno;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 0a7b22f97e..692f45daa5 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -2663,6 +2663,7 @@ expression_tree_mutator_impl(Node *node,
Var *newnode;
FLATCOPY(newnode, var, Var);
+ /* Assume we need not copy the varnullingrels bitmapset */
return (Node *) newnode;
}
break;
@@ -3257,7 +3258,7 @@ expression_tree_mutator_impl(Node *node,
FLATCOPY(newnode, phv, PlaceHolderVar);
MUTATE(newnode->phexpr, phv->phexpr, Expr *);
- /* Assume we need not copy the relids bitmapset */
+ /* Assume we need not copy the relids bitmapsets */
return (Node *) newnode;
}
break;
diff --git a/src/backend/rewrite/rewriteManip.c b/src/backend/rewrite/rewriteManip.c
index 101c39553a..ab24547e6d 100644
--- a/src/backend/rewrite/rewriteManip.c
+++ b/src/backend/rewrite/rewriteManip.c
@@ -40,6 +40,20 @@ typedef struct
int win_location;
} locate_windowfunc_context;
+typedef struct
+{
+ const Bitmapset *target_relids;
+ const Bitmapset *added_relids;
+ int sublevels_up;
+} add_nulling_relids_context;
+
+typedef struct
+{
+ const Bitmapset *removable_relids;
+ const Bitmapset *except_relids;
+ int sublevels_up;
+} remove_nulling_relids_context;
+
static bool contain_aggs_of_level_walker(Node *node,
contain_aggs_of_level_context *context);
static bool locate_agg_of_level_walker(Node *node,
@@ -50,6 +64,10 @@ static bool locate_windowfunc_walker(Node *node,
static bool checkExprHasSubLink_walker(Node *node, void *context);
static Relids offset_relid_set(Relids relids, int offset);
static Relids adjust_relid_set(Relids relids, int oldrelid, int newrelid);
+static Node *add_nulling_relids_mutator(Node *node,
+ add_nulling_relids_context *context);
+static Node *remove_nulling_relids_mutator(Node *node,
+ remove_nulling_relids_context *context);
/*
@@ -348,6 +366,8 @@ OffsetVarNodes_walker(Node *node, OffsetVarNodes_context *context)
if (var->varlevelsup == context->sublevels_up)
{
var->varno += context->offset;
+ var->varnullingrels = offset_relid_set(var->varnullingrels,
+ context->offset);
if (var->varnosyn > 0)
var->varnosyn += context->offset;
}
@@ -386,6 +406,8 @@ OffsetVarNodes_walker(Node *node, OffsetVarNodes_context *context)
{
phv->phrels = offset_relid_set(phv->phrels,
context->offset);
+ phv->phnullingrels = offset_relid_set(phv->phnullingrels,
+ context->offset);
}
/* fall through to examine children */
}
@@ -510,11 +532,13 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context)
{
Var *var = (Var *) node;
- if (var->varlevelsup == context->sublevels_up &&
- var->varno == context->rt_index)
+ if (var->varlevelsup == context->sublevels_up)
{
- var->varno = context->new_index;
- /* If the syntactic referent is same RTE, fix it too */
+ if (var->varno == context->rt_index)
+ var->varno = context->new_index;
+ var->varnullingrels = adjust_relid_set(var->varnullingrels,
+ context->rt_index,
+ context->new_index);
if (var->varnosyn == context->rt_index)
var->varnosyn = context->new_index;
}
@@ -557,6 +581,9 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context)
phv->phrels = adjust_relid_set(phv->phrels,
context->rt_index,
context->new_index);
+ phv->phnullingrels = adjust_relid_set(phv->phnullingrels,
+ context->rt_index,
+ context->new_index);
}
/* fall through to examine children */
}
@@ -833,7 +860,8 @@ rangeTableEntry_used_walker(Node *node,
Var *var = (Var *) node;
if (var->varlevelsup == context->sublevels_up &&
- var->varno == context->rt_index)
+ (var->varno == context->rt_index ||
+ bms_is_member(context->rt_index, var->varnullingrels)))
return true;
return false;
}
@@ -1061,6 +1089,195 @@ AddInvertedQual(Query *parsetree, Node *qual)
}
+/*
+ * add_nulling_relids() finds Vars and PlaceHolderVars that belong to any
+ * of the target_relids, and adds added_relids to their varnullingrels
+ * and phnullingrels fields.
+ */
+Node *
+add_nulling_relids(Node *node,
+ const Bitmapset *target_relids,
+ const Bitmapset *added_relids)
+{
+ add_nulling_relids_context context;
+
+ context.target_relids = target_relids;
+ context.added_relids = added_relids;
+ context.sublevels_up = 0;
+ return query_or_expression_tree_mutator(node,
+ add_nulling_relids_mutator,
+ &context,
+ 0);
+}
+
+static Node *
+add_nulling_relids_mutator(Node *node,
+ add_nulling_relids_context *context)
+{
+ if (node == NULL)
+ return NULL;
+ if (IsA(node, Var))
+ {
+ Var *var = (Var *) node;
+
+ if (var->varlevelsup == context->sublevels_up &&
+ bms_is_member(var->varno, context->target_relids))
+ {
+ Relids newnullingrels = bms_union(var->varnullingrels,
+ context->added_relids);
+
+ /* Copy the Var ... */
+ var = copyObject(var);
+ /* ... and replace the copy's varnullingrels field */
+ var->varnullingrels = newnullingrels;
+ return (Node *) var;
+ }
+ /* Otherwise fall through to copy the Var normally */
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ if (phv->phlevelsup == context->sublevels_up &&
+ bms_overlap(phv->phrels, context->target_relids))
+ {
+ Relids newnullingrels = bms_union(phv->phnullingrels,
+ context->added_relids);
+
+ /*
+ * We don't modify the contents of the PHV's expression, only add
+ * to phnullingrels. This corresponds to assuming that the PHV
+ * will be evaluated at the same level as before, then perhaps be
+ * nulled as it bubbles up. Hence, just flat-copy the node ...
+ */
+ phv = makeNode(PlaceHolderVar);
+ memcpy(phv, node, sizeof(PlaceHolderVar));
+ /* ... and replace the copy's phnullingrels field */
+ phv->phnullingrels = newnullingrels;
+ return (Node *) phv;
+ }
+ /* Otherwise fall through to copy the PlaceHolderVar normally */
+ }
+ else if (IsA(node, Query))
+ {
+ /* Recurse into RTE or sublink subquery */
+ Query *newnode;
+
+ context->sublevels_up++;
+ newnode = query_tree_mutator((Query *) node,
+ add_nulling_relids_mutator,
+ (void *) context,
+ 0);
+ context->sublevels_up--;
+ return (Node *) newnode;
+ }
+ return expression_tree_mutator(node, add_nulling_relids_mutator,
+ (void *) context);
+}
+
+/*
+ * remove_nulling_relids() removes mentions of the specified RT index(es)
+ * in Var.varnullingrels and PlaceHolderVar.phnullingrels fields within
+ * the given expression, except in nodes belonging to rels listed in
+ * except_relids.
+ */
+Node *
+remove_nulling_relids(Node *node,
+ const Bitmapset *removable_relids,
+ const Bitmapset *except_relids)
+{
+ remove_nulling_relids_context context;
+
+ context.removable_relids = removable_relids;
+ context.except_relids = except_relids;
+ context.sublevels_up = 0;
+ return query_or_expression_tree_mutator(node,
+ remove_nulling_relids_mutator,
+ &context,
+ 0);
+}
+
+static Node *
+remove_nulling_relids_mutator(Node *node,
+ remove_nulling_relids_context *context)
+{
+ if (node == NULL)
+ return NULL;
+ if (IsA(node, Var))
+ {
+ Var *var = (Var *) node;
+
+ if (var->varlevelsup == context->sublevels_up &&
+ !bms_is_member(var->varno, context->except_relids) &&
+ bms_overlap(var->varnullingrels, context->removable_relids))
+ {
+ Relids newnullingrels = bms_difference(var->varnullingrels,
+ context->removable_relids);
+
+ /* Micro-optimization: ensure nullingrels is NULL if empty */
+ if (bms_is_empty(newnullingrels))
+ newnullingrels = NULL;
+ /* Copy the Var ... */
+ var = copyObject(var);
+ /* ... and replace the copy's varnullingrels field */
+ var->varnullingrels = newnullingrels;
+ return (Node *) var;
+ }
+ /* Otherwise fall through to copy the Var normally */
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ if (phv->phlevelsup == context->sublevels_up &&
+ !bms_overlap(phv->phrels, context->except_relids))
+ {
+ Relids newnullingrels = bms_difference(phv->phnullingrels,
+ context->removable_relids);
+
+ /*
+ * Micro-optimization: ensure nullingrels is NULL if empty.
+ *
+ * Note: it might seem desirable to remove the PHV altogether if
+ * phnullingrels goes to empty. Currently we dare not do that
+ * because we use PHVs in some cases to enforce separate identity
+ * of subexpressions; see wrap_non_vars usages in prepjointree.c.
+ */
+ if (bms_is_empty(newnullingrels))
+ newnullingrels = NULL;
+ /* Copy the PlaceHolderVar and mutate what's below ... */
+ phv = (PlaceHolderVar *)
+ expression_tree_mutator(node,
+ remove_nulling_relids_mutator,
+ (void *) context);
+ /* ... and replace the copy's phnullingrels field */
+ phv->phnullingrels = newnullingrels;
+ /* We must also update phrels, if it contains a removable RTI */
+ phv->phrels = bms_difference(phv->phrels,
+ context->removable_relids);
+ Assert(!bms_is_empty(phv->phrels));
+ return (Node *) phv;
+ }
+ /* Otherwise fall through to copy the PlaceHolderVar normally */
+ }
+ else if (IsA(node, Query))
+ {
+ /* Recurse into RTE or sublink subquery */
+ Query *newnode;
+
+ context->sublevels_up++;
+ newnode = query_tree_mutator((Query *) node,
+ remove_nulling_relids_mutator,
+ (void *) context,
+ 0);
+ context->sublevels_up--;
+ return (Node *) newnode;
+ }
+ return expression_tree_mutator(node, remove_nulling_relids_mutator,
+ (void *) context);
+}
+
+
/*
* replace_rte_variables() finds all Vars in an expression tree
* that reference a particular RTE, and replaces them with substitute
diff --git a/src/backend/utils/misc/queryjumble.c b/src/backend/utils/misc/queryjumble.c
index a8508463e7..c0b254fc6e 100644
--- a/src/backend/utils/misc/queryjumble.c
+++ b/src/backend/utils/misc/queryjumble.c
@@ -383,6 +383,11 @@ JumbleExpr(JumbleState *jstate, Node *node)
APP_JUMB(var->varno);
APP_JUMB(var->varattno);
APP_JUMB(var->varlevelsup);
+
+ /*
+ * We can omit varnullingrels, because it's fully determined
+ * by varno/varlevelsup plus the Var's query location.
+ */
}
break;
case T_Const:
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 09342d128d..2c6d5ca58f 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -874,7 +874,7 @@ typedef struct RelOptInfo
int32 *attr_widths pg_node_attr(read_write_ignore);
/* LATERAL Vars and PHVs referenced by rel */
List *lateral_vars;
- /* rels that reference me laterally */
+ /* rels that reference this baserel laterally */
Relids lateral_referencers;
/* list of IndexOptInfo */
List *indexlist;
@@ -884,10 +884,7 @@ typedef struct RelOptInfo
BlockNumber pages;
Cardinality tuples;
double allvisfrac;
-
- /*
- * Indexes in PlannerInfo's eq_classes list of ECs that mention this rel
- */
+ /* indexes in PlannerInfo's eq_classes list of ECs that mention this rel */
Bitmapset *eclass_indexes;
PlannerInfo *subroot; /* if subquery */
List *subplan_params; /* if subquery */
@@ -2586,10 +2583,15 @@ typedef struct MergeScanSelCache
* of a plan tree. This is used during planning to represent the contained
* expression. At the end of the planning process it is replaced by either
* the contained expression or a Var referring to a lower-level evaluation of
- * the contained expression. Typically the evaluation occurs below an outer
+ * the contained expression. Generally the evaluation occurs below an outer
* join, and Var references above the outer join might thereby yield NULL
* instead of the expression value.
*
+ * phrels and phlevelsup correspond to the varno/varlevelsup fields of a
+ * plain Var, except that phrels has to be a relid set since the evaluation
+ * level of a PlaceHolderVar might be a join rather than a base relation.
+ * Likewise, phnullingrels corresponds to varnullingrels.
+ *
* Although the planner treats this as an expression node type, it is not
* recognized by the parser or executor, so we declare it here rather than
* in primnodes.h.
@@ -2602,8 +2604,10 @@ typedef struct MergeScanSelCache
* PHV. Another way in which it can happen is that initplan sublinks
* could get replaced by differently-numbered Params when sublink folding
* is done. (The end result of such a situation would be some
- * unreferenced initplans, which is annoying but not really a problem.) On
- * the same reasoning, there is no need to examine phrels.
+ * unreferenced initplans, which is annoying but not really a problem.)
+ * On the same reasoning, there is no need to examine phrels. But we do
+ * need to compare phnullingrels, as that represents effects that are
+ * external to the original value of the PHV.
*/
typedef struct PlaceHolderVar
@@ -2613,9 +2617,12 @@ typedef struct PlaceHolderVar
/* the represented expression */
Expr *phexpr pg_node_attr(equal_ignore);
- /* base relids syntactically within expr src */
+ /* base+OJ relids syntactically within expr src */
Relids phrels pg_node_attr(equal_ignore);
+ /* RT indexes of outer joins that can null PHV's value */
+ Relids phnullingrels;
+
/* ID for PHV (unique within planner run) */
Index phid;
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index f71f551782..8f133e12ac 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -180,6 +180,14 @@ typedef struct Expr
* row identity information during UPDATE/DELETE/MERGE. This value should
* never be seen outside the planner.
*
+ * varnullingrels is the set of RT indexes of outer joins that can force
+ * the Var's value to null (at the point where it appears in the query).
+ * See optimizer/README for discussion of that.
+ *
+ * varlevelsup is greater than zero in Vars that represent outer references.
+ * Note that it affects the meaning of all of varno, varnullingrels, and
+ * varnosyn, all of which refer to the range table of that query level.
+ *
* In the parser, varnosyn and varattnosyn are either identical to
* varno/varattno, or they specify the column's position in an aliased JOIN
* RTE that hides the semantic referent RTE's refname. This is a syntactic
@@ -222,6 +230,8 @@ typedef struct Var
int32 vartypmod;
/* OID of collation, or InvalidOid if none */
Oid varcollid;
+ /* RT indexes of outer joins that can replace the Var's value with null */
+ Bitmapset *varnullingrels;
/*
* for subquery variables referencing outer relations; 0 in a normal var,
diff --git a/src/include/rewrite/rewriteManip.h b/src/include/rewrite/rewriteManip.h
index f001ca41bb..351ec15612 100644
--- a/src/include/rewrite/rewriteManip.h
+++ b/src/include/rewrite/rewriteManip.h
@@ -63,6 +63,13 @@ extern bool contain_windowfuncs(Node *node);
extern int locate_windowfunc(Node *node);
extern bool checkExprHasSubLink(Node *node);
+extern Node *add_nulling_relids(Node *node,
+ const Bitmapset *target_relids,
+ const Bitmapset *added_relids);
+extern Node *remove_nulling_relids(Node *node,
+ const Bitmapset *removable_relids,
+ const Bitmapset *except_relids);
+
extern Node *replace_rte_variables(Node *node,
int target_varno, int sublevels_up,
replace_rte_variables_callback callback,
commit f2d852846983b671a277f2321396a61e9a3d984b
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu Oct 27 14:59:43 2022 -0400
Teach the parser to fill Var.varnullingrels correctly.
Vars emitted by the parser are now marked with RT indexes of outer
joins that can null them. (This is done purely according to the
syntax of the query; we don't consider whether an outer join could
be strength-reduced, for example.)
Although the result of this step compiles, it will fail some
regression tests due to the planner not yet knowing what to do.
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 6688c2a865..dff3b1e349 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -670,6 +670,7 @@ transformInsertStmt(ParseState *pstate, InsertStmt *stmt)
*/
sub_pstate->p_rtable = sub_rtable;
sub_pstate->p_joinexprs = NIL; /* sub_rtable has no joins */
+ sub_pstate->p_nullingrels = NIL;
sub_pstate->p_namespace = sub_namespace;
sub_pstate->p_resolve_unknowns = false;
@@ -851,7 +852,7 @@ transformInsertStmt(ParseState *pstate, InsertStmt *stmt)
/*
* Generate list of Vars referencing the RTE
*/
- exprList = expandNSItemVars(nsitem, 0, -1, NULL);
+ exprList = expandNSItemVars(pstate, nsitem, 0, -1, NULL);
/*
* Re-apply any indirection on the target column specs to the Vars
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index e01c0734d1..95590d9ed2 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -52,7 +52,8 @@
#include "utils/syscache.h"
-static int extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
+static int extractRemainingColumns(ParseState *pstate,
+ ParseNamespaceColumn *src_nscolumns,
List *src_colnames,
List **src_colnos,
List **res_colnames, List **res_colvars,
@@ -75,9 +76,11 @@ static ParseNamespaceItem *getNSItemForSpecialRelationTypes(ParseState *pstate,
static Node *transformFromClauseItem(ParseState *pstate, Node *n,
ParseNamespaceItem **top_nsitem,
List **namespace);
-static Var *buildVarFromNSColumn(ParseNamespaceColumn *nscol);
+static Var *buildVarFromNSColumn(ParseState *pstate,
+ ParseNamespaceColumn *nscol);
static Node *buildMergedJoinVar(ParseState *pstate, JoinType jointype,
Var *l_colvar, Var *r_colvar);
+static void markRelsAsNulledBy(ParseState *pstate, Node *n, int jindex);
static void setNamespaceColumnVisibility(List *namespace, bool cols_visible);
static void setNamespaceLateralState(List *namespace,
bool lateral_only, bool lateral_ok);
@@ -251,7 +254,8 @@ setTargetTable(ParseState *pstate, RangeVar *relation,
* Returns the number of columns added.
*/
static int
-extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
+extractRemainingColumns(ParseState *pstate,
+ ParseNamespaceColumn *src_nscolumns,
List *src_colnames,
List **src_colnos,
List **res_colnames, List **res_colvars,
@@ -287,7 +291,8 @@ extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
*src_colnos = lappend_int(*src_colnos, attnum);
*res_colnames = lappend(*res_colnames, lfirst(lc));
*res_colvars = lappend(*res_colvars,
- buildVarFromNSColumn(src_nscolumns + attnum - 1));
+ buildVarFromNSColumn(pstate,
+ src_nscolumns + attnum - 1));
/* Copy the input relation's nscolumn data for this column */
res_nscolumns[colcount] = src_nscolumns[attnum - 1];
colcount++;
@@ -1288,8 +1293,7 @@ transformFromClauseItem(ParseState *pstate, Node *n,
{
/*
* JOIN/USING (or NATURAL JOIN, as transformed above). Transform
- * the list into an explicit ON-condition, and generate a list of
- * merged result columns.
+ * the list into an explicit ON-condition.
*/
List *ucols = j->usingClause;
List *l_usingvars = NIL;
@@ -1307,8 +1311,6 @@ transformFromClauseItem(ParseState *pstate, Node *n,
int r_index = -1;
Var *l_colvar,
*r_colvar;
- Node *u_colvar;
- ParseNamespaceColumn *res_nscolumn;
Assert(u_colname[0] != '\0');
@@ -1372,17 +1374,109 @@ transformFromClauseItem(ParseState *pstate, Node *n,
u_colname)));
r_colnos = lappend_int(r_colnos, r_index + 1);
- l_colvar = buildVarFromNSColumn(l_nscolumns + l_index);
+ /* Build Vars to use in the generated JOIN ON clause */
+ l_colvar = buildVarFromNSColumn(pstate, l_nscolumns + l_index);
l_usingvars = lappend(l_usingvars, l_colvar);
- r_colvar = buildVarFromNSColumn(r_nscolumns + r_index);
+ r_colvar = buildVarFromNSColumn(pstate, r_nscolumns + r_index);
r_usingvars = lappend(r_usingvars, r_colvar);
+ /*
+ * While we're here, add column names to the res_colnames
+ * list. It's a bit ugly to do this here while the
+ * corresponding res_colvars entries are not made till later,
+ * but doing this later would require an additional traversal
+ * of the usingClause list.
+ */
res_colnames = lappend(res_colnames, lfirst(ucol));
+ }
+
+ /* Construct the generated JOIN ON clause */
+ j->quals = transformJoinUsingClause(pstate,
+ l_usingvars,
+ r_usingvars);
+ }
+ else if (j->quals)
+ {
+ /* User-written ON-condition; transform it */
+ j->quals = transformJoinOnClause(pstate, j, my_namespace);
+ }
+ else
+ {
+ /* CROSS JOIN: no quals */
+ }
+
+ /*
+ * If this is an outer join, now mark the appropriate child RTEs as
+ * being nulled by this join. We have finished processing the child
+ * join expressions as well as the current join's quals, which deal in
+ * non-nulled input columns. All future references to those RTEs will
+ * see possibly-nulled values, and we should mark generated Vars to
+ * account for that. In particular, the join alias Vars that we're
+ * about to build should reflect the nulling effects of this join.
+ *
+ * A difficulty with doing this is that we need the join's RT index,
+ * which we don't officially have yet. However, no other RTE can get
+ * made between here and the addRangeTableEntryForJoin call, so we can
+ * predict what the assignment will be. (Alternatively, we could call
+ * addRangeTableEntryForJoin before we have all the data computed, but
+ * this seems less ugly.)
+ */
+ j->rtindex = list_length(pstate->p_rtable) + 1;
+
+ switch (j->jointype)
+ {
+ case JOIN_INNER:
+ break;
+ case JOIN_LEFT:
+ markRelsAsNulledBy(pstate, j->rarg, j->rtindex);
+ break;
+ case JOIN_FULL:
+ markRelsAsNulledBy(pstate, j->larg, j->rtindex);
+ markRelsAsNulledBy(pstate, j->rarg, j->rtindex);
+ break;
+ case JOIN_RIGHT:
+ markRelsAsNulledBy(pstate, j->larg, j->rtindex);
+ break;
+ default:
+ /* shouldn't see any other types here */
+ elog(ERROR, "unrecognized join type: %d",
+ (int) j->jointype);
+ break;
+ }
+
+ /*
+ * Now we can construct join alias expressions for the USING columns.
+ */
+ if (j->usingClause)
+ {
+ ListCell *lc1,
+ *lc2;
+
+ /* Scan the colnos lists to recover info from the previous loop */
+ forboth(lc1, l_colnos, lc2, r_colnos)
+ {
+ int l_index = lfirst_int(lc1) - 1;
+ int r_index = lfirst_int(lc2) - 1;
+ Var *l_colvar,
+ *r_colvar;
+ Node *u_colvar;
+ ParseNamespaceColumn *res_nscolumn;
+
+ /*
+ * Note we re-build these Vars: they might have different
+ * varnullingrels than the ones made in the previous loop.
+ */
+ l_colvar = buildVarFromNSColumn(pstate, l_nscolumns + l_index);
+ r_colvar = buildVarFromNSColumn(pstate, r_nscolumns + r_index);
+
+ /* Construct the join alias Var for this column */
u_colvar = buildMergedJoinVar(pstate,
j->jointype,
l_colvar,
r_colvar);
res_colvars = lappend(res_colvars, u_colvar);
+
+ /* Construct column's res_nscolumns[] entry */
res_nscolumn = res_nscolumns + res_colindex;
res_colindex++;
if (u_colvar == (Node *) l_colvar)
@@ -1400,47 +1494,45 @@ transformFromClauseItem(ParseState *pstate, Node *n,
/*
* Merged column is not semantically equivalent to either
* input, so it needs to be referenced as the join output
- * column. We don't know the join's varno yet, so we'll
- * replace these zeroes below.
+ * column.
*/
- res_nscolumn->p_varno = 0;
+ res_nscolumn->p_varno = j->rtindex;
res_nscolumn->p_varattno = res_colindex;
res_nscolumn->p_vartype = exprType(u_colvar);
res_nscolumn->p_vartypmod = exprTypmod(u_colvar);
res_nscolumn->p_varcollid = exprCollation(u_colvar);
- res_nscolumn->p_varnosyn = 0;
+ res_nscolumn->p_varnosyn = j->rtindex;
res_nscolumn->p_varattnosyn = res_colindex;
}
}
-
- j->quals = transformJoinUsingClause(pstate,
- l_usingvars,
- r_usingvars);
- }
- else if (j->quals)
- {
- /* User-written ON-condition; transform it */
- j->quals = transformJoinOnClause(pstate, j, my_namespace);
- }
- else
- {
- /* CROSS JOIN: no quals */
}
/* Add remaining columns from each side to the output columns */
res_colindex +=
- extractRemainingColumns(l_nscolumns, l_colnames, &l_colnos,
+ extractRemainingColumns(pstate,
+ l_nscolumns, l_colnames, &l_colnos,
&res_colnames, &res_colvars,
res_nscolumns + res_colindex);
res_colindex +=
- extractRemainingColumns(r_nscolumns, r_colnames, &r_colnos,
+ extractRemainingColumns(pstate,
+ r_nscolumns, r_colnames, &r_colnos,
&res_colnames, &res_colvars,
res_nscolumns + res_colindex);
+ /* If join has an alias, it syntactically hides all inputs */
+ if (j->alias)
+ {
+ for (k = 0; k < res_colindex; k++)
+ {
+ ParseNamespaceColumn *nscol = res_nscolumns + k;
+
+ nscol->p_varnosyn = j->rtindex;
+ nscol->p_varattnosyn = k + 1;
+ }
+ }
+
/*
* Now build an RTE and nsitem for the result of the join.
- * res_nscolumns isn't totally done yet, but that's OK because
- * addRangeTableEntryForJoin doesn't examine it, only store a pointer.
*/
nsitem = addRangeTableEntryForJoin(pstate,
res_colnames,
@@ -1454,31 +1546,16 @@ transformFromClauseItem(ParseState *pstate, Node *n,
j->alias,
true);
- j->rtindex = nsitem->p_rtindex;
+ /* Verify that we correctly predicted the join's RT index */
+ Assert(j->rtindex == nsitem->p_rtindex);
+ /* Cross-check number of columns, too */
+ Assert(res_colindex == list_length(nsitem->p_names->colnames));
/*
- * Now that we know the join RTE's rangetable index, we can fix up the
- * res_nscolumns data in places where it should contain that.
+ * Save a link to the JoinExpr in the proper element of p_joinexprs.
+ * Since we maintain that list lazily, it may be necessary to fill in
+ * empty entries before we can add the JoinExpr in the right place.
*/
- Assert(res_colindex == list_length(nsitem->p_names->colnames));
- for (k = 0; k < res_colindex; k++)
- {
- ParseNamespaceColumn *nscol = res_nscolumns + k;
-
- /* fill in join RTI for merged columns */
- if (nscol->p_varno == 0)
- nscol->p_varno = j->rtindex;
- if (nscol->p_varnosyn == 0)
- nscol->p_varnosyn = j->rtindex;
- /* if join has an alias, it syntactically hides all inputs */
- if (j->alias)
- {
- nscol->p_varnosyn = j->rtindex;
- nscol->p_varattnosyn = k + 1;
- }
- }
-
- /* make a matching link to the JoinExpr for later use */
for (k = list_length(pstate->p_joinexprs) + 1; k < j->rtindex; k++)
pstate->p_joinexprs = lappend(pstate->p_joinexprs, NULL);
pstate->p_joinexprs = lappend(pstate->p_joinexprs, j);
@@ -1547,10 +1624,13 @@ transformFromClauseItem(ParseState *pstate, Node *n,
* buildVarFromNSColumn -
* build a Var node using ParseNamespaceColumn data
*
- * We assume varlevelsup should be 0, and no location is specified
+ * This is used to construct joinaliasvars entries.
+ * We can assume varlevelsup should be 0, and no location is specified.
+ * Note also that no column SELECT privilege is requested here; that would
+ * happen only if the column is actually referenced in the query.
*/
static Var *
-buildVarFromNSColumn(ParseNamespaceColumn *nscol)
+buildVarFromNSColumn(ParseState *pstate, ParseNamespaceColumn *nscol)
{
Var *var;
@@ -1564,6 +1644,10 @@ buildVarFromNSColumn(ParseNamespaceColumn *nscol)
/* makeVar doesn't offer parameters for these, so set by hand: */
var->varnosyn = nscol->p_varnosyn;
var->varattnosyn = nscol->p_varattnosyn;
+
+ /* ... and update varnullingrels */
+ markNullableIfNeeded(pstate, var);
+
return var;
}
@@ -1675,6 +1759,47 @@ buildMergedJoinVar(ParseState *pstate, JoinType jointype,
return res_node;
}
+/*
+ * markRelsAsNulledBy -
+ * Mark the given jointree node and its children as nulled by join jindex
+ */
+static void
+markRelsAsNulledBy(ParseState *pstate, Node *n, int jindex)
+{
+ int varno;
+ ListCell *lc;
+
+ /* Note: we can't see FromExpr here */
+ if (IsA(n, RangeTblRef))
+ {
+ varno = ((RangeTblRef *) n)->rtindex;
+ }
+ else if (IsA(n, JoinExpr))
+ {
+ JoinExpr *j = (JoinExpr *) n;
+
+ /* recurse to children */
+ markRelsAsNulledBy(pstate, j->larg, jindex);
+ markRelsAsNulledBy(pstate, j->rarg, jindex);
+ varno = j->rtindex;
+ }
+ else
+ {
+ elog(ERROR, "unrecognized node type: %d", (int) nodeTag(n));
+ varno = 0; /* keep compiler quiet */
+ }
+
+ /*
+ * Now add jindex to the p_nullingrels set for relation varno. Since we
+ * maintain the p_nullingrels list lazily, we might need to extend it to
+ * make the varno'th entry exist.
+ */
+ while (list_length(pstate->p_nullingrels) < varno)
+ pstate->p_nullingrels = lappend(pstate->p_nullingrels, NULL);
+ lc = list_nth_cell(pstate->p_nullingrels, varno - 1);
+ lfirst(lc) = bms_add_member((Bitmapset *) lfirst(lc), jindex);
+}
+
/*
* setNamespaceColumnVisibility -
* Convenience subroutine to update cols_visible flags in a namespace list.
diff --git a/src/backend/parser/parse_coerce.c b/src/backend/parser/parse_coerce.c
index 60908111c8..606491bd66 100644
--- a/src/backend/parser/parse_coerce.c
+++ b/src/backend/parser/parse_coerce.c
@@ -1042,7 +1042,7 @@ coerce_record_to_complex(ParseState *pstate, Node *node,
ParseNamespaceItem *nsitem;
nsitem = GetNSItemByRangeTablePosn(pstate, rtindex, sublevels_up);
- args = expandNSItemVars(nsitem, sublevels_up, vlocation, NULL);
+ args = expandNSItemVars(pstate, nsitem, sublevels_up, vlocation, NULL);
}
else
ereport(ERROR,
diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c
index e5fc708c8a..3fce9c5b62 100644
--- a/src/backend/parser/parse_expr.c
+++ b/src/backend/parser/parse_expr.c
@@ -2538,6 +2538,9 @@ transformWholeRowRef(ParseState *pstate, ParseNamespaceItem *nsitem,
/* location is not filled in by makeWholeRowVar */
result->location = location;
+ /* mark Var if it's nulled by any outer joins */
+ markNullableIfNeeded(pstate, result);
+
/* mark relation as requiring whole-row SELECT access */
markVarForSelectPriv(pstate, result);
@@ -2565,6 +2568,8 @@ transformWholeRowRef(ParseState *pstate, ParseNamespaceItem *nsitem,
rowexpr->colnames = copyObject(nsitem->p_names->colnames);
rowexpr->location = location;
+ /* XXX we ought to mark the row as possibly nullable */
+
return (Node *) rowexpr;
}
}
diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c
index 81f9ae2f02..fd1631fe75 100644
--- a/src/backend/parser/parse_relation.c
+++ b/src/backend/parser/parse_relation.c
@@ -751,6 +751,9 @@ scanNSItemForColumn(ParseState *pstate, ParseNamespaceItem *nsitem,
}
var->location = location;
+ /* Mark Var if it's nulled by any outer joins */
+ markNullableIfNeeded(pstate, var);
+
/* Require read access to the column */
markVarForSelectPriv(pstate, var);
@@ -1007,6 +1010,35 @@ searchRangeTableForCol(ParseState *pstate, const char *alias, const char *colnam
return fuzzystate;
}
+/*
+ * markNullableIfNeeded
+ * If the RTE referenced by the Var is nullable by outer join(s)
+ * at this point in the query, set var->varnullingrels to show that.
+ */
+void
+markNullableIfNeeded(ParseState *pstate, Var *var)
+{
+ int rtindex = var->varno;
+ Bitmapset *relids;
+
+ /* Find the appropriate pstate */
+ for (int lv = 0; lv < var->varlevelsup; lv++)
+ pstate = pstate->parentParseState;
+
+ /* Find currently-relevant join relids for the Var's rel */
+ if (rtindex > 0 && rtindex <= list_length(pstate->p_nullingrels))
+ relids = (Bitmapset *) list_nth(pstate->p_nullingrels, rtindex - 1);
+ else
+ relids = NULL;
+
+ /*
+ * Merge with any already-declared nulling rels. (Typically there won't
+ * be any, but let's get it right if there are.)
+ */
+ if (relids != NULL)
+ var->varnullingrels = bms_union(var->varnullingrels, relids);
+}
+
/*
* markRTEForSelectPriv
* Mark the specified column of the RTE with index rtindex
@@ -3109,7 +3141,7 @@ expandTupleDesc(TupleDesc tupdesc, Alias *eref, int count, int offset,
* the list elements mustn't be modified.
*/
List *
-expandNSItemVars(ParseNamespaceItem *nsitem,
+expandNSItemVars(ParseState *pstate, ParseNamespaceItem *nsitem,
int sublevels_up, int location,
List **colnames)
{
@@ -3145,6 +3177,10 @@ expandNSItemVars(ParseNamespaceItem *nsitem,
var->varnosyn = nscol->p_varnosyn;
var->varattnosyn = nscol->p_varattnosyn;
var->location = location;
+
+ /* ... and update varnullingrels */
+ markNullableIfNeeded(pstate, var);
+
result = lappend(result, var);
if (colnames)
*colnames = lappend(*colnames, colnameval);
@@ -3179,7 +3215,7 @@ expandNSItemAttrs(ParseState *pstate, ParseNamespaceItem *nsitem,
*var;
List *te_list = NIL;
- vars = expandNSItemVars(nsitem, sublevels_up, location, &names);
+ vars = expandNSItemVars(pstate, nsitem, sublevels_up, location, &names);
/*
* Require read access to the table. This is normally redundant with the
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index bd8057bc3e..4f5dd2e99f 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -1370,7 +1370,7 @@ ExpandSingleTable(ParseState *pstate, ParseNamespaceItem *nsitem,
List *vars;
ListCell *l;
- vars = expandNSItemVars(nsitem, sublevels_up, location, NULL);
+ vars = expandNSItemVars(pstate, nsitem, sublevels_up, location, NULL);
/*
* Require read access to the table. This is normally redundant with
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 7e7ad3f7e4..9d986b0dee 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1079,6 +1079,14 @@ typedef struct RangeTblEntry
* alias Vars are generated only for merged columns). We keep these
* entries only because they're needed in expandRTE() and similar code.
*
+ * Vars appearing within joinaliasvars are marked with varnullingrels sets
+ * that describe the nulling effects of this join and lower ones. This is
+ * essential for FULL JOIN cases, because the COALESCE expression only
+ * describes the semantics correctly if its inputs have been nulled by the
+ * join. For other cases, it allows expandRTE() to generate a valid
+ * representation of the join's output without consulting additional
+ * parser state.
+ *
* Within a Query loaded from a stored rule, it is possible for non-merged
* joinaliasvars items to be null pointers, which are placeholders for
* (necessarily unreferenced) columns dropped since the rule was made.
diff --git a/src/include/parser/parse_node.h b/src/include/parser/parse_node.h
index 962ebf65de..636d3231cd 100644
--- a/src/include/parser/parse_node.h
+++ b/src/include/parser/parse_node.h
@@ -115,6 +115,13 @@ typedef Node *(*CoerceParamHook) (ParseState *pstate, Param *param,
* This is one-for-one with p_rtable, but contains NULLs for non-join
* RTEs, and may be shorter than p_rtable if the last RTE(s) aren't joins.
*
+ * p_nullingrels: list of Bitmapsets associated with p_rtable entries, each
+ * containing the set of outer-join RTE indexes that can null that relation
+ * at the current point in the parse tree. This is one-for-one with p_rtable,
+ * but may be shorter than p_rtable, in which case the missing entries are
+ * implicitly empty (NULL). That rule allows us to save work when the query
+ * contains no outer joins.
+ *
* p_joinlist: list of join items (RangeTblRef and JoinExpr nodes) that
* will become the fromlist of the query's top-level FromExpr node.
*
@@ -182,6 +189,7 @@ struct ParseState
const char *p_sourcetext; /* source text, or NULL if not available */
List *p_rtable; /* range table so far */
List *p_joinexprs; /* JoinExprs for RTE_JOIN p_rtable entries */
+ List *p_nullingrels; /* Bitmapsets showing nulling outer joins */
List *p_joinlist; /* join items so far (will become FromExpr
* node's fromlist) */
List *p_namespace; /* currently-referenceable RTEs (List of
diff --git a/src/include/parser/parse_relation.h b/src/include/parser/parse_relation.h
index 484db165db..e7e72d6f3e 100644
--- a/src/include/parser/parse_relation.h
+++ b/src/include/parser/parse_relation.h
@@ -41,6 +41,7 @@ extern Node *scanNSItemForColumn(ParseState *pstate, ParseNamespaceItem *nsitem,
int location);
extern Node *colNameToVar(ParseState *pstate, const char *colname, bool localonly,
int location);
+extern void markNullableIfNeeded(ParseState *pstate, Var *var);
extern void markVarForSelectPriv(ParseState *pstate, Var *var);
extern Relation parserOpenTable(ParseState *pstate, const RangeVar *relation,
int lockmode);
@@ -109,7 +110,7 @@ extern void errorMissingColumn(ParseState *pstate,
extern void expandRTE(RangeTblEntry *rte, int rtindex, int sublevels_up,
int location, bool include_dropped,
List **colnames, List **colvars);
-extern List *expandNSItemVars(ParseNamespaceItem *nsitem,
+extern List *expandNSItemVars(ParseState *pstate, ParseNamespaceItem *nsitem,
int sublevels_up, int location,
List **colnames);
extern List *expandNSItemAttrs(ParseState *pstate, ParseNamespaceItem *nsitem,
commit aefa1a9788165a3be592b493a0821fdbda7c2598
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Oct 31 11:51:22 2022 -0400
Teach the planner to cope with Vars bearing nullingrels.
The core idea of this step is to include varnullingrels in the
relid sets that qual clauses are considered to depend on.
So that we can still easily compare quals' relids to RelOptInfos'
relids, that means also adding outer join relids to the identifying
relids of join relations. Much of the bulk of this step is concerned
with fallout from the latter change.
I've resolved the previous squishiness entailed by outer join identity 3
by generating multiple versions of outer-join quals that could get moved
to a join level where they need to contain different nullingrels sets.
Now we have versions of such quals with the correct nullingrels for
each level where they could appear.
This requires a bit of new mechanism (RestrictInfo.has_clone/is_clone)
to prevent multiple versions of the same qual from getting used in the
plan. My worry about how that could work with EquivalenceClasses is
resolved by creating EquivalenceClasses only from the least-marked
version of a qual. (This doesn't really lose anything, since versions
with more nullingrels bits don't correspond to any equalities available
outside the nest of commuting outer joins.)
These extra versions of quals would also result in generating multiple
parameterized paths that differ only in what nullingrels they expect
for the Vars from the parameterization rel(s). That seems like it'd
be very wasteful, so I've arranged to generate such paths only from
the least-marked version of a qual (the has_clone version).
Unlike in the previous version of this patch, setrefs.c is able to
cross-check the nullingrel sets of most Vars and PlaceHolderVars to
ensure that they match up with what the previous plan step produces.
But there are three cases that I've so far punted on:
1. The targetlist and qpqual of an outer join node will contain
nullingrels bits for the outer join itself. To check exact matching to
the input, we'd need to know the OJ's relid as well as which input(s)
got nulled, neither of which is cheaply available in setrefs.c. For
now, it's just checking that such Vars have a superset of the input's
nullingrels bits.
2. Parameterized paths will generally refer to the least-marked version
of whichever outer-side Vars they use, which may not be what's actually
available from the outside of the nestloop. (We're relying on the join
ordering rules for that to be sensible.) Again, setrefs.c is in no
position to pass judgment on correctness, so it's just checking that
the parameter expression has a subset of the outer-side marking.
3. Row identity variables are not marked with any nullingrels, which
may not correspond to reality. I've punted on this by skipping the
checks when varattno <= 0.
Point 1 could be addressed if we were willing to add informational
fields to join plan nodes, which might be worth doing, but I'm not sure.
The other two points seem like the extra mechanisms needed for a
bulletproof check would be considerably more trouble than they'd be
worth.
There is still some confusion about which versions of a cloned qual
are actually necessary to check, which results in some extra filter
conditions showing up in a couple of regression test plans. There are
also some failure cases involving full joins that remain to be fixed.
This patch is already mighty big, so I'll address those failures
separately.
This step removes some low-hanging fruit from the old implementation,
such as the need to track lowest_nulling_outer_join during subquery
pullup. There's much more to do in that line, though.
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index 004481d608..1c921879a9 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -273,7 +273,7 @@ merge_clump(PlannerInfo *root, List *clumps, Clump *new_clump, int num_gene,
* rel once we know the final targetlist (see
* grouping_planner).
*/
- if (!bms_equal(joinrel->relids, root->all_baserels))
+ if (!bms_equal(joinrel->relids, root->all_query_rels))
generate_useful_gather_paths(root, joinrel, false);
/* Find and save the cheapest paths for this joinrel */
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 4ddaed31a4..5902c80747 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -159,27 +159,6 @@ make_one_rel(PlannerInfo *root, List *joinlist)
Index rti;
double total_pages;
- /*
- * Construct the all_baserels Relids set.
- */
- root->all_baserels = NULL;
- for (rti = 1; rti < root->simple_rel_array_size; rti++)
- {
- RelOptInfo *brel = root->simple_rel_array[rti];
-
- /* there may be empty slots corresponding to non-baserel RTEs */
- if (brel == NULL)
- continue;
-
- Assert(brel->relid == rti); /* sanity check on array */
-
- /* ignore RTEs that are "other rels" */
- if (brel->reloptkind != RELOPT_BASEREL)
- continue;
-
- root->all_baserels = bms_add_member(root->all_baserels, brel->relid);
- }
-
/* Mark base rels as to whether we care about fast-start plans */
set_base_rel_consider_startup(root);
@@ -207,6 +186,7 @@ make_one_rel(PlannerInfo *root, List *joinlist)
{
RelOptInfo *brel = root->simple_rel_array[rti];
+ /* there may be empty slots corresponding to non-baserel RTEs */
if (brel == NULL)
continue;
@@ -231,9 +211,9 @@ make_one_rel(PlannerInfo *root, List *joinlist)
rel = make_rel_from_joinlist(root, joinlist);
/*
- * The result should join all and only the query's base rels.
+ * The result should join all and only the query's base + outer-join rels.
*/
- Assert(bms_equal(rel->relids, root->all_baserels));
+ Assert(bms_equal(rel->relids, root->all_query_rels));
return rel;
}
@@ -558,7 +538,7 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* the final scan/join targetlist is available (see grouping_planner).
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- !bms_equal(rel->relids, root->all_baserels))
+ !bms_equal(rel->relids, root->all_query_rels))
generate_useful_gather_paths(root, rel, false);
/* Now find the cheapest of the paths for this rel */
@@ -879,7 +859,7 @@ set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *
* to support an uncommon usage of second-rate sampling methods. Instead,
* if there is a risk that the query might perform an unsafe join, just
* wrap the SampleScan in a Materialize node. We can check for joins by
- * counting the membership of all_baserels (note that this correctly
+ * counting the membership of all_query_rels (note that this correctly
* counts inheritance trees as single rels). If we're inside a subquery,
* we can't easily check whether a join might occur in the outer query, so
* just assume one is possible.
@@ -888,7 +868,7 @@ set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *
* so check repeatable_across_scans last, even though that's a bit odd.
*/
if ((root->query_level > 1 ||
- bms_membership(root->all_baserels) != BMS_SINGLETON) &&
+ bms_membership(root->all_query_rels) != BMS_SINGLETON) &&
!(GetTsmRoutine(rte->tablesample->tsmhandler)->repeatable_across_scans))
{
path = (Path *) create_material_path(rel, path);
@@ -970,7 +950,7 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
if (enable_partitionwise_join &&
rel->reloptkind == RELOPT_BASEREL &&
rte->relkind == RELKIND_PARTITIONED_TABLE &&
- rel->attr_needed[InvalidAttrNumber - rel->min_attr] == NULL)
+ bms_is_empty(rel->attr_needed[InvalidAttrNumber - rel->min_attr]))
rel->consider_partitionwise_join = true;
/*
@@ -3435,7 +3415,7 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
* partial paths. We'll do the same for the topmost scan/join rel
* once we know the final targetlist (see grouping_planner).
*/
- if (!bms_equal(rel->relids, root->all_baserels))
+ if (!bms_equal(rel->relids, root->all_query_rels))
generate_useful_gather_paths(root, rel, false);
/* Find and save the cheapest paths for this rel */
diff --git a/src/backend/optimizer/path/clausesel.c b/src/backend/optimizer/path/clausesel.c
index 06f836308d..c08eb2b1c5 100644
--- a/src/backend/optimizer/path/clausesel.c
+++ b/src/backend/optimizer/path/clausesel.c
@@ -218,7 +218,7 @@ clauselist_selectivity_ext(PlannerInfo *root,
if (rinfo)
{
- ok = (bms_membership(rinfo->clause_relids) == BMS_SINGLETON) &&
+ ok = (rinfo->num_base_rels == 1) &&
(is_pseudo_constant_clause_relids(lsecond(expr->args),
rinfo->right_relids) ||
(varonleft = false,
@@ -579,30 +579,6 @@ find_single_rel_for_clauses(PlannerInfo *root, List *clauses)
return NULL; /* no clauses */
}
-/*
- * bms_is_subset_singleton
- *
- * Same result as bms_is_subset(s, bms_make_singleton(x)),
- * but a little faster and doesn't leak memory.
- *
- * Is this of use anywhere else? If so move to bitmapset.c ...
- */
-static bool
-bms_is_subset_singleton(const Bitmapset *s, int x)
-{
- switch (bms_membership(s))
- {
- case BMS_EMPTY_SET:
- return true;
- case BMS_SINGLETON:
- return bms_is_member(x, s);
- case BMS_MULTIPLE:
- return false;
- }
- /* can't get here... */
- return false;
-}
-
/*
* treat_as_join_clause -
* Decide whether an operator clause is to be handled by the
@@ -631,17 +607,20 @@ treat_as_join_clause(PlannerInfo *root, Node *clause, RestrictInfo *rinfo,
else
{
/*
- * Otherwise, it's a join if there's more than one relation used. We
- * can optimize this calculation if an rinfo was passed.
+ * Otherwise, it's a join if there's more than one base relation used.
+ * We can optimize this calculation if an rinfo was passed.
*
* XXX Since we know the clause is being evaluated at a join, the
* only way it could be single-relation is if it was delayed by outer
- * joins. Although we can make use of the restriction qual estimators
- * anyway, it seems likely that we ought to account for the
- * probability of injected nulls somehow.
+ * joins. We intentionally count only baserels here, not OJs that
+ * might be present in rinfo->clause_relids, so that we direct such
+ * cases to the restriction qual estimators not join estimators.
+ * Eventually some notice should be taken of the possibility of
+ * injected nulls, but we'll likely want to do that in the restriction
+ * estimators rather than starting to treat such cases as join quals.
*/
if (rinfo)
- return (bms_membership(rinfo->clause_relids) == BMS_MULTIPLE);
+ return (rinfo->num_base_rels > 1);
else
return (NumRelids(root, clause) > 1);
}
@@ -754,7 +733,9 @@ clause_selectivity_ext(PlannerInfo *root,
* for all non-JOIN_INNER cases.
*/
if (varRelid == 0 ||
- bms_is_subset_singleton(rinfo->clause_relids, varRelid))
+ rinfo->num_base_rels == 0 ||
+ (rinfo->num_base_rels == 1 &&
+ bms_is_member(varRelid, rinfo->clause_relids)))
{
/* Cacheable --- do we already have the result? */
if (jointype == JOIN_INNER)
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 4c6b1d1f55..30ac8ae721 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4783,6 +4783,11 @@ compute_semi_anti_join_factors(PlannerInfo *root,
norm_sjinfo.syn_lefthand = outerrel->relids;
norm_sjinfo.syn_righthand = innerrel->relids;
norm_sjinfo.jointype = JOIN_INNER;
+ norm_sjinfo.ojrelid = 0;
+ norm_sjinfo.commute_above_l = NULL;
+ norm_sjinfo.commute_above_r = NULL;
+ norm_sjinfo.commute_below = NULL;
+ norm_sjinfo.oj_joinclause = NIL;
/* we don't bother trying to make the remaining fields valid */
norm_sjinfo.lhs_strict = false;
norm_sjinfo.delay_upper_joins = false;
@@ -4948,6 +4953,11 @@ approx_tuple_count(PlannerInfo *root, JoinPath *path, List *quals)
sjinfo.syn_lefthand = path->outerjoinpath->parent->relids;
sjinfo.syn_righthand = path->innerjoinpath->parent->relids;
sjinfo.jointype = JOIN_INNER;
+ sjinfo.ojrelid = 0;
+ sjinfo.commute_above_l = NULL;
+ sjinfo.commute_above_r = NULL;
+ sjinfo.commute_below = NULL;
+ sjinfo.oj_joinclause = NIL;
/* we don't bother trying to make the remaining fields valid */
sjinfo.lhs_strict = false;
sjinfo.delay_upper_joins = false;
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index e65b967b1f..349e183372 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -29,6 +29,7 @@
#include "optimizer/paths.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "rewrite/rewriteManip.h"
#include "utils/lsyscache.h"
@@ -64,7 +65,7 @@ static bool reconsider_outer_join_clause(PlannerInfo *root,
RestrictInfo *rinfo,
bool outer_on_left);
static bool reconsider_full_join_clause(PlannerInfo *root,
- RestrictInfo *rinfo);
+ FullJoinClauseInfo *fjinfo);
static Bitmapset *get_eclass_indexes_for_relids(PlannerInfo *root,
Relids relids);
static Bitmapset *get_common_eclass_indexes(PlannerInfo *root, Relids relids1,
@@ -757,6 +758,12 @@ get_eclass_for_sort_expr(PlannerInfo *root,
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL) /* must be an outer join */
+ {
+ Assert(bms_is_member(i, root->outer_join_rels));
+ continue;
+ }
+
Assert(rel->reloptkind == RELOPT_BASEREL ||
rel->reloptkind == RELOPT_DEADREL);
@@ -1113,6 +1120,12 @@ generate_base_implied_equalities(PlannerInfo *root)
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL) /* must be an outer join */
+ {
+ Assert(bms_is_member(i, root->outer_join_rels));
+ continue;
+ }
+
Assert(rel->reloptkind == RELOPT_BASEREL);
rel->eclass_indexes = bms_add_member(rel->eclass_indexes,
@@ -2015,10 +2028,12 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* Process the FULL JOIN clauses */
foreach(cell, root->full_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ FullJoinClauseInfo *fjinfo = (FullJoinClauseInfo *) lfirst(cell);
- if (reconsider_full_join_clause(root, rinfo))
+ if (reconsider_full_join_clause(root, fjinfo))
{
+ RestrictInfo *rinfo = fjinfo->rinfo;
+
found = true;
/* remove it from the list */
root->full_join_clauses =
@@ -2047,9 +2062,9 @@ reconsider_outer_join_clauses(PlannerInfo *root)
}
foreach(cell, root->full_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ FullJoinClauseInfo *fjinfo = (FullJoinClauseInfo *) lfirst(cell);
- distribute_restrictinfo_to_rels(root, rinfo);
+ distribute_restrictinfo_to_rels(root, fjinfo->rinfo);
}
}
@@ -2185,8 +2200,11 @@ reconsider_outer_join_clause(PlannerInfo *root, RestrictInfo *rinfo,
* Returns true if we were able to propagate a constant through the clause.
*/
static bool
-reconsider_full_join_clause(PlannerInfo *root, RestrictInfo *rinfo)
+reconsider_full_join_clause(PlannerInfo *root, FullJoinClauseInfo *fjinfo)
{
+ RestrictInfo *rinfo = fjinfo->rinfo;
+ SpecialJoinInfo *sjinfo = fjinfo->sjinfo;
+ Relids fjrelids = bms_make_singleton(sjinfo->ojrelid);
Expr *leftvar;
Expr *rightvar;
Oid opno,
@@ -2268,6 +2286,18 @@ reconsider_full_join_clause(PlannerInfo *root, RestrictInfo *rinfo)
cfirst = (Node *) linitial(cexpr->args);
csecond = (Node *) lsecond(cexpr->args);
+ /*
+ * The COALESCE arguments will be marked as possibly nulled by
+ * the full join, while we wish to generate clauses that apply
+ * to the join's inputs. So we must strip the join from the
+ * nullingrels fields of cfirst/csecond before comparing them
+ * to leftvar/rightvar. (Perhaps with a less hokey
+ * representation for FULL JOIN USING output columns, this
+ * wouldn't be needed?)
+ */
+ cfirst = remove_nulling_relids(cfirst, fjrelids, NULL);
+ csecond = remove_nulling_relids(csecond, fjrelids, NULL);
+
if (equal(leftvar, cfirst) && equal(rightvar, csecond))
{
coal_idx = foreach_current_index(lc2);
@@ -3204,6 +3234,12 @@ get_eclass_indexes_for_relids(PlannerInfo *root, Relids relids)
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL) /* must be an outer join */
+ {
+ Assert(bms_is_member(i, root->outer_join_rels));
+ continue;
+ }
+
ec_indexes = bms_add_members(ec_indexes, rel->eclass_indexes);
}
return ec_indexes;
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index 77f3f81bcb..7fa405e5b9 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -3372,13 +3372,13 @@ check_index_predicates(PlannerInfo *root, RelOptInfo *rel)
* Add on any equivalence-derivable join clauses. Computing the correct
* relid sets for generate_join_implied_equalities is slightly tricky
* because the rel could be a child rel rather than a true baserel, and in
- * that case we must remove its parents' relid(s) from all_baserels.
+ * that case we must subtract its parents' relid(s) from all_query_rels.
*/
if (rel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- otherrels = bms_difference(root->all_baserels,
+ otherrels = bms_difference(root->all_query_rels,
find_childrel_parents(root, rel));
else
- otherrels = bms_difference(root->all_baserels, rel->relids);
+ otherrels = bms_difference(root->all_query_rels, rel->relids);
if (!bms_is_empty(otherrels))
clauselist =
@@ -3756,7 +3756,8 @@ match_index_to_operand(Node *operand,
*/
if (operand && IsA(operand, Var) &&
index->rel->relid == ((Var *) operand)->varno &&
- indkey == ((Var *) operand)->varattno)
+ indkey == ((Var *) operand)->varattno &&
+ ((Var *) operand)->varnullingrels == NULL)
return true;
}
else
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 2a3f0ab7bf..cd3f9fa0af 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -234,7 +234,9 @@ add_paths_to_joinrel(PlannerInfo *root,
* reduces the number of parameterized paths we have to deal with at
* higher join levels, without compromising the quality of the resulting
* plan. We express the restriction as a Relids set that must overlap the
- * parameterization of any proposed join path.
+ * parameterization of any proposed join path. Note: param_source_rels
+ * should contain only baserels, not OJ relids, so starting from
+ * all_baserels not all_query_rels is correct.
*/
foreach(lc, root->join_info_list)
{
@@ -365,6 +367,47 @@ allow_star_schema_join(PlannerInfo *root,
bms_nonempty_difference(inner_paramrels, outerrelids));
}
+/*
+ * If the parameterization is only partly satisfied by the outer rel,
+ * the unsatisfied part can't include any outer-join relids that could
+ * null rels of the satisfied part. That would imply that we're trying
+ * to use a clause involving a Var with nonempty varnullingrels at
+ * a join level where that value isn't yet computable.
+ */
+static inline bool
+have_unsafe_outer_join_ref(PlannerInfo *root,
+ Relids outerrelids,
+ Relids inner_paramrels)
+{
+ bool result = false;
+ Relids unsatisfied = bms_difference(inner_paramrels, outerrelids);
+
+ if (bms_overlap(unsatisfied, root->outer_join_rels))
+ {
+ ListCell *lc;
+
+ foreach(lc, root->join_info_list)
+ {
+ SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+
+ if (!bms_is_member(sjinfo->ojrelid, unsatisfied))
+ continue; /* not relevant */
+ if (bms_overlap(inner_paramrels, sjinfo->min_righthand) ||
+ (sjinfo->jointype == JOIN_FULL &&
+ bms_overlap(inner_paramrels, sjinfo->min_lefthand)))
+ {
+ result = true; /* doesn't work */
+ break;
+ }
+ }
+ }
+
+ /* Waste no memory when we reject a path here */
+ bms_free(unsatisfied);
+
+ return result;
+}
+
/*
* paraminfo_get_equal_hashops
* Determine if param_info and innerrel's lateral_vars can be hashed.
@@ -656,15 +699,16 @@ try_nestloop_path(PlannerInfo *root,
/*
* Check to see if proposed path is still parameterized, and reject if the
* parameterization wouldn't be sensible --- unless allow_star_schema_join
- * says to allow it anyway. Also, we must reject if have_dangerous_phv
- * doesn't like the look of it, which could only happen if the nestloop is
- * still parameterized.
+ * says to allow it anyway. Also, we must reject if either
+ * have_unsafe_outer_join_ref or have_dangerous_phv don't like the look of
+ * it, which could only happen if the nestloop is still parameterized.
*/
required_outer = calc_nestloop_required_outer(outerrelids, outer_paramrels,
innerrelids, inner_paramrels);
if (required_outer &&
((!bms_overlap(required_outer, extra->param_source_rels) &&
!allow_star_schema_join(root, outerrelids, inner_paramrels)) ||
+ have_unsafe_outer_join_ref(root, outerrelids, inner_paramrels) ||
have_dangerous_phv(root, outerrelids, inner_paramrels)))
{
/* Waste no memory when we reject a path here */
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 9da3ff2f9a..6c28b0a057 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -353,7 +353,10 @@ make_rels_by_clauseless_joins(PlannerInfo *root,
*
* Caller must supply not only the two rels, but the union of their relids.
* (We could simplify the API by computing joinrelids locally, but this
- * would be redundant work in the normal path through make_join_rel.)
+ * would be redundant work in the normal path through make_join_rel.
+ * Note that this value does NOT include the RT index of any outer join that
+ * might need to be performed here, so it's not the canonical identifier
+ * of the join relation.)
*
* On success, *sjinfo_p is set to NULL if this is to be a plain inner join,
* else it's set to point to the associated SpecialJoinInfo node. Also,
@@ -695,7 +698,7 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(rel1->relids, rel2->relids));
- /* Construct Relids set that identifies the joinrel. */
+ /* Construct Relids set that identifies the joinrel (without OJ as yet). */
joinrelids = bms_union(rel1->relids, rel2->relids);
/* Check validity and determine join type. */
@@ -707,6 +710,10 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
return NULL;
}
+ /* If we have an outer join, add its RTI to form the canonical relids. */
+ if (sjinfo && sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
+
/* Swap rels if needed to match the join info. */
if (reversed)
{
@@ -730,6 +737,11 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
sjinfo->syn_lefthand = rel1->relids;
sjinfo->syn_righthand = rel2->relids;
sjinfo->jointype = JOIN_INNER;
+ sjinfo->ojrelid = 0;
+ sjinfo->commute_above_l = NULL;
+ sjinfo->commute_above_r = NULL;
+ sjinfo->commute_below = NULL;
+ sjinfo->oj_joinclause = NIL;
/* we don't bother trying to make the remaining fields valid */
sjinfo->lhs_strict = false;
sjinfo->delay_upper_joins = false;
@@ -1510,8 +1522,6 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(child_rel1->relids, child_rel2->relids));
- child_joinrelids = bms_union(child_rel1->relids, child_rel2->relids);
- appinfos = find_appinfos_by_relids(root, child_joinrelids, &nappinfos);
/*
* Construct SpecialJoinInfo from parent join relations's
@@ -1521,6 +1531,15 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
child_rel1->relids,
child_rel2->relids);
+ /* Build correct join relids for child join */
+ child_joinrelids = bms_union(child_rel1->relids, child_rel2->relids);
+ if (child_sjinfo->ojrelid != 0)
+ child_joinrelids = bms_add_member(child_joinrelids,
+ child_sjinfo->ojrelid);
+
+ /* Find the AppendRelInfo structures */
+ appinfos = find_appinfos_by_relids(root, child_joinrelids, &nappinfos);
+
/*
* Construct restrictions applicable to the child join from those
* applicable to the parent join.
@@ -1536,8 +1555,7 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
{
child_joinrel = build_child_join_rel(root, child_rel1, child_rel2,
joinrel, child_restrictlist,
- child_sjinfo,
- child_sjinfo->jointype);
+ child_sjinfo);
joinrel->part_rels[cnt_parts] = child_joinrel;
joinrel->live_parts = bms_add_member(joinrel->live_parts, cnt_parts);
joinrel->all_partrels = bms_add_members(joinrel->all_partrels,
@@ -1583,6 +1601,8 @@ build_child_join_sjinfo(PlannerInfo *root, SpecialJoinInfo *parent_sjinfo,
sjinfo->syn_righthand = adjust_child_relids(sjinfo->syn_righthand,
right_nappinfos,
right_appinfos);
+ /* outer-join relids need no adjustment */
+ Assert(sjinfo->oj_joinclause == NIL); /* should be empty now */
sjinfo->semi_rhs_exprs = (List *) adjust_appendrel_attrs(root,
(Node *) sjinfo->semi_rhs_exprs,
right_nappinfos,
diff --git a/src/backend/optimizer/path/tidpath.c b/src/backend/optimizer/path/tidpath.c
index c4e035b049..71488cec00 100644
--- a/src/backend/optimizer/path/tidpath.c
+++ b/src/backend/optimizer/path/tidpath.c
@@ -59,6 +59,7 @@ IsCTIDVar(Var *var, RelOptInfo *rel)
if (var->varattno == SelfItemPointerAttributeNumber &&
var->vartype == TIDOID &&
var->varno == rel->relid &&
+ var->varnullingrels == NULL &&
var->varlevelsup == 0)
return true;
return false;
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index bbeca9a9ab..0652b3200a 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -34,7 +34,7 @@
/* local functions */
static bool join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo);
-static void remove_rel_from_query(PlannerInfo *root, int relid,
+static void remove_rel_from_query(PlannerInfo *root, int relid, int ojrelid,
Relids joinrelids);
static List *remove_rel_from_joinlist(List *joinlist, int relid, int *nremoved);
static bool rel_supports_distinctness(PlannerInfo *root, RelOptInfo *rel);
@@ -70,6 +70,7 @@ restart:
foreach(lc, root->join_info_list)
{
SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+ Relids joinrelids;
int innerrelid;
int nremoved;
@@ -84,9 +85,12 @@ restart:
*/
innerrelid = bms_singleton_member(sjinfo->min_righthand);
- remove_rel_from_query(root, innerrelid,
- bms_union(sjinfo->min_lefthand,
- sjinfo->min_righthand));
+ /* Compute the relid set for the join we are considering */
+ joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ if (sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
+
+ remove_rel_from_query(root, innerrelid, sjinfo->ojrelid, joinrelids);
/* We verify that exactly one reference gets removed from joinlist */
nremoved = 0;
@@ -188,6 +192,8 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
/* Compute the relid set for the join we are considering */
joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ if (sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
/*
* We can't remove the join if any inner-rel attributes are used above the
@@ -247,6 +253,17 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
{
RestrictInfo *restrictinfo = (RestrictInfo *) lfirst(l);
+ /*
+ * If the current join commutes with some other outer join(s) via
+ * outer join identity 3, there will be multiple clones of its join
+ * clauses in the joininfo list. We want to consider only the
+ * has_clone form of such clauses. Processing more than one form
+ * would be wasteful, and also some of the others would confuse the
+ * RINFO_IS_PUSHED_DOWN test below.
+ */
+ if (restrictinfo->is_clone)
+ continue; /* ignore it */
+
/*
* If it's not a join clause for this outer join, we can't use it.
* Note that if the clause is pushed-down, then it is logically from
@@ -306,10 +323,12 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
* no longer treated as a baserel, and that attributes of other baserels
* are no longer marked as being needed at joins involving this rel.
* Also, join quals involving the rel have to be removed from the joininfo
- * lists, but only if they belong to the outer join identified by joinrelids.
+ * lists, but only if they belong to the outer join identified by ojrelid
+ * and joinrelids.
*/
static void
-remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
+remove_rel_from_query(PlannerInfo *root, int relid, int ojrelid,
+ Relids joinrelids)
{
RelOptInfo *rel = find_base_rel(root, relid);
List *joininfos;
@@ -349,6 +368,14 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
}
}
+ /*
+ * Update all_baserels and related relid sets.
+ */
+ root->all_baserels = bms_del_member(root->all_baserels, relid);
+ root->outer_join_rels = bms_del_member(root->outer_join_rels, ojrelid);
+ root->all_query_rels = bms_del_member(root->all_query_rels, relid);
+ root->all_query_rels = bms_del_member(root->all_query_rels, ojrelid);
+
/*
* Likewise remove references from SpecialJoinInfo data structures.
*
@@ -365,6 +392,15 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
sjinfo->min_righthand = bms_del_member(sjinfo->min_righthand, relid);
sjinfo->syn_lefthand = bms_del_member(sjinfo->syn_lefthand, relid);
sjinfo->syn_righthand = bms_del_member(sjinfo->syn_righthand, relid);
+ sjinfo->min_lefthand = bms_del_member(sjinfo->min_lefthand, ojrelid);
+ sjinfo->min_righthand = bms_del_member(sjinfo->min_righthand, ojrelid);
+ sjinfo->syn_lefthand = bms_del_member(sjinfo->syn_lefthand, ojrelid);
+ sjinfo->syn_righthand = bms_del_member(sjinfo->syn_righthand, ojrelid);
+ /* relid cannot appear in these fields, but ojrelid can: */
+ sjinfo->commute_above_l = bms_del_member(sjinfo->commute_above_l, ojrelid);
+ sjinfo->commute_above_r = bms_del_member(sjinfo->commute_above_r, ojrelid);
+ sjinfo->commute_below = bms_del_member(sjinfo->commute_below, ojrelid);
+ Assert(sjinfo->oj_joinclause == NIL); /* should be empty now */
}
/*
@@ -396,8 +432,10 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
else
{
phinfo->ph_eval_at = bms_del_member(phinfo->ph_eval_at, relid);
+ phinfo->ph_eval_at = bms_del_member(phinfo->ph_eval_at, ojrelid);
Assert(!bms_is_empty(phinfo->ph_eval_at));
phinfo->ph_needed = bms_del_member(phinfo->ph_needed, relid);
+ phinfo->ph_needed = bms_del_member(phinfo->ph_needed, ojrelid);
}
}
@@ -422,7 +460,12 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
remove_join_clause_from_rels(root, rinfo, rinfo->required_relids);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrelids))
+ /*
+ * If the qual lists ojrelid in its required_relids, it must have come
+ * from above the outer join we're removing (so we need to keep it);
+ * if it does not, then it didn't and we can discard it.
+ */
+ if (bms_is_member(ojrelid, rinfo->required_relids))
{
/* Recheck that qual doesn't actually reference the target rel */
Assert(!bms_is_member(relid, rinfo->clause_relids));
@@ -434,6 +477,8 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
rinfo->required_relids = bms_copy(rinfo->required_relids);
rinfo->required_relids = bms_del_member(rinfo->required_relids,
relid);
+ rinfo->required_relids = bms_del_member(rinfo->required_relids,
+ ojrelid);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
@@ -548,6 +593,7 @@ reduce_unique_semijoins(PlannerInfo *root)
/* Compute the relid set for the join we are considering */
joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ Assert(sjinfo->ojrelid == 0); /* SEMI joins don't have RT indexes */
/*
* Since we're only considering a single-rel RHS, any join clauses it
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index fd8cbb1dc7..41c69b29a7 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -60,16 +60,34 @@ static void process_security_barrier_quals(PlannerInfo *root,
static SpecialJoinInfo *make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
- JoinType jointype, List *clause);
+ JoinType jointype, Index ojrelid,
+ List *clause);
static void compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo,
List *clause);
+static void process_postponed_left_join_quals(PlannerInfo *root);
+static void distribute_quals_to_rels(PlannerInfo *root, List *clauses,
+ bool below_outer_join,
+ SpecialJoinInfo *sjinfo,
+ Index security_level,
+ Relids qualscope,
+ Relids ojscope,
+ Relids outerjoin_nonnullable,
+ bool allow_equivalence,
+ bool postpone_nondegenerate_clauses,
+ bool has_clone,
+ bool is_clone,
+ List **postponed_qual_list);
static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool below_outer_join,
- JoinType jointype,
+ SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
Relids ojscope,
Relids outerjoin_nonnullable,
+ bool allow_equivalence,
+ bool postpone_nondegenerate_clauses,
+ bool has_clone,
+ bool is_clone,
List **postponed_qual_list);
static bool check_outerjoin_delay(PlannerInfo *root, Relids *relids_p,
Relids *nullable_relids_p, bool is_pushed_down);
@@ -92,7 +110,7 @@ static void check_memoizable(RestrictInfo *restrictinfo);
*
* Scan the query's jointree and create baserel RelOptInfos for all
* the base relations (e.g., table, subquery, and function RTEs)
- * appearing in the jointree.
+ * appearing in the jointree. Also add their relids to all_baserels.
*
* The initial invocation must pass root->parse->jointree as the value of
* jtnode. Internally, the function recurses through the jointree.
@@ -112,6 +130,7 @@ add_base_rels_to_query(PlannerInfo *root, Node *jtnode)
int varno = ((RangeTblRef *) jtnode)->rtindex;
(void) build_simple_rel(root, varno, NULL);
+ root->all_baserels = bms_add_member(root->all_baserels, varno);
}
else if (IsA(jtnode, FromExpr))
{
@@ -230,6 +249,23 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
{
ListCell *temp;
+ /*
+ * By convention, attr_needed and ph_needed values contain only baserel
+ * relids (and of course "relation 0"), not outer-join relids. It's
+ * sufficient to keep track of this at baserel granularity, since whether
+ * an outer join has been computed at a particular join level is fully
+ * determined by the set of baserels in the join. If we included outer
+ * joins then we'd get confused by the varying sets of outer-join relids
+ * appearing in different versions of commutable outer-join clauses, and
+ * think that vars need to propagate higher than they really do. However,
+ * the presented value of where_needed will be a join relid set that may
+ * contain OJ relids, so we gotta mask here. This code assumes that
+ * "relation 0" is always presented alone, not along with other bits.
+ */
+ if (!bms_is_member(0, where_needed))
+ where_needed = bms_intersect(where_needed, root->all_baserels);
+
+ /* Should (still) have a nonempty set */
Assert(!bms_is_empty(where_needed));
foreach(temp, vars)
@@ -248,10 +284,16 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
attno -= rel->min_attr;
if (rel->attr_needed[attno] == NULL)
{
- /* Variable not yet requested, so add to rel's targetlist */
- /* XXX is copyObject necessary here? */
- rel->reltarget->exprs = lappend(rel->reltarget->exprs,
- copyObject(var));
+ /*
+ * Variable not yet requested, so add to rel's targetlist.
+ *
+ * The value available at the rel's scan level has not been
+ * nulled by any outer join, so drop its varnullingrels.
+ * (We'll put those back as we climb up the join tree.)
+ */
+ var = copyObject(var);
+ var->varnullingrels = NULL;
+ rel->reltarget->exprs = lappend(rel->reltarget->exprs, var);
/* reltarget cost and width will be computed later */
}
rel->attr_needed[attno] = bms_add_members(rel->attr_needed[attno],
@@ -547,8 +589,10 @@ create_lateral_join_info(PlannerInfo *root)
varno = -1;
while ((varno = bms_next_member(eval_at, varno)) >= 0)
{
- RelOptInfo *brel = find_base_rel(root, varno);
+ RelOptInfo *brel = find_base_rel_ignore_join(root, varno);
+ if (brel == NULL)
+ continue; /* ignore outer joins in eval_at */
brel->lateral_relids = bms_add_members(brel->lateral_relids,
phinfo->ph_lateral);
}
@@ -639,7 +683,10 @@ create_lateral_join_info(PlannerInfo *root)
{
RelOptInfo *brel2 = root->simple_rel_array[rti2];
- Assert(brel2 != NULL && brel2->reloptkind == RELOPT_BASEREL);
+ if (brel2 == NULL)
+ continue; /* must be an OJ */
+
+ Assert(brel2->reloptkind == RELOPT_BASEREL);
brel2->lateral_referencers =
bms_add_member(brel2->lateral_referencers, rti);
}
@@ -699,16 +746,27 @@ deconstruct_jointree(PlannerInfo *root)
Assert(root->parse->jointree != NULL &&
IsA(root->parse->jointree, FromExpr));
- /* this is filled as we scan the jointree */
+ /* These are filled as we scan the jointree */
+ root->outer_join_rels = NULL;
root->nullable_baserels = NULL;
result = deconstruct_recurse(root, (Node *) root->parse->jointree, false,
&qualscope, &inner_join_rels,
&postponed_qual_list);
- /* Shouldn't be any leftover quals */
+ /* Now we can form the value of all_query_rels, too */
+ root->all_query_rels = bms_union(root->all_baserels, root->outer_join_rels);
+
+ /* Shouldn't be any leftover postponed quals */
Assert(postponed_qual_list == NIL);
+ /*
+ * However, if there were any special joins then we may well have some
+ * postponed LEFT JOIN clauses to deal with.
+ */
+ if (root->join_info_list)
+ process_postponed_left_join_quals(root);
+
return result;
}
@@ -721,7 +779,7 @@ deconstruct_jointree(PlannerInfo *root)
* below_outer_join is true if this node is within the nullable side of a
* higher-level outer join
* Outputs:
- * *qualscope gets the set of base Relids syntactically included in this
+ * *qualscope gets the set of base+OJ Relids syntactically included in this
* jointree node (do not modify or free this, as it may also be pointed
* to by RestrictInfo and SpecialJoinInfo nodes)
* *inner_join_rels gets the set of base Relids syntactically included in
@@ -806,6 +864,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
* there was exactly one element, we should (and already did) report
* whatever its inner_join_rels were. If there were no elements (is
* that still possible?) the initialization before the loop fixed it.
+ *
+ * XXX now wrong, do we care?
*/
if (list_length(f->fromlist) > 1)
*inner_join_rels = *qualscope;
@@ -820,10 +880,10 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
if (bms_is_subset(pq->relids, *qualscope))
distribute_qual_to_rels(root, pq->qual,
- below_outer_join, JOIN_INNER,
+ below_outer_join, NULL,
root->qual_security_level,
*qualscope, NULL, NULL,
- NULL);
+ true, false, false, false, NULL);
else
*postponed_qual_list = lappend(*postponed_qual_list, pq);
}
@@ -831,16 +891,12 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
/*
* Now process the top-level quals.
*/
- foreach(l, (List *) f->quals)
- {
- Node *qual = (Node *) lfirst(l);
-
- distribute_qual_to_rels(root, qual,
- below_outer_join, JOIN_INNER,
- root->qual_security_level,
- *qualscope, NULL, NULL,
- postponed_qual_list);
- }
+ distribute_quals_to_rels(root, (List *) f->quals,
+ below_outer_join, NULL,
+ root->qual_security_level,
+ *qualscope, NULL, NULL,
+ true, false, false, false,
+ postponed_qual_list);
}
else if (IsA(jtnode, JoinExpr))
{
@@ -857,6 +913,7 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
*rightjoinlist;
List *my_quals;
SpecialJoinInfo *sjinfo;
+ bool postpone_nondegenerate_clauses;
ListCell *l;
/*
@@ -900,6 +957,13 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ /* caution: ANTI join derived from SEMI will lack rtindex */
+ if (j->rtindex != 0)
+ {
+ *qualscope = bms_add_member(*qualscope, j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
+ j->rtindex);
+ }
*inner_join_rels = bms_union(left_inners, right_inners);
nonnullable_rels = leftids;
nullable_rels = rightids;
@@ -914,6 +978,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ /* SEMI join never has rtindex, so don't add to qualscope */
+ Assert(j->rtindex == 0);
*inner_join_rels = bms_union(left_inners, right_inners);
/* Semi join adds no restrictions for quals */
nonnullable_rels = NULL;
@@ -935,6 +1001,10 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ Assert(j->rtindex != 0);
+ *qualscope = bms_add_member(*qualscope, j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
+ j->rtindex);
*inner_join_rels = bms_union(left_inners, right_inners);
/* each side is both outer and inner */
nonnullable_rels = *qualscope;
@@ -994,12 +1064,28 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
leftids, rightids,
*inner_join_rels,
j->jointype,
+ j->rtindex,
my_quals);
if (j->jointype == JOIN_SEMI)
ojscope = NULL;
else
+ {
ojscope = bms_union(sjinfo->min_lefthand,
sjinfo->min_righthand);
+
+ /*
+ * Add back any commutable lower OJ relids that were removed
+ * from min_lefthand or min_righthand, else the ojscope
+ * cross-check in distribute_qual_to_rels will complain. If
+ * any such OJs were removed, we will postpone processing of
+ * non-degenerate clauses, so this addition doesn't affect
+ * anything except that cross-check and some Asserts. Real
+ * clause positioning decisions will be made later, when we
+ * revisit the postponed clauses.
+ */
+ if (sjinfo->commute_below)
+ ojscope = bms_add_members(ojscope, sjinfo->commute_below);
+ }
}
else
{
@@ -1007,18 +1093,26 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
ojscope = NULL;
}
+ /*
+ * If it's a left join with a join clause that is strict for the LHS,
+ * then we need to postpone handling of any non-degenerate join
+ * clauses, in case the join is able to commute with another left join
+ * per identity 3. (Degenerate clauses need not be postponed, since
+ * they will drop down below this join anyway.)
+ */
+ postpone_nondegenerate_clauses = (j->jointype == JOIN_LEFT &&
+ sjinfo->lhs_strict);
+
/* Process the JOIN's qual clauses */
- foreach(l, my_quals)
- {
- Node *qual = (Node *) lfirst(l);
-
- distribute_qual_to_rels(root, qual,
- below_outer_join, j->jointype,
- root->qual_security_level,
- *qualscope,
- ojscope, nonnullable_rels,
- postponed_qual_list);
- }
+ distribute_quals_to_rels(root, my_quals,
+ below_outer_join, sjinfo,
+ root->qual_security_level,
+ *qualscope,
+ ojscope, nonnullable_rels,
+ true, /* allow_equivalence */
+ postpone_nondegenerate_clauses,
+ false, false, /* not clones */
+ postponed_qual_list);
/* Now we can add the SpecialJoinInfo to join_info_list */
if (sjinfo)
@@ -1102,27 +1196,24 @@ process_security_barrier_quals(PlannerInfo *root,
foreach(lc, rte->securityQuals)
{
List *qualset = (List *) lfirst(lc);
- ListCell *lc2;
-
- foreach(lc2, qualset)
- {
- Node *qual = (Node *) lfirst(lc2);
- /*
- * We cheat to the extent of passing ojscope = qualscope rather
- * than its more logical value of NULL. The only effect this has
- * is to force a Var-free qual to be evaluated at the rel rather
- * than being pushed up to top of tree, which we don't want.
- */
- distribute_qual_to_rels(root, qual,
- below_outer_join,
- JOIN_INNER,
- security_level,
- qualscope,
- qualscope,
- NULL,
- NULL);
- }
+ /*
+ * We cheat to the extent of passing ojscope = qualscope rather than
+ * its more logical value of NULL. The only effect this has is to
+ * force a Var-free qual to be evaluated at the rel rather than being
+ * pushed up to top of tree, which we don't want.
+ */
+ distribute_quals_to_rels(root, qualset,
+ below_outer_join,
+ NULL,
+ security_level,
+ qualscope,
+ qualscope,
+ NULL,
+ true,
+ false,
+ false, false, /* not clones */
+ NULL);
security_level++;
}
@@ -1135,10 +1226,11 @@ process_security_barrier_quals(PlannerInfo *root,
* Build a SpecialJoinInfo for the current outer join
*
* Inputs:
- * left_rels: the base Relids syntactically on outer side of join
- * right_rels: the base Relids syntactically on inner side of join
+ * left_rels: the base+OJ Relids syntactically on outer side of join
+ * right_rels: the base+OJ Relids syntactically on inner side of join
* inner_join_rels: base Relids participating in inner joins below this one
* jointype: what it says (must always be LEFT, FULL, SEMI, or ANTI)
+ * ojrelid: RT index of the join RTE (0 for SEMI, which isn't in the RT list)
* clause: the outer join's join condition (in implicit-AND format)
*
* The node should eventually be appended to root->join_info_list, but we
@@ -1152,7 +1244,8 @@ static SpecialJoinInfo *
make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
- JoinType jointype, List *clause)
+ JoinType jointype, Index ojrelid,
+ List *clause)
{
SpecialJoinInfo *sjinfo = makeNode(SpecialJoinInfo);
Relids clause_relids;
@@ -1200,6 +1293,12 @@ make_outerjoininfo(PlannerInfo *root,
sjinfo->syn_lefthand = left_rels;
sjinfo->syn_righthand = right_rels;
sjinfo->jointype = jointype;
+ sjinfo->ojrelid = ojrelid;
+ /* these fields may get added to later: */
+ sjinfo->commute_above_l = NULL;
+ sjinfo->commute_above_r = NULL;
+ sjinfo->commute_below = NULL;
+ sjinfo->oj_joinclause = NIL;
/* this always starts out false */
sjinfo->delay_upper_joins = false;
@@ -1247,6 +1346,7 @@ make_outerjoininfo(PlannerInfo *root,
foreach(l, root->join_info_list)
{
SpecialJoinInfo *otherinfo = (SpecialJoinInfo *) lfirst(l);
+ bool have_unsafe_phvs;
/*
* A full join is an optimization barrier: we can't associate into or
@@ -1262,6 +1362,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_lefthand = bms_add_member(min_lefthand,
+ otherinfo->ojrelid);
}
if (bms_overlap(right_rels, otherinfo->syn_lefthand) ||
bms_overlap(right_rels, otherinfo->syn_righthand))
@@ -1270,11 +1373,26 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_righthand = bms_add_member(min_righthand,
+ otherinfo->ojrelid);
}
/* Needn't do anything else with the full join */
continue;
}
+ /*
+ * If our join condition contains any PlaceHolderVars that need to be
+ * evaluated above the lower OJ, then we can't commute with it.
+ */
+ if (otherinfo->ojrelid != 0)
+ have_unsafe_phvs =
+ contain_placeholder_references_to(root,
+ (Node *) clause,
+ otherinfo->ojrelid);
+ else
+ have_unsafe_phvs = false;
+
/*
* For a lower OJ in our LHS, if our join condition uses the lower
* join's RHS and is not strict for that rel, we must preserve the
@@ -1282,23 +1400,44 @@ make_outerjoininfo(PlannerInfo *root,
* min_lefthand. (We must use its full syntactic relset, not just its
* min_lefthand + min_righthand. This is because there might be other
* OJs below this one that this one can commute with, but we cannot
- * commute with them if we don't with this one.) Also, if the current
- * join is a semijoin or antijoin, we must preserve ordering
- * regardless of strictness.
+ * commute with them if we don't with this one.) Also, if we have
+ * unsafe PHVs or the current join is a semijoin or antijoin, we must
+ * preserve ordering regardless of strictness.
*
* Note: I believe we have to insist on being strict for at least one
* rel in the lower OJ's min_righthand, not its whole syn_righthand.
+ *
+ * When we don't need to preserve ordering, check to see if outer join
+ * identity 3 applies, and if so, remove the lower OJ's ojrelid from
+ * our min_lefthand so that commutation is allowed.
*/
if (bms_overlap(left_rels, otherinfo->syn_righthand))
{
if (bms_overlap(clause_relids, otherinfo->syn_righthand) &&
- (jointype == JOIN_SEMI || jointype == JOIN_ANTI ||
+ (have_unsafe_phvs ||
+ jointype == JOIN_SEMI || jointype == JOIN_ANTI ||
!bms_overlap(strict_relids, otherinfo->min_righthand)))
{
+ /* Preserve ordering */
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_lefthand);
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_lefthand = bms_add_member(min_lefthand,
+ otherinfo->ojrelid);
+ }
+ else if (jointype == JOIN_LEFT &&
+ otherinfo->jointype == JOIN_LEFT &&
+ bms_overlap(strict_relids, otherinfo->min_righthand))
+ {
+ /* Identity 3 applies, so remove the ordering restriction */
+ min_lefthand = bms_del_member(min_lefthand, otherinfo->ojrelid);
+ /* Add commutability markers to both SpecialJoinInfos */
+ otherinfo->commute_above_l =
+ bms_add_member(otherinfo->commute_above_l, ojrelid);
+ sjinfo->commute_below =
+ bms_add_member(sjinfo->commute_below, otherinfo->ojrelid);
}
}
@@ -1313,8 +1452,8 @@ make_outerjoininfo(PlannerInfo *root,
* up with SpecialJoinInfos with identical min_righthands, which can
* confuse join_is_legal (see discussion in backend/optimizer/README).
*
- * Also, we must preserve ordering anyway if either the current join
- * or the lower OJ is either a semijoin or an antijoin.
+ * Also, we must preserve ordering anyway if we have unsafe PHVs, or
+ * if either this join or the lower OJ is a semijoin or antijoin.
*
* Here, we have to consider that "our join condition" includes any
* clauses that syntactically appeared above the lower OJ and below
@@ -1326,21 +1465,43 @@ make_outerjoininfo(PlannerInfo *root,
* join condition are not affected by them. The net effect is
* therefore sufficiently represented by the delay_upper_joins flag
* saved for us by check_outerjoin_delay.
+ *
+ * When we don't need to preserve ordering, check to see if outer join
+ * identity 3 applies, and if so, remove the lower OJ's ojrelid from
+ * our min_righthand so that commutation is allowed.
*/
if (bms_overlap(right_rels, otherinfo->syn_righthand))
{
if (bms_overlap(clause_relids, otherinfo->syn_righthand) ||
!bms_overlap(clause_relids, otherinfo->min_lefthand) ||
+ have_unsafe_phvs ||
jointype == JOIN_SEMI ||
jointype == JOIN_ANTI ||
otherinfo->jointype == JOIN_SEMI ||
otherinfo->jointype == JOIN_ANTI ||
!otherinfo->lhs_strict || otherinfo->delay_upper_joins)
{
+ /* Preserve ordering */
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_lefthand);
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_righthand = bms_add_member(min_righthand,
+ otherinfo->ojrelid);
+ }
+ else if (jointype == JOIN_LEFT &&
+ otherinfo->jointype == JOIN_LEFT &&
+ otherinfo->lhs_strict)
+ {
+ /* Identity 3 applies, so remove the ordering restriction */
+ min_righthand = bms_del_member(min_righthand,
+ otherinfo->ojrelid);
+ /* Add commutability markers to both SpecialJoinInfos */
+ otherinfo->commute_above_r =
+ bms_add_member(otherinfo->commute_above_r, ojrelid);
+ sjinfo->commute_below =
+ bms_add_member(sjinfo->commute_below, otherinfo->ojrelid);
}
}
}
@@ -1565,6 +1726,231 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
sjinfo->semi_rhs_exprs = semi_rhs_exprs;
}
+/*
+ * process_postponed_left_join_quals
+ * Adjust LEFT JOIN quals to be suitable for commuted-left-join cases,
+ * then push them into the joinqual lists and EquivalenceClass structures.
+ *
+ * This runs immediately after we've completed the deconstruct_recurse scan.
+ */
+static void
+process_postponed_left_join_quals(PlannerInfo *root)
+{
+ List *join_info_list_orig = root->join_info_list;
+ ListCell *lc;
+
+ /*
+ * XXX hack: when we call distribute_qual_to_rels to process one of these
+ * quals, neither the owning SpecialJoinInfo nor any later ones can appear
+ * in root->join_info_list, else the wrong things will happen. Fake it
+ * out by emptying join_info_list and rebuilding it as we go. This works
+ * because join_info_list is only appended to during deconstruct_recurse,
+ * so we know we are examining SpecialJoinInfos bottom-up, just like the
+ * first time. Maybe we can get rid of this hack later, if we can fix
+ * things so that distribute_qual_to_rels doesn't consult join_info_list.
+ */
+ root->join_info_list = NIL;
+
+ foreach(lc, join_info_list_orig)
+ {
+ SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+ Relids qualscope,
+ ojscope,
+ nonnullable_rels;
+
+ if (sjinfo->oj_joinclause == NIL) /* nothing to do here */
+ {
+ root->join_info_list = lappend(root->join_info_list, sjinfo);
+ continue;
+ }
+
+ /* Recompute syntactic and semantic scopes of this left join */
+ qualscope = bms_union(sjinfo->syn_lefthand, sjinfo->syn_righthand);
+ qualscope = bms_add_member(qualscope, sjinfo->ojrelid);
+ ojscope = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ nonnullable_rels = sjinfo->syn_lefthand;
+
+ /*
+ * If this join can commute with any other ones per outer-join
+ * identity 3, and it is the one providing the join clause with
+ * flexible semantics, then we have to generate variants of the join
+ * clause with different nullingrels labeling. Otherwise, just push
+ * out the postponed clause as-is.
+ */
+ Assert(sjinfo->lhs_strict); /* else we shouldn't be here */
+ if (sjinfo->commute_above_r ||
+ bms_overlap(sjinfo->commute_below, sjinfo->syn_lefthand))
+ {
+ Relids joins_above;
+ Relids joins_below;
+ Relids joins_so_far;
+ List *quals;
+ ListCell *lc2;
+
+ /*
+ * Put any OJ relids that were removed from min_righthand back
+ * into ojscope, else distribute_qual_to_rels will complain.
+ */
+ ojscope = bms_join(ojscope, bms_intersect(sjinfo->commute_below,
+ sjinfo->syn_righthand));
+
+ /* Identify the outer joins this one commutes with */
+ joins_above = sjinfo->commute_above_r;
+ joins_below = bms_intersect(sjinfo->commute_below,
+ sjinfo->syn_lefthand);
+
+ /*
+ * Generate qual variants with different sets of nullingrels bits.
+ * We only need bit-sets that correspond to the successively less
+ * deeply syntactically-nested subsets of this join and its
+ * commutators. That's true first because obviously only those
+ * forms of the Vars and PHVs could appear elsewhere in the query,
+ * and second because the outer join identities do not provide a
+ * way to re-order such joins in a way that would require
+ * different marking. (That is, while the current join may
+ * commute with several others, none of those others can commute
+ * with each other.) To visit the interesting SpecialJoinInfos in
+ * syntactic nesting order, we rely on the join_info_list to be
+ * ordered that way.
+ *
+ * We first strip out all the nullingrels bits corresponding to
+ * commutating joins below this one, and then successively put
+ * them back as we crawl up the join stack.
+ */
+ quals = sjinfo->oj_joinclause;
+ if (!bms_is_empty(joins_below))
+ quals = (List *) remove_nulling_relids((Node *) quals,
+ joins_below,
+ NULL);
+
+ joins_so_far = NULL;
+ foreach(lc2, join_info_list_orig)
+ {
+ SpecialJoinInfo *othersj = (SpecialJoinInfo *) lfirst(lc2);
+ bool below_sjinfo = false;
+ bool above_sjinfo = false;
+ Relids this_qualscope;
+ Relids this_ojscope;
+ bool allow_equivalence,
+ has_clone,
+ is_clone;
+
+ if (bms_is_member(othersj->ojrelid, joins_below))
+ {
+ /* othersj commutes with sjinfo from below left */
+ below_sjinfo = true;
+ }
+ else if (othersj == sjinfo)
+ {
+ /* found our join in syntactic order */
+ Assert(bms_equal(joins_so_far, joins_below));
+ }
+ else if (bms_is_member(othersj->ojrelid, joins_above))
+ {
+ /* othersj commutes with sjinfo from above */
+ above_sjinfo = true;
+ }
+ else
+ {
+ /* othersj is not relevant, ignore */
+ continue;
+ }
+
+ /*
+ * When we are looking at joins above sjinfo, we are
+ * envisioning pushing sjinfo to above othersj, so add
+ * othersj's nulling bit before distributing the quals.
+ */
+ if (above_sjinfo)
+ quals = (List *)
+ add_nulling_relids((Node *) quals,
+ othersj->min_righthand,
+ bms_make_singleton(othersj->ojrelid));
+
+ /* Compute qualscope and ojscope for this join level */
+ this_qualscope = bms_union(qualscope, joins_so_far);
+ this_ojscope = bms_union(ojscope, joins_so_far);
+ if (above_sjinfo)
+ {
+ /* othersj is not yet in joins_so_far, but we need it */
+ this_qualscope = bms_add_member(this_qualscope,
+ othersj->ojrelid);
+ this_ojscope = bms_add_member(this_ojscope,
+ othersj->ojrelid);
+ /* sjinfo is in joins_so_far, and we don't want it */
+ this_ojscope = bms_del_member(this_ojscope,
+ sjinfo->ojrelid);
+ }
+
+ /*
+ * We generate EquivalenceClasses only from the first form of
+ * the quals, with the fewest nullingrels bits set. An EC
+ * made from this version of the quals can be useful below the
+ * outer-join nest, whereas versions with some nullingrels
+ * bits set would not be. We cannot generate ECs from more
+ * than one version, or we'll make nonsensical conclusions
+ * that Vars with nullingrels bits set are equal to their
+ * versions without. Fortunately, such ECs wouldn't be very
+ * useful anyway, because they'd equate values not observable
+ * outside the join nest. (See optimizer/README.)
+ *
+ * The first form of the quals is also the only one marked as
+ * has_clone rather than is_clone.
+ */
+ allow_equivalence = (joins_so_far == NULL);
+ has_clone = allow_equivalence;
+ is_clone = !has_clone;
+
+ distribute_quals_to_rels(root, quals,
+ false, /* XXX below_outer_join? */
+ sjinfo,
+ root->qual_security_level,
+ this_qualscope,
+ this_ojscope, nonnullable_rels,
+ allow_equivalence,
+ false, /* no more postponement */
+ has_clone,
+ is_clone,
+ NULL);
+
+ /*
+ * Adjust qual nulling bits for next level up, if needed. We
+ * don't want to put sjinfo's own bit in at all, and if we're
+ * above sjinfo then we did it already.
+ */
+ if (below_sjinfo)
+ quals = (List *)
+ add_nulling_relids((Node *) quals,
+ othersj->min_righthand,
+ bms_make_singleton(othersj->ojrelid));
+
+ /* ... and track joins processed so far */
+ joins_so_far = bms_add_member(joins_so_far, othersj->ojrelid);
+ }
+ }
+ else
+ {
+ /* No commutation possible, just process the postponed clauses */
+ distribute_quals_to_rels(root, sjinfo->oj_joinclause,
+ false, /* XXX below_outer_join? */
+ sjinfo,
+ root->qual_security_level,
+ qualscope,
+ ojscope, nonnullable_rels,
+ true, /* allow_equivalence */
+ false, /* no more postponement */
+ false, false, /* not clones */
+ NULL);
+ }
+
+ /* Clear out the list, just so we don't have multiply-linked trees */
+ sjinfo->oj_joinclause = NIL;
+
+ /* Now add sjinfo to the new join_info_list */
+ root->join_info_list = lappend(root->join_info_list, sjinfo);
+ }
+}
+
/*****************************************************************************
*
@@ -1572,6 +1958,46 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
*
*****************************************************************************/
+/*
+ * distribute_quals_to_rels
+ * Convenience routine to apply distribute_qual_to_rels to each element
+ * of an AND'ed list of clauses.
+ */
+static void
+distribute_quals_to_rels(PlannerInfo *root, List *clauses,
+ bool below_outer_join,
+ SpecialJoinInfo *sjinfo,
+ Index security_level,
+ Relids qualscope,
+ Relids ojscope,
+ Relids outerjoin_nonnullable,
+ bool allow_equivalence,
+ bool postpone_nondegenerate_clauses,
+ bool has_clone,
+ bool is_clone,
+ List **postponed_qual_list)
+{
+ ListCell *lc;
+
+ foreach(lc, clauses)
+ {
+ Node *clause = (Node *) lfirst(lc);
+
+ distribute_qual_to_rels(root, clause,
+ below_outer_join,
+ sjinfo,
+ security_level,
+ qualscope,
+ ojscope,
+ outerjoin_nonnullable,
+ allow_equivalence,
+ postpone_nondegenerate_clauses,
+ has_clone,
+ is_clone,
+ postponed_qual_list);
+ }
+}
+
/*
* distribute_qual_to_rels
* Add clause information to either the baserestrictinfo or joininfo list
@@ -1586,7 +2012,7 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
* 'clause': the qual clause to be distributed
* 'below_outer_join': true if the qual is from a JOIN/ON that is below the
* nullable side of a higher-level outer join
- * 'jointype': type of join the qual is from (JOIN_INNER for a WHERE clause)
+ * 'sjinfo': join's SpecialJoinInfo (NULL for an inner join or WHERE clause)
* 'security_level': security_level to assign to the qual
* 'qualscope': set of baserels the qual's syntactic scope covers
* 'ojscope': NULL if not an outer-join qual, else the minimum set of baserels
@@ -1595,6 +2021,12 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
* baserels appearing on the outer (nonnullable) side of the join
* (for FULL JOIN this includes both sides of the join, and must in fact
* equal qualscope)
+ * 'allow_equivalence': true if it's okay to convert clause into an
+ * EquivalenceClass
+ * 'postpone_nondegenerate_clauses': true if non-degenerate outer join clauses
+ * should be added to sjinfo->oj_joinclause instead of being processed
+ * 'has_clone': has_clone property to assign to the qual
+ * 'is_clone': is_clone property to assign to the qual
* 'postponed_qual_list': list of PostponedQual structs, which we can add
* this qual to if it turns out to belong to a higher join level.
* Can be NULL if caller knows postponement is impossible.
@@ -1604,16 +2036,21 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
* level, which will be ojscope not necessarily qualscope.
*
* At the time this is called, root->join_info_list must contain entries for
- * all and only those special joins that are syntactically below this qual.
+ * all and only those special joins that are syntactically below this qual;
+ * in particular, the passed-in SpecialJoinInfo isn't yet in that list.
*/
static void
distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool below_outer_join,
- JoinType jointype,
+ SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
Relids ojscope,
Relids outerjoin_nonnullable,
+ bool allow_equivalence,
+ bool postpone_nondegenerate_clauses,
+ bool has_clone,
+ bool is_clone,
List **postponed_qual_list)
{
Relids relids;
@@ -1646,7 +2083,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
PostponedQual *pq = (PostponedQual *) palloc(sizeof(PostponedQual));
Assert(root->hasLateralRTEs); /* shouldn't happen otherwise */
- Assert(jointype == JOIN_INNER); /* mustn't postpone past outer join */
+ Assert(sjinfo == NULL); /* mustn't postpone past outer join */
pq->qual = clause;
pq->relids = relids;
*postponed_qual_list = lappend(*postponed_qual_list, pq);
@@ -1708,7 +2145,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
{
relids =
get_relids_in_jointree((Node *) root->parse->jointree,
- false);
+ true, false);
qualscope = bms_copy(relids);
}
}
@@ -1751,8 +2188,18 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
{
/*
* The qual is attached to an outer join and mentions (some of the)
- * rels on the nonnullable side, so it's not degenerate.
- *
+ * rels on the nonnullable side, so it's not degenerate. If the
+ * caller wants to postpone handling such clauses, just add it to
+ * sjinfo->oj_joinclause and return. (The work we've done up to here
+ * will have to be redone later, but there's not much of it.)
+ */
+ if (postpone_nondegenerate_clauses)
+ {
+ sjinfo->oj_joinclause = lappend(sjinfo->oj_joinclause, clause);
+ return;
+ }
+
+ /*
* We can't use such a clause to deduce equivalence (the left and
* right sides might be unequal above the join because one of them has
* gone to NULL) ... but we might be able to use it for more limited
@@ -1818,6 +2265,11 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
if (check_redundant_nullability_qual(root, clause))
return;
}
+ else if (!allow_equivalence)
+ {
+ /* Caller says it mustn't become an equivalence class */
+ maybe_equivalence = false;
+ }
else
{
/*
@@ -1852,6 +2304,10 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
outerjoin_nonnullable,
nullable_relids);
+ /* Apply appropriate clone marking, too */
+ restrictinfo->has_clone = has_clone;
+ restrictinfo->is_clone = is_clone;
+
/*
* If it's a join clause (either naturally, or because delayed by
* outer-join rules), add vars used in the clause to targetlists of their
@@ -1950,11 +2406,15 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
restrictinfo);
return;
}
- if (jointype == JOIN_FULL)
+ if (sjinfo && sjinfo->jointype == JOIN_FULL)
{
/* FULL JOIN (above tests cannot match in this case) */
+ FullJoinClauseInfo *fjinfo = makeNode(FullJoinClauseInfo);
+
+ fjinfo->rinfo = restrictinfo;
+ fjinfo->sjinfo = sjinfo;
root->full_join_clauses = lappend(root->full_join_clauses,
- restrictinfo);
+ fjinfo);
return;
}
/* nope, so fall through to distribute_restrictinfo_to_rels */
@@ -2348,7 +2808,7 @@ process_implied_equality(PlannerInfo *root,
{
relids =
get_relids_in_jointree((Node *) root->parse->jointree,
- false);
+ true, false);
}
}
}
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 63deed27c9..69e725d159 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -158,13 +158,15 @@ query_planner(PlannerInfo *root,
/*
* Construct RelOptInfo nodes for all base relations used in the query.
- * Appendrel member relations ("other rels") will be added later.
+ * Appendrel member relations ("other rels") will be added later. We also
+ * construct a bitmapset of all the baserel relids.
*
* Note: the reason we find the baserels by searching the jointree, rather
* than scanning the rangetable, is that the rangetable may contain RTEs
* for rels not actively part of the query, for example views. We don't
* want to make RelOptInfos for them.
*/
+ root->all_baserels = NULL;
add_base_rels_to_query(root, (Node *) parse->jointree);
/*
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 493a3af0fa..e743a5d9fe 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2223,7 +2223,7 @@ preprocess_rowmarks(PlannerInfo *root)
* make a bitmapset of all base rels and then remove the items we don't
* need or have FOR [KEY] UPDATE/SHARE marks for.
*/
- rels = get_relids_in_jointree((Node *) parse->jointree, false);
+ rels = get_relids_in_jointree((Node *) parse->jointree, false, false);
if (parse->resultRelation)
rels = bms_del_member(rels, parse->resultRelation);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 1cb0abdbc1..8fff731756 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -29,11 +29,21 @@
#include "utils/syscache.h"
+typedef enum
+{
+ NRM_EQUAL, /* expect exact match of nullingrels */
+ NRM_SUBSET, /* actual Var may have a subset of input */
+ NRM_SUPERSET /* actual Var may have a superset of input */
+} NullingRelsMatch;
+
typedef struct
{
int varno; /* RT index of Var */
AttrNumber varattno; /* attr number of Var */
AttrNumber resno; /* TLE position of Var */
+#ifdef USE_ASSERT_CHECKING
+ Bitmapset *varnullingrels; /* Var's varnullingrels */
+#endif
} tlist_vinfo;
typedef struct
@@ -59,6 +69,7 @@ typedef struct
indexed_tlist *inner_itlist;
Index acceptable_rel;
int rtoffset;
+ NullingRelsMatch nrm_match;
double num_exec;
} fix_join_expr_context;
@@ -68,6 +79,7 @@ typedef struct
indexed_tlist *subplan_itlist;
int newvarno;
int rtoffset;
+ NullingRelsMatch nrm_match;
double num_exec;
} fix_upper_expr_context;
@@ -150,7 +162,12 @@ static indexed_tlist *build_tlist_index(List *tlist);
static Var *search_indexed_tlist_for_var(Var *var,
indexed_tlist *itlist,
int newvarno,
- int rtoffset);
+ int rtoffset,
+ NullingRelsMatch nrm_match);
+static Var *search_indexed_tlist_for_phv(PlaceHolderVar *phv,
+ indexed_tlist *itlist,
+ int newvarno,
+ NullingRelsMatch nrm_match);
static Var *search_indexed_tlist_for_non_var(Expr *node,
indexed_tlist *itlist,
int newvarno);
@@ -163,14 +180,18 @@ static List *fix_join_expr(PlannerInfo *root,
indexed_tlist *outer_itlist,
indexed_tlist *inner_itlist,
Index acceptable_rel,
- int rtoffset, double num_exec);
+ int rtoffset,
+ NullingRelsMatch nrm_match,
+ double num_exec);
static Node *fix_join_expr_mutator(Node *node,
fix_join_expr_context *context);
static Node *fix_upper_expr(PlannerInfo *root,
Node *node,
indexed_tlist *subplan_itlist,
int newvarno,
- int rtoffset, double num_exec);
+ int rtoffset,
+ NullingRelsMatch nrm_match,
+ double num_exec);
static Node *fix_upper_expr_mutator(Node *node,
fix_upper_expr_context *context);
static List *set_returning_clause_references(PlannerInfo *root,
@@ -1045,13 +1066,13 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
fix_join_expr(root, splan->onConflictSet,
NULL, itlist,
linitial_int(splan->resultRelations),
- rtoffset, NUM_EXEC_QUAL(plan));
+ rtoffset, NRM_EQUAL, NUM_EXEC_QUAL(plan));
splan->onConflictWhere = (Node *)
fix_join_expr(root, (List *) splan->onConflictWhere,
NULL, itlist,
linitial_int(splan->resultRelations),
- rtoffset, NUM_EXEC_QUAL(plan));
+ rtoffset, NRM_EQUAL, NUM_EXEC_QUAL(plan));
pfree(itlist);
@@ -1108,6 +1129,7 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
NULL, itlist,
resultrel,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST(plan));
/* Fix quals too. */
@@ -1116,6 +1138,7 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
NULL, itlist,
resultrel,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL(plan));
}
}
@@ -1261,6 +1284,7 @@ set_indexonlyscan_references(PlannerInfo *root,
index_itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST((Plan *) plan));
plan->scan.plan.qual = (List *)
fix_upper_expr(root,
@@ -1268,6 +1292,7 @@ set_indexonlyscan_references(PlannerInfo *root,
index_itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) plan));
plan->recheckqual = (List *)
fix_upper_expr(root,
@@ -1275,6 +1300,7 @@ set_indexonlyscan_references(PlannerInfo *root,
index_itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) plan));
/* indexqual is already transformed to reference index columns */
plan->indexqual = fix_scan_list(root, plan->indexqual,
@@ -1481,6 +1507,7 @@ set_foreignscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST((Plan *) fscan));
fscan->scan.plan.qual = (List *)
fix_upper_expr(root,
@@ -1488,6 +1515,7 @@ set_foreignscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) fscan));
fscan->fdw_exprs = (List *)
fix_upper_expr(root,
@@ -1495,6 +1523,7 @@ set_foreignscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) fscan));
fscan->fdw_recheck_quals = (List *)
fix_upper_expr(root,
@@ -1502,6 +1531,7 @@ set_foreignscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) fscan));
pfree(itlist);
/* fdw_scan_tlist itself just needs fix_scan_list() adjustments */
@@ -1562,6 +1592,7 @@ set_customscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST((Plan *) cscan));
cscan->scan.plan.qual = (List *)
fix_upper_expr(root,
@@ -1569,6 +1600,7 @@ set_customscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) cscan));
cscan->custom_exprs = (List *)
fix_upper_expr(root,
@@ -1576,6 +1608,7 @@ set_customscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) cscan));
pfree(itlist);
/* custom_scan_tlist itself just needs fix_scan_list() adjustments */
@@ -1780,6 +1813,7 @@ set_hash_references(PlannerInfo *root, Plan *plan, int rtoffset)
outer_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL(plan));
/* Hash doesn't project */
@@ -2115,6 +2149,7 @@ fix_scan_expr_mutator(Node *node, fix_scan_expr_context *context)
/* At scan level, we should always just evaluate the contained expr */
PlaceHolderVar *phv = (PlaceHolderVar *) node;
+ Assert(phv->phnullingrels == NULL);
return fix_scan_expr_mutator((Node *) phv->phexpr, context);
}
if (IsA(node, AlternativeSubPlan))
@@ -2172,6 +2207,7 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
inner_itlist,
(Index) 0,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) join));
/* Now do join-type-specific stuff */
@@ -2184,11 +2220,21 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
{
NestLoopParam *nlp = (NestLoopParam *) lfirst(lc);
+ /*
+ * Because we don't reparameterize parameterized paths to match
+ * the outer-join level at which they are used, Vars seen in the
+ * NestLoopParam expression may have nullingrels that are just a
+ * subset of those in the Vars actually available from the outer
+ * side. Not checking this exactly is a bit grotty, but the work
+ * needed to make things match up perfectly seems well out of
+ * proportion to the value.
+ */
nlp->paramval = (Var *) fix_upper_expr(root,
(Node *) nlp->paramval,
outer_itlist,
OUTER_VAR,
rtoffset,
+ NRM_SUBSET,
NUM_EXEC_TLIST(outer_plan));
/* Check we replaced any PlaceHolderVar with simple Var */
if (!(IsA(nlp->paramval, Var) &&
@@ -2206,6 +2252,7 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
inner_itlist,
(Index) 0,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) join));
}
else if (IsA(join, HashJoin))
@@ -2218,6 +2265,7 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
inner_itlist,
(Index) 0,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) join));
/*
@@ -2229,45 +2277,27 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
outer_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) join));
}
/*
* Now we need to fix up the targetlist and qpqual, which are logically
- * above the join. This means they should not re-use any input expression
- * that was computed in the nullable side of an outer join. Vars and
- * PlaceHolderVars are fine, so we can implement this restriction just by
- * clearing has_non_vars in the indexed_tlist structs.
- *
- * XXX This is a grotty workaround for the fact that we don't clearly
- * distinguish between a Var appearing below an outer join and the "same"
- * Var appearing above it. If we did, we'd not need to hack the matching
- * rules this way.
+ * above the join. This means that, if it's not an inner join, any Vars
+ * and PHVs appearing here should have nullingrels that include the
+ * effects of the outer join, ie they will have nullingrels equal to the
+ * input Vars' nullingrels plus the bit added by the outer join. We don't
+ * currently have enough info available here to identify what that should
+ * be, so we just tell fix_join_expr to accept superset nullingrels
+ * matches instead of exact ones.
*/
- switch (join->jointype)
- {
- case JOIN_LEFT:
- case JOIN_SEMI:
- case JOIN_ANTI:
- inner_itlist->has_non_vars = false;
- break;
- case JOIN_RIGHT:
- outer_itlist->has_non_vars = false;
- break;
- case JOIN_FULL:
- outer_itlist->has_non_vars = false;
- inner_itlist->has_non_vars = false;
- break;
- default:
- break;
- }
-
join->plan.targetlist = fix_join_expr(root,
join->plan.targetlist,
outer_itlist,
inner_itlist,
(Index) 0,
rtoffset,
+ (join->jointype == JOIN_INNER ? NRM_EQUAL : NRM_SUPERSET),
NUM_EXEC_TLIST((Plan *) join));
join->plan.qual = fix_join_expr(root,
join->plan.qual,
@@ -2275,6 +2305,7 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
inner_itlist,
(Index) 0,
rtoffset,
+ (join->jointype == JOIN_INNER ? NRM_EQUAL : NRM_SUPERSET),
NUM_EXEC_QUAL((Plan *) join));
pfree(outer_itlist);
@@ -2329,6 +2360,7 @@ set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset)
subplan_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST(plan));
}
else
@@ -2337,6 +2369,7 @@ set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset)
subplan_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST(plan));
tle = flatCopyTargetEntry(tle);
tle->expr = (Expr *) newexpr;
@@ -2350,6 +2383,7 @@ set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset)
subplan_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL(plan));
pfree(subplan_itlist);
@@ -2550,7 +2584,7 @@ set_dummy_tlist_references(Plan *plan, int rtoffset)
* tlist_member() searches.
*
* The result of this function is an indexed_tlist struct to pass to
- * search_indexed_tlist_for_var() or search_indexed_tlist_for_non_var().
+ * search_indexed_tlist_for_var() and siblings.
* When done, the indexed_tlist may be freed with a single pfree().
*/
static indexed_tlist *
@@ -2582,6 +2616,9 @@ build_tlist_index(List *tlist)
vinfo->varno = var->varno;
vinfo->varattno = var->varattno;
vinfo->resno = tle->resno;
+#ifdef USE_ASSERT_CHECKING
+ vinfo->varnullingrels = var->varnullingrels;
+#endif
vinfo++;
}
else if (tle->expr && IsA(tle->expr, PlaceHolderVar))
@@ -2634,6 +2671,9 @@ build_tlist_index_other_vars(List *tlist, int ignore_rel)
vinfo->varno = var->varno;
vinfo->varattno = var->varattno;
vinfo->resno = tle->resno;
+#ifdef USE_ASSERT_CHECKING
+ vinfo->varnullingrels = var->varnullingrels;
+#endif
vinfo++;
}
}
@@ -2653,10 +2693,17 @@ build_tlist_index_other_vars(List *tlist, int ignore_rel)
* modified varno/varattno (to wit, newvarno and the resno of the TLE entry).
* Also ensure that varnosyn is incremented by rtoffset.
* If no match, return NULL.
+ *
+ * In debugging builds, we cross-check the varnullingrels of the subplan
+ * output Var based on nrm_match. Most call sites should pass NRM_EQUAL
+ * indicating we expect an exact match. However, there are places where
+ * we haven't cleaned things up completely, and we have to settle for
+ * allowing subset or superset matches.
*/
static Var *
search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
- int newvarno, int rtoffset)
+ int newvarno, int rtoffset,
+ NullingRelsMatch nrm_match)
{
int varno = var->varno;
AttrNumber varattno = var->varattno;
@@ -2672,6 +2719,36 @@ search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
/* Found a match */
Var *newvar = copyVar(var);
+ /*
+ * Assert that we kept all the nullingrels machinations straight.
+ *
+ * XXX eventually reduce this to a plain Assert. Right now it's
+ * more useful to warn and keep going.
+ *
+ * XXX skip this check for system columns and whole-row Vars.
+ * That's because such Vars might be row identity Vars, which are
+ * generated without any varnullingrels. It'd be hard to do
+ * otherwise, since they're normally made very early in planning,
+ * when we haven't looked at the jointree yet and don't know which
+ * joins might null such Vars. Doesn't seem worth the expense to
+ * make them fully valid. (While it's slightly annoying that we
+ * thereby lose checking for user-written references to such
+ * columns, it seems unlikely that a bug in nullingrels logic
+ * would affect only system columns.)
+ */
+#ifdef USE_ASSERT_CHECKING
+ if (!(varattno <= 0 ||
+ (nrm_match == NRM_SUBSET ?
+ bms_is_subset(var->varnullingrels, vinfo->varnullingrels) :
+ nrm_match == NRM_SUPERSET ?
+ bms_is_subset(vinfo->varnullingrels, var->varnullingrels) :
+ bms_equal(vinfo->varnullingrels, var->varnullingrels))))
+ elog(WARNING, "bogus varnullingrels for (%d,%d): expected %s, found %s in subplan",
+ varno, varattno,
+ bmsToString(var->varnullingrels),
+ bmsToString(vinfo->varnullingrels));
+#endif
+
newvar->varno = newvarno;
newvar->varattno = vinfo->resno;
if (newvar->varnosyn > 0)
@@ -2684,15 +2761,74 @@ search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
}
/*
- * search_indexed_tlist_for_non_var --- find a non-Var in an indexed tlist
+ * search_indexed_tlist_for_phv --- find a PlaceHolderVar in an indexed tlist
*
* If a match is found, return a Var constructed to reference the tlist item.
* If no match, return NULL.
*
- * NOTE: it is a waste of time to call this unless itlist->has_ph_vars or
- * itlist->has_non_vars. Furthermore, set_join_references() relies on being
- * able to prevent matching of non-Vars by clearing itlist->has_non_vars,
- * so there's a correctness reason not to call it unless that's set.
+ * Cross-check phnullingrels as in search_indexed_tlist_for_var.
+ *
+ * NOTE: it is a waste of time to call this unless itlist->has_ph_vars.
+ */
+static Var *
+search_indexed_tlist_for_phv(PlaceHolderVar *phv,
+ indexed_tlist *itlist, int newvarno,
+ NullingRelsMatch nrm_match)
+{
+ ListCell *lc;
+
+ foreach(lc, itlist->tlist)
+ {
+ TargetEntry *tle = (TargetEntry *) lfirst(lc);
+
+ if (tle->expr && IsA(tle->expr, PlaceHolderVar))
+ {
+ PlaceHolderVar *subphv = (PlaceHolderVar *) tle->expr;
+ Var *newvar;
+
+ /*
+ * Analogously to search_indexed_tlist_for_var, we match on phid
+ * only. We don't use equal(), partially for speed but mostly
+ * because phnullingrels might not be exactly equal.
+ */
+ if (phv->phid != subphv->phid)
+ continue;
+
+ /*
+ * Assert that we kept all the nullingrels machinations straight.
+ *
+ * XXX eventually reduce this to a plain Assert. Right now it's
+ * more useful to warn and keep going.
+ */
+#ifdef USE_ASSERT_CHECKING
+ if (!(nrm_match == NRM_SUBSET ?
+ bms_is_subset(phv->phnullingrels, subphv->phnullingrels) :
+ nrm_match == NRM_SUPERSET ?
+ bms_is_subset(subphv->phnullingrels, phv->phnullingrels) :
+ bms_equal(subphv->phnullingrels, phv->phnullingrels)))
+ elog(WARNING, "bogus phnullingrels for %d: expected %s, found %s in subplan",
+ phv->phid,
+ bmsToString(phv->phnullingrels),
+ bmsToString(subphv->phnullingrels));
+#endif
+
+ /* Found a matching subplan output expression */
+ newvar = makeVarFromTargetEntry(newvarno, tle);
+ newvar->varnosyn = 0; /* wasn't ever a plain Var */
+ newvar->varattnosyn = 0;
+ return newvar;
+ }
+ }
+ return NULL; /* no match */
+}
+
+/*
+ * search_indexed_tlist_for_non_var --- find a non-Var/PHV in an indexed tlist
+ *
+ * If a match is found, return a Var constructed to reference the tlist item.
+ * If no match, return NULL.
+ *
+ * NOTE: it is a waste of time to call this unless itlist->has_non_vars.
*/
static Var *
search_indexed_tlist_for_non_var(Expr *node,
@@ -2799,6 +2935,7 @@ search_indexed_tlist_for_sortgroupref(Expr *node,
* 'acceptable_rel' is either zero or the rangetable index of a relation
* whose Vars may appear in the clause without provoking an error
* 'rtoffset': how much to increment varnos by
+ * 'nrm_match': as for search_indexed_tlist_for_var()
* 'num_exec': estimated number of executions of expression
*
* Returns the new expression tree. The original clause structure is
@@ -2811,6 +2948,7 @@ fix_join_expr(PlannerInfo *root,
indexed_tlist *inner_itlist,
Index acceptable_rel,
int rtoffset,
+ NullingRelsMatch nrm_match,
double num_exec)
{
fix_join_expr_context context;
@@ -2820,6 +2958,7 @@ fix_join_expr(PlannerInfo *root,
context.inner_itlist = inner_itlist;
context.acceptable_rel = acceptable_rel;
context.rtoffset = rtoffset;
+ context.nrm_match = nrm_match;
context.num_exec = num_exec;
return (List *) fix_join_expr_mutator((Node *) clauses, &context);
}
@@ -2841,7 +2980,8 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
newvar = search_indexed_tlist_for_var(var,
context->outer_itlist,
OUTER_VAR,
- context->rtoffset);
+ context->rtoffset,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
@@ -2852,7 +2992,8 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
newvar = search_indexed_tlist_for_var(var,
context->inner_itlist,
INNER_VAR,
- context->rtoffset);
+ context->rtoffset,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
@@ -2877,22 +3018,25 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* See if the PlaceHolderVar has bubbled up from a lower plan node */
if (context->outer_itlist && context->outer_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->outer_itlist,
- OUTER_VAR);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->outer_itlist,
+ OUTER_VAR,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
if (context->inner_itlist && context->inner_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->inner_itlist,
- INNER_VAR);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->inner_itlist,
+ INNER_VAR,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
/* If not supplied by input plans, evaluate the contained expr */
+ /* XXX can we assert something about phnullingrels? */
return fix_join_expr_mutator((Node *) phv->phexpr, context);
}
/* Try matching more complex expressions too, if tlists have any */
@@ -2951,6 +3095,7 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
* 'subplan_itlist': indexed target list for subplan (or index)
* 'newvarno': varno to use for Vars referencing tlist elements
* 'rtoffset': how much to increment varnos by
+ * 'nrm_match': as for search_indexed_tlist_for_var()
* 'num_exec': estimated number of executions of expression
*
* The resulting tree is a copy of the original in which all Var nodes have
@@ -2963,6 +3108,7 @@ fix_upper_expr(PlannerInfo *root,
indexed_tlist *subplan_itlist,
int newvarno,
int rtoffset,
+ NullingRelsMatch nrm_match,
double num_exec)
{
fix_upper_expr_context context;
@@ -2971,6 +3117,7 @@ fix_upper_expr(PlannerInfo *root,
context.subplan_itlist = subplan_itlist;
context.newvarno = newvarno;
context.rtoffset = rtoffset;
+ context.nrm_match = nrm_match;
context.num_exec = num_exec;
return fix_upper_expr_mutator(node, &context);
}
@@ -2989,7 +3136,8 @@ fix_upper_expr_mutator(Node *node, fix_upper_expr_context *context)
newvar = search_indexed_tlist_for_var(var,
context->subplan_itlist,
context->newvarno,
- context->rtoffset);
+ context->rtoffset,
+ context->nrm_match);
if (!newvar)
elog(ERROR, "variable not found in subplan target list");
return (Node *) newvar;
@@ -3001,13 +3149,15 @@ fix_upper_expr_mutator(Node *node, fix_upper_expr_context *context)
/* See if the PlaceHolderVar has bubbled up from a lower plan node */
if (context->subplan_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->subplan_itlist,
- context->newvarno);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->subplan_itlist,
+ context->newvarno,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
/* If not supplied by input plan, evaluate the contained expr */
+ /* XXX can we assert something about phnullingrels? */
return fix_upper_expr_mutator((Node *) phv->phexpr, context);
}
/* Try matching more complex expressions too, if tlist has any */
@@ -3114,6 +3264,7 @@ set_returning_clause_references(PlannerInfo *root,
NULL,
resultRelation,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST(topplan));
pfree(itlist);
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 41c7066d90..4c60a5858e 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -49,17 +49,28 @@ typedef struct pullup_replace_vars_context
* pullup (set only if target_rte->lateral) */
bool *outer_hasSubLinks; /* -> outer query's hasSubLinks */
int varno; /* varno of subquery */
- bool need_phvs; /* do we need PlaceHolderVars? */
- bool wrap_non_vars; /* do we need 'em on *all* non-Vars? */
+ bool wrap_non_vars; /* do we need all non-Var outputs to be PHVs? */
Node **rv_cache; /* cache for results with PHVs */
} pullup_replace_vars_context;
-typedef struct reduce_outer_joins_state
+typedef struct reduce_outer_joins_pass1_state
{
Relids relids; /* base relids within this subtree */
bool contains_outer; /* does subtree contain outer join(s)? */
List *sub_states; /* List of states for subtree components */
-} reduce_outer_joins_state;
+} reduce_outer_joins_pass1_state;
+
+typedef struct reduce_outer_joins_pass2_state
+{
+ Relids inner_reduced; /* OJ relids reduced to plain inner joins */
+ List *partial_reduced; /* List of partially reduced FULL joins */
+} reduce_outer_joins_pass2_state;
+
+typedef struct reduce_outer_joins_partial_state
+{
+ int full_join_rti; /* RT index of a formerly-FULL join */
+ Relids unreduced_side; /* relids in its still-nullable side */
+} reduce_outer_joins_partial_state;
static Node *pull_up_sublinks_jointree_recurse(PlannerInfo *root, Node *jtnode,
Relids *relids);
@@ -68,12 +79,10 @@ static Node *pull_up_sublinks_qual_recurse(PlannerInfo *root, Node *node,
Node **jtlink2, Relids available_rels2);
static Node *pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static Node *pull_up_simple_subquery(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static Node *pull_up_simple_union_all(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte);
@@ -90,7 +99,6 @@ static Node *pull_up_simple_values(PlannerInfo *root, Node *jtnode,
static bool is_simple_values(PlannerInfo *root, RangeTblEntry *rte);
static Node *pull_up_constant_function(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static bool is_simple_union_all(Query *subquery);
static bool is_simple_union_all_recurse(Node *setOp, Query *setOpQuery,
@@ -101,25 +109,27 @@ static bool jointree_contains_lateral_outer_refs(PlannerInfo *root,
Relids safe_upper_varnos);
static void perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars_context *rvcontext,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static void replace_vars_in_jointree(Node *jtnode,
- pullup_replace_vars_context *context,
- JoinExpr *lowest_nulling_outer_join);
+ pullup_replace_vars_context *context);
static Node *pullup_replace_vars(Node *expr,
pullup_replace_vars_context *context);
static Node *pullup_replace_vars_callback(Var *var,
replace_rte_variables_context *context);
static Query *pullup_replace_vars_subquery(Query *query,
pullup_replace_vars_context *context);
-static reduce_outer_joins_state *reduce_outer_joins_pass1(Node *jtnode);
+static reduce_outer_joins_pass1_state *reduce_outer_joins_pass1(Node *jtnode);
static void reduce_outer_joins_pass2(Node *jtnode,
- reduce_outer_joins_state *state,
+ reduce_outer_joins_pass1_state *state1,
+ reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
List *nonnullable_vars,
List *forced_null_vars);
-static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode);
+static void report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
+ int rtindex, Relids relids);
+static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Relids *dropped_outer_joins);
static int get_result_relid(PlannerInfo *root, Node *jtnode);
static void remove_result_refs(PlannerInfo *root, int varno, Node *newjtloc);
static bool find_dependent_phvs(PlannerInfo *root, int varno);
@@ -764,7 +774,7 @@ pull_up_subqueries(PlannerInfo *root)
/* Recursion starts with no containing join nor appendrel */
root->parse->jointree = (FromExpr *)
pull_up_subqueries_recurse(root, (Node *) root->parse->jointree,
- NULL, NULL, NULL);
+ NULL, NULL);
/* We should still have a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
}
@@ -779,12 +789,6 @@ pull_up_subqueries(PlannerInfo *root)
* lowest_outer_join references the lowest such JoinExpr node; otherwise
* it is NULL. We use this to constrain the effects of LATERAL subqueries.
*
- * If this jointree node is within the nullable side of an outer join, then
- * lowest_nulling_outer_join references the lowest such JoinExpr node;
- * otherwise it is NULL. This forces use of the PlaceHolderVar mechanism for
- * references to non-nullable targetlist items, but only for references above
- * that join.
- *
* If we are looking at a member subquery of an append relation,
* containing_appendrel describes that relation; else it is NULL.
* This forces use of the PlaceHolderVar mechanism for all non-Var targetlist
@@ -801,15 +805,14 @@ pull_up_subqueries(PlannerInfo *root)
* Notice also that we can't turn pullup_replace_vars loose on the whole
* jointree, because it'd return a mutated copy of the tree; we have to
* invoke it just on the quals, instead. This behavior is what makes it
- * reasonable to pass lowest_outer_join and lowest_nulling_outer_join as
- * pointers rather than some more-indirect way of identifying the lowest
- * OJs. Likewise, we don't replace append_rel_list members but only their
- * substructure, so the containing_appendrel reference is safe to use.
+ * reasonable to pass lowest_outer_join as a pointer rather than some
+ * more-indirect way of identifying the lowest OJ. Likewise, we don't
+ * replace append_rel_list members but only their substructure, so the
+ * containing_appendrel reference is safe to use.
*/
static Node *
pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Assert(jtnode != NULL);
@@ -831,7 +834,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
is_safe_append_member(rte->subquery)))
return pull_up_simple_subquery(root, jtnode, rte,
lowest_outer_join,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -864,7 +866,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
*/
if (rte->rtekind == RTE_FUNCTION)
return pull_up_constant_function(root, jtnode, rte,
- lowest_nulling_outer_join,
containing_appendrel);
/* Otherwise, do nothing at this node. */
@@ -880,7 +881,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
{
lfirst(l) = pull_up_subqueries_recurse(root, lfirst(l),
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
}
}
@@ -895,11 +895,9 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
case JOIN_INNER:
j->larg = pull_up_subqueries_recurse(root, j->larg,
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
break;
case JOIN_LEFT:
@@ -907,31 +905,25 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
case JOIN_ANTI:
j->larg = pull_up_subqueries_recurse(root, j->larg,
j,
- lowest_nulling_outer_join,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
- j,
j,
NULL);
break;
case JOIN_FULL:
j->larg = pull_up_subqueries_recurse(root, j->larg,
- j,
j,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
- j,
j,
NULL);
break;
case JOIN_RIGHT:
j->larg = pull_up_subqueries_recurse(root, j->larg,
- j,
j,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
j,
- lowest_nulling_outer_join,
NULL);
break;
default:
@@ -961,7 +953,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
static Node *
pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -1108,31 +1099,25 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* The subquery's targetlist items are now in the appropriate form to
* insert into the top query, except that we may need to wrap them in
* PlaceHolderVars. Set up required context data for pullup_replace_vars.
+ * (Note that we should include the subquery's inner joins in relids,
+ * since it may include join alias vars referencing them.)
*/
rvcontext.root = root;
rvcontext.targetlist = subquery->targetList;
rvcontext.target_rte = rte;
if (rte->lateral)
rvcontext.relids = get_relids_in_jointree((Node *) subquery->jointree,
- true);
+ true, true);
else /* won't need relids */
rvcontext.relids = NULL;
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = varno;
- /* these flags will be set below, if needed */
- rvcontext.need_phvs = false;
+ /* this flag will be set below, if needed */
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(subquery->targetList) + 1) *
sizeof(Node *));
- /*
- * If we are under an outer join then non-nullable items and lateral
- * references may have to be turned into PlaceHolderVars.
- */
- if (lowest_nulling_outer_join != NULL)
- rvcontext.need_phvs = true;
-
/*
* If we are dealing with an appendrel member then anything that's not a
* simple Var has to be turned into a PlaceHolderVar. We force this to
@@ -1141,10 +1126,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* expression actually available from the appendrel.
*/
if (containing_appendrel != NULL)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* If the parent query uses grouping sets, we need a PlaceHolderVar for
@@ -1156,10 +1138,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* that pullup_replace_vars hasn't currently got.)
*/
if (parse->groupingSets)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* Replace all of the top query's references to the subquery's outputs
@@ -1167,7 +1146,6 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* replace any of the jointree structure.
*/
perform_pullup_replace_vars(root, &rvcontext,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -1234,7 +1212,8 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
{
Relids subrelids;
- subrelids = get_relids_in_jointree((Node *) subquery->jointree, false);
+ subrelids = get_relids_in_jointree((Node *) subquery->jointree,
+ true, false);
substitute_phv_relids((Node *) parse, varno, subrelids);
fix_append_rel_relids(root->append_rel_list, varno, subrelids);
}
@@ -1425,7 +1404,7 @@ pull_up_union_leaf_queries(Node *setOp, PlannerInfo *root, int parentRTindex,
rtr = makeNode(RangeTblRef);
rtr->rtindex = childRTindex;
(void) pull_up_subqueries_recurse(root, (Node *) rtr,
- NULL, NULL, appinfo);
+ NULL, appinfo);
}
else if (IsA(setOp, SetOperationStmt))
{
@@ -1562,7 +1541,7 @@ is_simple_subquery(PlannerInfo *root, Query *subquery, RangeTblEntry *rte,
{
restricted = true;
safe_upper_varnos = get_relids_in_jointree((Node *) lowest_outer_join,
- true);
+ true, true);
}
else
{
@@ -1674,7 +1653,6 @@ pull_up_simple_values(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte)
rvcontext.relids = NULL;
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = varno;
- rvcontext.need_phvs = false;
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(tlist) + 1) *
@@ -1686,7 +1664,7 @@ pull_up_simple_values(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte)
* any of the jointree structure. We can assume there's no outer joins or
* appendrels in the dummy Query that surrounds a VALUES RTE.
*/
- perform_pullup_replace_vars(root, &rvcontext, NULL, NULL);
+ perform_pullup_replace_vars(root, &rvcontext, NULL);
/*
* There should be no appendrels to fix, nor any outer joins and hence no
@@ -1785,7 +1763,6 @@ is_simple_values(PlannerInfo *root, RangeTblEntry *rte)
static Node *
pull_up_constant_function(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -1837,40 +1814,26 @@ pull_up_constant_function(PlannerInfo *root, Node *jtnode,
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = ((RangeTblRef *) jtnode)->rtindex;
- /* these flags will be set below, if needed */
- rvcontext.need_phvs = false;
+ /* this flag will be set below, if needed */
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(rvcontext.targetlist) + 1) *
sizeof(Node *));
- /*
- * If we are under an outer join then non-nullable items and lateral
- * references may have to be turned into PlaceHolderVars.
- */
- if (lowest_nulling_outer_join != NULL)
- rvcontext.need_phvs = true;
-
/*
* If we are dealing with an appendrel member then anything that's not a
* simple Var has to be turned into a PlaceHolderVar. (See comments in
* pull_up_simple_subquery().)
*/
if (containing_appendrel != NULL)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* If the parent query uses grouping sets, we need a PlaceHolderVar for
* anything that's not a simple Var.
*/
if (parse->groupingSets)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* Replace all of the top query's references to the RTE's output with
@@ -1878,7 +1841,6 @@ pull_up_constant_function(PlannerInfo *root, Node *jtnode,
* jointree structure.
*/
perform_pullup_replace_vars(root, &rvcontext,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -2100,13 +2062,11 @@ jointree_contains_lateral_outer_refs(PlannerInfo *root, Node *jtnode,
*
* Caller has already filled *rvcontext with data describing what to
* substitute for Vars referencing the target subquery. In addition
- * we need the identity of the lowest outer join that can null the
- * target subquery, and its containing appendrel if any.
+ * we need the identity of the containing appendrel if any.
*/
static void
perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars_context *rvcontext,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -2150,38 +2110,31 @@ perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars((Node *) action->targetList, rvcontext);
}
}
- replace_vars_in_jointree((Node *) parse->jointree, rvcontext,
- lowest_nulling_outer_join);
+ replace_vars_in_jointree((Node *) parse->jointree, rvcontext);
Assert(parse->setOperations == NULL);
parse->havingQual = pullup_replace_vars(parse->havingQual, rvcontext);
/*
* Replace references in the translated_vars lists of appendrels. When
- * pulling up an appendrel member, we do not need PHVs in the list of the
- * parent appendrel --- there isn't any outer join between. Elsewhere,
- * use PHVs for safety. (This analysis could be made tighter but it seems
- * unlikely to be worth much trouble.)
+ * pulling up an appendrel member, we do not want to force PHVs in the
+ * list of the parent appendrel --- there isn't any outer join between.
+ * Elsewhere, use PHVs for safety. (This analysis could be made tighter
+ * but it seems unlikely to be worth much trouble.)
*/
foreach(lc, root->append_rel_list)
{
AppendRelInfo *appinfo = (AppendRelInfo *) lfirst(lc);
- bool save_need_phvs = rvcontext->need_phvs;
+ bool save_wrap_non_vars = rvcontext->wrap_non_vars;
if (appinfo == containing_appendrel)
- rvcontext->need_phvs = false;
+ rvcontext->wrap_non_vars = false;
appinfo->translated_vars = (List *)
pullup_replace_vars((Node *) appinfo->translated_vars, rvcontext);
- rvcontext->need_phvs = save_need_phvs;
+ rvcontext->wrap_non_vars = save_wrap_non_vars;
}
/*
* Replace references in the joinaliasvars lists of join RTEs.
- *
- * You might think that we could avoid using PHVs for alias vars of joins
- * below lowest_nulling_outer_join, but that doesn't work because the
- * alias vars could be referenced above that join; we need the PHVs to be
- * present in such references after the alias vars get flattened. (It
- * might be worth trying to be smarter here, someday.)
*/
foreach(lc, parse->rtable)
{
@@ -2198,14 +2151,10 @@ perform_pullup_replace_vars(PlannerInfo *root,
* Helper routine for perform_pullup_replace_vars: do pullup_replace_vars on
* every expression in the jointree, without changing the jointree structure
* itself. Ugly, but there's no other way...
- *
- * If we are at or below lowest_nulling_outer_join, we can suppress use of
- * PlaceHolderVars wrapped around the replacement expressions.
*/
static void
replace_vars_in_jointree(Node *jtnode,
- pullup_replace_vars_context *context,
- JoinExpr *lowest_nulling_outer_join)
+ pullup_replace_vars_context *context)
{
if (jtnode == NULL)
return;
@@ -2215,10 +2164,8 @@ replace_vars_in_jointree(Node *jtnode,
* If the RangeTblRef refers to a LATERAL subquery (that isn't the
* same subquery we're pulling up), it might contain references to the
* target subquery, which we must replace. We drive this from the
- * jointree scan, rather than a scan of the rtable, for a couple of
- * reasons: we can avoid processing no-longer-referenced RTEs, and we
- * can use the appropriate setting of need_phvs depending on whether
- * the RTE is above possibly-nulling outer joins or not.
+ * jointree scan, rather than a scan of the rtable, so that we can
+ * avoid processing no-longer-referenced RTEs.
*/
int varno = ((RangeTblRef *) jtnode)->rtindex;
@@ -2275,42 +2222,30 @@ replace_vars_in_jointree(Node *jtnode,
ListCell *l;
foreach(l, f->fromlist)
- replace_vars_in_jointree(lfirst(l), context,
- lowest_nulling_outer_join);
+ replace_vars_in_jointree(lfirst(l), context);
f->quals = pullup_replace_vars(f->quals, context);
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- bool save_need_phvs = context->need_phvs;
+ bool save_wrap_non_vars = context->wrap_non_vars;
- if (j == lowest_nulling_outer_join)
- {
- /* no more PHVs in or below this join */
- context->need_phvs = false;
- lowest_nulling_outer_join = NULL;
- }
- replace_vars_in_jointree(j->larg, context, lowest_nulling_outer_join);
- replace_vars_in_jointree(j->rarg, context, lowest_nulling_outer_join);
+ replace_vars_in_jointree(j->larg, context);
+ replace_vars_in_jointree(j->rarg, context);
/*
- * Use PHVs within the join quals of a full join, even when it's the
- * lowest nulling outer join. Otherwise, we cannot identify which
- * side of the join a pulled-up var-free expression came from, which
- * can lead to failure to make a plan at all because none of the quals
- * appear to be mergeable or hashable conditions. For this purpose we
- * don't care about the state of wrap_non_vars, so leave it alone.
+ * Use PHVs within the join quals of a full join. Otherwise, we
+ * cannot identify which side of the join a pulled-up var-free
+ * expression came from, which can lead to failure to make a plan at
+ * all because none of the quals appear to be mergeable or hashable
+ * conditions.
*/
if (j->jointype == JOIN_FULL)
- context->need_phvs = true;
+ context->wrap_non_vars = true;
j->quals = pullup_replace_vars(j->quals, context);
- /*
- * We don't bother to update the colvars list, since it won't be used
- * again ...
- */
- context->need_phvs = save_need_phvs;
+ context->wrap_non_vars = save_wrap_non_vars;
}
else
elog(ERROR, "unrecognized node type: %d",
@@ -2339,8 +2274,18 @@ pullup_replace_vars_callback(Var *var,
{
pullup_replace_vars_context *rcon = (pullup_replace_vars_context *) context->callback_arg;
int varattno = var->varattno;
+ bool need_phv;
Node *newnode;
+ /*
+ * We need a PlaceHolderVar if the Var-to-be-replaced has nonempty
+ * varnullingrels (unless we find below that the replacement expression is
+ * a Var or PlaceHolderVar that we can just add the nullingrels to). We
+ * also need one if the caller has instructed us that all non-Var/PHV
+ * replacements need to be wrapped for identification purposes.
+ */
+ need_phv = (var->varnullingrels != NULL) || rcon->wrap_non_vars;
+
/*
* If PlaceHolderVars are needed, we cache the modified expressions in
* rcon->rv_cache[]. This is not in hopes of any material speed gain
@@ -2349,13 +2294,16 @@ pullup_replace_vars_callback(Var *var,
* and possibly prevent optimizations that rely on recognizing different
* references to the same subquery output as being equal(). So it's worth
* a bit of extra effort to avoid it.
+ *
+ * The cached items have phlevelsup = 0 and phnullingrels = NULL; we'll
+ * copy them and adjust those values for this reference site below.
*/
- if (rcon->need_phvs &&
+ if (need_phv &&
varattno >= InvalidAttrNumber &&
varattno <= list_length(rcon->targetlist) &&
rcon->rv_cache[varattno] != NULL)
{
- /* Just copy the entry and fall through to adjust its varlevelsup */
+ /* Just copy the entry and fall through to adjust phlevelsup etc */
newnode = copyObject(rcon->rv_cache[varattno]);
}
else if (varattno == InvalidAttrNumber)
@@ -2364,7 +2312,7 @@ pullup_replace_vars_callback(Var *var,
RowExpr *rowexpr;
List *colnames;
List *fields;
- bool save_need_phvs = rcon->need_phvs;
+ bool save_wrap_non_vars = rcon->wrap_non_vars;
int save_sublevelsup = context->sublevels_up;
/*
@@ -2375,18 +2323,18 @@ pullup_replace_vars_callback(Var *var,
* the RowExpr for use of the executor and ruleutils.c.
*
* In order to be able to cache the results, we always generate the
- * expansion with varlevelsup = 0, and then adjust if needed.
+ * expansion with varlevelsup = 0, and then adjust below if needed.
*/
expandRTE(rcon->target_rte,
var->varno, 0 /* not varlevelsup */ , var->location,
(var->vartype != RECORDOID),
&colnames, &fields);
- /* Adjust the generated per-field Vars, but don't insert PHVs */
- rcon->need_phvs = false;
+ /* Expand the generated per-field Vars, but don't insert PHVs there */
+ rcon->wrap_non_vars = false;
context->sublevels_up = 0; /* to match the expandRTE output */
fields = (List *) replace_rte_variables_mutator((Node *) fields,
context);
- rcon->need_phvs = save_need_phvs;
+ rcon->wrap_non_vars = save_wrap_non_vars;
context->sublevels_up = save_sublevelsup;
rowexpr = makeNode(RowExpr);
@@ -2404,14 +2352,13 @@ pullup_replace_vars_callback(Var *var,
* expression to yield NULL, not ROW(NULL,NULL,...) when it is forced
* to null by an outer join.
*/
- if (rcon->need_phvs)
+ if (need_phv)
{
- /* RowExpr is certainly not strict, so always need PHV */
newnode = (Node *)
make_placeholder_expr(rcon->root,
(Expr *) newnode,
bms_make_singleton(rcon->varno));
- /* cache it with the PHV, and with varlevelsup still zero */
+ /* cache it with the PHV, and with phlevelsup etc not set yet */
rcon->rv_cache[InvalidAttrNumber] = copyObject(newnode);
}
}
@@ -2428,7 +2375,7 @@ pullup_replace_vars_callback(Var *var,
newnode = (Node *) copyObject(tle->expr);
/* Insert PlaceHolderVar if needed */
- if (rcon->need_phvs)
+ if (need_phv)
{
bool wrap;
@@ -2454,69 +2401,61 @@ pullup_replace_vars_callback(Var *var,
/* No need to wrap a PlaceHolderVar with another one, either */
wrap = false;
}
- else if (rcon->wrap_non_vars)
- {
- /* Wrap all non-Vars in a PlaceHolderVar */
- wrap = true;
- }
else
{
/*
- * If it contains a Var of the subquery being pulled up, and
- * does not contain any non-strict constructs, then it's
- * certainly nullable so we don't need to insert a
- * PlaceHolderVar.
- *
- * This analysis could be tighter: in particular, a non-strict
- * construct hidden within a lower-level PlaceHolderVar is not
- * reason to add another PHV. But for now it doesn't seem
- * worth the code to be more exact.
- *
- * Note: in future maybe we should insert a PlaceHolderVar
- * anyway, if the tlist item is expensive to evaluate?
- *
- * For a LATERAL subquery, we have to check the actual var
- * membership of the node, but if it's non-lateral then any
- * level-zero var must belong to the subquery.
+ * Must wrap, either because we need a place to insert
+ * varnullingrels or because caller told us to wrap
+ * everything.
*/
- if ((rcon->target_rte->lateral ?
- bms_overlap(pull_varnos(rcon->root, (Node *) newnode),
- rcon->relids) :
- contain_vars_of_level((Node *) newnode, 0)) &&
- !contain_nonstrict_functions((Node *) newnode))
- {
- /* No wrap needed */
- wrap = false;
- }
- else
- {
- /* Else wrap it in a PlaceHolderVar */
- wrap = true;
- }
+ wrap = true;
}
if (wrap)
+ {
newnode = (Node *)
make_placeholder_expr(rcon->root,
(Expr *) newnode,
bms_make_singleton(rcon->varno));
- /*
- * Cache it if possible (ie, if the attno is in range, which it
- * probably always should be). We can cache the value even if we
- * decided we didn't need a PHV, since this result will be
- * suitable for any request that has need_phvs.
- */
- if (varattno > InvalidAttrNumber &&
- varattno <= list_length(rcon->targetlist))
- rcon->rv_cache[varattno] = copyObject(newnode);
+ /*
+ * Cache it if possible (ie, if the attno is in range, which
+ * it probably always should be).
+ */
+ if (varattno > InvalidAttrNumber &&
+ varattno <= list_length(rcon->targetlist))
+ rcon->rv_cache[varattno] = copyObject(newnode);
+ }
}
}
- /* Must adjust varlevelsup if tlist item is from higher query */
+ /* Must adjust varlevelsup if replaced Var is within a subquery */
if (var->varlevelsup > 0)
IncrementVarSublevelsUp(newnode, var->varlevelsup, 0);
+ /* Propagate any varnullingrels into the replacement Var or PHV */
+ if (var->varnullingrels != NULL)
+ {
+ if (IsA(newnode, Var))
+ {
+ Var *newvar = (Var *) newnode;
+
+ Assert(newvar->varlevelsup == var->varlevelsup);
+ newvar->varnullingrels = bms_add_members(newvar->varnullingrels,
+ var->varnullingrels);
+ }
+ else if (IsA(newnode, PlaceHolderVar))
+ {
+ PlaceHolderVar *newphv = (PlaceHolderVar *) newnode;
+
+ Assert(newphv->phlevelsup == var->varlevelsup);
+ newphv->phnullingrels = bms_add_members(newphv->phnullingrels,
+ var->varnullingrels);
+ }
+ else
+ elog(ERROR, "failed to wrap a non-Var");
+ }
+
return newnode;
}
@@ -2675,7 +2614,9 @@ flatten_simple_union_all(PlannerInfo *root)
void
reduce_outer_joins(PlannerInfo *root)
{
- reduce_outer_joins_state *state;
+ reduce_outer_joins_pass1_state *state1;
+ reduce_outer_joins_pass2_state state2;
+ ListCell *lc;
/*
* To avoid doing strictness checks on more quals than necessary, we want
@@ -2686,14 +2627,56 @@ reduce_outer_joins(PlannerInfo *root)
* join(s) below each side of each join clause. The second pass examines
* qual clauses and changes join types as it descends the tree.
*/
- state = reduce_outer_joins_pass1((Node *) root->parse->jointree);
+ state1 = reduce_outer_joins_pass1((Node *) root->parse->jointree);
/* planner.c shouldn't have called me if no outer joins */
- if (state == NULL || !state->contains_outer)
+ if (state1 == NULL || !state1->contains_outer)
elog(ERROR, "so where are the outer joins?");
+ state2.inner_reduced = NULL;
+ state2.partial_reduced = NIL;
+
reduce_outer_joins_pass2((Node *) root->parse->jointree,
- state, root, NULL, NIL, NIL);
+ state1, &state2,
+ root, NULL, NIL, NIL);
+
+ /*
+ * If we successfully reduced the strength of any outer joins, we must
+ * remove references to those joins as nulling rels. This is handled as
+ * an additional pass, for simplicity and because we can handle all
+ * fully-reduced joins in a single pass over the parse tree.
+ */
+ if (!bms_is_empty(state2.inner_reduced))
+ {
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ state2.inner_reduced,
+ NULL);
+ /* There could be references in the append_rel_list, too */
+ root->append_rel_list = (List *)
+ remove_nulling_relids((Node *) root->append_rel_list,
+ state2.inner_reduced,
+ NULL);
+ }
+
+ /*
+ * Partially-reduced full joins have to be done one at a time, since
+ * they'll each need a different setting of except_relids.
+ */
+ foreach(lc, state2.partial_reduced)
+ {
+ reduce_outer_joins_partial_state *statep = lfirst(lc);
+ Relids full_join_relids = bms_make_singleton(statep->full_join_rti);
+
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ full_join_relids,
+ statep->unreduced_side);
+ root->append_rel_list = (List *)
+ remove_nulling_relids((Node *) root->append_rel_list,
+ full_join_relids,
+ statep->unreduced_side);
+ }
}
/*
@@ -2701,13 +2684,13 @@ reduce_outer_joins(PlannerInfo *root)
*
* Returns a state node describing the given jointree node.
*/
-static reduce_outer_joins_state *
+static reduce_outer_joins_pass1_state *
reduce_outer_joins_pass1(Node *jtnode)
{
- reduce_outer_joins_state *result;
+ reduce_outer_joins_pass1_state *result;
- result = (reduce_outer_joins_state *)
- palloc(sizeof(reduce_outer_joins_state));
+ result = (reduce_outer_joins_pass1_state *)
+ palloc(sizeof(reduce_outer_joins_pass1_state));
result->relids = NULL;
result->contains_outer = false;
result->sub_states = NIL;
@@ -2727,7 +2710,7 @@ reduce_outer_joins_pass1(Node *jtnode)
foreach(l, f->fromlist)
{
- reduce_outer_joins_state *sub_state;
+ reduce_outer_joins_pass1_state *sub_state;
sub_state = reduce_outer_joins_pass1(lfirst(l));
result->relids = bms_add_members(result->relids,
@@ -2739,7 +2722,7 @@ reduce_outer_joins_pass1(Node *jtnode)
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- reduce_outer_joins_state *sub_state;
+ reduce_outer_joins_pass1_state *sub_state;
/* join's own RT index is not wanted in result->relids */
if (IS_OUTER_JOIN(j->jointype))
@@ -2767,15 +2750,23 @@ reduce_outer_joins_pass1(Node *jtnode)
* reduce_outer_joins_pass2 - phase 2 processing
*
* jtnode: current jointree node
- * state: state data collected by phase 1 for this node
+ * state1: state data collected by phase 1 for this node
+ * state2: where to accumulate info about successfully-reduced joins
* root: toplevel planner state
* nonnullable_rels: set of base relids forced non-null by upper quals
* nonnullable_vars: list of Vars forced non-null by upper quals
* forced_null_vars: list of Vars forced null by upper quals
+ *
+ * Returns info in state2 about outer joins that were successfully simplified.
+ * Joins that were fully reduced to inner joins are all added to
+ * state2->inner_reduced. If a full join is reduced to a left join,
+ * it needs its own entry in state2->partial_reduced, since that will
+ * require custom processing to remove only the correct nullingrel markers.
*/
static void
reduce_outer_joins_pass2(Node *jtnode,
- reduce_outer_joins_state *state,
+ reduce_outer_joins_pass1_state *state1,
+ reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
List *nonnullable_vars,
@@ -2809,13 +2800,14 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_forced_null_vars = list_concat(pass_forced_null_vars,
forced_null_vars);
/* And recurse --- but only into interesting subtrees */
- Assert(list_length(f->fromlist) == list_length(state->sub_states));
- forboth(l, f->fromlist, s, state->sub_states)
+ Assert(list_length(f->fromlist) == list_length(state1->sub_states));
+ forboth(l, f->fromlist, s, state1->sub_states)
{
- reduce_outer_joins_state *sub_state = lfirst(s);
+ reduce_outer_joins_pass1_state *sub_state = lfirst(s);
if (sub_state->contains_outer)
- reduce_outer_joins_pass2(lfirst(l), sub_state, root,
+ reduce_outer_joins_pass2(lfirst(l), sub_state,
+ state2, root,
pass_nonnullable_rels,
pass_nonnullable_vars,
pass_forced_null_vars);
@@ -2828,8 +2820,8 @@ reduce_outer_joins_pass2(Node *jtnode,
JoinExpr *j = (JoinExpr *) jtnode;
int rtindex = j->rtindex;
JoinType jointype = j->jointype;
- reduce_outer_joins_state *left_state = linitial(state->sub_states);
- reduce_outer_joins_state *right_state = lsecond(state->sub_states);
+ reduce_outer_joins_pass1_state *left_state = linitial(state1->sub_states);
+ reduce_outer_joins_pass1_state *right_state = lsecond(state1->sub_states);
List *local_nonnullable_vars = NIL;
bool computed_local_nonnullable_vars = false;
@@ -2852,12 +2844,22 @@ reduce_outer_joins_pass2(Node *jtnode,
if (bms_overlap(nonnullable_rels, right_state->relids))
jointype = JOIN_INNER;
else
+ {
jointype = JOIN_LEFT;
+ /* Also report partial reduction in state2 */
+ report_reduced_full_join(state2, rtindex,
+ right_state->relids);
+ }
}
else
{
if (bms_overlap(nonnullable_rels, right_state->relids))
+ {
jointype = JOIN_RIGHT;
+ /* Also report partial reduction in state2 */
+ report_reduced_full_join(state2, rtindex,
+ left_state->relids);
+ }
}
break;
case JOIN_SEMI:
@@ -2890,8 +2892,8 @@ reduce_outer_joins_pass2(Node *jtnode,
j->larg = j->rarg;
j->rarg = tmparg;
jointype = JOIN_LEFT;
- right_state = linitial(state->sub_states);
- left_state = lsecond(state->sub_states);
+ right_state = linitial(state1->sub_states);
+ left_state = lsecond(state1->sub_states);
}
/*
@@ -2924,7 +2926,10 @@ reduce_outer_joins_pass2(Node *jtnode,
jointype = JOIN_ANTI;
}
- /* Apply the jointype change, if any, to both jointree node and RTE */
+ /*
+ * Apply the jointype change, if any, to both jointree node and RTE.
+ * Also, if we changed an RTE to INNER, add its RTI to inner_reduced.
+ */
if (rtindex && jointype != j->jointype)
{
RangeTblEntry *rte = rt_fetch(rtindex, root->parse->rtable);
@@ -2932,6 +2937,9 @@ reduce_outer_joins_pass2(Node *jtnode,
Assert(rte->rtekind == RTE_JOIN);
Assert(rte->jointype == j->jointype);
rte->jointype = jointype;
+ if (jointype == JOIN_INNER)
+ state2->inner_reduced = bms_add_member(state2->inner_reduced,
+ rtindex);
}
j->jointype = jointype;
@@ -3012,7 +3020,8 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_nonnullable_vars = NIL;
pass_forced_null_vars = NIL;
}
- reduce_outer_joins_pass2(j->larg, left_state, root,
+ reduce_outer_joins_pass2(j->larg, left_state,
+ state2, root,
pass_nonnullable_rels,
pass_nonnullable_vars,
pass_forced_null_vars);
@@ -3034,7 +3043,8 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_nonnullable_vars = NIL;
pass_forced_null_vars = NIL;
}
- reduce_outer_joins_pass2(j->rarg, right_state, root,
+ reduce_outer_joins_pass2(j->rarg, right_state,
+ state2, root,
pass_nonnullable_rels,
pass_nonnullable_vars,
pass_forced_null_vars);
@@ -3047,6 +3057,19 @@ reduce_outer_joins_pass2(Node *jtnode,
(int) nodeTag(jtnode));
}
+/* Helper for reduce_outer_joins_pass2 */
+static void
+report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
+ int rtindex, Relids relids)
+{
+ reduce_outer_joins_partial_state *statep;
+
+ statep = palloc(sizeof(reduce_outer_joins_partial_state));
+ statep->full_join_rti = rtindex;
+ statep->unreduced_side = relids;
+ state2->partial_reduced = lappend(state2->partial_reduced, statep);
+}
+
/*
* remove_useless_result_rtes
@@ -3088,16 +3111,41 @@ reduce_outer_joins_pass2(Node *jtnode,
void
remove_useless_result_rtes(PlannerInfo *root)
{
+ Relids dropped_outer_joins = NULL;
ListCell *cell;
/* Top level of jointree must always be a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
/* Recurse ... */
root->parse->jointree = (FromExpr *)
- remove_useless_results_recurse(root, (Node *) root->parse->jointree);
+ remove_useless_results_recurse(root,
+ (Node *) root->parse->jointree,
+ &dropped_outer_joins);
/* We should still have a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
+ /*
+ * If we removed any outer-join nodes from the jointree, run around and
+ * remove references to those joins as nulling rels. (There could be such
+ * references in PHVs that we pulled up out of the original subquery that
+ * the RESULT rel replaced. This is kosher on the grounds that we now
+ * know that such an outer join wouldn't really have nulled anything.) We
+ * don't do this during the main recursion, for simplicity and because we
+ * can handle all such joins in a single pass over the parse tree.
+ */
+ if (!bms_is_empty(dropped_outer_joins))
+ {
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ dropped_outer_joins,
+ NULL);
+ /* There could be references in the append_rel_list, too */
+ root->append_rel_list = (List *)
+ remove_nulling_relids((Node *) root->append_rel_list,
+ dropped_outer_joins,
+ NULL);
+ }
+
/*
* Remove any PlanRowMark referencing an RTE_RESULT RTE. We obviously
* must do that for any RTE_RESULT that we just removed. But one for a
@@ -3123,9 +3171,12 @@ remove_useless_result_rtes(PlannerInfo *root)
* Recursive guts of remove_useless_result_rtes.
*
* This recursively processes the jointree and returns a modified jointree.
+ * In addition, the RT indexes of any removed outer-join nodes are added to
+ * *dropped_outer_joins.
*/
static Node *
-remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
+remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Relids *dropped_outer_joins)
{
Assert(jtnode != NULL);
if (IsA(jtnode, RangeTblRef))
@@ -3153,7 +3204,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
int varno;
/* Recursively transform child ... */
- child = remove_useless_results_recurse(root, child);
+ child = remove_useless_results_recurse(root, child,
+ dropped_outer_joins);
/* ... and stick it back into the tree */
lfirst(cell) = child;
@@ -3202,8 +3254,10 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
int varno;
/* First, recurse */
- j->larg = remove_useless_results_recurse(root, j->larg);
- j->rarg = remove_useless_results_recurse(root, j->rarg);
+ j->larg = remove_useless_results_recurse(root, j->larg,
+ dropped_outer_joins);
+ j->rarg = remove_useless_results_recurse(root, j->rarg,
+ dropped_outer_joins);
/* Apply join-type-specific optimization rules */
switch (j->jointype)
@@ -3271,6 +3325,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
!find_dependent_phvs(root, varno)))
{
remove_result_refs(root, varno, j->larg);
+ *dropped_outer_joins = bms_add_member(*dropped_outer_joins,
+ j->rtindex);
jtnode = j->larg;
}
break;
@@ -3281,6 +3337,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
!find_dependent_phvs(root, varno)))
{
remove_result_refs(root, varno, j->rarg);
+ *dropped_outer_joins = bms_add_member(*dropped_outer_joins,
+ j->rtindex);
jtnode = j->rarg;
}
break;
@@ -3295,11 +3353,14 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
* Unlike the LEFT/RIGHT cases, we just Assert that there are
* no PHVs that need to be evaluated at the semijoin's RHS,
* since the rest of the query couldn't reference any outputs
- * of the semijoin's RHS.
+ * of the semijoin's RHS. Also, we don't need to worry about
+ * removing traces of the join's rtindex, since it hasn't got
+ * one.
*/
if ((varno = get_result_relid(root, j->rarg)) != 0)
{
Assert(!find_dependent_phvs(root, varno));
+ Assert(j->rtindex == 0);
remove_result_refs(root, varno, j->larg);
if (j->quals)
jtnode = (Node *)
@@ -3368,7 +3429,7 @@ remove_result_refs(PlannerInfo *root, int varno, Node *newjtloc)
{
Relids subrelids;
- subrelids = get_relids_in_jointree(newjtloc, false);
+ subrelids = get_relids_in_jointree(newjtloc, true, false);
Assert(!bms_is_empty(subrelids));
substitute_phv_relids((Node *) root->parse, varno, subrelids);
fix_append_rel_relids(root->append_rel_list, varno, subrelids);
@@ -3425,9 +3486,8 @@ find_dependent_phvs_walker(Node *node,
context->sublevels_up--;
return result;
}
- /* Shouldn't need to handle planner auxiliary nodes here */
+ /* Shouldn't need to handle most planner auxiliary nodes here */
Assert(!IsA(node, SpecialJoinInfo));
- Assert(!IsA(node, AppendRelInfo));
Assert(!IsA(node, PlaceHolderInfo));
Assert(!IsA(node, MinMaxAggInfo));
@@ -3447,10 +3507,17 @@ find_dependent_phvs(PlannerInfo *root, int varno)
context.relids = bms_make_singleton(varno);
context.sublevels_up = 0;
- return query_tree_walker(root->parse,
- find_dependent_phvs_walker,
- (void *) &context,
- 0);
+ if (query_tree_walker(root->parse,
+ find_dependent_phvs_walker,
+ (void *) &context,
+ 0))
+ return true;
+ /* The append_rel_list could be populated already, so check it too */
+ if (expression_tree_walker((Node *) root->append_rel_list,
+ find_dependent_phvs_walker,
+ (void *) &context))
+ return true;
+ return false;
}
static bool
@@ -3480,7 +3547,7 @@ find_dependent_phvs_in_jointree(PlannerInfo *root, Node *node, int varno)
* are not marked LATERAL, though, since they couldn't possibly contain
* any cross-references to other RTEs.
*/
- subrelids = get_relids_in_jointree(node, false);
+ subrelids = get_relids_in_jointree(node, false, false);
relid = -1;
while ((relid = bms_next_member(subrelids, relid)) >= 0)
{
@@ -3624,11 +3691,17 @@ fix_append_rel_relids(List *append_rel_list, int varno, Relids subrelids)
/*
* get_relids_in_jointree: get set of RT indexes present in a jointree
*
- * If include_joins is true, join RT indexes are included; if false,
- * only base rels are included.
+ * Base-relation relids are always included in the result.
+ * If include_outer_joins is true, outer-join RT indexes are included.
+ * If include_inner_joins is true, inner-join RT indexes are included.
+ *
+ * Note that for most purposes in the planner, outer joins are included
+ * in standard relid sets. Setting include_inner_joins true is only
+ * appropriate for special purposes during subquery flattening.
*/
Relids
-get_relids_in_jointree(Node *jtnode, bool include_joins)
+get_relids_in_jointree(Node *jtnode, bool include_outer_joins,
+ bool include_inner_joins)
{
Relids result = NULL;
@@ -3649,18 +3722,34 @@ get_relids_in_jointree(Node *jtnode, bool include_joins)
{
result = bms_join(result,
get_relids_in_jointree(lfirst(l),
- include_joins));
+ include_outer_joins,
+ include_inner_joins));
}
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- result = get_relids_in_jointree(j->larg, include_joins);
+ result = get_relids_in_jointree(j->larg,
+ include_outer_joins,
+ include_inner_joins);
result = bms_join(result,
- get_relids_in_jointree(j->rarg, include_joins));
- if (include_joins && j->rtindex)
- result = bms_add_member(result, j->rtindex);
+ get_relids_in_jointree(j->rarg,
+ include_outer_joins,
+ include_inner_joins));
+ if (j->rtindex)
+ {
+ if (j->jointype == JOIN_INNER)
+ {
+ if (include_inner_joins)
+ result = bms_add_member(result, j->rtindex);
+ }
+ else
+ {
+ if (include_outer_joins)
+ result = bms_add_member(result, j->rtindex);
+ }
+ }
}
else
elog(ERROR, "unrecognized node type: %d",
@@ -3669,7 +3758,7 @@ get_relids_in_jointree(Node *jtnode, bool include_joins)
}
/*
- * get_relids_for_join: get set of base RT indexes making up a join
+ * get_relids_for_join: get set of base+OJ RT indexes making up a join
*/
Relids
get_relids_for_join(Query *query, int joinrelid)
@@ -3680,7 +3769,7 @@ get_relids_for_join(Query *query, int joinrelid)
joinrelid);
if (!jtnode)
elog(ERROR, "could not find join node %d", joinrelid);
- return get_relids_in_jointree(jtnode, false);
+ return get_relids_in_jointree(jtnode, true, false);
}
/*
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index f6fc62aa5d..11c6bbaba6 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -228,6 +228,14 @@ adjust_appendrel_attrs_mutator(Node *node,
if (var->varlevelsup != 0)
return (Node *) var; /* no changes needed */
+ /*
+ * You might think we need to adjust var->varnullingrels, but that
+ * shouldn't need any changes. It will contain outer-join relids,
+ * while the transformation we are making affects only baserels.
+ * Below, we just propagate var->varnullingrels into the translated
+ * Var. (XXX what to do if translation is not a Var??)
+ */
+
for (cnt = 0; cnt < nappinfos; cnt++)
{
if (var->varno == appinfos[cnt]->parent_relid)
@@ -255,6 +263,8 @@ adjust_appendrel_attrs_mutator(Node *node,
if (newnode == NULL)
elog(ERROR, "attribute %d of relation \"%s\" does not exist",
var->varattno, get_rel_name(appinfo->parent_reloid));
+ if (IsA(newnode, Var))
+ ((Var *) newnode)->varnullingrels = var->varnullingrels;
return newnode;
}
else if (var->varattno == 0)
@@ -348,6 +358,8 @@ adjust_appendrel_attrs_mutator(Node *node,
var = copyObject(ridinfo->rowidvar);
/* ... but use the correct relid */
var->varno = leaf_relid;
+ /* identity vars shouldn't have nulling rels */
+ Assert(var->varnullingrels == NULL);
/* varnosyn in the RowIdentityVarInfo is probably wrong */
var->varnosyn = 0;
var->varattnosyn = 0;
@@ -392,8 +404,11 @@ adjust_appendrel_attrs_mutator(Node *node,
(void *) context);
/* now fix PlaceHolderVar's relid sets */
if (phv->phlevelsup == 0)
- phv->phrels = adjust_child_relids(phv->phrels, context->nappinfos,
- context->appinfos);
+ {
+ phv->phrels = adjust_child_relids(phv->phrels,
+ nappinfos, appinfos);
+ /* as above, we needn't touch phnullingrels */
+ }
return (Node *) phv;
}
/* Shouldn't need to handle planner auxiliary nodes here */
@@ -688,7 +703,11 @@ get_translated_update_targetlist(PlannerInfo *root, Index relid,
/*
* find_appinfos_by_relids
- * Find AppendRelInfo structures for all relations specified by relids.
+ * Find AppendRelInfo structures for base relations listed in relids.
+ *
+ * The relids argument is typically a join relation's relids, which can
+ * include outer-join RT indexes in addition to baserels. We silently
+ * ignore the outer joins.
*
* The AppendRelInfos are returned in an array, which can be pfree'd by the
* caller. *nappinfos is set to the number of entries in the array.
@@ -700,8 +719,9 @@ find_appinfos_by_relids(PlannerInfo *root, Relids relids, int *nappinfos)
int cnt = 0;
int i;
- *nappinfos = bms_num_members(relids);
- appinfos = (AppendRelInfo **) palloc(sizeof(AppendRelInfo *) * *nappinfos);
+ /* Allocate an array that's certainly big enough */
+ appinfos = (AppendRelInfo **)
+ palloc(sizeof(AppendRelInfo *) * bms_num_members(relids));
i = -1;
while ((i = bms_next_member(relids, i)) >= 0)
@@ -709,10 +729,17 @@ find_appinfos_by_relids(PlannerInfo *root, Relids relids, int *nappinfos)
AppendRelInfo *appinfo = root->append_rel_array[i];
if (!appinfo)
+ {
+ /* Probably i is an OJ index, but let's check */
+ if (find_base_rel_ignore_join(root, i) == NULL)
+ continue;
+ /* It's a base rel, but we lack an append_rel_array entry */
elog(ERROR, "child rel %d not found in append_rel_array", i);
+ }
appinfos[cnt++] = appinfo;
}
+ *nappinfos = cnt;
return appinfos;
}
@@ -754,6 +781,7 @@ add_row_identity_var(PlannerInfo *root, Var *orig_var,
Assert(IsA(orig_var, Var));
Assert(orig_var->varno == rtindex);
Assert(orig_var->varlevelsup == 0);
+ Assert(orig_var->varnullingrels == NULL);
/*
* If we're doing non-inherited UPDATE/DELETE/MERGE, there's little need
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 7fb32a0710..f9913ce3b5 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -1967,14 +1967,16 @@ is_pseudo_constant_clause_relids(Node *clause, Relids relids)
* NumRelids
* (formerly clause_relids)
*
- * Returns the number of different relations referenced in 'clause'.
+ * Returns the number of different base relations referenced in 'clause'.
*/
int
NumRelids(PlannerInfo *root, Node *clause)
{
+ int result;
Relids varnos = pull_varnos(root, clause);
- int result = bms_num_members(varnos);
+ varnos = bms_del_members(varnos, root->outer_join_rels);
+ result = bms_num_members(varnos);
bms_free(varnos);
return result;
}
diff --git a/src/backend/optimizer/util/joininfo.c b/src/backend/optimizer/util/joininfo.c
index d4cffdb198..afd243f5d8 100644
--- a/src/backend/optimizer/util/joininfo.c
+++ b/src/backend/optimizer/util/joininfo.c
@@ -88,8 +88,8 @@ have_relevant_joinclause(PlannerInfo *root,
* not depend on context).
*
* 'restrictinfo' describes the join clause
- * 'join_relids' is the list of relations participating in the join clause
- * (there must be more than one)
+ * 'join_relids' is the set of relations participating in the join clause
+ * (some of these could be outer joins)
*/
void
add_join_clause_to_rels(PlannerInfo *root,
@@ -101,8 +101,11 @@ add_join_clause_to_rels(PlannerInfo *root,
cur_relid = -1;
while ((cur_relid = bms_next_member(join_relids, cur_relid)) >= 0)
{
- RelOptInfo *rel = find_base_rel(root, cur_relid);
+ RelOptInfo *rel = find_base_rel_ignore_join(root, cur_relid);
+ /* We only need to add the clause to baserels */
+ if (rel == NULL)
+ continue;
rel->joininfo = lappend(rel->joininfo, restrictinfo);
}
}
@@ -115,8 +118,8 @@ add_join_clause_to_rels(PlannerInfo *root,
* discover that a relation need not be joined at all.
*
* 'restrictinfo' describes the join clause
- * 'join_relids' is the list of relations participating in the join clause
- * (there must be more than one)
+ * 'join_relids' is the set of relations participating in the join clause
+ * (some of these could be outer joins)
*/
void
remove_join_clause_from_rels(PlannerInfo *root,
@@ -128,7 +131,11 @@ remove_join_clause_from_rels(PlannerInfo *root,
cur_relid = -1;
while ((cur_relid = bms_next_member(join_relids, cur_relid)) >= 0)
{
- RelOptInfo *rel = find_base_rel(root, cur_relid);
+ RelOptInfo *rel = find_base_rel_ignore_join(root, cur_relid);
+
+ /* We would only have added the clause to baserels */
+ if (rel == NULL)
+ continue;
/*
* Remove the restrictinfo from the list. Pointer comparison is
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index b1363df065..9cfde2f790 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -338,6 +338,11 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
sjinfo.syn_lefthand = sjinfo.min_lefthand;
sjinfo.syn_righthand = sjinfo.min_righthand;
sjinfo.jointype = JOIN_INNER;
+ sjinfo.ojrelid = 0;
+ sjinfo.commute_above_l = NULL;
+ sjinfo.commute_above_r = NULL;
+ sjinfo.commute_below = NULL;
+ sjinfo.oj_joinclause = NIL;
/* we don't bother trying to make the remaining fields valid */
sjinfo.lhs_strict = false;
sjinfo.delay_upper_joins = false;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 6dd11329fb..bf35d1989c 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -1307,7 +1307,7 @@ create_append_path(PlannerInfo *root,
* Apply query-wide LIMIT if known and path is for sole base relation.
* (Handling this at this low level is a bit klugy.)
*/
- if (root != NULL && bms_equal(rel->relids, root->all_baserels))
+ if (root != NULL && bms_equal(rel->relids, root->all_query_rels))
pathnode->limit_tuples = root->limit_tuples;
else
pathnode->limit_tuples = -1.0;
@@ -1436,7 +1436,7 @@ create_merge_append_path(PlannerInfo *root,
* Apply query-wide LIMIT if known and path is for sole base relation.
* (Handling this at this low level is a bit klugy.)
*/
- if (bms_equal(rel->relids, root->all_baserels))
+ if (bms_equal(rel->relids, root->all_query_rels))
pathnode->limit_tuples = root->limit_tuples;
else
pathnode->limit_tuples = -1.0;
diff --git a/src/backend/optimizer/util/placeholder.c b/src/backend/optimizer/util/placeholder.c
index c7bfa293c9..bbc39ac3c5 100644
--- a/src/backend/optimizer/util/placeholder.c
+++ b/src/backend/optimizer/util/placeholder.c
@@ -23,17 +23,32 @@
#include "optimizer/planmain.h"
#include "utils/lsyscache.h"
+
+typedef struct contain_placeholder_references_context
+{
+ int relid;
+ int sublevels_up;
+} contain_placeholder_references_context;
+
/* Local functions */
static void find_placeholders_recurse(PlannerInfo *root, Node *jtnode);
static void find_placeholders_in_expr(PlannerInfo *root, Node *expr);
+static bool contain_placeholder_references_walker(Node *node,
+ contain_placeholder_references_context *context);
/*
* make_placeholder_expr
* Make a PlaceHolderVar for the given expression.
*
- * phrels is the syntactic location (as a set of baserels) to attribute
+ * phrels is the syntactic location (as a set of relids) to attribute
* to the expression.
+ *
+ * The caller is responsible for adjusting phlevelsup and phnullingrels
+ * as needed. Because we do not know here which query level the PHV
+ * will be associated with, it's important that this function touches
+ * only root->glob; messing with other parts of PlannerInfo would be
+ * likely to do the wrong thing.
*/
PlaceHolderVar *
make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
@@ -42,8 +57,9 @@ make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
phv->phexpr = expr;
phv->phrels = phrels;
+ phv->phnullingrels = NULL; /* caller may change this later */
phv->phid = ++(root->glob->lastPHId);
- phv->phlevelsup = 0;
+ phv->phlevelsup = 0; /* caller may change this later */
return phv;
}
@@ -92,6 +108,15 @@ find_placeholder_info(PlannerInfo *root, PlaceHolderVar *phv)
phinfo->phid = phv->phid;
phinfo->ph_var = copyObject(phv);
+ /*
+ * By convention, phinfo->ph_var->phnullingrels is always empty, since the
+ * PlaceHolderInfo represents the initially-calculated state of the
+ * PlaceHolderVar. PlaceHolderVars appearing in the query tree might have
+ * varying values of phnullingrels, reflecting outer joins applied above
+ * the calculation level.
+ */
+ phinfo->ph_var->phnullingrels = NULL;
+
/*
* Any referenced rels that are outside the PHV's syntactic scope are
* LATERAL references, which should be included in ph_lateral but not in
@@ -344,6 +369,8 @@ update_placeholder_eval_levels(PlannerInfo *root, SpecialJoinInfo *new_sjinfo)
sjinfo->min_lefthand);
eval_at = bms_add_members(eval_at,
sjinfo->min_righthand);
+ if (sjinfo->ojrelid)
+ eval_at = bms_add_member(eval_at, sjinfo->ojrelid);
/* we'll need another iteration */
found_some = true;
}
@@ -418,6 +445,14 @@ add_placeholders_to_base_rels(PlannerInfo *root)
{
RelOptInfo *rel = find_base_rel(root, varno);
+ /*
+ * As in add_vars_to_targetlist(), a value computed at scan level
+ * has not yet been nulled by any outer join, so its phnullingrels
+ * should be empty.
+ */
+ Assert(phinfo->ph_var->phnullingrels == NULL);
+
+ /* Copying the PHV might be unnecessary here, but be safe */
rel->reltarget->exprs = lappend(rel->reltarget->exprs,
copyObject(phinfo->ph_var));
/* reltarget's cost and width fields will be updated later */
@@ -440,7 +475,8 @@ add_placeholders_to_base_rels(PlannerInfo *root)
*/
void
add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *outer_rel, RelOptInfo *inner_rel)
+ RelOptInfo *outer_rel, RelOptInfo *inner_rel,
+ SpecialJoinInfo *sjinfo)
{
Relids relids = joinrel->relids;
ListCell *lc;
@@ -471,9 +507,17 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
if (!bms_is_subset(phinfo->ph_eval_at, outer_rel->relids) &&
!bms_is_subset(phinfo->ph_eval_at, inner_rel->relids))
{
- PlaceHolderVar *phv = phinfo->ph_var;
+ /* Copying might be unnecessary here, but be safe */
+ PlaceHolderVar *phv = copyObject(phinfo->ph_var);
QualCost cost;
+ /*
+ * It'll start out not nulled by anything. Joins above
+ * this one might add to its phnullingrels later, in much
+ * the same way as for Vars.
+ */
+ Assert(phv->phnullingrels == NULL);
+
joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
phv);
cost_qual_eval_node(&cost, (Node *) phv->phexpr, root);
@@ -504,3 +548,74 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
}
}
}
+
+/*
+ * contain_placeholder_references_to
+ * Detect whether any PlaceHolderVars in the given clause contain
+ * references to the given relid (typically an OJ relid).
+ *
+ * "Contain" means that there's a use of the relid inside the PHV's
+ * contained expression, so that changing the nullability status of
+ * the rel might change what the PHV computes.
+ *
+ * The code here to cope with upper-level PHVs is likely dead, but keep it
+ * anyway just in case.
+ */
+bool
+contain_placeholder_references_to(PlannerInfo *root, Node *clause,
+ int relid)
+{
+ contain_placeholder_references_context context;
+
+ /* We can answer quickly in the common case that there's no PHVs at all */
+ if (root->glob->lastPHId == 0)
+ return false;
+ /* Else run the recursive search */
+ context.relid = relid;
+ context.sublevels_up = 0;
+ return contain_placeholder_references_walker(clause, &context);
+}
+
+static bool
+contain_placeholder_references_walker(Node *node,
+ contain_placeholder_references_context *context)
+{
+ if (node == NULL)
+ return false;
+ if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ /* We should just look through PHVs of other query levels */
+ if (phv->phlevelsup == context->sublevels_up)
+ {
+ /* If phrels matches, we found what we came for */
+ if (bms_is_member(context->relid, phv->phrels))
+ return true;
+
+ /*
+ * We should not examine phnullingrels: what we are looking for is
+ * references in the contained expression, not OJs that might null
+ * the result afterwards. Also, we don't need to recurse into the
+ * contained expression, because phrels should adequately
+ * summarize what's in there. So we're done here.
+ */
+ return false;
+ }
+ }
+ else if (IsA(node, Query))
+ {
+ /* Recurse into RTE subquery or not-yet-planned sublink subquery */
+ bool result;
+
+ context->sublevels_up++;
+ result = query_tree_walker((Query *) node,
+ contain_placeholder_references_walker,
+ context,
+ 0);
+ context->sublevels_up--;
+ return result;
+ }
+ return expression_tree_walker(node, contain_placeholder_references_walker,
+ context);
+}
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 1786a3dadd..226914498a 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -28,6 +28,7 @@
#include "optimizer/plancat.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+#include "rewrite/rewriteManip.h"
#include "utils/hsearch.h"
#include "utils/lsyscache.h"
@@ -39,7 +40,9 @@ typedef struct JoinHashEntry
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *input_rel);
+ RelOptInfo *input_rel,
+ SpecialJoinInfo *sjinfo,
+ bool can_null);
static List *build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outer_rel,
@@ -47,8 +50,10 @@ static List *build_joinrel_restrictlist(PlannerInfo *root,
static void build_joinrel_joinlist(RelOptInfo *joinrel,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel);
-static List *subbuild_joinrel_restrictlist(RelOptInfo *joinrel,
- List *joininfo_list,
+static List *subbuild_joinrel_restrictlist(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *input_rel,
+ Relids both_input_relids,
List *new_restrictlist);
static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
List *joininfo_list,
@@ -56,10 +61,12 @@ static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
static void set_foreign_rel_properties(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel);
static void add_join_rel(PlannerInfo *root, RelOptInfo *joinrel);
-static void build_joinrel_partition_info(RelOptInfo *joinrel,
+static void build_joinrel_partition_info(PlannerInfo *root,
+ RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
- List *restrictlist, JoinType jointype);
-static bool have_partkey_equi_join(RelOptInfo *joinrel,
+ SpecialJoinInfo *sjinfo,
+ List *restrictlist);
+static bool have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *rel1, RelOptInfo *rel2,
JoinType jointype, List *restrictlist);
static int match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel,
@@ -367,7 +374,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
/*
* find_base_rel
- * Find a base or other relation entry, which must already exist.
+ * Find a base or otherrel relation entry, which must already exist.
*/
RelOptInfo *
find_base_rel(PlannerInfo *root, int relid)
@@ -388,6 +395,44 @@ find_base_rel(PlannerInfo *root, int relid)
return NULL; /* keep compiler quiet */
}
+/*
+ * find_base_rel_ignore_join
+ * Find a base or otherrel relation entry, which must already exist.
+ *
+ * Unlike find_base_rel, if relid references an outer join then this
+ * will return NULL rather than raising an error. This is convenient
+ * for callers that must deal with relid sets including both base and
+ * outer joins.
+ */
+RelOptInfo *
+find_base_rel_ignore_join(PlannerInfo *root, int relid)
+{
+ Assert(relid > 0);
+
+ if (relid < root->simple_rel_array_size)
+ {
+ RelOptInfo *rel;
+ RangeTblEntry *rte;
+
+ rel = root->simple_rel_array[relid];
+ if (rel)
+ return rel;
+
+ /*
+ * We could just return NULL here, but for debugging purposes it seems
+ * best to actually verify that the relid is an outer join and not
+ * something weird.
+ */
+ rte = root->simple_rte_array[relid];
+ if (rte && rte->rtekind == RTE_JOIN && rte->jointype != JOIN_INNER)
+ return NULL;
+ }
+
+ elog(ERROR, "no relation entry for relid %d", relid);
+
+ return NULL; /* keep compiler quiet */
+}
+
/*
* build_join_rel_hash
* Construct the auxiliary hash table for join relations.
@@ -687,9 +732,11 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ build_joinrel_tlist(root, joinrel, outer_rel, sjinfo,
+ (sjinfo->jointype == JOIN_FULL));
+ build_joinrel_tlist(root, joinrel, inner_rel, sjinfo,
+ (sjinfo->jointype != JOIN_INNER));
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel, sjinfo);
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -721,8 +768,8 @@ build_join_rel(PlannerInfo *root,
joinrel->has_eclass_joins = has_relevant_eclass_joinclause(root, joinrel);
/* Store the partition information. */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- sjinfo->jointype);
+ build_joinrel_partition_info(root, joinrel, outer_rel, inner_rel, sjinfo,
+ restrictlist);
/*
* Set estimates of the joinrel's size.
@@ -778,16 +825,14 @@ build_join_rel(PlannerInfo *root,
* 'parent_joinrel' is the RelOptInfo representing the join between parent
* relations. Some of the members of new RelOptInfo are produced by
* translating corresponding members of this RelOptInfo
- * 'sjinfo': child-join context info
* 'restrictlist': list of RestrictInfo nodes that apply to this particular
* pair of joinable relations
- * 'jointype' is the join type (inner, left, full, etc)
+ * 'sjinfo': child join's join-type details
*/
RelOptInfo *
build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
RelOptInfo *inner_rel, RelOptInfo *parent_joinrel,
- List *restrictlist, SpecialJoinInfo *sjinfo,
- JoinType jointype)
+ List *restrictlist, SpecialJoinInfo *sjinfo)
{
RelOptInfo *joinrel = makeNode(RelOptInfo);
AppendRelInfo **appinfos;
@@ -801,6 +846,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->reloptkind = RELOPT_OTHER_JOINREL;
joinrel->relids = bms_union(outer_rel->relids, inner_rel->relids);
+ if (sjinfo->ojrelid != 0)
+ joinrel->relids = bms_add_member(joinrel->relids, sjinfo->ojrelid);
joinrel->rows = 0;
/* cheap startup cost is interesting iff not all tuples to be retrieved */
joinrel->consider_startup = (root->tuple_fraction > 0);
@@ -887,8 +934,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->has_eclass_joins = parent_joinrel->has_eclass_joins;
/* Is the join between partitions itself partitioned? */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- jointype);
+ build_joinrel_partition_info(root, joinrel, outer_rel, inner_rel, sjinfo,
+ restrictlist);
/* Child joinrel is parallel safe if parent is parallel safe. */
joinrel->consider_parallel = parent_joinrel->consider_parallel;
@@ -970,10 +1017,41 @@ min_join_parameterization(PlannerInfo *root,
*
* We also compute the expected width of the join's output, making use
* of data that was cached at the baserel level by set_rel_width().
+ *
+ * Pass can_null as true if the join is an outer join that can null Vars
+ * from this input relation. If so, we will (normally) add the join's relid
+ * to the nulling bitmaps of Vars and PHVs bubbled up from the input.
+ *
+ * When forming an outer join's target list, special handling is needed
+ * in case the outer join was commuted with another one per outer join
+ * identity 3 (see optimizer/README). We must take steps to ensure that
+ * the output Vars have the same nulling bitmaps that they would if the
+ * two joins had been done in syntactic order; else they won't match Vars
+ * appearing higher in the query tree. We need to do two things:
+ *
+ * First, sjinfo->commute_above_r is added to the nulling bitmaps of RHS Vars.
+ * This takes care of the case where we implement
+ * A leftjoin (B leftjoin C on (Pbc)) on (Pab)
+ * as
+ * (A leftjoin B on (Pab)) leftjoin C on (Pbc)
+ * The C columns emitted by the B/C join need to be shown as nulled by both
+ * the B/C and A/B joins, even though they've not traversed the A/B join.
+ * (If the joins haven't been commuted, we are adding the nullingrel bits
+ * prematurely; but that's okay because the C columns can't be referenced
+ * between here and the upper join.)
+ *
+ * Second, if a RHS Var has any of the relids in sjinfo->commute_above_l
+ * already set in its nulling bitmap, then we *don't* add sjinfo->ojrelid
+ * to its nulling bitmap (but we do still add commute_above_r). This takes
+ * care of the reverse transformation: if the original syntax was
+ * (A leftjoin B on (Pab)) leftjoin C on (Pbc)
+ * then the now-upper A/B join must not mark C columns as nulled by itself.
*/
static void
build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *input_rel)
+ RelOptInfo *input_rel,
+ SpecialJoinInfo *sjinfo,
+ bool can_null)
{
Relids relids = joinrel->relids;
ListCell *vars;
@@ -993,7 +1071,24 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
/* Is it still needed above this joinrel? */
if (bms_nonempty_difference(phinfo->ph_needed, relids))
{
- /* Yup, add it to the output */
+ /*
+ * Yup, add it to the output. If this join potentially nulls
+ * this input, we have to update the PHV's phnullingrels,
+ * which means making a copy.
+ */
+ if (can_null)
+ {
+ phv = copyObject(phv);
+ /* See comments above to understand this logic */
+ if (sjinfo->ojrelid != 0 &&
+ !bms_overlap(phv->phnullingrels, sjinfo->commute_above_l))
+ phv->phnullingrels = bms_add_member(phv->phnullingrels,
+ sjinfo->ojrelid);
+ if (sjinfo->commute_above_r)
+ phv->phnullingrels = bms_add_members(phv->phnullingrels,
+ sjinfo->commute_above_r);
+ }
+
joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
phv);
/* Bubbling up the precomputed result has cost zero */
@@ -1017,9 +1112,7 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
RowIdentityVarInfo *ridinfo = (RowIdentityVarInfo *)
list_nth(root->row_identity_vars, var->varattno - 1);
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- var);
- /* Vars have cost zero, so no need to adjust reltarget->cost */
+ /* Update reltarget width estimate from RowIdentityVarInfo */
joinrel->reltarget->width += ridinfo->rowidwidth;
}
else
@@ -1032,15 +1125,35 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
/* Is it still needed above this joinrel? */
ndx = var->varattno - baserel->min_attr;
- if (bms_nonempty_difference(baserel->attr_needed[ndx], relids))
- {
- /* Yup, add it to the output */
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- var);
- /* Vars have cost zero, so no need to adjust reltarget->cost */
- joinrel->reltarget->width += baserel->attr_widths[ndx];
- }
+ if (!bms_nonempty_difference(baserel->attr_needed[ndx], relids))
+ continue; /* nope, skip it */
+
+ /* Update reltarget width estimate from baserel's attr_widths */
+ joinrel->reltarget->width += baserel->attr_widths[ndx];
+ }
+
+ /*
+ * Add the Var to the output. If this join potentially nulls this
+ * input, we have to update the Var's varnullingrels, which means
+ * making a copy.
+ */
+ if (can_null)
+ {
+ var = copyObject(var);
+ /* See comments above to understand this logic */
+ if (sjinfo->ojrelid != 0 &&
+ !bms_overlap(var->varnullingrels, sjinfo->commute_above_l))
+ var->varnullingrels = bms_add_member(var->varnullingrels,
+ sjinfo->ojrelid);
+ if (sjinfo->commute_above_r)
+ var->varnullingrels = bms_add_members(var->varnullingrels,
+ sjinfo->commute_above_r);
}
+
+ joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
+ var);
+
+ /* Vars have cost zero, so no need to adjust reltarget->cost */
}
}
@@ -1059,7 +1172,7 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
* is not handled in the sub-relations, so it depends on which
* sub-relations are considered.
*
- * If a join clause from an input relation refers to base rels still not
+ * If a join clause from an input relation refers to base+OJ rels still not
* present in the joinrel, then it is still a join clause for the joinrel;
* we put it into the joininfo list for the joinrel. Otherwise,
* the clause is now a restrict clause for the joined relation, and we
@@ -1093,14 +1206,19 @@ build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *inner_rel)
{
List *result;
+ Relids both_input_relids;
+
+ both_input_relids = bms_union(outer_rel->relids, inner_rel->relids);
/*
* Collect all the clauses that syntactically belong at this level,
* eliminating any duplicates (important since we will see many of the
* same clauses arriving from both input relations).
*/
- result = subbuild_joinrel_restrictlist(joinrel, outer_rel->joininfo, NIL);
- result = subbuild_joinrel_restrictlist(joinrel, inner_rel->joininfo, result);
+ result = subbuild_joinrel_restrictlist(root, joinrel, outer_rel,
+ both_input_relids, NIL);
+ result = subbuild_joinrel_restrictlist(root, joinrel, inner_rel,
+ both_input_relids, result);
/*
* Add on any clauses derived from EquivalenceClasses. These cannot be
@@ -1135,24 +1253,63 @@ build_joinrel_joinlist(RelOptInfo *joinrel,
}
static List *
-subbuild_joinrel_restrictlist(RelOptInfo *joinrel,
- List *joininfo_list,
+subbuild_joinrel_restrictlist(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *input_rel,
+ Relids both_input_relids,
List *new_restrictlist)
{
ListCell *l;
- foreach(l, joininfo_list)
+ foreach(l, input_rel->joininfo)
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(l);
if (bms_is_subset(rinfo->required_relids, joinrel->relids))
{
/*
- * This clause becomes a restriction clause for the joinrel, since
- * it refers to no outside rels. Add it to the list, being
- * careful to eliminate duplicates. (Since RestrictInfo nodes in
- * different joinlists will have been multiply-linked rather than
- * copied, pointer equality should be a sufficient test.)
+ * This clause should become a restriction clause for the joinrel,
+ * since it refers to no outside rels. However, if it's a clone
+ * clause then it might be too late to evaluate it, so we have to
+ * check. (If it is too late, just ignore the clause, taking it
+ * on faith that another clone was or will be selected.) Clone
+ * clauses should always be outer-join clauses, so we compare
+ * against both_input_relids.
+ */
+ if (rinfo->has_clone || rinfo->is_clone)
+ {
+ Assert(!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids));
+ if (!bms_is_subset(rinfo->required_relids, both_input_relids))
+ continue;
+ if (!clause_is_computable_at(root, rinfo->clause_relids,
+ both_input_relids))
+ continue;
+ }
+ else
+ {
+ /*
+ * For non-clone clauses, we just Assert it's OK. These might
+ * be either join or filter clauses.
+ */
+#ifdef USE_ASSERT_CHECKING
+ if (RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ Assert(clause_is_computable_at(root, rinfo->clause_relids,
+ joinrel->relids));
+ else
+ {
+ Assert(bms_is_subset(rinfo->required_relids,
+ both_input_relids));
+ Assert(clause_is_computable_at(root, rinfo->clause_relids,
+ both_input_relids));
+ }
+#endif
+ }
+
+ /*
+ * OK, so add it to the list, being careful to eliminate
+ * duplicates. (Since RestrictInfo nodes in different joinlists
+ * will have been multiply-linked rather than copied, pointer
+ * equality should be a sufficient test.)
*/
new_restrictlist = list_append_unique_ptr(new_restrictlist, rinfo);
}
@@ -1659,9 +1816,10 @@ find_param_path_info(RelOptInfo *rel, Relids required_outer)
* partitioned join relation.
*/
static void
-build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
- RelOptInfo *inner_rel, List *restrictlist,
- JoinType jointype)
+build_joinrel_partition_info(PlannerInfo *root,
+ RelOptInfo *joinrel, RelOptInfo *outer_rel,
+ RelOptInfo *inner_rel, SpecialJoinInfo *sjinfo,
+ List *restrictlist)
{
PartitionScheme part_scheme;
@@ -1687,8 +1845,8 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
!outer_rel->consider_partitionwise_join ||
!inner_rel->consider_partitionwise_join ||
outer_rel->part_scheme != inner_rel->part_scheme ||
- !have_partkey_equi_join(joinrel, outer_rel, inner_rel,
- jointype, restrictlist))
+ !have_partkey_equi_join(root, joinrel, outer_rel, inner_rel,
+ sjinfo->jointype, restrictlist))
{
Assert(!IS_PARTITIONED_REL(joinrel));
return;
@@ -1712,7 +1870,8 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* child-join relations of the join relation in try_partitionwise_join().
*/
joinrel->part_scheme = part_scheme;
- set_joinrel_partition_key_exprs(joinrel, outer_rel, inner_rel, jointype);
+ set_joinrel_partition_key_exprs(joinrel, outer_rel, inner_rel,
+ sjinfo->jointype);
/*
* Set the consider_partitionwise_join flag.
@@ -1730,7 +1889,7 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* partition keys.
*/
static bool
-have_partkey_equi_join(RelOptInfo *joinrel,
+have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *rel1, RelOptInfo *rel2,
JoinType jointype, List *restrictlist)
{
@@ -1795,6 +1954,24 @@ have_partkey_equi_join(RelOptInfo *joinrel,
*/
strict_op = op_strict(opexpr->opno);
+ /*
+ * Vars appearing in the relation's partition keys will not have any
+ * varnullingrels, but those in expr1 and expr2 will if we're above
+ * outer joins that could null the respective rels. It's okay to
+ * match anyway, if the join operator is strict.
+ */
+ if (strict_op)
+ {
+ if (bms_overlap(rel1->relids, root->outer_join_rels))
+ expr1 = (Expr *) remove_nulling_relids((Node *) expr1,
+ root->outer_join_rels,
+ NULL);
+ if (bms_overlap(rel2->relids, root->outer_join_rels))
+ expr2 = (Expr *) remove_nulling_relids((Node *) expr2,
+ root->outer_join_rels,
+ NULL);
+ }
+
/*
* Only clauses referencing the partition keys are useful for
* partitionwise join.
@@ -2007,7 +2184,12 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
* partitionwise nesting of any outer join.) We assume no
* type coercions are needed to make the coalesce expressions,
* since columns of different types won't have gotten
- * classified as the same PartitionScheme.
+ * classified as the same PartitionScheme. Note that we
+ * intentionally leave out the varnullingrels decoration that
+ * would ordinarily appear on the Vars inside these
+ * CoalesceExprs, because have_partkey_equi_join will strip
+ * varnullingrels from the expressions it will compare to the
+ * partexprs.
*/
foreach(lc, list_concat_copy(outer_expr, outer_null_expr))
{
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index ef8df3d098..327c3ba563 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -53,6 +53,10 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
* required_relids can be NULL, in which case it defaults to the actual clause
* contents (i.e., clause_relids).
*
+ * Note that there aren't options to set the has_clone and is_clone flags:
+ * we always initialize those to false. There's just one place that wants
+ * something different, so making all callers pass them seems inconvenient.
+ *
* We initialize fields that depend only on the given subexpression, leaving
* others that depend on context (or may never be needed at all) to be filled
* later.
@@ -116,12 +120,15 @@ make_restrictinfo_internal(PlannerInfo *root,
Relids nullable_relids)
{
RestrictInfo *restrictinfo = makeNode(RestrictInfo);
+ Relids baserels;
restrictinfo->clause = clause;
restrictinfo->orclause = orclause;
restrictinfo->is_pushed_down = is_pushed_down;
restrictinfo->outerjoin_delayed = outerjoin_delayed;
restrictinfo->pseudoconstant = pseudoconstant;
+ restrictinfo->has_clone = false; /* may get set by caller */
+ restrictinfo->is_clone = false; /* may get set by caller */
restrictinfo->can_join = false; /* may get set below */
restrictinfo->security_level = security_level;
restrictinfo->outer_relids = outer_relids;
@@ -187,6 +194,20 @@ make_restrictinfo_internal(PlannerInfo *root,
else
restrictinfo->required_relids = restrictinfo->clause_relids;
+ /*
+ * Count the number of base rels appearing in clause_relids. To do this,
+ * we just delete rels mentioned in root->outer_join_rels and count the
+ * survivors. Because we are called during deconstruct_jointree which is
+ * the same tree walk that populates outer_join_rels, this is a little bit
+ * unsafe-looking; but it should be fine because the recursion in
+ * deconstruct_jointree should already have visited any outer join that
+ * could be mentioned in this clause.
+ */
+ baserels = bms_difference(restrictinfo->clause_relids,
+ root->outer_join_rels);
+ restrictinfo->num_base_rels = bms_num_members(baserels);
+ bms_free(baserels);
+
/*
* Fill in all the cacheable fields with "not yet set" markers. None of
* these will be computed until/unless needed. Note in particular that we
@@ -497,6 +518,58 @@ extract_actual_join_clauses(List *restrictinfo_list,
}
}
+/*
+ * clause_is_computable_at
+ * Test whether a clause is computable at a given evaluation level.
+ *
+ * There are two conditions for whether an expression can actually be
+ * evaluated at a given join level: the evaluation context must include
+ * all the relids (both base and OJ) used by the expression, and we must
+ * not have already evaluated any outer joins that null Vars/PHVs of the
+ * expression and are not listed in their nullingrels.
+ *
+ * This function checks the second condition; we assume the caller already
+ * saw to the first one.
+ *
+ * For speed reasons, we don't individually examine each Var/PHV of the
+ * expression, but just look at the overall clause_relids (the union of the
+ * varnos and varnullingrels). This could give a misleading answer if the
+ * Vars of a given varno don't all have the same varnullingrels; but that
+ * really shouldn't happen within a single scalar expression or RestrictInfo
+ * clause. Despite that, this is still annoyingly expensive :-(
+ */
+bool
+clause_is_computable_at(PlannerInfo *root,
+ Relids clause_relids,
+ Relids eval_relids)
+{
+ ListCell *lc;
+
+ /* Nothing to do if no outer joins have been performed yet. */
+ if (!bms_overlap(eval_relids, root->outer_join_rels))
+ return true;
+
+ foreach(lc, root->join_info_list)
+ {
+ SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+
+ /* Ignore outer joins that are not yet performed. */
+ if (!bms_is_member(sjinfo->ojrelid, eval_relids))
+ continue;
+
+ /* OK if clause lists it (we assume all Vars in it agree). */
+ if (bms_is_member(sjinfo->ojrelid, clause_relids))
+ continue;
+
+ /* Else, trouble if clause mentions any nullable Vars. */
+ if (bms_overlap(clause_relids, sjinfo->min_righthand) ||
+ (sjinfo->jointype == JOIN_FULL &&
+ bms_overlap(clause_relids, sjinfo->min_lefthand)))
+ return false; /* doesn't work */
+ }
+
+ return true; /* OK */
+}
/*
* join_clause_is_movable_to
@@ -522,6 +595,12 @@ extract_actual_join_clauses(List *restrictinfo_list,
* Also, the join clause must not use any relations that have LATERAL
* references to the target relation, since we could not put such rels on
* the outer side of a nestloop with the target relation.
+ *
+ * Also, we reject is_clone versions of outer-join clauses. This has the
+ * effect of preventing us from generating variant parameterized paths
+ * that differ only in which outer joins null the parameterization rel(s).
+ * Generating one path from the minimally-parameterized has_clone version
+ * is sufficient.
*/
bool
join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
@@ -542,6 +621,10 @@ join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
if (bms_overlap(baserel->lateral_referencers, rinfo->clause_relids))
return false;
+ /* Ignore clones, too */
+ if (rinfo->is_clone)
+ return false;
+
return true;
}
@@ -587,6 +670,9 @@ join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
* moved for some valid set of outer rels, so we don't have the benefit of
* relying on prior checks for lateral-reference validity.
*
+ * Likewise, we don't check is_clone here: rejecting the inappropriate
+ * variants of a cloned clause must be handled upstream.
+ *
* Note: if this returns true, it means that the clause could be moved to
* this join relation, but that doesn't mean that this is the lowest join
* it could be moved to. Caller may need to make additional calls to verify
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
index 7db86c39ef..8d8c9136f8 100644
--- a/src/backend/optimizer/util/var.c
+++ b/src/backend/optimizer/util/var.c
@@ -88,6 +88,9 @@ static Relids alias_relid_set(Query *query, Relids relids);
* Create a set of all the distinct varnos present in a parsetree.
* Only varnos that reference level-zero rtable entries are considered.
*
+ * The result includes outer-join relids mentioned in Var.varnullingrels and
+ * PlaceHolderVar.phnullingrels fields in the parsetree.
+ *
* "root" can be passed as NULL if it is not necessary to process
* PlaceHolderVars.
*
@@ -153,7 +156,11 @@ pull_varnos_walker(Node *node, pull_varnos_context *context)
Var *var = (Var *) node;
if (var->varlevelsup == context->sublevels_up)
+ {
context->varnos = bms_add_member(context->varnos, var->varno);
+ context->varnos = bms_add_members(context->varnos,
+ var->varnullingrels);
+ }
return false;
}
if (IsA(node, CurrentOfExpr))
@@ -244,6 +251,14 @@ pull_varnos_walker(Node *node, pull_varnos_context *context)
context->varnos = bms_join(context->varnos,
newevalat);
}
+
+ /*
+ * In all three cases, include phnullingrels in the result. We
+ * don't worry about possibly needing to translate it, because
+ * appendrels only translate varnos of baserels, not outer joins.
+ */
+ context->varnos = bms_add_members(context->varnos,
+ phv->phnullingrels);
return false; /* don't recurse into expression */
}
}
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index 69e0fb98f5..8602308a45 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -2205,7 +2205,7 @@ rowcomparesel(PlannerInfo *root,
else
{
/*
- * Otherwise, it's a join if there's more than one relation used.
+ * Otherwise, it's a join if there's more than one base relation used.
*/
is_join_clause = (NumRelids(root, (Node *) opargs) > 1);
}
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 2c6d5ca58f..a734676293 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -243,13 +243,26 @@ struct PlannerInfo
struct AppendRelInfo **append_rel_array pg_node_attr(read_write_ignore);
/*
- * all_baserels is a Relids set of all base relids (but not "other"
- * relids) in the query; that is, the Relids identifier of the final join
- * we need to form. This is computed in make_one_rel, just before we
- * start making Paths.
+ * all_baserels is a Relids set of all base relids (but not joins or
+ * "other" relids) in the query. This is computed in
+ * add_base_rels_to_query.
*/
Relids all_baserels;
+ /*
+ * outer_join_rels is a Relids set of all outer-join relids in the query.
+ * This is computed in deconstruct_jointree.
+ */
+ Relids outer_join_rels;
+
+ /*
+ * all_query_rels is a Relids set of all base relids and outer join relids
+ * (but not "other" relids) in the query. This is the Relids identifier
+ * of the final join we need to form. This is computed in
+ * deconstruct_jointree.
+ */
+ Relids all_query_rels;
+
/*
* nullable_baserels is a Relids set of base relids that are nullable by
* some outer join in the jointree; these are rels that are potentially
@@ -319,7 +332,7 @@ struct PlannerInfo
List *right_join_clauses;
/*
- * list of RestrictInfos for mergejoinable full join clauses
+ * list of FullJoinClauseInfos for mergejoinable full join clauses
*/
List *full_join_clauses;
@@ -555,9 +568,10 @@ typedef struct PartitionSchemeData *PartitionScheme;
* or the output of a sub-SELECT or function that appears in the range table.
* In either case it is uniquely identified by an RT index. A "joinrel"
* is the joining of two or more base rels. A joinrel is identified by
- * the set of RT indexes for its component baserels. We create RelOptInfo
- * nodes for each baserel and joinrel, and store them in the PlannerInfo's
- * simple_rel_array and join_rel_list respectively.
+ * the set of RT indexes for its component baserels, along with RT indexes
+ * for any outer joins it has computed. We create RelOptInfo nodes for each
+ * baserel and joinrel, and store them in the PlannerInfo's simple_rel_array
+ * and join_rel_list respectively.
*
* Note that there is only one joinrel for any given set of component
* baserels, no matter what order we assemble them in; so an unordered
@@ -596,8 +610,10 @@ typedef struct PartitionSchemeData *PartitionScheme;
* Parts of this data structure are specific to various scan and join
* mechanisms. It didn't seem worth creating new node types for them.
*
- * relids - Set of base-relation identifiers; it is a base relation
- * if there is just one, a join relation if more than one
+ * relids - Set of relation identifiers (RT indexes). This is a base
+ * relation if there is just one, a join relation if more;
+ * in the join case, RT indexes of any outer joins formed
+ * at or below this join are included along with baserels
* rows - estimated number of tuples in the relation after restriction
* clauses have been applied (ie, output rows of a plan for it)
* consider_startup - true if there is any value in keeping plain paths for
@@ -636,7 +652,9 @@ typedef struct PartitionSchemeData *PartitionScheme;
* min_attr, max_attr - range of valid AttrNumbers for rel
* attr_needed - array of bitmapsets indicating the highest joinrel
* in which each attribute is needed; if bit 0 is set then
- * the attribute is needed as part of final targetlist
+ * the attribute is needed as part of final targetlist.
+ * By convention, attr_needed includes only baserels not
+ * outer-join relids.
* attr_widths - cache space for per-attribute width estimates;
* zero means not computed yet
* lateral_vars - lateral cross-references of rel, if any (list of
@@ -809,7 +827,7 @@ typedef struct RelOptInfo
RelOptKind reloptkind;
/*
- * all relations included in this RelOptInfo; set of base relids
+ * all relations included in this RelOptInfo; set of base + OJ relids
* (rangetable indexes)
*/
Relids relids;
@@ -2278,17 +2296,17 @@ typedef struct LimitPath
* If a restriction clause references a single base relation, it will appear
* in the baserestrictinfo list of the RelOptInfo for that base rel.
*
- * If a restriction clause references more than one base rel, it will
+ * If a restriction clause references more than one base+OJ relation, it will
* appear in the joininfo list of every RelOptInfo that describes a strict
- * subset of the base rels mentioned in the clause. The joininfo lists are
+ * subset of the relations mentioned in the clause. The joininfo lists are
* used to drive join tree building by selecting plausible join candidates.
* The clause cannot actually be applied until we have built a join rel
- * containing all the base rels it references, however.
+ * containing all the relations it references, however.
*
- * When we construct a join rel that includes all the base rels referenced
+ * When we construct a join rel that includes all the relations referenced
* in a multi-relation restriction clause, we place that clause into the
* joinrestrictinfo lists of paths for the join rel, if neither left nor
- * right sub-path includes all base rels referenced in the clause. The clause
+ * right sub-path includes all relations referenced in the clause. The clause
* will be applied at that join level, and will not propagate any further up
* the join tree. (Note: the "predicate migration" code was once intended to
* push restriction clauses up and down the plan tree based on evaluation
@@ -2309,12 +2327,15 @@ typedef struct LimitPath
* or join to enforce that all members of each EquivalenceClass are in fact
* equal in all rows emitted by the scan or join.
*
- * When dealing with outer joins we have to be very careful about pushing qual
- * clauses up and down the tree. An outer join's own JOIN/ON conditions must
- * be evaluated exactly at that join node, unless they are "degenerate"
- * conditions that reference only Vars from the nullable side of the join.
- * Quals appearing in WHERE or in a JOIN above the outer join cannot be pushed
- * down below the outer join, if they reference any nullable Vars.
+ * The clause_relids field lists the base plus outer-join RT indexes that
+ * actually appear in the clause. required_relids lists the minimum set of
+ * relids needed to evaluate the clause; while this is often equal to
+ * clause_relids, it can be more. We will add relids to required_relids when
+ * we need to force an outer join ON clause to be evaluated exactly at the
+ * level of the outer join, which is true except when it is a "degenerate"
+ * condition that references only Vars from the nullable side of the join.
+ *
+ * XXX rewrite or remove me:
* RestrictInfo nodes contain a flag to indicate whether a qual has been
* pushed down to a lower level than its original syntactic placement in the
* join tree would suggest. If an outer join prevents us from pushing a qual
@@ -2399,6 +2420,12 @@ typedef struct LimitPath
* or merge or hash join clause, so it's of no interest to large parts of
* the planner.
*
+ * When we generate multiple versions of a clause so as to have versions
+ * that will work after commuting some left joins per outer join identity 3,
+ * we mark the one with the fewest nullingrels bits with has_clone = true,
+ * and the rest with is_clone = true. This allows proper filtering of
+ * these redundant clauses, so that we apply only one version of them.
+ *
* When join clauses are generated from EquivalenceClasses, there may be
* several equally valid ways to enforce join equivalence, of which we need
* apply only one. We mark clauses of this kind by setting parent_ec to
@@ -2433,16 +2460,23 @@ typedef struct RestrictInfo
/* see comment above */
bool pseudoconstant pg_node_attr(equal_ignore);
+ /* see comment above */
+ bool has_clone;
+ bool is_clone;
+
/* true if known to contain no leaked Vars */
bool leakproof pg_node_attr(equal_ignore);
- /* to indicate if clause contains any volatile functions. */
+ /* indicates if clause contains any volatile functions */
VolatileFunctionStatus has_volatile pg_node_attr(equal_ignore);
/* see comment above */
Index security_level;
- /* The set of relids (varnos) actually referenced in the clause: */
+ /* number of base rels in clause_relids */
+ int num_base_rels pg_node_attr(equal_ignore);
+
+ /* The relids (varnos+varnullingrels) actually referenced in the clause: */
Relids clause_relids pg_node_attr(equal_ignore);
/* The set of relids required to evaluate the clause: */
@@ -2544,6 +2578,7 @@ typedef struct RestrictInfo
} RestrictInfo;
/*
+ * XXX this will need work:
* This macro embodies the correct way to test whether a RestrictInfo is
* "pushed down" to a given outer join, that is, should be treated as a filter
* clause rather than a join clause at that outer join. This is certainly so
@@ -2646,7 +2681,7 @@ typedef struct PlaceHolderVar
* We make SpecialJoinInfos for FULL JOINs even though there is no flexibility
* of planning for them, because this simplifies make_join_rel()'s API.
*
- * min_lefthand and min_righthand are the sets of base relids that must be
+ * min_lefthand and min_righthand are the sets of base+OJ relids that must be
* available on each side when performing the special join. lhs_strict is
* true if the special join's condition cannot succeed when the LHS variables
* are all NULL (this means that an outer join can commute with upper-level
@@ -2656,7 +2691,7 @@ typedef struct PlaceHolderVar
* It is not valid for either min_lefthand or min_righthand to be empty sets;
* if they were, this would break the logic that enforces join order.
*
- * syn_lefthand and syn_righthand are the sets of base relids that are
+ * syn_lefthand and syn_righthand are the sets of base+OJ relids that are
* syntactically below this special join. (These are needed to help compute
* min_lefthand and min_righthand for higher joins.)
*
@@ -2678,6 +2713,37 @@ typedef struct PlaceHolderVar
* the inputs to make it a LEFT JOIN. So the allowed values of jointype
* in a join_info_list member are only LEFT, FULL, SEMI, or ANTI.
*
+ * ojrelid is the RT index of the join RTE representing this outer join,
+ * if there is one. It is zero when jointype is INNER or SEMI, and can be
+ * zero for jointype ANTI (if the join was transformed from a SEMI join).
+ * One use for this field is that when constructing the output targetlist of a
+ * join relation that implements this OJ, we add ojrelid to the varnullingrels
+ * and phnullingrels fields of nullable (RHS) output columns, so that the
+ * output Vars and PlaceHolderVars correctly reflect the nulling that has
+ * potentially happened to them.
+ *
+ * commute_above_l is filled with the relids of syntactically-higher outer
+ * joins that have been found to commute with this one per outer join identity
+ * 3 (see optimizer/README), when this join is in the LHS of the upper join
+ * (so, this is the lower join in the first form of the identity).
+ *
+ * commute_above_r is filled with the relids of syntactically-higher outer
+ * joins that have been found to commute with this one per outer join identity
+ * 3, when this join is in the RHS of the upper join (so, this is the lower
+ * join in the second form of the identity).
+ *
+ * commute_below is filled with the relids of syntactically-lower outer joins
+ * that have been found to commute with this one per outer join identity 3.
+ * (We need not record which side they are on, since that can be determined
+ * by seeing whether the lower join's relid appears in syn_lefthand or
+ * syn_righthand.)
+ *
+ * oj_joinclause is used during deconstruct_jointree() to hold the JOIN/ON
+ * quals of a possibly-commutable outer join until the end of the jointree
+ * walk (at which time we'll know whether any other outer joins actually
+ * commute with it, and can decorate the quals properly). These quals do not
+ * have RestrictInfos yet.
+ *
* For purposes of join selectivity estimation, we create transient
* SpecialJoinInfo structures for regular inner joins; so it is possible
* to have jointype == JOIN_INNER in such a structure, even though this is
@@ -2697,11 +2763,16 @@ struct SpecialJoinInfo
pg_node_attr(no_read)
NodeTag type;
- Relids min_lefthand; /* base relids in minimum LHS for join */
- Relids min_righthand; /* base relids in minimum RHS for join */
- Relids syn_lefthand; /* base relids syntactically within LHS */
- Relids syn_righthand; /* base relids syntactically within RHS */
+ Relids min_lefthand; /* base+OJ relids in minimum LHS for join */
+ Relids min_righthand; /* base+OJ relids in minimum RHS for join */
+ Relids syn_lefthand; /* base+OJ relids syntactically within LHS */
+ Relids syn_righthand; /* base+OJ relids syntactically within RHS */
JoinType jointype; /* always INNER, LEFT, FULL, SEMI, or ANTI */
+ Index ojrelid; /* outer join's RT index; 0 if none */
+ Relids commute_above_l; /* commuting OJs above this one, if LHS */
+ Relids commute_above_r; /* commuting OJs above this one, if RHS */
+ Relids commute_below; /* commuting OJs below this one */
+ List *oj_joinclause; /* outer join quals not yet distributed */
bool lhs_strict; /* joinclause is strict for some LHS rel */
bool delay_upper_joins; /* can't commute with upper RHS */
/* Remaining fields are set only for JOIN_SEMI jointype: */
@@ -2711,6 +2782,21 @@ struct SpecialJoinInfo
List *semi_rhs_exprs; /* righthand-side expressions of these ops */
};
+/*
+ * FULL JOIN clause info.
+ *
+ * We set aside every FULL JOIN ON clause that looks mergejoinable, and
+ * process it specially at the end of qual distribution.
+ */
+typedef struct FullJoinClauseInfo
+{
+ pg_node_attr(no_copy_equal, no_read)
+
+ NodeTag type;
+ RestrictInfo *rinfo; /* a mergejoinable FULL JOIN clause */
+ SpecialJoinInfo *sjinfo; /* the FULL JOIN's SpecialJoinInfo */
+} FullJoinClauseInfo;
+
/*
* Append-relation info.
*
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 050f00e79a..197234d44c 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -304,6 +304,7 @@ extern void expand_planner_arrays(PlannerInfo *root, int add_size);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
+extern RelOptInfo *find_base_rel_ignore_join(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
@@ -335,6 +336,6 @@ extern ParamPathInfo *find_param_path_info(RelOptInfo *rel,
extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
- SpecialJoinInfo *sjinfo, JoinType jointype);
+ SpecialJoinInfo *sjinfo);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/placeholder.h b/src/include/optimizer/placeholder.h
index 507dbc6175..3fe9b57415 100644
--- a/src/include/optimizer/placeholder.h
+++ b/src/include/optimizer/placeholder.h
@@ -27,6 +27,9 @@ extern void update_placeholder_eval_levels(PlannerInfo *root,
extern void fix_placeholder_input_needed_levels(PlannerInfo *root);
extern void add_placeholders_to_base_rels(PlannerInfo *root);
extern void add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *outer_rel, RelOptInfo *inner_rel);
+ RelOptInfo *outer_rel, RelOptInfo *inner_rel,
+ SpecialJoinInfo *sjinfo);
+extern bool contain_placeholder_references_to(PlannerInfo *root, Node *clause,
+ int relid);
#endif /* PLACEHOLDER_H */
diff --git a/src/include/optimizer/prep.h b/src/include/optimizer/prep.h
index 5b4f350b33..0847cfd5f4 100644
--- a/src/include/optimizer/prep.h
+++ b/src/include/optimizer/prep.h
@@ -29,7 +29,8 @@ extern void pull_up_subqueries(PlannerInfo *root);
extern void flatten_simple_union_all(PlannerInfo *root);
extern void reduce_outer_joins(PlannerInfo *root);
extern void remove_useless_result_rtes(PlannerInfo *root);
-extern Relids get_relids_in_jointree(Node *jtnode, bool include_joins);
+extern Relids get_relids_in_jointree(Node *jtnode, bool include_outer_joins,
+ bool include_inner_joins);
extern Relids get_relids_for_join(Query *query, int joinrelid);
/*
diff --git a/src/include/optimizer/restrictinfo.h b/src/include/optimizer/restrictinfo.h
index 6d30bd5e9d..17d3b4ab05 100644
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -41,6 +41,9 @@ extern void extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
List **joinquals,
List **otherquals);
+extern bool clause_is_computable_at(PlannerInfo *root,
+ Relids clause_relids,
+ Relids eval_relids);
extern bool join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel);
extern bool join_clause_is_movable_into(RestrictInfo *rinfo,
Relids currentrelids,
commit 6cebc8bb1df3186cb5033fb5454bf922da00a28f
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Oct 31 12:12:44 2022 -0400
Detect duplicated pushed-down conditions using RestrictInfo ID numbers.
create_nestloop_path needs to identify which candidates for join
restriction quals were already enforced in the parameterized inner
path. Currently we do that by relying on join_clause_is_movable_into
to give consistent answers, but that is not working very well with
variant clauses generated to satisfy outer join identity 3. We may
have a clause that (correctly) shows the outer-side Var as nulled by
a previous outer join, which makes it dependent on the nestloop outer
side having included that join, so that it appears to not be pushable
into a parameterized path that uses the un-nulled version of that Var.
Nonetheless, the cloned clause *is* redundant and we don't want
to check it again.
This patch offers a somewhat brute-force solution, which is to assign
serial numbers to RestrictInfo nodes, then check for redundancy using
serial number match rather than trusting join_clause_is_movable_into.
The variant-clause problem can be solved by allowing clauses to share
a serial number when we know that they are equivalent. Both the
outer-join variant generator and equivclass.c need to be in on that
trick in order to handle all cases that were handled well before.
It'd be nicer if we could continue to trust join_clause_is_movable_into
for this, but on the other hand this mechanism does provide a much more
concrete, harder-to-break way of verifying that we already enforced
(some version of) a qual. Any failure mode would almost certainly
be in the safe direction of enforcing a qual redundantly, which is
not a claim that the existing method can make.
This patch results in two changes to the core regression test outputs:
* One query in join.sql changes to a different join order. Examining
the cost estimates that are normally not shown, the new order is
estimated as very slightly faster, so this seems like an improvement.
I'm not quite sure why the old code did not find this join order.
* Some of the queries in partition_join.sql revert equivalence-clause
ordering back to what it was before a5fc46414. That's probably a
consequence of investigating parameterized paths in a different order
than before. Anyway, it's visibly harmless.
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 349e183372..d4f8b7893d 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -35,7 +35,8 @@
static EquivalenceMember *add_eq_member(EquivalenceClass *ec,
Expr *expr, Relids relids, Relids nullable_relids,
- bool is_child, Oid datatype);
+ EquivalenceMember *parent,
+ Oid datatype);
static bool is_exprlist_member(Expr *node, List *exprs);
static void generate_base_implied_equalities_const(PlannerInfo *root,
EquivalenceClass *ec);
@@ -400,7 +401,7 @@ process_equivalence(PlannerInfo *root,
{
/* Case 3: add item2 to ec1 */
em2 = add_eq_member(ec1, item2, item2_relids, item2_nullable_relids,
- false, item2_type);
+ NULL, item2_type);
ec1->ec_sources = lappend(ec1->ec_sources, restrictinfo);
ec1->ec_below_outer_join |= below_outer_join;
ec1->ec_min_security = Min(ec1->ec_min_security,
@@ -418,7 +419,7 @@ process_equivalence(PlannerInfo *root,
{
/* Case 3: add item1 to ec2 */
em1 = add_eq_member(ec2, item1, item1_relids, item1_nullable_relids,
- false, item1_type);
+ NULL, item1_type);
ec2->ec_sources = lappend(ec2->ec_sources, restrictinfo);
ec2->ec_below_outer_join |= below_outer_join;
ec2->ec_min_security = Min(ec2->ec_min_security,
@@ -452,9 +453,9 @@ process_equivalence(PlannerInfo *root,
ec->ec_max_security = restrictinfo->security_level;
ec->ec_merged = NULL;
em1 = add_eq_member(ec, item1, item1_relids, item1_nullable_relids,
- false, item1_type);
+ NULL, item1_type);
em2 = add_eq_member(ec, item2, item2_relids, item2_nullable_relids,
- false, item2_type);
+ NULL, item2_type);
root->eq_classes = lappend(root->eq_classes, ec);
@@ -544,7 +545,7 @@ canonicalize_ec_expression(Expr *expr, Oid req_type, Oid req_collation)
*/
static EquivalenceMember *
add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
- Relids nullable_relids, bool is_child, Oid datatype)
+ Relids nullable_relids, EquivalenceMember *parent, Oid datatype)
{
EquivalenceMember *em = makeNode(EquivalenceMember);
@@ -552,8 +553,9 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
em->em_relids = relids;
em->em_nullable_relids = nullable_relids;
em->em_is_const = false;
- em->em_is_child = is_child;
+ em->em_is_child = (parent != NULL);
em->em_datatype = datatype;
+ em->em_parent = parent;
if (bms_is_empty(relids))
{
@@ -565,12 +567,12 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
* get_eclass_for_sort_expr() has to work harder. We put the tests
* there not here to save cycles in the equivalence case.
*/
- Assert(!is_child);
+ Assert(!parent);
em->em_is_const = true;
ec->ec_has_const = true;
/* it can't affect ec_relids */
}
- else if (!is_child) /* child members don't add to ec_relids */
+ else if (!parent) /* child members don't add to ec_relids */
{
ec->ec_relids = bms_add_members(ec->ec_relids, relids);
}
@@ -723,7 +725,7 @@ get_eclass_for_sort_expr(PlannerInfo *root,
nullable_relids = bms_intersect(nullable_relids, expr_relids);
newem = add_eq_member(newec, copyObject(expr), expr_relids,
- nullable_relids, false, opcintype);
+ nullable_relids, NULL, opcintype);
/*
* add_eq_member doesn't check for volatile functions, set-returning
@@ -1821,6 +1823,7 @@ create_join_clause(PlannerInfo *root,
EquivalenceClass *parent_ec)
{
RestrictInfo *rinfo;
+ RestrictInfo *parent_rinfo = NULL;
ListCell *lc;
MemoryContext oldcontext;
@@ -1865,6 +1868,20 @@ create_join_clause(PlannerInfo *root,
*/
oldcontext = MemoryContextSwitchTo(root->planner_cxt);
+ /*
+ * If either EM is a child, recursively create the corresponding
+ * parent-to-parent clause, so that we can duplicate its rinfo_serial.
+ */
+ if (leftem->em_is_child || rightem->em_is_child)
+ {
+ EquivalenceMember *leftp = leftem->em_parent ? leftem->em_parent : leftem;
+ EquivalenceMember *rightp = rightem->em_parent ? rightem->em_parent : rightem;
+
+ parent_rinfo = create_join_clause(root, ec, opno,
+ leftp, rightp,
+ parent_ec);
+ }
+
rinfo = build_implied_join_equality(root,
opno,
ec->ec_collation,
@@ -1876,6 +1893,10 @@ create_join_clause(PlannerInfo *root,
rightem->em_nullable_relids),
ec->ec_min_security);
+ /* If it's a child clause, copy the parent's rinfo_serial */
+ if (parent_rinfo)
+ rinfo->rinfo_serial = parent_rinfo->rinfo_serial;
+
/* Mark the clause as redundant, or not */
rinfo->parent_ec = parent_ec;
@@ -2686,7 +2707,7 @@ add_child_rel_equivalences(PlannerInfo *root,
(void) add_eq_member(cur_ec, child_expr,
new_relids, new_nullable_relids,
- true, cur_em->em_datatype);
+ cur_em, cur_em->em_datatype);
/* Record this EC index for the child rel */
child_rel->eclass_indexes = bms_add_member(child_rel->eclass_indexes, i);
@@ -2827,7 +2848,7 @@ add_child_join_rel_equivalences(PlannerInfo *root,
(void) add_eq_member(cur_ec, child_expr,
new_relids, new_nullable_relids,
- true, cur_em->em_datatype);
+ cur_em, cur_em->em_datatype);
}
}
}
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 41c69b29a7..96e8033930 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -1785,6 +1785,7 @@ process_postponed_left_join_quals(PlannerInfo *root)
Relids joins_below;
Relids joins_so_far;
List *quals;
+ int save_last_rinfo_serial;
ListCell *lc2;
/*
@@ -1823,6 +1824,16 @@ process_postponed_left_join_quals(PlannerInfo *root)
joins_below,
NULL);
+ /*
+ * Each time we produce RestrictInfo(s) from these quals, reset
+ * the last_rinfo_serial counter, so that the RestrictInfos for
+ * the "same" qual condition get identical serial numbers. (This
+ * relies on the fact that we're not changing the qual list in any
+ * way that'd affect the number of RestrictInfos built from it.)
+ * This'll allow us to detect duplicative qual usage later.
+ */
+ save_last_rinfo_serial = root->last_rinfo_serial;
+
joins_so_far = NULL;
foreach(lc2, join_info_list_orig)
{
@@ -1856,6 +1867,9 @@ process_postponed_left_join_quals(PlannerInfo *root)
continue;
}
+ /* Reset serial counter for this version of the quals */
+ root->last_rinfo_serial = save_last_rinfo_serial;
+
/*
* When we are looking at joins above sjinfo, we are
* envisioning pushing sjinfo to above othersj, so add
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index e743a5d9fe..d52c2a3595 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -620,6 +620,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
root->multiexpr_params = NIL;
root->eq_classes = NIL;
root->ec_merging_done = false;
+ root->last_rinfo_serial = 0;
root->all_result_relids =
parse->resultRelation ? bms_make_singleton(parse->resultRelation) : NULL;
root->leaf_result_relids = NULL; /* we'll find out leaf-ness later */
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 4c60a5858e..65436dd143 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -990,6 +990,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
subroot->multiexpr_params = NIL;
subroot->eq_classes = NIL;
subroot->ec_merging_done = false;
+ subroot->last_rinfo_serial = 0;
subroot->all_result_relids = NULL;
subroot->leaf_result_relids = NULL;
subroot->append_rel_list = NIL;
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index 11c6bbaba6..e18d64b6dc 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -427,7 +427,7 @@ adjust_appendrel_attrs_mutator(Node *node,
RestrictInfo *oldinfo = (RestrictInfo *) node;
RestrictInfo *newinfo = makeNode(RestrictInfo);
- /* Copy all flat-copiable fields */
+ /* Copy all flat-copiable fields, notably including rinfo_serial */
memcpy(newinfo, oldinfo, sizeof(RestrictInfo));
/* Recursively fix the clause itself */
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index bf35d1989c..c77399ca92 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2442,12 +2442,12 @@ create_nestloop_path(PlannerInfo *root,
* restrict_clauses that are due to be moved into the inner path. We have
* to do this now, rather than postpone the work till createplan time,
* because the restrict_clauses list can affect the size and cost
- * estimates for this path.
+ * estimates for this path. We detect such clauses by checking for serial
+ * number match to clauses already enforced in the inner path.
*/
if (bms_overlap(inner_req_outer, outer_path->parent->relids))
{
- Relids inner_and_outer = bms_union(inner_path->parent->relids,
- inner_req_outer);
+ Bitmapset *enforced_serials = get_param_path_clause_serials(inner_path);
List *jclauses = NIL;
ListCell *lc;
@@ -2455,9 +2455,7 @@ create_nestloop_path(PlannerInfo *root,
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
- if (!join_clause_is_movable_into(rinfo,
- inner_path->parent->relids,
- inner_and_outer))
+ if (!bms_is_member(rinfo->rinfo_serial, enforced_serials))
jclauses = lappend(jclauses, rinfo);
}
restrict_clauses = jclauses;
@@ -4268,6 +4266,7 @@ do { \
new_ppi->ppi_rows = old_ppi->ppi_rows;
new_ppi->ppi_clauses = old_ppi->ppi_clauses;
ADJUST_CHILD_ATTRS(new_ppi->ppi_clauses);
+ new_ppi->ppi_serials = bms_copy(old_ppi->ppi_serials);
rel->ppilist = lappend(rel->ppilist, new_ppi);
MemoryContextSwitchTo(oldcontext);
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 226914498a..cea298c633 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -1471,6 +1471,7 @@ get_baserel_parampathinfo(PlannerInfo *root, RelOptInfo *baserel,
ParamPathInfo *ppi;
Relids joinrelids;
List *pclauses;
+ Bitmapset *pserials;
double rows;
ListCell *lc;
@@ -1513,6 +1514,15 @@ get_baserel_parampathinfo(PlannerInfo *root, RelOptInfo *baserel,
required_outer,
baserel));
+ /* Compute set of serial numbers of the enforced clauses */
+ pserials = NULL;
+ foreach(lc, pclauses)
+ {
+ RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
+
+ pserials = bms_add_member(pserials, rinfo->rinfo_serial);
+ }
+
/* Estimate the number of rows returned by the parameterized scan */
rows = get_parameterized_baserel_size(root, baserel, pclauses);
@@ -1521,6 +1531,7 @@ get_baserel_parampathinfo(PlannerInfo *root, RelOptInfo *baserel,
ppi->ppi_req_outer = required_outer;
ppi->ppi_rows = rows;
ppi->ppi_clauses = pclauses;
+ ppi->ppi_serials = pserials;
baserel->ppilist = lappend(baserel->ppilist, ppi);
return ppi;
@@ -1746,6 +1757,7 @@ get_joinrel_parampathinfo(PlannerInfo *root, RelOptInfo *joinrel,
ppi->ppi_req_outer = required_outer;
ppi->ppi_rows = rows;
ppi->ppi_clauses = NIL;
+ ppi->ppi_serials = NULL;
joinrel->ppilist = lappend(joinrel->ppilist, ppi);
return ppi;
@@ -1784,6 +1796,7 @@ get_appendrel_parampathinfo(RelOptInfo *appendrel, Relids required_outer)
ppi->ppi_req_outer = required_outer;
ppi->ppi_rows = 0;
ppi->ppi_clauses = NIL;
+ ppi->ppi_serials = NULL;
appendrel->ppilist = lappend(appendrel->ppilist, ppi);
return ppi;
@@ -1809,6 +1822,100 @@ find_param_path_info(RelOptInfo *rel, Relids required_outer)
return NULL;
}
+/*
+ * get_param_path_clause_serials
+ * Given a parameterized Path, return the set of pushed-down clauses
+ * (identified by rinfo_serial numbers) enforced within the Path.
+ */
+Bitmapset *
+get_param_path_clause_serials(Path *path)
+{
+ if (path->param_info == NULL)
+ return NULL; /* not parameterized */
+ if (IsA(path, NestPath) ||
+ IsA(path, MergePath) ||
+ IsA(path, HashPath))
+ {
+ /*
+ * For a join path, combine clauses enforced within either input path
+ * with those enforced as joinrestrictinfo in this path. Note that
+ * joinrestrictinfo may include some non-pushed-down clauses, but for
+ * current purposes it's okay if we include those in the result. (To
+ * be more careful, we could check for clause_relids overlapping the
+ * path parameterization, but it's not worth the cycles for now.)
+ */
+ JoinPath *jpath = (JoinPath *) path;
+ Bitmapset *pserials;
+ ListCell *lc;
+
+ pserials = NULL;
+ pserials = bms_add_members(pserials,
+ get_param_path_clause_serials(jpath->outerjoinpath));
+ pserials = bms_add_members(pserials,
+ get_param_path_clause_serials(jpath->innerjoinpath));
+ foreach(lc, jpath->joinrestrictinfo)
+ {
+ RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
+
+ pserials = bms_add_member(pserials, rinfo->rinfo_serial);
+ }
+ return pserials;
+ }
+ else if (IsA(path, AppendPath))
+ {
+ /*
+ * For an appendrel, take the intersection of the sets of clauses
+ * enforced in each input path.
+ */
+ AppendPath *apath = (AppendPath *) path;
+ Bitmapset *pserials;
+ ListCell *lc;
+
+ pserials = NULL;
+ foreach(lc, apath->subpaths)
+ {
+ Path *subpath = (Path *) lfirst(lc);
+ Bitmapset *subserials;
+
+ subserials = get_param_path_clause_serials(subpath);
+ if (lc == list_head(apath->subpaths))
+ pserials = bms_copy(subserials);
+ else
+ pserials = bms_int_members(pserials, subserials);
+ }
+ return pserials;
+ }
+ else if (IsA(path, MergeAppendPath))
+ {
+ /* Same as AppendPath case */
+ MergeAppendPath *apath = (MergeAppendPath *) path;
+ Bitmapset *pserials;
+ ListCell *lc;
+
+ pserials = NULL;
+ foreach(lc, apath->subpaths)
+ {
+ Path *subpath = (Path *) lfirst(lc);
+ Bitmapset *subserials;
+
+ subserials = get_param_path_clause_serials(subpath);
+ if (lc == list_head(apath->subpaths))
+ pserials = bms_copy(subserials);
+ else
+ pserials = bms_int_members(pserials, subserials);
+ }
+ return pserials;
+ }
+ else
+ {
+ /*
+ * Otherwise, it's a baserel path and we can use the
+ * previously-computed set of serial numbers.
+ */
+ return path->param_info->ppi_serials;
+ }
+}
+
/*
* build_joinrel_partition_info
* Checks if the two relations being joined can use partitionwise join
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index 327c3ba563..bcbee8f943 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -208,6 +208,11 @@ make_restrictinfo_internal(PlannerInfo *root,
restrictinfo->num_base_rels = bms_num_members(baserels);
bms_free(baserels);
+ /*
+ * Label this RestrictInfo with a fresh serial number.
+ */
+ restrictinfo->rinfo_serial = ++(root->last_rinfo_serial);
+
/*
* Fill in all the cacheable fields with "not yet set" markers. None of
* these will be computed until/unless needed. Note in particular that we
@@ -371,7 +376,7 @@ commute_restrictinfo(RestrictInfo *rinfo, Oid comm_op)
* ... and adjust those we need to change. Note in particular that we can
* preserve any cached selectivity or cost estimates, since those ought to
* be the same for the new clause. Likewise we can keep the source's
- * parent_ec.
+ * parent_ec. It's also important that we keep the same rinfo_serial.
*/
result->clause = (Expr *) newclause;
result->left_relids = rinfo->right_relids;
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index a734676293..9d89b0c9eb 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -339,6 +339,9 @@ struct PlannerInfo
/* list of SpecialJoinInfos */
List *join_info_list;
+ /* counter for assigning RestrictInfo serial numbers */
+ int last_rinfo_serial;
+
/*
* all_result_relids is empty for SELECT, otherwise it contains at least
* parse->resultRelation. For UPDATE/DELETE/MERGE across an inheritance
@@ -1356,6 +1359,8 @@ typedef struct EquivalenceMember
bool em_is_const; /* expression is pseudoconstant? */
bool em_is_child; /* derived version for a child relation? */
Oid em_datatype; /* the "nominal type" used by the opfamily */
+ /* if em_is_child is true, this links to corresponding EM for top parent */
+ struct EquivalenceMember *em_parent pg_node_attr(read_write_ignore);
} EquivalenceMember;
/*
@@ -1461,7 +1466,13 @@ typedef struct PathTarget
* Note: ppi_clauses is only used in ParamPathInfos for base relation paths;
* in join cases it's NIL because the set of relevant clauses varies depending
* on how the join is formed. The relevant clauses will appear in each
- * parameterized join path's joinrestrictinfo list, instead.
+ * parameterized join path's joinrestrictinfo list, instead. ParamPathInfos
+ * for append relations don't bother with this, either.
+ *
+ * ppi_serials is the set of rinfo_serial numbers for quals that are enforced
+ * by this path. As with ppi_clauses, it's only maintained for baserels.
+ * (We could construct it on-the-fly from ppi_clauses, but it seems better
+ * to materialize a copy.)
*/
typedef struct ParamPathInfo
{
@@ -1472,6 +1483,7 @@ typedef struct ParamPathInfo
Relids ppi_req_outer; /* rels supplying parameters used by path */
Cardinality ppi_rows; /* estimated number of result tuples */
List *ppi_clauses; /* join clauses available from outer rels */
+ Bitmapset *ppi_serials; /* set of rinfo_serial for enforced quals */
} ParamPathInfo;
@@ -2501,6 +2513,25 @@ typedef struct RestrictInfo
*/
Expr *orclause pg_node_attr(equal_ignore);
+ /*----------
+ * Serial number of this RestrictInfo. This is unique within the current
+ * PlannerInfo context, with a few critical exceptions:
+ * 1. When we generate multiple clones of the same qual condition to
+ * cope with outer join identity 3, all the clones get the same serial
+ * number. This reflects that we only want to apply one of them in any
+ * given plan.
+ * 2. If we manufacture a commuted version of a qual to use as an index
+ * condition, it copies the original's rinfo_serial, since it is in
+ * practice the same condition.
+ * 3. RestrictInfos made for a child relation copy their parent's
+ * rinfo_serial. Likewise, when an EquivalenceClass makes a derived
+ * equality clause for a child relation, it copies the rinfo_serial of
+ * the matching equality clause for the parent. This allows detection
+ * of redundant pushed-down equality clauses.
+ *----------
+ */
+ int rinfo_serial;
+
/*
* Generating EquivalenceClass. This field is NULL unless clause is
* potentially redundant.
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 197234d44c..3440455a2e 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -333,6 +333,7 @@ extern ParamPathInfo *get_appendrel_parampathinfo(RelOptInfo *appendrel,
Relids required_outer);
extern ParamPathInfo *find_param_path_info(RelOptInfo *rel,
Relids required_outer);
+extern Bitmapset *get_param_path_clause_serials(Path *path);
extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out
index 9b69a8c122..340c57a270 100644
--- a/src/test/regress/expected/join.out
+++ b/src/test/regress/expected/join.out
@@ -2335,17 +2335,17 @@ select a.f1, b.f1, t.thousand, t.tenthous from
(select sum(f1)+1 as f1 from int4_tbl i4a) a,
(select sum(f1) as f1 from int4_tbl i4b) b
where b.f1 = t.thousand and a.f1 = b.f1 and (a.f1+b.f1+999) = t.tenthous;
- QUERY PLAN
------------------------------------------------------------------------------------------------------------------------
+ QUERY PLAN
+-----------------------------------------------------------------------------------------------------------------
Nested Loop
- -> Aggregate
- -> Seq Scan on int4_tbl i4b
-> Nested Loop
Join Filter: ((sum(i4b.f1)) = ((sum(i4a.f1) + 1)))
-> Aggregate
-> Seq Scan on int4_tbl i4a
- -> Index Only Scan using tenk1_thous_tenthous on tenk1 t
- Index Cond: ((thousand = (sum(i4b.f1))) AND (tenthous = ((((sum(i4a.f1) + 1)) + (sum(i4b.f1))) + 999)))
+ -> Aggregate
+ -> Seq Scan on int4_tbl i4b
+ -> Index Only Scan using tenk1_thous_tenthous on tenk1 t
+ Index Cond: ((thousand = (sum(i4b.f1))) AND (tenthous = ((((sum(i4a.f1) + 1)) + (sum(i4b.f1))) + 999)))
(9 rows)
select a.f1, b.f1, t.thousand, t.tenthous from
diff --git a/src/test/regress/expected/partition_join.out b/src/test/regress/expected/partition_join.out
index b20facc19f..bb5b7c47a4 100644
--- a/src/test/regress/expected/partition_join.out
+++ b/src/test/regress/expected/partition_join.out
@@ -304,7 +304,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t2.b FROM prt2 t2 WHERE t2.a = 0)
-> Seq Scan on prt2_p2 t2_2
Filter: (a = 0)
-> Nested Loop Semi Join
- Join Filter: (t2_3.b = t1_3.a)
+ Join Filter: (t1_3.a = t2_3.b)
-> Seq Scan on prt1_p3 t1_3
Filter: (b = 0)
-> Materialize
@@ -601,7 +601,7 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM prt1 t1, prt2 t2, prt1_e t
Sort Key: t1.a
-> Append
-> Nested Loop
- Join Filter: (((t3_1.a + t3_1.b) / 2) = t1_1.a)
+ Join Filter: (t1_1.a = ((t3_1.a + t3_1.b) / 2))
-> Hash Join
Hash Cond: (t2_1.b = t1_1.a)
-> Seq Scan on prt2_p1 t2_1
@@ -611,7 +611,7 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM prt1 t1, prt2 t2, prt1_e t
-> Index Scan using iprt1_e_p1_ab2 on prt1_e_p1 t3_1
Index Cond: (((a + b) / 2) = t2_1.b)
-> Nested Loop
- Join Filter: (((t3_2.a + t3_2.b) / 2) = t1_2.a)
+ Join Filter: (t1_2.a = ((t3_2.a + t3_2.b) / 2))
-> Hash Join
Hash Cond: (t2_2.b = t1_2.a)
-> Seq Scan on prt2_p2 t2_2
@@ -621,7 +621,7 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM prt1 t1, prt2 t2, prt1_e t
-> Index Scan using iprt1_e_p2_ab2 on prt1_e_p2 t3_2
Index Cond: (((a + b) / 2) = t2_2.b)
-> Nested Loop
- Join Filter: (((t3_3.a + t3_3.b) / 2) = t1_3.a)
+ Join Filter: (t1_3.a = ((t3_3.a + t3_3.b) / 2))
-> Hash Join
Hash Cond: (t2_3.b = t1_3.a)
-> Seq Scan on prt2_p3 t2_3
@@ -926,7 +926,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t1.b FROM prt2 t1, prt1_e t2 WHER
Sort Key: t1.a
-> Append
-> Nested Loop
- Join Filter: (t1_5.b = t1_2.a)
+ Join Filter: (t1_2.a = t1_5.b)
-> HashAggregate
Group Key: t1_5.b
-> Hash Join
@@ -939,7 +939,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t1.b FROM prt2 t1, prt1_e t2 WHER
Index Cond: (a = ((t2_1.a + t2_1.b) / 2))
Filter: (b = 0)
-> Nested Loop
- Join Filter: (t1_6.b = t1_3.a)
+ Join Filter: (t1_3.a = t1_6.b)
-> HashAggregate
Group Key: t1_6.b
-> Hash Join
@@ -952,7 +952,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t1.b FROM prt2 t1, prt1_e t2 WHER
Index Cond: (a = ((t2_2.a + t2_2.b) / 2))
Filter: (b = 0)
-> Nested Loop
- Join Filter: (t1_7.b = t1_4.a)
+ Join Filter: (t1_4.a = t1_7.b)
-> HashAggregate
Group Key: t1_7.b
-> Nested Loop
commit 3cd087cbaeb3e7be4b0872d49f211083eed1f3b0
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Oct 31 12:39:31 2022 -0400
Fix flatten_join_alias_vars() to handle varnullingrels correctly.
The remaining core regression test failures occur because
flatten_join_alias_vars() isn't doing the right thing. The
alias Var it needs to replace may have acquired varnullingrels
bits signifying the effect of upper outer joins, and if so we
must preserve that information in the replacement expression.
The simplest way to do that is to wrap the replacement expression
in a PlaceHolderVar, and that's what we have to do in the general
case where subquery pullup has mutated the replacement joinaliasvars
entry into an arbitrary expression. But in simpler cases, such as
where the joinaliasvars entry is just a Var, we'd prefer to do it
by merging the alias Var's varnullingrels into the replacement Var.
In that way the flattened alias will compare equal() to semantically
equivalent references that didn't use the alias name.
Moreover, the parser also uses this code while checking certain
semantic constraints, and in that context we *must not* generate
PlaceHolderVars. PHVs shouldn't appear in parse-time expressions,
and adding one would certainly cause the parser to decide the
query is invalid (because the result wouldn't compare equal() to
what it needs to). Fortunately, during parsing the set of possible
contents of a joinaliasvars entry is quite constrained, so we can
guarantee to apply the nullingrels info to the Vars therein.
The result of this step passes all core regression tests, but there
are still loose ends for FDWs (so that contrib/postgres_fdw will fail).
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index d52c2a3595..dc089306ae 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -901,7 +901,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
*/
if (rte->lateral && root->hasJoinRTEs)
rte->subquery = (Query *)
- flatten_join_alias_vars(root->parse,
+ flatten_join_alias_vars(root, root->parse,
(Node *) rte->subquery);
}
else if (rte->rtekind == RTE_FUNCTION)
@@ -1102,7 +1102,7 @@ preprocess_expression(PlannerInfo *root, Node *expr, int kind)
kind == EXPRKIND_VALUES ||
kind == EXPRKIND_TABLESAMPLE ||
kind == EXPRKIND_TABLEFUNC))
- expr = flatten_join_alias_vars(root->parse, expr);
+ expr = flatten_join_alias_vars(root, root->parse, expr);
/*
* Simplify constant expressions. For function RTEs, this was already
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 65436dd143..cf558264eb 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -1078,7 +1078,8 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* maybe even in the rewriter; but for now let's just fix this case here.)
*/
subquery->targetList = (List *)
- flatten_join_alias_vars(subroot->parse, (Node *) subquery->targetList);
+ flatten_join_alias_vars(subroot, subroot->parse,
+ (Node *) subquery->targetList);
/*
* Adjust level-0 varnos in subquery so that we can append its rangetable
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
index 8d8c9136f8..69c2019553 100644
--- a/src/backend/optimizer/util/var.c
+++ b/src/backend/optimizer/util/var.c
@@ -62,6 +62,7 @@ typedef struct
typedef struct
{
+ PlannerInfo *root; /* could be NULL! */
Query *query; /* outer Query */
int sublevels_up;
bool possible_sublink; /* could aliases include a SubLink? */
@@ -80,6 +81,10 @@ static bool pull_var_clause_walker(Node *node,
pull_var_clause_context *context);
static Node *flatten_join_alias_vars_mutator(Node *node,
flatten_join_alias_vars_context *context);
+static Node *add_nullingrels_if_needed(PlannerInfo *root, Node *newnode,
+ Var *oldvar);
+static bool is_standard_join_alias_expression(Node *newnode, Var *oldvar);
+static void adjust_standard_join_alias_expression(Node *newnode, Var *oldvar);
static Relids alias_relid_set(Query *query, Relids relids);
@@ -722,26 +727,42 @@ pull_var_clause_walker(Node *node, pull_var_clause_context *context)
* is the only way that the executor can directly handle whole-row Vars.
*
* This also adjusts relid sets found in some expression node types to
- * substitute the contained base rels for any join relid.
+ * substitute the contained base+OJ rels for any join relid.
*
* If a JOIN contains sub-selects that have been flattened, its join alias
* entries might now be arbitrary expressions, not just Vars. This affects
- * this function in one important way: we might find ourselves inserting
- * SubLink expressions into subqueries, and we must make sure that their
- * Query.hasSubLinks fields get set to true if so. If there are any
+ * this function in two important ways. First, we might find ourselves
+ * inserting SubLink expressions into subqueries, and we must make sure that
+ * their Query.hasSubLinks fields get set to true if so. If there are any
* SubLinks in the join alias lists, the outer Query should already have
* hasSubLinks = true, so this is only relevant to un-flattened subqueries.
+ * Second, we have to preserve any varnullingrels info attached to the
+ * alias Vars we're replacing. If the replacement expression is a Var or
+ * PlaceHolderVar or constructed from those, we can just add the
+ * varnullingrels bits to the existing nullingrels field(s); otherwise
+ * we have to add a PlaceHolderVar wrapper.
*
- * NOTE: this is used on not-yet-planned expressions. We do not expect it
- * to be applied directly to the whole Query, so if we see a Query to start
- * with, we do want to increment sublevels_up (this occurs for LATERAL
- * subqueries).
+ * NOTE: this is also used by the parser, to expand join alias Vars before
+ * checking GROUP BY validity. For that use-case, root will be NULL, which
+ * is why we have to pass the Query separately. We need the root itself only
+ * for making PlaceHolderVars. We can avoid making PlaceHolderVars in the
+ * parser's usage because it won't be dealing with arbitrary expressions:
+ * so long as adjust_standard_join_alias_expression can handle everything
+ * the parser would make as a join alias expression, we're OK.
*/
Node *
-flatten_join_alias_vars(Query *query, Node *node)
+flatten_join_alias_vars(PlannerInfo *root, Query *query, Node *node)
{
flatten_join_alias_vars_context context;
+ /*
+ * We do not expect this to be applied to the whole Query, only to
+ * expressions or LATERAL subqueries. Hence, if the top node is a Query,
+ * it's okay to immediately increment sublevels_up.
+ */
+ Assert(node != (Node *) query);
+
+ context.root = root;
context.query = query;
context.sublevels_up = 0;
/* flag whether join aliases could possibly contain SubLinks */
@@ -812,7 +833,9 @@ flatten_join_alias_vars_mutator(Node *node,
rowexpr->colnames = colnames;
rowexpr->location = var->location;
- return (Node *) rowexpr;
+ /* Lastly, add any varnullingrels to the replacement expression */
+ return add_nullingrels_if_needed(context->root, (Node *) rowexpr,
+ var);
}
/* Expand join alias reference */
@@ -839,7 +862,8 @@ flatten_join_alias_vars_mutator(Node *node,
if (context->possible_sublink && !context->inserted_sublink)
context->inserted_sublink = checkExprHasSubLink(newvar);
- return newvar;
+ /* Lastly, add any varnullingrels to the replacement expression */
+ return add_nullingrels_if_needed(context->root, newvar, var);
}
if (IsA(node, PlaceHolderVar))
{
@@ -854,6 +878,7 @@ flatten_join_alias_vars_mutator(Node *node,
{
phv->phrels = alias_relid_set(context->query,
phv->phrels);
+ /* we *don't* change phnullingrels */
}
return (Node *) phv;
}
@@ -887,9 +912,145 @@ flatten_join_alias_vars_mutator(Node *node,
(void *) context);
}
+/*
+ * Add oldvar's varnullingrels, if any, to a flattened join alias expression.
+ * The newnode has been copied, so we can modify it freely.
+ */
+static Node *
+add_nullingrels_if_needed(PlannerInfo *root, Node *newnode, Var *oldvar)
+{
+ if (oldvar->varnullingrels == NULL)
+ return newnode; /* nothing to do */
+ /* If possible, do it by adding to existing nullingrel fields */
+ if (is_standard_join_alias_expression(newnode, oldvar))
+ adjust_standard_join_alias_expression(newnode, oldvar);
+ else if (root)
+ {
+ /* We can insert a PlaceHolderVar to carry the nullingrels */
+ PlaceHolderVar *newphv;
+ Relids phrels = pull_varnos(root, newnode);
+
+ /* XXX what if phrels is empty? */
+ Assert(!bms_is_empty(phrels)); /* probably wrong */
+ newphv = make_placeholder_expr(root, (Expr *) newnode, phrels);
+ /* newphv has zero phlevelsup and NULL phnullingrels; fix it */
+ newphv->phlevelsup = oldvar->varlevelsup;
+ newphv->phnullingrels = bms_copy(oldvar->varnullingrels);
+ newnode = (Node *) newphv;
+ }
+ else
+ {
+ /* ooops, we're missing support for something the parser can make */
+ elog(ERROR, "unsupported join alias expression");
+ }
+ return newnode;
+}
+
+/*
+ * Check to see if we can insert nullingrels into this join alias expression
+ * without use of a separate PlaceHolderVar.
+ *
+ * This will handle Vars, PlaceHolderVars, and implicit-coercion and COALESCE
+ * expressions built from those. This coverage needs to handle anything
+ * that the parser would put into joinaliasvars.
+ * XXX it's probably incomplete at the moment.
+ */
+static bool
+is_standard_join_alias_expression(Node *newnode, Var *oldvar)
+{
+ if (newnode == NULL)
+ return false;
+ if (IsA(newnode, Var) &&
+ ((Var *) newnode)->varlevelsup == oldvar->varlevelsup)
+ return true;
+ else if (IsA(newnode, PlaceHolderVar) &&
+ ((PlaceHolderVar *) newnode)->phlevelsup == oldvar->varlevelsup)
+ return true;
+ else if (IsA(newnode, FuncExpr))
+ {
+ FuncExpr *fexpr = (FuncExpr *) newnode;
+
+ /*
+ * We need to assume that the function wouldn't produce non-NULL from
+ * NULL, which is reasonable for implicit coercions but otherwise not
+ * so much. (Looking at its strictness is likely overkill, and anyway
+ * it would cause us to fail if someone forgot to mark an implicit
+ * coercion as strict.)
+ */
+ if (fexpr->funcformat != COERCE_IMPLICIT_CAST ||
+ fexpr->args == NIL)
+ return false;
+
+ /*
+ * Examine only the first argument --- coercions might have additional
+ * arguments that are constants.
+ */
+ return is_standard_join_alias_expression(linitial(fexpr->args), oldvar);
+ }
+ else if (IsA(newnode, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) newnode;
+ ListCell *lc;
+
+ Assert(cexpr->args != NIL);
+ foreach(lc, cexpr->args)
+ {
+ if (!is_standard_join_alias_expression(lfirst(lc), oldvar))
+ return false;
+ }
+ return true;
+ }
+ else
+ return false;
+}
+
+/*
+ * Insert nullingrels into an expression accepted by
+ * is_standard_join_alias_expression.
+ */
+static void
+adjust_standard_join_alias_expression(Node *newnode, Var *oldvar)
+{
+ if (IsA(newnode, Var) &&
+ ((Var *) newnode)->varlevelsup == oldvar->varlevelsup)
+ {
+ Var *newvar = (Var *) newnode;
+
+ newvar->varnullingrels = bms_add_members(newvar->varnullingrels,
+ oldvar->varnullingrels);
+ }
+ else if (IsA(newnode, PlaceHolderVar) &&
+ ((PlaceHolderVar *) newnode)->phlevelsup == oldvar->varlevelsup)
+ {
+ PlaceHolderVar *newphv = (PlaceHolderVar *) newnode;
+
+ newphv->phnullingrels = bms_add_members(newphv->phnullingrels,
+ oldvar->varnullingrels);
+ }
+ else if (IsA(newnode, FuncExpr))
+ {
+ FuncExpr *fexpr = (FuncExpr *) newnode;
+
+ adjust_standard_join_alias_expression(linitial(fexpr->args), oldvar);
+ }
+ else if (IsA(newnode, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) newnode;
+ ListCell *lc;
+
+ Assert(cexpr->args != NIL);
+ foreach(lc, cexpr->args)
+ {
+ adjust_standard_join_alias_expression(lfirst(lc), oldvar);
+ }
+ }
+ else
+ Assert(false);
+}
+
/*
* alias_relid_set: in a set of RT indexes, replace joins by their
- * underlying base relids
+ * underlying base+OJ relids
*/
static Relids
alias_relid_set(Query *query, Relids relids)
diff --git a/src/backend/parser/parse_agg.c b/src/backend/parser/parse_agg.c
index 3ef9e8ee5e..c15fab0f68 100644
--- a/src/backend/parser/parse_agg.c
+++ b/src/backend/parser/parse_agg.c
@@ -1162,7 +1162,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
* entries are RTE_JOIN kind.
*/
if (hasJoinRTEs)
- groupClauses = (List *) flatten_join_alias_vars(qry,
+ groupClauses = (List *) flatten_join_alias_vars(NULL, qry,
(Node *) groupClauses);
/*
@@ -1206,7 +1206,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
groupClauses, hasJoinRTEs,
have_non_var_grouping);
if (hasJoinRTEs)
- clause = flatten_join_alias_vars(qry, clause);
+ clause = flatten_join_alias_vars(NULL, qry, clause);
check_ungrouped_columns(clause, pstate, qry,
groupClauses, groupClauseCommonVars,
have_non_var_grouping,
@@ -1217,7 +1217,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
groupClauses, hasJoinRTEs,
have_non_var_grouping);
if (hasJoinRTEs)
- clause = flatten_join_alias_vars(qry, clause);
+ clause = flatten_join_alias_vars(NULL, qry, clause);
check_ungrouped_columns(clause, pstate, qry,
groupClauses, groupClauseCommonVars,
have_non_var_grouping,
@@ -1546,7 +1546,7 @@ finalize_grouping_exprs_walker(Node *node,
Index ref = 0;
if (context->hasJoinRTEs)
- expr = flatten_join_alias_vars(context->qry, expr);
+ expr = flatten_join_alias_vars(NULL, context->qry, expr);
/*
* Each expression must match a grouping entry at the current
diff --git a/src/include/optimizer/optimizer.h b/src/include/optimizer/optimizer.h
index 409005bae9..95f3461a3d 100644
--- a/src/include/optimizer/optimizer.h
+++ b/src/include/optimizer/optimizer.h
@@ -197,6 +197,6 @@ extern bool contain_var_clause(Node *node);
extern bool contain_vars_of_level(Node *node, int levelsup);
extern int locate_var_of_level(Node *node, int levelsup);
extern List *pull_var_clause(Node *node, int flags);
-extern Node *flatten_join_alias_vars(Query *query, Node *node);
+extern Node *flatten_join_alias_vars(PlannerInfo *root, Query *query, Node *node);
#endif /* OPTIMIZER_H */
commit eb118a954e4afbae9c32a8079b0525c6fd28ce7a
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Oct 31 12:54:09 2022 -0400
Teach FDWs about base-plus-outer-join relids.
Conversion of the planner to include OJ relids in join relids
affects FDWs that want to plan foreign joins. They *must* follow
suit when labeling foreign joins in order to match with the core
planner, but for many purposes (if postgres_fdw is any guide)
they'd prefer to consider only base relations within the join.
To support both requirements, redefine ForeignScan.fs_relids as
base+OJ relids, and add a new field fs_base_relids that's set up
by the core planner.
Another way we could do this is to keep fs_relids as just base
relids and make the new field be the one with OJ relids added.
While that would be more backwards-compatible in some sense,
it would be inconsistent with the naming used in the core planner,
and I think that it might allow some types of bugs to escape
quick detection.
postgres_fdw also has one place where it needs to ignore varnullingrels
while matching Vars. It's not clear whether it's worth trying to
improve that. (This too is probably only an issue for FDWs that do
join planning, since Vars seen in a base relation scan should never
have any varnullingrels.)
As of this step, this patch series passes check-world.
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
index 9524765650..94dd7b2c96 100644
--- a/contrib/postgres_fdw/deparse.c
+++ b/contrib/postgres_fdw/deparse.c
@@ -3950,7 +3950,17 @@ get_relation_column_alias_ids(Var *node, RelOptInfo *foreignrel,
i = 1;
foreach(lc, foreignrel->reltarget->exprs)
{
- if (equal(lfirst(lc), (Node *) node))
+ Var *tlvar = (Var *) lfirst(lc);
+
+ /*
+ * Match reltarget entries only on varno/varattno. Ideally there
+ * would be some cross-check on varnullingrels, but it's unclear what
+ * to do exactly; we don't have enough context to know what that value
+ * should be.
+ */
+ if (IsA(tlvar, Var) &&
+ tlvar->varno == node->varno &&
+ tlvar->varattno == node->varattno)
{
*colno = i;
return;
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 8d7500abfb..39cc37053c 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -1511,13 +1511,13 @@ postgresBeginForeignScan(ForeignScanState *node, int eflags)
/*
* Identify which user to do the remote access as. This should match what
* ExecCheckRTEPerms() does. In case of a join or aggregate, use the
- * lowest-numbered member RTE as a representative; we would get the same
- * result from any.
+ * lowest-numbered member base RTE as a representative; we would get the
+ * same result from any.
*/
if (fsplan->scan.scanrelid > 0)
rtindex = fsplan->scan.scanrelid;
else
- rtindex = bms_next_member(fsplan->fs_relids, -1);
+ rtindex = bms_next_member(fsplan->fs_base_relids, -1);
rte = exec_rt_fetch(rtindex, estate);
userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
@@ -2414,7 +2414,7 @@ find_modifytable_subplan(PlannerInfo *root,
{
ForeignScan *fscan = (ForeignScan *) subplan;
- if (bms_is_member(rtindex, fscan->fs_relids))
+ if (bms_is_member(rtindex, fscan->fs_base_relids))
return fscan;
}
@@ -2840,8 +2840,8 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
* that setrefs.c won't update the string when flattening the
* rangetable. To find out what rtoffset was applied, identify the
* minimum RT index appearing in the string and compare it to the
- * minimum member of plan->fs_relids. (We expect all the relids in
- * the join will have been offset by the same amount; the Asserts
+ * minimum member of plan->fs_base_relids. (We expect all the relids
+ * in the join will have been offset by the same amount; the Asserts
* below should catch it if that ever changes.)
*/
minrti = INT_MAX;
@@ -2858,7 +2858,7 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
else
ptr++;
}
- rtoffset = bms_next_member(plan->fs_relids, -1) - minrti;
+ rtoffset = bms_next_member(plan->fs_base_relids, -1) - minrti;
/* Now we can translate the string */
relations = makeStringInfo();
@@ -2873,7 +2873,7 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
char *refname;
rti += rtoffset;
- Assert(bms_is_member(rti, plan->fs_relids));
+ Assert(bms_is_member(rti, plan->fs_base_relids));
rte = rt_fetch(rti, es->rtable);
Assert(rte->rtekind == RTE_RELATION);
/* This logic should agree with explain.c's ExplainTargetRel */
diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml
index 94263c628f..ac1717bc3c 100644
--- a/doc/src/sgml/fdwhandler.sgml
+++ b/doc/src/sgml/fdwhandler.sgml
@@ -351,6 +351,17 @@ GetForeignJoinPaths(PlannerInfo *root,
it will supply at run time in the tuples it returns.
</para>
+ <note>
+ <para>
+ Beginning with <productname>PostgreSQL</productname> 16,
+ <structfield>fs_relids</structfield> includes the rangetable indexes
+ of outer joins, if any were involved in this join. The new field
+ <structfield>fs_base_relids</structfield> includes only base
+ relation indexes, and thus
+ mimics <structfield>fs_relids</structfield>'s old semantics.
+ </para>
+ </note>
+
<para>
See <xref linkend="fdw-planning"/> for additional information.
</para>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index f86983c660..ed9a118416 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1114,7 +1114,7 @@ ExplainPreScanNode(PlanState *planstate, Bitmapset **rels_used)
break;
case T_ForeignScan:
*rels_used = bms_add_members(*rels_used,
- ((ForeignScan *) plan)->fs_relids);
+ ((ForeignScan *) plan)->fs_base_relids);
break;
case T_CustomScan:
*rels_used = bms_add_members(*rels_used,
diff --git a/src/backend/executor/execScan.c b/src/backend/executor/execScan.c
index 043bb83f55..2b37266b6a 100644
--- a/src/backend/executor/execScan.c
+++ b/src/backend/executor/execScan.c
@@ -325,7 +325,7 @@ ExecScanReScan(ScanState *node)
* all of them.
*/
if (IsA(node->ps.plan, ForeignScan))
- relids = ((ForeignScan *) node->ps.plan)->fs_relids;
+ relids = ((ForeignScan *) node->ps.plan)->fs_base_relids;
else if (IsA(node->ps.plan, CustomScan))
relids = ((CustomScan *) node->ps.plan)->custom_relids;
else
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index ac86ce9003..13f46e4f23 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -4153,14 +4153,22 @@ create_foreignscan_plan(PlannerInfo *root, ForeignPath *best_path,
/*
* Likewise, copy the relids that are represented by this foreign scan. An
- * upper rel doesn't have relids set, but it covers all the base relations
- * participating in the underlying scan, so use root's all_baserels.
+ * upper rel doesn't have relids set, but it covers all the relations
+ * participating in the underlying scan/join, so use root->all_query_rels.
*/
if (rel->reloptkind == RELOPT_UPPER_REL)
- scan_plan->fs_relids = root->all_baserels;
+ scan_plan->fs_relids = root->all_query_rels;
else
scan_plan->fs_relids = best_path->path.parent->relids;
+ /*
+ * Join relid sets include relevant outer joins, but FDWs may need to know
+ * which are the included base rels. That's a bit tedious to get without
+ * access to the plan-time data structures, so compute it here.
+ */
+ scan_plan->fs_base_relids = bms_difference(scan_plan->fs_relids,
+ root->outer_join_rels);
+
/*
* If this is a foreign join, and to make it valid to push down we had to
* assume that the current user is the same as some user explicitly named
@@ -5800,8 +5808,9 @@ make_foreignscan(List *qptlist,
node->fdw_private = fdw_private;
node->fdw_scan_tlist = fdw_scan_tlist;
node->fdw_recheck_quals = fdw_recheck_quals;
- /* fs_relids will be filled in by create_foreignscan_plan */
+ /* fs_relids, fs_base_relids will be filled by create_foreignscan_plan */
node->fs_relids = NULL;
+ node->fs_base_relids = NULL;
/* fsSystemCol will be filled in by create_foreignscan_plan */
node->fsSystemCol = false;
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 8fff731756..6ed6b950a4 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1560,6 +1560,7 @@ set_foreignscan_references(PlannerInfo *root,
}
fscan->fs_relids = offset_relid_set(fscan->fs_relids, rtoffset);
+ fscan->fs_base_relids = offset_relid_set(fscan->fs_base_relids, rtoffset);
/* Adjust resultRelation if it's valid */
if (fscan->resultRelation > 0)
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 5c2ab1b379..25bc3e61eb 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -689,6 +689,7 @@ typedef struct WorkTableScan
* When the plan node represents a foreign join, scan.scanrelid is zero and
* fs_relids must be consulted to identify the join relation. (fs_relids
* is valid for simple scans as well, but will always match scan.scanrelid.)
+ * fs_relids includes outer joins; fs_base_relids does not.
*
* If the FDW's PlanDirectModify() callback decides to repurpose a ForeignScan
* node to perform the UPDATE or DELETE operation directly in the remote
@@ -708,7 +709,8 @@ typedef struct ForeignScan
List *fdw_private; /* private data for FDW */
List *fdw_scan_tlist; /* optional tlist describing scan tuple */
List *fdw_recheck_quals; /* original quals not in scan.plan.qual */
- Bitmapset *fs_relids; /* RTIs generated by this scan */
+ Bitmapset *fs_relids; /* base+OJ RTIs generated by this scan */
+ Bitmapset *fs_base_relids; /* base RTIs generated by this scan */
bool fsSystemCol; /* true if any "system column" is needed */
} ForeignScan;
commit b602befb346a13edbbb883c4beac901849fe5d65
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Oct 31 13:03:47 2022 -0400
Don't use RestrictInfo.nullable_relids in join_clause_is_movable_to.
Instead of using per-clause nullable_relids data, compute a
per-baserel set of outer joins that can null each relation, and
check for overlap between that and clause_relids to detect whether
the clause can safely be pushed down to relation scan level.
join_clause_is_movable_into also uses nullable_relids, but it
turns out that that test can just be dropped entirely. Now that
clause_relids includes nulling outer joins, the preceding tests
in the function are sufficient to reject clauses that can't be
pushed down.
This might seem like a net loss given that we have to add a bit
of code to initsplan.c to compute RelOptInfo.nulling_relids.
However, that's not much code at all, and the payoff is this:
we no longer need RestrictInfo.nullable_relids at all.
The next patch, which removes that field and the extensive
infrastructure that maintains it, saves way more code and cycles
than we add here. Also, I think there are likely going to be
other uses for RelOptInfo.nulling_relids.
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 96e8033930..364c26badf 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -57,6 +57,8 @@ static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
static void process_security_barrier_quals(PlannerInfo *root,
int rti, Relids qualscope,
bool below_outer_join);
+static void mark_rels_nulled_by_join(PlannerInfo *root, Index ojrelid,
+ Relids lower_rels);
static SpecialJoinInfo *make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
@@ -963,6 +965,7 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
*qualscope = bms_add_member(*qualscope, j->rtindex);
root->outer_join_rels = bms_add_member(root->outer_join_rels,
j->rtindex);
+ mark_rels_nulled_by_join(root, j->rtindex, rightids);
}
*inner_join_rels = bms_union(left_inners, right_inners);
nonnullable_rels = leftids;
@@ -1005,6 +1008,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
*qualscope = bms_add_member(*qualscope, j->rtindex);
root->outer_join_rels = bms_add_member(root->outer_join_rels,
j->rtindex);
+ mark_rels_nulled_by_join(root, j->rtindex, leftids);
+ mark_rels_nulled_by_join(root, j->rtindex, rightids);
*inner_join_rels = bms_union(left_inners, right_inners);
/* each side is both outer and inner */
nonnullable_rels = *qualscope;
@@ -1221,6 +1226,33 @@ process_security_barrier_quals(PlannerInfo *root,
Assert(security_level <= root->qual_security_level);
}
+/*
+ * mark_rels_nulled_by_join
+ * Fill RelOptInfo.nulling_relids of baserels nulled by this outer join
+ *
+ * Inputs:
+ * ojrelid: RT index of the join RTE (must not be 0)
+ * lower_rels: the base+OJ Relids syntactically below nullable side of join
+ */
+static void
+mark_rels_nulled_by_join(PlannerInfo *root, Index ojrelid,
+ Relids lower_rels)
+{
+ int relid = -1;
+
+ while ((relid = bms_next_member(lower_rels, relid)) > 0)
+ {
+ RelOptInfo *rel = root->simple_rel_array[relid];
+
+ if (rel == NULL) /* must be an outer join */
+ {
+ Assert(bms_is_member(relid, root->outer_join_rels));
+ continue;
+ }
+ rel->nulling_relids = bms_add_member(rel->nulling_relids, ojrelid);
+ }
+}
+
/*
* make_outerjoininfo
* Build a SpecialJoinInfo for the current outer join
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index cea298c633..3dd9317320 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -277,6 +277,12 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
rel->top_parent = parent->top_parent ? parent->top_parent : parent;
rel->top_parent_relids = rel->top_parent->relids;
+ /*
+ * A child rel is below the same outer joins as its parent. (We
+ * presume this info was already calculated for the parent.)
+ */
+ rel->nulling_relids = parent->nulling_relids;
+
/*
* Also propagate lateral-reference information from appendrel parent
* rels to their child rels. We intentionally give each child rel the
@@ -300,6 +306,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
rel->parent = NULL;
rel->top_parent = NULL;
rel->top_parent_relids = NULL;
+ rel->nulling_relids = NULL;
rel->direct_lateral_relids = NULL;
rel->lateral_relids = NULL;
rel->lateral_referencers = NULL;
@@ -680,6 +687,7 @@ build_join_rel(PlannerInfo *root,
joinrel->max_attr = 0;
joinrel->attr_needed = NULL;
joinrel->attr_widths = NULL;
+ joinrel->nulling_relids = NULL;
joinrel->lateral_vars = NIL;
joinrel->lateral_referencers = NULL;
joinrel->indexlist = NIL;
@@ -869,6 +877,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->max_attr = 0;
joinrel->attr_needed = NULL;
joinrel->attr_widths = NULL;
+ joinrel->nulling_relids = NULL;
joinrel->lateral_vars = NIL;
joinrel->lateral_referencers = NULL;
joinrel->indexlist = NIL;
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index bcbee8f943..15f410cf36 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -618,8 +618,17 @@ join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
if (bms_is_member(baserel->relid, rinfo->outer_relids))
return false;
- /* Target rel must not be nullable below the clause */
- if (bms_is_member(baserel->relid, rinfo->nullable_relids))
+ /*
+ * Target rel's Vars must not be nulled by any outer join. We can check
+ * this without groveling through the individual Vars by seeing whether
+ * clause_relids (which includes all such Vars' varnullingrels) includes
+ * any outer join that can null the target rel. You might object that
+ * this could reject the clause on the basis of an OJ relid that came from
+ * some other rel's Var. However, that would still mean that the clause
+ * came from above that outer join and shouldn't be pushed down; so there
+ * should be no false positives.
+ */
+ if (bms_overlap(rinfo->clause_relids, baserel->nulling_relids))
return false;
/* Clause must not use any rels with LATERAL references to this rel */
@@ -651,16 +660,17 @@ join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
* relation plus the outer rels. We also check that it does reference at
* least one current Var, ensuring that the clause will be pushed down to
* a unique place in a parameterized join tree. And we check that we're
- * not pushing the clause into its outer-join outer side, nor down into
- * a lower outer join's inner side.
- *
- * The check about pushing a clause down into a lower outer join's inner side
- * is only approximate; it sometimes returns "false" when actually it would
- * be safe to use the clause here because we're still above the outer join
- * in question. This is okay as long as the answers at different join levels
- * are consistent: it just means we might sometimes fail to push a clause as
- * far down as it could safely be pushed. It's unclear whether it would be
- * worthwhile to do this more precisely. (But if it's ever fixed to be
+ * not pushing the clause into its outer-join outer side.
+ *
+ * We used to need to check that we're not pushing the clause into a lower
+ * outer join's inner side. However, now that clause_relids includes
+ * references to potentially-nulling outer joins, the other tests handle that
+ * concern. If the clause references any Var coming from the inside of a
+ * lower outer join, its clause_relids will mention that outer join, causing
+ * the evaluability check to fail; while if it references no such Vars, the
+ * references-a-target-rel check will fail.
+ *
+ * XXX not clear if we can do this yet: (But if it's ever fixed to be
* exactly accurate, there's an Assert in get_joinrel_parampathinfo() that
* should be re-enabled.)
*
@@ -704,14 +714,5 @@ join_clause_is_movable_into(RestrictInfo *rinfo,
if (bms_overlap(currentrelids, rinfo->outer_relids))
return false;
- /*
- * Target rel(s) must not be nullable below the clause. This is
- * approximate, in the safe direction, because the current join might be
- * above the join where the nulling would happen, in which case the clause
- * would work correctly here. But we don't have enough info to be sure.
- */
- if (bms_overlap(currentrelids, rinfo->nullable_relids))
- return false;
-
return true;
}
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 9d89b0c9eb..3f89199e64 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -660,6 +660,7 @@ typedef struct PartitionSchemeData *PartitionScheme;
* outer-join relids.
* attr_widths - cache space for per-attribute width estimates;
* zero means not computed yet
+ * nulling_relids - relids of outer joins that can null this rel
* lateral_vars - lateral cross-references of rel, if any (list of
* Vars and PlaceHolderVars)
* lateral_referencers - relids of rels that reference this one laterally
@@ -893,6 +894,8 @@ typedef struct RelOptInfo
Relids *attr_needed pg_node_attr(read_write_ignore);
/* array indexed [min_attr .. max_attr] */
int32 *attr_widths pg_node_attr(read_write_ignore);
+ /* relids of outer joins that can null this baserel */
+ Relids nulling_relids;
/* LATERAL Vars and PHVs referenced by rel */
List *lateral_vars;
/* rels that reference this baserel laterally */
commit c7837187e8c6652811d91431413f404cef844505
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Oct 31 13:26:51 2022 -0400
Remove RestrictInfo.nullable_relids and associated infrastructure.
There is no more code using this field, only code computing it,
so just delete all that. We can likewise get rid of
EquivalenceMember.em_nullable_relids and
PlannerInfo.nullable_baserels.
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 39cc37053c..10aa27a78a 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -6313,7 +6313,6 @@ foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
false,
root->qual_security_level,
grouped_rel->relids,
- NULL,
NULL);
if (is_foreign_expr(root, grouped_rel, expr))
fpinfo->remote_conds = lappend(fpinfo->remote_conds, rinfo);
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 5902c80747..8a6a40f672 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -2733,7 +2733,6 @@ set_function_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
if (var)
pathkeys = build_expression_pathkey(root,
(Expr *) var,
- NULL, /* below outer joins */
Int8LessOperator,
rel->relids,
false);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index d4f8b7893d..0737cc355f 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -34,7 +34,7 @@
static EquivalenceMember *add_eq_member(EquivalenceClass *ec,
- Expr *expr, Relids relids, Relids nullable_relids,
+ Expr *expr, Relids relids,
EquivalenceMember *parent,
Oid datatype);
static bool is_exprlist_member(Expr *node, List *exprs);
@@ -131,9 +131,7 @@ process_equivalence(PlannerInfo *root,
Expr *item1;
Expr *item2;
Relids item1_relids,
- item2_relids,
- item1_nullable_relids,
- item2_nullable_relids;
+ item2_relids;
List *opfamilies;
EquivalenceClass *ec1,
*ec2;
@@ -206,8 +204,7 @@ process_equivalence(PlannerInfo *root,
restrictinfo->pseudoconstant,
restrictinfo->security_level,
NULL,
- restrictinfo->outer_relids,
- restrictinfo->nullable_relids);
+ restrictinfo->outer_relids);
}
return false;
}
@@ -225,12 +222,6 @@ process_equivalence(PlannerInfo *root,
return false; /* RHS is non-strict but not constant */
}
- /* Calculate nullable-relid sets for each side of the clause */
- item1_nullable_relids = bms_intersect(item1_relids,
- restrictinfo->nullable_relids);
- item2_nullable_relids = bms_intersect(item2_relids,
- restrictinfo->nullable_relids);
-
/*
* We use the declared input types of the operator, not exprType() of the
* inputs, as the nominal datatypes for opfamily lookup. This presumes
@@ -400,7 +391,7 @@ process_equivalence(PlannerInfo *root,
else if (ec1)
{
/* Case 3: add item2 to ec1 */
- em2 = add_eq_member(ec1, item2, item2_relids, item2_nullable_relids,
+ em2 = add_eq_member(ec1, item2, item2_relids,
NULL, item2_type);
ec1->ec_sources = lappend(ec1->ec_sources, restrictinfo);
ec1->ec_below_outer_join |= below_outer_join;
@@ -418,7 +409,7 @@ process_equivalence(PlannerInfo *root,
else if (ec2)
{
/* Case 3: add item1 to ec2 */
- em1 = add_eq_member(ec2, item1, item1_relids, item1_nullable_relids,
+ em1 = add_eq_member(ec2, item1, item1_relids,
NULL, item1_type);
ec2->ec_sources = lappend(ec2->ec_sources, restrictinfo);
ec2->ec_below_outer_join |= below_outer_join;
@@ -452,9 +443,9 @@ process_equivalence(PlannerInfo *root,
ec->ec_min_security = restrictinfo->security_level;
ec->ec_max_security = restrictinfo->security_level;
ec->ec_merged = NULL;
- em1 = add_eq_member(ec, item1, item1_relids, item1_nullable_relids,
+ em1 = add_eq_member(ec, item1, item1_relids,
NULL, item1_type);
- em2 = add_eq_member(ec, item2, item2_relids, item2_nullable_relids,
+ em2 = add_eq_member(ec, item2, item2_relids,
NULL, item2_type);
root->eq_classes = lappend(root->eq_classes, ec);
@@ -545,13 +536,12 @@ canonicalize_ec_expression(Expr *expr, Oid req_type, Oid req_collation)
*/
static EquivalenceMember *
add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
- Relids nullable_relids, EquivalenceMember *parent, Oid datatype)
+ EquivalenceMember *parent, Oid datatype)
{
EquivalenceMember *em = makeNode(EquivalenceMember);
em->em_expr = expr;
em->em_relids = relids;
- em->em_nullable_relids = nullable_relids;
em->em_is_const = false;
em->em_is_child = (parent != NULL);
em->em_datatype = datatype;
@@ -588,13 +578,6 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
* equivalence class it is a member of; if none, optionally build a new
* single-member EquivalenceClass for it.
*
- * expr is the expression, and nullable_relids is the set of base relids
- * that are potentially nullable below it. We actually only care about
- * the set of such relids that are used in the expression; but for caller
- * convenience, we perform that intersection step here. The caller need
- * only be sure that nullable_relids doesn't omit any nullable rels that
- * might appear in the expr.
- *
* sortref is the SortGroupRef of the originating SortGroupClause, if any,
* or zero if not. (It should never be zero if the expression is volatile!)
*
@@ -623,7 +606,6 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
EquivalenceClass *
get_eclass_for_sort_expr(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
List *opfamilies,
Oid opcintype,
Oid collation,
@@ -719,13 +701,12 @@ get_eclass_for_sort_expr(PlannerInfo *root,
elog(ERROR, "volatile EquivalenceClass has no sortref");
/*
- * Get the precise set of nullable relids appearing in the expression.
+ * Get the precise set of relids appearing in the expression.
*/
expr_relids = pull_varnos(root, (Node *) expr);
- nullable_relids = bms_intersect(nullable_relids, expr_relids);
newem = add_eq_member(newec, copyObject(expr), expr_relids,
- nullable_relids, NULL, opcintype);
+ NULL, opcintype);
/*
* add_eq_member doesn't check for volatile functions, set-returning
@@ -1211,8 +1192,6 @@ generate_base_implied_equalities_const(PlannerInfo *root,
rinfo = process_implied_equality(root, eq_op, ec->ec_collation,
cur_em->em_expr, const_em->em_expr,
bms_copy(ec->ec_relids),
- bms_union(cur_em->em_nullable_relids,
- const_em->em_nullable_relids),
ec->ec_min_security,
ec->ec_below_outer_join,
cur_em->em_is_const);
@@ -1285,8 +1264,6 @@ generate_base_implied_equalities_no_const(PlannerInfo *root,
rinfo = process_implied_equality(root, eq_op, ec->ec_collation,
prev_em->em_expr, cur_em->em_expr,
bms_copy(ec->ec_relids),
- bms_union(prev_em->em_nullable_relids,
- cur_em->em_nullable_relids),
ec->ec_min_security,
ec->ec_below_outer_join,
false);
@@ -1889,8 +1866,6 @@ create_join_clause(PlannerInfo *root,
rightem->em_expr,
bms_union(leftem->em_relids,
rightem->em_relids),
- bms_union(leftem->em_nullable_relids,
- rightem->em_nullable_relids),
ec->ec_min_security);
/* If it's a child clause, copy the parent's rinfo_serial */
@@ -2105,8 +2080,7 @@ reconsider_outer_join_clause(PlannerInfo *root, RestrictInfo *rinfo,
left_type,
right_type,
inner_datatype;
- Relids inner_relids,
- inner_nullable_relids;
+ Relids inner_relids;
ListCell *lc1;
Assert(is_opclause(rinfo->clause));
@@ -2133,8 +2107,6 @@ reconsider_outer_join_clause(PlannerInfo *root, RestrictInfo *rinfo,
inner_datatype = left_type;
inner_relids = rinfo->left_relids;
}
- inner_nullable_relids = bms_intersect(inner_relids,
- rinfo->nullable_relids);
/* Scan EquivalenceClasses for a match to outervar */
foreach(lc1, root->eq_classes)
@@ -2195,7 +2167,6 @@ reconsider_outer_join_clause(PlannerInfo *root, RestrictInfo *rinfo,
innervar,
cur_em->em_expr,
bms_copy(inner_relids),
- bms_copy(inner_nullable_relids),
cur_ec->ec_min_security);
if (process_equivalence(root, &newrinfo, true))
match = true;
@@ -2233,9 +2204,7 @@ reconsider_full_join_clause(PlannerInfo *root, FullJoinClauseInfo *fjinfo)
left_type,
right_type;
Relids left_relids,
- right_relids,
- left_nullable_relids,
- right_nullable_relids;
+ right_relids;
ListCell *lc1;
/* Can't use an outerjoin_delayed clause here */
@@ -2251,10 +2220,6 @@ reconsider_full_join_clause(PlannerInfo *root, FullJoinClauseInfo *fjinfo)
rightvar = (Expr *) get_rightop(rinfo->clause);
left_relids = rinfo->left_relids;
right_relids = rinfo->right_relids;
- left_nullable_relids = bms_intersect(left_relids,
- rinfo->nullable_relids);
- right_nullable_relids = bms_intersect(right_relids,
- rinfo->nullable_relids);
foreach(lc1, root->eq_classes)
{
@@ -2356,7 +2321,6 @@ reconsider_full_join_clause(PlannerInfo *root, FullJoinClauseInfo *fjinfo)
leftvar,
cur_em->em_expr,
bms_copy(left_relids),
- bms_copy(left_nullable_relids),
cur_ec->ec_min_security);
if (process_equivalence(root, &newrinfo, true))
matchleft = true;
@@ -2372,7 +2336,6 @@ reconsider_full_join_clause(PlannerInfo *root, FullJoinClauseInfo *fjinfo)
rightvar,
cur_em->em_expr,
bms_copy(right_relids),
- bms_copy(right_nullable_relids),
cur_ec->ec_min_security);
if (process_equivalence(root, &newrinfo, true))
matchright = true;
@@ -2662,7 +2625,6 @@ add_child_rel_equivalences(PlannerInfo *root,
/* Yes, generate transformed child version */
Expr *child_expr;
Relids new_relids;
- Relids new_nullable_relids;
if (parent_rel->reloptkind == RELOPT_BASEREL)
{
@@ -2692,21 +2654,7 @@ add_child_rel_equivalences(PlannerInfo *root,
top_parent_relids);
new_relids = bms_add_members(new_relids, child_relids);
- /*
- * And likewise for nullable_relids. Note this code assumes
- * parent and child relids are singletons.
- */
- new_nullable_relids = cur_em->em_nullable_relids;
- if (bms_overlap(new_nullable_relids, top_parent_relids))
- {
- new_nullable_relids = bms_difference(new_nullable_relids,
- top_parent_relids);
- new_nullable_relids = bms_add_members(new_nullable_relids,
- child_relids);
- }
-
- (void) add_eq_member(cur_ec, child_expr,
- new_relids, new_nullable_relids,
+ (void) add_eq_member(cur_ec, child_expr, new_relids,
cur_em, cur_em->em_datatype);
/* Record this EC index for the child rel */
@@ -2803,7 +2751,6 @@ add_child_join_rel_equivalences(PlannerInfo *root,
/* Yes, generate transformed child version */
Expr *child_expr;
Relids new_relids;
- Relids new_nullable_relids;
if (parent_joinrel->reloptkind == RELOPT_JOINREL)
{
@@ -2834,20 +2781,7 @@ add_child_join_rel_equivalences(PlannerInfo *root,
top_parent_relids);
new_relids = bms_add_members(new_relids, child_relids);
- /*
- * For nullable_relids, we must selectively replace parent
- * nullable relids with child ones.
- */
- new_nullable_relids = cur_em->em_nullable_relids;
- if (bms_overlap(new_nullable_relids, top_parent_relids))
- new_nullable_relids =
- adjust_child_relids_multilevel(root,
- new_nullable_relids,
- child_joinrel,
- child_joinrel->top_parent);
-
- (void) add_eq_member(cur_ec, child_expr,
- new_relids, new_nullable_relids,
+ (void) add_eq_member(cur_ec, child_expr, new_relids,
cur_em, cur_em->em_datatype);
}
}
diff --git a/src/backend/optimizer/path/pathkeys.c b/src/backend/optimizer/path/pathkeys.c
index a9943cd6e0..bf919ca97f 100644
--- a/src/backend/optimizer/path/pathkeys.c
+++ b/src/backend/optimizer/path/pathkeys.c
@@ -180,9 +180,6 @@ pathkey_is_redundant(PathKey *new_pathkey, List *pathkeys)
* Given an expression and sort-order information, create a PathKey.
* The result is always a "canonical" PathKey, but it might be redundant.
*
- * expr is the expression, and nullable_relids is the set of base relids
- * that are potentially nullable below it.
- *
* If the PathKey is being generated from a SortGroupClause, sortref should be
* the SortGroupClause's SortGroupRef; otherwise zero.
*
@@ -198,7 +195,6 @@ pathkey_is_redundant(PathKey *new_pathkey, List *pathkeys)
static PathKey *
make_pathkey_from_sortinfo(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
Oid opfamily,
Oid opcintype,
Oid collation,
@@ -234,7 +230,7 @@ make_pathkey_from_sortinfo(PlannerInfo *root,
equality_op);
/* Now find or (optionally) create a matching EquivalenceClass */
- eclass = get_eclass_for_sort_expr(root, expr, nullable_relids,
+ eclass = get_eclass_for_sort_expr(root, expr,
opfamilies, opcintype, collation,
sortref, rel, create_it);
@@ -257,7 +253,6 @@ make_pathkey_from_sortinfo(PlannerInfo *root,
static PathKey *
make_pathkey_from_sortop(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
Oid ordering_op,
bool nulls_first,
Index sortref,
@@ -279,7 +274,6 @@ make_pathkey_from_sortop(PlannerInfo *root,
return make_pathkey_from_sortinfo(root,
expr,
- nullable_relids,
opfamily,
opcintype,
collation,
@@ -584,12 +578,10 @@ build_index_pathkeys(PlannerInfo *root,
}
/*
- * OK, try to make a canonical pathkey for this sort key. Note we're
- * underneath any outer joins, so nullable_relids should be NULL.
+ * OK, try to make a canonical pathkey for this sort key.
*/
cpathkey = make_pathkey_from_sortinfo(root,
indexkey,
- NULL,
index->sortopfamily[i],
index->opcintype[i],
index->indexcollations[i],
@@ -743,14 +735,12 @@ build_partition_pathkeys(PlannerInfo *root, RelOptInfo *partrel,
/*
* Try to make a canonical pathkey for this partkey.
*
- * We're considering a baserel scan, so nullable_relids should be
- * NULL. Also, we assume the PartitionDesc lists any NULL partition
- * last, so we treat the scan like a NULLS LAST index: we have
- * nulls_first for backwards scan only.
+ * We assume the PartitionDesc lists any NULL partition last, so we
+ * treat the scan like a NULLS LAST index: we have nulls_first for
+ * backwards scan only.
*/
cpathkey = make_pathkey_from_sortinfo(root,
keyCol,
- NULL,
partscheme->partopfamily[i],
partscheme->partopcintype[i],
partscheme->partcollation[i],
@@ -799,7 +789,7 @@ build_partition_pathkeys(PlannerInfo *root, RelOptInfo *partrel,
* Build a pathkeys list that describes an ordering by a single expression
* using the given sort operator.
*
- * expr, nullable_relids, and rel are as for make_pathkey_from_sortinfo.
+ * expr and rel are as for make_pathkey_from_sortinfo.
* We induce the other arguments assuming default sort order for the operator.
*
* Similarly to make_pathkey_from_sortinfo, the result is NIL if create_it
@@ -808,7 +798,6 @@ build_partition_pathkeys(PlannerInfo *root, RelOptInfo *partrel,
List *
build_expression_pathkey(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
Oid opno,
Relids rel,
bool create_it)
@@ -827,7 +816,6 @@ build_expression_pathkey(PlannerInfo *root,
cpathkey = make_pathkey_from_sortinfo(root,
expr,
- nullable_relids,
opfamily,
opcintype,
exprCollation((Node *) expr),
@@ -908,14 +896,11 @@ convert_subquery_pathkeys(PlannerInfo *root, RelOptInfo *rel,
* expression is *not* volatile in the outer query: it's just
* a Var referencing whatever the subquery emitted. (IOW, the
* outer query isn't going to re-execute the volatile
- * expression itself.) So this is okay. Likewise, it's
- * correct to pass nullable_relids = NULL, because we're
- * underneath any outer joins appearing in the outer query.
+ * expression itself.) So this is okay.
*/
outer_ec =
get_eclass_for_sort_expr(root,
(Expr *) outer_var,
- NULL,
sub_eclass->ec_opfamilies,
sub_member->em_datatype,
sub_eclass->ec_collation,
@@ -997,7 +982,6 @@ convert_subquery_pathkeys(PlannerInfo *root, RelOptInfo *rel,
/* See if we have a matching EC for the TLE */
outer_ec = get_eclass_for_sort_expr(root,
(Expr *) outer_var,
- NULL,
sub_eclass->ec_opfamilies,
sub_expr_type,
sub_expr_coll,
@@ -1138,13 +1122,6 @@ build_join_pathkeys(PlannerInfo *root,
* The resulting PathKeys are always in canonical form. (Actually, there
* is no longer any code anywhere that creates non-canonical PathKeys.)
*
- * We assume that root->nullable_baserels is the set of base relids that could
- * have gone to NULL below the SortGroupClause expressions. This is okay if
- * the expressions came from the query's top level (ORDER BY, DISTINCT, etc)
- * and if this function is only invoked after deconstruct_jointree. In the
- * future we might have to make callers pass in the appropriate
- * nullable-relids set, but for now it seems unnecessary.
- *
* 'sortclauses' is a list of SortGroupClause nodes
* 'tlist' is the targetlist to find the referenced tlist entries in
*/
@@ -1166,7 +1143,6 @@ make_pathkeys_for_sortclauses(PlannerInfo *root,
Assert(OidIsValid(sortcl->sortop));
pathkey = make_pathkey_from_sortop(root,
sortkey,
- root->nullable_baserels,
sortcl->sortop,
sortcl->nulls_first,
sortcl->tleSortGroupRef,
@@ -1222,7 +1198,6 @@ initialize_mergeclause_eclasses(PlannerInfo *root, RestrictInfo *restrictinfo)
restrictinfo->left_ec =
get_eclass_for_sort_expr(root,
(Expr *) get_leftop(clause),
- restrictinfo->nullable_relids,
restrictinfo->mergeopfamilies,
lefttype,
((OpExpr *) clause)->inputcollid,
@@ -1232,7 +1207,6 @@ initialize_mergeclause_eclasses(PlannerInfo *root, RestrictInfo *restrictinfo)
restrictinfo->right_ec =
get_eclass_for_sort_expr(root,
(Expr *) get_rightop(clause),
- restrictinfo->nullable_relids,
restrictinfo->mergeopfamilies,
righttype,
((OpExpr *) clause)->inputcollid,
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 364c26badf..4b554967d4 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -92,7 +92,7 @@ static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool is_clone,
List **postponed_qual_list);
static bool check_outerjoin_delay(PlannerInfo *root, Relids *relids_p,
- Relids *nullable_relids_p, bool is_pushed_down);
+ bool is_pushed_down);
static bool check_equivalence_delay(PlannerInfo *root,
RestrictInfo *restrictinfo);
static bool check_redundant_nullability_qual(PlannerInfo *root, Node *clause);
@@ -748,9 +748,8 @@ deconstruct_jointree(PlannerInfo *root)
Assert(root->parse->jointree != NULL &&
IsA(root->parse->jointree, FromExpr));
- /* These are filled as we scan the jointree */
+ /* This is filled as we scan the jointree */
root->outer_join_rels = NULL;
- root->nullable_baserels = NULL;
result = deconstruct_recurse(root, (Node *) root->parse->jointree, false,
&qualscope, &inner_join_rels,
@@ -909,7 +908,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
left_inners,
right_inners,
nonnullable_rels,
- nullable_rels,
ojscope;
List *leftjoinlist,
*rightjoinlist;
@@ -945,8 +943,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
*inner_join_rels = *qualscope;
/* Inner join adds no restrictions for quals */
nonnullable_rels = NULL;
- /* and it doesn't force anything to null, either */
- nullable_rels = NULL;
break;
case JOIN_LEFT:
case JOIN_ANTI:
@@ -969,7 +965,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
}
*inner_join_rels = bms_union(left_inners, right_inners);
nonnullable_rels = leftids;
- nullable_rels = rightids;
break;
case JOIN_SEMI:
leftjoinlist = deconstruct_recurse(root, j->larg,
@@ -986,13 +981,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
*inner_join_rels = bms_union(left_inners, right_inners);
/* Semi join adds no restrictions for quals */
nonnullable_rels = NULL;
-
- /*
- * Theoretically, a semijoin would null the RHS; but since the
- * RHS can't be accessed above the join, this is immaterial
- * and we needn't account for it.
- */
- nullable_rels = NULL;
break;
case JOIN_FULL:
leftjoinlist = deconstruct_recurse(root, j->larg,
@@ -1013,22 +1001,16 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
*inner_join_rels = bms_union(left_inners, right_inners);
/* each side is both outer and inner */
nonnullable_rels = *qualscope;
- nullable_rels = *qualscope;
break;
default:
/* JOIN_RIGHT was eliminated during reduce_outer_joins() */
elog(ERROR, "unrecognized join type: %d",
(int) j->jointype);
nonnullable_rels = NULL; /* keep compiler quiet */
- nullable_rels = NULL;
leftjoinlist = rightjoinlist = NIL;
break;
}
- /* Report all rels that will be nulled anywhere in the jointree */
- root->nullable_baserels = bms_add_members(root->nullable_baserels,
- nullable_rels);
-
/*
* Try to process any quals postponed by children. If they need
* further postponement, add them to my output postponed_qual_list.
@@ -2105,7 +2087,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool pseudoconstant = false;
bool maybe_equivalence;
bool maybe_outer_join;
- Relids nullable_relids;
RestrictInfo *restrictinfo;
/*
@@ -2259,7 +2240,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
/* Check to see if must be delayed by lower outer join */
outerjoin_delayed = check_outerjoin_delay(root,
&relids,
- &nullable_relids,
false);
/*
@@ -2287,7 +2267,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
/* Check to see if must be delayed by lower outer join */
outerjoin_delayed = check_outerjoin_delay(root,
&relids,
- &nullable_relids,
true);
if (outerjoin_delayed)
@@ -2347,8 +2326,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
pseudoconstant,
security_level,
relids,
- outerjoin_nonnullable,
- nullable_relids);
+ outerjoin_nonnullable);
/* Apply appropriate clone marking, too */
restrictinfo->has_clone = has_clone;
@@ -2485,9 +2463,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* If the qual must be delayed, add relids to *relids_p to reflect the lowest
* safe level for evaluating the qual, and return true. Any extra delay for
* higher-level joins is reflected by setting delay_upper_joins to true in
- * SpecialJoinInfo structs. We also compute nullable_relids, the set of
- * referenced relids that are nullable by lower outer joins (note that this
- * can be nonempty even for a non-delayed qual).
+ * SpecialJoinInfo structs.
*
* For an is_pushed_down qual, we can evaluate the qual as soon as (1) we have
* all the rels it mentions, and (2) we are at or above any outer joins that
@@ -2510,8 +2486,8 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* mentioning only C cannot be applied below the join to A.
*
* For a non-pushed-down qual, this isn't going to determine where we place the
- * qual, but we need to determine outerjoin_delayed and nullable_relids anyway
- * for use later in the planning process.
+ * qual, but we need to determine outerjoin_delayed anyway for use later in
+ * the planning process.
*
* Lastly, a pushed-down qual that references the nullable side of any current
* join_info_list member and has to be evaluated above that OJ (because its
@@ -2529,24 +2505,18 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
static bool
check_outerjoin_delay(PlannerInfo *root,
Relids *relids_p, /* in/out parameter */
- Relids *nullable_relids_p, /* output parameter */
bool is_pushed_down)
{
Relids relids;
- Relids nullable_relids;
bool outerjoin_delayed;
bool found_some;
/* fast path if no special joins */
if (root->join_info_list == NIL)
- {
- *nullable_relids_p = NULL;
return false;
- }
/* must copy relids because we need the original value at the end */
relids = bms_copy(*relids_p);
- nullable_relids = NULL;
outerjoin_delayed = false;
do
{
@@ -2573,12 +2543,6 @@ check_outerjoin_delay(PlannerInfo *root,
/* we'll need another iteration */
found_some = true;
}
- /* track all the nullable rels of relevant OJs */
- nullable_relids = bms_add_members(nullable_relids,
- sjinfo->min_righthand);
- if (sjinfo->jointype == JOIN_FULL)
- nullable_relids = bms_add_members(nullable_relids,
- sjinfo->min_lefthand);
/* set delay_upper_joins if needed */
if (is_pushed_down && sjinfo->jointype != JOIN_FULL &&
bms_overlap(relids, sjinfo->min_lefthand))
@@ -2587,13 +2551,9 @@ check_outerjoin_delay(PlannerInfo *root,
}
} while (found_some);
- /* identify just the actually-referenced nullable rels */
- nullable_relids = bms_int_members(nullable_relids, *relids_p);
-
- /* replace *relids_p, and return nullable_relids */
+ /* replace *relids_p */
bms_free(*relids_p);
*relids_p = relids;
- *nullable_relids_p = nullable_relids;
return outerjoin_delayed;
}
@@ -2615,7 +2575,6 @@ check_equivalence_delay(PlannerInfo *root,
RestrictInfo *restrictinfo)
{
Relids relids;
- Relids nullable_relids;
/* fast path if no special joins */
if (root->join_info_list == NIL)
@@ -2624,12 +2583,12 @@ check_equivalence_delay(PlannerInfo *root,
/* must copy restrictinfo's relids to avoid changing it */
relids = bms_copy(restrictinfo->left_relids);
/* check left side does not need delay */
- if (check_outerjoin_delay(root, &relids, &nullable_relids, true))
+ if (check_outerjoin_delay(root, &relids, true))
return false;
/* and similarly for the right side */
relids = bms_copy(restrictinfo->right_relids);
- if (check_outerjoin_delay(root, &relids, &nullable_relids, true))
+ if (check_outerjoin_delay(root, &relids, true))
return false;
return true;
@@ -2755,11 +2714,6 @@ distribute_restrictinfo_to_rels(PlannerInfo *root,
* variable-free. Otherwise the qual is applied at the lowest join level
* that provides all its variables.
*
- * "nullable_relids" is the set of relids used in the expressions that are
- * potentially nullable below the expressions. (This has to be supplied by
- * caller because this function is used after deconstruct_jointree, so we
- * don't have knowledge of where the clause items came from.)
- *
* "security_level" is the security level to assign to the new restrictinfo.
*
* "both_const" indicates whether both items are known pseudo-constant;
@@ -2785,7 +2739,6 @@ process_implied_equality(PlannerInfo *root,
Expr *item1,
Expr *item2,
Relids qualscope,
- Relids nullable_relids,
Index security_level,
bool below_outer_join,
bool both_const)
@@ -2869,8 +2822,7 @@ process_implied_equality(PlannerInfo *root,
pseudoconstant,
security_level,
relids,
- NULL, /* outer_relids */
- nullable_relids);
+ NULL); /* outer_relids */
/*
* If it's a join clause, add vars used in the clause to targetlists of
@@ -2935,7 +2887,6 @@ build_implied_join_equality(PlannerInfo *root,
Expr *item1,
Expr *item2,
Relids qualscope,
- Relids nullable_relids,
Index security_level)
{
RestrictInfo *restrictinfo;
@@ -2963,8 +2914,7 @@ build_implied_join_equality(PlannerInfo *root,
false, /* pseudoconstant */
security_level, /* security_level */
qualscope, /* required_relids */
- NULL, /* outer_relids */
- nullable_relids); /* nullable_relids */
+ NULL); /* outer_relids */
/* Set mergejoinability/hashjoinability flags */
check_mergejoinable(restrictinfo);
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index e18d64b6dc..662d2c1f17 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -448,9 +448,6 @@ adjust_appendrel_attrs_mutator(Node *node,
newinfo->outer_relids = adjust_child_relids(oldinfo->outer_relids,
context->nappinfos,
context->appinfos);
- newinfo->nullable_relids = adjust_child_relids(oldinfo->nullable_relids,
- context->nappinfos,
- context->appinfos);
newinfo->left_relids = adjust_child_relids(oldinfo->left_relids,
context->nappinfos,
context->appinfos);
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 3d270e91d6..eb20583f75 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -815,7 +815,7 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
rinfo->outerjoin_delayed,
pseudoconstant,
rinfo->security_level,
- NULL, NULL, NULL));
+ NULL, NULL));
/* track minimum security level among child quals */
cq_min_security = Min(cq_min_security, rinfo->security_level);
}
@@ -850,7 +850,7 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
make_restrictinfo(root, qual,
true, false, false,
security_level,
- NULL, NULL, NULL));
+ NULL, NULL));
cq_min_security = Min(cq_min_security, security_level);
}
security_level++;
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index 9cfde2f790..336a73d3b9 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -275,7 +275,6 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
false,
join_or_rinfo->security_level,
NULL,
- NULL,
NULL);
/*
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index 15f410cf36..1d8912608b 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -29,8 +29,7 @@ static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids);
+ Relids outer_relids);
static Expr *make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
bool is_pushed_down,
@@ -38,8 +37,7 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids);
+ Relids outer_relids);
/*
@@ -49,7 +47,7 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
*
* The is_pushed_down, outerjoin_delayed, and pseudoconstant flags for the
* RestrictInfo must be supplied by the caller, as well as the correct values
- * for security_level, outer_relids, and nullable_relids.
+ * for security_level and outer_relids.
* required_relids can be NULL, in which case it defaults to the actual clause
* contents (i.e., clause_relids).
*
@@ -69,8 +67,7 @@ make_restrictinfo(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids)
+ Relids outer_relids)
{
/*
* If it's an OR clause, build a modified copy with RestrictInfos inserted
@@ -84,8 +81,7 @@ make_restrictinfo(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids);
+ outer_relids);
/* Shouldn't be an AND clause, else AND/OR flattening messed up */
Assert(!is_andclause(clause));
@@ -98,8 +94,7 @@ make_restrictinfo(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids);
+ outer_relids);
}
/*
@@ -116,8 +111,7 @@ make_restrictinfo_internal(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids)
+ Relids outer_relids)
{
RestrictInfo *restrictinfo = makeNode(RestrictInfo);
Relids baserels;
@@ -132,7 +126,6 @@ make_restrictinfo_internal(PlannerInfo *root,
restrictinfo->can_join = false; /* may get set below */
restrictinfo->security_level = security_level;
restrictinfo->outer_relids = outer_relids;
- restrictinfo->nullable_relids = nullable_relids;
/*
* If it's potentially delayable by lower-level security quals, figure out
@@ -260,7 +253,7 @@ make_restrictinfo_internal(PlannerInfo *root,
*
* The same is_pushed_down, outerjoin_delayed, and pseudoconstant flag
* values can be applied to all RestrictInfo nodes in the result. Likewise
- * for security_level, outer_relids, and nullable_relids.
+ * for security_level and outer_relids.
*
* The given required_relids are attached to our top-level output,
* but any OR-clause constituents are allowed to default to just the
@@ -274,8 +267,7 @@ make_sub_restrictinfos(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids)
+ Relids outer_relids)
{
if (is_orclause(clause))
{
@@ -291,8 +283,7 @@ make_sub_restrictinfos(PlannerInfo *root,
pseudoconstant,
security_level,
NULL,
- outer_relids,
- nullable_relids));
+ outer_relids));
return (Expr *) make_restrictinfo_internal(root,
clause,
make_orclause(orlist),
@@ -301,8 +292,7 @@ make_sub_restrictinfos(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids);
+ outer_relids);
}
else if (is_andclause(clause))
{
@@ -318,8 +308,7 @@ make_sub_restrictinfos(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids));
+ outer_relids));
return make_andclause(andlist);
}
else
@@ -331,8 +320,7 @@ make_sub_restrictinfos(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids);
+ outer_relids);
}
/*
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 3f89199e64..ea43313f11 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -263,14 +263,6 @@ struct PlannerInfo
*/
Relids all_query_rels;
- /*
- * nullable_baserels is a Relids set of base relids that are nullable by
- * some outer join in the jointree; these are rels that are potentially
- * nullable below the WHERE clause, SELECT targetlist, etc. This is
- * computed in deconstruct_jointree.
- */
- Relids nullable_baserels;
-
/*
* join_rel_list is a list of all join-relation RelOptInfos we have
* considered in this planning run. For small problems we just scan the
@@ -1358,7 +1350,6 @@ typedef struct EquivalenceMember
Expr *em_expr; /* the expression represented */
Relids em_relids; /* all relids appearing in em_expr */
- Relids em_nullable_relids; /* nullable by lower outer joins */
bool em_is_const; /* expression is pseudoconstant? */
bool em_is_child; /* derived version for a child relation? */
Oid em_datatype; /* the "nominal type" used by the opfamily */
@@ -2384,9 +2375,7 @@ typedef struct LimitPath
* in parameterized scans, since pushing it into the join's outer side would
* lead to wrong answers.)
*
- * There is also a nullable_relids field, which is the set of rels the clause
- * references that can be forced null by some outer join below the clause.
- *
+ * XXX this comment needs work, if we don't remove it completely:
* outerjoin_delayed = true is subtly different from nullable_relids != NULL:
* a clause might reference some nullable rels and yet not be
* outerjoin_delayed because it also references all the other rels of the
@@ -2500,9 +2489,6 @@ typedef struct RestrictInfo
/* If an outer-join clause, the outer-side relations, else NULL: */
Relids outer_relids;
- /* The relids used in the clause that are nullable by lower outer joins: */
- Relids nullable_relids;
-
/*
* Relids in the left/right side of the clause. These fields are set for
* any binary opclause.
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 41f765d342..03866de136 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -128,7 +128,6 @@ extern Expr *canonicalize_ec_expression(Expr *expr,
extern void reconsider_outer_join_clauses(PlannerInfo *root);
extern EquivalenceClass *get_eclass_for_sort_expr(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
List *opfamilies,
Oid opcintype,
Oid collation,
@@ -216,7 +215,7 @@ extern List *build_index_pathkeys(PlannerInfo *root, IndexOptInfo *index,
extern List *build_partition_pathkeys(PlannerInfo *root, RelOptInfo *partrel,
ScanDirection scandir, bool *partialkeys);
extern List *build_expression_pathkey(PlannerInfo *root, Expr *expr,
- Relids nullable_relids, Oid opno,
+ Oid opno,
Relids rel, bool create_it);
extern List *convert_subquery_pathkeys(PlannerInfo *root, RelOptInfo *rel,
List *subquery_pathkeys,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index 9dffdcfd1e..57b963c0f7 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -83,7 +83,6 @@ extern RestrictInfo *process_implied_equality(PlannerInfo *root,
Expr *item1,
Expr *item2,
Relids qualscope,
- Relids nullable_relids,
Index security_level,
bool below_outer_join,
bool both_const);
@@ -93,7 +92,6 @@ extern RestrictInfo *build_implied_join_equality(PlannerInfo *root,
Expr *item1,
Expr *item2,
Relids qualscope,
- Relids nullable_relids,
Index security_level);
extern void match_foreign_keys_to_quals(PlannerInfo *root);
diff --git a/src/include/optimizer/restrictinfo.h b/src/include/optimizer/restrictinfo.h
index 17d3b4ab05..1f092371ea 100644
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -19,7 +19,7 @@
/* Convenience macro for the common case of a valid-everywhere qual */
#define make_simple_restrictinfo(root, clause) \
- make_restrictinfo(root, clause, true, false, false, 0, NULL, NULL, NULL)
+ make_restrictinfo(root, clause, true, false, false, 0, NULL, NULL)
extern RestrictInfo *make_restrictinfo(PlannerInfo *root,
Expr *clause,
@@ -28,8 +28,7 @@ extern RestrictInfo *make_restrictinfo(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids);
+ Relids outer_relids);
extern RestrictInfo *commute_restrictinfo(RestrictInfo *rinfo, Oid comm_op);
extern bool restriction_is_or_clause(RestrictInfo *restrictinfo);
extern bool restriction_is_securely_promotable(RestrictInfo *restrictinfo,
commit 30d9a0d7c0201152cced5ba040ecd8d5a5bbb3b3
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Oct 31 13:44:10 2022 -0400
Use constant TRUE for "dummy" clauses when throwing back outer joins.
This improves on a hack I introduced in commit 6a6522529. If we
have a left-join clause l.x = r.y, and a WHERE clause l.x = constant,
we generate r.y = constant and then don't really have a need for the
join clause. Currently we throw the join clause back anyway after
marking it redundant, so that the join search heuristics won't think
this is a clauseless join and avoid it. That was a kluge introduced
under time pressure, and after looking at it I thought of a better
way: let's just introduce constant-TRUE "join clauses" instead,
and get rid of them at the end.
This improves the generated plans for such cases by not having to
test a redundant join clause. We can also get rid of the ugly hack
used to mark such clauses as redundant for selectivity estimation.
The code added here should go away again, once we handle these cases
as ordinary eclasses. But it seemed worth committing separately
so as to get the regression changes and selectivity simplifications
in place.
diff --git a/src/backend/optimizer/path/clausesel.c b/src/backend/optimizer/path/clausesel.c
index c08eb2b1c5..1cf565ee59 100644
--- a/src/backend/optimizer/path/clausesel.c
+++ b/src/backend/optimizer/path/clausesel.c
@@ -715,12 +715,6 @@ clause_selectivity_ext(PlannerInfo *root,
return (Selectivity) 1.0;
}
- /*
- * If the clause is marked redundant, always return 1.0.
- */
- if (rinfo->norm_selec > 1)
- return (Selectivity) 1.0;
-
/*
* If possible, cache the result of the selectivity calculation for
* the clause. We can cache if varRelid is zero or the clause
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 0737cc355f..a9f5db3ef6 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -1954,14 +1954,11 @@ create_join_clause(PlannerInfo *root,
* If we don't find any match for a set-aside outer join clause, we must
* throw it back into the regular joinclause processing by passing it to
* distribute_restrictinfo_to_rels(). If we do generate a derived clause,
- * however, the outer-join clause is redundant. We still throw it back,
- * because otherwise the join will be seen as a clauseless join and avoided
- * during join order searching; but we mark it as redundant to keep from
- * messing up the joinrel's size estimate. (This behavior means that the
- * API for this routine is uselessly complex: we could have just put all
- * the clauses into the regular processing initially. We keep it because
- * someday we might want to do something else, such as inserting "dummy"
- * joinclauses instead of real ones.)
+ * however, the outer-join clause is redundant. We must still put some
+ * clause into the regular processing, because otherwise the join will be
+ * seen as a clauseless join and avoided during join order searching.
+ * We handle this by generating a constant-TRUE clause that is marked with
+ * required_relids that make it a join between the correct relations.
*
* Outer join clauses that are marked outerjoin_delayed are special: this
* condition means that one or both VARs might go to null due to a lower
@@ -1994,10 +1991,15 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* remove it from the list */
root->left_join_clauses =
foreach_delete_current(root->left_join_clauses, cell);
- /* we throw it back anyway (see notes above) */
- /* but the thrown-back clause has no extra selectivity */
- rinfo->norm_selec = 2.0;
- rinfo->outer_selec = 1.0;
+ /* throw back a dummy replacement clause (see notes above) */
+ rinfo = make_restrictinfo(root,
+ (Expr *) makeBoolConst(true, false),
+ true, /* is_pushed_down */
+ false, /* outerjoin_delayed */
+ false, /* pseudoconstant */
+ 0, /* security_level */
+ rinfo->required_relids,
+ rinfo->outer_relids);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
@@ -2013,10 +2015,15 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* remove it from the list */
root->right_join_clauses =
foreach_delete_current(root->right_join_clauses, cell);
- /* we throw it back anyway (see notes above) */
- /* but the thrown-back clause has no extra selectivity */
- rinfo->norm_selec = 2.0;
- rinfo->outer_selec = 1.0;
+ /* throw back a dummy replacement clause (see notes above) */
+ rinfo = make_restrictinfo(root,
+ (Expr *) makeBoolConst(true, false),
+ true, /* is_pushed_down */
+ false, /* outerjoin_delayed */
+ false, /* pseudoconstant */
+ 0, /* security_level */
+ rinfo->required_relids,
+ rinfo->outer_relids);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
@@ -2034,10 +2041,15 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* remove it from the list */
root->full_join_clauses =
foreach_delete_current(root->full_join_clauses, cell);
- /* we throw it back anyway (see notes above) */
- /* but the thrown-back clause has no extra selectivity */
- rinfo->norm_selec = 2.0;
- rinfo->outer_selec = 1.0;
+ /* throw back a dummy replacement clause (see notes above) */
+ rinfo = make_restrictinfo(root,
+ (Expr *) makeBoolConst(true, false),
+ true, /* is_pushed_down */
+ false, /* outerjoin_delayed */
+ false, /* pseudoconstant */
+ 0, /* security_level */
+ rinfo->required_relids,
+ rinfo->outer_relids);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index 336a73d3b9..4b98692189 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -98,18 +98,13 @@ extract_restriction_or_clauses(PlannerInfo *root)
* joinclause that is considered safe to move to this rel by the
* parameterized-path machinery, even though what we are going to do
* with it is not exactly a parameterized path.
- *
- * However, it seems best to ignore clauses that have been marked
- * redundant (by setting norm_selec > 1). That likely can't happen
- * for OR clauses, but let's be safe.
*/
foreach(lc, rel->joininfo)
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
if (restriction_is_or_clause(rinfo) &&
- join_clause_is_movable_to(rinfo, rel) &&
- rinfo->norm_selec <= 1)
+ join_clause_is_movable_to(rinfo, rel))
{
/* Try to extract a qual for this rel only */
Expr *orclause = extract_or_clause(rinfo, rel);
@@ -356,7 +351,7 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
/* And hack cached selectivity so join size remains the same */
join_or_rinfo->norm_selec = orig_selec / or_selec;
- /* ensure result stays in sane range, in particular not "redundant" */
+ /* ensure result stays in sane range */
if (join_or_rinfo->norm_selec > 1)
join_or_rinfo->norm_selec = 1;
/* as explained above, we don't touch outer_selec */
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index 1d8912608b..c3af845acd 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -424,6 +424,21 @@ restriction_is_securely_promotable(RestrictInfo *restrictinfo,
return false;
}
+/*
+ * Detect whether a RestrictInfo's clause is constant TRUE (note that it's
+ * surely of type boolean). No such WHERE clause could survive qual
+ * canonicalization, but equivclass.c may generate such RestrictInfos for
+ * reasons discussed therein. We should drop them again when creating
+ * the finished plan, which is handled by the next few functions.
+ */
+static inline bool
+rinfo_is_constant_true(RestrictInfo *rinfo)
+{
+ return IsA(rinfo->clause, Const) &&
+ !((Const *) rinfo->clause)->constisnull &&
+ DatumGetBool(((Const *) rinfo->clause)->constvalue);
+}
+
/*
* get_actual_clauses
*
@@ -443,6 +458,7 @@ get_actual_clauses(List *restrictinfo_list)
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
Assert(!rinfo->pseudoconstant);
+ Assert(!rinfo_is_constant_true(rinfo));
result = lappend(result, rinfo->clause);
}
@@ -454,6 +470,7 @@ get_actual_clauses(List *restrictinfo_list)
*
* Extract bare clauses from 'restrictinfo_list', returning either the
* regular ones or the pseudoconstant ones per 'pseudoconstant'.
+ * Constant-TRUE clauses are dropped in any case.
*/
List *
extract_actual_clauses(List *restrictinfo_list,
@@ -466,7 +483,8 @@ extract_actual_clauses(List *restrictinfo_list,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (rinfo->pseudoconstant == pseudoconstant)
+ if (rinfo->pseudoconstant == pseudoconstant &&
+ !rinfo_is_constant_true(rinfo))
result = lappend(result, rinfo->clause);
}
return result;
@@ -477,7 +495,7 @@ extract_actual_clauses(List *restrictinfo_list,
*
* Extract bare clauses from 'restrictinfo_list', separating those that
* semantically match the join level from those that were pushed down.
- * Pseudoconstant clauses are excluded from the results.
+ * Pseudoconstant and constant-TRUE clauses are excluded from the results.
*
* This is only used at outer joins, since for plain joins we don't care
* about pushed-down-ness.
@@ -499,13 +517,15 @@ extract_actual_join_clauses(List *restrictinfo_list,
if (RINFO_IS_PUSHED_DOWN(rinfo, joinrelids))
{
- if (!rinfo->pseudoconstant)
+ if (!rinfo->pseudoconstant &&
+ !rinfo_is_constant_true(rinfo))
*otherquals = lappend(*otherquals, rinfo->clause);
}
else
{
/* joinquals shouldn't have been marked pseudoconstant */
Assert(!rinfo->pseudoconstant);
+ Assert(!rinfo_is_constant_true(rinfo));
*joinquals = lappend(*joinquals, rinfo->clause);
}
}
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index ea43313f11..6a5ff13d39 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2534,10 +2534,7 @@ typedef struct RestrictInfo
/* eval cost of clause; -1 if not yet set */
QualCost eval_cost pg_node_attr(equal_ignore);
- /*
- * selectivity for "normal" (JOIN_INNER) semantics; -1 if not yet set; >1
- * means a redundant clause
- */
+ /* selectivity for "normal" (JOIN_INNER) semantics; -1 if not yet set */
Selectivity norm_selec pg_node_attr(equal_ignore);
/* selectivity for outer join semantics; -1 if not yet set */
Selectivity outer_selec pg_node_attr(equal_ignore);
diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out
index 340c57a270..dd0f622b78 100644
--- a/src/test/regress/expected/join.out
+++ b/src/test/regress/expected/join.out
@@ -3952,8 +3952,8 @@ explain (costs off)
select a.unique1, b.unique1, c.unique1, coalesce(b.twothousand, a.twothousand)
from tenk1 a left join tenk1 b on b.thousand = a.unique1 left join tenk1 c on c.unique2 =
coalesce(b.twothousand,a.twothousand)
where a.unique2 < 10 and coalesce(b.twothousand, a.twothousand) = 44;
- QUERY PLAN
----------------------------------------------------------------------------------------------
+ QUERY PLAN
+---------------------------------------------------------------
Nested Loop Left Join
-> Nested Loop Left Join
Filter: (COALESCE(b.twothousand, a.twothousand) = 44)
@@ -3964,7 +3964,7 @@ select a.unique1, b.unique1, c.unique1, coalesce(b.twothousand, a.twothousand)
-> Bitmap Index Scan on tenk1_thous_tenthous
Index Cond: (thousand = a.unique1)
-> Index Scan using tenk1_unique2 on tenk1 c
- Index Cond: ((unique2 = COALESCE(b.twothousand, a.twothousand)) AND (unique2 = 44))
+ Index Cond: (unique2 = 44)
(11 rows)
select a.unique1, b.unique1, c.unique1, coalesce(b.twothousand, a.twothousand)
@@ -4395,7 +4395,6 @@ where tt1.f1 = ss1.c0;
Output: tt4.f1
-> Nested Loop Left Join
Output: tt4.f1
- Join Filter: (tt3.f1 = tt4.f1)
-> Seq Scan on public.text_tbl tt3
Output: tt3.f1
Filter: (tt3.f1 = 'foo'::text)
@@ -4413,7 +4412,7 @@ where tt1.f1 = ss1.c0;
Output: (tt4.f1)
-> Seq Scan on public.text_tbl tt5
Output: tt4.f1
-(33 rows)
+(32 rows)
select 1 from
text_tbl as tt1
@@ -4520,24 +4519,22 @@ explain (costs off)
QUERY PLAN
-------------------------------------------------
Nested Loop Left Join
- Join Filter: (a.f1 = b.unique2)
-> Seq Scan on int4_tbl a
Filter: (f1 = 0)
-> Index Scan using tenk1_unique2 on tenk1 b
Index Cond: (unique2 = 0)
-(6 rows)
+(5 rows)
explain (costs off)
select * from tenk1 a full join tenk1 b using(unique2) where unique2 = 42;
QUERY PLAN
-------------------------------------------------
Merge Full Join
- Merge Cond: (a.unique2 = b.unique2)
-> Index Scan using tenk1_unique2 on tenk1 a
Index Cond: (unique2 = 42)
-> Index Scan using tenk1_unique2 on tenk1 b
Index Cond: (unique2 = 42)
-(6 rows)
+(5 rows)
--
-- test that quals attached to an outer join have correct semantics,
commit a2e0c5f8ddcab95f5004ba6d54feef72c0dea876
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Oct 31 19:47:31 2022 -0400
Rewrite reduce_outer_joins' matching of Vars.
My draft patch for adding nulling-rel marks to Vars broke the logic in
reduce_outer_joins_pass2 that detects antijoins by matching upper-level
"Var IS NULL" tests to strict join quals. The problem of course is
that the upper Var is no longer equal() to the one in the join qual,
since the former will now be marked as being nulled by the outer join.
Now, this logic was always pretty brute-force: doing list_intersect
on a list full of Vars isn't especially cheap. So let's fix it by
creating a better-suited data structure, namely an array of per-RTE
bitmaps of relevant Vars' attnos.
(I wonder if there aren't other applications for an array-of-bitmaps
data structure. But for now I just settled for writing enough
primitives for the immediate problem.)
Discussion: https://postgr.es/m/CAMbWs4-mvPPCJ1W6iK6dD5HiNwoJdi6mZp=-7mE8N9Sh+cd0tQ@mail.gmail.com
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index cf558264eb..b1d5105bcd 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -124,8 +124,8 @@ static void reduce_outer_joins_pass2(Node *jtnode,
reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
- List *nonnullable_vars,
- List *forced_null_vars);
+ VarAttnoSet *nonnullable_vars,
+ VarAttnoSet *forced_null_vars);
static void report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
int rtindex, Relids relids);
static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
@@ -2640,7 +2640,7 @@ reduce_outer_joins(PlannerInfo *root)
reduce_outer_joins_pass2((Node *) root->parse->jointree,
state1, &state2,
- root, NULL, NIL, NIL);
+ root, NULL, NULL, NULL);
/*
* If we successfully reduced the strength of any outer joins, we must
@@ -2756,8 +2756,8 @@ reduce_outer_joins_pass1(Node *jtnode)
* state2: where to accumulate info about successfully-reduced joins
* root: toplevel planner state
* nonnullable_rels: set of base relids forced non-null by upper quals
- * nonnullable_vars: list of Vars forced non-null by upper quals
- * forced_null_vars: list of Vars forced null by upper quals
+ * nonnullable_vars: set of Vars forced non-null by upper quals
+ * forced_null_vars: set of Vars forced null by upper quals
*
* Returns info in state2 about outer joins that were successfully simplified.
* Joins that were fully reduced to inner joins are all added to
@@ -2771,8 +2771,8 @@ reduce_outer_joins_pass2(Node *jtnode,
reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
- List *nonnullable_vars,
- List *forced_null_vars)
+ VarAttnoSet *nonnullable_vars,
+ VarAttnoSet *forced_null_vars)
{
/*
* pass 2 should never descend as far as an empty subnode or base rel,
@@ -2788,19 +2788,21 @@ reduce_outer_joins_pass2(Node *jtnode,
ListCell *l;
ListCell *s;
Relids pass_nonnullable_rels;
- List *pass_nonnullable_vars;
- List *pass_forced_null_vars;
+ VarAttnoSet *pass_nonnullable_vars;
+ VarAttnoSet *pass_forced_null_vars;
/* Scan quals to see if we can add any constraints */
pass_nonnullable_rels = find_nonnullable_rels(f->quals);
pass_nonnullable_rels = bms_add_members(pass_nonnullable_rels,
nonnullable_rels);
- pass_nonnullable_vars = find_nonnullable_vars(f->quals);
- pass_nonnullable_vars = list_concat(pass_nonnullable_vars,
- nonnullable_vars);
- pass_forced_null_vars = find_forced_null_vars(f->quals);
- pass_forced_null_vars = list_concat(pass_forced_null_vars,
- forced_null_vars);
+ pass_nonnullable_vars = make_empty_varattnoset(list_length(root->parse->rtable));
+
+ find_nonnullable_vars(f->quals, pass_nonnullable_vars);
+ varattnoset_add_members(pass_nonnullable_vars, nonnullable_vars);
+ pass_forced_null_vars = make_empty_varattnoset(list_length(root->parse->rtable));
+ find_forced_null_vars(f->quals, pass_forced_null_vars);
+ varattnoset_add_members(pass_forced_null_vars,
+ forced_null_vars);
/* And recurse --- but only into interesting subtrees */
Assert(list_length(f->fromlist) == list_length(state1->sub_states));
forboth(l, f->fromlist, s, state1->sub_states)
@@ -2824,8 +2826,7 @@ reduce_outer_joins_pass2(Node *jtnode,
JoinType jointype = j->jointype;
reduce_outer_joins_pass1_state *left_state = linitial(state1->sub_states);
reduce_outer_joins_pass1_state *right_state = lsecond(state1->sub_states);
- List *local_nonnullable_vars = NIL;
- bool computed_local_nonnullable_vars = false;
+ VarAttnoSet *local_nonnullable_vars = NULL;
/* Can we simplify this join? */
switch (jointype)
@@ -2910,21 +2911,19 @@ reduce_outer_joins_pass2(Node *jtnode,
*/
if (jointype == JOIN_LEFT)
{
- List *overlap;
+ Relids overlap;
- local_nonnullable_vars = find_nonnullable_vars(j->quals);
- computed_local_nonnullable_vars = true;
+ local_nonnullable_vars = make_empty_varattnoset(list_length(root->parse->rtable));
+ find_nonnullable_vars(j->quals, local_nonnullable_vars);
/*
* It's not sufficient to check whether local_nonnullable_vars and
* forced_null_vars overlap: we need to know if the overlap
* includes any RHS variables.
*/
- overlap = list_intersection(local_nonnullable_vars,
- forced_null_vars);
- if (overlap != NIL &&
- bms_overlap(pull_varnos(root, (Node *) overlap),
- right_state->relids))
+ overlap = varattnoset_intersect_relids(local_nonnullable_vars,
+ forced_null_vars);
+ if (bms_overlap(overlap, right_state->relids))
jointype = JOIN_ANTI;
}
@@ -2949,10 +2948,10 @@ reduce_outer_joins_pass2(Node *jtnode,
if (left_state->contains_outer || right_state->contains_outer)
{
Relids local_nonnullable_rels;
- List *local_forced_null_vars;
+ VarAttnoSet *local_forced_null_vars;
Relids pass_nonnullable_rels;
- List *pass_nonnullable_vars;
- List *pass_forced_null_vars;
+ VarAttnoSet *pass_nonnullable_vars;
+ VarAttnoSet *pass_forced_null_vars;
/*
* If this join is (now) inner, we can add any constraints its
@@ -2978,25 +2977,30 @@ reduce_outer_joins_pass2(Node *jtnode,
if (jointype != JOIN_FULL)
{
local_nonnullable_rels = find_nonnullable_rels(j->quals);
- if (!computed_local_nonnullable_vars)
- local_nonnullable_vars = find_nonnullable_vars(j->quals);
- local_forced_null_vars = find_forced_null_vars(j->quals);
+ if (!local_nonnullable_vars)
+ {
+ local_nonnullable_vars = make_empty_varattnoset(list_length(root->parse->rtable));
+ find_nonnullable_vars(j->quals, local_nonnullable_vars);
+ }
+ local_forced_null_vars = make_empty_varattnoset(list_length(root->parse->rtable));
+
+ find_forced_null_vars(j->quals, local_forced_null_vars);
if (jointype == JOIN_INNER || jointype == JOIN_SEMI)
{
/* OK to merge upper and local constraints */
local_nonnullable_rels = bms_add_members(local_nonnullable_rels,
nonnullable_rels);
- local_nonnullable_vars = list_concat(local_nonnullable_vars,
- nonnullable_vars);
- local_forced_null_vars = list_concat(local_forced_null_vars,
- forced_null_vars);
+ varattnoset_add_members(local_nonnullable_vars,
+ nonnullable_vars);
+ varattnoset_add_members(local_forced_null_vars,
+ forced_null_vars);
}
}
else
{
/* no use in calculating these */
local_nonnullable_rels = NULL;
- local_forced_null_vars = NIL;
+ local_forced_null_vars = NULL;
}
if (left_state->contains_outer)
@@ -3019,8 +3023,8 @@ reduce_outer_joins_pass2(Node *jtnode,
{
/* no constraints pass through JOIN_FULL */
pass_nonnullable_rels = NULL;
- pass_nonnullable_vars = NIL;
- pass_forced_null_vars = NIL;
+ pass_nonnullable_vars = NULL;
+ pass_forced_null_vars = NULL;
}
reduce_outer_joins_pass2(j->larg, left_state,
state2, root,
@@ -3042,8 +3046,8 @@ reduce_outer_joins_pass2(Node *jtnode,
{
/* no constraints pass through JOIN_FULL */
pass_nonnullable_rels = NULL;
- pass_nonnullable_vars = NIL;
- pass_forced_null_vars = NIL;
+ pass_nonnullable_vars = NULL;
+ pass_forced_null_vars = NULL;
}
reduce_outer_joins_pass2(j->rarg, right_state,
state2, root,
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index f9913ce3b5..3972e10804 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -105,7 +105,8 @@ static bool contain_context_dependent_node(Node *clause);
static bool contain_context_dependent_node_walker(Node *node, int *flags);
static bool contain_leaked_vars_walker(Node *node, void *context);
static Relids find_nonnullable_rels_walker(Node *node, bool top_level);
-static List *find_nonnullable_vars_walker(Node *node, bool top_level);
+static void find_nonnullable_vars_walker(Node *node, VarAttnoSet *attnos,
+ bool top_level);
static bool is_strict_saop(ScalarArrayOpExpr *expr, bool falseOK);
static bool convert_saop_to_hashed_saop_walker(Node *node, void *context);
static Node *eval_const_expressions_mutator(Node *node,
@@ -1308,6 +1309,85 @@ contain_leaked_vars_walker(Node *node, void *context)
context);
}
+
+/*
+ * make_empty_varattnoset
+ * Create an empty VarAttnoSet structure for use with following routines.
+ *
+ * The maximum varno we expect to deal with is rangetable_length.
+ */
+VarAttnoSet *
+make_empty_varattnoset(int rangetable_length)
+{
+ VarAttnoSet *result;
+
+ /* palloc0 is sufficient to initialize all the bitmapsets to empty */
+ result = (VarAttnoSet *)
+ palloc0(offsetof(VarAttnoSet, varattnos) +
+ (rangetable_length + 1) * sizeof(Bitmapset *));
+ result->max_varno = rangetable_length;
+ return result;
+}
+
+/*
+ * varattnoset_add_members
+ * Add all members of set b to set a.
+ *
+ * This is like bms_add_members, but for sets of bitmapsets.
+ * For convenience, we allow b (but not a) to be a NULL pointer.
+ */
+void
+varattnoset_add_members(VarAttnoSet *a, const VarAttnoSet *b)
+{
+ if (b != NULL)
+ {
+ /* We don't really expect the max_varnos to differ, but allow b < a */
+ Assert(b->max_varno <= a->max_varno);
+ for (int i = 1; i <= b->max_varno; i++)
+ a->varattnos[i] = bms_add_members(a->varattnos[i],
+ b->varattnos[i]);
+ }
+}
+
+/*
+ * varattnoset_int_members
+ * Reduce set a to its intersection with set b.
+ *
+ * This is like bms_int_members, but for sets of bitmapsets.
+ */
+static void
+varattnoset_int_members(VarAttnoSet *a, const VarAttnoSet *b)
+{
+ /* We don't really expect the max_varnos to differ, but allow a < b */
+ Assert(a->max_varno <= b->max_varno);
+ for (int i = 1; i <= a->max_varno; i++)
+ a->varattnos[i] = bms_int_members(a->varattnos[i],
+ b->varattnos[i]);
+}
+
+/*
+ * varattnoset_intersect_relids
+ * Identify the relations having common members in a and b.
+ *
+ * For convenience, we allow NULL inputs.
+ */
+Relids
+varattnoset_intersect_relids(const VarAttnoSet *a, const VarAttnoSet *b)
+{
+ Relids result = NULL;
+
+ if (a == NULL || b == NULL)
+ return NULL;
+ Assert(a->max_varno == b->max_varno);
+ for (int i = 1; i <= a->max_varno; i++)
+ {
+ if (bms_overlap(a->varattnos[i], b->varattnos[i]))
+ result = bms_add_member(result, i);
+ }
+ return result;
+}
+
+
/*
* find_nonnullable_rels
* Determine which base rels are forced nonnullable by given clause.
@@ -1541,11 +1621,13 @@ find_nonnullable_rels_walker(Node *node, bool top_level)
* find_nonnullable_vars
* Determine which Vars are forced nonnullable by given clause.
*
- * Returns a list of all level-zero Vars that are referenced in the clause in
+ * Builds a set of all level-zero Vars that are referenced in the clause in
* such a way that the clause cannot possibly return TRUE if any of these Vars
* is NULL. (It is OK to err on the side of conservatism; hence the analysis
* here is simplistic.)
*
+ * Attnos of the identified Vars are added to a caller-supplied VarAttnoSet.
+ *
* The semantics here are subtly different from contain_nonstrict_functions:
* that function is concerned with NULL results from arbitrary expressions,
* but here we assume that the input is a Boolean expression, and wish to
@@ -1553,9 +1635,6 @@ find_nonnullable_rels_walker(Node *node, bool top_level)
* the expression to have been AND/OR flattened and converted to implicit-AND
* format.
*
- * The result is a palloc'd List, but we have not copied the member Var nodes.
- * Also, we don't bother trying to eliminate duplicate entries.
- *
* top_level is true while scanning top-level AND/OR structure; here, showing
* the result is either FALSE or NULL is good enough. top_level is false when
* we have descended below a NOT or a strict function: now we must be able to
@@ -1564,26 +1643,30 @@ find_nonnullable_rels_walker(Node *node, bool top_level)
* We don't use expression_tree_walker here because we don't want to descend
* through very many kinds of nodes; only the ones we can be sure are strict.
*/
-List *
-find_nonnullable_vars(Node *clause)
+void
+find_nonnullable_vars(Node *clause, VarAttnoSet *attnos)
{
- return find_nonnullable_vars_walker(clause, true);
+ return find_nonnullable_vars_walker(clause, attnos, true);
}
-static List *
-find_nonnullable_vars_walker(Node *node, bool top_level)
+static void
+find_nonnullable_vars_walker(Node *node, VarAttnoSet *attnos, bool top_level)
{
- List *result = NIL;
ListCell *l;
if (node == NULL)
- return NIL;
+ return;
if (IsA(node, Var))
{
Var *var = (Var *) node;
if (var->varlevelsup == 0)
- result = list_make1(var);
+ {
+ Assert(var->varno > 0 && var->varno <= attnos->max_varno);
+ attnos->varattnos[var->varno] =
+ bms_add_member(attnos->varattnos[var->varno],
+ var->varattno - FirstLowInvalidHeapAttributeNumber);
+ }
}
else if (IsA(node, List))
{
@@ -1598,9 +1681,7 @@ find_nonnullable_vars_walker(Node *node, bool top_level)
*/
foreach(l, (List *) node)
{
- result = list_concat(result,
- find_nonnullable_vars_walker(lfirst(l),
- top_level));
+ find_nonnullable_vars_walker(lfirst(l), attnos, top_level);
}
}
else if (IsA(node, FuncExpr))
@@ -1608,7 +1689,7 @@ find_nonnullable_vars_walker(Node *node, bool top_level)
FuncExpr *expr = (FuncExpr *) node;
if (func_strict(expr->funcid))
- result = find_nonnullable_vars_walker((Node *) expr->args, false);
+ find_nonnullable_vars_walker((Node *) expr->args, attnos, false);
}
else if (IsA(node, OpExpr))
{
@@ -1616,14 +1697,14 @@ find_nonnullable_vars_walker(Node *node, bool top_level)
set_opfuncid(expr);
if (func_strict(expr->opfuncid))
- result = find_nonnullable_vars_walker((Node *) expr->args, false);
+ find_nonnullable_vars_walker((Node *) expr->args, attnos, false);
}
else if (IsA(node, ScalarArrayOpExpr))
{
ScalarArrayOpExpr *expr = (ScalarArrayOpExpr *) node;
if (is_strict_saop(expr, true))
- result = find_nonnullable_vars_walker((Node *) expr->args, false);
+ find_nonnullable_vars_walker((Node *) expr->args, attnos, false);
}
else if (IsA(node, BoolExpr))
{
@@ -1632,11 +1713,16 @@ find_nonnullable_vars_walker(Node *node, bool top_level)
switch (expr->boolop)
{
case AND_EXPR:
- /* At top level we can just recurse (to the List case) */
+
+ /*
+ * At top level we can just recurse (to the List case), since
+ * the result should be the union of what we can prove in each
+ * arm.
+ */
if (top_level)
{
- result = find_nonnullable_vars_walker((Node *) expr->args,
- top_level);
+ find_nonnullable_vars_walker((Node *) expr->args, attnos,
+ top_level);
break;
}
@@ -1654,30 +1740,36 @@ find_nonnullable_vars_walker(Node *node, bool top_level)
* OR is strict if all of its arms are, so we can take the
* intersection of the sets of nonnullable vars for each arm.
* This works for both values of top_level.
+ *
+ * The pfree's below miss cleaning up individual bitmapsets in
+ * each VarAttnoSet. Doesn't seem worth working harder.
*/
- foreach(l, expr->args)
{
- List *subresult;
+ VarAttnoSet *int_attnos = NULL;
- subresult = find_nonnullable_vars_walker(lfirst(l),
- top_level);
- if (result == NIL) /* first subresult? */
- result = subresult;
- else
- result = list_intersection(result, subresult);
-
- /*
- * If the intersection is empty, we can stop looking. This
- * also justifies the test for first-subresult above.
- */
- if (result == NIL)
- break;
+ foreach(l, expr->args)
+ {
+ VarAttnoSet *sub_attnos;
+
+ sub_attnos = make_empty_varattnoset(attnos->max_varno);
+ find_nonnullable_vars_walker(lfirst(l), sub_attnos,
+ top_level);
+ if (int_attnos == NULL) /* first subresult? */
+ int_attnos = sub_attnos;
+ else
+ {
+ varattnoset_int_members(int_attnos, sub_attnos);
+ pfree(sub_attnos);
+ }
+ }
+ varattnoset_add_members(attnos, int_attnos);
+ pfree(int_attnos);
}
break;
case NOT_EXPR:
/* NOT will return null if its arg is null */
- result = find_nonnullable_vars_walker((Node *) expr->args,
- false);
+ find_nonnullable_vars_walker((Node *) expr->args, attnos,
+ false);
break;
default:
elog(ERROR, "unrecognized boolop: %d", (int) expr->boolop);
@@ -1688,34 +1780,34 @@ find_nonnullable_vars_walker(Node *node, bool top_level)
{
RelabelType *expr = (RelabelType *) node;
- result = find_nonnullable_vars_walker((Node *) expr->arg, top_level);
+ find_nonnullable_vars_walker((Node *) expr->arg, attnos, top_level);
}
else if (IsA(node, CoerceViaIO))
{
/* not clear this is useful, but it can't hurt */
CoerceViaIO *expr = (CoerceViaIO *) node;
- result = find_nonnullable_vars_walker((Node *) expr->arg, false);
+ find_nonnullable_vars_walker((Node *) expr->arg, attnos, false);
}
else if (IsA(node, ArrayCoerceExpr))
{
/* ArrayCoerceExpr is strict at the array level; ignore elemexpr */
ArrayCoerceExpr *expr = (ArrayCoerceExpr *) node;
- result = find_nonnullable_vars_walker((Node *) expr->arg, top_level);
+ find_nonnullable_vars_walker((Node *) expr->arg, attnos, top_level);
}
else if (IsA(node, ConvertRowtypeExpr))
{
/* not clear this is useful, but it can't hurt */
ConvertRowtypeExpr *expr = (ConvertRowtypeExpr *) node;
- result = find_nonnullable_vars_walker((Node *) expr->arg, top_level);
+ find_nonnullable_vars_walker((Node *) expr->arg, attnos, top_level);
}
else if (IsA(node, CollateExpr))
{
CollateExpr *expr = (CollateExpr *) node;
- result = find_nonnullable_vars_walker((Node *) expr->arg, top_level);
+ find_nonnullable_vars_walker((Node *) expr->arg, attnos, top_level);
}
else if (IsA(node, NullTest))
{
@@ -1723,7 +1815,7 @@ find_nonnullable_vars_walker(Node *node, bool top_level)
NullTest *expr = (NullTest *) node;
if (top_level && expr->nulltesttype == IS_NOT_NULL && !expr->argisrow)
- result = find_nonnullable_vars_walker((Node *) expr->arg, false);
+ find_nonnullable_vars_walker((Node *) expr->arg, attnos, false);
}
else if (IsA(node, BooleanTest))
{
@@ -1734,15 +1826,14 @@ find_nonnullable_vars_walker(Node *node, bool top_level)
(expr->booltesttype == IS_TRUE ||
expr->booltesttype == IS_FALSE ||
expr->booltesttype == IS_NOT_UNKNOWN))
- result = find_nonnullable_vars_walker((Node *) expr->arg, false);
+ find_nonnullable_vars_walker((Node *) expr->arg, attnos, false);
}
else if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
- result = find_nonnullable_vars_walker((Node *) phv->phexpr, top_level);
+ find_nonnullable_vars_walker((Node *) phv->phexpr, attnos, top_level);
}
- return result;
}
/*
@@ -1754,23 +1845,25 @@ find_nonnullable_vars_walker(Node *node, bool top_level)
* side of conservatism; hence the analysis here is simplistic. In fact,
* we only detect simple "var IS NULL" tests at the top level.)
*
- * The result is a palloc'd List, but we have not copied the member Var nodes.
- * Also, we don't bother trying to eliminate duplicate entries.
+ * As with find_nonnullable_vars, we add the varattnos of the identified Vars
+ * to a caller-provided VarAttnoSet.
*/
-List *
-find_forced_null_vars(Node *node)
+void
+find_forced_null_vars(Node *node, VarAttnoSet *attnos)
{
- List *result = NIL;
Var *var;
ListCell *l;
if (node == NULL)
- return NIL;
+ return;
/* Check single-clause cases using subroutine */
var = find_forced_null_var(node);
if (var)
{
- result = list_make1(var);
+ Assert(var->varno > 0 && var->varno <= attnos->max_varno);
+ attnos->varattnos[var->varno] =
+ bms_add_member(attnos->varattnos[var->varno],
+ var->varattno - FirstLowInvalidHeapAttributeNumber);
}
/* Otherwise, handle AND-conditions */
else if (IsA(node, List))
@@ -1781,8 +1874,7 @@ find_forced_null_vars(Node *node)
*/
foreach(l, (List *) node)
{
- result = list_concat(result,
- find_forced_null_vars(lfirst(l)));
+ find_forced_null_vars((Node *) lfirst(l), attnos);
}
}
else if (IsA(node, BoolExpr))
@@ -1797,10 +1889,9 @@ find_forced_null_vars(Node *node)
if (expr->boolop == AND_EXPR)
{
/* At top level we can just recurse (to the List case) */
- result = find_forced_null_vars((Node *) expr->args);
+ find_forced_null_vars((Node *) expr->args, attnos);
}
}
- return result;
}
/*
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index ff242d1b6d..5466ada7ba 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -23,6 +23,14 @@ typedef struct
List **windowFuncs; /* lists of WindowFuncs for each winref */
} WindowFuncLists;
+/* Data structure to represent all level-zero Vars meeting some condition */
+typedef struct
+{
+ int max_varno; /* maximum index in varattnos[] */
+ /* Attnos in these sets are offset by FirstLowInvalidHeapAttributeNumber */
+ Bitmapset *varattnos[FLEXIBLE_ARRAY_MEMBER];
+} VarAttnoSet;
+
extern bool contain_agg_clause(Node *clause);
extern bool contain_window_function(Node *clause);
@@ -38,9 +46,14 @@ extern bool contain_nonstrict_functions(Node *clause);
extern bool contain_exec_param(Node *clause, List *param_ids);
extern bool contain_leaked_vars(Node *clause);
+extern VarAttnoSet *make_empty_varattnoset(int rangetable_length);
+extern void varattnoset_add_members(VarAttnoSet *a, const VarAttnoSet *b);
+extern Relids varattnoset_intersect_relids(const VarAttnoSet *a,
+ const VarAttnoSet *b);
+
extern Relids find_nonnullable_rels(Node *clause);
-extern List *find_nonnullable_vars(Node *clause);
-extern List *find_forced_null_vars(Node *node);
+extern void find_nonnullable_vars(Node *clause, VarAttnoSet *attnos);
+extern void find_forced_null_vars(Node *node, VarAttnoSet *attnos);
extern Var *find_forced_null_var(Node *node);
extern bool is_pseudo_constant_clause(Node *clause);
I wrote:
> I've been working away at this patch series, and here is an up-to-date
> version.
This needs a rebase after ff8fa0bf7 and b0b72c64a. I also re-ordered
the patches so that the commit messages' claims about when regression
tests start to pass are true again. No interesting changes, though.
regards, tom lane
commit 60c3d589ccc990a387e91f63f17eb78fbc3d9c3f
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sun Oct 30 15:39:58 2022 -0400
Add overview documentation.
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index 41c120e0cd..360d37bcaa 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -295,6 +295,239 @@ Therefore, we don't merge FROM-lists if the result would have too many
FROM-items in one list.
+Vars and PlaceHolderVars
+------------------------
+
+A Var node is simply the parse-tree representation of a table column
+reference. However, in the presence of outer joins, that concept is
+more subtle than it might seem. We need to distinguish the values of
+a Var "above" and "below" any outer join that could force the Var to
+null. As an example, consider
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE foo(t2.z)
+
+(Assume foo() is not strict, so that we can't reduce the left join to
+a plain join.) A naive implementation might try to push the foo(t2.z)
+call down to the scan of t2, but that is not correct because
+(a) what foo() should actually see for a null-extended join row is NULL,
+and (b) if foo() returns false, we should suppress the t1 row from the
+join altogether, not emit it with a null-extended t2 row. On the other
+hand, it *would* be correct (and desirable) to push the call down to
+the scan level if the query were
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y AND foo(t2.z))
+
+This motivates considering "t2.z" within the left join's ON clause
+to be a different value from "t2.z" outside the JOIN clause. The
+former can be identified with t2.z as seen at the relation scan level,
+but the latter can't.
+
+Another example occurs in connection with EquivalenceClasses (discussed
+below). Given
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE t1.x = 42
+
+we would like to use the EquivalenceClass mechanisms to derive "t2.y = 42"
+to use as a restriction clause for the scan of t2. (That works, because t2
+rows having y different from 42 cannot affect the query result.) However,
+it'd be wrong to conclude that t2.y will be equal to t1.x in every joined
+row. Part of the solution to this problem is to deem that "t2.y" in the
+ON clause refers to the relation-scan-level value of t2.y, but not to the
+value that y will have in joined rows, where it might be NULL rather than
+equal to t1.x.
+
+Therefore, Var nodes are decorated with "varnullingrels", which are sets
+of the rangetable indexes of outer joins that potentially null the Var
+at the point where it appears in the query. (Using a set, not an ordered
+list, is fine since it doesn't matter which join forced the value to null;
+and that avoids having to change the representation when we consider
+different outer-join orders.) In the examples above, all occurrences of
+t1.x would have empty varnullingrels, since the left join doesn't null t1.
+The t2 references within the JOIN ON clauses would also have empty
+varnullingrels. But outside the JOIN clauses, any Vars referencing t2
+would have varnullingrels containing the index of the JOIN's rangetable
+entry (RTE), so that they'd be understood as potentially different from
+the t2 values seen at scan level. Labeling t2.z in the WHERE clause with
+the JOIN's RT index lets us recognize that that occurrence of foo(t2.z)
+cannot be pushed down to the t2 scan level: we cannot evaluate that value
+at the scan level, but only after the join has been done.
+
+For LEFT and RIGHT outer joins, only Vars coming from the nullable side
+of the join are marked with that join's RT index. For FULL joins, Vars
+from both inputs are marked. (Such marking doesn't let us tell which
+side of the full join a Var came from; but that information can be found
+elsewhere at need.)
+
+Notionally, a Var having nonempty varnullingrels can be thought of as
+ CASE WHEN any-of-these-outer-joins-produced-a-null-extended-row
+ THEN NULL
+ ELSE the-scan-level-value-of-the-column
+ END
+It's only notional, because no such calculation is ever done explicitly.
+In a finished plan, Vars occurring in scan-level plan nodes represent
+the actual table column values, but upper-level Vars are always
+references to outputs of lower-level plan nodes. When a join node emits
+a null-extended row, it just returns nulls for the relevant output
+columns rather than copying up values from its input. Because we don't
+ever have to do this calculation explicitly, it's not necessary to
+distinguish which side of an outer join got null-extended, which'd
+otherwise be essential information for FULL JOIN cases.
+
+Outer join identity 3 (discussed above) complicates this picture
+a bit. In the form
+ A leftjoin (B leftjoin C on (Pbc)) on (Pab)
+all of the Vars in clauses Pbc and Pab will have empty varnullingrels,
+but if we start with
+ (A leftjoin B on (Pab)) leftjoin C on (Pbc)
+then the parser will have marked Pbc's B Vars with the A/B join's
+RT index, making this form artificially different from the first.
+For discussion's sake, let's denote this marking with a star:
+ (A leftjoin B on (Pab)) leftjoin C on (Pb*c)
+To cope with this, once we have detected that commuting these joins
+is legal, we generate both the Pbc and Pb*c forms of that ON clause,
+by either removing or adding the first join's RT index in the B Vars
+that the parser created. While generating paths for a plan step that
+joins B and C, we include as a relevant join qual only the form that
+is appropriate depending on whether A has already been joined to B.
+
+It's also worth noting that identity 3 makes "the left join's RT index"
+itself a bit of a fuzzy concept, since the syntactic scope of each join
+RTE will depend on which form was produced by the parser. We resolve
+this by considering that a left join's identity is determined by its
+minimum set of right-hand-side input relations. In both forms allowed
+by identity 3, we can identify the first join as having minimum RHS B
+and the second join as having minimum RHS C.
+
+Another thing to notice is that C Vars appearing outside the nested
+JOIN clauses will be marked as nulled by both left joins if the
+original parser input was in the first form of identity 3, but if the
+parser input was in the second form, such Vars will only be marked as
+nulled by the second join. This is not really a semantic problem:
+such Vars will be marked the same way throughout the upper part of the
+query, so they will all look equal() which is correct; and they will not
+look equal() to any C Var appearing in the JOIN ON clause or below these
+joins. However, when building Vars representing the outputs of join
+relations, we need to ensure that their varnullingrels are set to
+values consistent with the syntactic join order, so that they will
+appear equal() to pre-existing Vars in the upper part of the query.
+
+Outer joins also complicate handling of subquery pull-up. Consider
+
+ SELECT ..., ss.x FROM tab1
+ LEFT JOIN (SELECT *, 42 AS x FROM tab2) ss ON ...
+
+We want to be able to pull up the subquery as discussed previously,
+but we can't just replace the "ss.x" Var in the top-level SELECT list
+with the constant 42. That'd result in always emitting 42, rather
+than emitting NULL in null-extended join rows.
+
+To solve this, we introduce the concept of PlaceHolderVars.
+A PlaceHolderVar is somewhat like a Var, in that its value originates
+at a relation scan level and can then be forced to null by higher-level
+outer joins; hence PlaceHolderVars carry a set of nulling rel IDs just
+like Vars. Unlike a Var, whose original value comes from a table,
+a PlaceHolderVar's original value is defined by a query-determined
+expression ("42" in this example); so we represent the PlaceHolderVar
+as a node with that expression as child. We insert a PlaceHolderVar
+whenever subquery pullup needs to replace a subquery-referencing Var
+that has nonempty varnullingrels with an expression that is not simply a
+Var. (When the replacement expression is a pulled-up Var, we can just
+add the replaced Var's varnullingrels to its set. Also, if the replaced
+Var has empty varnullingrels, we don't need a PlaceHolderVar: there is
+nothing that'd force the value to null, so the pulled-up expression is
+fine to use as-is.) In a finished plan, a PlaceHolderVar becomes just
+the contained expression at whatever plan level it's supposed to be
+evaluated at, and then upper-level occurrences are replaced by
+references to that output column of the lower plan level. That causes
+the value to go to null when appropriate at an outer join, in the same
+way as for Vars. Thus, PlaceHolderVars are never seen outside the
+planner.
+
+PlaceHolderVars (PHVs) are more complicated than Vars in another way:
+their original value might need to be calculated at a join, not a
+base-level relation scan. This can happen if a pulled-up subquery
+contains a join. Because of this, a PHV can create a join order
+constraint that wouldn't otherwise exist, to ensure that it can
+be calculated before it is used. A PHV's expression can also contain
+LATERAL references, adding complications that are discussed below.
+
+
+Relation Identification and Qual Clause Placement
+-------------------------------------------------
+
+A qual clause obtained from WHERE or JOIN/ON can be enforced at the lowest
+scan or join level that includes all relations used in the clause. For
+this purpose we consider that outer joins listed in varnullingrels or
+phnullingrels are used in the clause, since we can't compute the qual's
+result correctly until we know whether such Vars have gone to null.
+
+The one exception to this general rule is that a non-degenerate outer
+JOIN/ON qual (one that references the non-nullable side of the join)
+cannot be enforced below that join, even if it doesn't reference the
+nullable side. Pushing it down into the non-nullable side would result
+in rows disappearing from the join's result, rather than appearing as
+null-extended rows. To handle that, when we identify such a qual we
+artificially add the join's minimum input relid set to the set of
+relations it is considered to use, forcing it to be evaluated exactly at
+that join level. The same happens for outer-join quals that mention no
+relations at all.
+
+When attaching a qual clause to a join plan node that is performing an
+outer join, the qual clause is considered a "join clause" (that is, it is
+applied before the join performs null-extension) if it does not reference
+that outer join in any varnullingrels or phnullingrels set, or a "filter
+clause" (applied after null-extension) if it does reference that outer
+join. A qual clause that originally appeared in that outer join's JOIN/ON
+will fall into the first category, since the parser would not have marked
+any of its Vars as referencing the outer join. A qual clause that
+originally came from some upper ON clause or WHERE clause will be seen as
+referencing the outer join if it references any of the nullable side's
+Vars, since those Vars will be so marked by the parser. But, if such a
+qual does not reference any nullable-side Vars, it's okay to push it down
+into the non-nullable side, so it won't get attached to the join node in
+the first place.
+
+These things lead us to identify join relations within the planner
+by the sets of base relation RT indexes plus outer join RT indexes
+that they include. In that way, the sets of relations used by qual
+clauses can be directly compared to join relations' relid sets to
+see where to place the clauses. These identifying sets are unique
+because, for any given collection of base relations, there is only
+one valid set of outer joins to have performed along the way to
+joining that set of base relations (although the order of applying
+them could vary, as discussed above).
+
+SEMI joins do not have RT indexes, because they are artifacts made by
+the planner rather than the parser. (We could create rangetable
+entries for them, but there seems no need at present.) This does not
+cause a problem for qual placement, because the nullable side of a
+semijoin is not referenceable from above the join, so there is never a
+need to cite it in varnullingrels or phnullingrels. It does not cause a
+problem for join relation identification either, since whether a semijoin
+has been completed is again implicit in the set of base relations
+included in the join.
+
+There is one additional complication for qual clause placement, which
+occurs when we have made multiple versions of an outer-join clause as
+described previously (that is, we have both "Pbc" and "Pb*c" forms of
+the same clause seen in outer join identity 3). When forming an outer
+join we only want to apply one of the redundant versions of the clause.
+If we are forming the B/C join without having yet computed the A/B
+join, it's easy to reject the "Pb*c" form since its required relid
+set includes the A/B join relid which is not in the input. However,
+if we form B/C after A/B, then both forms of the clause are applicable
+so far as that test can tell. We have to look more closely to notice
+that the "Pbc" clause form refers to relation B which is no longer
+directly accessible. While this check is straightforward, it's not
+especially cheap (see clause_is_computable_at()). To avoid doing it
+unnecessarily, we mark the variant versions of a redundant clause as
+either "has_clone" or "is_clone". A production build of Postgres
+checks restriction_is_computable_at() to disentangle which clone copy
+to apply at a given join level. In debug builds, we also Assert that
+non-clone clauses are validly computable, but that seems too expensive
+for production usage.
+
+
Optimizer Functions
-------------------
@@ -437,11 +670,10 @@ inputs.
EquivalenceClasses
------------------
-During the deconstruct_jointree() scan of the query's qual clauses, we look
-for mergejoinable equality clauses A = B whose applicability is not delayed
-by an outer join; these are called "equivalence clauses". When we find
-one, we create an EquivalenceClass containing the expressions A and B to
-record this knowledge. If we later find another equivalence clause B = C,
+During the deconstruct_jointree() scan of the query's qual clauses, we
+look for mergejoinable equality clauses A = B. When we find one, we
+create an EquivalenceClass containing the expressions A and B to record
+that they are equal. If we later find another equivalence clause B = C,
we add C to the existing EquivalenceClass for {A B}; this may require
merging two existing EquivalenceClasses. At the end of the scan, we have
sets of values that are known all transitively equal to each other. We can
@@ -473,15 +705,54 @@ asserts that at any plan node where more than one of its member values
can be computed, output rows in which the values are not all equal may
be discarded without affecting the query result. (We require all levels
of the plan to enforce EquivalenceClasses, hence a join need not recheck
-equality of values that were computable by one of its children.) For an
-ordinary EquivalenceClass that is "valid everywhere", we can further infer
-that the values are all non-null, because all mergejoinable operators are
-strict. However, we also allow equivalence clauses that appear below the
-nullable side of an outer join to form EquivalenceClasses; for these
-classes, the interpretation is that either all the values are equal, or
-all (except pseudo-constants) have gone to null. (This requires a
-limitation that non-constant members be strict, else they might not go
-to null when the other members do.) Consider for example
+equality of values that were computable by one of its children.)
+
+It's tempting to include equality clauses appearing in outer-join
+conditions as sources of EquivalenceClasses, but there's a serious
+difficulty: the resulting deductions are not valid everywhere.
+For example, given
+
+ SELECT * FROM a LEFT JOIN b ON a.x = b.y WHERE a.x = 42;
+
+we could safely derive b.y = 42 and use that in the scan of B,
+because B rows not having b.y = 42 will not contribute to the
+join result. Likewise, given
+
+ SELECT * FROM a LEFT JOIN b ON a.x = b.y AND a.x = b.z;
+
+it's all right to apply b.y = b.z while scanning B, and then only
+one of the two equality conditions need be tested at the join.
+However, if we have
+
+ SELECT * FROM a LEFT JOIN b ON a.x1 = b.y AND a.x2 = b.y;
+
+it'd be completely incorrect to push "a.x1 = a.x2" down to the scan
+of A. Rows where they are different should not be eliminated from
+the join result, but instead produce null-extended join rows.
+
+In general, therefore, we can treat outer-join equalities somewhat like
+real equivalences, but we can only produce derived clauses at that
+outer join and at scans and joins contained within its nullable side.
+(FULL JOIN conditions can't be optimized at all this way, since derived
+clauses couldn't be enforced on either side.)
+
+Another instructive example is:
+
+ SELECT *
+ FROM a LEFT JOIN
+ (SELECT * FROM b JOIN c ON b.y = c.z WHERE b.y = 10) ss
+ ON a.x = ss.y
+ ORDER BY ss.y;
+
+We can form the EquivalenceClass {b.y c.z 10} and thereby apply c.z = 10
+while scanning c. However, this does not tell us anything about the
+ss.y reference appearing in ORDER BY (which is another name for b.y*,
+that is the possibly-nulled form of b.y), so we don't get to conclude
+that sorting for the ORDER BY is unnecessary, as it would be if we could
+prove that b.y* is equal to a constant (see discussion of PathKeys
+below).
+
+Also consider this variant:
SELECT *
FROM a LEFT JOIN
@@ -489,40 +760,60 @@ to null when the other members do.) Consider for example
ON a.x = ss.y
WHERE a.x = 42;
-We can form the below-outer-join EquivalenceClass {b.y c.z 10} and thereby
-apply c.z = 10 while scanning c. (The reason we disallow outerjoin-delayed
-clauses from forming EquivalenceClasses is exactly that we want to be able
-to push any derived clauses as far down as possible.) But once above the
-outer join it's no longer necessarily the case that b.y = 10, and thus we
-cannot use such EquivalenceClasses to conclude that sorting is unnecessary
-(see discussion of PathKeys below).
-
-In this example, notice also that a.x = ss.y (really a.x = b.y) is not an
-equivalence clause because its applicability to b is delayed by the outer
-join; thus we do not try to insert b.y into the equivalence class {a.x 42}.
-But since we see that a.x has been equated to 42 above the outer join, we
-are able to form a below-outer-join class {b.y 42}; this restriction can be
-added because no b/c row not having b.y = 42 can contribute to the result
-of the outer join, and so we need not compute such rows. Now this class
-will get merged with {b.y c.z 10}, leading to the contradiction 10 = 42,
+Here, we have an EquivalenceClass {a.x 42} in addition to {b.y c.z 10},
+and we have an outer-join condition a.x = b.y (not b.y*). That lets us
+derive b.y = 42, but we can only constrain scans/joins below the left join
+that way. Nonetheless, we can still produce the contradiction 10 = 42,
which lets the planner deduce that the b/c join need not be computed at all
because none of its rows can contribute to the outer join. (This gets
implemented as a gating Result filter, since more usually the potential
contradiction involves Param values rather than just Consts, and thus has
to be checked at runtime.)
+To handle outer-join conditions this way, we put their left and right
+operands into EquivalenceClasses in the usual way. (This may result in
+creating single-item equivalence "classes", though of course these are
+still subject to merging if other equivalence clauses are found that
+mention the same Vars.) We do not merge those two EquivalenceClasses
+as would happen with an ordinary equivalence condition. Instead, the
+outer-join condition is recorded in a separate "ConstrainedEquivalence"
+data structure, showing the EquivalenceClasses it connects and the scope
+of the outer join that it is valid within. We can make deductions as
+if the two classes were one, but only when considering a scan or join
+within the scope of the constrained equivalence.
+
To aid in determining the sort ordering(s) that can work with a mergejoin,
we mark each mergejoinable clause with the EquivalenceClasses of its left
-and right inputs. For an equivalence clause, these are of course the same
-EquivalenceClass. For a non-equivalence mergejoinable clause (such as an
-outer-join qualification), we generate two separate EquivalenceClasses for
-the left and right inputs. This may result in creating single-item
-equivalence "classes", though of course these are still subject to merging
-if other equivalence clauses are later found to bear on the same
-expressions.
+and right inputs. For an ordinary equivalence clause these will be the
+same EquivalenceClass, since processing of the clause itself causes its
+inputs to be put into the same EquivalenceClass. But as described above,
+mergejoinable outer-join clauses will end up with different
+EquivalenceClasses for left and right sides.
+
+There is an additional complication when re-ordering outer joins according
+to identity 3. Recall that the two choices we consider for such joins are
+ A leftjoin (B leftjoin C on (Pbc)) on (Pab)
+ (A leftjoin B on (Pab)) leftjoin C on (Pb*c)
+where the star denotes varnullingrels markers on B's Vars. When Pbc
+is (or includes) a mergejoinable clause, we have something like
+ A leftjoin (B leftjoin C on (b.b = c.c)) on (Pab)
+ (A leftjoin B on (Pab)) leftjoin C on (b.b* = c.c)
+We could generate a ConstrainedEquivalence linking b.b and c.c, and
+another one linking b.b* and c.c. (b.b and b.b* are necessarily in
+different EquivalenceClasses: there is no mechanism whereby they
+could be found to be equal.) However, these would generate largely
+duplicative conditions. Conditions involving b.b* can't be computed
+below this join nest, and any that can be computed would be duplicative
+of what we'd get from the b.b/c.c ConstrainedEquivalence. Therefore,
+we choose to generate a ConstrainedEquivalence for b.b and c.c, but
+"b.b* = c.c" is handled as just an ordinary clause.
Another way that we may form a single-item EquivalenceClass is in creation
-of a PathKey to represent a desired sort order (see below). This is a bit
+of a PathKey to represent a desired sort order (see below). This happens
+if an ORDER BY or GROUP BY key is not mentioned in any equivalence
+clause. We need to reason about sort orders in such queries, and our
+representation of sort ordering is a PathKey (see below) which uses an
+EquivalenceClass, so we have to make an EquivalenceClass. This is a bit
different from the above cases because such an EquivalenceClass might
contain an aggregate function or volatile expression. (A clause containing
a volatile function will never be considered mergejoinable, even if its top
@@ -579,7 +870,7 @@ Index scans have Path.pathkeys that represent the chosen index's ordering,
if any. A single-key index would create a single-PathKey list, while a
multi-column index generates a list with one element per key index column.
Non-key columns specified in the INCLUDE clause of covering indexes don't
-have corresponding PathKeys in the list, because the have no influence on
+have corresponding PathKeys in the list, because they have no influence on
index ordering. (Actually, since an index can be scanned either forward or
backward, there are two possible sort orders and two possible PathKey lists
it can generate.)
@@ -655,14 +946,9 @@ redundancy, we save time and improve planning, since the planner will more
easily recognize equivalent orderings as being equivalent.
Another interesting property is that if the underlying EquivalenceClass
-contains a constant and is not below an outer join, then the pathkey is
-completely redundant and need not be sorted by at all! Every row must
-contain the same constant value, so there's no need to sort. (If the EC is
-below an outer join, we still have to sort, since some of the rows might
-have gone to null and others not. In this case we must be careful to pick
-a non-const member to sort by. The assumption that all the non-const
-members go to null at the same plan level is critical here, else they might
-not produce the same sort order.) This might seem pointless because users
+contains a constant, then the pathkey is completely redundant and need not
+be sorted by at all! Every interesting row must contain the same value,
+so there's no need to sort. This might seem pointless because users
are unlikely to write "... WHERE x = 42 ORDER BY x", but it allows us to
recognize when particular index columns are irrelevant to the sort order:
if we have "... WHERE x = 42 ORDER BY y", scanning an index on (x,y)
@@ -670,15 +956,6 @@ produces correctly ordered data without a sort step. We used to have very
ugly ad-hoc code to recognize that in limited contexts, but discarding
constant ECs from pathkeys makes it happen cleanly and automatically.
-You might object that a below-outer-join EquivalenceClass doesn't always
-represent the same values at every level of the join tree, and so using
-it to uniquely identify a sort order is dubious. This is true, but we
-can avoid dealing with the fact explicitly because we always consider that
-an outer join destroys any ordering of its nullable inputs. Thus, even
-if a path was sorted by {a.x} below an outer join, we'll re-sort if that
-sort ordering was important; and so using the same PathKey for both sort
-orderings doesn't create any real problem.
-
Order of processing for EquivalenceClasses and PathKeys
-------------------------------------------------------
commit c53fadf1db050c55cf8b331cdff96f32b859a4ac
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu Oct 27 14:56:08 2022 -0400
Add Var.varnullingrels and PlaceHolderVar.phnullingrels fields.
These fields are always empty as of this commit, so they don't
affect any behavior, even though equal() will compare them.
Update backend/nodes/ and backend/rewrite/ infrastructure as needed.
Also add some rewrite functions we'll need later.
Note this will require a catversion bump when committed.
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index c85d8fe975..cced668f58 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -80,11 +80,13 @@ makeVar(int varno,
var->varlevelsup = varlevelsup;
/*
- * Only a few callers need to make Var nodes with varnosyn/varattnosyn
- * different from varno/varattno. We don't provide separate arguments for
- * them, but just initialize them to the given varno/varattno. This
- * reduces code clutter and chance of error for most callers.
+ * Only a few callers need to make Var nodes with non-null varnullingrels,
+ * or with varnosyn/varattnosyn different from varno/varattno. We don't
+ * provide separate arguments for them, but just initialize them to NULL
+ * and the given varno/varattno. This reduces code clutter and chance of
+ * error for most callers.
*/
+ var->varnullingrels = NULL;
var->varnosyn = (Index) varno;
var->varattnosyn = varattno;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 0a7b22f97e..692f45daa5 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -2663,6 +2663,7 @@ expression_tree_mutator_impl(Node *node,
Var *newnode;
FLATCOPY(newnode, var, Var);
+ /* Assume we need not copy the varnullingrels bitmapset */
return (Node *) newnode;
}
break;
@@ -3257,7 +3258,7 @@ expression_tree_mutator_impl(Node *node,
FLATCOPY(newnode, phv, PlaceHolderVar);
MUTATE(newnode->phexpr, phv->phexpr, Expr *);
- /* Assume we need not copy the relids bitmapset */
+ /* Assume we need not copy the relids bitmapsets */
return (Node *) newnode;
}
break;
diff --git a/src/backend/rewrite/rewriteManip.c b/src/backend/rewrite/rewriteManip.c
index 101c39553a..ab24547e6d 100644
--- a/src/backend/rewrite/rewriteManip.c
+++ b/src/backend/rewrite/rewriteManip.c
@@ -40,6 +40,20 @@ typedef struct
int win_location;
} locate_windowfunc_context;
+typedef struct
+{
+ const Bitmapset *target_relids;
+ const Bitmapset *added_relids;
+ int sublevels_up;
+} add_nulling_relids_context;
+
+typedef struct
+{
+ const Bitmapset *removable_relids;
+ const Bitmapset *except_relids;
+ int sublevels_up;
+} remove_nulling_relids_context;
+
static bool contain_aggs_of_level_walker(Node *node,
contain_aggs_of_level_context *context);
static bool locate_agg_of_level_walker(Node *node,
@@ -50,6 +64,10 @@ static bool locate_windowfunc_walker(Node *node,
static bool checkExprHasSubLink_walker(Node *node, void *context);
static Relids offset_relid_set(Relids relids, int offset);
static Relids adjust_relid_set(Relids relids, int oldrelid, int newrelid);
+static Node *add_nulling_relids_mutator(Node *node,
+ add_nulling_relids_context *context);
+static Node *remove_nulling_relids_mutator(Node *node,
+ remove_nulling_relids_context *context);
/*
@@ -348,6 +366,8 @@ OffsetVarNodes_walker(Node *node, OffsetVarNodes_context *context)
if (var->varlevelsup == context->sublevels_up)
{
var->varno += context->offset;
+ var->varnullingrels = offset_relid_set(var->varnullingrels,
+ context->offset);
if (var->varnosyn > 0)
var->varnosyn += context->offset;
}
@@ -386,6 +406,8 @@ OffsetVarNodes_walker(Node *node, OffsetVarNodes_context *context)
{
phv->phrels = offset_relid_set(phv->phrels,
context->offset);
+ phv->phnullingrels = offset_relid_set(phv->phnullingrels,
+ context->offset);
}
/* fall through to examine children */
}
@@ -510,11 +532,13 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context)
{
Var *var = (Var *) node;
- if (var->varlevelsup == context->sublevels_up &&
- var->varno == context->rt_index)
+ if (var->varlevelsup == context->sublevels_up)
{
- var->varno = context->new_index;
- /* If the syntactic referent is same RTE, fix it too */
+ if (var->varno == context->rt_index)
+ var->varno = context->new_index;
+ var->varnullingrels = adjust_relid_set(var->varnullingrels,
+ context->rt_index,
+ context->new_index);
if (var->varnosyn == context->rt_index)
var->varnosyn = context->new_index;
}
@@ -557,6 +581,9 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context)
phv->phrels = adjust_relid_set(phv->phrels,
context->rt_index,
context->new_index);
+ phv->phnullingrels = adjust_relid_set(phv->phnullingrels,
+ context->rt_index,
+ context->new_index);
}
/* fall through to examine children */
}
@@ -833,7 +860,8 @@ rangeTableEntry_used_walker(Node *node,
Var *var = (Var *) node;
if (var->varlevelsup == context->sublevels_up &&
- var->varno == context->rt_index)
+ (var->varno == context->rt_index ||
+ bms_is_member(context->rt_index, var->varnullingrels)))
return true;
return false;
}
@@ -1061,6 +1089,195 @@ AddInvertedQual(Query *parsetree, Node *qual)
}
+/*
+ * add_nulling_relids() finds Vars and PlaceHolderVars that belong to any
+ * of the target_relids, and adds added_relids to their varnullingrels
+ * and phnullingrels fields.
+ */
+Node *
+add_nulling_relids(Node *node,
+ const Bitmapset *target_relids,
+ const Bitmapset *added_relids)
+{
+ add_nulling_relids_context context;
+
+ context.target_relids = target_relids;
+ context.added_relids = added_relids;
+ context.sublevels_up = 0;
+ return query_or_expression_tree_mutator(node,
+ add_nulling_relids_mutator,
+ &context,
+ 0);
+}
+
+static Node *
+add_nulling_relids_mutator(Node *node,
+ add_nulling_relids_context *context)
+{
+ if (node == NULL)
+ return NULL;
+ if (IsA(node, Var))
+ {
+ Var *var = (Var *) node;
+
+ if (var->varlevelsup == context->sublevels_up &&
+ bms_is_member(var->varno, context->target_relids))
+ {
+ Relids newnullingrels = bms_union(var->varnullingrels,
+ context->added_relids);
+
+ /* Copy the Var ... */
+ var = copyObject(var);
+ /* ... and replace the copy's varnullingrels field */
+ var->varnullingrels = newnullingrels;
+ return (Node *) var;
+ }
+ /* Otherwise fall through to copy the Var normally */
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ if (phv->phlevelsup == context->sublevels_up &&
+ bms_overlap(phv->phrels, context->target_relids))
+ {
+ Relids newnullingrels = bms_union(phv->phnullingrels,
+ context->added_relids);
+
+ /*
+ * We don't modify the contents of the PHV's expression, only add
+ * to phnullingrels. This corresponds to assuming that the PHV
+ * will be evaluated at the same level as before, then perhaps be
+ * nulled as it bubbles up. Hence, just flat-copy the node ...
+ */
+ phv = makeNode(PlaceHolderVar);
+ memcpy(phv, node, sizeof(PlaceHolderVar));
+ /* ... and replace the copy's phnullingrels field */
+ phv->phnullingrels = newnullingrels;
+ return (Node *) phv;
+ }
+ /* Otherwise fall through to copy the PlaceHolderVar normally */
+ }
+ else if (IsA(node, Query))
+ {
+ /* Recurse into RTE or sublink subquery */
+ Query *newnode;
+
+ context->sublevels_up++;
+ newnode = query_tree_mutator((Query *) node,
+ add_nulling_relids_mutator,
+ (void *) context,
+ 0);
+ context->sublevels_up--;
+ return (Node *) newnode;
+ }
+ return expression_tree_mutator(node, add_nulling_relids_mutator,
+ (void *) context);
+}
+
+/*
+ * remove_nulling_relids() removes mentions of the specified RT index(es)
+ * in Var.varnullingrels and PlaceHolderVar.phnullingrels fields within
+ * the given expression, except in nodes belonging to rels listed in
+ * except_relids.
+ */
+Node *
+remove_nulling_relids(Node *node,
+ const Bitmapset *removable_relids,
+ const Bitmapset *except_relids)
+{
+ remove_nulling_relids_context context;
+
+ context.removable_relids = removable_relids;
+ context.except_relids = except_relids;
+ context.sublevels_up = 0;
+ return query_or_expression_tree_mutator(node,
+ remove_nulling_relids_mutator,
+ &context,
+ 0);
+}
+
+static Node *
+remove_nulling_relids_mutator(Node *node,
+ remove_nulling_relids_context *context)
+{
+ if (node == NULL)
+ return NULL;
+ if (IsA(node, Var))
+ {
+ Var *var = (Var *) node;
+
+ if (var->varlevelsup == context->sublevels_up &&
+ !bms_is_member(var->varno, context->except_relids) &&
+ bms_overlap(var->varnullingrels, context->removable_relids))
+ {
+ Relids newnullingrels = bms_difference(var->varnullingrels,
+ context->removable_relids);
+
+ /* Micro-optimization: ensure nullingrels is NULL if empty */
+ if (bms_is_empty(newnullingrels))
+ newnullingrels = NULL;
+ /* Copy the Var ... */
+ var = copyObject(var);
+ /* ... and replace the copy's varnullingrels field */
+ var->varnullingrels = newnullingrels;
+ return (Node *) var;
+ }
+ /* Otherwise fall through to copy the Var normally */
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ if (phv->phlevelsup == context->sublevels_up &&
+ !bms_overlap(phv->phrels, context->except_relids))
+ {
+ Relids newnullingrels = bms_difference(phv->phnullingrels,
+ context->removable_relids);
+
+ /*
+ * Micro-optimization: ensure nullingrels is NULL if empty.
+ *
+ * Note: it might seem desirable to remove the PHV altogether if
+ * phnullingrels goes to empty. Currently we dare not do that
+ * because we use PHVs in some cases to enforce separate identity
+ * of subexpressions; see wrap_non_vars usages in prepjointree.c.
+ */
+ if (bms_is_empty(newnullingrels))
+ newnullingrels = NULL;
+ /* Copy the PlaceHolderVar and mutate what's below ... */
+ phv = (PlaceHolderVar *)
+ expression_tree_mutator(node,
+ remove_nulling_relids_mutator,
+ (void *) context);
+ /* ... and replace the copy's phnullingrels field */
+ phv->phnullingrels = newnullingrels;
+ /* We must also update phrels, if it contains a removable RTI */
+ phv->phrels = bms_difference(phv->phrels,
+ context->removable_relids);
+ Assert(!bms_is_empty(phv->phrels));
+ return (Node *) phv;
+ }
+ /* Otherwise fall through to copy the PlaceHolderVar normally */
+ }
+ else if (IsA(node, Query))
+ {
+ /* Recurse into RTE or sublink subquery */
+ Query *newnode;
+
+ context->sublevels_up++;
+ newnode = query_tree_mutator((Query *) node,
+ remove_nulling_relids_mutator,
+ (void *) context,
+ 0);
+ context->sublevels_up--;
+ return (Node *) newnode;
+ }
+ return expression_tree_mutator(node, remove_nulling_relids_mutator,
+ (void *) context);
+}
+
+
/*
* replace_rte_variables() finds all Vars in an expression tree
* that reference a particular RTE, and replaces them with substitute
diff --git a/src/backend/utils/misc/queryjumble.c b/src/backend/utils/misc/queryjumble.c
index a8508463e7..c0b254fc6e 100644
--- a/src/backend/utils/misc/queryjumble.c
+++ b/src/backend/utils/misc/queryjumble.c
@@ -383,6 +383,11 @@ JumbleExpr(JumbleState *jstate, Node *node)
APP_JUMB(var->varno);
APP_JUMB(var->varattno);
APP_JUMB(var->varlevelsup);
+
+ /*
+ * We can omit varnullingrels, because it's fully determined
+ * by varno/varlevelsup plus the Var's query location.
+ */
}
break;
case T_Const:
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 09342d128d..2c6d5ca58f 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -874,7 +874,7 @@ typedef struct RelOptInfo
int32 *attr_widths pg_node_attr(read_write_ignore);
/* LATERAL Vars and PHVs referenced by rel */
List *lateral_vars;
- /* rels that reference me laterally */
+ /* rels that reference this baserel laterally */
Relids lateral_referencers;
/* list of IndexOptInfo */
List *indexlist;
@@ -884,10 +884,7 @@ typedef struct RelOptInfo
BlockNumber pages;
Cardinality tuples;
double allvisfrac;
-
- /*
- * Indexes in PlannerInfo's eq_classes list of ECs that mention this rel
- */
+ /* indexes in PlannerInfo's eq_classes list of ECs that mention this rel */
Bitmapset *eclass_indexes;
PlannerInfo *subroot; /* if subquery */
List *subplan_params; /* if subquery */
@@ -2586,10 +2583,15 @@ typedef struct MergeScanSelCache
* of a plan tree. This is used during planning to represent the contained
* expression. At the end of the planning process it is replaced by either
* the contained expression or a Var referring to a lower-level evaluation of
- * the contained expression. Typically the evaluation occurs below an outer
+ * the contained expression. Generally the evaluation occurs below an outer
* join, and Var references above the outer join might thereby yield NULL
* instead of the expression value.
*
+ * phrels and phlevelsup correspond to the varno/varlevelsup fields of a
+ * plain Var, except that phrels has to be a relid set since the evaluation
+ * level of a PlaceHolderVar might be a join rather than a base relation.
+ * Likewise, phnullingrels corresponds to varnullingrels.
+ *
* Although the planner treats this as an expression node type, it is not
* recognized by the parser or executor, so we declare it here rather than
* in primnodes.h.
@@ -2602,8 +2604,10 @@ typedef struct MergeScanSelCache
* PHV. Another way in which it can happen is that initplan sublinks
* could get replaced by differently-numbered Params when sublink folding
* is done. (The end result of such a situation would be some
- * unreferenced initplans, which is annoying but not really a problem.) On
- * the same reasoning, there is no need to examine phrels.
+ * unreferenced initplans, which is annoying but not really a problem.)
+ * On the same reasoning, there is no need to examine phrels. But we do
+ * need to compare phnullingrels, as that represents effects that are
+ * external to the original value of the PHV.
*/
typedef struct PlaceHolderVar
@@ -2613,9 +2617,12 @@ typedef struct PlaceHolderVar
/* the represented expression */
Expr *phexpr pg_node_attr(equal_ignore);
- /* base relids syntactically within expr src */
+ /* base+OJ relids syntactically within expr src */
Relids phrels pg_node_attr(equal_ignore);
+ /* RT indexes of outer joins that can null PHV's value */
+ Relids phnullingrels;
+
/* ID for PHV (unique within planner run) */
Index phid;
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index f71f551782..8f133e12ac 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -180,6 +180,14 @@ typedef struct Expr
* row identity information during UPDATE/DELETE/MERGE. This value should
* never be seen outside the planner.
*
+ * varnullingrels is the set of RT indexes of outer joins that can force
+ * the Var's value to null (at the point where it appears in the query).
+ * See optimizer/README for discussion of that.
+ *
+ * varlevelsup is greater than zero in Vars that represent outer references.
+ * Note that it affects the meaning of all of varno, varnullingrels, and
+ * varnosyn, all of which refer to the range table of that query level.
+ *
* In the parser, varnosyn and varattnosyn are either identical to
* varno/varattno, or they specify the column's position in an aliased JOIN
* RTE that hides the semantic referent RTE's refname. This is a syntactic
@@ -222,6 +230,8 @@ typedef struct Var
int32 vartypmod;
/* OID of collation, or InvalidOid if none */
Oid varcollid;
+ /* RT indexes of outer joins that can replace the Var's value with null */
+ Bitmapset *varnullingrels;
/*
* for subquery variables referencing outer relations; 0 in a normal var,
diff --git a/src/include/rewrite/rewriteManip.h b/src/include/rewrite/rewriteManip.h
index f001ca41bb..351ec15612 100644
--- a/src/include/rewrite/rewriteManip.h
+++ b/src/include/rewrite/rewriteManip.h
@@ -63,6 +63,13 @@ extern bool contain_windowfuncs(Node *node);
extern int locate_windowfunc(Node *node);
extern bool checkExprHasSubLink(Node *node);
+extern Node *add_nulling_relids(Node *node,
+ const Bitmapset *target_relids,
+ const Bitmapset *added_relids);
+extern Node *remove_nulling_relids(Node *node,
+ const Bitmapset *removable_relids,
+ const Bitmapset *except_relids);
+
extern Node *replace_rte_variables(Node *node,
int target_varno, int sublevels_up,
replace_rte_variables_callback callback,
commit 5b4b3e7992bee83dba970267799cce8bda194ae1
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu Oct 27 14:59:43 2022 -0400
Teach the parser to fill Var.varnullingrels correctly.
Vars emitted by the parser are now marked with RT indexes of outer
joins that can null them. (This is done purely according to the
syntax of the query; we don't consider whether an outer join could
be strength-reduced, for example.)
Although the result of this step compiles, it will fail some
regression tests due to the planner not yet knowing what to do.
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 6688c2a865..dff3b1e349 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -670,6 +670,7 @@ transformInsertStmt(ParseState *pstate, InsertStmt *stmt)
*/
sub_pstate->p_rtable = sub_rtable;
sub_pstate->p_joinexprs = NIL; /* sub_rtable has no joins */
+ sub_pstate->p_nullingrels = NIL;
sub_pstate->p_namespace = sub_namespace;
sub_pstate->p_resolve_unknowns = false;
@@ -851,7 +852,7 @@ transformInsertStmt(ParseState *pstate, InsertStmt *stmt)
/*
* Generate list of Vars referencing the RTE
*/
- exprList = expandNSItemVars(nsitem, 0, -1, NULL);
+ exprList = expandNSItemVars(pstate, nsitem, 0, -1, NULL);
/*
* Re-apply any indirection on the target column specs to the Vars
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index e01c0734d1..95590d9ed2 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -52,7 +52,8 @@
#include "utils/syscache.h"
-static int extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
+static int extractRemainingColumns(ParseState *pstate,
+ ParseNamespaceColumn *src_nscolumns,
List *src_colnames,
List **src_colnos,
List **res_colnames, List **res_colvars,
@@ -75,9 +76,11 @@ static ParseNamespaceItem *getNSItemForSpecialRelationTypes(ParseState *pstate,
static Node *transformFromClauseItem(ParseState *pstate, Node *n,
ParseNamespaceItem **top_nsitem,
List **namespace);
-static Var *buildVarFromNSColumn(ParseNamespaceColumn *nscol);
+static Var *buildVarFromNSColumn(ParseState *pstate,
+ ParseNamespaceColumn *nscol);
static Node *buildMergedJoinVar(ParseState *pstate, JoinType jointype,
Var *l_colvar, Var *r_colvar);
+static void markRelsAsNulledBy(ParseState *pstate, Node *n, int jindex);
static void setNamespaceColumnVisibility(List *namespace, bool cols_visible);
static void setNamespaceLateralState(List *namespace,
bool lateral_only, bool lateral_ok);
@@ -251,7 +254,8 @@ setTargetTable(ParseState *pstate, RangeVar *relation,
* Returns the number of columns added.
*/
static int
-extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
+extractRemainingColumns(ParseState *pstate,
+ ParseNamespaceColumn *src_nscolumns,
List *src_colnames,
List **src_colnos,
List **res_colnames, List **res_colvars,
@@ -287,7 +291,8 @@ extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
*src_colnos = lappend_int(*src_colnos, attnum);
*res_colnames = lappend(*res_colnames, lfirst(lc));
*res_colvars = lappend(*res_colvars,
- buildVarFromNSColumn(src_nscolumns + attnum - 1));
+ buildVarFromNSColumn(pstate,
+ src_nscolumns + attnum - 1));
/* Copy the input relation's nscolumn data for this column */
res_nscolumns[colcount] = src_nscolumns[attnum - 1];
colcount++;
@@ -1288,8 +1293,7 @@ transformFromClauseItem(ParseState *pstate, Node *n,
{
/*
* JOIN/USING (or NATURAL JOIN, as transformed above). Transform
- * the list into an explicit ON-condition, and generate a list of
- * merged result columns.
+ * the list into an explicit ON-condition.
*/
List *ucols = j->usingClause;
List *l_usingvars = NIL;
@@ -1307,8 +1311,6 @@ transformFromClauseItem(ParseState *pstate, Node *n,
int r_index = -1;
Var *l_colvar,
*r_colvar;
- Node *u_colvar;
- ParseNamespaceColumn *res_nscolumn;
Assert(u_colname[0] != '\0');
@@ -1372,17 +1374,109 @@ transformFromClauseItem(ParseState *pstate, Node *n,
u_colname)));
r_colnos = lappend_int(r_colnos, r_index + 1);
- l_colvar = buildVarFromNSColumn(l_nscolumns + l_index);
+ /* Build Vars to use in the generated JOIN ON clause */
+ l_colvar = buildVarFromNSColumn(pstate, l_nscolumns + l_index);
l_usingvars = lappend(l_usingvars, l_colvar);
- r_colvar = buildVarFromNSColumn(r_nscolumns + r_index);
+ r_colvar = buildVarFromNSColumn(pstate, r_nscolumns + r_index);
r_usingvars = lappend(r_usingvars, r_colvar);
+ /*
+ * While we're here, add column names to the res_colnames
+ * list. It's a bit ugly to do this here while the
+ * corresponding res_colvars entries are not made till later,
+ * but doing this later would require an additional traversal
+ * of the usingClause list.
+ */
res_colnames = lappend(res_colnames, lfirst(ucol));
+ }
+
+ /* Construct the generated JOIN ON clause */
+ j->quals = transformJoinUsingClause(pstate,
+ l_usingvars,
+ r_usingvars);
+ }
+ else if (j->quals)
+ {
+ /* User-written ON-condition; transform it */
+ j->quals = transformJoinOnClause(pstate, j, my_namespace);
+ }
+ else
+ {
+ /* CROSS JOIN: no quals */
+ }
+
+ /*
+ * If this is an outer join, now mark the appropriate child RTEs as
+ * being nulled by this join. We have finished processing the child
+ * join expressions as well as the current join's quals, which deal in
+ * non-nulled input columns. All future references to those RTEs will
+ * see possibly-nulled values, and we should mark generated Vars to
+ * account for that. In particular, the join alias Vars that we're
+ * about to build should reflect the nulling effects of this join.
+ *
+ * A difficulty with doing this is that we need the join's RT index,
+ * which we don't officially have yet. However, no other RTE can get
+ * made between here and the addRangeTableEntryForJoin call, so we can
+ * predict what the assignment will be. (Alternatively, we could call
+ * addRangeTableEntryForJoin before we have all the data computed, but
+ * this seems less ugly.)
+ */
+ j->rtindex = list_length(pstate->p_rtable) + 1;
+
+ switch (j->jointype)
+ {
+ case JOIN_INNER:
+ break;
+ case JOIN_LEFT:
+ markRelsAsNulledBy(pstate, j->rarg, j->rtindex);
+ break;
+ case JOIN_FULL:
+ markRelsAsNulledBy(pstate, j->larg, j->rtindex);
+ markRelsAsNulledBy(pstate, j->rarg, j->rtindex);
+ break;
+ case JOIN_RIGHT:
+ markRelsAsNulledBy(pstate, j->larg, j->rtindex);
+ break;
+ default:
+ /* shouldn't see any other types here */
+ elog(ERROR, "unrecognized join type: %d",
+ (int) j->jointype);
+ break;
+ }
+
+ /*
+ * Now we can construct join alias expressions for the USING columns.
+ */
+ if (j->usingClause)
+ {
+ ListCell *lc1,
+ *lc2;
+
+ /* Scan the colnos lists to recover info from the previous loop */
+ forboth(lc1, l_colnos, lc2, r_colnos)
+ {
+ int l_index = lfirst_int(lc1) - 1;
+ int r_index = lfirst_int(lc2) - 1;
+ Var *l_colvar,
+ *r_colvar;
+ Node *u_colvar;
+ ParseNamespaceColumn *res_nscolumn;
+
+ /*
+ * Note we re-build these Vars: they might have different
+ * varnullingrels than the ones made in the previous loop.
+ */
+ l_colvar = buildVarFromNSColumn(pstate, l_nscolumns + l_index);
+ r_colvar = buildVarFromNSColumn(pstate, r_nscolumns + r_index);
+
+ /* Construct the join alias Var for this column */
u_colvar = buildMergedJoinVar(pstate,
j->jointype,
l_colvar,
r_colvar);
res_colvars = lappend(res_colvars, u_colvar);
+
+ /* Construct column's res_nscolumns[] entry */
res_nscolumn = res_nscolumns + res_colindex;
res_colindex++;
if (u_colvar == (Node *) l_colvar)
@@ -1400,47 +1494,45 @@ transformFromClauseItem(ParseState *pstate, Node *n,
/*
* Merged column is not semantically equivalent to either
* input, so it needs to be referenced as the join output
- * column. We don't know the join's varno yet, so we'll
- * replace these zeroes below.
+ * column.
*/
- res_nscolumn->p_varno = 0;
+ res_nscolumn->p_varno = j->rtindex;
res_nscolumn->p_varattno = res_colindex;
res_nscolumn->p_vartype = exprType(u_colvar);
res_nscolumn->p_vartypmod = exprTypmod(u_colvar);
res_nscolumn->p_varcollid = exprCollation(u_colvar);
- res_nscolumn->p_varnosyn = 0;
+ res_nscolumn->p_varnosyn = j->rtindex;
res_nscolumn->p_varattnosyn = res_colindex;
}
}
-
- j->quals = transformJoinUsingClause(pstate,
- l_usingvars,
- r_usingvars);
- }
- else if (j->quals)
- {
- /* User-written ON-condition; transform it */
- j->quals = transformJoinOnClause(pstate, j, my_namespace);
- }
- else
- {
- /* CROSS JOIN: no quals */
}
/* Add remaining columns from each side to the output columns */
res_colindex +=
- extractRemainingColumns(l_nscolumns, l_colnames, &l_colnos,
+ extractRemainingColumns(pstate,
+ l_nscolumns, l_colnames, &l_colnos,
&res_colnames, &res_colvars,
res_nscolumns + res_colindex);
res_colindex +=
- extractRemainingColumns(r_nscolumns, r_colnames, &r_colnos,
+ extractRemainingColumns(pstate,
+ r_nscolumns, r_colnames, &r_colnos,
&res_colnames, &res_colvars,
res_nscolumns + res_colindex);
+ /* If join has an alias, it syntactically hides all inputs */
+ if (j->alias)
+ {
+ for (k = 0; k < res_colindex; k++)
+ {
+ ParseNamespaceColumn *nscol = res_nscolumns + k;
+
+ nscol->p_varnosyn = j->rtindex;
+ nscol->p_varattnosyn = k + 1;
+ }
+ }
+
/*
* Now build an RTE and nsitem for the result of the join.
- * res_nscolumns isn't totally done yet, but that's OK because
- * addRangeTableEntryForJoin doesn't examine it, only store a pointer.
*/
nsitem = addRangeTableEntryForJoin(pstate,
res_colnames,
@@ -1454,31 +1546,16 @@ transformFromClauseItem(ParseState *pstate, Node *n,
j->alias,
true);
- j->rtindex = nsitem->p_rtindex;
+ /* Verify that we correctly predicted the join's RT index */
+ Assert(j->rtindex == nsitem->p_rtindex);
+ /* Cross-check number of columns, too */
+ Assert(res_colindex == list_length(nsitem->p_names->colnames));
/*
- * Now that we know the join RTE's rangetable index, we can fix up the
- * res_nscolumns data in places where it should contain that.
+ * Save a link to the JoinExpr in the proper element of p_joinexprs.
+ * Since we maintain that list lazily, it may be necessary to fill in
+ * empty entries before we can add the JoinExpr in the right place.
*/
- Assert(res_colindex == list_length(nsitem->p_names->colnames));
- for (k = 0; k < res_colindex; k++)
- {
- ParseNamespaceColumn *nscol = res_nscolumns + k;
-
- /* fill in join RTI for merged columns */
- if (nscol->p_varno == 0)
- nscol->p_varno = j->rtindex;
- if (nscol->p_varnosyn == 0)
- nscol->p_varnosyn = j->rtindex;
- /* if join has an alias, it syntactically hides all inputs */
- if (j->alias)
- {
- nscol->p_varnosyn = j->rtindex;
- nscol->p_varattnosyn = k + 1;
- }
- }
-
- /* make a matching link to the JoinExpr for later use */
for (k = list_length(pstate->p_joinexprs) + 1; k < j->rtindex; k++)
pstate->p_joinexprs = lappend(pstate->p_joinexprs, NULL);
pstate->p_joinexprs = lappend(pstate->p_joinexprs, j);
@@ -1547,10 +1624,13 @@ transformFromClauseItem(ParseState *pstate, Node *n,
* buildVarFromNSColumn -
* build a Var node using ParseNamespaceColumn data
*
- * We assume varlevelsup should be 0, and no location is specified
+ * This is used to construct joinaliasvars entries.
+ * We can assume varlevelsup should be 0, and no location is specified.
+ * Note also that no column SELECT privilege is requested here; that would
+ * happen only if the column is actually referenced in the query.
*/
static Var *
-buildVarFromNSColumn(ParseNamespaceColumn *nscol)
+buildVarFromNSColumn(ParseState *pstate, ParseNamespaceColumn *nscol)
{
Var *var;
@@ -1564,6 +1644,10 @@ buildVarFromNSColumn(ParseNamespaceColumn *nscol)
/* makeVar doesn't offer parameters for these, so set by hand: */
var->varnosyn = nscol->p_varnosyn;
var->varattnosyn = nscol->p_varattnosyn;
+
+ /* ... and update varnullingrels */
+ markNullableIfNeeded(pstate, var);
+
return var;
}
@@ -1675,6 +1759,47 @@ buildMergedJoinVar(ParseState *pstate, JoinType jointype,
return res_node;
}
+/*
+ * markRelsAsNulledBy -
+ * Mark the given jointree node and its children as nulled by join jindex
+ */
+static void
+markRelsAsNulledBy(ParseState *pstate, Node *n, int jindex)
+{
+ int varno;
+ ListCell *lc;
+
+ /* Note: we can't see FromExpr here */
+ if (IsA(n, RangeTblRef))
+ {
+ varno = ((RangeTblRef *) n)->rtindex;
+ }
+ else if (IsA(n, JoinExpr))
+ {
+ JoinExpr *j = (JoinExpr *) n;
+
+ /* recurse to children */
+ markRelsAsNulledBy(pstate, j->larg, jindex);
+ markRelsAsNulledBy(pstate, j->rarg, jindex);
+ varno = j->rtindex;
+ }
+ else
+ {
+ elog(ERROR, "unrecognized node type: %d", (int) nodeTag(n));
+ varno = 0; /* keep compiler quiet */
+ }
+
+ /*
+ * Now add jindex to the p_nullingrels set for relation varno. Since we
+ * maintain the p_nullingrels list lazily, we might need to extend it to
+ * make the varno'th entry exist.
+ */
+ while (list_length(pstate->p_nullingrels) < varno)
+ pstate->p_nullingrels = lappend(pstate->p_nullingrels, NULL);
+ lc = list_nth_cell(pstate->p_nullingrels, varno - 1);
+ lfirst(lc) = bms_add_member((Bitmapset *) lfirst(lc), jindex);
+}
+
/*
* setNamespaceColumnVisibility -
* Convenience subroutine to update cols_visible flags in a namespace list.
diff --git a/src/backend/parser/parse_coerce.c b/src/backend/parser/parse_coerce.c
index 60908111c8..606491bd66 100644
--- a/src/backend/parser/parse_coerce.c
+++ b/src/backend/parser/parse_coerce.c
@@ -1042,7 +1042,7 @@ coerce_record_to_complex(ParseState *pstate, Node *node,
ParseNamespaceItem *nsitem;
nsitem = GetNSItemByRangeTablePosn(pstate, rtindex, sublevels_up);
- args = expandNSItemVars(nsitem, sublevels_up, vlocation, NULL);
+ args = expandNSItemVars(pstate, nsitem, sublevels_up, vlocation, NULL);
}
else
ereport(ERROR,
diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c
index e5fc708c8a..3fce9c5b62 100644
--- a/src/backend/parser/parse_expr.c
+++ b/src/backend/parser/parse_expr.c
@@ -2538,6 +2538,9 @@ transformWholeRowRef(ParseState *pstate, ParseNamespaceItem *nsitem,
/* location is not filled in by makeWholeRowVar */
result->location = location;
+ /* mark Var if it's nulled by any outer joins */
+ markNullableIfNeeded(pstate, result);
+
/* mark relation as requiring whole-row SELECT access */
markVarForSelectPriv(pstate, result);
@@ -2565,6 +2568,8 @@ transformWholeRowRef(ParseState *pstate, ParseNamespaceItem *nsitem,
rowexpr->colnames = copyObject(nsitem->p_names->colnames);
rowexpr->location = location;
+ /* XXX we ought to mark the row as possibly nullable */
+
return (Node *) rowexpr;
}
}
diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c
index 81f9ae2f02..fd1631fe75 100644
--- a/src/backend/parser/parse_relation.c
+++ b/src/backend/parser/parse_relation.c
@@ -751,6 +751,9 @@ scanNSItemForColumn(ParseState *pstate, ParseNamespaceItem *nsitem,
}
var->location = location;
+ /* Mark Var if it's nulled by any outer joins */
+ markNullableIfNeeded(pstate, var);
+
/* Require read access to the column */
markVarForSelectPriv(pstate, var);
@@ -1007,6 +1010,35 @@ searchRangeTableForCol(ParseState *pstate, const char *alias, const char *colnam
return fuzzystate;
}
+/*
+ * markNullableIfNeeded
+ * If the RTE referenced by the Var is nullable by outer join(s)
+ * at this point in the query, set var->varnullingrels to show that.
+ */
+void
+markNullableIfNeeded(ParseState *pstate, Var *var)
+{
+ int rtindex = var->varno;
+ Bitmapset *relids;
+
+ /* Find the appropriate pstate */
+ for (int lv = 0; lv < var->varlevelsup; lv++)
+ pstate = pstate->parentParseState;
+
+ /* Find currently-relevant join relids for the Var's rel */
+ if (rtindex > 0 && rtindex <= list_length(pstate->p_nullingrels))
+ relids = (Bitmapset *) list_nth(pstate->p_nullingrels, rtindex - 1);
+ else
+ relids = NULL;
+
+ /*
+ * Merge with any already-declared nulling rels. (Typically there won't
+ * be any, but let's get it right if there are.)
+ */
+ if (relids != NULL)
+ var->varnullingrels = bms_union(var->varnullingrels, relids);
+}
+
/*
* markRTEForSelectPriv
* Mark the specified column of the RTE with index rtindex
@@ -3109,7 +3141,7 @@ expandTupleDesc(TupleDesc tupdesc, Alias *eref, int count, int offset,
* the list elements mustn't be modified.
*/
List *
-expandNSItemVars(ParseNamespaceItem *nsitem,
+expandNSItemVars(ParseState *pstate, ParseNamespaceItem *nsitem,
int sublevels_up, int location,
List **colnames)
{
@@ -3145,6 +3177,10 @@ expandNSItemVars(ParseNamespaceItem *nsitem,
var->varnosyn = nscol->p_varnosyn;
var->varattnosyn = nscol->p_varattnosyn;
var->location = location;
+
+ /* ... and update varnullingrels */
+ markNullableIfNeeded(pstate, var);
+
result = lappend(result, var);
if (colnames)
*colnames = lappend(*colnames, colnameval);
@@ -3179,7 +3215,7 @@ expandNSItemAttrs(ParseState *pstate, ParseNamespaceItem *nsitem,
*var;
List *te_list = NIL;
- vars = expandNSItemVars(nsitem, sublevels_up, location, &names);
+ vars = expandNSItemVars(pstate, nsitem, sublevels_up, location, &names);
/*
* Require read access to the table. This is normally redundant with the
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index bd8057bc3e..4f5dd2e99f 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -1370,7 +1370,7 @@ ExpandSingleTable(ParseState *pstate, ParseNamespaceItem *nsitem,
List *vars;
ListCell *l;
- vars = expandNSItemVars(nsitem, sublevels_up, location, NULL);
+ vars = expandNSItemVars(pstate, nsitem, sublevels_up, location, NULL);
/*
* Require read access to the table. This is normally redundant with
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 7caff62af7..63725c8322 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1080,6 +1080,14 @@ typedef struct RangeTblEntry
* alias Vars are generated only for merged columns). We keep these
* entries only because they're needed in expandRTE() and similar code.
*
+ * Vars appearing within joinaliasvars are marked with varnullingrels sets
+ * that describe the nulling effects of this join and lower ones. This is
+ * essential for FULL JOIN cases, because the COALESCE expression only
+ * describes the semantics correctly if its inputs have been nulled by the
+ * join. For other cases, it allows expandRTE() to generate a valid
+ * representation of the join's output without consulting additional
+ * parser state.
+ *
* Within a Query loaded from a stored rule, it is possible for non-merged
* joinaliasvars items to be null pointers, which are placeholders for
* (necessarily unreferenced) columns dropped since the rule was made.
diff --git a/src/include/parser/parse_node.h b/src/include/parser/parse_node.h
index 962ebf65de..636d3231cd 100644
--- a/src/include/parser/parse_node.h
+++ b/src/include/parser/parse_node.h
@@ -115,6 +115,13 @@ typedef Node *(*CoerceParamHook) (ParseState *pstate, Param *param,
* This is one-for-one with p_rtable, but contains NULLs for non-join
* RTEs, and may be shorter than p_rtable if the last RTE(s) aren't joins.
*
+ * p_nullingrels: list of Bitmapsets associated with p_rtable entries, each
+ * containing the set of outer-join RTE indexes that can null that relation
+ * at the current point in the parse tree. This is one-for-one with p_rtable,
+ * but may be shorter than p_rtable, in which case the missing entries are
+ * implicitly empty (NULL). That rule allows us to save work when the query
+ * contains no outer joins.
+ *
* p_joinlist: list of join items (RangeTblRef and JoinExpr nodes) that
* will become the fromlist of the query's top-level FromExpr node.
*
@@ -182,6 +189,7 @@ struct ParseState
const char *p_sourcetext; /* source text, or NULL if not available */
List *p_rtable; /* range table so far */
List *p_joinexprs; /* JoinExprs for RTE_JOIN p_rtable entries */
+ List *p_nullingrels; /* Bitmapsets showing nulling outer joins */
List *p_joinlist; /* join items so far (will become FromExpr
* node's fromlist) */
List *p_namespace; /* currently-referenceable RTEs (List of
diff --git a/src/include/parser/parse_relation.h b/src/include/parser/parse_relation.h
index 484db165db..e7e72d6f3e 100644
--- a/src/include/parser/parse_relation.h
+++ b/src/include/parser/parse_relation.h
@@ -41,6 +41,7 @@ extern Node *scanNSItemForColumn(ParseState *pstate, ParseNamespaceItem *nsitem,
int location);
extern Node *colNameToVar(ParseState *pstate, const char *colname, bool localonly,
int location);
+extern void markNullableIfNeeded(ParseState *pstate, Var *var);
extern void markVarForSelectPriv(ParseState *pstate, Var *var);
extern Relation parserOpenTable(ParseState *pstate, const RangeVar *relation,
int lockmode);
@@ -109,7 +110,7 @@ extern void errorMissingColumn(ParseState *pstate,
extern void expandRTE(RangeTblEntry *rte, int rtindex, int sublevels_up,
int location, bool include_dropped,
List **colnames, List **colvars);
-extern List *expandNSItemVars(ParseNamespaceItem *nsitem,
+extern List *expandNSItemVars(ParseState *pstate, ParseNamespaceItem *nsitem,
int sublevels_up, int location,
List **colnames);
extern List *expandNSItemAttrs(ParseState *pstate, ParseNamespaceItem *nsitem,
commit 299ca73aee68b567418c344a4a574fd69dae4ba5
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sat Nov 5 16:32:19 2022 -0400
Teach the planner to cope with Vars bearing nullingrels.
The core idea of this step is to include varnullingrels in the
relid sets that qual clauses are considered to depend on.
So that we can still easily compare quals' relids to RelOptInfos'
relids, that means also adding outer join relids to the identifying
relids of join relations. Much of the bulk of this step is concerned
with fallout from the latter change.
I've resolved the previous squishiness entailed by outer join identity 3
by generating multiple versions of outer-join quals that could get moved
to a join level where they need to contain different nullingrels sets.
Now we have versions of such quals with the correct nullingrels for
each level where they could appear.
This requires a bit of new mechanism (RestrictInfo.has_clone/is_clone)
to prevent multiple versions of the same qual from getting used in the
plan. My worry about how that could work with EquivalenceClasses is
resolved by creating EquivalenceClasses only from the least-marked
version of a qual. (This doesn't really lose anything, since versions
with more nullingrels bits don't correspond to any equalities available
outside the nest of commuting outer joins.)
These extra versions of quals would also result in generating multiple
parameterized paths that differ only in what nullingrels they expect
for the Vars from the parameterization rel(s). That seems like it'd
be very wasteful, so I've arranged to generate such paths only from
the least-marked version of a qual (the has_clone version).
Unlike in the previous version of this patch, setrefs.c is able to
cross-check the nullingrel sets of most Vars and PlaceHolderVars to
ensure that they match up with what the previous plan step produces.
But there are three cases that I've so far punted on:
1. The targetlist and qpqual of an outer join node will contain
nullingrels bits for the outer join itself. To check exact matching to
the input, we'd need to know the OJ's relid as well as which input(s)
got nulled, neither of which is cheaply available in setrefs.c. For
now, it's just checking that such Vars have a superset of the input's
nullingrels bits.
2. Parameterized paths will generally refer to the least-marked version
of whichever outer-side Vars they use, which may not be what's actually
available from the outside of the nestloop. (We're relying on the join
ordering rules for that to be sensible.) Again, setrefs.c is in no
position to pass judgment on correctness, so it's just checking that
the parameter expression has a subset of the outer-side marking.
3. Row identity variables are not marked with any nullingrels, which
may not correspond to reality. I've punted on this by skipping the
checks when varattno <= 0.
Point 1 could be addressed if we were willing to add informational
fields to join plan nodes, which might be worth doing, but I'm not sure.
The other two points seem like the extra mechanisms needed for a
bulletproof check would be considerably more trouble than they'd be
worth.
There is still some confusion about which versions of a cloned qual
are actually necessary to check, which results in some extra filter
conditions showing up in a couple of regression test plans. There are
also some failure cases involving antijoins and full joins that remain
to be fixed. This patch is already mighty big, so I'll address those
failures separately.
This step removes some low-hanging fruit from the old implementation,
such as the need to track lowest_nulling_outer_join during subquery
pullup. There's much more to do in that line, though.
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index 004481d608..1c921879a9 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -273,7 +273,7 @@ merge_clump(PlannerInfo *root, List *clumps, Clump *new_clump, int num_gene,
* rel once we know the final targetlist (see
* grouping_planner).
*/
- if (!bms_equal(joinrel->relids, root->all_baserels))
+ if (!bms_equal(joinrel->relids, root->all_query_rels))
generate_useful_gather_paths(root, joinrel, false);
/* Find and save the cheapest paths for this joinrel */
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 4ddaed31a4..5902c80747 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -159,27 +159,6 @@ make_one_rel(PlannerInfo *root, List *joinlist)
Index rti;
double total_pages;
- /*
- * Construct the all_baserels Relids set.
- */
- root->all_baserels = NULL;
- for (rti = 1; rti < root->simple_rel_array_size; rti++)
- {
- RelOptInfo *brel = root->simple_rel_array[rti];
-
- /* there may be empty slots corresponding to non-baserel RTEs */
- if (brel == NULL)
- continue;
-
- Assert(brel->relid == rti); /* sanity check on array */
-
- /* ignore RTEs that are "other rels" */
- if (brel->reloptkind != RELOPT_BASEREL)
- continue;
-
- root->all_baserels = bms_add_member(root->all_baserels, brel->relid);
- }
-
/* Mark base rels as to whether we care about fast-start plans */
set_base_rel_consider_startup(root);
@@ -207,6 +186,7 @@ make_one_rel(PlannerInfo *root, List *joinlist)
{
RelOptInfo *brel = root->simple_rel_array[rti];
+ /* there may be empty slots corresponding to non-baserel RTEs */
if (brel == NULL)
continue;
@@ -231,9 +211,9 @@ make_one_rel(PlannerInfo *root, List *joinlist)
rel = make_rel_from_joinlist(root, joinlist);
/*
- * The result should join all and only the query's base rels.
+ * The result should join all and only the query's base + outer-join rels.
*/
- Assert(bms_equal(rel->relids, root->all_baserels));
+ Assert(bms_equal(rel->relids, root->all_query_rels));
return rel;
}
@@ -558,7 +538,7 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* the final scan/join targetlist is available (see grouping_planner).
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- !bms_equal(rel->relids, root->all_baserels))
+ !bms_equal(rel->relids, root->all_query_rels))
generate_useful_gather_paths(root, rel, false);
/* Now find the cheapest of the paths for this rel */
@@ -879,7 +859,7 @@ set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *
* to support an uncommon usage of second-rate sampling methods. Instead,
* if there is a risk that the query might perform an unsafe join, just
* wrap the SampleScan in a Materialize node. We can check for joins by
- * counting the membership of all_baserels (note that this correctly
+ * counting the membership of all_query_rels (note that this correctly
* counts inheritance trees as single rels). If we're inside a subquery,
* we can't easily check whether a join might occur in the outer query, so
* just assume one is possible.
@@ -888,7 +868,7 @@ set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *
* so check repeatable_across_scans last, even though that's a bit odd.
*/
if ((root->query_level > 1 ||
- bms_membership(root->all_baserels) != BMS_SINGLETON) &&
+ bms_membership(root->all_query_rels) != BMS_SINGLETON) &&
!(GetTsmRoutine(rte->tablesample->tsmhandler)->repeatable_across_scans))
{
path = (Path *) create_material_path(rel, path);
@@ -970,7 +950,7 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
if (enable_partitionwise_join &&
rel->reloptkind == RELOPT_BASEREL &&
rte->relkind == RELKIND_PARTITIONED_TABLE &&
- rel->attr_needed[InvalidAttrNumber - rel->min_attr] == NULL)
+ bms_is_empty(rel->attr_needed[InvalidAttrNumber - rel->min_attr]))
rel->consider_partitionwise_join = true;
/*
@@ -3435,7 +3415,7 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
* partial paths. We'll do the same for the topmost scan/join rel
* once we know the final targetlist (see grouping_planner).
*/
- if (!bms_equal(rel->relids, root->all_baserels))
+ if (!bms_equal(rel->relids, root->all_query_rels))
generate_useful_gather_paths(root, rel, false);
/* Find and save the cheapest paths for this rel */
diff --git a/src/backend/optimizer/path/clausesel.c b/src/backend/optimizer/path/clausesel.c
index 06f836308d..c08eb2b1c5 100644
--- a/src/backend/optimizer/path/clausesel.c
+++ b/src/backend/optimizer/path/clausesel.c
@@ -218,7 +218,7 @@ clauselist_selectivity_ext(PlannerInfo *root,
if (rinfo)
{
- ok = (bms_membership(rinfo->clause_relids) == BMS_SINGLETON) &&
+ ok = (rinfo->num_base_rels == 1) &&
(is_pseudo_constant_clause_relids(lsecond(expr->args),
rinfo->right_relids) ||
(varonleft = false,
@@ -579,30 +579,6 @@ find_single_rel_for_clauses(PlannerInfo *root, List *clauses)
return NULL; /* no clauses */
}
-/*
- * bms_is_subset_singleton
- *
- * Same result as bms_is_subset(s, bms_make_singleton(x)),
- * but a little faster and doesn't leak memory.
- *
- * Is this of use anywhere else? If so move to bitmapset.c ...
- */
-static bool
-bms_is_subset_singleton(const Bitmapset *s, int x)
-{
- switch (bms_membership(s))
- {
- case BMS_EMPTY_SET:
- return true;
- case BMS_SINGLETON:
- return bms_is_member(x, s);
- case BMS_MULTIPLE:
- return false;
- }
- /* can't get here... */
- return false;
-}
-
/*
* treat_as_join_clause -
* Decide whether an operator clause is to be handled by the
@@ -631,17 +607,20 @@ treat_as_join_clause(PlannerInfo *root, Node *clause, RestrictInfo *rinfo,
else
{
/*
- * Otherwise, it's a join if there's more than one relation used. We
- * can optimize this calculation if an rinfo was passed.
+ * Otherwise, it's a join if there's more than one base relation used.
+ * We can optimize this calculation if an rinfo was passed.
*
* XXX Since we know the clause is being evaluated at a join, the
* only way it could be single-relation is if it was delayed by outer
- * joins. Although we can make use of the restriction qual estimators
- * anyway, it seems likely that we ought to account for the
- * probability of injected nulls somehow.
+ * joins. We intentionally count only baserels here, not OJs that
+ * might be present in rinfo->clause_relids, so that we direct such
+ * cases to the restriction qual estimators not join estimators.
+ * Eventually some notice should be taken of the possibility of
+ * injected nulls, but we'll likely want to do that in the restriction
+ * estimators rather than starting to treat such cases as join quals.
*/
if (rinfo)
- return (bms_membership(rinfo->clause_relids) == BMS_MULTIPLE);
+ return (rinfo->num_base_rels > 1);
else
return (NumRelids(root, clause) > 1);
}
@@ -754,7 +733,9 @@ clause_selectivity_ext(PlannerInfo *root,
* for all non-JOIN_INNER cases.
*/
if (varRelid == 0 ||
- bms_is_subset_singleton(rinfo->clause_relids, varRelid))
+ rinfo->num_base_rels == 0 ||
+ (rinfo->num_base_rels == 1 &&
+ bms_is_member(varRelid, rinfo->clause_relids)))
{
/* Cacheable --- do we already have the result? */
if (jointype == JOIN_INNER)
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 4c6b1d1f55..30ac8ae721 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4783,6 +4783,11 @@ compute_semi_anti_join_factors(PlannerInfo *root,
norm_sjinfo.syn_lefthand = outerrel->relids;
norm_sjinfo.syn_righthand = innerrel->relids;
norm_sjinfo.jointype = JOIN_INNER;
+ norm_sjinfo.ojrelid = 0;
+ norm_sjinfo.commute_above_l = NULL;
+ norm_sjinfo.commute_above_r = NULL;
+ norm_sjinfo.commute_below = NULL;
+ norm_sjinfo.oj_joinclause = NIL;
/* we don't bother trying to make the remaining fields valid */
norm_sjinfo.lhs_strict = false;
norm_sjinfo.delay_upper_joins = false;
@@ -4948,6 +4953,11 @@ approx_tuple_count(PlannerInfo *root, JoinPath *path, List *quals)
sjinfo.syn_lefthand = path->outerjoinpath->parent->relids;
sjinfo.syn_righthand = path->innerjoinpath->parent->relids;
sjinfo.jointype = JOIN_INNER;
+ sjinfo.ojrelid = 0;
+ sjinfo.commute_above_l = NULL;
+ sjinfo.commute_above_r = NULL;
+ sjinfo.commute_below = NULL;
+ sjinfo.oj_joinclause = NIL;
/* we don't bother trying to make the remaining fields valid */
sjinfo.lhs_strict = false;
sjinfo.delay_upper_joins = false;
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index e65b967b1f..349e183372 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -29,6 +29,7 @@
#include "optimizer/paths.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "rewrite/rewriteManip.h"
#include "utils/lsyscache.h"
@@ -64,7 +65,7 @@ static bool reconsider_outer_join_clause(PlannerInfo *root,
RestrictInfo *rinfo,
bool outer_on_left);
static bool reconsider_full_join_clause(PlannerInfo *root,
- RestrictInfo *rinfo);
+ FullJoinClauseInfo *fjinfo);
static Bitmapset *get_eclass_indexes_for_relids(PlannerInfo *root,
Relids relids);
static Bitmapset *get_common_eclass_indexes(PlannerInfo *root, Relids relids1,
@@ -757,6 +758,12 @@ get_eclass_for_sort_expr(PlannerInfo *root,
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL) /* must be an outer join */
+ {
+ Assert(bms_is_member(i, root->outer_join_rels));
+ continue;
+ }
+
Assert(rel->reloptkind == RELOPT_BASEREL ||
rel->reloptkind == RELOPT_DEADREL);
@@ -1113,6 +1120,12 @@ generate_base_implied_equalities(PlannerInfo *root)
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL) /* must be an outer join */
+ {
+ Assert(bms_is_member(i, root->outer_join_rels));
+ continue;
+ }
+
Assert(rel->reloptkind == RELOPT_BASEREL);
rel->eclass_indexes = bms_add_member(rel->eclass_indexes,
@@ -2015,10 +2028,12 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* Process the FULL JOIN clauses */
foreach(cell, root->full_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ FullJoinClauseInfo *fjinfo = (FullJoinClauseInfo *) lfirst(cell);
- if (reconsider_full_join_clause(root, rinfo))
+ if (reconsider_full_join_clause(root, fjinfo))
{
+ RestrictInfo *rinfo = fjinfo->rinfo;
+
found = true;
/* remove it from the list */
root->full_join_clauses =
@@ -2047,9 +2062,9 @@ reconsider_outer_join_clauses(PlannerInfo *root)
}
foreach(cell, root->full_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ FullJoinClauseInfo *fjinfo = (FullJoinClauseInfo *) lfirst(cell);
- distribute_restrictinfo_to_rels(root, rinfo);
+ distribute_restrictinfo_to_rels(root, fjinfo->rinfo);
}
}
@@ -2185,8 +2200,11 @@ reconsider_outer_join_clause(PlannerInfo *root, RestrictInfo *rinfo,
* Returns true if we were able to propagate a constant through the clause.
*/
static bool
-reconsider_full_join_clause(PlannerInfo *root, RestrictInfo *rinfo)
+reconsider_full_join_clause(PlannerInfo *root, FullJoinClauseInfo *fjinfo)
{
+ RestrictInfo *rinfo = fjinfo->rinfo;
+ SpecialJoinInfo *sjinfo = fjinfo->sjinfo;
+ Relids fjrelids = bms_make_singleton(sjinfo->ojrelid);
Expr *leftvar;
Expr *rightvar;
Oid opno,
@@ -2268,6 +2286,18 @@ reconsider_full_join_clause(PlannerInfo *root, RestrictInfo *rinfo)
cfirst = (Node *) linitial(cexpr->args);
csecond = (Node *) lsecond(cexpr->args);
+ /*
+ * The COALESCE arguments will be marked as possibly nulled by
+ * the full join, while we wish to generate clauses that apply
+ * to the join's inputs. So we must strip the join from the
+ * nullingrels fields of cfirst/csecond before comparing them
+ * to leftvar/rightvar. (Perhaps with a less hokey
+ * representation for FULL JOIN USING output columns, this
+ * wouldn't be needed?)
+ */
+ cfirst = remove_nulling_relids(cfirst, fjrelids, NULL);
+ csecond = remove_nulling_relids(csecond, fjrelids, NULL);
+
if (equal(leftvar, cfirst) && equal(rightvar, csecond))
{
coal_idx = foreach_current_index(lc2);
@@ -3204,6 +3234,12 @@ get_eclass_indexes_for_relids(PlannerInfo *root, Relids relids)
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL) /* must be an outer join */
+ {
+ Assert(bms_is_member(i, root->outer_join_rels));
+ continue;
+ }
+
ec_indexes = bms_add_members(ec_indexes, rel->eclass_indexes);
}
return ec_indexes;
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index 77f3f81bcb..7fa405e5b9 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -3372,13 +3372,13 @@ check_index_predicates(PlannerInfo *root, RelOptInfo *rel)
* Add on any equivalence-derivable join clauses. Computing the correct
* relid sets for generate_join_implied_equalities is slightly tricky
* because the rel could be a child rel rather than a true baserel, and in
- * that case we must remove its parents' relid(s) from all_baserels.
+ * that case we must subtract its parents' relid(s) from all_query_rels.
*/
if (rel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- otherrels = bms_difference(root->all_baserels,
+ otherrels = bms_difference(root->all_query_rels,
find_childrel_parents(root, rel));
else
- otherrels = bms_difference(root->all_baserels, rel->relids);
+ otherrels = bms_difference(root->all_query_rels, rel->relids);
if (!bms_is_empty(otherrels))
clauselist =
@@ -3756,7 +3756,8 @@ match_index_to_operand(Node *operand,
*/
if (operand && IsA(operand, Var) &&
index->rel->relid == ((Var *) operand)->varno &&
- indkey == ((Var *) operand)->varattno)
+ indkey == ((Var *) operand)->varattno &&
+ ((Var *) operand)->varnullingrels == NULL)
return true;
}
else
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 2a3f0ab7bf..cd3f9fa0af 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -234,7 +234,9 @@ add_paths_to_joinrel(PlannerInfo *root,
* reduces the number of parameterized paths we have to deal with at
* higher join levels, without compromising the quality of the resulting
* plan. We express the restriction as a Relids set that must overlap the
- * parameterization of any proposed join path.
+ * parameterization of any proposed join path. Note: param_source_rels
+ * should contain only baserels, not OJ relids, so starting from
+ * all_baserels not all_query_rels is correct.
*/
foreach(lc, root->join_info_list)
{
@@ -365,6 +367,47 @@ allow_star_schema_join(PlannerInfo *root,
bms_nonempty_difference(inner_paramrels, outerrelids));
}
+/*
+ * If the parameterization is only partly satisfied by the outer rel,
+ * the unsatisfied part can't include any outer-join relids that could
+ * null rels of the satisfied part. That would imply that we're trying
+ * to use a clause involving a Var with nonempty varnullingrels at
+ * a join level where that value isn't yet computable.
+ */
+static inline bool
+have_unsafe_outer_join_ref(PlannerInfo *root,
+ Relids outerrelids,
+ Relids inner_paramrels)
+{
+ bool result = false;
+ Relids unsatisfied = bms_difference(inner_paramrels, outerrelids);
+
+ if (bms_overlap(unsatisfied, root->outer_join_rels))
+ {
+ ListCell *lc;
+
+ foreach(lc, root->join_info_list)
+ {
+ SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+
+ if (!bms_is_member(sjinfo->ojrelid, unsatisfied))
+ continue; /* not relevant */
+ if (bms_overlap(inner_paramrels, sjinfo->min_righthand) ||
+ (sjinfo->jointype == JOIN_FULL &&
+ bms_overlap(inner_paramrels, sjinfo->min_lefthand)))
+ {
+ result = true; /* doesn't work */
+ break;
+ }
+ }
+ }
+
+ /* Waste no memory when we reject a path here */
+ bms_free(unsatisfied);
+
+ return result;
+}
+
/*
* paraminfo_get_equal_hashops
* Determine if param_info and innerrel's lateral_vars can be hashed.
@@ -656,15 +699,16 @@ try_nestloop_path(PlannerInfo *root,
/*
* Check to see if proposed path is still parameterized, and reject if the
* parameterization wouldn't be sensible --- unless allow_star_schema_join
- * says to allow it anyway. Also, we must reject if have_dangerous_phv
- * doesn't like the look of it, which could only happen if the nestloop is
- * still parameterized.
+ * says to allow it anyway. Also, we must reject if either
+ * have_unsafe_outer_join_ref or have_dangerous_phv don't like the look of
+ * it, which could only happen if the nestloop is still parameterized.
*/
required_outer = calc_nestloop_required_outer(outerrelids, outer_paramrels,
innerrelids, inner_paramrels);
if (required_outer &&
((!bms_overlap(required_outer, extra->param_source_rels) &&
!allow_star_schema_join(root, outerrelids, inner_paramrels)) ||
+ have_unsafe_outer_join_ref(root, outerrelids, inner_paramrels) ||
have_dangerous_phv(root, outerrelids, inner_paramrels)))
{
/* Waste no memory when we reject a path here */
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 9da3ff2f9a..6c28b0a057 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -353,7 +353,10 @@ make_rels_by_clauseless_joins(PlannerInfo *root,
*
* Caller must supply not only the two rels, but the union of their relids.
* (We could simplify the API by computing joinrelids locally, but this
- * would be redundant work in the normal path through make_join_rel.)
+ * would be redundant work in the normal path through make_join_rel.
+ * Note that this value does NOT include the RT index of any outer join that
+ * might need to be performed here, so it's not the canonical identifier
+ * of the join relation.)
*
* On success, *sjinfo_p is set to NULL if this is to be a plain inner join,
* else it's set to point to the associated SpecialJoinInfo node. Also,
@@ -695,7 +698,7 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(rel1->relids, rel2->relids));
- /* Construct Relids set that identifies the joinrel. */
+ /* Construct Relids set that identifies the joinrel (without OJ as yet). */
joinrelids = bms_union(rel1->relids, rel2->relids);
/* Check validity and determine join type. */
@@ -707,6 +710,10 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
return NULL;
}
+ /* If we have an outer join, add its RTI to form the canonical relids. */
+ if (sjinfo && sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
+
/* Swap rels if needed to match the join info. */
if (reversed)
{
@@ -730,6 +737,11 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
sjinfo->syn_lefthand = rel1->relids;
sjinfo->syn_righthand = rel2->relids;
sjinfo->jointype = JOIN_INNER;
+ sjinfo->ojrelid = 0;
+ sjinfo->commute_above_l = NULL;
+ sjinfo->commute_above_r = NULL;
+ sjinfo->commute_below = NULL;
+ sjinfo->oj_joinclause = NIL;
/* we don't bother trying to make the remaining fields valid */
sjinfo->lhs_strict = false;
sjinfo->delay_upper_joins = false;
@@ -1510,8 +1522,6 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(child_rel1->relids, child_rel2->relids));
- child_joinrelids = bms_union(child_rel1->relids, child_rel2->relids);
- appinfos = find_appinfos_by_relids(root, child_joinrelids, &nappinfos);
/*
* Construct SpecialJoinInfo from parent join relations's
@@ -1521,6 +1531,15 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
child_rel1->relids,
child_rel2->relids);
+ /* Build correct join relids for child join */
+ child_joinrelids = bms_union(child_rel1->relids, child_rel2->relids);
+ if (child_sjinfo->ojrelid != 0)
+ child_joinrelids = bms_add_member(child_joinrelids,
+ child_sjinfo->ojrelid);
+
+ /* Find the AppendRelInfo structures */
+ appinfos = find_appinfos_by_relids(root, child_joinrelids, &nappinfos);
+
/*
* Construct restrictions applicable to the child join from those
* applicable to the parent join.
@@ -1536,8 +1555,7 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
{
child_joinrel = build_child_join_rel(root, child_rel1, child_rel2,
joinrel, child_restrictlist,
- child_sjinfo,
- child_sjinfo->jointype);
+ child_sjinfo);
joinrel->part_rels[cnt_parts] = child_joinrel;
joinrel->live_parts = bms_add_member(joinrel->live_parts, cnt_parts);
joinrel->all_partrels = bms_add_members(joinrel->all_partrels,
@@ -1583,6 +1601,8 @@ build_child_join_sjinfo(PlannerInfo *root, SpecialJoinInfo *parent_sjinfo,
sjinfo->syn_righthand = adjust_child_relids(sjinfo->syn_righthand,
right_nappinfos,
right_appinfos);
+ /* outer-join relids need no adjustment */
+ Assert(sjinfo->oj_joinclause == NIL); /* should be empty now */
sjinfo->semi_rhs_exprs = (List *) adjust_appendrel_attrs(root,
(Node *) sjinfo->semi_rhs_exprs,
right_nappinfos,
diff --git a/src/backend/optimizer/path/tidpath.c b/src/backend/optimizer/path/tidpath.c
index c4e035b049..71488cec00 100644
--- a/src/backend/optimizer/path/tidpath.c
+++ b/src/backend/optimizer/path/tidpath.c
@@ -59,6 +59,7 @@ IsCTIDVar(Var *var, RelOptInfo *rel)
if (var->varattno == SelfItemPointerAttributeNumber &&
var->vartype == TIDOID &&
var->varno == rel->relid &&
+ var->varnullingrels == NULL &&
var->varlevelsup == 0)
return true;
return false;
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index bbeca9a9ab..0652b3200a 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -34,7 +34,7 @@
/* local functions */
static bool join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo);
-static void remove_rel_from_query(PlannerInfo *root, int relid,
+static void remove_rel_from_query(PlannerInfo *root, int relid, int ojrelid,
Relids joinrelids);
static List *remove_rel_from_joinlist(List *joinlist, int relid, int *nremoved);
static bool rel_supports_distinctness(PlannerInfo *root, RelOptInfo *rel);
@@ -70,6 +70,7 @@ restart:
foreach(lc, root->join_info_list)
{
SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+ Relids joinrelids;
int innerrelid;
int nremoved;
@@ -84,9 +85,12 @@ restart:
*/
innerrelid = bms_singleton_member(sjinfo->min_righthand);
- remove_rel_from_query(root, innerrelid,
- bms_union(sjinfo->min_lefthand,
- sjinfo->min_righthand));
+ /* Compute the relid set for the join we are considering */
+ joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ if (sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
+
+ remove_rel_from_query(root, innerrelid, sjinfo->ojrelid, joinrelids);
/* We verify that exactly one reference gets removed from joinlist */
nremoved = 0;
@@ -188,6 +192,8 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
/* Compute the relid set for the join we are considering */
joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ if (sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
/*
* We can't remove the join if any inner-rel attributes are used above the
@@ -247,6 +253,17 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
{
RestrictInfo *restrictinfo = (RestrictInfo *) lfirst(l);
+ /*
+ * If the current join commutes with some other outer join(s) via
+ * outer join identity 3, there will be multiple clones of its join
+ * clauses in the joininfo list. We want to consider only the
+ * has_clone form of such clauses. Processing more than one form
+ * would be wasteful, and also some of the others would confuse the
+ * RINFO_IS_PUSHED_DOWN test below.
+ */
+ if (restrictinfo->is_clone)
+ continue; /* ignore it */
+
/*
* If it's not a join clause for this outer join, we can't use it.
* Note that if the clause is pushed-down, then it is logically from
@@ -306,10 +323,12 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
* no longer treated as a baserel, and that attributes of other baserels
* are no longer marked as being needed at joins involving this rel.
* Also, join quals involving the rel have to be removed from the joininfo
- * lists, but only if they belong to the outer join identified by joinrelids.
+ * lists, but only if they belong to the outer join identified by ojrelid
+ * and joinrelids.
*/
static void
-remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
+remove_rel_from_query(PlannerInfo *root, int relid, int ojrelid,
+ Relids joinrelids)
{
RelOptInfo *rel = find_base_rel(root, relid);
List *joininfos;
@@ -349,6 +368,14 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
}
}
+ /*
+ * Update all_baserels and related relid sets.
+ */
+ root->all_baserels = bms_del_member(root->all_baserels, relid);
+ root->outer_join_rels = bms_del_member(root->outer_join_rels, ojrelid);
+ root->all_query_rels = bms_del_member(root->all_query_rels, relid);
+ root->all_query_rels = bms_del_member(root->all_query_rels, ojrelid);
+
/*
* Likewise remove references from SpecialJoinInfo data structures.
*
@@ -365,6 +392,15 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
sjinfo->min_righthand = bms_del_member(sjinfo->min_righthand, relid);
sjinfo->syn_lefthand = bms_del_member(sjinfo->syn_lefthand, relid);
sjinfo->syn_righthand = bms_del_member(sjinfo->syn_righthand, relid);
+ sjinfo->min_lefthand = bms_del_member(sjinfo->min_lefthand, ojrelid);
+ sjinfo->min_righthand = bms_del_member(sjinfo->min_righthand, ojrelid);
+ sjinfo->syn_lefthand = bms_del_member(sjinfo->syn_lefthand, ojrelid);
+ sjinfo->syn_righthand = bms_del_member(sjinfo->syn_righthand, ojrelid);
+ /* relid cannot appear in these fields, but ojrelid can: */
+ sjinfo->commute_above_l = bms_del_member(sjinfo->commute_above_l, ojrelid);
+ sjinfo->commute_above_r = bms_del_member(sjinfo->commute_above_r, ojrelid);
+ sjinfo->commute_below = bms_del_member(sjinfo->commute_below, ojrelid);
+ Assert(sjinfo->oj_joinclause == NIL); /* should be empty now */
}
/*
@@ -396,8 +432,10 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
else
{
phinfo->ph_eval_at = bms_del_member(phinfo->ph_eval_at, relid);
+ phinfo->ph_eval_at = bms_del_member(phinfo->ph_eval_at, ojrelid);
Assert(!bms_is_empty(phinfo->ph_eval_at));
phinfo->ph_needed = bms_del_member(phinfo->ph_needed, relid);
+ phinfo->ph_needed = bms_del_member(phinfo->ph_needed, ojrelid);
}
}
@@ -422,7 +460,12 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
remove_join_clause_from_rels(root, rinfo, rinfo->required_relids);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrelids))
+ /*
+ * If the qual lists ojrelid in its required_relids, it must have come
+ * from above the outer join we're removing (so we need to keep it);
+ * if it does not, then it didn't and we can discard it.
+ */
+ if (bms_is_member(ojrelid, rinfo->required_relids))
{
/* Recheck that qual doesn't actually reference the target rel */
Assert(!bms_is_member(relid, rinfo->clause_relids));
@@ -434,6 +477,8 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
rinfo->required_relids = bms_copy(rinfo->required_relids);
rinfo->required_relids = bms_del_member(rinfo->required_relids,
relid);
+ rinfo->required_relids = bms_del_member(rinfo->required_relids,
+ ojrelid);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
@@ -548,6 +593,7 @@ reduce_unique_semijoins(PlannerInfo *root)
/* Compute the relid set for the join we are considering */
joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ Assert(sjinfo->ojrelid == 0); /* SEMI joins don't have RT indexes */
/*
* Since we're only considering a single-rel RHS, any join clauses it
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index fd8cbb1dc7..41c69b29a7 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -60,16 +60,34 @@ static void process_security_barrier_quals(PlannerInfo *root,
static SpecialJoinInfo *make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
- JoinType jointype, List *clause);
+ JoinType jointype, Index ojrelid,
+ List *clause);
static void compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo,
List *clause);
+static void process_postponed_left_join_quals(PlannerInfo *root);
+static void distribute_quals_to_rels(PlannerInfo *root, List *clauses,
+ bool below_outer_join,
+ SpecialJoinInfo *sjinfo,
+ Index security_level,
+ Relids qualscope,
+ Relids ojscope,
+ Relids outerjoin_nonnullable,
+ bool allow_equivalence,
+ bool postpone_nondegenerate_clauses,
+ bool has_clone,
+ bool is_clone,
+ List **postponed_qual_list);
static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool below_outer_join,
- JoinType jointype,
+ SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
Relids ojscope,
Relids outerjoin_nonnullable,
+ bool allow_equivalence,
+ bool postpone_nondegenerate_clauses,
+ bool has_clone,
+ bool is_clone,
List **postponed_qual_list);
static bool check_outerjoin_delay(PlannerInfo *root, Relids *relids_p,
Relids *nullable_relids_p, bool is_pushed_down);
@@ -92,7 +110,7 @@ static void check_memoizable(RestrictInfo *restrictinfo);
*
* Scan the query's jointree and create baserel RelOptInfos for all
* the base relations (e.g., table, subquery, and function RTEs)
- * appearing in the jointree.
+ * appearing in the jointree. Also add their relids to all_baserels.
*
* The initial invocation must pass root->parse->jointree as the value of
* jtnode. Internally, the function recurses through the jointree.
@@ -112,6 +130,7 @@ add_base_rels_to_query(PlannerInfo *root, Node *jtnode)
int varno = ((RangeTblRef *) jtnode)->rtindex;
(void) build_simple_rel(root, varno, NULL);
+ root->all_baserels = bms_add_member(root->all_baserels, varno);
}
else if (IsA(jtnode, FromExpr))
{
@@ -230,6 +249,23 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
{
ListCell *temp;
+ /*
+ * By convention, attr_needed and ph_needed values contain only baserel
+ * relids (and of course "relation 0"), not outer-join relids. It's
+ * sufficient to keep track of this at baserel granularity, since whether
+ * an outer join has been computed at a particular join level is fully
+ * determined by the set of baserels in the join. If we included outer
+ * joins then we'd get confused by the varying sets of outer-join relids
+ * appearing in different versions of commutable outer-join clauses, and
+ * think that vars need to propagate higher than they really do. However,
+ * the presented value of where_needed will be a join relid set that may
+ * contain OJ relids, so we gotta mask here. This code assumes that
+ * "relation 0" is always presented alone, not along with other bits.
+ */
+ if (!bms_is_member(0, where_needed))
+ where_needed = bms_intersect(where_needed, root->all_baserels);
+
+ /* Should (still) have a nonempty set */
Assert(!bms_is_empty(where_needed));
foreach(temp, vars)
@@ -248,10 +284,16 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
attno -= rel->min_attr;
if (rel->attr_needed[attno] == NULL)
{
- /* Variable not yet requested, so add to rel's targetlist */
- /* XXX is copyObject necessary here? */
- rel->reltarget->exprs = lappend(rel->reltarget->exprs,
- copyObject(var));
+ /*
+ * Variable not yet requested, so add to rel's targetlist.
+ *
+ * The value available at the rel's scan level has not been
+ * nulled by any outer join, so drop its varnullingrels.
+ * (We'll put those back as we climb up the join tree.)
+ */
+ var = copyObject(var);
+ var->varnullingrels = NULL;
+ rel->reltarget->exprs = lappend(rel->reltarget->exprs, var);
/* reltarget cost and width will be computed later */
}
rel->attr_needed[attno] = bms_add_members(rel->attr_needed[attno],
@@ -547,8 +589,10 @@ create_lateral_join_info(PlannerInfo *root)
varno = -1;
while ((varno = bms_next_member(eval_at, varno)) >= 0)
{
- RelOptInfo *brel = find_base_rel(root, varno);
+ RelOptInfo *brel = find_base_rel_ignore_join(root, varno);
+ if (brel == NULL)
+ continue; /* ignore outer joins in eval_at */
brel->lateral_relids = bms_add_members(brel->lateral_relids,
phinfo->ph_lateral);
}
@@ -639,7 +683,10 @@ create_lateral_join_info(PlannerInfo *root)
{
RelOptInfo *brel2 = root->simple_rel_array[rti2];
- Assert(brel2 != NULL && brel2->reloptkind == RELOPT_BASEREL);
+ if (brel2 == NULL)
+ continue; /* must be an OJ */
+
+ Assert(brel2->reloptkind == RELOPT_BASEREL);
brel2->lateral_referencers =
bms_add_member(brel2->lateral_referencers, rti);
}
@@ -699,16 +746,27 @@ deconstruct_jointree(PlannerInfo *root)
Assert(root->parse->jointree != NULL &&
IsA(root->parse->jointree, FromExpr));
- /* this is filled as we scan the jointree */
+ /* These are filled as we scan the jointree */
+ root->outer_join_rels = NULL;
root->nullable_baserels = NULL;
result = deconstruct_recurse(root, (Node *) root->parse->jointree, false,
&qualscope, &inner_join_rels,
&postponed_qual_list);
- /* Shouldn't be any leftover quals */
+ /* Now we can form the value of all_query_rels, too */
+ root->all_query_rels = bms_union(root->all_baserels, root->outer_join_rels);
+
+ /* Shouldn't be any leftover postponed quals */
Assert(postponed_qual_list == NIL);
+ /*
+ * However, if there were any special joins then we may well have some
+ * postponed LEFT JOIN clauses to deal with.
+ */
+ if (root->join_info_list)
+ process_postponed_left_join_quals(root);
+
return result;
}
@@ -721,7 +779,7 @@ deconstruct_jointree(PlannerInfo *root)
* below_outer_join is true if this node is within the nullable side of a
* higher-level outer join
* Outputs:
- * *qualscope gets the set of base Relids syntactically included in this
+ * *qualscope gets the set of base+OJ Relids syntactically included in this
* jointree node (do not modify or free this, as it may also be pointed
* to by RestrictInfo and SpecialJoinInfo nodes)
* *inner_join_rels gets the set of base Relids syntactically included in
@@ -806,6 +864,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
* there was exactly one element, we should (and already did) report
* whatever its inner_join_rels were. If there were no elements (is
* that still possible?) the initialization before the loop fixed it.
+ *
+ * XXX now wrong, do we care?
*/
if (list_length(f->fromlist) > 1)
*inner_join_rels = *qualscope;
@@ -820,10 +880,10 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
if (bms_is_subset(pq->relids, *qualscope))
distribute_qual_to_rels(root, pq->qual,
- below_outer_join, JOIN_INNER,
+ below_outer_join, NULL,
root->qual_security_level,
*qualscope, NULL, NULL,
- NULL);
+ true, false, false, false, NULL);
else
*postponed_qual_list = lappend(*postponed_qual_list, pq);
}
@@ -831,16 +891,12 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
/*
* Now process the top-level quals.
*/
- foreach(l, (List *) f->quals)
- {
- Node *qual = (Node *) lfirst(l);
-
- distribute_qual_to_rels(root, qual,
- below_outer_join, JOIN_INNER,
- root->qual_security_level,
- *qualscope, NULL, NULL,
- postponed_qual_list);
- }
+ distribute_quals_to_rels(root, (List *) f->quals,
+ below_outer_join, NULL,
+ root->qual_security_level,
+ *qualscope, NULL, NULL,
+ true, false, false, false,
+ postponed_qual_list);
}
else if (IsA(jtnode, JoinExpr))
{
@@ -857,6 +913,7 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
*rightjoinlist;
List *my_quals;
SpecialJoinInfo *sjinfo;
+ bool postpone_nondegenerate_clauses;
ListCell *l;
/*
@@ -900,6 +957,13 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ /* caution: ANTI join derived from SEMI will lack rtindex */
+ if (j->rtindex != 0)
+ {
+ *qualscope = bms_add_member(*qualscope, j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
+ j->rtindex);
+ }
*inner_join_rels = bms_union(left_inners, right_inners);
nonnullable_rels = leftids;
nullable_rels = rightids;
@@ -914,6 +978,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ /* SEMI join never has rtindex, so don't add to qualscope */
+ Assert(j->rtindex == 0);
*inner_join_rels = bms_union(left_inners, right_inners);
/* Semi join adds no restrictions for quals */
nonnullable_rels = NULL;
@@ -935,6 +1001,10 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ Assert(j->rtindex != 0);
+ *qualscope = bms_add_member(*qualscope, j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
+ j->rtindex);
*inner_join_rels = bms_union(left_inners, right_inners);
/* each side is both outer and inner */
nonnullable_rels = *qualscope;
@@ -994,12 +1064,28 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
leftids, rightids,
*inner_join_rels,
j->jointype,
+ j->rtindex,
my_quals);
if (j->jointype == JOIN_SEMI)
ojscope = NULL;
else
+ {
ojscope = bms_union(sjinfo->min_lefthand,
sjinfo->min_righthand);
+
+ /*
+ * Add back any commutable lower OJ relids that were removed
+ * from min_lefthand or min_righthand, else the ojscope
+ * cross-check in distribute_qual_to_rels will complain. If
+ * any such OJs were removed, we will postpone processing of
+ * non-degenerate clauses, so this addition doesn't affect
+ * anything except that cross-check and some Asserts. Real
+ * clause positioning decisions will be made later, when we
+ * revisit the postponed clauses.
+ */
+ if (sjinfo->commute_below)
+ ojscope = bms_add_members(ojscope, sjinfo->commute_below);
+ }
}
else
{
@@ -1007,18 +1093,26 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
ojscope = NULL;
}
+ /*
+ * If it's a left join with a join clause that is strict for the LHS,
+ * then we need to postpone handling of any non-degenerate join
+ * clauses, in case the join is able to commute with another left join
+ * per identity 3. (Degenerate clauses need not be postponed, since
+ * they will drop down below this join anyway.)
+ */
+ postpone_nondegenerate_clauses = (j->jointype == JOIN_LEFT &&
+ sjinfo->lhs_strict);
+
/* Process the JOIN's qual clauses */
- foreach(l, my_quals)
- {
- Node *qual = (Node *) lfirst(l);
-
- distribute_qual_to_rels(root, qual,
- below_outer_join, j->jointype,
- root->qual_security_level,
- *qualscope,
- ojscope, nonnullable_rels,
- postponed_qual_list);
- }
+ distribute_quals_to_rels(root, my_quals,
+ below_outer_join, sjinfo,
+ root->qual_security_level,
+ *qualscope,
+ ojscope, nonnullable_rels,
+ true, /* allow_equivalence */
+ postpone_nondegenerate_clauses,
+ false, false, /* not clones */
+ postponed_qual_list);
/* Now we can add the SpecialJoinInfo to join_info_list */
if (sjinfo)
@@ -1102,27 +1196,24 @@ process_security_barrier_quals(PlannerInfo *root,
foreach(lc, rte->securityQuals)
{
List *qualset = (List *) lfirst(lc);
- ListCell *lc2;
-
- foreach(lc2, qualset)
- {
- Node *qual = (Node *) lfirst(lc2);
- /*
- * We cheat to the extent of passing ojscope = qualscope rather
- * than its more logical value of NULL. The only effect this has
- * is to force a Var-free qual to be evaluated at the rel rather
- * than being pushed up to top of tree, which we don't want.
- */
- distribute_qual_to_rels(root, qual,
- below_outer_join,
- JOIN_INNER,
- security_level,
- qualscope,
- qualscope,
- NULL,
- NULL);
- }
+ /*
+ * We cheat to the extent of passing ojscope = qualscope rather than
+ * its more logical value of NULL. The only effect this has is to
+ * force a Var-free qual to be evaluated at the rel rather than being
+ * pushed up to top of tree, which we don't want.
+ */
+ distribute_quals_to_rels(root, qualset,
+ below_outer_join,
+ NULL,
+ security_level,
+ qualscope,
+ qualscope,
+ NULL,
+ true,
+ false,
+ false, false, /* not clones */
+ NULL);
security_level++;
}
@@ -1135,10 +1226,11 @@ process_security_barrier_quals(PlannerInfo *root,
* Build a SpecialJoinInfo for the current outer join
*
* Inputs:
- * left_rels: the base Relids syntactically on outer side of join
- * right_rels: the base Relids syntactically on inner side of join
+ * left_rels: the base+OJ Relids syntactically on outer side of join
+ * right_rels: the base+OJ Relids syntactically on inner side of join
* inner_join_rels: base Relids participating in inner joins below this one
* jointype: what it says (must always be LEFT, FULL, SEMI, or ANTI)
+ * ojrelid: RT index of the join RTE (0 for SEMI, which isn't in the RT list)
* clause: the outer join's join condition (in implicit-AND format)
*
* The node should eventually be appended to root->join_info_list, but we
@@ -1152,7 +1244,8 @@ static SpecialJoinInfo *
make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
- JoinType jointype, List *clause)
+ JoinType jointype, Index ojrelid,
+ List *clause)
{
SpecialJoinInfo *sjinfo = makeNode(SpecialJoinInfo);
Relids clause_relids;
@@ -1200,6 +1293,12 @@ make_outerjoininfo(PlannerInfo *root,
sjinfo->syn_lefthand = left_rels;
sjinfo->syn_righthand = right_rels;
sjinfo->jointype = jointype;
+ sjinfo->ojrelid = ojrelid;
+ /* these fields may get added to later: */
+ sjinfo->commute_above_l = NULL;
+ sjinfo->commute_above_r = NULL;
+ sjinfo->commute_below = NULL;
+ sjinfo->oj_joinclause = NIL;
/* this always starts out false */
sjinfo->delay_upper_joins = false;
@@ -1247,6 +1346,7 @@ make_outerjoininfo(PlannerInfo *root,
foreach(l, root->join_info_list)
{
SpecialJoinInfo *otherinfo = (SpecialJoinInfo *) lfirst(l);
+ bool have_unsafe_phvs;
/*
* A full join is an optimization barrier: we can't associate into or
@@ -1262,6 +1362,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_lefthand = bms_add_member(min_lefthand,
+ otherinfo->ojrelid);
}
if (bms_overlap(right_rels, otherinfo->syn_lefthand) ||
bms_overlap(right_rels, otherinfo->syn_righthand))
@@ -1270,11 +1373,26 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_righthand = bms_add_member(min_righthand,
+ otherinfo->ojrelid);
}
/* Needn't do anything else with the full join */
continue;
}
+ /*
+ * If our join condition contains any PlaceHolderVars that need to be
+ * evaluated above the lower OJ, then we can't commute with it.
+ */
+ if (otherinfo->ojrelid != 0)
+ have_unsafe_phvs =
+ contain_placeholder_references_to(root,
+ (Node *) clause,
+ otherinfo->ojrelid);
+ else
+ have_unsafe_phvs = false;
+
/*
* For a lower OJ in our LHS, if our join condition uses the lower
* join's RHS and is not strict for that rel, we must preserve the
@@ -1282,23 +1400,44 @@ make_outerjoininfo(PlannerInfo *root,
* min_lefthand. (We must use its full syntactic relset, not just its
* min_lefthand + min_righthand. This is because there might be other
* OJs below this one that this one can commute with, but we cannot
- * commute with them if we don't with this one.) Also, if the current
- * join is a semijoin or antijoin, we must preserve ordering
- * regardless of strictness.
+ * commute with them if we don't with this one.) Also, if we have
+ * unsafe PHVs or the current join is a semijoin or antijoin, we must
+ * preserve ordering regardless of strictness.
*
* Note: I believe we have to insist on being strict for at least one
* rel in the lower OJ's min_righthand, not its whole syn_righthand.
+ *
+ * When we don't need to preserve ordering, check to see if outer join
+ * identity 3 applies, and if so, remove the lower OJ's ojrelid from
+ * our min_lefthand so that commutation is allowed.
*/
if (bms_overlap(left_rels, otherinfo->syn_righthand))
{
if (bms_overlap(clause_relids, otherinfo->syn_righthand) &&
- (jointype == JOIN_SEMI || jointype == JOIN_ANTI ||
+ (have_unsafe_phvs ||
+ jointype == JOIN_SEMI || jointype == JOIN_ANTI ||
!bms_overlap(strict_relids, otherinfo->min_righthand)))
{
+ /* Preserve ordering */
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_lefthand);
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_lefthand = bms_add_member(min_lefthand,
+ otherinfo->ojrelid);
+ }
+ else if (jointype == JOIN_LEFT &&
+ otherinfo->jointype == JOIN_LEFT &&
+ bms_overlap(strict_relids, otherinfo->min_righthand))
+ {
+ /* Identity 3 applies, so remove the ordering restriction */
+ min_lefthand = bms_del_member(min_lefthand, otherinfo->ojrelid);
+ /* Add commutability markers to both SpecialJoinInfos */
+ otherinfo->commute_above_l =
+ bms_add_member(otherinfo->commute_above_l, ojrelid);
+ sjinfo->commute_below =
+ bms_add_member(sjinfo->commute_below, otherinfo->ojrelid);
}
}
@@ -1313,8 +1452,8 @@ make_outerjoininfo(PlannerInfo *root,
* up with SpecialJoinInfos with identical min_righthands, which can
* confuse join_is_legal (see discussion in backend/optimizer/README).
*
- * Also, we must preserve ordering anyway if either the current join
- * or the lower OJ is either a semijoin or an antijoin.
+ * Also, we must preserve ordering anyway if we have unsafe PHVs, or
+ * if either this join or the lower OJ is a semijoin or antijoin.
*
* Here, we have to consider that "our join condition" includes any
* clauses that syntactically appeared above the lower OJ and below
@@ -1326,21 +1465,43 @@ make_outerjoininfo(PlannerInfo *root,
* join condition are not affected by them. The net effect is
* therefore sufficiently represented by the delay_upper_joins flag
* saved for us by check_outerjoin_delay.
+ *
+ * When we don't need to preserve ordering, check to see if outer join
+ * identity 3 applies, and if so, remove the lower OJ's ojrelid from
+ * our min_righthand so that commutation is allowed.
*/
if (bms_overlap(right_rels, otherinfo->syn_righthand))
{
if (bms_overlap(clause_relids, otherinfo->syn_righthand) ||
!bms_overlap(clause_relids, otherinfo->min_lefthand) ||
+ have_unsafe_phvs ||
jointype == JOIN_SEMI ||
jointype == JOIN_ANTI ||
otherinfo->jointype == JOIN_SEMI ||
otherinfo->jointype == JOIN_ANTI ||
!otherinfo->lhs_strict || otherinfo->delay_upper_joins)
{
+ /* Preserve ordering */
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_lefthand);
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_righthand = bms_add_member(min_righthand,
+ otherinfo->ojrelid);
+ }
+ else if (jointype == JOIN_LEFT &&
+ otherinfo->jointype == JOIN_LEFT &&
+ otherinfo->lhs_strict)
+ {
+ /* Identity 3 applies, so remove the ordering restriction */
+ min_righthand = bms_del_member(min_righthand,
+ otherinfo->ojrelid);
+ /* Add commutability markers to both SpecialJoinInfos */
+ otherinfo->commute_above_r =
+ bms_add_member(otherinfo->commute_above_r, ojrelid);
+ sjinfo->commute_below =
+ bms_add_member(sjinfo->commute_below, otherinfo->ojrelid);
}
}
}
@@ -1565,6 +1726,231 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
sjinfo->semi_rhs_exprs = semi_rhs_exprs;
}
+/*
+ * process_postponed_left_join_quals
+ * Adjust LEFT JOIN quals to be suitable for commuted-left-join cases,
+ * then push them into the joinqual lists and EquivalenceClass structures.
+ *
+ * This runs immediately after we've completed the deconstruct_recurse scan.
+ */
+static void
+process_postponed_left_join_quals(PlannerInfo *root)
+{
+ List *join_info_list_orig = root->join_info_list;
+ ListCell *lc;
+
+ /*
+ * XXX hack: when we call distribute_qual_to_rels to process one of these
+ * quals, neither the owning SpecialJoinInfo nor any later ones can appear
+ * in root->join_info_list, else the wrong things will happen. Fake it
+ * out by emptying join_info_list and rebuilding it as we go. This works
+ * because join_info_list is only appended to during deconstruct_recurse,
+ * so we know we are examining SpecialJoinInfos bottom-up, just like the
+ * first time. Maybe we can get rid of this hack later, if we can fix
+ * things so that distribute_qual_to_rels doesn't consult join_info_list.
+ */
+ root->join_info_list = NIL;
+
+ foreach(lc, join_info_list_orig)
+ {
+ SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+ Relids qualscope,
+ ojscope,
+ nonnullable_rels;
+
+ if (sjinfo->oj_joinclause == NIL) /* nothing to do here */
+ {
+ root->join_info_list = lappend(root->join_info_list, sjinfo);
+ continue;
+ }
+
+ /* Recompute syntactic and semantic scopes of this left join */
+ qualscope = bms_union(sjinfo->syn_lefthand, sjinfo->syn_righthand);
+ qualscope = bms_add_member(qualscope, sjinfo->ojrelid);
+ ojscope = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ nonnullable_rels = sjinfo->syn_lefthand;
+
+ /*
+ * If this join can commute with any other ones per outer-join
+ * identity 3, and it is the one providing the join clause with
+ * flexible semantics, then we have to generate variants of the join
+ * clause with different nullingrels labeling. Otherwise, just push
+ * out the postponed clause as-is.
+ */
+ Assert(sjinfo->lhs_strict); /* else we shouldn't be here */
+ if (sjinfo->commute_above_r ||
+ bms_overlap(sjinfo->commute_below, sjinfo->syn_lefthand))
+ {
+ Relids joins_above;
+ Relids joins_below;
+ Relids joins_so_far;
+ List *quals;
+ ListCell *lc2;
+
+ /*
+ * Put any OJ relids that were removed from min_righthand back
+ * into ojscope, else distribute_qual_to_rels will complain.
+ */
+ ojscope = bms_join(ojscope, bms_intersect(sjinfo->commute_below,
+ sjinfo->syn_righthand));
+
+ /* Identify the outer joins this one commutes with */
+ joins_above = sjinfo->commute_above_r;
+ joins_below = bms_intersect(sjinfo->commute_below,
+ sjinfo->syn_lefthand);
+
+ /*
+ * Generate qual variants with different sets of nullingrels bits.
+ * We only need bit-sets that correspond to the successively less
+ * deeply syntactically-nested subsets of this join and its
+ * commutators. That's true first because obviously only those
+ * forms of the Vars and PHVs could appear elsewhere in the query,
+ * and second because the outer join identities do not provide a
+ * way to re-order such joins in a way that would require
+ * different marking. (That is, while the current join may
+ * commute with several others, none of those others can commute
+ * with each other.) To visit the interesting SpecialJoinInfos in
+ * syntactic nesting order, we rely on the join_info_list to be
+ * ordered that way.
+ *
+ * We first strip out all the nullingrels bits corresponding to
+ * commutating joins below this one, and then successively put
+ * them back as we crawl up the join stack.
+ */
+ quals = sjinfo->oj_joinclause;
+ if (!bms_is_empty(joins_below))
+ quals = (List *) remove_nulling_relids((Node *) quals,
+ joins_below,
+ NULL);
+
+ joins_so_far = NULL;
+ foreach(lc2, join_info_list_orig)
+ {
+ SpecialJoinInfo *othersj = (SpecialJoinInfo *) lfirst(lc2);
+ bool below_sjinfo = false;
+ bool above_sjinfo = false;
+ Relids this_qualscope;
+ Relids this_ojscope;
+ bool allow_equivalence,
+ has_clone,
+ is_clone;
+
+ if (bms_is_member(othersj->ojrelid, joins_below))
+ {
+ /* othersj commutes with sjinfo from below left */
+ below_sjinfo = true;
+ }
+ else if (othersj == sjinfo)
+ {
+ /* found our join in syntactic order */
+ Assert(bms_equal(joins_so_far, joins_below));
+ }
+ else if (bms_is_member(othersj->ojrelid, joins_above))
+ {
+ /* othersj commutes with sjinfo from above */
+ above_sjinfo = true;
+ }
+ else
+ {
+ /* othersj is not relevant, ignore */
+ continue;
+ }
+
+ /*
+ * When we are looking at joins above sjinfo, we are
+ * envisioning pushing sjinfo to above othersj, so add
+ * othersj's nulling bit before distributing the quals.
+ */
+ if (above_sjinfo)
+ quals = (List *)
+ add_nulling_relids((Node *) quals,
+ othersj->min_righthand,
+ bms_make_singleton(othersj->ojrelid));
+
+ /* Compute qualscope and ojscope for this join level */
+ this_qualscope = bms_union(qualscope, joins_so_far);
+ this_ojscope = bms_union(ojscope, joins_so_far);
+ if (above_sjinfo)
+ {
+ /* othersj is not yet in joins_so_far, but we need it */
+ this_qualscope = bms_add_member(this_qualscope,
+ othersj->ojrelid);
+ this_ojscope = bms_add_member(this_ojscope,
+ othersj->ojrelid);
+ /* sjinfo is in joins_so_far, and we don't want it */
+ this_ojscope = bms_del_member(this_ojscope,
+ sjinfo->ojrelid);
+ }
+
+ /*
+ * We generate EquivalenceClasses only from the first form of
+ * the quals, with the fewest nullingrels bits set. An EC
+ * made from this version of the quals can be useful below the
+ * outer-join nest, whereas versions with some nullingrels
+ * bits set would not be. We cannot generate ECs from more
+ * than one version, or we'll make nonsensical conclusions
+ * that Vars with nullingrels bits set are equal to their
+ * versions without. Fortunately, such ECs wouldn't be very
+ * useful anyway, because they'd equate values not observable
+ * outside the join nest. (See optimizer/README.)
+ *
+ * The first form of the quals is also the only one marked as
+ * has_clone rather than is_clone.
+ */
+ allow_equivalence = (joins_so_far == NULL);
+ has_clone = allow_equivalence;
+ is_clone = !has_clone;
+
+ distribute_quals_to_rels(root, quals,
+ false, /* XXX below_outer_join? */
+ sjinfo,
+ root->qual_security_level,
+ this_qualscope,
+ this_ojscope, nonnullable_rels,
+ allow_equivalence,
+ false, /* no more postponement */
+ has_clone,
+ is_clone,
+ NULL);
+
+ /*
+ * Adjust qual nulling bits for next level up, if needed. We
+ * don't want to put sjinfo's own bit in at all, and if we're
+ * above sjinfo then we did it already.
+ */
+ if (below_sjinfo)
+ quals = (List *)
+ add_nulling_relids((Node *) quals,
+ othersj->min_righthand,
+ bms_make_singleton(othersj->ojrelid));
+
+ /* ... and track joins processed so far */
+ joins_so_far = bms_add_member(joins_so_far, othersj->ojrelid);
+ }
+ }
+ else
+ {
+ /* No commutation possible, just process the postponed clauses */
+ distribute_quals_to_rels(root, sjinfo->oj_joinclause,
+ false, /* XXX below_outer_join? */
+ sjinfo,
+ root->qual_security_level,
+ qualscope,
+ ojscope, nonnullable_rels,
+ true, /* allow_equivalence */
+ false, /* no more postponement */
+ false, false, /* not clones */
+ NULL);
+ }
+
+ /* Clear out the list, just so we don't have multiply-linked trees */
+ sjinfo->oj_joinclause = NIL;
+
+ /* Now add sjinfo to the new join_info_list */
+ root->join_info_list = lappend(root->join_info_list, sjinfo);
+ }
+}
+
/*****************************************************************************
*
@@ -1572,6 +1958,46 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
*
*****************************************************************************/
+/*
+ * distribute_quals_to_rels
+ * Convenience routine to apply distribute_qual_to_rels to each element
+ * of an AND'ed list of clauses.
+ */
+static void
+distribute_quals_to_rels(PlannerInfo *root, List *clauses,
+ bool below_outer_join,
+ SpecialJoinInfo *sjinfo,
+ Index security_level,
+ Relids qualscope,
+ Relids ojscope,
+ Relids outerjoin_nonnullable,
+ bool allow_equivalence,
+ bool postpone_nondegenerate_clauses,
+ bool has_clone,
+ bool is_clone,
+ List **postponed_qual_list)
+{
+ ListCell *lc;
+
+ foreach(lc, clauses)
+ {
+ Node *clause = (Node *) lfirst(lc);
+
+ distribute_qual_to_rels(root, clause,
+ below_outer_join,
+ sjinfo,
+ security_level,
+ qualscope,
+ ojscope,
+ outerjoin_nonnullable,
+ allow_equivalence,
+ postpone_nondegenerate_clauses,
+ has_clone,
+ is_clone,
+ postponed_qual_list);
+ }
+}
+
/*
* distribute_qual_to_rels
* Add clause information to either the baserestrictinfo or joininfo list
@@ -1586,7 +2012,7 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
* 'clause': the qual clause to be distributed
* 'below_outer_join': true if the qual is from a JOIN/ON that is below the
* nullable side of a higher-level outer join
- * 'jointype': type of join the qual is from (JOIN_INNER for a WHERE clause)
+ * 'sjinfo': join's SpecialJoinInfo (NULL for an inner join or WHERE clause)
* 'security_level': security_level to assign to the qual
* 'qualscope': set of baserels the qual's syntactic scope covers
* 'ojscope': NULL if not an outer-join qual, else the minimum set of baserels
@@ -1595,6 +2021,12 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
* baserels appearing on the outer (nonnullable) side of the join
* (for FULL JOIN this includes both sides of the join, and must in fact
* equal qualscope)
+ * 'allow_equivalence': true if it's okay to convert clause into an
+ * EquivalenceClass
+ * 'postpone_nondegenerate_clauses': true if non-degenerate outer join clauses
+ * should be added to sjinfo->oj_joinclause instead of being processed
+ * 'has_clone': has_clone property to assign to the qual
+ * 'is_clone': is_clone property to assign to the qual
* 'postponed_qual_list': list of PostponedQual structs, which we can add
* this qual to if it turns out to belong to a higher join level.
* Can be NULL if caller knows postponement is impossible.
@@ -1604,16 +2036,21 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
* level, which will be ojscope not necessarily qualscope.
*
* At the time this is called, root->join_info_list must contain entries for
- * all and only those special joins that are syntactically below this qual.
+ * all and only those special joins that are syntactically below this qual;
+ * in particular, the passed-in SpecialJoinInfo isn't yet in that list.
*/
static void
distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool below_outer_join,
- JoinType jointype,
+ SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
Relids ojscope,
Relids outerjoin_nonnullable,
+ bool allow_equivalence,
+ bool postpone_nondegenerate_clauses,
+ bool has_clone,
+ bool is_clone,
List **postponed_qual_list)
{
Relids relids;
@@ -1646,7 +2083,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
PostponedQual *pq = (PostponedQual *) palloc(sizeof(PostponedQual));
Assert(root->hasLateralRTEs); /* shouldn't happen otherwise */
- Assert(jointype == JOIN_INNER); /* mustn't postpone past outer join */
+ Assert(sjinfo == NULL); /* mustn't postpone past outer join */
pq->qual = clause;
pq->relids = relids;
*postponed_qual_list = lappend(*postponed_qual_list, pq);
@@ -1708,7 +2145,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
{
relids =
get_relids_in_jointree((Node *) root->parse->jointree,
- false);
+ true, false);
qualscope = bms_copy(relids);
}
}
@@ -1751,8 +2188,18 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
{
/*
* The qual is attached to an outer join and mentions (some of the)
- * rels on the nonnullable side, so it's not degenerate.
- *
+ * rels on the nonnullable side, so it's not degenerate. If the
+ * caller wants to postpone handling such clauses, just add it to
+ * sjinfo->oj_joinclause and return. (The work we've done up to here
+ * will have to be redone later, but there's not much of it.)
+ */
+ if (postpone_nondegenerate_clauses)
+ {
+ sjinfo->oj_joinclause = lappend(sjinfo->oj_joinclause, clause);
+ return;
+ }
+
+ /*
* We can't use such a clause to deduce equivalence (the left and
* right sides might be unequal above the join because one of them has
* gone to NULL) ... but we might be able to use it for more limited
@@ -1818,6 +2265,11 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
if (check_redundant_nullability_qual(root, clause))
return;
}
+ else if (!allow_equivalence)
+ {
+ /* Caller says it mustn't become an equivalence class */
+ maybe_equivalence = false;
+ }
else
{
/*
@@ -1852,6 +2304,10 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
outerjoin_nonnullable,
nullable_relids);
+ /* Apply appropriate clone marking, too */
+ restrictinfo->has_clone = has_clone;
+ restrictinfo->is_clone = is_clone;
+
/*
* If it's a join clause (either naturally, or because delayed by
* outer-join rules), add vars used in the clause to targetlists of their
@@ -1950,11 +2406,15 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
restrictinfo);
return;
}
- if (jointype == JOIN_FULL)
+ if (sjinfo && sjinfo->jointype == JOIN_FULL)
{
/* FULL JOIN (above tests cannot match in this case) */
+ FullJoinClauseInfo *fjinfo = makeNode(FullJoinClauseInfo);
+
+ fjinfo->rinfo = restrictinfo;
+ fjinfo->sjinfo = sjinfo;
root->full_join_clauses = lappend(root->full_join_clauses,
- restrictinfo);
+ fjinfo);
return;
}
/* nope, so fall through to distribute_restrictinfo_to_rels */
@@ -2348,7 +2808,7 @@ process_implied_equality(PlannerInfo *root,
{
relids =
get_relids_in_jointree((Node *) root->parse->jointree,
- false);
+ true, false);
}
}
}
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 63deed27c9..69e725d159 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -158,13 +158,15 @@ query_planner(PlannerInfo *root,
/*
* Construct RelOptInfo nodes for all base relations used in the query.
- * Appendrel member relations ("other rels") will be added later.
+ * Appendrel member relations ("other rels") will be added later. We also
+ * construct a bitmapset of all the baserel relids.
*
* Note: the reason we find the baserels by searching the jointree, rather
* than scanning the rangetable, is that the rangetable may contain RTEs
* for rels not actively part of the query, for example views. We don't
* want to make RelOptInfos for them.
*/
+ root->all_baserels = NULL;
add_base_rels_to_query(root, (Node *) parse->jointree);
/*
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 493a3af0fa..e743a5d9fe 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2223,7 +2223,7 @@ preprocess_rowmarks(PlannerInfo *root)
* make a bitmapset of all base rels and then remove the items we don't
* need or have FOR [KEY] UPDATE/SHARE marks for.
*/
- rels = get_relids_in_jointree((Node *) parse->jointree, false);
+ rels = get_relids_in_jointree((Node *) parse->jointree, false, false);
if (parse->resultRelation)
rels = bms_del_member(rels, parse->resultRelation);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 1cb0abdbc1..8fff731756 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -29,11 +29,21 @@
#include "utils/syscache.h"
+typedef enum
+{
+ NRM_EQUAL, /* expect exact match of nullingrels */
+ NRM_SUBSET, /* actual Var may have a subset of input */
+ NRM_SUPERSET /* actual Var may have a superset of input */
+} NullingRelsMatch;
+
typedef struct
{
int varno; /* RT index of Var */
AttrNumber varattno; /* attr number of Var */
AttrNumber resno; /* TLE position of Var */
+#ifdef USE_ASSERT_CHECKING
+ Bitmapset *varnullingrels; /* Var's varnullingrels */
+#endif
} tlist_vinfo;
typedef struct
@@ -59,6 +69,7 @@ typedef struct
indexed_tlist *inner_itlist;
Index acceptable_rel;
int rtoffset;
+ NullingRelsMatch nrm_match;
double num_exec;
} fix_join_expr_context;
@@ -68,6 +79,7 @@ typedef struct
indexed_tlist *subplan_itlist;
int newvarno;
int rtoffset;
+ NullingRelsMatch nrm_match;
double num_exec;
} fix_upper_expr_context;
@@ -150,7 +162,12 @@ static indexed_tlist *build_tlist_index(List *tlist);
static Var *search_indexed_tlist_for_var(Var *var,
indexed_tlist *itlist,
int newvarno,
- int rtoffset);
+ int rtoffset,
+ NullingRelsMatch nrm_match);
+static Var *search_indexed_tlist_for_phv(PlaceHolderVar *phv,
+ indexed_tlist *itlist,
+ int newvarno,
+ NullingRelsMatch nrm_match);
static Var *search_indexed_tlist_for_non_var(Expr *node,
indexed_tlist *itlist,
int newvarno);
@@ -163,14 +180,18 @@ static List *fix_join_expr(PlannerInfo *root,
indexed_tlist *outer_itlist,
indexed_tlist *inner_itlist,
Index acceptable_rel,
- int rtoffset, double num_exec);
+ int rtoffset,
+ NullingRelsMatch nrm_match,
+ double num_exec);
static Node *fix_join_expr_mutator(Node *node,
fix_join_expr_context *context);
static Node *fix_upper_expr(PlannerInfo *root,
Node *node,
indexed_tlist *subplan_itlist,
int newvarno,
- int rtoffset, double num_exec);
+ int rtoffset,
+ NullingRelsMatch nrm_match,
+ double num_exec);
static Node *fix_upper_expr_mutator(Node *node,
fix_upper_expr_context *context);
static List *set_returning_clause_references(PlannerInfo *root,
@@ -1045,13 +1066,13 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
fix_join_expr(root, splan->onConflictSet,
NULL, itlist,
linitial_int(splan->resultRelations),
- rtoffset, NUM_EXEC_QUAL(plan));
+ rtoffset, NRM_EQUAL, NUM_EXEC_QUAL(plan));
splan->onConflictWhere = (Node *)
fix_join_expr(root, (List *) splan->onConflictWhere,
NULL, itlist,
linitial_int(splan->resultRelations),
- rtoffset, NUM_EXEC_QUAL(plan));
+ rtoffset, NRM_EQUAL, NUM_EXEC_QUAL(plan));
pfree(itlist);
@@ -1108,6 +1129,7 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
NULL, itlist,
resultrel,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST(plan));
/* Fix quals too. */
@@ -1116,6 +1138,7 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
NULL, itlist,
resultrel,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL(plan));
}
}
@@ -1261,6 +1284,7 @@ set_indexonlyscan_references(PlannerInfo *root,
index_itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST((Plan *) plan));
plan->scan.plan.qual = (List *)
fix_upper_expr(root,
@@ -1268,6 +1292,7 @@ set_indexonlyscan_references(PlannerInfo *root,
index_itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) plan));
plan->recheckqual = (List *)
fix_upper_expr(root,
@@ -1275,6 +1300,7 @@ set_indexonlyscan_references(PlannerInfo *root,
index_itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) plan));
/* indexqual is already transformed to reference index columns */
plan->indexqual = fix_scan_list(root, plan->indexqual,
@@ -1481,6 +1507,7 @@ set_foreignscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST((Plan *) fscan));
fscan->scan.plan.qual = (List *)
fix_upper_expr(root,
@@ -1488,6 +1515,7 @@ set_foreignscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) fscan));
fscan->fdw_exprs = (List *)
fix_upper_expr(root,
@@ -1495,6 +1523,7 @@ set_foreignscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) fscan));
fscan->fdw_recheck_quals = (List *)
fix_upper_expr(root,
@@ -1502,6 +1531,7 @@ set_foreignscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) fscan));
pfree(itlist);
/* fdw_scan_tlist itself just needs fix_scan_list() adjustments */
@@ -1562,6 +1592,7 @@ set_customscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST((Plan *) cscan));
cscan->scan.plan.qual = (List *)
fix_upper_expr(root,
@@ -1569,6 +1600,7 @@ set_customscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) cscan));
cscan->custom_exprs = (List *)
fix_upper_expr(root,
@@ -1576,6 +1608,7 @@ set_customscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) cscan));
pfree(itlist);
/* custom_scan_tlist itself just needs fix_scan_list() adjustments */
@@ -1780,6 +1813,7 @@ set_hash_references(PlannerInfo *root, Plan *plan, int rtoffset)
outer_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL(plan));
/* Hash doesn't project */
@@ -2115,6 +2149,7 @@ fix_scan_expr_mutator(Node *node, fix_scan_expr_context *context)
/* At scan level, we should always just evaluate the contained expr */
PlaceHolderVar *phv = (PlaceHolderVar *) node;
+ Assert(phv->phnullingrels == NULL);
return fix_scan_expr_mutator((Node *) phv->phexpr, context);
}
if (IsA(node, AlternativeSubPlan))
@@ -2172,6 +2207,7 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
inner_itlist,
(Index) 0,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) join));
/* Now do join-type-specific stuff */
@@ -2184,11 +2220,21 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
{
NestLoopParam *nlp = (NestLoopParam *) lfirst(lc);
+ /*
+ * Because we don't reparameterize parameterized paths to match
+ * the outer-join level at which they are used, Vars seen in the
+ * NestLoopParam expression may have nullingrels that are just a
+ * subset of those in the Vars actually available from the outer
+ * side. Not checking this exactly is a bit grotty, but the work
+ * needed to make things match up perfectly seems well out of
+ * proportion to the value.
+ */
nlp->paramval = (Var *) fix_upper_expr(root,
(Node *) nlp->paramval,
outer_itlist,
OUTER_VAR,
rtoffset,
+ NRM_SUBSET,
NUM_EXEC_TLIST(outer_plan));
/* Check we replaced any PlaceHolderVar with simple Var */
if (!(IsA(nlp->paramval, Var) &&
@@ -2206,6 +2252,7 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
inner_itlist,
(Index) 0,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) join));
}
else if (IsA(join, HashJoin))
@@ -2218,6 +2265,7 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
inner_itlist,
(Index) 0,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) join));
/*
@@ -2229,45 +2277,27 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
outer_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) join));
}
/*
* Now we need to fix up the targetlist and qpqual, which are logically
- * above the join. This means they should not re-use any input expression
- * that was computed in the nullable side of an outer join. Vars and
- * PlaceHolderVars are fine, so we can implement this restriction just by
- * clearing has_non_vars in the indexed_tlist structs.
- *
- * XXX This is a grotty workaround for the fact that we don't clearly
- * distinguish between a Var appearing below an outer join and the "same"
- * Var appearing above it. If we did, we'd not need to hack the matching
- * rules this way.
+ * above the join. This means that, if it's not an inner join, any Vars
+ * and PHVs appearing here should have nullingrels that include the
+ * effects of the outer join, ie they will have nullingrels equal to the
+ * input Vars' nullingrels plus the bit added by the outer join. We don't
+ * currently have enough info available here to identify what that should
+ * be, so we just tell fix_join_expr to accept superset nullingrels
+ * matches instead of exact ones.
*/
- switch (join->jointype)
- {
- case JOIN_LEFT:
- case JOIN_SEMI:
- case JOIN_ANTI:
- inner_itlist->has_non_vars = false;
- break;
- case JOIN_RIGHT:
- outer_itlist->has_non_vars = false;
- break;
- case JOIN_FULL:
- outer_itlist->has_non_vars = false;
- inner_itlist->has_non_vars = false;
- break;
- default:
- break;
- }
-
join->plan.targetlist = fix_join_expr(root,
join->plan.targetlist,
outer_itlist,
inner_itlist,
(Index) 0,
rtoffset,
+ (join->jointype == JOIN_INNER ? NRM_EQUAL : NRM_SUPERSET),
NUM_EXEC_TLIST((Plan *) join));
join->plan.qual = fix_join_expr(root,
join->plan.qual,
@@ -2275,6 +2305,7 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
inner_itlist,
(Index) 0,
rtoffset,
+ (join->jointype == JOIN_INNER ? NRM_EQUAL : NRM_SUPERSET),
NUM_EXEC_QUAL((Plan *) join));
pfree(outer_itlist);
@@ -2329,6 +2360,7 @@ set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset)
subplan_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST(plan));
}
else
@@ -2337,6 +2369,7 @@ set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset)
subplan_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST(plan));
tle = flatCopyTargetEntry(tle);
tle->expr = (Expr *) newexpr;
@@ -2350,6 +2383,7 @@ set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset)
subplan_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL(plan));
pfree(subplan_itlist);
@@ -2550,7 +2584,7 @@ set_dummy_tlist_references(Plan *plan, int rtoffset)
* tlist_member() searches.
*
* The result of this function is an indexed_tlist struct to pass to
- * search_indexed_tlist_for_var() or search_indexed_tlist_for_non_var().
+ * search_indexed_tlist_for_var() and siblings.
* When done, the indexed_tlist may be freed with a single pfree().
*/
static indexed_tlist *
@@ -2582,6 +2616,9 @@ build_tlist_index(List *tlist)
vinfo->varno = var->varno;
vinfo->varattno = var->varattno;
vinfo->resno = tle->resno;
+#ifdef USE_ASSERT_CHECKING
+ vinfo->varnullingrels = var->varnullingrels;
+#endif
vinfo++;
}
else if (tle->expr && IsA(tle->expr, PlaceHolderVar))
@@ -2634,6 +2671,9 @@ build_tlist_index_other_vars(List *tlist, int ignore_rel)
vinfo->varno = var->varno;
vinfo->varattno = var->varattno;
vinfo->resno = tle->resno;
+#ifdef USE_ASSERT_CHECKING
+ vinfo->varnullingrels = var->varnullingrels;
+#endif
vinfo++;
}
}
@@ -2653,10 +2693,17 @@ build_tlist_index_other_vars(List *tlist, int ignore_rel)
* modified varno/varattno (to wit, newvarno and the resno of the TLE entry).
* Also ensure that varnosyn is incremented by rtoffset.
* If no match, return NULL.
+ *
+ * In debugging builds, we cross-check the varnullingrels of the subplan
+ * output Var based on nrm_match. Most call sites should pass NRM_EQUAL
+ * indicating we expect an exact match. However, there are places where
+ * we haven't cleaned things up completely, and we have to settle for
+ * allowing subset or superset matches.
*/
static Var *
search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
- int newvarno, int rtoffset)
+ int newvarno, int rtoffset,
+ NullingRelsMatch nrm_match)
{
int varno = var->varno;
AttrNumber varattno = var->varattno;
@@ -2672,6 +2719,36 @@ search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
/* Found a match */
Var *newvar = copyVar(var);
+ /*
+ * Assert that we kept all the nullingrels machinations straight.
+ *
+ * XXX eventually reduce this to a plain Assert. Right now it's
+ * more useful to warn and keep going.
+ *
+ * XXX skip this check for system columns and whole-row Vars.
+ * That's because such Vars might be row identity Vars, which are
+ * generated without any varnullingrels. It'd be hard to do
+ * otherwise, since they're normally made very early in planning,
+ * when we haven't looked at the jointree yet and don't know which
+ * joins might null such Vars. Doesn't seem worth the expense to
+ * make them fully valid. (While it's slightly annoying that we
+ * thereby lose checking for user-written references to such
+ * columns, it seems unlikely that a bug in nullingrels logic
+ * would affect only system columns.)
+ */
+#ifdef USE_ASSERT_CHECKING
+ if (!(varattno <= 0 ||
+ (nrm_match == NRM_SUBSET ?
+ bms_is_subset(var->varnullingrels, vinfo->varnullingrels) :
+ nrm_match == NRM_SUPERSET ?
+ bms_is_subset(vinfo->varnullingrels, var->varnullingrels) :
+ bms_equal(vinfo->varnullingrels, var->varnullingrels))))
+ elog(WARNING, "bogus varnullingrels for (%d,%d): expected %s, found %s in subplan",
+ varno, varattno,
+ bmsToString(var->varnullingrels),
+ bmsToString(vinfo->varnullingrels));
+#endif
+
newvar->varno = newvarno;
newvar->varattno = vinfo->resno;
if (newvar->varnosyn > 0)
@@ -2684,15 +2761,74 @@ search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
}
/*
- * search_indexed_tlist_for_non_var --- find a non-Var in an indexed tlist
+ * search_indexed_tlist_for_phv --- find a PlaceHolderVar in an indexed tlist
*
* If a match is found, return a Var constructed to reference the tlist item.
* If no match, return NULL.
*
- * NOTE: it is a waste of time to call this unless itlist->has_ph_vars or
- * itlist->has_non_vars. Furthermore, set_join_references() relies on being
- * able to prevent matching of non-Vars by clearing itlist->has_non_vars,
- * so there's a correctness reason not to call it unless that's set.
+ * Cross-check phnullingrels as in search_indexed_tlist_for_var.
+ *
+ * NOTE: it is a waste of time to call this unless itlist->has_ph_vars.
+ */
+static Var *
+search_indexed_tlist_for_phv(PlaceHolderVar *phv,
+ indexed_tlist *itlist, int newvarno,
+ NullingRelsMatch nrm_match)
+{
+ ListCell *lc;
+
+ foreach(lc, itlist->tlist)
+ {
+ TargetEntry *tle = (TargetEntry *) lfirst(lc);
+
+ if (tle->expr && IsA(tle->expr, PlaceHolderVar))
+ {
+ PlaceHolderVar *subphv = (PlaceHolderVar *) tle->expr;
+ Var *newvar;
+
+ /*
+ * Analogously to search_indexed_tlist_for_var, we match on phid
+ * only. We don't use equal(), partially for speed but mostly
+ * because phnullingrels might not be exactly equal.
+ */
+ if (phv->phid != subphv->phid)
+ continue;
+
+ /*
+ * Assert that we kept all the nullingrels machinations straight.
+ *
+ * XXX eventually reduce this to a plain Assert. Right now it's
+ * more useful to warn and keep going.
+ */
+#ifdef USE_ASSERT_CHECKING
+ if (!(nrm_match == NRM_SUBSET ?
+ bms_is_subset(phv->phnullingrels, subphv->phnullingrels) :
+ nrm_match == NRM_SUPERSET ?
+ bms_is_subset(subphv->phnullingrels, phv->phnullingrels) :
+ bms_equal(subphv->phnullingrels, phv->phnullingrels)))
+ elog(WARNING, "bogus phnullingrels for %d: expected %s, found %s in subplan",
+ phv->phid,
+ bmsToString(phv->phnullingrels),
+ bmsToString(subphv->phnullingrels));
+#endif
+
+ /* Found a matching subplan output expression */
+ newvar = makeVarFromTargetEntry(newvarno, tle);
+ newvar->varnosyn = 0; /* wasn't ever a plain Var */
+ newvar->varattnosyn = 0;
+ return newvar;
+ }
+ }
+ return NULL; /* no match */
+}
+
+/*
+ * search_indexed_tlist_for_non_var --- find a non-Var/PHV in an indexed tlist
+ *
+ * If a match is found, return a Var constructed to reference the tlist item.
+ * If no match, return NULL.
+ *
+ * NOTE: it is a waste of time to call this unless itlist->has_non_vars.
*/
static Var *
search_indexed_tlist_for_non_var(Expr *node,
@@ -2799,6 +2935,7 @@ search_indexed_tlist_for_sortgroupref(Expr *node,
* 'acceptable_rel' is either zero or the rangetable index of a relation
* whose Vars may appear in the clause without provoking an error
* 'rtoffset': how much to increment varnos by
+ * 'nrm_match': as for search_indexed_tlist_for_var()
* 'num_exec': estimated number of executions of expression
*
* Returns the new expression tree. The original clause structure is
@@ -2811,6 +2948,7 @@ fix_join_expr(PlannerInfo *root,
indexed_tlist *inner_itlist,
Index acceptable_rel,
int rtoffset,
+ NullingRelsMatch nrm_match,
double num_exec)
{
fix_join_expr_context context;
@@ -2820,6 +2958,7 @@ fix_join_expr(PlannerInfo *root,
context.inner_itlist = inner_itlist;
context.acceptable_rel = acceptable_rel;
context.rtoffset = rtoffset;
+ context.nrm_match = nrm_match;
context.num_exec = num_exec;
return (List *) fix_join_expr_mutator((Node *) clauses, &context);
}
@@ -2841,7 +2980,8 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
newvar = search_indexed_tlist_for_var(var,
context->outer_itlist,
OUTER_VAR,
- context->rtoffset);
+ context->rtoffset,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
@@ -2852,7 +2992,8 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
newvar = search_indexed_tlist_for_var(var,
context->inner_itlist,
INNER_VAR,
- context->rtoffset);
+ context->rtoffset,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
@@ -2877,22 +3018,25 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* See if the PlaceHolderVar has bubbled up from a lower plan node */
if (context->outer_itlist && context->outer_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->outer_itlist,
- OUTER_VAR);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->outer_itlist,
+ OUTER_VAR,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
if (context->inner_itlist && context->inner_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->inner_itlist,
- INNER_VAR);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->inner_itlist,
+ INNER_VAR,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
/* If not supplied by input plans, evaluate the contained expr */
+ /* XXX can we assert something about phnullingrels? */
return fix_join_expr_mutator((Node *) phv->phexpr, context);
}
/* Try matching more complex expressions too, if tlists have any */
@@ -2951,6 +3095,7 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
* 'subplan_itlist': indexed target list for subplan (or index)
* 'newvarno': varno to use for Vars referencing tlist elements
* 'rtoffset': how much to increment varnos by
+ * 'nrm_match': as for search_indexed_tlist_for_var()
* 'num_exec': estimated number of executions of expression
*
* The resulting tree is a copy of the original in which all Var nodes have
@@ -2963,6 +3108,7 @@ fix_upper_expr(PlannerInfo *root,
indexed_tlist *subplan_itlist,
int newvarno,
int rtoffset,
+ NullingRelsMatch nrm_match,
double num_exec)
{
fix_upper_expr_context context;
@@ -2971,6 +3117,7 @@ fix_upper_expr(PlannerInfo *root,
context.subplan_itlist = subplan_itlist;
context.newvarno = newvarno;
context.rtoffset = rtoffset;
+ context.nrm_match = nrm_match;
context.num_exec = num_exec;
return fix_upper_expr_mutator(node, &context);
}
@@ -2989,7 +3136,8 @@ fix_upper_expr_mutator(Node *node, fix_upper_expr_context *context)
newvar = search_indexed_tlist_for_var(var,
context->subplan_itlist,
context->newvarno,
- context->rtoffset);
+ context->rtoffset,
+ context->nrm_match);
if (!newvar)
elog(ERROR, "variable not found in subplan target list");
return (Node *) newvar;
@@ -3001,13 +3149,15 @@ fix_upper_expr_mutator(Node *node, fix_upper_expr_context *context)
/* See if the PlaceHolderVar has bubbled up from a lower plan node */
if (context->subplan_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->subplan_itlist,
- context->newvarno);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->subplan_itlist,
+ context->newvarno,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
/* If not supplied by input plan, evaluate the contained expr */
+ /* XXX can we assert something about phnullingrels? */
return fix_upper_expr_mutator((Node *) phv->phexpr, context);
}
/* Try matching more complex expressions too, if tlist has any */
@@ -3114,6 +3264,7 @@ set_returning_clause_references(PlannerInfo *root,
NULL,
resultRelation,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST(topplan));
pfree(itlist);
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index b4b9099eb6..4821b7a71f 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -49,17 +49,28 @@ typedef struct pullup_replace_vars_context
* pullup (set only if target_rte->lateral) */
bool *outer_hasSubLinks; /* -> outer query's hasSubLinks */
int varno; /* varno of subquery */
- bool need_phvs; /* do we need PlaceHolderVars? */
- bool wrap_non_vars; /* do we need 'em on *all* non-Vars? */
+ bool wrap_non_vars; /* do we need all non-Var outputs to be PHVs? */
Node **rv_cache; /* cache for results with PHVs */
} pullup_replace_vars_context;
-typedef struct reduce_outer_joins_state
+typedef struct reduce_outer_joins_pass1_state
{
Relids relids; /* base relids within this subtree */
bool contains_outer; /* does subtree contain outer join(s)? */
List *sub_states; /* List of states for subtree components */
-} reduce_outer_joins_state;
+} reduce_outer_joins_pass1_state;
+
+typedef struct reduce_outer_joins_pass2_state
+{
+ Relids inner_reduced; /* OJ relids reduced to plain inner joins */
+ List *partial_reduced; /* List of partially reduced FULL joins */
+} reduce_outer_joins_pass2_state;
+
+typedef struct reduce_outer_joins_partial_state
+{
+ int full_join_rti; /* RT index of a formerly-FULL join */
+ Relids unreduced_side; /* relids in its still-nullable side */
+} reduce_outer_joins_partial_state;
static Node *pull_up_sublinks_jointree_recurse(PlannerInfo *root, Node *jtnode,
Relids *relids);
@@ -68,12 +79,10 @@ static Node *pull_up_sublinks_qual_recurse(PlannerInfo *root, Node *node,
Node **jtlink2, Relids available_rels2);
static Node *pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static Node *pull_up_simple_subquery(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static Node *pull_up_simple_union_all(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte);
@@ -90,7 +99,6 @@ static Node *pull_up_simple_values(PlannerInfo *root, Node *jtnode,
static bool is_simple_values(PlannerInfo *root, RangeTblEntry *rte);
static Node *pull_up_constant_function(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static bool is_simple_union_all(Query *subquery);
static bool is_simple_union_all_recurse(Node *setOp, Query *setOpQuery,
@@ -101,24 +109,26 @@ static bool jointree_contains_lateral_outer_refs(PlannerInfo *root,
Relids safe_upper_varnos);
static void perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars_context *rvcontext,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static void replace_vars_in_jointree(Node *jtnode,
- pullup_replace_vars_context *context,
- JoinExpr *lowest_nulling_outer_join);
+ pullup_replace_vars_context *context);
static Node *pullup_replace_vars(Node *expr,
pullup_replace_vars_context *context);
static Node *pullup_replace_vars_callback(Var *var,
replace_rte_variables_context *context);
static Query *pullup_replace_vars_subquery(Query *query,
pullup_replace_vars_context *context);
-static reduce_outer_joins_state *reduce_outer_joins_pass1(Node *jtnode);
+static reduce_outer_joins_pass1_state *reduce_outer_joins_pass1(Node *jtnode);
static void reduce_outer_joins_pass2(Node *jtnode,
- reduce_outer_joins_state *state,
+ reduce_outer_joins_pass1_state *state1,
+ reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
List *forced_null_vars);
-static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode);
+static void report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
+ int rtindex, Relids relids);
+static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Relids *dropped_outer_joins);
static int get_result_relid(PlannerInfo *root, Node *jtnode);
static void remove_result_refs(PlannerInfo *root, int varno, Node *newjtloc);
static bool find_dependent_phvs(PlannerInfo *root, int varno);
@@ -763,7 +773,7 @@ pull_up_subqueries(PlannerInfo *root)
/* Recursion starts with no containing join nor appendrel */
root->parse->jointree = (FromExpr *)
pull_up_subqueries_recurse(root, (Node *) root->parse->jointree,
- NULL, NULL, NULL);
+ NULL, NULL);
/* We should still have a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
}
@@ -778,12 +788,6 @@ pull_up_subqueries(PlannerInfo *root)
* lowest_outer_join references the lowest such JoinExpr node; otherwise
* it is NULL. We use this to constrain the effects of LATERAL subqueries.
*
- * If this jointree node is within the nullable side of an outer join, then
- * lowest_nulling_outer_join references the lowest such JoinExpr node;
- * otherwise it is NULL. This forces use of the PlaceHolderVar mechanism for
- * references to non-nullable targetlist items, but only for references above
- * that join.
- *
* If we are looking at a member subquery of an append relation,
* containing_appendrel describes that relation; else it is NULL.
* This forces use of the PlaceHolderVar mechanism for all non-Var targetlist
@@ -800,15 +804,14 @@ pull_up_subqueries(PlannerInfo *root)
* Notice also that we can't turn pullup_replace_vars loose on the whole
* jointree, because it'd return a mutated copy of the tree; we have to
* invoke it just on the quals, instead. This behavior is what makes it
- * reasonable to pass lowest_outer_join and lowest_nulling_outer_join as
- * pointers rather than some more-indirect way of identifying the lowest
- * OJs. Likewise, we don't replace append_rel_list members but only their
- * substructure, so the containing_appendrel reference is safe to use.
+ * reasonable to pass lowest_outer_join as a pointer rather than some
+ * more-indirect way of identifying the lowest OJ. Likewise, we don't
+ * replace append_rel_list members but only their substructure, so the
+ * containing_appendrel reference is safe to use.
*/
static Node *
pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Assert(jtnode != NULL);
@@ -830,7 +833,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
is_safe_append_member(rte->subquery)))
return pull_up_simple_subquery(root, jtnode, rte,
lowest_outer_join,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -863,7 +865,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
*/
if (rte->rtekind == RTE_FUNCTION)
return pull_up_constant_function(root, jtnode, rte,
- lowest_nulling_outer_join,
containing_appendrel);
/* Otherwise, do nothing at this node. */
@@ -879,7 +880,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
{
lfirst(l) = pull_up_subqueries_recurse(root, lfirst(l),
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
}
}
@@ -894,11 +894,9 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
case JOIN_INNER:
j->larg = pull_up_subqueries_recurse(root, j->larg,
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
break;
case JOIN_LEFT:
@@ -906,31 +904,25 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
case JOIN_ANTI:
j->larg = pull_up_subqueries_recurse(root, j->larg,
j,
- lowest_nulling_outer_join,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
- j,
j,
NULL);
break;
case JOIN_FULL:
j->larg = pull_up_subqueries_recurse(root, j->larg,
- j,
j,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
- j,
j,
NULL);
break;
case JOIN_RIGHT:
j->larg = pull_up_subqueries_recurse(root, j->larg,
- j,
j,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
j,
- lowest_nulling_outer_join,
NULL);
break;
default:
@@ -960,7 +952,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
static Node *
pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -1107,31 +1098,25 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* The subquery's targetlist items are now in the appropriate form to
* insert into the top query, except that we may need to wrap them in
* PlaceHolderVars. Set up required context data for pullup_replace_vars.
+ * (Note that we should include the subquery's inner joins in relids,
+ * since it may include join alias vars referencing them.)
*/
rvcontext.root = root;
rvcontext.targetlist = subquery->targetList;
rvcontext.target_rte = rte;
if (rte->lateral)
rvcontext.relids = get_relids_in_jointree((Node *) subquery->jointree,
- true);
+ true, true);
else /* won't need relids */
rvcontext.relids = NULL;
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = varno;
- /* these flags will be set below, if needed */
- rvcontext.need_phvs = false;
+ /* this flag will be set below, if needed */
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(subquery->targetList) + 1) *
sizeof(Node *));
- /*
- * If we are under an outer join then non-nullable items and lateral
- * references may have to be turned into PlaceHolderVars.
- */
- if (lowest_nulling_outer_join != NULL)
- rvcontext.need_phvs = true;
-
/*
* If we are dealing with an appendrel member then anything that's not a
* simple Var has to be turned into a PlaceHolderVar. We force this to
@@ -1140,10 +1125,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* expression actually available from the appendrel.
*/
if (containing_appendrel != NULL)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* If the parent query uses grouping sets, we need a PlaceHolderVar for
@@ -1155,10 +1137,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* that pullup_replace_vars hasn't currently got.)
*/
if (parse->groupingSets)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* Replace all of the top query's references to the subquery's outputs
@@ -1166,7 +1145,6 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* replace any of the jointree structure.
*/
perform_pullup_replace_vars(root, &rvcontext,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -1233,7 +1211,8 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
{
Relids subrelids;
- subrelids = get_relids_in_jointree((Node *) subquery->jointree, false);
+ subrelids = get_relids_in_jointree((Node *) subquery->jointree,
+ true, false);
substitute_phv_relids((Node *) parse, varno, subrelids);
fix_append_rel_relids(root->append_rel_list, varno, subrelids);
}
@@ -1424,7 +1403,7 @@ pull_up_union_leaf_queries(Node *setOp, PlannerInfo *root, int parentRTindex,
rtr = makeNode(RangeTblRef);
rtr->rtindex = childRTindex;
(void) pull_up_subqueries_recurse(root, (Node *) rtr,
- NULL, NULL, appinfo);
+ NULL, appinfo);
}
else if (IsA(setOp, SetOperationStmt))
{
@@ -1561,7 +1540,7 @@ is_simple_subquery(PlannerInfo *root, Query *subquery, RangeTblEntry *rte,
{
restricted = true;
safe_upper_varnos = get_relids_in_jointree((Node *) lowest_outer_join,
- true);
+ true, true);
}
else
{
@@ -1673,7 +1652,6 @@ pull_up_simple_values(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte)
rvcontext.relids = NULL;
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = varno;
- rvcontext.need_phvs = false;
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(tlist) + 1) *
@@ -1685,7 +1663,7 @@ pull_up_simple_values(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte)
* any of the jointree structure. We can assume there's no outer joins or
* appendrels in the dummy Query that surrounds a VALUES RTE.
*/
- perform_pullup_replace_vars(root, &rvcontext, NULL, NULL);
+ perform_pullup_replace_vars(root, &rvcontext, NULL);
/*
* There should be no appendrels to fix, nor any outer joins and hence no
@@ -1784,7 +1762,6 @@ is_simple_values(PlannerInfo *root, RangeTblEntry *rte)
static Node *
pull_up_constant_function(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -1836,40 +1813,26 @@ pull_up_constant_function(PlannerInfo *root, Node *jtnode,
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = ((RangeTblRef *) jtnode)->rtindex;
- /* these flags will be set below, if needed */
- rvcontext.need_phvs = false;
+ /* this flag will be set below, if needed */
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(rvcontext.targetlist) + 1) *
sizeof(Node *));
- /*
- * If we are under an outer join then non-nullable items and lateral
- * references may have to be turned into PlaceHolderVars.
- */
- if (lowest_nulling_outer_join != NULL)
- rvcontext.need_phvs = true;
-
/*
* If we are dealing with an appendrel member then anything that's not a
* simple Var has to be turned into a PlaceHolderVar. (See comments in
* pull_up_simple_subquery().)
*/
if (containing_appendrel != NULL)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* If the parent query uses grouping sets, we need a PlaceHolderVar for
* anything that's not a simple Var.
*/
if (parse->groupingSets)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* Replace all of the top query's references to the RTE's output with
@@ -1877,7 +1840,6 @@ pull_up_constant_function(PlannerInfo *root, Node *jtnode,
* jointree structure.
*/
perform_pullup_replace_vars(root, &rvcontext,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -2099,13 +2061,11 @@ jointree_contains_lateral_outer_refs(PlannerInfo *root, Node *jtnode,
*
* Caller has already filled *rvcontext with data describing what to
* substitute for Vars referencing the target subquery. In addition
- * we need the identity of the lowest outer join that can null the
- * target subquery, and its containing appendrel if any.
+ * we need the identity of the containing appendrel if any.
*/
static void
perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars_context *rvcontext,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -2149,38 +2109,31 @@ perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars((Node *) action->targetList, rvcontext);
}
}
- replace_vars_in_jointree((Node *) parse->jointree, rvcontext,
- lowest_nulling_outer_join);
+ replace_vars_in_jointree((Node *) parse->jointree, rvcontext);
Assert(parse->setOperations == NULL);
parse->havingQual = pullup_replace_vars(parse->havingQual, rvcontext);
/*
* Replace references in the translated_vars lists of appendrels. When
- * pulling up an appendrel member, we do not need PHVs in the list of the
- * parent appendrel --- there isn't any outer join between. Elsewhere,
- * use PHVs for safety. (This analysis could be made tighter but it seems
- * unlikely to be worth much trouble.)
+ * pulling up an appendrel member, we do not want to force PHVs in the
+ * list of the parent appendrel --- there isn't any outer join between.
+ * Elsewhere, use PHVs for safety. (This analysis could be made tighter
+ * but it seems unlikely to be worth much trouble.)
*/
foreach(lc, root->append_rel_list)
{
AppendRelInfo *appinfo = (AppendRelInfo *) lfirst(lc);
- bool save_need_phvs = rvcontext->need_phvs;
+ bool save_wrap_non_vars = rvcontext->wrap_non_vars;
if (appinfo == containing_appendrel)
- rvcontext->need_phvs = false;
+ rvcontext->wrap_non_vars = false;
appinfo->translated_vars = (List *)
pullup_replace_vars((Node *) appinfo->translated_vars, rvcontext);
- rvcontext->need_phvs = save_need_phvs;
+ rvcontext->wrap_non_vars = save_wrap_non_vars;
}
/*
* Replace references in the joinaliasvars lists of join RTEs.
- *
- * You might think that we could avoid using PHVs for alias vars of joins
- * below lowest_nulling_outer_join, but that doesn't work because the
- * alias vars could be referenced above that join; we need the PHVs to be
- * present in such references after the alias vars get flattened. (It
- * might be worth trying to be smarter here, someday.)
*/
foreach(lc, parse->rtable)
{
@@ -2197,14 +2150,10 @@ perform_pullup_replace_vars(PlannerInfo *root,
* Helper routine for perform_pullup_replace_vars: do pullup_replace_vars on
* every expression in the jointree, without changing the jointree structure
* itself. Ugly, but there's no other way...
- *
- * If we are at or below lowest_nulling_outer_join, we can suppress use of
- * PlaceHolderVars wrapped around the replacement expressions.
*/
static void
replace_vars_in_jointree(Node *jtnode,
- pullup_replace_vars_context *context,
- JoinExpr *lowest_nulling_outer_join)
+ pullup_replace_vars_context *context)
{
if (jtnode == NULL)
return;
@@ -2214,10 +2163,8 @@ replace_vars_in_jointree(Node *jtnode,
* If the RangeTblRef refers to a LATERAL subquery (that isn't the
* same subquery we're pulling up), it might contain references to the
* target subquery, which we must replace. We drive this from the
- * jointree scan, rather than a scan of the rtable, for a couple of
- * reasons: we can avoid processing no-longer-referenced RTEs, and we
- * can use the appropriate setting of need_phvs depending on whether
- * the RTE is above possibly-nulling outer joins or not.
+ * jointree scan, rather than a scan of the rtable, so that we can
+ * avoid processing no-longer-referenced RTEs.
*/
int varno = ((RangeTblRef *) jtnode)->rtindex;
@@ -2274,42 +2221,30 @@ replace_vars_in_jointree(Node *jtnode,
ListCell *l;
foreach(l, f->fromlist)
- replace_vars_in_jointree(lfirst(l), context,
- lowest_nulling_outer_join);
+ replace_vars_in_jointree(lfirst(l), context);
f->quals = pullup_replace_vars(f->quals, context);
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- bool save_need_phvs = context->need_phvs;
+ bool save_wrap_non_vars = context->wrap_non_vars;
- if (j == lowest_nulling_outer_join)
- {
- /* no more PHVs in or below this join */
- context->need_phvs = false;
- lowest_nulling_outer_join = NULL;
- }
- replace_vars_in_jointree(j->larg, context, lowest_nulling_outer_join);
- replace_vars_in_jointree(j->rarg, context, lowest_nulling_outer_join);
+ replace_vars_in_jointree(j->larg, context);
+ replace_vars_in_jointree(j->rarg, context);
/*
- * Use PHVs within the join quals of a full join, even when it's the
- * lowest nulling outer join. Otherwise, we cannot identify which
- * side of the join a pulled-up var-free expression came from, which
- * can lead to failure to make a plan at all because none of the quals
- * appear to be mergeable or hashable conditions. For this purpose we
- * don't care about the state of wrap_non_vars, so leave it alone.
+ * Use PHVs within the join quals of a full join. Otherwise, we
+ * cannot identify which side of the join a pulled-up var-free
+ * expression came from, which can lead to failure to make a plan at
+ * all because none of the quals appear to be mergeable or hashable
+ * conditions.
*/
if (j->jointype == JOIN_FULL)
- context->need_phvs = true;
+ context->wrap_non_vars = true;
j->quals = pullup_replace_vars(j->quals, context);
- /*
- * We don't bother to update the colvars list, since it won't be used
- * again ...
- */
- context->need_phvs = save_need_phvs;
+ context->wrap_non_vars = save_wrap_non_vars;
}
else
elog(ERROR, "unrecognized node type: %d",
@@ -2338,8 +2273,18 @@ pullup_replace_vars_callback(Var *var,
{
pullup_replace_vars_context *rcon = (pullup_replace_vars_context *) context->callback_arg;
int varattno = var->varattno;
+ bool need_phv;
Node *newnode;
+ /*
+ * We need a PlaceHolderVar if the Var-to-be-replaced has nonempty
+ * varnullingrels (unless we find below that the replacement expression is
+ * a Var or PlaceHolderVar that we can just add the nullingrels to). We
+ * also need one if the caller has instructed us that all non-Var/PHV
+ * replacements need to be wrapped for identification purposes.
+ */
+ need_phv = (var->varnullingrels != NULL) || rcon->wrap_non_vars;
+
/*
* If PlaceHolderVars are needed, we cache the modified expressions in
* rcon->rv_cache[]. This is not in hopes of any material speed gain
@@ -2348,13 +2293,16 @@ pullup_replace_vars_callback(Var *var,
* and possibly prevent optimizations that rely on recognizing different
* references to the same subquery output as being equal(). So it's worth
* a bit of extra effort to avoid it.
+ *
+ * The cached items have phlevelsup = 0 and phnullingrels = NULL; we'll
+ * copy them and adjust those values for this reference site below.
*/
- if (rcon->need_phvs &&
+ if (need_phv &&
varattno >= InvalidAttrNumber &&
varattno <= list_length(rcon->targetlist) &&
rcon->rv_cache[varattno] != NULL)
{
- /* Just copy the entry and fall through to adjust its varlevelsup */
+ /* Just copy the entry and fall through to adjust phlevelsup etc */
newnode = copyObject(rcon->rv_cache[varattno]);
}
else if (varattno == InvalidAttrNumber)
@@ -2363,7 +2311,7 @@ pullup_replace_vars_callback(Var *var,
RowExpr *rowexpr;
List *colnames;
List *fields;
- bool save_need_phvs = rcon->need_phvs;
+ bool save_wrap_non_vars = rcon->wrap_non_vars;
int save_sublevelsup = context->sublevels_up;
/*
@@ -2374,18 +2322,18 @@ pullup_replace_vars_callback(Var *var,
* the RowExpr for use of the executor and ruleutils.c.
*
* In order to be able to cache the results, we always generate the
- * expansion with varlevelsup = 0, and then adjust if needed.
+ * expansion with varlevelsup = 0, and then adjust below if needed.
*/
expandRTE(rcon->target_rte,
var->varno, 0 /* not varlevelsup */ , var->location,
(var->vartype != RECORDOID),
&colnames, &fields);
- /* Adjust the generated per-field Vars, but don't insert PHVs */
- rcon->need_phvs = false;
+ /* Expand the generated per-field Vars, but don't insert PHVs there */
+ rcon->wrap_non_vars = false;
context->sublevels_up = 0; /* to match the expandRTE output */
fields = (List *) replace_rte_variables_mutator((Node *) fields,
context);
- rcon->need_phvs = save_need_phvs;
+ rcon->wrap_non_vars = save_wrap_non_vars;
context->sublevels_up = save_sublevelsup;
rowexpr = makeNode(RowExpr);
@@ -2403,14 +2351,13 @@ pullup_replace_vars_callback(Var *var,
* expression to yield NULL, not ROW(NULL,NULL,...) when it is forced
* to null by an outer join.
*/
- if (rcon->need_phvs)
+ if (need_phv)
{
- /* RowExpr is certainly not strict, so always need PHV */
newnode = (Node *)
make_placeholder_expr(rcon->root,
(Expr *) newnode,
bms_make_singleton(rcon->varno));
- /* cache it with the PHV, and with varlevelsup still zero */
+ /* cache it with the PHV, and with phlevelsup etc not set yet */
rcon->rv_cache[InvalidAttrNumber] = copyObject(newnode);
}
}
@@ -2427,7 +2374,7 @@ pullup_replace_vars_callback(Var *var,
newnode = (Node *) copyObject(tle->expr);
/* Insert PlaceHolderVar if needed */
- if (rcon->need_phvs)
+ if (need_phv)
{
bool wrap;
@@ -2453,69 +2400,61 @@ pullup_replace_vars_callback(Var *var,
/* No need to wrap a PlaceHolderVar with another one, either */
wrap = false;
}
- else if (rcon->wrap_non_vars)
- {
- /* Wrap all non-Vars in a PlaceHolderVar */
- wrap = true;
- }
else
{
/*
- * If it contains a Var of the subquery being pulled up, and
- * does not contain any non-strict constructs, then it's
- * certainly nullable so we don't need to insert a
- * PlaceHolderVar.
- *
- * This analysis could be tighter: in particular, a non-strict
- * construct hidden within a lower-level PlaceHolderVar is not
- * reason to add another PHV. But for now it doesn't seem
- * worth the code to be more exact.
- *
- * Note: in future maybe we should insert a PlaceHolderVar
- * anyway, if the tlist item is expensive to evaluate?
- *
- * For a LATERAL subquery, we have to check the actual var
- * membership of the node, but if it's non-lateral then any
- * level-zero var must belong to the subquery.
+ * Must wrap, either because we need a place to insert
+ * varnullingrels or because caller told us to wrap
+ * everything.
*/
- if ((rcon->target_rte->lateral ?
- bms_overlap(pull_varnos(rcon->root, (Node *) newnode),
- rcon->relids) :
- contain_vars_of_level((Node *) newnode, 0)) &&
- !contain_nonstrict_functions((Node *) newnode))
- {
- /* No wrap needed */
- wrap = false;
- }
- else
- {
- /* Else wrap it in a PlaceHolderVar */
- wrap = true;
- }
+ wrap = true;
}
if (wrap)
+ {
newnode = (Node *)
make_placeholder_expr(rcon->root,
(Expr *) newnode,
bms_make_singleton(rcon->varno));
- /*
- * Cache it if possible (ie, if the attno is in range, which it
- * probably always should be). We can cache the value even if we
- * decided we didn't need a PHV, since this result will be
- * suitable for any request that has need_phvs.
- */
- if (varattno > InvalidAttrNumber &&
- varattno <= list_length(rcon->targetlist))
- rcon->rv_cache[varattno] = copyObject(newnode);
+ /*
+ * Cache it if possible (ie, if the attno is in range, which
+ * it probably always should be).
+ */
+ if (varattno > InvalidAttrNumber &&
+ varattno <= list_length(rcon->targetlist))
+ rcon->rv_cache[varattno] = copyObject(newnode);
+ }
}
}
- /* Must adjust varlevelsup if tlist item is from higher query */
+ /* Must adjust varlevelsup if replaced Var is within a subquery */
if (var->varlevelsup > 0)
IncrementVarSublevelsUp(newnode, var->varlevelsup, 0);
+ /* Propagate any varnullingrels into the replacement Var or PHV */
+ if (var->varnullingrels != NULL)
+ {
+ if (IsA(newnode, Var))
+ {
+ Var *newvar = (Var *) newnode;
+
+ Assert(newvar->varlevelsup == var->varlevelsup);
+ newvar->varnullingrels = bms_add_members(newvar->varnullingrels,
+ var->varnullingrels);
+ }
+ else if (IsA(newnode, PlaceHolderVar))
+ {
+ PlaceHolderVar *newphv = (PlaceHolderVar *) newnode;
+
+ Assert(newphv->phlevelsup == var->varlevelsup);
+ newphv->phnullingrels = bms_add_members(newphv->phnullingrels,
+ var->varnullingrels);
+ }
+ else
+ elog(ERROR, "failed to wrap a non-Var");
+ }
+
return newnode;
}
@@ -2674,7 +2613,9 @@ flatten_simple_union_all(PlannerInfo *root)
void
reduce_outer_joins(PlannerInfo *root)
{
- reduce_outer_joins_state *state;
+ reduce_outer_joins_pass1_state *state1;
+ reduce_outer_joins_pass2_state state2;
+ ListCell *lc;
/*
* To avoid doing strictness checks on more quals than necessary, we want
@@ -2685,14 +2626,56 @@ reduce_outer_joins(PlannerInfo *root)
* join(s) below each side of each join clause. The second pass examines
* qual clauses and changes join types as it descends the tree.
*/
- state = reduce_outer_joins_pass1((Node *) root->parse->jointree);
+ state1 = reduce_outer_joins_pass1((Node *) root->parse->jointree);
/* planner.c shouldn't have called me if no outer joins */
- if (state == NULL || !state->contains_outer)
+ if (state1 == NULL || !state1->contains_outer)
elog(ERROR, "so where are the outer joins?");
+ state2.inner_reduced = NULL;
+ state2.partial_reduced = NIL;
+
reduce_outer_joins_pass2((Node *) root->parse->jointree,
- state, root, NULL, NIL);
+ state1, &state2,
+ root, NULL, NIL);
+
+ /*
+ * If we successfully reduced the strength of any outer joins, we must
+ * remove references to those joins as nulling rels. This is handled as
+ * an additional pass, for simplicity and because we can handle all
+ * fully-reduced joins in a single pass over the parse tree.
+ */
+ if (!bms_is_empty(state2.inner_reduced))
+ {
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ state2.inner_reduced,
+ NULL);
+ /* There could be references in the append_rel_list, too */
+ root->append_rel_list = (List *)
+ remove_nulling_relids((Node *) root->append_rel_list,
+ state2.inner_reduced,
+ NULL);
+ }
+
+ /*
+ * Partially-reduced full joins have to be done one at a time, since
+ * they'll each need a different setting of except_relids.
+ */
+ foreach(lc, state2.partial_reduced)
+ {
+ reduce_outer_joins_partial_state *statep = lfirst(lc);
+ Relids full_join_relids = bms_make_singleton(statep->full_join_rti);
+
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ full_join_relids,
+ statep->unreduced_side);
+ root->append_rel_list = (List *)
+ remove_nulling_relids((Node *) root->append_rel_list,
+ full_join_relids,
+ statep->unreduced_side);
+ }
}
/*
@@ -2700,13 +2683,13 @@ reduce_outer_joins(PlannerInfo *root)
*
* Returns a state node describing the given jointree node.
*/
-static reduce_outer_joins_state *
+static reduce_outer_joins_pass1_state *
reduce_outer_joins_pass1(Node *jtnode)
{
- reduce_outer_joins_state *result;
+ reduce_outer_joins_pass1_state *result;
- result = (reduce_outer_joins_state *)
- palloc(sizeof(reduce_outer_joins_state));
+ result = (reduce_outer_joins_pass1_state *)
+ palloc(sizeof(reduce_outer_joins_pass1_state));
result->relids = NULL;
result->contains_outer = false;
result->sub_states = NIL;
@@ -2726,7 +2709,7 @@ reduce_outer_joins_pass1(Node *jtnode)
foreach(l, f->fromlist)
{
- reduce_outer_joins_state *sub_state;
+ reduce_outer_joins_pass1_state *sub_state;
sub_state = reduce_outer_joins_pass1(lfirst(l));
result->relids = bms_add_members(result->relids,
@@ -2738,7 +2721,7 @@ reduce_outer_joins_pass1(Node *jtnode)
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- reduce_outer_joins_state *sub_state;
+ reduce_outer_joins_pass1_state *sub_state;
/* join's own RT index is not wanted in result->relids */
if (IS_OUTER_JOIN(j->jointype))
@@ -2766,14 +2749,22 @@ reduce_outer_joins_pass1(Node *jtnode)
* reduce_outer_joins_pass2 - phase 2 processing
*
* jtnode: current jointree node
- * state: state data collected by phase 1 for this node
+ * state1: state data collected by phase 1 for this node
+ * state2: where to accumulate info about successfully-reduced joins
* root: toplevel planner state
* nonnullable_rels: set of base relids forced non-null by upper quals
* forced_null_vars: list of Vars forced null by upper quals
+ *
+ * Returns info in state2 about outer joins that were successfully simplified.
+ * Joins that were fully reduced to inner joins are all added to
+ * state2->inner_reduced. If a full join is reduced to a left join,
+ * it needs its own entry in state2->partial_reduced, since that will
+ * require custom processing to remove only the correct nullingrel markers.
*/
static void
reduce_outer_joins_pass2(Node *jtnode,
- reduce_outer_joins_state *state,
+ reduce_outer_joins_pass1_state *state1,
+ reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
List *forced_null_vars)
@@ -2802,13 +2793,14 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_forced_null_vars = list_concat(pass_forced_null_vars,
forced_null_vars);
/* And recurse --- but only into interesting subtrees */
- Assert(list_length(f->fromlist) == list_length(state->sub_states));
- forboth(l, f->fromlist, s, state->sub_states)
+ Assert(list_length(f->fromlist) == list_length(state1->sub_states));
+ forboth(l, f->fromlist, s, state1->sub_states)
{
- reduce_outer_joins_state *sub_state = lfirst(s);
+ reduce_outer_joins_pass1_state *sub_state = lfirst(s);
if (sub_state->contains_outer)
- reduce_outer_joins_pass2(lfirst(l), sub_state, root,
+ reduce_outer_joins_pass2(lfirst(l), sub_state,
+ state2, root,
pass_nonnullable_rels,
pass_forced_null_vars);
}
@@ -2820,8 +2812,8 @@ reduce_outer_joins_pass2(Node *jtnode,
JoinExpr *j = (JoinExpr *) jtnode;
int rtindex = j->rtindex;
JoinType jointype = j->jointype;
- reduce_outer_joins_state *left_state = linitial(state->sub_states);
- reduce_outer_joins_state *right_state = lsecond(state->sub_states);
+ reduce_outer_joins_pass1_state *left_state = linitial(state1->sub_states);
+ reduce_outer_joins_pass1_state *right_state = lsecond(state1->sub_states);
/* Can we simplify this join? */
switch (jointype)
@@ -2842,12 +2834,22 @@ reduce_outer_joins_pass2(Node *jtnode,
if (bms_overlap(nonnullable_rels, right_state->relids))
jointype = JOIN_INNER;
else
+ {
jointype = JOIN_LEFT;
+ /* Also report partial reduction in state2 */
+ report_reduced_full_join(state2, rtindex,
+ right_state->relids);
+ }
}
else
{
if (bms_overlap(nonnullable_rels, right_state->relids))
+ {
jointype = JOIN_RIGHT;
+ /* Also report partial reduction in state2 */
+ report_reduced_full_join(state2, rtindex,
+ left_state->relids);
+ }
}
break;
case JOIN_SEMI:
@@ -2880,8 +2882,8 @@ reduce_outer_joins_pass2(Node *jtnode,
j->larg = j->rarg;
j->rarg = tmparg;
jointype = JOIN_LEFT;
- right_state = linitial(state->sub_states);
- left_state = lsecond(state->sub_states);
+ right_state = linitial(state1->sub_states);
+ left_state = lsecond(state1->sub_states);
}
/*
@@ -2915,7 +2917,10 @@ reduce_outer_joins_pass2(Node *jtnode,
jointype = JOIN_ANTI;
}
- /* Apply the jointype change, if any, to both jointree node and RTE */
+ /*
+ * Apply the jointype change, if any, to both jointree node and RTE.
+ * Also, if we changed an RTE to INNER, add its RTI to inner_reduced.
+ */
if (rtindex && jointype != j->jointype)
{
RangeTblEntry *rte = rt_fetch(rtindex, root->parse->rtable);
@@ -2923,6 +2928,9 @@ reduce_outer_joins_pass2(Node *jtnode,
Assert(rte->rtekind == RTE_JOIN);
Assert(rte->jointype == j->jointype);
rte->jointype = jointype;
+ if (jointype == JOIN_INNER)
+ state2->inner_reduced = bms_add_member(state2->inner_reduced,
+ rtindex);
}
j->jointype = jointype;
@@ -2995,7 +3003,8 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_nonnullable_rels = NULL;
pass_forced_null_vars = NIL;
}
- reduce_outer_joins_pass2(j->larg, left_state, root,
+ reduce_outer_joins_pass2(j->larg, left_state,
+ state2, root,
pass_nonnullable_rels,
pass_forced_null_vars);
}
@@ -3014,7 +3023,8 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_nonnullable_rels = NULL;
pass_forced_null_vars = NIL;
}
- reduce_outer_joins_pass2(j->rarg, right_state, root,
+ reduce_outer_joins_pass2(j->rarg, right_state,
+ state2, root,
pass_nonnullable_rels,
pass_forced_null_vars);
}
@@ -3026,6 +3036,19 @@ reduce_outer_joins_pass2(Node *jtnode,
(int) nodeTag(jtnode));
}
+/* Helper for reduce_outer_joins_pass2 */
+static void
+report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
+ int rtindex, Relids relids)
+{
+ reduce_outer_joins_partial_state *statep;
+
+ statep = palloc(sizeof(reduce_outer_joins_partial_state));
+ statep->full_join_rti = rtindex;
+ statep->unreduced_side = relids;
+ state2->partial_reduced = lappend(state2->partial_reduced, statep);
+}
+
/*
* remove_useless_result_rtes
@@ -3067,16 +3090,41 @@ reduce_outer_joins_pass2(Node *jtnode,
void
remove_useless_result_rtes(PlannerInfo *root)
{
+ Relids dropped_outer_joins = NULL;
ListCell *cell;
/* Top level of jointree must always be a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
/* Recurse ... */
root->parse->jointree = (FromExpr *)
- remove_useless_results_recurse(root, (Node *) root->parse->jointree);
+ remove_useless_results_recurse(root,
+ (Node *) root->parse->jointree,
+ &dropped_outer_joins);
/* We should still have a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
+ /*
+ * If we removed any outer-join nodes from the jointree, run around and
+ * remove references to those joins as nulling rels. (There could be such
+ * references in PHVs that we pulled up out of the original subquery that
+ * the RESULT rel replaced. This is kosher on the grounds that we now
+ * know that such an outer join wouldn't really have nulled anything.) We
+ * don't do this during the main recursion, for simplicity and because we
+ * can handle all such joins in a single pass over the parse tree.
+ */
+ if (!bms_is_empty(dropped_outer_joins))
+ {
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ dropped_outer_joins,
+ NULL);
+ /* There could be references in the append_rel_list, too */
+ root->append_rel_list = (List *)
+ remove_nulling_relids((Node *) root->append_rel_list,
+ dropped_outer_joins,
+ NULL);
+ }
+
/*
* Remove any PlanRowMark referencing an RTE_RESULT RTE. We obviously
* must do that for any RTE_RESULT that we just removed. But one for a
@@ -3102,9 +3150,12 @@ remove_useless_result_rtes(PlannerInfo *root)
* Recursive guts of remove_useless_result_rtes.
*
* This recursively processes the jointree and returns a modified jointree.
+ * In addition, the RT indexes of any removed outer-join nodes are added to
+ * *dropped_outer_joins.
*/
static Node *
-remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
+remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Relids *dropped_outer_joins)
{
Assert(jtnode != NULL);
if (IsA(jtnode, RangeTblRef))
@@ -3132,7 +3183,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
int varno;
/* Recursively transform child ... */
- child = remove_useless_results_recurse(root, child);
+ child = remove_useless_results_recurse(root, child,
+ dropped_outer_joins);
/* ... and stick it back into the tree */
lfirst(cell) = child;
@@ -3181,8 +3233,10 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
int varno;
/* First, recurse */
- j->larg = remove_useless_results_recurse(root, j->larg);
- j->rarg = remove_useless_results_recurse(root, j->rarg);
+ j->larg = remove_useless_results_recurse(root, j->larg,
+ dropped_outer_joins);
+ j->rarg = remove_useless_results_recurse(root, j->rarg,
+ dropped_outer_joins);
/* Apply join-type-specific optimization rules */
switch (j->jointype)
@@ -3250,6 +3304,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
!find_dependent_phvs(root, varno)))
{
remove_result_refs(root, varno, j->larg);
+ *dropped_outer_joins = bms_add_member(*dropped_outer_joins,
+ j->rtindex);
jtnode = j->larg;
}
break;
@@ -3260,6 +3316,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
!find_dependent_phvs(root, varno)))
{
remove_result_refs(root, varno, j->rarg);
+ *dropped_outer_joins = bms_add_member(*dropped_outer_joins,
+ j->rtindex);
jtnode = j->rarg;
}
break;
@@ -3274,11 +3332,14 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
* Unlike the LEFT/RIGHT cases, we just Assert that there are
* no PHVs that need to be evaluated at the semijoin's RHS,
* since the rest of the query couldn't reference any outputs
- * of the semijoin's RHS.
+ * of the semijoin's RHS. Also, we don't need to worry about
+ * removing traces of the join's rtindex, since it hasn't got
+ * one.
*/
if ((varno = get_result_relid(root, j->rarg)) != 0)
{
Assert(!find_dependent_phvs(root, varno));
+ Assert(j->rtindex == 0);
remove_result_refs(root, varno, j->larg);
if (j->quals)
jtnode = (Node *)
@@ -3347,7 +3408,7 @@ remove_result_refs(PlannerInfo *root, int varno, Node *newjtloc)
{
Relids subrelids;
- subrelids = get_relids_in_jointree(newjtloc, false);
+ subrelids = get_relids_in_jointree(newjtloc, true, false);
Assert(!bms_is_empty(subrelids));
substitute_phv_relids((Node *) root->parse, varno, subrelids);
fix_append_rel_relids(root->append_rel_list, varno, subrelids);
@@ -3404,9 +3465,8 @@ find_dependent_phvs_walker(Node *node,
context->sublevels_up--;
return result;
}
- /* Shouldn't need to handle planner auxiliary nodes here */
+ /* Shouldn't need to handle most planner auxiliary nodes here */
Assert(!IsA(node, SpecialJoinInfo));
- Assert(!IsA(node, AppendRelInfo));
Assert(!IsA(node, PlaceHolderInfo));
Assert(!IsA(node, MinMaxAggInfo));
@@ -3426,10 +3486,17 @@ find_dependent_phvs(PlannerInfo *root, int varno)
context.relids = bms_make_singleton(varno);
context.sublevels_up = 0;
- return query_tree_walker(root->parse,
- find_dependent_phvs_walker,
- (void *) &context,
- 0);
+ if (query_tree_walker(root->parse,
+ find_dependent_phvs_walker,
+ (void *) &context,
+ 0))
+ return true;
+ /* The append_rel_list could be populated already, so check it too */
+ if (expression_tree_walker((Node *) root->append_rel_list,
+ find_dependent_phvs_walker,
+ (void *) &context))
+ return true;
+ return false;
}
static bool
@@ -3459,7 +3526,7 @@ find_dependent_phvs_in_jointree(PlannerInfo *root, Node *node, int varno)
* are not marked LATERAL, though, since they couldn't possibly contain
* any cross-references to other RTEs.
*/
- subrelids = get_relids_in_jointree(node, false);
+ subrelids = get_relids_in_jointree(node, false, false);
relid = -1;
while ((relid = bms_next_member(subrelids, relid)) >= 0)
{
@@ -3603,11 +3670,17 @@ fix_append_rel_relids(List *append_rel_list, int varno, Relids subrelids)
/*
* get_relids_in_jointree: get set of RT indexes present in a jointree
*
- * If include_joins is true, join RT indexes are included; if false,
- * only base rels are included.
+ * Base-relation relids are always included in the result.
+ * If include_outer_joins is true, outer-join RT indexes are included.
+ * If include_inner_joins is true, inner-join RT indexes are included.
+ *
+ * Note that for most purposes in the planner, outer joins are included
+ * in standard relid sets. Setting include_inner_joins true is only
+ * appropriate for special purposes during subquery flattening.
*/
Relids
-get_relids_in_jointree(Node *jtnode, bool include_joins)
+get_relids_in_jointree(Node *jtnode, bool include_outer_joins,
+ bool include_inner_joins)
{
Relids result = NULL;
@@ -3628,18 +3701,34 @@ get_relids_in_jointree(Node *jtnode, bool include_joins)
{
result = bms_join(result,
get_relids_in_jointree(lfirst(l),
- include_joins));
+ include_outer_joins,
+ include_inner_joins));
}
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- result = get_relids_in_jointree(j->larg, include_joins);
+ result = get_relids_in_jointree(j->larg,
+ include_outer_joins,
+ include_inner_joins);
result = bms_join(result,
- get_relids_in_jointree(j->rarg, include_joins));
- if (include_joins && j->rtindex)
- result = bms_add_member(result, j->rtindex);
+ get_relids_in_jointree(j->rarg,
+ include_outer_joins,
+ include_inner_joins));
+ if (j->rtindex)
+ {
+ if (j->jointype == JOIN_INNER)
+ {
+ if (include_inner_joins)
+ result = bms_add_member(result, j->rtindex);
+ }
+ else
+ {
+ if (include_outer_joins)
+ result = bms_add_member(result, j->rtindex);
+ }
+ }
}
else
elog(ERROR, "unrecognized node type: %d",
@@ -3648,7 +3737,7 @@ get_relids_in_jointree(Node *jtnode, bool include_joins)
}
/*
- * get_relids_for_join: get set of base RT indexes making up a join
+ * get_relids_for_join: get set of base+OJ RT indexes making up a join
*/
Relids
get_relids_for_join(Query *query, int joinrelid)
@@ -3659,7 +3748,7 @@ get_relids_for_join(Query *query, int joinrelid)
joinrelid);
if (!jtnode)
elog(ERROR, "could not find join node %d", joinrelid);
- return get_relids_in_jointree(jtnode, false);
+ return get_relids_in_jointree(jtnode, true, false);
}
/*
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index f6fc62aa5d..11c6bbaba6 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -228,6 +228,14 @@ adjust_appendrel_attrs_mutator(Node *node,
if (var->varlevelsup != 0)
return (Node *) var; /* no changes needed */
+ /*
+ * You might think we need to adjust var->varnullingrels, but that
+ * shouldn't need any changes. It will contain outer-join relids,
+ * while the transformation we are making affects only baserels.
+ * Below, we just propagate var->varnullingrels into the translated
+ * Var. (XXX what to do if translation is not a Var??)
+ */
+
for (cnt = 0; cnt < nappinfos; cnt++)
{
if (var->varno == appinfos[cnt]->parent_relid)
@@ -255,6 +263,8 @@ adjust_appendrel_attrs_mutator(Node *node,
if (newnode == NULL)
elog(ERROR, "attribute %d of relation \"%s\" does not exist",
var->varattno, get_rel_name(appinfo->parent_reloid));
+ if (IsA(newnode, Var))
+ ((Var *) newnode)->varnullingrels = var->varnullingrels;
return newnode;
}
else if (var->varattno == 0)
@@ -348,6 +358,8 @@ adjust_appendrel_attrs_mutator(Node *node,
var = copyObject(ridinfo->rowidvar);
/* ... but use the correct relid */
var->varno = leaf_relid;
+ /* identity vars shouldn't have nulling rels */
+ Assert(var->varnullingrels == NULL);
/* varnosyn in the RowIdentityVarInfo is probably wrong */
var->varnosyn = 0;
var->varattnosyn = 0;
@@ -392,8 +404,11 @@ adjust_appendrel_attrs_mutator(Node *node,
(void *) context);
/* now fix PlaceHolderVar's relid sets */
if (phv->phlevelsup == 0)
- phv->phrels = adjust_child_relids(phv->phrels, context->nappinfos,
- context->appinfos);
+ {
+ phv->phrels = adjust_child_relids(phv->phrels,
+ nappinfos, appinfos);
+ /* as above, we needn't touch phnullingrels */
+ }
return (Node *) phv;
}
/* Shouldn't need to handle planner auxiliary nodes here */
@@ -688,7 +703,11 @@ get_translated_update_targetlist(PlannerInfo *root, Index relid,
/*
* find_appinfos_by_relids
- * Find AppendRelInfo structures for all relations specified by relids.
+ * Find AppendRelInfo structures for base relations listed in relids.
+ *
+ * The relids argument is typically a join relation's relids, which can
+ * include outer-join RT indexes in addition to baserels. We silently
+ * ignore the outer joins.
*
* The AppendRelInfos are returned in an array, which can be pfree'd by the
* caller. *nappinfos is set to the number of entries in the array.
@@ -700,8 +719,9 @@ find_appinfos_by_relids(PlannerInfo *root, Relids relids, int *nappinfos)
int cnt = 0;
int i;
- *nappinfos = bms_num_members(relids);
- appinfos = (AppendRelInfo **) palloc(sizeof(AppendRelInfo *) * *nappinfos);
+ /* Allocate an array that's certainly big enough */
+ appinfos = (AppendRelInfo **)
+ palloc(sizeof(AppendRelInfo *) * bms_num_members(relids));
i = -1;
while ((i = bms_next_member(relids, i)) >= 0)
@@ -709,10 +729,17 @@ find_appinfos_by_relids(PlannerInfo *root, Relids relids, int *nappinfos)
AppendRelInfo *appinfo = root->append_rel_array[i];
if (!appinfo)
+ {
+ /* Probably i is an OJ index, but let's check */
+ if (find_base_rel_ignore_join(root, i) == NULL)
+ continue;
+ /* It's a base rel, but we lack an append_rel_array entry */
elog(ERROR, "child rel %d not found in append_rel_array", i);
+ }
appinfos[cnt++] = appinfo;
}
+ *nappinfos = cnt;
return appinfos;
}
@@ -754,6 +781,7 @@ add_row_identity_var(PlannerInfo *root, Var *orig_var,
Assert(IsA(orig_var, Var));
Assert(orig_var->varno == rtindex);
Assert(orig_var->varlevelsup == 0);
+ Assert(orig_var->varnullingrels == NULL);
/*
* If we're doing non-inherited UPDATE/DELETE/MERGE, there's little need
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 5e791333cb..8c80f5a1a6 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -2001,14 +2001,16 @@ is_pseudo_constant_clause_relids(Node *clause, Relids relids)
* NumRelids
* (formerly clause_relids)
*
- * Returns the number of different relations referenced in 'clause'.
+ * Returns the number of different base relations referenced in 'clause'.
*/
int
NumRelids(PlannerInfo *root, Node *clause)
{
+ int result;
Relids varnos = pull_varnos(root, clause);
- int result = bms_num_members(varnos);
+ varnos = bms_del_members(varnos, root->outer_join_rels);
+ result = bms_num_members(varnos);
bms_free(varnos);
return result;
}
diff --git a/src/backend/optimizer/util/joininfo.c b/src/backend/optimizer/util/joininfo.c
index d4cffdb198..afd243f5d8 100644
--- a/src/backend/optimizer/util/joininfo.c
+++ b/src/backend/optimizer/util/joininfo.c
@@ -88,8 +88,8 @@ have_relevant_joinclause(PlannerInfo *root,
* not depend on context).
*
* 'restrictinfo' describes the join clause
- * 'join_relids' is the list of relations participating in the join clause
- * (there must be more than one)
+ * 'join_relids' is the set of relations participating in the join clause
+ * (some of these could be outer joins)
*/
void
add_join_clause_to_rels(PlannerInfo *root,
@@ -101,8 +101,11 @@ add_join_clause_to_rels(PlannerInfo *root,
cur_relid = -1;
while ((cur_relid = bms_next_member(join_relids, cur_relid)) >= 0)
{
- RelOptInfo *rel = find_base_rel(root, cur_relid);
+ RelOptInfo *rel = find_base_rel_ignore_join(root, cur_relid);
+ /* We only need to add the clause to baserels */
+ if (rel == NULL)
+ continue;
rel->joininfo = lappend(rel->joininfo, restrictinfo);
}
}
@@ -115,8 +118,8 @@ add_join_clause_to_rels(PlannerInfo *root,
* discover that a relation need not be joined at all.
*
* 'restrictinfo' describes the join clause
- * 'join_relids' is the list of relations participating in the join clause
- * (there must be more than one)
+ * 'join_relids' is the set of relations participating in the join clause
+ * (some of these could be outer joins)
*/
void
remove_join_clause_from_rels(PlannerInfo *root,
@@ -128,7 +131,11 @@ remove_join_clause_from_rels(PlannerInfo *root,
cur_relid = -1;
while ((cur_relid = bms_next_member(join_relids, cur_relid)) >= 0)
{
- RelOptInfo *rel = find_base_rel(root, cur_relid);
+ RelOptInfo *rel = find_base_rel_ignore_join(root, cur_relid);
+
+ /* We would only have added the clause to baserels */
+ if (rel == NULL)
+ continue;
/*
* Remove the restrictinfo from the list. Pointer comparison is
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index b1363df065..9cfde2f790 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -338,6 +338,11 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
sjinfo.syn_lefthand = sjinfo.min_lefthand;
sjinfo.syn_righthand = sjinfo.min_righthand;
sjinfo.jointype = JOIN_INNER;
+ sjinfo.ojrelid = 0;
+ sjinfo.commute_above_l = NULL;
+ sjinfo.commute_above_r = NULL;
+ sjinfo.commute_below = NULL;
+ sjinfo.oj_joinclause = NIL;
/* we don't bother trying to make the remaining fields valid */
sjinfo.lhs_strict = false;
sjinfo.delay_upper_joins = false;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 6dd11329fb..bf35d1989c 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -1307,7 +1307,7 @@ create_append_path(PlannerInfo *root,
* Apply query-wide LIMIT if known and path is for sole base relation.
* (Handling this at this low level is a bit klugy.)
*/
- if (root != NULL && bms_equal(rel->relids, root->all_baserels))
+ if (root != NULL && bms_equal(rel->relids, root->all_query_rels))
pathnode->limit_tuples = root->limit_tuples;
else
pathnode->limit_tuples = -1.0;
@@ -1436,7 +1436,7 @@ create_merge_append_path(PlannerInfo *root,
* Apply query-wide LIMIT if known and path is for sole base relation.
* (Handling this at this low level is a bit klugy.)
*/
- if (bms_equal(rel->relids, root->all_baserels))
+ if (bms_equal(rel->relids, root->all_query_rels))
pathnode->limit_tuples = root->limit_tuples;
else
pathnode->limit_tuples = -1.0;
diff --git a/src/backend/optimizer/util/placeholder.c b/src/backend/optimizer/util/placeholder.c
index c7bfa293c9..bbc39ac3c5 100644
--- a/src/backend/optimizer/util/placeholder.c
+++ b/src/backend/optimizer/util/placeholder.c
@@ -23,17 +23,32 @@
#include "optimizer/planmain.h"
#include "utils/lsyscache.h"
+
+typedef struct contain_placeholder_references_context
+{
+ int relid;
+ int sublevels_up;
+} contain_placeholder_references_context;
+
/* Local functions */
static void find_placeholders_recurse(PlannerInfo *root, Node *jtnode);
static void find_placeholders_in_expr(PlannerInfo *root, Node *expr);
+static bool contain_placeholder_references_walker(Node *node,
+ contain_placeholder_references_context *context);
/*
* make_placeholder_expr
* Make a PlaceHolderVar for the given expression.
*
- * phrels is the syntactic location (as a set of baserels) to attribute
+ * phrels is the syntactic location (as a set of relids) to attribute
* to the expression.
+ *
+ * The caller is responsible for adjusting phlevelsup and phnullingrels
+ * as needed. Because we do not know here which query level the PHV
+ * will be associated with, it's important that this function touches
+ * only root->glob; messing with other parts of PlannerInfo would be
+ * likely to do the wrong thing.
*/
PlaceHolderVar *
make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
@@ -42,8 +57,9 @@ make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
phv->phexpr = expr;
phv->phrels = phrels;
+ phv->phnullingrels = NULL; /* caller may change this later */
phv->phid = ++(root->glob->lastPHId);
- phv->phlevelsup = 0;
+ phv->phlevelsup = 0; /* caller may change this later */
return phv;
}
@@ -92,6 +108,15 @@ find_placeholder_info(PlannerInfo *root, PlaceHolderVar *phv)
phinfo->phid = phv->phid;
phinfo->ph_var = copyObject(phv);
+ /*
+ * By convention, phinfo->ph_var->phnullingrels is always empty, since the
+ * PlaceHolderInfo represents the initially-calculated state of the
+ * PlaceHolderVar. PlaceHolderVars appearing in the query tree might have
+ * varying values of phnullingrels, reflecting outer joins applied above
+ * the calculation level.
+ */
+ phinfo->ph_var->phnullingrels = NULL;
+
/*
* Any referenced rels that are outside the PHV's syntactic scope are
* LATERAL references, which should be included in ph_lateral but not in
@@ -344,6 +369,8 @@ update_placeholder_eval_levels(PlannerInfo *root, SpecialJoinInfo *new_sjinfo)
sjinfo->min_lefthand);
eval_at = bms_add_members(eval_at,
sjinfo->min_righthand);
+ if (sjinfo->ojrelid)
+ eval_at = bms_add_member(eval_at, sjinfo->ojrelid);
/* we'll need another iteration */
found_some = true;
}
@@ -418,6 +445,14 @@ add_placeholders_to_base_rels(PlannerInfo *root)
{
RelOptInfo *rel = find_base_rel(root, varno);
+ /*
+ * As in add_vars_to_targetlist(), a value computed at scan level
+ * has not yet been nulled by any outer join, so its phnullingrels
+ * should be empty.
+ */
+ Assert(phinfo->ph_var->phnullingrels == NULL);
+
+ /* Copying the PHV might be unnecessary here, but be safe */
rel->reltarget->exprs = lappend(rel->reltarget->exprs,
copyObject(phinfo->ph_var));
/* reltarget's cost and width fields will be updated later */
@@ -440,7 +475,8 @@ add_placeholders_to_base_rels(PlannerInfo *root)
*/
void
add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *outer_rel, RelOptInfo *inner_rel)
+ RelOptInfo *outer_rel, RelOptInfo *inner_rel,
+ SpecialJoinInfo *sjinfo)
{
Relids relids = joinrel->relids;
ListCell *lc;
@@ -471,9 +507,17 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
if (!bms_is_subset(phinfo->ph_eval_at, outer_rel->relids) &&
!bms_is_subset(phinfo->ph_eval_at, inner_rel->relids))
{
- PlaceHolderVar *phv = phinfo->ph_var;
+ /* Copying might be unnecessary here, but be safe */
+ PlaceHolderVar *phv = copyObject(phinfo->ph_var);
QualCost cost;
+ /*
+ * It'll start out not nulled by anything. Joins above
+ * this one might add to its phnullingrels later, in much
+ * the same way as for Vars.
+ */
+ Assert(phv->phnullingrels == NULL);
+
joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
phv);
cost_qual_eval_node(&cost, (Node *) phv->phexpr, root);
@@ -504,3 +548,74 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
}
}
}
+
+/*
+ * contain_placeholder_references_to
+ * Detect whether any PlaceHolderVars in the given clause contain
+ * references to the given relid (typically an OJ relid).
+ *
+ * "Contain" means that there's a use of the relid inside the PHV's
+ * contained expression, so that changing the nullability status of
+ * the rel might change what the PHV computes.
+ *
+ * The code here to cope with upper-level PHVs is likely dead, but keep it
+ * anyway just in case.
+ */
+bool
+contain_placeholder_references_to(PlannerInfo *root, Node *clause,
+ int relid)
+{
+ contain_placeholder_references_context context;
+
+ /* We can answer quickly in the common case that there's no PHVs at all */
+ if (root->glob->lastPHId == 0)
+ return false;
+ /* Else run the recursive search */
+ context.relid = relid;
+ context.sublevels_up = 0;
+ return contain_placeholder_references_walker(clause, &context);
+}
+
+static bool
+contain_placeholder_references_walker(Node *node,
+ contain_placeholder_references_context *context)
+{
+ if (node == NULL)
+ return false;
+ if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ /* We should just look through PHVs of other query levels */
+ if (phv->phlevelsup == context->sublevels_up)
+ {
+ /* If phrels matches, we found what we came for */
+ if (bms_is_member(context->relid, phv->phrels))
+ return true;
+
+ /*
+ * We should not examine phnullingrels: what we are looking for is
+ * references in the contained expression, not OJs that might null
+ * the result afterwards. Also, we don't need to recurse into the
+ * contained expression, because phrels should adequately
+ * summarize what's in there. So we're done here.
+ */
+ return false;
+ }
+ }
+ else if (IsA(node, Query))
+ {
+ /* Recurse into RTE subquery or not-yet-planned sublink subquery */
+ bool result;
+
+ context->sublevels_up++;
+ result = query_tree_walker((Query *) node,
+ contain_placeholder_references_walker,
+ context,
+ 0);
+ context->sublevels_up--;
+ return result;
+ }
+ return expression_tree_walker(node, contain_placeholder_references_walker,
+ context);
+}
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 1786a3dadd..226914498a 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -28,6 +28,7 @@
#include "optimizer/plancat.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+#include "rewrite/rewriteManip.h"
#include "utils/hsearch.h"
#include "utils/lsyscache.h"
@@ -39,7 +40,9 @@ typedef struct JoinHashEntry
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *input_rel);
+ RelOptInfo *input_rel,
+ SpecialJoinInfo *sjinfo,
+ bool can_null);
static List *build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outer_rel,
@@ -47,8 +50,10 @@ static List *build_joinrel_restrictlist(PlannerInfo *root,
static void build_joinrel_joinlist(RelOptInfo *joinrel,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel);
-static List *subbuild_joinrel_restrictlist(RelOptInfo *joinrel,
- List *joininfo_list,
+static List *subbuild_joinrel_restrictlist(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *input_rel,
+ Relids both_input_relids,
List *new_restrictlist);
static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
List *joininfo_list,
@@ -56,10 +61,12 @@ static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
static void set_foreign_rel_properties(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel);
static void add_join_rel(PlannerInfo *root, RelOptInfo *joinrel);
-static void build_joinrel_partition_info(RelOptInfo *joinrel,
+static void build_joinrel_partition_info(PlannerInfo *root,
+ RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
- List *restrictlist, JoinType jointype);
-static bool have_partkey_equi_join(RelOptInfo *joinrel,
+ SpecialJoinInfo *sjinfo,
+ List *restrictlist);
+static bool have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *rel1, RelOptInfo *rel2,
JoinType jointype, List *restrictlist);
static int match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel,
@@ -367,7 +374,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
/*
* find_base_rel
- * Find a base or other relation entry, which must already exist.
+ * Find a base or otherrel relation entry, which must already exist.
*/
RelOptInfo *
find_base_rel(PlannerInfo *root, int relid)
@@ -388,6 +395,44 @@ find_base_rel(PlannerInfo *root, int relid)
return NULL; /* keep compiler quiet */
}
+/*
+ * find_base_rel_ignore_join
+ * Find a base or otherrel relation entry, which must already exist.
+ *
+ * Unlike find_base_rel, if relid references an outer join then this
+ * will return NULL rather than raising an error. This is convenient
+ * for callers that must deal with relid sets including both base and
+ * outer joins.
+ */
+RelOptInfo *
+find_base_rel_ignore_join(PlannerInfo *root, int relid)
+{
+ Assert(relid > 0);
+
+ if (relid < root->simple_rel_array_size)
+ {
+ RelOptInfo *rel;
+ RangeTblEntry *rte;
+
+ rel = root->simple_rel_array[relid];
+ if (rel)
+ return rel;
+
+ /*
+ * We could just return NULL here, but for debugging purposes it seems
+ * best to actually verify that the relid is an outer join and not
+ * something weird.
+ */
+ rte = root->simple_rte_array[relid];
+ if (rte && rte->rtekind == RTE_JOIN && rte->jointype != JOIN_INNER)
+ return NULL;
+ }
+
+ elog(ERROR, "no relation entry for relid %d", relid);
+
+ return NULL; /* keep compiler quiet */
+}
+
/*
* build_join_rel_hash
* Construct the auxiliary hash table for join relations.
@@ -687,9 +732,11 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ build_joinrel_tlist(root, joinrel, outer_rel, sjinfo,
+ (sjinfo->jointype == JOIN_FULL));
+ build_joinrel_tlist(root, joinrel, inner_rel, sjinfo,
+ (sjinfo->jointype != JOIN_INNER));
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel, sjinfo);
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -721,8 +768,8 @@ build_join_rel(PlannerInfo *root,
joinrel->has_eclass_joins = has_relevant_eclass_joinclause(root, joinrel);
/* Store the partition information. */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- sjinfo->jointype);
+ build_joinrel_partition_info(root, joinrel, outer_rel, inner_rel, sjinfo,
+ restrictlist);
/*
* Set estimates of the joinrel's size.
@@ -778,16 +825,14 @@ build_join_rel(PlannerInfo *root,
* 'parent_joinrel' is the RelOptInfo representing the join between parent
* relations. Some of the members of new RelOptInfo are produced by
* translating corresponding members of this RelOptInfo
- * 'sjinfo': child-join context info
* 'restrictlist': list of RestrictInfo nodes that apply to this particular
* pair of joinable relations
- * 'jointype' is the join type (inner, left, full, etc)
+ * 'sjinfo': child join's join-type details
*/
RelOptInfo *
build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
RelOptInfo *inner_rel, RelOptInfo *parent_joinrel,
- List *restrictlist, SpecialJoinInfo *sjinfo,
- JoinType jointype)
+ List *restrictlist, SpecialJoinInfo *sjinfo)
{
RelOptInfo *joinrel = makeNode(RelOptInfo);
AppendRelInfo **appinfos;
@@ -801,6 +846,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->reloptkind = RELOPT_OTHER_JOINREL;
joinrel->relids = bms_union(outer_rel->relids, inner_rel->relids);
+ if (sjinfo->ojrelid != 0)
+ joinrel->relids = bms_add_member(joinrel->relids, sjinfo->ojrelid);
joinrel->rows = 0;
/* cheap startup cost is interesting iff not all tuples to be retrieved */
joinrel->consider_startup = (root->tuple_fraction > 0);
@@ -887,8 +934,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->has_eclass_joins = parent_joinrel->has_eclass_joins;
/* Is the join between partitions itself partitioned? */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- jointype);
+ build_joinrel_partition_info(root, joinrel, outer_rel, inner_rel, sjinfo,
+ restrictlist);
/* Child joinrel is parallel safe if parent is parallel safe. */
joinrel->consider_parallel = parent_joinrel->consider_parallel;
@@ -970,10 +1017,41 @@ min_join_parameterization(PlannerInfo *root,
*
* We also compute the expected width of the join's output, making use
* of data that was cached at the baserel level by set_rel_width().
+ *
+ * Pass can_null as true if the join is an outer join that can null Vars
+ * from this input relation. If so, we will (normally) add the join's relid
+ * to the nulling bitmaps of Vars and PHVs bubbled up from the input.
+ *
+ * When forming an outer join's target list, special handling is needed
+ * in case the outer join was commuted with another one per outer join
+ * identity 3 (see optimizer/README). We must take steps to ensure that
+ * the output Vars have the same nulling bitmaps that they would if the
+ * two joins had been done in syntactic order; else they won't match Vars
+ * appearing higher in the query tree. We need to do two things:
+ *
+ * First, sjinfo->commute_above_r is added to the nulling bitmaps of RHS Vars.
+ * This takes care of the case where we implement
+ * A leftjoin (B leftjoin C on (Pbc)) on (Pab)
+ * as
+ * (A leftjoin B on (Pab)) leftjoin C on (Pbc)
+ * The C columns emitted by the B/C join need to be shown as nulled by both
+ * the B/C and A/B joins, even though they've not traversed the A/B join.
+ * (If the joins haven't been commuted, we are adding the nullingrel bits
+ * prematurely; but that's okay because the C columns can't be referenced
+ * between here and the upper join.)
+ *
+ * Second, if a RHS Var has any of the relids in sjinfo->commute_above_l
+ * already set in its nulling bitmap, then we *don't* add sjinfo->ojrelid
+ * to its nulling bitmap (but we do still add commute_above_r). This takes
+ * care of the reverse transformation: if the original syntax was
+ * (A leftjoin B on (Pab)) leftjoin C on (Pbc)
+ * then the now-upper A/B join must not mark C columns as nulled by itself.
*/
static void
build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *input_rel)
+ RelOptInfo *input_rel,
+ SpecialJoinInfo *sjinfo,
+ bool can_null)
{
Relids relids = joinrel->relids;
ListCell *vars;
@@ -993,7 +1071,24 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
/* Is it still needed above this joinrel? */
if (bms_nonempty_difference(phinfo->ph_needed, relids))
{
- /* Yup, add it to the output */
+ /*
+ * Yup, add it to the output. If this join potentially nulls
+ * this input, we have to update the PHV's phnullingrels,
+ * which means making a copy.
+ */
+ if (can_null)
+ {
+ phv = copyObject(phv);
+ /* See comments above to understand this logic */
+ if (sjinfo->ojrelid != 0 &&
+ !bms_overlap(phv->phnullingrels, sjinfo->commute_above_l))
+ phv->phnullingrels = bms_add_member(phv->phnullingrels,
+ sjinfo->ojrelid);
+ if (sjinfo->commute_above_r)
+ phv->phnullingrels = bms_add_members(phv->phnullingrels,
+ sjinfo->commute_above_r);
+ }
+
joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
phv);
/* Bubbling up the precomputed result has cost zero */
@@ -1017,9 +1112,7 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
RowIdentityVarInfo *ridinfo = (RowIdentityVarInfo *)
list_nth(root->row_identity_vars, var->varattno - 1);
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- var);
- /* Vars have cost zero, so no need to adjust reltarget->cost */
+ /* Update reltarget width estimate from RowIdentityVarInfo */
joinrel->reltarget->width += ridinfo->rowidwidth;
}
else
@@ -1032,15 +1125,35 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
/* Is it still needed above this joinrel? */
ndx = var->varattno - baserel->min_attr;
- if (bms_nonempty_difference(baserel->attr_needed[ndx], relids))
- {
- /* Yup, add it to the output */
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- var);
- /* Vars have cost zero, so no need to adjust reltarget->cost */
- joinrel->reltarget->width += baserel->attr_widths[ndx];
- }
+ if (!bms_nonempty_difference(baserel->attr_needed[ndx], relids))
+ continue; /* nope, skip it */
+
+ /* Update reltarget width estimate from baserel's attr_widths */
+ joinrel->reltarget->width += baserel->attr_widths[ndx];
+ }
+
+ /*
+ * Add the Var to the output. If this join potentially nulls this
+ * input, we have to update the Var's varnullingrels, which means
+ * making a copy.
+ */
+ if (can_null)
+ {
+ var = copyObject(var);
+ /* See comments above to understand this logic */
+ if (sjinfo->ojrelid != 0 &&
+ !bms_overlap(var->varnullingrels, sjinfo->commute_above_l))
+ var->varnullingrels = bms_add_member(var->varnullingrels,
+ sjinfo->ojrelid);
+ if (sjinfo->commute_above_r)
+ var->varnullingrels = bms_add_members(var->varnullingrels,
+ sjinfo->commute_above_r);
}
+
+ joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
+ var);
+
+ /* Vars have cost zero, so no need to adjust reltarget->cost */
}
}
@@ -1059,7 +1172,7 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
* is not handled in the sub-relations, so it depends on which
* sub-relations are considered.
*
- * If a join clause from an input relation refers to base rels still not
+ * If a join clause from an input relation refers to base+OJ rels still not
* present in the joinrel, then it is still a join clause for the joinrel;
* we put it into the joininfo list for the joinrel. Otherwise,
* the clause is now a restrict clause for the joined relation, and we
@@ -1093,14 +1206,19 @@ build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *inner_rel)
{
List *result;
+ Relids both_input_relids;
+
+ both_input_relids = bms_union(outer_rel->relids, inner_rel->relids);
/*
* Collect all the clauses that syntactically belong at this level,
* eliminating any duplicates (important since we will see many of the
* same clauses arriving from both input relations).
*/
- result = subbuild_joinrel_restrictlist(joinrel, outer_rel->joininfo, NIL);
- result = subbuild_joinrel_restrictlist(joinrel, inner_rel->joininfo, result);
+ result = subbuild_joinrel_restrictlist(root, joinrel, outer_rel,
+ both_input_relids, NIL);
+ result = subbuild_joinrel_restrictlist(root, joinrel, inner_rel,
+ both_input_relids, result);
/*
* Add on any clauses derived from EquivalenceClasses. These cannot be
@@ -1135,24 +1253,63 @@ build_joinrel_joinlist(RelOptInfo *joinrel,
}
static List *
-subbuild_joinrel_restrictlist(RelOptInfo *joinrel,
- List *joininfo_list,
+subbuild_joinrel_restrictlist(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *input_rel,
+ Relids both_input_relids,
List *new_restrictlist)
{
ListCell *l;
- foreach(l, joininfo_list)
+ foreach(l, input_rel->joininfo)
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(l);
if (bms_is_subset(rinfo->required_relids, joinrel->relids))
{
/*
- * This clause becomes a restriction clause for the joinrel, since
- * it refers to no outside rels. Add it to the list, being
- * careful to eliminate duplicates. (Since RestrictInfo nodes in
- * different joinlists will have been multiply-linked rather than
- * copied, pointer equality should be a sufficient test.)
+ * This clause should become a restriction clause for the joinrel,
+ * since it refers to no outside rels. However, if it's a clone
+ * clause then it might be too late to evaluate it, so we have to
+ * check. (If it is too late, just ignore the clause, taking it
+ * on faith that another clone was or will be selected.) Clone
+ * clauses should always be outer-join clauses, so we compare
+ * against both_input_relids.
+ */
+ if (rinfo->has_clone || rinfo->is_clone)
+ {
+ Assert(!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids));
+ if (!bms_is_subset(rinfo->required_relids, both_input_relids))
+ continue;
+ if (!clause_is_computable_at(root, rinfo->clause_relids,
+ both_input_relids))
+ continue;
+ }
+ else
+ {
+ /*
+ * For non-clone clauses, we just Assert it's OK. These might
+ * be either join or filter clauses.
+ */
+#ifdef USE_ASSERT_CHECKING
+ if (RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ Assert(clause_is_computable_at(root, rinfo->clause_relids,
+ joinrel->relids));
+ else
+ {
+ Assert(bms_is_subset(rinfo->required_relids,
+ both_input_relids));
+ Assert(clause_is_computable_at(root, rinfo->clause_relids,
+ both_input_relids));
+ }
+#endif
+ }
+
+ /*
+ * OK, so add it to the list, being careful to eliminate
+ * duplicates. (Since RestrictInfo nodes in different joinlists
+ * will have been multiply-linked rather than copied, pointer
+ * equality should be a sufficient test.)
*/
new_restrictlist = list_append_unique_ptr(new_restrictlist, rinfo);
}
@@ -1659,9 +1816,10 @@ find_param_path_info(RelOptInfo *rel, Relids required_outer)
* partitioned join relation.
*/
static void
-build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
- RelOptInfo *inner_rel, List *restrictlist,
- JoinType jointype)
+build_joinrel_partition_info(PlannerInfo *root,
+ RelOptInfo *joinrel, RelOptInfo *outer_rel,
+ RelOptInfo *inner_rel, SpecialJoinInfo *sjinfo,
+ List *restrictlist)
{
PartitionScheme part_scheme;
@@ -1687,8 +1845,8 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
!outer_rel->consider_partitionwise_join ||
!inner_rel->consider_partitionwise_join ||
outer_rel->part_scheme != inner_rel->part_scheme ||
- !have_partkey_equi_join(joinrel, outer_rel, inner_rel,
- jointype, restrictlist))
+ !have_partkey_equi_join(root, joinrel, outer_rel, inner_rel,
+ sjinfo->jointype, restrictlist))
{
Assert(!IS_PARTITIONED_REL(joinrel));
return;
@@ -1712,7 +1870,8 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* child-join relations of the join relation in try_partitionwise_join().
*/
joinrel->part_scheme = part_scheme;
- set_joinrel_partition_key_exprs(joinrel, outer_rel, inner_rel, jointype);
+ set_joinrel_partition_key_exprs(joinrel, outer_rel, inner_rel,
+ sjinfo->jointype);
/*
* Set the consider_partitionwise_join flag.
@@ -1730,7 +1889,7 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* partition keys.
*/
static bool
-have_partkey_equi_join(RelOptInfo *joinrel,
+have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *rel1, RelOptInfo *rel2,
JoinType jointype, List *restrictlist)
{
@@ -1795,6 +1954,24 @@ have_partkey_equi_join(RelOptInfo *joinrel,
*/
strict_op = op_strict(opexpr->opno);
+ /*
+ * Vars appearing in the relation's partition keys will not have any
+ * varnullingrels, but those in expr1 and expr2 will if we're above
+ * outer joins that could null the respective rels. It's okay to
+ * match anyway, if the join operator is strict.
+ */
+ if (strict_op)
+ {
+ if (bms_overlap(rel1->relids, root->outer_join_rels))
+ expr1 = (Expr *) remove_nulling_relids((Node *) expr1,
+ root->outer_join_rels,
+ NULL);
+ if (bms_overlap(rel2->relids, root->outer_join_rels))
+ expr2 = (Expr *) remove_nulling_relids((Node *) expr2,
+ root->outer_join_rels,
+ NULL);
+ }
+
/*
* Only clauses referencing the partition keys are useful for
* partitionwise join.
@@ -2007,7 +2184,12 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
* partitionwise nesting of any outer join.) We assume no
* type coercions are needed to make the coalesce expressions,
* since columns of different types won't have gotten
- * classified as the same PartitionScheme.
+ * classified as the same PartitionScheme. Note that we
+ * intentionally leave out the varnullingrels decoration that
+ * would ordinarily appear on the Vars inside these
+ * CoalesceExprs, because have_partkey_equi_join will strip
+ * varnullingrels from the expressions it will compare to the
+ * partexprs.
*/
foreach(lc, list_concat_copy(outer_expr, outer_null_expr))
{
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index ef8df3d098..327c3ba563 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -53,6 +53,10 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
* required_relids can be NULL, in which case it defaults to the actual clause
* contents (i.e., clause_relids).
*
+ * Note that there aren't options to set the has_clone and is_clone flags:
+ * we always initialize those to false. There's just one place that wants
+ * something different, so making all callers pass them seems inconvenient.
+ *
* We initialize fields that depend only on the given subexpression, leaving
* others that depend on context (or may never be needed at all) to be filled
* later.
@@ -116,12 +120,15 @@ make_restrictinfo_internal(PlannerInfo *root,
Relids nullable_relids)
{
RestrictInfo *restrictinfo = makeNode(RestrictInfo);
+ Relids baserels;
restrictinfo->clause = clause;
restrictinfo->orclause = orclause;
restrictinfo->is_pushed_down = is_pushed_down;
restrictinfo->outerjoin_delayed = outerjoin_delayed;
restrictinfo->pseudoconstant = pseudoconstant;
+ restrictinfo->has_clone = false; /* may get set by caller */
+ restrictinfo->is_clone = false; /* may get set by caller */
restrictinfo->can_join = false; /* may get set below */
restrictinfo->security_level = security_level;
restrictinfo->outer_relids = outer_relids;
@@ -187,6 +194,20 @@ make_restrictinfo_internal(PlannerInfo *root,
else
restrictinfo->required_relids = restrictinfo->clause_relids;
+ /*
+ * Count the number of base rels appearing in clause_relids. To do this,
+ * we just delete rels mentioned in root->outer_join_rels and count the
+ * survivors. Because we are called during deconstruct_jointree which is
+ * the same tree walk that populates outer_join_rels, this is a little bit
+ * unsafe-looking; but it should be fine because the recursion in
+ * deconstruct_jointree should already have visited any outer join that
+ * could be mentioned in this clause.
+ */
+ baserels = bms_difference(restrictinfo->clause_relids,
+ root->outer_join_rels);
+ restrictinfo->num_base_rels = bms_num_members(baserels);
+ bms_free(baserels);
+
/*
* Fill in all the cacheable fields with "not yet set" markers. None of
* these will be computed until/unless needed. Note in particular that we
@@ -497,6 +518,58 @@ extract_actual_join_clauses(List *restrictinfo_list,
}
}
+/*
+ * clause_is_computable_at
+ * Test whether a clause is computable at a given evaluation level.
+ *
+ * There are two conditions for whether an expression can actually be
+ * evaluated at a given join level: the evaluation context must include
+ * all the relids (both base and OJ) used by the expression, and we must
+ * not have already evaluated any outer joins that null Vars/PHVs of the
+ * expression and are not listed in their nullingrels.
+ *
+ * This function checks the second condition; we assume the caller already
+ * saw to the first one.
+ *
+ * For speed reasons, we don't individually examine each Var/PHV of the
+ * expression, but just look at the overall clause_relids (the union of the
+ * varnos and varnullingrels). This could give a misleading answer if the
+ * Vars of a given varno don't all have the same varnullingrels; but that
+ * really shouldn't happen within a single scalar expression or RestrictInfo
+ * clause. Despite that, this is still annoyingly expensive :-(
+ */
+bool
+clause_is_computable_at(PlannerInfo *root,
+ Relids clause_relids,
+ Relids eval_relids)
+{
+ ListCell *lc;
+
+ /* Nothing to do if no outer joins have been performed yet. */
+ if (!bms_overlap(eval_relids, root->outer_join_rels))
+ return true;
+
+ foreach(lc, root->join_info_list)
+ {
+ SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+
+ /* Ignore outer joins that are not yet performed. */
+ if (!bms_is_member(sjinfo->ojrelid, eval_relids))
+ continue;
+
+ /* OK if clause lists it (we assume all Vars in it agree). */
+ if (bms_is_member(sjinfo->ojrelid, clause_relids))
+ continue;
+
+ /* Else, trouble if clause mentions any nullable Vars. */
+ if (bms_overlap(clause_relids, sjinfo->min_righthand) ||
+ (sjinfo->jointype == JOIN_FULL &&
+ bms_overlap(clause_relids, sjinfo->min_lefthand)))
+ return false; /* doesn't work */
+ }
+
+ return true; /* OK */
+}
/*
* join_clause_is_movable_to
@@ -522,6 +595,12 @@ extract_actual_join_clauses(List *restrictinfo_list,
* Also, the join clause must not use any relations that have LATERAL
* references to the target relation, since we could not put such rels on
* the outer side of a nestloop with the target relation.
+ *
+ * Also, we reject is_clone versions of outer-join clauses. This has the
+ * effect of preventing us from generating variant parameterized paths
+ * that differ only in which outer joins null the parameterization rel(s).
+ * Generating one path from the minimally-parameterized has_clone version
+ * is sufficient.
*/
bool
join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
@@ -542,6 +621,10 @@ join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
if (bms_overlap(baserel->lateral_referencers, rinfo->clause_relids))
return false;
+ /* Ignore clones, too */
+ if (rinfo->is_clone)
+ return false;
+
return true;
}
@@ -587,6 +670,9 @@ join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
* moved for some valid set of outer rels, so we don't have the benefit of
* relying on prior checks for lateral-reference validity.
*
+ * Likewise, we don't check is_clone here: rejecting the inappropriate
+ * variants of a cloned clause must be handled upstream.
+ *
* Note: if this returns true, it means that the clause could be moved to
* this join relation, but that doesn't mean that this is the lowest join
* it could be moved to. Caller may need to make additional calls to verify
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
index 7db86c39ef..8d8c9136f8 100644
--- a/src/backend/optimizer/util/var.c
+++ b/src/backend/optimizer/util/var.c
@@ -88,6 +88,9 @@ static Relids alias_relid_set(Query *query, Relids relids);
* Create a set of all the distinct varnos present in a parsetree.
* Only varnos that reference level-zero rtable entries are considered.
*
+ * The result includes outer-join relids mentioned in Var.varnullingrels and
+ * PlaceHolderVar.phnullingrels fields in the parsetree.
+ *
* "root" can be passed as NULL if it is not necessary to process
* PlaceHolderVars.
*
@@ -153,7 +156,11 @@ pull_varnos_walker(Node *node, pull_varnos_context *context)
Var *var = (Var *) node;
if (var->varlevelsup == context->sublevels_up)
+ {
context->varnos = bms_add_member(context->varnos, var->varno);
+ context->varnos = bms_add_members(context->varnos,
+ var->varnullingrels);
+ }
return false;
}
if (IsA(node, CurrentOfExpr))
@@ -244,6 +251,14 @@ pull_varnos_walker(Node *node, pull_varnos_context *context)
context->varnos = bms_join(context->varnos,
newevalat);
}
+
+ /*
+ * In all three cases, include phnullingrels in the result. We
+ * don't worry about possibly needing to translate it, because
+ * appendrels only translate varnos of baserels, not outer joins.
+ */
+ context->varnos = bms_add_members(context->varnos,
+ phv->phnullingrels);
return false; /* don't recurse into expression */
}
}
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index d597b7e81f..40129d435e 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -2205,7 +2205,7 @@ rowcomparesel(PlannerInfo *root,
else
{
/*
- * Otherwise, it's a join if there's more than one relation used.
+ * Otherwise, it's a join if there's more than one base relation used.
*/
is_join_clause = (NumRelids(root, (Node *) opargs) > 1);
}
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 2c6d5ca58f..a734676293 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -243,13 +243,26 @@ struct PlannerInfo
struct AppendRelInfo **append_rel_array pg_node_attr(read_write_ignore);
/*
- * all_baserels is a Relids set of all base relids (but not "other"
- * relids) in the query; that is, the Relids identifier of the final join
- * we need to form. This is computed in make_one_rel, just before we
- * start making Paths.
+ * all_baserels is a Relids set of all base relids (but not joins or
+ * "other" relids) in the query. This is computed in
+ * add_base_rels_to_query.
*/
Relids all_baserels;
+ /*
+ * outer_join_rels is a Relids set of all outer-join relids in the query.
+ * This is computed in deconstruct_jointree.
+ */
+ Relids outer_join_rels;
+
+ /*
+ * all_query_rels is a Relids set of all base relids and outer join relids
+ * (but not "other" relids) in the query. This is the Relids identifier
+ * of the final join we need to form. This is computed in
+ * deconstruct_jointree.
+ */
+ Relids all_query_rels;
+
/*
* nullable_baserels is a Relids set of base relids that are nullable by
* some outer join in the jointree; these are rels that are potentially
@@ -319,7 +332,7 @@ struct PlannerInfo
List *right_join_clauses;
/*
- * list of RestrictInfos for mergejoinable full join clauses
+ * list of FullJoinClauseInfos for mergejoinable full join clauses
*/
List *full_join_clauses;
@@ -555,9 +568,10 @@ typedef struct PartitionSchemeData *PartitionScheme;
* or the output of a sub-SELECT or function that appears in the range table.
* In either case it is uniquely identified by an RT index. A "joinrel"
* is the joining of two or more base rels. A joinrel is identified by
- * the set of RT indexes for its component baserels. We create RelOptInfo
- * nodes for each baserel and joinrel, and store them in the PlannerInfo's
- * simple_rel_array and join_rel_list respectively.
+ * the set of RT indexes for its component baserels, along with RT indexes
+ * for any outer joins it has computed. We create RelOptInfo nodes for each
+ * baserel and joinrel, and store them in the PlannerInfo's simple_rel_array
+ * and join_rel_list respectively.
*
* Note that there is only one joinrel for any given set of component
* baserels, no matter what order we assemble them in; so an unordered
@@ -596,8 +610,10 @@ typedef struct PartitionSchemeData *PartitionScheme;
* Parts of this data structure are specific to various scan and join
* mechanisms. It didn't seem worth creating new node types for them.
*
- * relids - Set of base-relation identifiers; it is a base relation
- * if there is just one, a join relation if more than one
+ * relids - Set of relation identifiers (RT indexes). This is a base
+ * relation if there is just one, a join relation if more;
+ * in the join case, RT indexes of any outer joins formed
+ * at or below this join are included along with baserels
* rows - estimated number of tuples in the relation after restriction
* clauses have been applied (ie, output rows of a plan for it)
* consider_startup - true if there is any value in keeping plain paths for
@@ -636,7 +652,9 @@ typedef struct PartitionSchemeData *PartitionScheme;
* min_attr, max_attr - range of valid AttrNumbers for rel
* attr_needed - array of bitmapsets indicating the highest joinrel
* in which each attribute is needed; if bit 0 is set then
- * the attribute is needed as part of final targetlist
+ * the attribute is needed as part of final targetlist.
+ * By convention, attr_needed includes only baserels not
+ * outer-join relids.
* attr_widths - cache space for per-attribute width estimates;
* zero means not computed yet
* lateral_vars - lateral cross-references of rel, if any (list of
@@ -809,7 +827,7 @@ typedef struct RelOptInfo
RelOptKind reloptkind;
/*
- * all relations included in this RelOptInfo; set of base relids
+ * all relations included in this RelOptInfo; set of base + OJ relids
* (rangetable indexes)
*/
Relids relids;
@@ -2278,17 +2296,17 @@ typedef struct LimitPath
* If a restriction clause references a single base relation, it will appear
* in the baserestrictinfo list of the RelOptInfo for that base rel.
*
- * If a restriction clause references more than one base rel, it will
+ * If a restriction clause references more than one base+OJ relation, it will
* appear in the joininfo list of every RelOptInfo that describes a strict
- * subset of the base rels mentioned in the clause. The joininfo lists are
+ * subset of the relations mentioned in the clause. The joininfo lists are
* used to drive join tree building by selecting plausible join candidates.
* The clause cannot actually be applied until we have built a join rel
- * containing all the base rels it references, however.
+ * containing all the relations it references, however.
*
- * When we construct a join rel that includes all the base rels referenced
+ * When we construct a join rel that includes all the relations referenced
* in a multi-relation restriction clause, we place that clause into the
* joinrestrictinfo lists of paths for the join rel, if neither left nor
- * right sub-path includes all base rels referenced in the clause. The clause
+ * right sub-path includes all relations referenced in the clause. The clause
* will be applied at that join level, and will not propagate any further up
* the join tree. (Note: the "predicate migration" code was once intended to
* push restriction clauses up and down the plan tree based on evaluation
@@ -2309,12 +2327,15 @@ typedef struct LimitPath
* or join to enforce that all members of each EquivalenceClass are in fact
* equal in all rows emitted by the scan or join.
*
- * When dealing with outer joins we have to be very careful about pushing qual
- * clauses up and down the tree. An outer join's own JOIN/ON conditions must
- * be evaluated exactly at that join node, unless they are "degenerate"
- * conditions that reference only Vars from the nullable side of the join.
- * Quals appearing in WHERE or in a JOIN above the outer join cannot be pushed
- * down below the outer join, if they reference any nullable Vars.
+ * The clause_relids field lists the base plus outer-join RT indexes that
+ * actually appear in the clause. required_relids lists the minimum set of
+ * relids needed to evaluate the clause; while this is often equal to
+ * clause_relids, it can be more. We will add relids to required_relids when
+ * we need to force an outer join ON clause to be evaluated exactly at the
+ * level of the outer join, which is true except when it is a "degenerate"
+ * condition that references only Vars from the nullable side of the join.
+ *
+ * XXX rewrite or remove me:
* RestrictInfo nodes contain a flag to indicate whether a qual has been
* pushed down to a lower level than its original syntactic placement in the
* join tree would suggest. If an outer join prevents us from pushing a qual
@@ -2399,6 +2420,12 @@ typedef struct LimitPath
* or merge or hash join clause, so it's of no interest to large parts of
* the planner.
*
+ * When we generate multiple versions of a clause so as to have versions
+ * that will work after commuting some left joins per outer join identity 3,
+ * we mark the one with the fewest nullingrels bits with has_clone = true,
+ * and the rest with is_clone = true. This allows proper filtering of
+ * these redundant clauses, so that we apply only one version of them.
+ *
* When join clauses are generated from EquivalenceClasses, there may be
* several equally valid ways to enforce join equivalence, of which we need
* apply only one. We mark clauses of this kind by setting parent_ec to
@@ -2433,16 +2460,23 @@ typedef struct RestrictInfo
/* see comment above */
bool pseudoconstant pg_node_attr(equal_ignore);
+ /* see comment above */
+ bool has_clone;
+ bool is_clone;
+
/* true if known to contain no leaked Vars */
bool leakproof pg_node_attr(equal_ignore);
- /* to indicate if clause contains any volatile functions. */
+ /* indicates if clause contains any volatile functions */
VolatileFunctionStatus has_volatile pg_node_attr(equal_ignore);
/* see comment above */
Index security_level;
- /* The set of relids (varnos) actually referenced in the clause: */
+ /* number of base rels in clause_relids */
+ int num_base_rels pg_node_attr(equal_ignore);
+
+ /* The relids (varnos+varnullingrels) actually referenced in the clause: */
Relids clause_relids pg_node_attr(equal_ignore);
/* The set of relids required to evaluate the clause: */
@@ -2544,6 +2578,7 @@ typedef struct RestrictInfo
} RestrictInfo;
/*
+ * XXX this will need work:
* This macro embodies the correct way to test whether a RestrictInfo is
* "pushed down" to a given outer join, that is, should be treated as a filter
* clause rather than a join clause at that outer join. This is certainly so
@@ -2646,7 +2681,7 @@ typedef struct PlaceHolderVar
* We make SpecialJoinInfos for FULL JOINs even though there is no flexibility
* of planning for them, because this simplifies make_join_rel()'s API.
*
- * min_lefthand and min_righthand are the sets of base relids that must be
+ * min_lefthand and min_righthand are the sets of base+OJ relids that must be
* available on each side when performing the special join. lhs_strict is
* true if the special join's condition cannot succeed when the LHS variables
* are all NULL (this means that an outer join can commute with upper-level
@@ -2656,7 +2691,7 @@ typedef struct PlaceHolderVar
* It is not valid for either min_lefthand or min_righthand to be empty sets;
* if they were, this would break the logic that enforces join order.
*
- * syn_lefthand and syn_righthand are the sets of base relids that are
+ * syn_lefthand and syn_righthand are the sets of base+OJ relids that are
* syntactically below this special join. (These are needed to help compute
* min_lefthand and min_righthand for higher joins.)
*
@@ -2678,6 +2713,37 @@ typedef struct PlaceHolderVar
* the inputs to make it a LEFT JOIN. So the allowed values of jointype
* in a join_info_list member are only LEFT, FULL, SEMI, or ANTI.
*
+ * ojrelid is the RT index of the join RTE representing this outer join,
+ * if there is one. It is zero when jointype is INNER or SEMI, and can be
+ * zero for jointype ANTI (if the join was transformed from a SEMI join).
+ * One use for this field is that when constructing the output targetlist of a
+ * join relation that implements this OJ, we add ojrelid to the varnullingrels
+ * and phnullingrels fields of nullable (RHS) output columns, so that the
+ * output Vars and PlaceHolderVars correctly reflect the nulling that has
+ * potentially happened to them.
+ *
+ * commute_above_l is filled with the relids of syntactically-higher outer
+ * joins that have been found to commute with this one per outer join identity
+ * 3 (see optimizer/README), when this join is in the LHS of the upper join
+ * (so, this is the lower join in the first form of the identity).
+ *
+ * commute_above_r is filled with the relids of syntactically-higher outer
+ * joins that have been found to commute with this one per outer join identity
+ * 3, when this join is in the RHS of the upper join (so, this is the lower
+ * join in the second form of the identity).
+ *
+ * commute_below is filled with the relids of syntactically-lower outer joins
+ * that have been found to commute with this one per outer join identity 3.
+ * (We need not record which side they are on, since that can be determined
+ * by seeing whether the lower join's relid appears in syn_lefthand or
+ * syn_righthand.)
+ *
+ * oj_joinclause is used during deconstruct_jointree() to hold the JOIN/ON
+ * quals of a possibly-commutable outer join until the end of the jointree
+ * walk (at which time we'll know whether any other outer joins actually
+ * commute with it, and can decorate the quals properly). These quals do not
+ * have RestrictInfos yet.
+ *
* For purposes of join selectivity estimation, we create transient
* SpecialJoinInfo structures for regular inner joins; so it is possible
* to have jointype == JOIN_INNER in such a structure, even though this is
@@ -2697,11 +2763,16 @@ struct SpecialJoinInfo
pg_node_attr(no_read)
NodeTag type;
- Relids min_lefthand; /* base relids in minimum LHS for join */
- Relids min_righthand; /* base relids in minimum RHS for join */
- Relids syn_lefthand; /* base relids syntactically within LHS */
- Relids syn_righthand; /* base relids syntactically within RHS */
+ Relids min_lefthand; /* base+OJ relids in minimum LHS for join */
+ Relids min_righthand; /* base+OJ relids in minimum RHS for join */
+ Relids syn_lefthand; /* base+OJ relids syntactically within LHS */
+ Relids syn_righthand; /* base+OJ relids syntactically within RHS */
JoinType jointype; /* always INNER, LEFT, FULL, SEMI, or ANTI */
+ Index ojrelid; /* outer join's RT index; 0 if none */
+ Relids commute_above_l; /* commuting OJs above this one, if LHS */
+ Relids commute_above_r; /* commuting OJs above this one, if RHS */
+ Relids commute_below; /* commuting OJs below this one */
+ List *oj_joinclause; /* outer join quals not yet distributed */
bool lhs_strict; /* joinclause is strict for some LHS rel */
bool delay_upper_joins; /* can't commute with upper RHS */
/* Remaining fields are set only for JOIN_SEMI jointype: */
@@ -2711,6 +2782,21 @@ struct SpecialJoinInfo
List *semi_rhs_exprs; /* righthand-side expressions of these ops */
};
+/*
+ * FULL JOIN clause info.
+ *
+ * We set aside every FULL JOIN ON clause that looks mergejoinable, and
+ * process it specially at the end of qual distribution.
+ */
+typedef struct FullJoinClauseInfo
+{
+ pg_node_attr(no_copy_equal, no_read)
+
+ NodeTag type;
+ RestrictInfo *rinfo; /* a mergejoinable FULL JOIN clause */
+ SpecialJoinInfo *sjinfo; /* the FULL JOIN's SpecialJoinInfo */
+} FullJoinClauseInfo;
+
/*
* Append-relation info.
*
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 050f00e79a..197234d44c 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -304,6 +304,7 @@ extern void expand_planner_arrays(PlannerInfo *root, int add_size);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
+extern RelOptInfo *find_base_rel_ignore_join(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
@@ -335,6 +336,6 @@ extern ParamPathInfo *find_param_path_info(RelOptInfo *rel,
extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
- SpecialJoinInfo *sjinfo, JoinType jointype);
+ SpecialJoinInfo *sjinfo);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/placeholder.h b/src/include/optimizer/placeholder.h
index 507dbc6175..3fe9b57415 100644
--- a/src/include/optimizer/placeholder.h
+++ b/src/include/optimizer/placeholder.h
@@ -27,6 +27,9 @@ extern void update_placeholder_eval_levels(PlannerInfo *root,
extern void fix_placeholder_input_needed_levels(PlannerInfo *root);
extern void add_placeholders_to_base_rels(PlannerInfo *root);
extern void add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *outer_rel, RelOptInfo *inner_rel);
+ RelOptInfo *outer_rel, RelOptInfo *inner_rel,
+ SpecialJoinInfo *sjinfo);
+extern bool contain_placeholder_references_to(PlannerInfo *root, Node *clause,
+ int relid);
#endif /* PLACEHOLDER_H */
diff --git a/src/include/optimizer/prep.h b/src/include/optimizer/prep.h
index 5b4f350b33..0847cfd5f4 100644
--- a/src/include/optimizer/prep.h
+++ b/src/include/optimizer/prep.h
@@ -29,7 +29,8 @@ extern void pull_up_subqueries(PlannerInfo *root);
extern void flatten_simple_union_all(PlannerInfo *root);
extern void reduce_outer_joins(PlannerInfo *root);
extern void remove_useless_result_rtes(PlannerInfo *root);
-extern Relids get_relids_in_jointree(Node *jtnode, bool include_joins);
+extern Relids get_relids_in_jointree(Node *jtnode, bool include_outer_joins,
+ bool include_inner_joins);
extern Relids get_relids_for_join(Query *query, int joinrelid);
/*
diff --git a/src/include/optimizer/restrictinfo.h b/src/include/optimizer/restrictinfo.h
index 6d30bd5e9d..17d3b4ab05 100644
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -41,6 +41,9 @@ extern void extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
List **joinquals,
List **otherquals);
+extern bool clause_is_computable_at(PlannerInfo *root,
+ Relids clause_relids,
+ Relids eval_relids);
extern bool join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel);
extern bool join_clause_is_movable_into(RestrictInfo *rinfo,
Relids currentrelids,
commit de13875dbaa05d7eb4392abb1ea043987048aa7c
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sat Nov 5 17:14:58 2022 -0400
Rewrite reduce_outer_joins' matching of Vars.
Adding varnullingrels breaks the logic in reduce_outer_joins_pass2
that detects antijoins by matching upper-level "Var IS NULL" tests to
strict join quals. The problem of course is that the upper Var is
no longer equal() to the one in the join qual, since the former will
now be marked as being nulled by the outer join.
Now, this logic was always pretty brute-force: doing list_intersect
on a list full of Vars isn't especially cheap. So let's fix it by
creating a better-suited data structure, namely an array of per-RTE
bitmaps of relevant Vars' attnos.
(I wonder if there aren't other applications for an array-of-bitmaps
data structure, which might lead to wanting this to be a general-purpose
data set on the same level as Bitmapset. But for now I just settled for
writing enough primitives for the immediate problem.)
Discussion: https://postgr.es/m/CAMbWs4-mvPPCJ1W6iK6dD5HiNwoJdi6mZp=-7mE8N9Sh+cd0tQ@mail.gmail.com
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 4821b7a71f..ce029dbf69 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -124,7 +124,7 @@ static void reduce_outer_joins_pass2(Node *jtnode,
reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
- List *forced_null_vars);
+ VarAttnoSet *forced_null_vars);
static void report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
int rtindex, Relids relids);
static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
@@ -2637,7 +2637,7 @@ reduce_outer_joins(PlannerInfo *root)
reduce_outer_joins_pass2((Node *) root->parse->jointree,
state1, &state2,
- root, NULL, NIL);
+ root, NULL, NULL);
/*
* If we successfully reduced the strength of any outer joins, we must
@@ -2753,7 +2753,7 @@ reduce_outer_joins_pass1(Node *jtnode)
* state2: where to accumulate info about successfully-reduced joins
* root: toplevel planner state
* nonnullable_rels: set of base relids forced non-null by upper quals
- * forced_null_vars: list of Vars forced null by upper quals
+ * forced_null_vars: set of Vars forced null by upper quals
*
* Returns info in state2 about outer joins that were successfully simplified.
* Joins that were fully reduced to inner joins are all added to
@@ -2767,7 +2767,7 @@ reduce_outer_joins_pass2(Node *jtnode,
reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
- List *forced_null_vars)
+ VarAttnoSet *forced_null_vars)
{
/*
* pass 2 should never descend as far as an empty subnode or base rel,
@@ -2783,15 +2783,16 @@ reduce_outer_joins_pass2(Node *jtnode,
ListCell *l;
ListCell *s;
Relids pass_nonnullable_rels;
- List *pass_forced_null_vars;
+ VarAttnoSet *pass_forced_null_vars;
/* Scan quals to see if we can add any constraints */
pass_nonnullable_rels = find_nonnullable_rels(f->quals);
pass_nonnullable_rels = bms_add_members(pass_nonnullable_rels,
nonnullable_rels);
- pass_forced_null_vars = find_forced_null_vars(f->quals);
- pass_forced_null_vars = list_concat(pass_forced_null_vars,
- forced_null_vars);
+ pass_forced_null_vars = make_empty_varattnoset(list_length(root->parse->rtable));
+ find_forced_null_vars(f->quals, pass_forced_null_vars);
+ varattnoset_add_members(pass_forced_null_vars,
+ forced_null_vars);
/* And recurse --- but only into interesting subtrees */
Assert(list_length(f->fromlist) == list_length(state1->sub_states));
forboth(l, f->fromlist, s, state1->sub_states)
@@ -2898,22 +2899,21 @@ reduce_outer_joins_pass2(Node *jtnode,
*/
if (jointype == JOIN_LEFT)
{
- List *nonnullable_vars;
- List *overlap;
+ VarAttnoSet *nonnullable_vars;
+ Relids overlap;
/* Find Vars in j->quals that must be non-null in joined rows */
- nonnullable_vars = find_nonnullable_vars(j->quals);
+ nonnullable_vars = make_empty_varattnoset(list_length(root->parse->rtable));
+ find_nonnullable_vars(j->quals, nonnullable_vars);
/*
* It's not sufficient to check whether nonnullable_vars and
* forced_null_vars overlap: we need to know if the overlap
* includes any RHS variables.
*/
- overlap = list_intersection(nonnullable_vars,
- forced_null_vars);
- if (overlap != NIL &&
- bms_overlap(pull_varnos(root, (Node *) overlap),
- right_state->relids))
+ overlap = varattnoset_intersect_relids(nonnullable_vars,
+ forced_null_vars);
+ if (bms_overlap(overlap, right_state->relids))
jointype = JOIN_ANTI;
}
@@ -2938,9 +2938,9 @@ reduce_outer_joins_pass2(Node *jtnode,
if (left_state->contains_outer || right_state->contains_outer)
{
Relids local_nonnullable_rels;
- List *local_forced_null_vars;
+ VarAttnoSet *local_forced_null_vars;
Relids pass_nonnullable_rels;
- List *pass_forced_null_vars;
+ VarAttnoSet *pass_forced_null_vars;
/*
* If this join is (now) inner, we can add any constraints its
@@ -2966,21 +2966,22 @@ reduce_outer_joins_pass2(Node *jtnode,
if (jointype != JOIN_FULL)
{
local_nonnullable_rels = find_nonnullable_rels(j->quals);
- local_forced_null_vars = find_forced_null_vars(j->quals);
+ local_forced_null_vars = make_empty_varattnoset(list_length(root->parse->rtable));
+ find_forced_null_vars(j->quals, local_forced_null_vars);
if (jointype == JOIN_INNER || jointype == JOIN_SEMI)
{
/* OK to merge upper and local constraints */
local_nonnullable_rels = bms_add_members(local_nonnullable_rels,
nonnullable_rels);
- local_forced_null_vars = list_concat(local_forced_null_vars,
- forced_null_vars);
+ varattnoset_add_members(local_forced_null_vars,
+ forced_null_vars);
}
}
else
{
/* no use in calculating these */
local_nonnullable_rels = NULL;
- local_forced_null_vars = NIL;
+ local_forced_null_vars = NULL;
}
if (left_state->contains_outer)
@@ -3001,7 +3002,7 @@ reduce_outer_joins_pass2(Node *jtnode,
{
/* no constraints pass through JOIN_FULL */
pass_nonnullable_rels = NULL;
- pass_forced_null_vars = NIL;
+ pass_forced_null_vars = NULL;
}
reduce_outer_joins_pass2(j->larg, left_state,
state2, root,
@@ -3021,7 +3022,7 @@ reduce_outer_joins_pass2(Node *jtnode,
{
/* no constraints pass through JOIN_FULL */
pass_nonnullable_rels = NULL;
- pass_forced_null_vars = NIL;
+ pass_forced_null_vars = NULL;
}
reduce_outer_joins_pass2(j->rarg, right_state,
state2, root,
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 8c80f5a1a6..7d1f729d97 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -105,7 +105,8 @@ static bool contain_context_dependent_node(Node *clause);
static bool contain_context_dependent_node_walker(Node *node, int *flags);
static bool contain_leaked_vars_walker(Node *node, void *context);
static Relids find_nonnullable_rels_walker(Node *node, bool top_level);
-static List *find_nonnullable_vars_walker(Node *node, bool top_level);
+static void find_nonnullable_vars_walker(Node *node, VarAttnoSet *attnos,
+ bool top_level);
static bool is_strict_saop(ScalarArrayOpExpr *expr, bool falseOK);
static bool convert_saop_to_hashed_saop_walker(Node *node, void *context);
static Node *eval_const_expressions_mutator(Node *node,
@@ -1308,6 +1309,85 @@ contain_leaked_vars_walker(Node *node, void *context)
context);
}
+
+/*
+ * make_empty_varattnoset
+ * Create an empty VarAttnoSet structure for use with following routines.
+ *
+ * The maximum varno we expect to deal with is rangetable_length.
+ */
+VarAttnoSet *
+make_empty_varattnoset(int rangetable_length)
+{
+ VarAttnoSet *result;
+
+ /* palloc0 is sufficient to initialize all the bitmapsets to empty */
+ result = (VarAttnoSet *)
+ palloc0(offsetof(VarAttnoSet, varattnos) +
+ (rangetable_length + 1) * sizeof(Bitmapset *));
+ result->max_varno = rangetable_length;
+ return result;
+}
+
+/*
+ * varattnoset_add_members
+ * Add all members of set b to set a.
+ *
+ * This is like bms_add_members, but for sets of bitmapsets.
+ * For convenience, we allow b (but not a) to be a NULL pointer.
+ */
+void
+varattnoset_add_members(VarAttnoSet *a, const VarAttnoSet *b)
+{
+ if (b != NULL)
+ {
+ /* We don't really expect the max_varnos to differ, but allow b < a */
+ Assert(b->max_varno <= a->max_varno);
+ for (int i = 1; i <= b->max_varno; i++)
+ a->varattnos[i] = bms_add_members(a->varattnos[i],
+ b->varattnos[i]);
+ }
+}
+
+/*
+ * varattnoset_int_members
+ * Reduce set a to its intersection with set b.
+ *
+ * This is like bms_int_members, but for sets of bitmapsets.
+ */
+static void
+varattnoset_int_members(VarAttnoSet *a, const VarAttnoSet *b)
+{
+ /* We don't really expect the max_varnos to differ, but allow a < b */
+ Assert(a->max_varno <= b->max_varno);
+ for (int i = 1; i <= a->max_varno; i++)
+ a->varattnos[i] = bms_int_members(a->varattnos[i],
+ b->varattnos[i]);
+}
+
+/*
+ * varattnoset_intersect_relids
+ * Identify the relations having common members in a and b.
+ *
+ * For convenience, we allow NULL inputs.
+ */
+Relids
+varattnoset_intersect_relids(const VarAttnoSet *a, const VarAttnoSet *b)
+{
+ Relids result = NULL;
+
+ if (a == NULL || b == NULL)
+ return NULL;
+ Assert(a->max_varno == b->max_varno);
+ for (int i = 1; i <= a->max_varno; i++)
+ {
+ if (bms_overlap(a->varattnos[i], b->varattnos[i]))
+ result = bms_add_member(result, i);
+ }
+ return result;
+}
+
+
/*
* find_nonnullable_rels
* Determine which base rels are forced nonnullable by given clause.
@@ -1566,11 +1646,13 @@ find_nonnullable_rels_walker(Node *node, bool top_level)
* find_nonnullable_vars
* Determine which Vars are forced nonnullable by given clause.
*
- * Returns a list of all level-zero Vars that are referenced in the clause in
+ * Builds a set of all level-zero Vars that are referenced in the clause in
* such a way that the clause cannot possibly return TRUE if any of these Vars
* is NULL. (It is OK to err on the side of conservatism; hence the analysis
* here is simplistic.)
*
+ * Attnos of the identified Vars are added to a caller-supplied VarAttnoSet.
+ *
* The semantics here are subtly different from contain_nonstrict_functions:
* that function is concerned with NULL results from arbitrary expressions,
* but here we assume that the input is a Boolean expression, and wish to
@@ -1578,9 +1660,6 @@ find_nonnullable_rels_walker(Node *node, bool top_level)
* the expression to have been AND/OR flattened and converted to implicit-AND
* format.
*
- * The result is a palloc'd List, but we have not copied the member Var nodes.
- * Also, we don't bother trying to eliminate duplicate entries.
- *
* top_level is true while scanning top-level AND/OR structure; here, showing
* the result is either FALSE or NULL is good enough. top_level is false when
* we have descended below a NOT or a strict function: now we must be able to
@@ -1589,26 +1668,30 @@ find_nonnullable_rels_walker(Node *node, bool top_level)
* We don't use expression_tree_walker here because we don't want to descend
* through very many kinds of nodes; only the ones we can be sure are strict.
*/
-List *
-find_nonnullable_vars(Node *clause)
+void
+find_nonnullable_vars(Node *clause, VarAttnoSet *attnos)
{
- return find_nonnullable_vars_walker(clause, true);
+ return find_nonnullable_vars_walker(clause, attnos, true);
}
-static List *
-find_nonnullable_vars_walker(Node *node, bool top_level)
+static void
+find_nonnullable_vars_walker(Node *node, VarAttnoSet *attnos, bool top_level)
{
- List *result = NIL;
ListCell *l;
if (node == NULL)
- return NIL;
+ return;
if (IsA(node, Var))
{
Var *var = (Var *) node;
if (var->varlevelsup == 0)
- result = list_make1(var);
+ {
+ Assert(var->varno > 0 && var->varno <= attnos->max_varno);
+ attnos->varattnos[var->varno] =
+ bms_add_member(attnos->varattnos[var->varno],
+ var->varattno - FirstLowInvalidHeapAttributeNumber);
+ }
}
else if (IsA(node, List))
{
@@ -1623,9 +1706,7 @@ find_nonnullable_vars_walker(Node *node, bool top_level)
*/
foreach(l, (List *) node)
{
- result = list_concat(result,
- find_nonnullable_vars_walker(lfirst(l),
- top_level));
+ find_nonnullable_vars_walker(lfirst(l), attnos, top_level);
}
}
else if (IsA(node, FuncExpr))
@@ -1633,7 +1714,7 @@ find_nonnullable_vars_walker(Node *node, bool top_level)
FuncExpr *expr = (FuncExpr *) node;
if (func_strict(expr->funcid))
- result = find_nonnullable_vars_walker((Node *) expr->args, false);
+ find_nonnullable_vars_walker((Node *) expr->args, attnos, false);
}
else if (IsA(node, OpExpr))
{
@@ -1641,14 +1722,14 @@ find_nonnullable_vars_walker(Node *node, bool top_level)
set_opfuncid(expr);
if (func_strict(expr->opfuncid))
- result = find_nonnullable_vars_walker((Node *) expr->args, false);
+ find_nonnullable_vars_walker((Node *) expr->args, attnos, false);
}
else if (IsA(node, ScalarArrayOpExpr))
{
ScalarArrayOpExpr *expr = (ScalarArrayOpExpr *) node;
if (is_strict_saop(expr, true))
- result = find_nonnullable_vars_walker((Node *) expr->args, false);
+ find_nonnullable_vars_walker((Node *) expr->args, attnos, false);
}
else if (IsA(node, BoolExpr))
{
@@ -1657,11 +1738,16 @@ find_nonnullable_vars_walker(Node *node, bool top_level)
switch (expr->boolop)
{
case AND_EXPR:
- /* At top level we can just recurse (to the List case) */
+
+ /*
+ * At top level we can just recurse (to the List case), since
+ * the result should be the union of what we can prove in each
+ * arm.
+ */
if (top_level)
{
- result = find_nonnullable_vars_walker((Node *) expr->args,
- top_level);
+ find_nonnullable_vars_walker((Node *) expr->args, attnos,
+ top_level);
break;
}
@@ -1679,30 +1765,36 @@ find_nonnullable_vars_walker(Node *node, bool top_level)
* OR is strict if all of its arms are, so we can take the
* intersection of the sets of nonnullable vars for each arm.
* This works for both values of top_level.
+ *
+ * The pfree's below miss cleaning up individual bitmapsets in
+ * each VarAttnoSet. Doesn't seem worth working harder.
*/
- foreach(l, expr->args)
{
- List *subresult;
+ VarAttnoSet *int_attnos = NULL;
- subresult = find_nonnullable_vars_walker(lfirst(l),
- top_level);
- if (result == NIL) /* first subresult? */
- result = subresult;
- else
- result = list_intersection(result, subresult);
-
- /*
- * If the intersection is empty, we can stop looking. This
- * also justifies the test for first-subresult above.
- */
- if (result == NIL)
- break;
+ foreach(l, expr->args)
+ {
+ VarAttnoSet *sub_attnos;
+
+ sub_attnos = make_empty_varattnoset(attnos->max_varno);
+ find_nonnullable_vars_walker(lfirst(l), sub_attnos,
+ top_level);
+ if (int_attnos == NULL) /* first subresult? */
+ int_attnos = sub_attnos;
+ else
+ {
+ varattnoset_int_members(int_attnos, sub_attnos);
+ pfree(sub_attnos);
+ }
+ }
+ varattnoset_add_members(attnos, int_attnos);
+ pfree(int_attnos);
}
break;
case NOT_EXPR:
/* NOT will return null if its arg is null */
- result = find_nonnullable_vars_walker((Node *) expr->args,
- false);
+ find_nonnullable_vars_walker((Node *) expr->args, attnos,
+ false);
break;
default:
elog(ERROR, "unrecognized boolop: %d", (int) expr->boolop);
@@ -1713,34 +1805,34 @@ find_nonnullable_vars_walker(Node *node, bool top_level)
{
RelabelType *expr = (RelabelType *) node;
- result = find_nonnullable_vars_walker((Node *) expr->arg, top_level);
+ find_nonnullable_vars_walker((Node *) expr->arg, attnos, top_level);
}
else if (IsA(node, CoerceViaIO))
{
/* not clear this is useful, but it can't hurt */
CoerceViaIO *expr = (CoerceViaIO *) node;
- result = find_nonnullable_vars_walker((Node *) expr->arg, false);
+ find_nonnullable_vars_walker((Node *) expr->arg, attnos, false);
}
else if (IsA(node, ArrayCoerceExpr))
{
/* ArrayCoerceExpr is strict at the array level; ignore elemexpr */
ArrayCoerceExpr *expr = (ArrayCoerceExpr *) node;
- result = find_nonnullable_vars_walker((Node *) expr->arg, top_level);
+ find_nonnullable_vars_walker((Node *) expr->arg, attnos, top_level);
}
else if (IsA(node, ConvertRowtypeExpr))
{
/* not clear this is useful, but it can't hurt */
ConvertRowtypeExpr *expr = (ConvertRowtypeExpr *) node;
- result = find_nonnullable_vars_walker((Node *) expr->arg, top_level);
+ find_nonnullable_vars_walker((Node *) expr->arg, attnos, top_level);
}
else if (IsA(node, CollateExpr))
{
CollateExpr *expr = (CollateExpr *) node;
- result = find_nonnullable_vars_walker((Node *) expr->arg, top_level);
+ find_nonnullable_vars_walker((Node *) expr->arg, attnos, top_level);
}
else if (IsA(node, NullTest))
{
@@ -1748,7 +1840,7 @@ find_nonnullable_vars_walker(Node *node, bool top_level)
NullTest *expr = (NullTest *) node;
if (top_level && expr->nulltesttype == IS_NOT_NULL && !expr->argisrow)
- result = find_nonnullable_vars_walker((Node *) expr->arg, false);
+ find_nonnullable_vars_walker((Node *) expr->arg, attnos, false);
}
else if (IsA(node, BooleanTest))
{
@@ -1759,7 +1851,7 @@ find_nonnullable_vars_walker(Node *node, bool top_level)
(expr->booltesttype == IS_TRUE ||
expr->booltesttype == IS_FALSE ||
expr->booltesttype == IS_NOT_UNKNOWN))
- result = find_nonnullable_vars_walker((Node *) expr->arg, false);
+ find_nonnullable_vars_walker((Node *) expr->arg, attnos, false);
}
else if (IsA(node, SubPlan))
{
@@ -1768,15 +1860,14 @@ find_nonnullable_vars_walker(Node *node, bool top_level)
/* See analysis in find_nonnullable_rels_walker */
if ((top_level && splan->subLinkType == ANY_SUBLINK) ||
splan->subLinkType == ROWCOMPARE_SUBLINK)
- result = find_nonnullable_vars_walker(splan->testexpr, top_level);
+ find_nonnullable_vars_walker(splan->testexpr, attnos, top_level);
}
else if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
- result = find_nonnullable_vars_walker((Node *) phv->phexpr, top_level);
+ find_nonnullable_vars_walker((Node *) phv->phexpr, attnos, top_level);
}
- return result;
}
/*
@@ -1788,23 +1879,25 @@ find_nonnullable_vars_walker(Node *node, bool top_level)
* side of conservatism; hence the analysis here is simplistic. In fact,
* we only detect simple "var IS NULL" tests at the top level.)
*
- * The result is a palloc'd List, but we have not copied the member Var nodes.
- * Also, we don't bother trying to eliminate duplicate entries.
+ * As with find_nonnullable_vars, we add the varattnos of the identified Vars
+ * to a caller-provided VarAttnoSet.
*/
-List *
-find_forced_null_vars(Node *node)
+void
+find_forced_null_vars(Node *node, VarAttnoSet *attnos)
{
- List *result = NIL;
Var *var;
ListCell *l;
if (node == NULL)
- return NIL;
+ return;
/* Check single-clause cases using subroutine */
var = find_forced_null_var(node);
if (var)
{
- result = list_make1(var);
+ Assert(var->varno > 0 && var->varno <= attnos->max_varno);
+ attnos->varattnos[var->varno] =
+ bms_add_member(attnos->varattnos[var->varno],
+ var->varattno - FirstLowInvalidHeapAttributeNumber);
}
/* Otherwise, handle AND-conditions */
else if (IsA(node, List))
@@ -1815,8 +1908,7 @@ find_forced_null_vars(Node *node)
*/
foreach(l, (List *) node)
{
- result = list_concat(result,
- find_forced_null_vars(lfirst(l)));
+ find_forced_null_vars((Node *) lfirst(l), attnos);
}
}
else if (IsA(node, BoolExpr))
@@ -1831,10 +1923,9 @@ find_forced_null_vars(Node *node)
if (expr->boolop == AND_EXPR)
{
/* At top level we can just recurse (to the List case) */
- result = find_forced_null_vars((Node *) expr->args);
+ find_forced_null_vars((Node *) expr->args, attnos);
}
}
- return result;
}
/*
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index ff242d1b6d..5466ada7ba 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -23,6 +23,14 @@ typedef struct
List **windowFuncs; /* lists of WindowFuncs for each winref */
} WindowFuncLists;
+/* Data structure to represent all level-zero Vars meeting some condition */
+typedef struct
+{
+ int max_varno; /* maximum index in varattnos[] */
+ /* Attnos in these sets are offset by FirstLowInvalidHeapAttributeNumber */
+ Bitmapset *varattnos[FLEXIBLE_ARRAY_MEMBER];
+} VarAttnoSet;
+
extern bool contain_agg_clause(Node *clause);
extern bool contain_window_function(Node *clause);
@@ -38,9 +46,14 @@ extern bool contain_nonstrict_functions(Node *clause);
extern bool contain_exec_param(Node *clause, List *param_ids);
extern bool contain_leaked_vars(Node *clause);
+extern VarAttnoSet *make_empty_varattnoset(int rangetable_length);
+extern void varattnoset_add_members(VarAttnoSet *a, const VarAttnoSet *b);
+extern Relids varattnoset_intersect_relids(const VarAttnoSet *a,
+ const VarAttnoSet *b);
+
extern Relids find_nonnullable_rels(Node *clause);
-extern List *find_nonnullable_vars(Node *clause);
-extern List *find_forced_null_vars(Node *node);
+extern void find_nonnullable_vars(Node *clause, VarAttnoSet *attnos);
+extern void find_forced_null_vars(Node *node, VarAttnoSet *attnos);
extern Var *find_forced_null_var(Node *node);
extern bool is_pseudo_constant_clause(Node *clause);
commit 12ade1ae36621376b813f6bb5755ba466765e74c
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sat Nov 5 17:21:46 2022 -0400
Detect duplicated pushed-down conditions using RestrictInfo ID numbers.
create_nestloop_path needs to identify which candidates for join
restriction quals were already enforced in the parameterized inner
path. Currently we do that by relying on join_clause_is_movable_into
to give consistent answers, but that is not working very well with
variant clauses generated to satisfy outer join identity 3. We may
have a clause that (correctly) shows the outer-side Var as nulled by
a previous outer join, which makes it dependent on the nestloop outer
side having included that join, so that it appears to not be pushable
into a parameterized path that uses the un-nulled version of that Var.
Nonetheless, the cloned clause *is* redundant and we don't want
to check it again.
This patch offers a somewhat brute-force solution, which is to assign
serial numbers to RestrictInfo nodes, then check for redundancy using
serial number match rather than trusting join_clause_is_movable_into.
The variant-clause problem can be solved by allowing clauses to share
a serial number when we know that they are equivalent. Both the
outer-join variant generator and equivclass.c need to be in on that
trick in order to handle all cases that were handled well before.
It'd be nicer if we could continue to trust join_clause_is_movable_into
for this, but on the other hand this mechanism does provide a much more
concrete, harder-to-break way of verifying that we already enforced
(some version of) a qual. Any failure mode would almost certainly
be in the safe direction of enforcing a qual redundantly, which is
not a claim that the existing method can make.
This patch results in two changes to the core regression test outputs:
* One query in join.sql changes to a different join order. Examining
the cost estimates that are normally not shown, the new order is
estimated as very slightly faster, so this seems like an improvement.
I'm not quite sure why the old code did not find this join order.
* Some of the queries in partition_join.sql revert equivalence-clause
ordering back to what it was before a5fc46414. That's probably a
consequence of investigating parameterized paths in a different order
than before. Anyway, it's visibly harmless.
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 349e183372..d4f8b7893d 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -35,7 +35,8 @@
static EquivalenceMember *add_eq_member(EquivalenceClass *ec,
Expr *expr, Relids relids, Relids nullable_relids,
- bool is_child, Oid datatype);
+ EquivalenceMember *parent,
+ Oid datatype);
static bool is_exprlist_member(Expr *node, List *exprs);
static void generate_base_implied_equalities_const(PlannerInfo *root,
EquivalenceClass *ec);
@@ -400,7 +401,7 @@ process_equivalence(PlannerInfo *root,
{
/* Case 3: add item2 to ec1 */
em2 = add_eq_member(ec1, item2, item2_relids, item2_nullable_relids,
- false, item2_type);
+ NULL, item2_type);
ec1->ec_sources = lappend(ec1->ec_sources, restrictinfo);
ec1->ec_below_outer_join |= below_outer_join;
ec1->ec_min_security = Min(ec1->ec_min_security,
@@ -418,7 +419,7 @@ process_equivalence(PlannerInfo *root,
{
/* Case 3: add item1 to ec2 */
em1 = add_eq_member(ec2, item1, item1_relids, item1_nullable_relids,
- false, item1_type);
+ NULL, item1_type);
ec2->ec_sources = lappend(ec2->ec_sources, restrictinfo);
ec2->ec_below_outer_join |= below_outer_join;
ec2->ec_min_security = Min(ec2->ec_min_security,
@@ -452,9 +453,9 @@ process_equivalence(PlannerInfo *root,
ec->ec_max_security = restrictinfo->security_level;
ec->ec_merged = NULL;
em1 = add_eq_member(ec, item1, item1_relids, item1_nullable_relids,
- false, item1_type);
+ NULL, item1_type);
em2 = add_eq_member(ec, item2, item2_relids, item2_nullable_relids,
- false, item2_type);
+ NULL, item2_type);
root->eq_classes = lappend(root->eq_classes, ec);
@@ -544,7 +545,7 @@ canonicalize_ec_expression(Expr *expr, Oid req_type, Oid req_collation)
*/
static EquivalenceMember *
add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
- Relids nullable_relids, bool is_child, Oid datatype)
+ Relids nullable_relids, EquivalenceMember *parent, Oid datatype)
{
EquivalenceMember *em = makeNode(EquivalenceMember);
@@ -552,8 +553,9 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
em->em_relids = relids;
em->em_nullable_relids = nullable_relids;
em->em_is_const = false;
- em->em_is_child = is_child;
+ em->em_is_child = (parent != NULL);
em->em_datatype = datatype;
+ em->em_parent = parent;
if (bms_is_empty(relids))
{
@@ -565,12 +567,12 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
* get_eclass_for_sort_expr() has to work harder. We put the tests
* there not here to save cycles in the equivalence case.
*/
- Assert(!is_child);
+ Assert(!parent);
em->em_is_const = true;
ec->ec_has_const = true;
/* it can't affect ec_relids */
}
- else if (!is_child) /* child members don't add to ec_relids */
+ else if (!parent) /* child members don't add to ec_relids */
{
ec->ec_relids = bms_add_members(ec->ec_relids, relids);
}
@@ -723,7 +725,7 @@ get_eclass_for_sort_expr(PlannerInfo *root,
nullable_relids = bms_intersect(nullable_relids, expr_relids);
newem = add_eq_member(newec, copyObject(expr), expr_relids,
- nullable_relids, false, opcintype);
+ nullable_relids, NULL, opcintype);
/*
* add_eq_member doesn't check for volatile functions, set-returning
@@ -1821,6 +1823,7 @@ create_join_clause(PlannerInfo *root,
EquivalenceClass *parent_ec)
{
RestrictInfo *rinfo;
+ RestrictInfo *parent_rinfo = NULL;
ListCell *lc;
MemoryContext oldcontext;
@@ -1865,6 +1868,20 @@ create_join_clause(PlannerInfo *root,
*/
oldcontext = MemoryContextSwitchTo(root->planner_cxt);
+ /*
+ * If either EM is a child, recursively create the corresponding
+ * parent-to-parent clause, so that we can duplicate its rinfo_serial.
+ */
+ if (leftem->em_is_child || rightem->em_is_child)
+ {
+ EquivalenceMember *leftp = leftem->em_parent ? leftem->em_parent : leftem;
+ EquivalenceMember *rightp = rightem->em_parent ? rightem->em_parent : rightem;
+
+ parent_rinfo = create_join_clause(root, ec, opno,
+ leftp, rightp,
+ parent_ec);
+ }
+
rinfo = build_implied_join_equality(root,
opno,
ec->ec_collation,
@@ -1876,6 +1893,10 @@ create_join_clause(PlannerInfo *root,
rightem->em_nullable_relids),
ec->ec_min_security);
+ /* If it's a child clause, copy the parent's rinfo_serial */
+ if (parent_rinfo)
+ rinfo->rinfo_serial = parent_rinfo->rinfo_serial;
+
/* Mark the clause as redundant, or not */
rinfo->parent_ec = parent_ec;
@@ -2686,7 +2707,7 @@ add_child_rel_equivalences(PlannerInfo *root,
(void) add_eq_member(cur_ec, child_expr,
new_relids, new_nullable_relids,
- true, cur_em->em_datatype);
+ cur_em, cur_em->em_datatype);
/* Record this EC index for the child rel */
child_rel->eclass_indexes = bms_add_member(child_rel->eclass_indexes, i);
@@ -2827,7 +2848,7 @@ add_child_join_rel_equivalences(PlannerInfo *root,
(void) add_eq_member(cur_ec, child_expr,
new_relids, new_nullable_relids,
- true, cur_em->em_datatype);
+ cur_em, cur_em->em_datatype);
}
}
}
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 41c69b29a7..96e8033930 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -1785,6 +1785,7 @@ process_postponed_left_join_quals(PlannerInfo *root)
Relids joins_below;
Relids joins_so_far;
List *quals;
+ int save_last_rinfo_serial;
ListCell *lc2;
/*
@@ -1823,6 +1824,16 @@ process_postponed_left_join_quals(PlannerInfo *root)
joins_below,
NULL);
+ /*
+ * Each time we produce RestrictInfo(s) from these quals, reset
+ * the last_rinfo_serial counter, so that the RestrictInfos for
+ * the "same" qual condition get identical serial numbers. (This
+ * relies on the fact that we're not changing the qual list in any
+ * way that'd affect the number of RestrictInfos built from it.)
+ * This'll allow us to detect duplicative qual usage later.
+ */
+ save_last_rinfo_serial = root->last_rinfo_serial;
+
joins_so_far = NULL;
foreach(lc2, join_info_list_orig)
{
@@ -1856,6 +1867,9 @@ process_postponed_left_join_quals(PlannerInfo *root)
continue;
}
+ /* Reset serial counter for this version of the quals */
+ root->last_rinfo_serial = save_last_rinfo_serial;
+
/*
* When we are looking at joins above sjinfo, we are
* envisioning pushing sjinfo to above othersj, so add
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index e743a5d9fe..d52c2a3595 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -620,6 +620,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
root->multiexpr_params = NIL;
root->eq_classes = NIL;
root->ec_merging_done = false;
+ root->last_rinfo_serial = 0;
root->all_result_relids =
parse->resultRelation ? bms_make_singleton(parse->resultRelation) : NULL;
root->leaf_result_relids = NULL; /* we'll find out leaf-ness later */
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index ce029dbf69..0a327e1d71 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -989,6 +989,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
subroot->multiexpr_params = NIL;
subroot->eq_classes = NIL;
subroot->ec_merging_done = false;
+ subroot->last_rinfo_serial = 0;
subroot->all_result_relids = NULL;
subroot->leaf_result_relids = NULL;
subroot->append_rel_list = NIL;
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index 11c6bbaba6..e18d64b6dc 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -427,7 +427,7 @@ adjust_appendrel_attrs_mutator(Node *node,
RestrictInfo *oldinfo = (RestrictInfo *) node;
RestrictInfo *newinfo = makeNode(RestrictInfo);
- /* Copy all flat-copiable fields */
+ /* Copy all flat-copiable fields, notably including rinfo_serial */
memcpy(newinfo, oldinfo, sizeof(RestrictInfo));
/* Recursively fix the clause itself */
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index bf35d1989c..c77399ca92 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2442,12 +2442,12 @@ create_nestloop_path(PlannerInfo *root,
* restrict_clauses that are due to be moved into the inner path. We have
* to do this now, rather than postpone the work till createplan time,
* because the restrict_clauses list can affect the size and cost
- * estimates for this path.
+ * estimates for this path. We detect such clauses by checking for serial
+ * number match to clauses already enforced in the inner path.
*/
if (bms_overlap(inner_req_outer, outer_path->parent->relids))
{
- Relids inner_and_outer = bms_union(inner_path->parent->relids,
- inner_req_outer);
+ Bitmapset *enforced_serials = get_param_path_clause_serials(inner_path);
List *jclauses = NIL;
ListCell *lc;
@@ -2455,9 +2455,7 @@ create_nestloop_path(PlannerInfo *root,
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
- if (!join_clause_is_movable_into(rinfo,
- inner_path->parent->relids,
- inner_and_outer))
+ if (!bms_is_member(rinfo->rinfo_serial, enforced_serials))
jclauses = lappend(jclauses, rinfo);
}
restrict_clauses = jclauses;
@@ -4268,6 +4266,7 @@ do { \
new_ppi->ppi_rows = old_ppi->ppi_rows;
new_ppi->ppi_clauses = old_ppi->ppi_clauses;
ADJUST_CHILD_ATTRS(new_ppi->ppi_clauses);
+ new_ppi->ppi_serials = bms_copy(old_ppi->ppi_serials);
rel->ppilist = lappend(rel->ppilist, new_ppi);
MemoryContextSwitchTo(oldcontext);
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 226914498a..cea298c633 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -1471,6 +1471,7 @@ get_baserel_parampathinfo(PlannerInfo *root, RelOptInfo *baserel,
ParamPathInfo *ppi;
Relids joinrelids;
List *pclauses;
+ Bitmapset *pserials;
double rows;
ListCell *lc;
@@ -1513,6 +1514,15 @@ get_baserel_parampathinfo(PlannerInfo *root, RelOptInfo *baserel,
required_outer,
baserel));
+ /* Compute set of serial numbers of the enforced clauses */
+ pserials = NULL;
+ foreach(lc, pclauses)
+ {
+ RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
+
+ pserials = bms_add_member(pserials, rinfo->rinfo_serial);
+ }
+
/* Estimate the number of rows returned by the parameterized scan */
rows = get_parameterized_baserel_size(root, baserel, pclauses);
@@ -1521,6 +1531,7 @@ get_baserel_parampathinfo(PlannerInfo *root, RelOptInfo *baserel,
ppi->ppi_req_outer = required_outer;
ppi->ppi_rows = rows;
ppi->ppi_clauses = pclauses;
+ ppi->ppi_serials = pserials;
baserel->ppilist = lappend(baserel->ppilist, ppi);
return ppi;
@@ -1746,6 +1757,7 @@ get_joinrel_parampathinfo(PlannerInfo *root, RelOptInfo *joinrel,
ppi->ppi_req_outer = required_outer;
ppi->ppi_rows = rows;
ppi->ppi_clauses = NIL;
+ ppi->ppi_serials = NULL;
joinrel->ppilist = lappend(joinrel->ppilist, ppi);
return ppi;
@@ -1784,6 +1796,7 @@ get_appendrel_parampathinfo(RelOptInfo *appendrel, Relids required_outer)
ppi->ppi_req_outer = required_outer;
ppi->ppi_rows = 0;
ppi->ppi_clauses = NIL;
+ ppi->ppi_serials = NULL;
appendrel->ppilist = lappend(appendrel->ppilist, ppi);
return ppi;
@@ -1809,6 +1822,100 @@ find_param_path_info(RelOptInfo *rel, Relids required_outer)
return NULL;
}
+/*
+ * get_param_path_clause_serials
+ * Given a parameterized Path, return the set of pushed-down clauses
+ * (identified by rinfo_serial numbers) enforced within the Path.
+ */
+Bitmapset *
+get_param_path_clause_serials(Path *path)
+{
+ if (path->param_info == NULL)
+ return NULL; /* not parameterized */
+ if (IsA(path, NestPath) ||
+ IsA(path, MergePath) ||
+ IsA(path, HashPath))
+ {
+ /*
+ * For a join path, combine clauses enforced within either input path
+ * with those enforced as joinrestrictinfo in this path. Note that
+ * joinrestrictinfo may include some non-pushed-down clauses, but for
+ * current purposes it's okay if we include those in the result. (To
+ * be more careful, we could check for clause_relids overlapping the
+ * path parameterization, but it's not worth the cycles for now.)
+ */
+ JoinPath *jpath = (JoinPath *) path;
+ Bitmapset *pserials;
+ ListCell *lc;
+
+ pserials = NULL;
+ pserials = bms_add_members(pserials,
+ get_param_path_clause_serials(jpath->outerjoinpath));
+ pserials = bms_add_members(pserials,
+ get_param_path_clause_serials(jpath->innerjoinpath));
+ foreach(lc, jpath->joinrestrictinfo)
+ {
+ RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
+
+ pserials = bms_add_member(pserials, rinfo->rinfo_serial);
+ }
+ return pserials;
+ }
+ else if (IsA(path, AppendPath))
+ {
+ /*
+ * For an appendrel, take the intersection of the sets of clauses
+ * enforced in each input path.
+ */
+ AppendPath *apath = (AppendPath *) path;
+ Bitmapset *pserials;
+ ListCell *lc;
+
+ pserials = NULL;
+ foreach(lc, apath->subpaths)
+ {
+ Path *subpath = (Path *) lfirst(lc);
+ Bitmapset *subserials;
+
+ subserials = get_param_path_clause_serials(subpath);
+ if (lc == list_head(apath->subpaths))
+ pserials = bms_copy(subserials);
+ else
+ pserials = bms_int_members(pserials, subserials);
+ }
+ return pserials;
+ }
+ else if (IsA(path, MergeAppendPath))
+ {
+ /* Same as AppendPath case */
+ MergeAppendPath *apath = (MergeAppendPath *) path;
+ Bitmapset *pserials;
+ ListCell *lc;
+
+ pserials = NULL;
+ foreach(lc, apath->subpaths)
+ {
+ Path *subpath = (Path *) lfirst(lc);
+ Bitmapset *subserials;
+
+ subserials = get_param_path_clause_serials(subpath);
+ if (lc == list_head(apath->subpaths))
+ pserials = bms_copy(subserials);
+ else
+ pserials = bms_int_members(pserials, subserials);
+ }
+ return pserials;
+ }
+ else
+ {
+ /*
+ * Otherwise, it's a baserel path and we can use the
+ * previously-computed set of serial numbers.
+ */
+ return path->param_info->ppi_serials;
+ }
+}
+
/*
* build_joinrel_partition_info
* Checks if the two relations being joined can use partitionwise join
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index 327c3ba563..bcbee8f943 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -208,6 +208,11 @@ make_restrictinfo_internal(PlannerInfo *root,
restrictinfo->num_base_rels = bms_num_members(baserels);
bms_free(baserels);
+ /*
+ * Label this RestrictInfo with a fresh serial number.
+ */
+ restrictinfo->rinfo_serial = ++(root->last_rinfo_serial);
+
/*
* Fill in all the cacheable fields with "not yet set" markers. None of
* these will be computed until/unless needed. Note in particular that we
@@ -371,7 +376,7 @@ commute_restrictinfo(RestrictInfo *rinfo, Oid comm_op)
* ... and adjust those we need to change. Note in particular that we can
* preserve any cached selectivity or cost estimates, since those ought to
* be the same for the new clause. Likewise we can keep the source's
- * parent_ec.
+ * parent_ec. It's also important that we keep the same rinfo_serial.
*/
result->clause = (Expr *) newclause;
result->left_relids = rinfo->right_relids;
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index a734676293..9d89b0c9eb 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -339,6 +339,9 @@ struct PlannerInfo
/* list of SpecialJoinInfos */
List *join_info_list;
+ /* counter for assigning RestrictInfo serial numbers */
+ int last_rinfo_serial;
+
/*
* all_result_relids is empty for SELECT, otherwise it contains at least
* parse->resultRelation. For UPDATE/DELETE/MERGE across an inheritance
@@ -1356,6 +1359,8 @@ typedef struct EquivalenceMember
bool em_is_const; /* expression is pseudoconstant? */
bool em_is_child; /* derived version for a child relation? */
Oid em_datatype; /* the "nominal type" used by the opfamily */
+ /* if em_is_child is true, this links to corresponding EM for top parent */
+ struct EquivalenceMember *em_parent pg_node_attr(read_write_ignore);
} EquivalenceMember;
/*
@@ -1461,7 +1466,13 @@ typedef struct PathTarget
* Note: ppi_clauses is only used in ParamPathInfos for base relation paths;
* in join cases it's NIL because the set of relevant clauses varies depending
* on how the join is formed. The relevant clauses will appear in each
- * parameterized join path's joinrestrictinfo list, instead.
+ * parameterized join path's joinrestrictinfo list, instead. ParamPathInfos
+ * for append relations don't bother with this, either.
+ *
+ * ppi_serials is the set of rinfo_serial numbers for quals that are enforced
+ * by this path. As with ppi_clauses, it's only maintained for baserels.
+ * (We could construct it on-the-fly from ppi_clauses, but it seems better
+ * to materialize a copy.)
*/
typedef struct ParamPathInfo
{
@@ -1472,6 +1483,7 @@ typedef struct ParamPathInfo
Relids ppi_req_outer; /* rels supplying parameters used by path */
Cardinality ppi_rows; /* estimated number of result tuples */
List *ppi_clauses; /* join clauses available from outer rels */
+ Bitmapset *ppi_serials; /* set of rinfo_serial for enforced quals */
} ParamPathInfo;
@@ -2501,6 +2513,25 @@ typedef struct RestrictInfo
*/
Expr *orclause pg_node_attr(equal_ignore);
+ /*----------
+ * Serial number of this RestrictInfo. This is unique within the current
+ * PlannerInfo context, with a few critical exceptions:
+ * 1. When we generate multiple clones of the same qual condition to
+ * cope with outer join identity 3, all the clones get the same serial
+ * number. This reflects that we only want to apply one of them in any
+ * given plan.
+ * 2. If we manufacture a commuted version of a qual to use as an index
+ * condition, it copies the original's rinfo_serial, since it is in
+ * practice the same condition.
+ * 3. RestrictInfos made for a child relation copy their parent's
+ * rinfo_serial. Likewise, when an EquivalenceClass makes a derived
+ * equality clause for a child relation, it copies the rinfo_serial of
+ * the matching equality clause for the parent. This allows detection
+ * of redundant pushed-down equality clauses.
+ *----------
+ */
+ int rinfo_serial;
+
/*
* Generating EquivalenceClass. This field is NULL unless clause is
* potentially redundant.
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 197234d44c..3440455a2e 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -333,6 +333,7 @@ extern ParamPathInfo *get_appendrel_parampathinfo(RelOptInfo *appendrel,
Relids required_outer);
extern ParamPathInfo *find_param_path_info(RelOptInfo *rel,
Relids required_outer);
+extern Bitmapset *get_param_path_clause_serials(Path *path);
extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out
index 9358371072..00f4c58238 100644
--- a/src/test/regress/expected/join.out
+++ b/src/test/regress/expected/join.out
@@ -2335,17 +2335,17 @@ select a.f1, b.f1, t.thousand, t.tenthous from
(select sum(f1)+1 as f1 from int4_tbl i4a) a,
(select sum(f1) as f1 from int4_tbl i4b) b
where b.f1 = t.thousand and a.f1 = b.f1 and (a.f1+b.f1+999) = t.tenthous;
- QUERY PLAN
------------------------------------------------------------------------------------------------------------------------
+ QUERY PLAN
+-----------------------------------------------------------------------------------------------------------------
Nested Loop
- -> Aggregate
- -> Seq Scan on int4_tbl i4b
-> Nested Loop
Join Filter: ((sum(i4b.f1)) = ((sum(i4a.f1) + 1)))
-> Aggregate
-> Seq Scan on int4_tbl i4a
- -> Index Only Scan using tenk1_thous_tenthous on tenk1 t
- Index Cond: ((thousand = (sum(i4b.f1))) AND (tenthous = ((((sum(i4a.f1) + 1)) + (sum(i4b.f1))) + 999)))
+ -> Aggregate
+ -> Seq Scan on int4_tbl i4b
+ -> Index Only Scan using tenk1_thous_tenthous on tenk1 t
+ Index Cond: ((thousand = (sum(i4b.f1))) AND (tenthous = ((((sum(i4a.f1) + 1)) + (sum(i4b.f1))) + 999)))
(9 rows)
select a.f1, b.f1, t.thousand, t.tenthous from
diff --git a/src/test/regress/expected/partition_join.out b/src/test/regress/expected/partition_join.out
index b20facc19f..bb5b7c47a4 100644
--- a/src/test/regress/expected/partition_join.out
+++ b/src/test/regress/expected/partition_join.out
@@ -304,7 +304,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t2.b FROM prt2 t2 WHERE t2.a = 0)
-> Seq Scan on prt2_p2 t2_2
Filter: (a = 0)
-> Nested Loop Semi Join
- Join Filter: (t2_3.b = t1_3.a)
+ Join Filter: (t1_3.a = t2_3.b)
-> Seq Scan on prt1_p3 t1_3
Filter: (b = 0)
-> Materialize
@@ -601,7 +601,7 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM prt1 t1, prt2 t2, prt1_e t
Sort Key: t1.a
-> Append
-> Nested Loop
- Join Filter: (((t3_1.a + t3_1.b) / 2) = t1_1.a)
+ Join Filter: (t1_1.a = ((t3_1.a + t3_1.b) / 2))
-> Hash Join
Hash Cond: (t2_1.b = t1_1.a)
-> Seq Scan on prt2_p1 t2_1
@@ -611,7 +611,7 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM prt1 t1, prt2 t2, prt1_e t
-> Index Scan using iprt1_e_p1_ab2 on prt1_e_p1 t3_1
Index Cond: (((a + b) / 2) = t2_1.b)
-> Nested Loop
- Join Filter: (((t3_2.a + t3_2.b) / 2) = t1_2.a)
+ Join Filter: (t1_2.a = ((t3_2.a + t3_2.b) / 2))
-> Hash Join
Hash Cond: (t2_2.b = t1_2.a)
-> Seq Scan on prt2_p2 t2_2
@@ -621,7 +621,7 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM prt1 t1, prt2 t2, prt1_e t
-> Index Scan using iprt1_e_p2_ab2 on prt1_e_p2 t3_2
Index Cond: (((a + b) / 2) = t2_2.b)
-> Nested Loop
- Join Filter: (((t3_3.a + t3_3.b) / 2) = t1_3.a)
+ Join Filter: (t1_3.a = ((t3_3.a + t3_3.b) / 2))
-> Hash Join
Hash Cond: (t2_3.b = t1_3.a)
-> Seq Scan on prt2_p3 t2_3
@@ -926,7 +926,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t1.b FROM prt2 t1, prt1_e t2 WHER
Sort Key: t1.a
-> Append
-> Nested Loop
- Join Filter: (t1_5.b = t1_2.a)
+ Join Filter: (t1_2.a = t1_5.b)
-> HashAggregate
Group Key: t1_5.b
-> Hash Join
@@ -939,7 +939,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t1.b FROM prt2 t1, prt1_e t2 WHER
Index Cond: (a = ((t2_1.a + t2_1.b) / 2))
Filter: (b = 0)
-> Nested Loop
- Join Filter: (t1_6.b = t1_3.a)
+ Join Filter: (t1_3.a = t1_6.b)
-> HashAggregate
Group Key: t1_6.b
-> Hash Join
@@ -952,7 +952,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t1.b FROM prt2 t1, prt1_e t2 WHER
Index Cond: (a = ((t2_2.a + t2_2.b) / 2))
Filter: (b = 0)
-> Nested Loop
- Join Filter: (t1_7.b = t1_4.a)
+ Join Filter: (t1_4.a = t1_7.b)
-> HashAggregate
Group Key: t1_7.b
-> Nested Loop
commit 4e2aa1ee681f751928083d50a9488a23703be16d
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sat Nov 5 17:25:11 2022 -0400
Fix flatten_join_alias_vars() to handle varnullingrels correctly.
The remaining core regression test failures occur because
flatten_join_alias_vars() isn't doing the right thing. The
alias Var it needs to replace may have acquired varnullingrels
bits signifying the effect of upper outer joins, and if so we
must preserve that information in the replacement expression.
The simplest way to do that is to wrap the replacement expression
in a PlaceHolderVar, and that's what we have to do in the general
case where subquery pullup has mutated the replacement joinaliasvars
entry into an arbitrary expression. But in simpler cases, such as
where the joinaliasvars entry is just a Var, we'd prefer to do it
by merging the alias Var's varnullingrels into the replacement Var.
In that way the flattened alias will compare equal() to semantically
equivalent references that didn't use the alias name.
Moreover, the parser also uses this code while checking certain
semantic constraints, and in that context we *must not* generate
PlaceHolderVars. PHVs shouldn't appear in parse-time expressions,
and adding one would certainly cause the parser to decide the
query is invalid (because the result wouldn't compare equal() to
what it needs to). Fortunately, during parsing the set of possible
contents of a joinaliasvars entry is quite constrained, so we can
guarantee to apply the nullingrels info to the Vars therein.
The result of this step passes all core regression tests, but there
are still loose ends for FDWs (so that contrib/postgres_fdw will fail).
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index d52c2a3595..dc089306ae 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -901,7 +901,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
*/
if (rte->lateral && root->hasJoinRTEs)
rte->subquery = (Query *)
- flatten_join_alias_vars(root->parse,
+ flatten_join_alias_vars(root, root->parse,
(Node *) rte->subquery);
}
else if (rte->rtekind == RTE_FUNCTION)
@@ -1102,7 +1102,7 @@ preprocess_expression(PlannerInfo *root, Node *expr, int kind)
kind == EXPRKIND_VALUES ||
kind == EXPRKIND_TABLESAMPLE ||
kind == EXPRKIND_TABLEFUNC))
- expr = flatten_join_alias_vars(root->parse, expr);
+ expr = flatten_join_alias_vars(root, root->parse, expr);
/*
* Simplify constant expressions. For function RTEs, this was already
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 0a327e1d71..74fcca3d9a 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -1077,7 +1077,8 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* maybe even in the rewriter; but for now let's just fix this case here.)
*/
subquery->targetList = (List *)
- flatten_join_alias_vars(subroot->parse, (Node *) subquery->targetList);
+ flatten_join_alias_vars(subroot, subroot->parse,
+ (Node *) subquery->targetList);
/*
* Adjust level-0 varnos in subquery so that we can append its rangetable
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
index 8d8c9136f8..69c2019553 100644
--- a/src/backend/optimizer/util/var.c
+++ b/src/backend/optimizer/util/var.c
@@ -62,6 +62,7 @@ typedef struct
typedef struct
{
+ PlannerInfo *root; /* could be NULL! */
Query *query; /* outer Query */
int sublevels_up;
bool possible_sublink; /* could aliases include a SubLink? */
@@ -80,6 +81,10 @@ static bool pull_var_clause_walker(Node *node,
pull_var_clause_context *context);
static Node *flatten_join_alias_vars_mutator(Node *node,
flatten_join_alias_vars_context *context);
+static Node *add_nullingrels_if_needed(PlannerInfo *root, Node *newnode,
+ Var *oldvar);
+static bool is_standard_join_alias_expression(Node *newnode, Var *oldvar);
+static void adjust_standard_join_alias_expression(Node *newnode, Var *oldvar);
static Relids alias_relid_set(Query *query, Relids relids);
@@ -722,26 +727,42 @@ pull_var_clause_walker(Node *node, pull_var_clause_context *context)
* is the only way that the executor can directly handle whole-row Vars.
*
* This also adjusts relid sets found in some expression node types to
- * substitute the contained base rels for any join relid.
+ * substitute the contained base+OJ rels for any join relid.
*
* If a JOIN contains sub-selects that have been flattened, its join alias
* entries might now be arbitrary expressions, not just Vars. This affects
- * this function in one important way: we might find ourselves inserting
- * SubLink expressions into subqueries, and we must make sure that their
- * Query.hasSubLinks fields get set to true if so. If there are any
+ * this function in two important ways. First, we might find ourselves
+ * inserting SubLink expressions into subqueries, and we must make sure that
+ * their Query.hasSubLinks fields get set to true if so. If there are any
* SubLinks in the join alias lists, the outer Query should already have
* hasSubLinks = true, so this is only relevant to un-flattened subqueries.
+ * Second, we have to preserve any varnullingrels info attached to the
+ * alias Vars we're replacing. If the replacement expression is a Var or
+ * PlaceHolderVar or constructed from those, we can just add the
+ * varnullingrels bits to the existing nullingrels field(s); otherwise
+ * we have to add a PlaceHolderVar wrapper.
*
- * NOTE: this is used on not-yet-planned expressions. We do not expect it
- * to be applied directly to the whole Query, so if we see a Query to start
- * with, we do want to increment sublevels_up (this occurs for LATERAL
- * subqueries).
+ * NOTE: this is also used by the parser, to expand join alias Vars before
+ * checking GROUP BY validity. For that use-case, root will be NULL, which
+ * is why we have to pass the Query separately. We need the root itself only
+ * for making PlaceHolderVars. We can avoid making PlaceHolderVars in the
+ * parser's usage because it won't be dealing with arbitrary expressions:
+ * so long as adjust_standard_join_alias_expression can handle everything
+ * the parser would make as a join alias expression, we're OK.
*/
Node *
-flatten_join_alias_vars(Query *query, Node *node)
+flatten_join_alias_vars(PlannerInfo *root, Query *query, Node *node)
{
flatten_join_alias_vars_context context;
+ /*
+ * We do not expect this to be applied to the whole Query, only to
+ * expressions or LATERAL subqueries. Hence, if the top node is a Query,
+ * it's okay to immediately increment sublevels_up.
+ */
+ Assert(node != (Node *) query);
+
+ context.root = root;
context.query = query;
context.sublevels_up = 0;
/* flag whether join aliases could possibly contain SubLinks */
@@ -812,7 +833,9 @@ flatten_join_alias_vars_mutator(Node *node,
rowexpr->colnames = colnames;
rowexpr->location = var->location;
- return (Node *) rowexpr;
+ /* Lastly, add any varnullingrels to the replacement expression */
+ return add_nullingrels_if_needed(context->root, (Node *) rowexpr,
+ var);
}
/* Expand join alias reference */
@@ -839,7 +862,8 @@ flatten_join_alias_vars_mutator(Node *node,
if (context->possible_sublink && !context->inserted_sublink)
context->inserted_sublink = checkExprHasSubLink(newvar);
- return newvar;
+ /* Lastly, add any varnullingrels to the replacement expression */
+ return add_nullingrels_if_needed(context->root, newvar, var);
}
if (IsA(node, PlaceHolderVar))
{
@@ -854,6 +878,7 @@ flatten_join_alias_vars_mutator(Node *node,
{
phv->phrels = alias_relid_set(context->query,
phv->phrels);
+ /* we *don't* change phnullingrels */
}
return (Node *) phv;
}
@@ -887,9 +912,145 @@ flatten_join_alias_vars_mutator(Node *node,
(void *) context);
}
+/*
+ * Add oldvar's varnullingrels, if any, to a flattened join alias expression.
+ * The newnode has been copied, so we can modify it freely.
+ */
+static Node *
+add_nullingrels_if_needed(PlannerInfo *root, Node *newnode, Var *oldvar)
+{
+ if (oldvar->varnullingrels == NULL)
+ return newnode; /* nothing to do */
+ /* If possible, do it by adding to existing nullingrel fields */
+ if (is_standard_join_alias_expression(newnode, oldvar))
+ adjust_standard_join_alias_expression(newnode, oldvar);
+ else if (root)
+ {
+ /* We can insert a PlaceHolderVar to carry the nullingrels */
+ PlaceHolderVar *newphv;
+ Relids phrels = pull_varnos(root, newnode);
+
+ /* XXX what if phrels is empty? */
+ Assert(!bms_is_empty(phrels)); /* probably wrong */
+ newphv = make_placeholder_expr(root, (Expr *) newnode, phrels);
+ /* newphv has zero phlevelsup and NULL phnullingrels; fix it */
+ newphv->phlevelsup = oldvar->varlevelsup;
+ newphv->phnullingrels = bms_copy(oldvar->varnullingrels);
+ newnode = (Node *) newphv;
+ }
+ else
+ {
+ /* ooops, we're missing support for something the parser can make */
+ elog(ERROR, "unsupported join alias expression");
+ }
+ return newnode;
+}
+
+/*
+ * Check to see if we can insert nullingrels into this join alias expression
+ * without use of a separate PlaceHolderVar.
+ *
+ * This will handle Vars, PlaceHolderVars, and implicit-coercion and COALESCE
+ * expressions built from those. This coverage needs to handle anything
+ * that the parser would put into joinaliasvars.
+ * XXX it's probably incomplete at the moment.
+ */
+static bool
+is_standard_join_alias_expression(Node *newnode, Var *oldvar)
+{
+ if (newnode == NULL)
+ return false;
+ if (IsA(newnode, Var) &&
+ ((Var *) newnode)->varlevelsup == oldvar->varlevelsup)
+ return true;
+ else if (IsA(newnode, PlaceHolderVar) &&
+ ((PlaceHolderVar *) newnode)->phlevelsup == oldvar->varlevelsup)
+ return true;
+ else if (IsA(newnode, FuncExpr))
+ {
+ FuncExpr *fexpr = (FuncExpr *) newnode;
+
+ /*
+ * We need to assume that the function wouldn't produce non-NULL from
+ * NULL, which is reasonable for implicit coercions but otherwise not
+ * so much. (Looking at its strictness is likely overkill, and anyway
+ * it would cause us to fail if someone forgot to mark an implicit
+ * coercion as strict.)
+ */
+ if (fexpr->funcformat != COERCE_IMPLICIT_CAST ||
+ fexpr->args == NIL)
+ return false;
+
+ /*
+ * Examine only the first argument --- coercions might have additional
+ * arguments that are constants.
+ */
+ return is_standard_join_alias_expression(linitial(fexpr->args), oldvar);
+ }
+ else if (IsA(newnode, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) newnode;
+ ListCell *lc;
+
+ Assert(cexpr->args != NIL);
+ foreach(lc, cexpr->args)
+ {
+ if (!is_standard_join_alias_expression(lfirst(lc), oldvar))
+ return false;
+ }
+ return true;
+ }
+ else
+ return false;
+}
+
+/*
+ * Insert nullingrels into an expression accepted by
+ * is_standard_join_alias_expression.
+ */
+static void
+adjust_standard_join_alias_expression(Node *newnode, Var *oldvar)
+{
+ if (IsA(newnode, Var) &&
+ ((Var *) newnode)->varlevelsup == oldvar->varlevelsup)
+ {
+ Var *newvar = (Var *) newnode;
+
+ newvar->varnullingrels = bms_add_members(newvar->varnullingrels,
+ oldvar->varnullingrels);
+ }
+ else if (IsA(newnode, PlaceHolderVar) &&
+ ((PlaceHolderVar *) newnode)->phlevelsup == oldvar->varlevelsup)
+ {
+ PlaceHolderVar *newphv = (PlaceHolderVar *) newnode;
+
+ newphv->phnullingrels = bms_add_members(newphv->phnullingrels,
+ oldvar->varnullingrels);
+ }
+ else if (IsA(newnode, FuncExpr))
+ {
+ FuncExpr *fexpr = (FuncExpr *) newnode;
+
+ adjust_standard_join_alias_expression(linitial(fexpr->args), oldvar);
+ }
+ else if (IsA(newnode, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) newnode;
+ ListCell *lc;
+
+ Assert(cexpr->args != NIL);
+ foreach(lc, cexpr->args)
+ {
+ adjust_standard_join_alias_expression(lfirst(lc), oldvar);
+ }
+ }
+ else
+ Assert(false);
+}
+
/*
* alias_relid_set: in a set of RT indexes, replace joins by their
- * underlying base relids
+ * underlying base+OJ relids
*/
static Relids
alias_relid_set(Query *query, Relids relids)
diff --git a/src/backend/parser/parse_agg.c b/src/backend/parser/parse_agg.c
index 3ef9e8ee5e..c15fab0f68 100644
--- a/src/backend/parser/parse_agg.c
+++ b/src/backend/parser/parse_agg.c
@@ -1162,7 +1162,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
* entries are RTE_JOIN kind.
*/
if (hasJoinRTEs)
- groupClauses = (List *) flatten_join_alias_vars(qry,
+ groupClauses = (List *) flatten_join_alias_vars(NULL, qry,
(Node *) groupClauses);
/*
@@ -1206,7 +1206,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
groupClauses, hasJoinRTEs,
have_non_var_grouping);
if (hasJoinRTEs)
- clause = flatten_join_alias_vars(qry, clause);
+ clause = flatten_join_alias_vars(NULL, qry, clause);
check_ungrouped_columns(clause, pstate, qry,
groupClauses, groupClauseCommonVars,
have_non_var_grouping,
@@ -1217,7 +1217,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
groupClauses, hasJoinRTEs,
have_non_var_grouping);
if (hasJoinRTEs)
- clause = flatten_join_alias_vars(qry, clause);
+ clause = flatten_join_alias_vars(NULL, qry, clause);
check_ungrouped_columns(clause, pstate, qry,
groupClauses, groupClauseCommonVars,
have_non_var_grouping,
@@ -1546,7 +1546,7 @@ finalize_grouping_exprs_walker(Node *node,
Index ref = 0;
if (context->hasJoinRTEs)
- expr = flatten_join_alias_vars(context->qry, expr);
+ expr = flatten_join_alias_vars(NULL, context->qry, expr);
/*
* Each expression must match a grouping entry at the current
diff --git a/src/include/optimizer/optimizer.h b/src/include/optimizer/optimizer.h
index 409005bae9..95f3461a3d 100644
--- a/src/include/optimizer/optimizer.h
+++ b/src/include/optimizer/optimizer.h
@@ -197,6 +197,6 @@ extern bool contain_var_clause(Node *node);
extern bool contain_vars_of_level(Node *node, int levelsup);
extern int locate_var_of_level(Node *node, int levelsup);
extern List *pull_var_clause(Node *node, int flags);
-extern Node *flatten_join_alias_vars(Query *query, Node *node);
+extern Node *flatten_join_alias_vars(PlannerInfo *root, Query *query, Node *node);
#endif /* OPTIMIZER_H */
commit 284e02cbe24b22c5c8b47081dfb4c5ee35ba74d6
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sat Nov 5 17:32:23 2022 -0400
Teach FDWs about base-plus-outer-join relids.
Conversion of the planner to include OJ relids in join relids
affects FDWs that want to plan foreign joins. They *must* follow
suit when labeling foreign joins in order to match with the core
planner, but for many purposes (if postgres_fdw is any guide)
they'd prefer to consider only base relations within the join.
To support both requirements, redefine ForeignScan.fs_relids as
base+OJ relids, and add a new field fs_base_relids that's set up
by the core planner.
Another way we could do this is to keep fs_relids as just base
relids and make the new field be the one with OJ relids added.
While that would be more backwards-compatible in some sense,
it would be inconsistent with the naming used in the core planner,
and I think that it might allow some types of bugs to escape
quick detection.
postgres_fdw also has one place where it needs to ignore varnullingrels
while matching Vars. It's not clear whether it's worth trying to
improve that. (This too is probably only an issue for FDWs that do
join planning, since Vars seen in a base relation scan should never
have any varnullingrels.)
As of this step, this patch series passes check-world.
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
index 9524765650..94dd7b2c96 100644
--- a/contrib/postgres_fdw/deparse.c
+++ b/contrib/postgres_fdw/deparse.c
@@ -3950,7 +3950,17 @@ get_relation_column_alias_ids(Var *node, RelOptInfo *foreignrel,
i = 1;
foreach(lc, foreignrel->reltarget->exprs)
{
- if (equal(lfirst(lc), (Node *) node))
+ Var *tlvar = (Var *) lfirst(lc);
+
+ /*
+ * Match reltarget entries only on varno/varattno. Ideally there
+ * would be some cross-check on varnullingrels, but it's unclear what
+ * to do exactly; we don't have enough context to know what that value
+ * should be.
+ */
+ if (IsA(tlvar, Var) &&
+ tlvar->varno == node->varno &&
+ tlvar->varattno == node->varattno)
{
*colno = i;
return;
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 8d7500abfb..39cc37053c 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -1511,13 +1511,13 @@ postgresBeginForeignScan(ForeignScanState *node, int eflags)
/*
* Identify which user to do the remote access as. This should match what
* ExecCheckRTEPerms() does. In case of a join or aggregate, use the
- * lowest-numbered member RTE as a representative; we would get the same
- * result from any.
+ * lowest-numbered member base RTE as a representative; we would get the
+ * same result from any.
*/
if (fsplan->scan.scanrelid > 0)
rtindex = fsplan->scan.scanrelid;
else
- rtindex = bms_next_member(fsplan->fs_relids, -1);
+ rtindex = bms_next_member(fsplan->fs_base_relids, -1);
rte = exec_rt_fetch(rtindex, estate);
userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
@@ -2414,7 +2414,7 @@ find_modifytable_subplan(PlannerInfo *root,
{
ForeignScan *fscan = (ForeignScan *) subplan;
- if (bms_is_member(rtindex, fscan->fs_relids))
+ if (bms_is_member(rtindex, fscan->fs_base_relids))
return fscan;
}
@@ -2840,8 +2840,8 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
* that setrefs.c won't update the string when flattening the
* rangetable. To find out what rtoffset was applied, identify the
* minimum RT index appearing in the string and compare it to the
- * minimum member of plan->fs_relids. (We expect all the relids in
- * the join will have been offset by the same amount; the Asserts
+ * minimum member of plan->fs_base_relids. (We expect all the relids
+ * in the join will have been offset by the same amount; the Asserts
* below should catch it if that ever changes.)
*/
minrti = INT_MAX;
@@ -2858,7 +2858,7 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
else
ptr++;
}
- rtoffset = bms_next_member(plan->fs_relids, -1) - minrti;
+ rtoffset = bms_next_member(plan->fs_base_relids, -1) - minrti;
/* Now we can translate the string */
relations = makeStringInfo();
@@ -2873,7 +2873,7 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
char *refname;
rti += rtoffset;
- Assert(bms_is_member(rti, plan->fs_relids));
+ Assert(bms_is_member(rti, plan->fs_base_relids));
rte = rt_fetch(rti, es->rtable);
Assert(rte->rtekind == RTE_RELATION);
/* This logic should agree with explain.c's ExplainTargetRel */
diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml
index 94263c628f..ac1717bc3c 100644
--- a/doc/src/sgml/fdwhandler.sgml
+++ b/doc/src/sgml/fdwhandler.sgml
@@ -351,6 +351,17 @@ GetForeignJoinPaths(PlannerInfo *root,
it will supply at run time in the tuples it returns.
</para>
+ <note>
+ <para>
+ Beginning with <productname>PostgreSQL</productname> 16,
+ <structfield>fs_relids</structfield> includes the rangetable indexes
+ of outer joins, if any were involved in this join. The new field
+ <structfield>fs_base_relids</structfield> includes only base
+ relation indexes, and thus
+ mimics <structfield>fs_relids</structfield>'s old semantics.
+ </para>
+ </note>
+
<para>
See <xref linkend="fdw-planning"/> for additional information.
</para>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index f86983c660..ed9a118416 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1114,7 +1114,7 @@ ExplainPreScanNode(PlanState *planstate, Bitmapset **rels_used)
break;
case T_ForeignScan:
*rels_used = bms_add_members(*rels_used,
- ((ForeignScan *) plan)->fs_relids);
+ ((ForeignScan *) plan)->fs_base_relids);
break;
case T_CustomScan:
*rels_used = bms_add_members(*rels_used,
diff --git a/src/backend/executor/execScan.c b/src/backend/executor/execScan.c
index 043bb83f55..2b37266b6a 100644
--- a/src/backend/executor/execScan.c
+++ b/src/backend/executor/execScan.c
@@ -325,7 +325,7 @@ ExecScanReScan(ScanState *node)
* all of them.
*/
if (IsA(node->ps.plan, ForeignScan))
- relids = ((ForeignScan *) node->ps.plan)->fs_relids;
+ relids = ((ForeignScan *) node->ps.plan)->fs_base_relids;
else if (IsA(node->ps.plan, CustomScan))
relids = ((CustomScan *) node->ps.plan)->custom_relids;
else
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index ac86ce9003..13f46e4f23 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -4153,14 +4153,22 @@ create_foreignscan_plan(PlannerInfo *root, ForeignPath *best_path,
/*
* Likewise, copy the relids that are represented by this foreign scan. An
- * upper rel doesn't have relids set, but it covers all the base relations
- * participating in the underlying scan, so use root's all_baserels.
+ * upper rel doesn't have relids set, but it covers all the relations
+ * participating in the underlying scan/join, so use root->all_query_rels.
*/
if (rel->reloptkind == RELOPT_UPPER_REL)
- scan_plan->fs_relids = root->all_baserels;
+ scan_plan->fs_relids = root->all_query_rels;
else
scan_plan->fs_relids = best_path->path.parent->relids;
+ /*
+ * Join relid sets include relevant outer joins, but FDWs may need to know
+ * which are the included base rels. That's a bit tedious to get without
+ * access to the plan-time data structures, so compute it here.
+ */
+ scan_plan->fs_base_relids = bms_difference(scan_plan->fs_relids,
+ root->outer_join_rels);
+
/*
* If this is a foreign join, and to make it valid to push down we had to
* assume that the current user is the same as some user explicitly named
@@ -5800,8 +5808,9 @@ make_foreignscan(List *qptlist,
node->fdw_private = fdw_private;
node->fdw_scan_tlist = fdw_scan_tlist;
node->fdw_recheck_quals = fdw_recheck_quals;
- /* fs_relids will be filled in by create_foreignscan_plan */
+ /* fs_relids, fs_base_relids will be filled by create_foreignscan_plan */
node->fs_relids = NULL;
+ node->fs_base_relids = NULL;
/* fsSystemCol will be filled in by create_foreignscan_plan */
node->fsSystemCol = false;
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 8fff731756..6ed6b950a4 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1560,6 +1560,7 @@ set_foreignscan_references(PlannerInfo *root,
}
fscan->fs_relids = offset_relid_set(fscan->fs_relids, rtoffset);
+ fscan->fs_base_relids = offset_relid_set(fscan->fs_base_relids, rtoffset);
/* Adjust resultRelation if it's valid */
if (fscan->resultRelation > 0)
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 5c2ab1b379..25bc3e61eb 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -689,6 +689,7 @@ typedef struct WorkTableScan
* When the plan node represents a foreign join, scan.scanrelid is zero and
* fs_relids must be consulted to identify the join relation. (fs_relids
* is valid for simple scans as well, but will always match scan.scanrelid.)
+ * fs_relids includes outer joins; fs_base_relids does not.
*
* If the FDW's PlanDirectModify() callback decides to repurpose a ForeignScan
* node to perform the UPDATE or DELETE operation directly in the remote
@@ -708,7 +709,8 @@ typedef struct ForeignScan
List *fdw_private; /* private data for FDW */
List *fdw_scan_tlist; /* optional tlist describing scan tuple */
List *fdw_recheck_quals; /* original quals not in scan.plan.qual */
- Bitmapset *fs_relids; /* RTIs generated by this scan */
+ Bitmapset *fs_relids; /* base+OJ RTIs generated by this scan */
+ Bitmapset *fs_base_relids; /* base RTIs generated by this scan */
bool fsSystemCol; /* true if any "system column" is needed */
} ForeignScan;
commit 9ffde4a92a31e9f6d1c04daa8ab64ad89107991d
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sat Nov 5 17:37:11 2022 -0400
Don't use RestrictInfo.nullable_relids in join_clause_is_movable_to.
Instead of using per-clause nullable_relids data, compute a
per-baserel set of outer joins that can null each relation, and
check for overlap between that and clause_relids to detect whether
the clause can safely be pushed down to relation scan level.
join_clause_is_movable_into also uses nullable_relids, but it
turns out that that test can just be dropped entirely. Now that
clause_relids includes nulling outer joins, the preceding tests
in the function are sufficient to reject clauses that can't be
pushed down.
This might seem like a net loss given that we have to add a bit
of code to initsplan.c to compute RelOptInfo.nulling_relids.
However, that's not much code at all, and the payoff is this:
we no longer need RestrictInfo.nullable_relids at all.
The next patch, which removes that field and the extensive
infrastructure that maintains it, saves way more code and cycles
than we add here. Also, I think there are likely going to be
other uses for RelOptInfo.nulling_relids.
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 96e8033930..364c26badf 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -57,6 +57,8 @@ static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
static void process_security_barrier_quals(PlannerInfo *root,
int rti, Relids qualscope,
bool below_outer_join);
+static void mark_rels_nulled_by_join(PlannerInfo *root, Index ojrelid,
+ Relids lower_rels);
static SpecialJoinInfo *make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
@@ -963,6 +965,7 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
*qualscope = bms_add_member(*qualscope, j->rtindex);
root->outer_join_rels = bms_add_member(root->outer_join_rels,
j->rtindex);
+ mark_rels_nulled_by_join(root, j->rtindex, rightids);
}
*inner_join_rels = bms_union(left_inners, right_inners);
nonnullable_rels = leftids;
@@ -1005,6 +1008,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
*qualscope = bms_add_member(*qualscope, j->rtindex);
root->outer_join_rels = bms_add_member(root->outer_join_rels,
j->rtindex);
+ mark_rels_nulled_by_join(root, j->rtindex, leftids);
+ mark_rels_nulled_by_join(root, j->rtindex, rightids);
*inner_join_rels = bms_union(left_inners, right_inners);
/* each side is both outer and inner */
nonnullable_rels = *qualscope;
@@ -1221,6 +1226,33 @@ process_security_barrier_quals(PlannerInfo *root,
Assert(security_level <= root->qual_security_level);
}
+/*
+ * mark_rels_nulled_by_join
+ * Fill RelOptInfo.nulling_relids of baserels nulled by this outer join
+ *
+ * Inputs:
+ * ojrelid: RT index of the join RTE (must not be 0)
+ * lower_rels: the base+OJ Relids syntactically below nullable side of join
+ */
+static void
+mark_rels_nulled_by_join(PlannerInfo *root, Index ojrelid,
+ Relids lower_rels)
+{
+ int relid = -1;
+
+ while ((relid = bms_next_member(lower_rels, relid)) > 0)
+ {
+ RelOptInfo *rel = root->simple_rel_array[relid];
+
+ if (rel == NULL) /* must be an outer join */
+ {
+ Assert(bms_is_member(relid, root->outer_join_rels));
+ continue;
+ }
+ rel->nulling_relids = bms_add_member(rel->nulling_relids, ojrelid);
+ }
+}
+
/*
* make_outerjoininfo
* Build a SpecialJoinInfo for the current outer join
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index cea298c633..3dd9317320 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -277,6 +277,12 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
rel->top_parent = parent->top_parent ? parent->top_parent : parent;
rel->top_parent_relids = rel->top_parent->relids;
+ /*
+ * A child rel is below the same outer joins as its parent. (We
+ * presume this info was already calculated for the parent.)
+ */
+ rel->nulling_relids = parent->nulling_relids;
+
/*
* Also propagate lateral-reference information from appendrel parent
* rels to their child rels. We intentionally give each child rel the
@@ -300,6 +306,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
rel->parent = NULL;
rel->top_parent = NULL;
rel->top_parent_relids = NULL;
+ rel->nulling_relids = NULL;
rel->direct_lateral_relids = NULL;
rel->lateral_relids = NULL;
rel->lateral_referencers = NULL;
@@ -680,6 +687,7 @@ build_join_rel(PlannerInfo *root,
joinrel->max_attr = 0;
joinrel->attr_needed = NULL;
joinrel->attr_widths = NULL;
+ joinrel->nulling_relids = NULL;
joinrel->lateral_vars = NIL;
joinrel->lateral_referencers = NULL;
joinrel->indexlist = NIL;
@@ -869,6 +877,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->max_attr = 0;
joinrel->attr_needed = NULL;
joinrel->attr_widths = NULL;
+ joinrel->nulling_relids = NULL;
joinrel->lateral_vars = NIL;
joinrel->lateral_referencers = NULL;
joinrel->indexlist = NIL;
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index bcbee8f943..15f410cf36 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -618,8 +618,17 @@ join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
if (bms_is_member(baserel->relid, rinfo->outer_relids))
return false;
- /* Target rel must not be nullable below the clause */
- if (bms_is_member(baserel->relid, rinfo->nullable_relids))
+ /*
+ * Target rel's Vars must not be nulled by any outer join. We can check
+ * this without groveling through the individual Vars by seeing whether
+ * clause_relids (which includes all such Vars' varnullingrels) includes
+ * any outer join that can null the target rel. You might object that
+ * this could reject the clause on the basis of an OJ relid that came from
+ * some other rel's Var. However, that would still mean that the clause
+ * came from above that outer join and shouldn't be pushed down; so there
+ * should be no false positives.
+ */
+ if (bms_overlap(rinfo->clause_relids, baserel->nulling_relids))
return false;
/* Clause must not use any rels with LATERAL references to this rel */
@@ -651,16 +660,17 @@ join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
* relation plus the outer rels. We also check that it does reference at
* least one current Var, ensuring that the clause will be pushed down to
* a unique place in a parameterized join tree. And we check that we're
- * not pushing the clause into its outer-join outer side, nor down into
- * a lower outer join's inner side.
- *
- * The check about pushing a clause down into a lower outer join's inner side
- * is only approximate; it sometimes returns "false" when actually it would
- * be safe to use the clause here because we're still above the outer join
- * in question. This is okay as long as the answers at different join levels
- * are consistent: it just means we might sometimes fail to push a clause as
- * far down as it could safely be pushed. It's unclear whether it would be
- * worthwhile to do this more precisely. (But if it's ever fixed to be
+ * not pushing the clause into its outer-join outer side.
+ *
+ * We used to need to check that we're not pushing the clause into a lower
+ * outer join's inner side. However, now that clause_relids includes
+ * references to potentially-nulling outer joins, the other tests handle that
+ * concern. If the clause references any Var coming from the inside of a
+ * lower outer join, its clause_relids will mention that outer join, causing
+ * the evaluability check to fail; while if it references no such Vars, the
+ * references-a-target-rel check will fail.
+ *
+ * XXX not clear if we can do this yet: (But if it's ever fixed to be
* exactly accurate, there's an Assert in get_joinrel_parampathinfo() that
* should be re-enabled.)
*
@@ -704,14 +714,5 @@ join_clause_is_movable_into(RestrictInfo *rinfo,
if (bms_overlap(currentrelids, rinfo->outer_relids))
return false;
- /*
- * Target rel(s) must not be nullable below the clause. This is
- * approximate, in the safe direction, because the current join might be
- * above the join where the nulling would happen, in which case the clause
- * would work correctly here. But we don't have enough info to be sure.
- */
- if (bms_overlap(currentrelids, rinfo->nullable_relids))
- return false;
-
return true;
}
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 9d89b0c9eb..3f89199e64 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -660,6 +660,7 @@ typedef struct PartitionSchemeData *PartitionScheme;
* outer-join relids.
* attr_widths - cache space for per-attribute width estimates;
* zero means not computed yet
+ * nulling_relids - relids of outer joins that can null this rel
* lateral_vars - lateral cross-references of rel, if any (list of
* Vars and PlaceHolderVars)
* lateral_referencers - relids of rels that reference this one laterally
@@ -893,6 +894,8 @@ typedef struct RelOptInfo
Relids *attr_needed pg_node_attr(read_write_ignore);
/* array indexed [min_attr .. max_attr] */
int32 *attr_widths pg_node_attr(read_write_ignore);
+ /* relids of outer joins that can null this baserel */
+ Relids nulling_relids;
/* LATERAL Vars and PHVs referenced by rel */
List *lateral_vars;
/* rels that reference this baserel laterally */
commit 8f75da0793f4310f0608137cb05d9029c8b11b4c
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sat Nov 5 17:39:57 2022 -0400
Remove RestrictInfo.nullable_relids and associated infrastructure.
There is no more code using this field, only code computing it,
so just delete all that. We can likewise get rid of
EquivalenceMember.em_nullable_relids and
PlannerInfo.nullable_baserels.
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 39cc37053c..10aa27a78a 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -6313,7 +6313,6 @@ foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
false,
root->qual_security_level,
grouped_rel->relids,
- NULL,
NULL);
if (is_foreign_expr(root, grouped_rel, expr))
fpinfo->remote_conds = lappend(fpinfo->remote_conds, rinfo);
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 5902c80747..8a6a40f672 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -2733,7 +2733,6 @@ set_function_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
if (var)
pathkeys = build_expression_pathkey(root,
(Expr *) var,
- NULL, /* below outer joins */
Int8LessOperator,
rel->relids,
false);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index d4f8b7893d..0737cc355f 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -34,7 +34,7 @@
static EquivalenceMember *add_eq_member(EquivalenceClass *ec,
- Expr *expr, Relids relids, Relids nullable_relids,
+ Expr *expr, Relids relids,
EquivalenceMember *parent,
Oid datatype);
static bool is_exprlist_member(Expr *node, List *exprs);
@@ -131,9 +131,7 @@ process_equivalence(PlannerInfo *root,
Expr *item1;
Expr *item2;
Relids item1_relids,
- item2_relids,
- item1_nullable_relids,
- item2_nullable_relids;
+ item2_relids;
List *opfamilies;
EquivalenceClass *ec1,
*ec2;
@@ -206,8 +204,7 @@ process_equivalence(PlannerInfo *root,
restrictinfo->pseudoconstant,
restrictinfo->security_level,
NULL,
- restrictinfo->outer_relids,
- restrictinfo->nullable_relids);
+ restrictinfo->outer_relids);
}
return false;
}
@@ -225,12 +222,6 @@ process_equivalence(PlannerInfo *root,
return false; /* RHS is non-strict but not constant */
}
- /* Calculate nullable-relid sets for each side of the clause */
- item1_nullable_relids = bms_intersect(item1_relids,
- restrictinfo->nullable_relids);
- item2_nullable_relids = bms_intersect(item2_relids,
- restrictinfo->nullable_relids);
-
/*
* We use the declared input types of the operator, not exprType() of the
* inputs, as the nominal datatypes for opfamily lookup. This presumes
@@ -400,7 +391,7 @@ process_equivalence(PlannerInfo *root,
else if (ec1)
{
/* Case 3: add item2 to ec1 */
- em2 = add_eq_member(ec1, item2, item2_relids, item2_nullable_relids,
+ em2 = add_eq_member(ec1, item2, item2_relids,
NULL, item2_type);
ec1->ec_sources = lappend(ec1->ec_sources, restrictinfo);
ec1->ec_below_outer_join |= below_outer_join;
@@ -418,7 +409,7 @@ process_equivalence(PlannerInfo *root,
else if (ec2)
{
/* Case 3: add item1 to ec2 */
- em1 = add_eq_member(ec2, item1, item1_relids, item1_nullable_relids,
+ em1 = add_eq_member(ec2, item1, item1_relids,
NULL, item1_type);
ec2->ec_sources = lappend(ec2->ec_sources, restrictinfo);
ec2->ec_below_outer_join |= below_outer_join;
@@ -452,9 +443,9 @@ process_equivalence(PlannerInfo *root,
ec->ec_min_security = restrictinfo->security_level;
ec->ec_max_security = restrictinfo->security_level;
ec->ec_merged = NULL;
- em1 = add_eq_member(ec, item1, item1_relids, item1_nullable_relids,
+ em1 = add_eq_member(ec, item1, item1_relids,
NULL, item1_type);
- em2 = add_eq_member(ec, item2, item2_relids, item2_nullable_relids,
+ em2 = add_eq_member(ec, item2, item2_relids,
NULL, item2_type);
root->eq_classes = lappend(root->eq_classes, ec);
@@ -545,13 +536,12 @@ canonicalize_ec_expression(Expr *expr, Oid req_type, Oid req_collation)
*/
static EquivalenceMember *
add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
- Relids nullable_relids, EquivalenceMember *parent, Oid datatype)
+ EquivalenceMember *parent, Oid datatype)
{
EquivalenceMember *em = makeNode(EquivalenceMember);
em->em_expr = expr;
em->em_relids = relids;
- em->em_nullable_relids = nullable_relids;
em->em_is_const = false;
em->em_is_child = (parent != NULL);
em->em_datatype = datatype;
@@ -588,13 +578,6 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
* equivalence class it is a member of; if none, optionally build a new
* single-member EquivalenceClass for it.
*
- * expr is the expression, and nullable_relids is the set of base relids
- * that are potentially nullable below it. We actually only care about
- * the set of such relids that are used in the expression; but for caller
- * convenience, we perform that intersection step here. The caller need
- * only be sure that nullable_relids doesn't omit any nullable rels that
- * might appear in the expr.
- *
* sortref is the SortGroupRef of the originating SortGroupClause, if any,
* or zero if not. (It should never be zero if the expression is volatile!)
*
@@ -623,7 +606,6 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
EquivalenceClass *
get_eclass_for_sort_expr(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
List *opfamilies,
Oid opcintype,
Oid collation,
@@ -719,13 +701,12 @@ get_eclass_for_sort_expr(PlannerInfo *root,
elog(ERROR, "volatile EquivalenceClass has no sortref");
/*
- * Get the precise set of nullable relids appearing in the expression.
+ * Get the precise set of relids appearing in the expression.
*/
expr_relids = pull_varnos(root, (Node *) expr);
- nullable_relids = bms_intersect(nullable_relids, expr_relids);
newem = add_eq_member(newec, copyObject(expr), expr_relids,
- nullable_relids, NULL, opcintype);
+ NULL, opcintype);
/*
* add_eq_member doesn't check for volatile functions, set-returning
@@ -1211,8 +1192,6 @@ generate_base_implied_equalities_const(PlannerInfo *root,
rinfo = process_implied_equality(root, eq_op, ec->ec_collation,
cur_em->em_expr, const_em->em_expr,
bms_copy(ec->ec_relids),
- bms_union(cur_em->em_nullable_relids,
- const_em->em_nullable_relids),
ec->ec_min_security,
ec->ec_below_outer_join,
cur_em->em_is_const);
@@ -1285,8 +1264,6 @@ generate_base_implied_equalities_no_const(PlannerInfo *root,
rinfo = process_implied_equality(root, eq_op, ec->ec_collation,
prev_em->em_expr, cur_em->em_expr,
bms_copy(ec->ec_relids),
- bms_union(prev_em->em_nullable_relids,
- cur_em->em_nullable_relids),
ec->ec_min_security,
ec->ec_below_outer_join,
false);
@@ -1889,8 +1866,6 @@ create_join_clause(PlannerInfo *root,
rightem->em_expr,
bms_union(leftem->em_relids,
rightem->em_relids),
- bms_union(leftem->em_nullable_relids,
- rightem->em_nullable_relids),
ec->ec_min_security);
/* If it's a child clause, copy the parent's rinfo_serial */
@@ -2105,8 +2080,7 @@ reconsider_outer_join_clause(PlannerInfo *root, RestrictInfo *rinfo,
left_type,
right_type,
inner_datatype;
- Relids inner_relids,
- inner_nullable_relids;
+ Relids inner_relids;
ListCell *lc1;
Assert(is_opclause(rinfo->clause));
@@ -2133,8 +2107,6 @@ reconsider_outer_join_clause(PlannerInfo *root, RestrictInfo *rinfo,
inner_datatype = left_type;
inner_relids = rinfo->left_relids;
}
- inner_nullable_relids = bms_intersect(inner_relids,
- rinfo->nullable_relids);
/* Scan EquivalenceClasses for a match to outervar */
foreach(lc1, root->eq_classes)
@@ -2195,7 +2167,6 @@ reconsider_outer_join_clause(PlannerInfo *root, RestrictInfo *rinfo,
innervar,
cur_em->em_expr,
bms_copy(inner_relids),
- bms_copy(inner_nullable_relids),
cur_ec->ec_min_security);
if (process_equivalence(root, &newrinfo, true))
match = true;
@@ -2233,9 +2204,7 @@ reconsider_full_join_clause(PlannerInfo *root, FullJoinClauseInfo *fjinfo)
left_type,
right_type;
Relids left_relids,
- right_relids,
- left_nullable_relids,
- right_nullable_relids;
+ right_relids;
ListCell *lc1;
/* Can't use an outerjoin_delayed clause here */
@@ -2251,10 +2220,6 @@ reconsider_full_join_clause(PlannerInfo *root, FullJoinClauseInfo *fjinfo)
rightvar = (Expr *) get_rightop(rinfo->clause);
left_relids = rinfo->left_relids;
right_relids = rinfo->right_relids;
- left_nullable_relids = bms_intersect(left_relids,
- rinfo->nullable_relids);
- right_nullable_relids = bms_intersect(right_relids,
- rinfo->nullable_relids);
foreach(lc1, root->eq_classes)
{
@@ -2356,7 +2321,6 @@ reconsider_full_join_clause(PlannerInfo *root, FullJoinClauseInfo *fjinfo)
leftvar,
cur_em->em_expr,
bms_copy(left_relids),
- bms_copy(left_nullable_relids),
cur_ec->ec_min_security);
if (process_equivalence(root, &newrinfo, true))
matchleft = true;
@@ -2372,7 +2336,6 @@ reconsider_full_join_clause(PlannerInfo *root, FullJoinClauseInfo *fjinfo)
rightvar,
cur_em->em_expr,
bms_copy(right_relids),
- bms_copy(right_nullable_relids),
cur_ec->ec_min_security);
if (process_equivalence(root, &newrinfo, true))
matchright = true;
@@ -2662,7 +2625,6 @@ add_child_rel_equivalences(PlannerInfo *root,
/* Yes, generate transformed child version */
Expr *child_expr;
Relids new_relids;
- Relids new_nullable_relids;
if (parent_rel->reloptkind == RELOPT_BASEREL)
{
@@ -2692,21 +2654,7 @@ add_child_rel_equivalences(PlannerInfo *root,
top_parent_relids);
new_relids = bms_add_members(new_relids, child_relids);
- /*
- * And likewise for nullable_relids. Note this code assumes
- * parent and child relids are singletons.
- */
- new_nullable_relids = cur_em->em_nullable_relids;
- if (bms_overlap(new_nullable_relids, top_parent_relids))
- {
- new_nullable_relids = bms_difference(new_nullable_relids,
- top_parent_relids);
- new_nullable_relids = bms_add_members(new_nullable_relids,
- child_relids);
- }
-
- (void) add_eq_member(cur_ec, child_expr,
- new_relids, new_nullable_relids,
+ (void) add_eq_member(cur_ec, child_expr, new_relids,
cur_em, cur_em->em_datatype);
/* Record this EC index for the child rel */
@@ -2803,7 +2751,6 @@ add_child_join_rel_equivalences(PlannerInfo *root,
/* Yes, generate transformed child version */
Expr *child_expr;
Relids new_relids;
- Relids new_nullable_relids;
if (parent_joinrel->reloptkind == RELOPT_JOINREL)
{
@@ -2834,20 +2781,7 @@ add_child_join_rel_equivalences(PlannerInfo *root,
top_parent_relids);
new_relids = bms_add_members(new_relids, child_relids);
- /*
- * For nullable_relids, we must selectively replace parent
- * nullable relids with child ones.
- */
- new_nullable_relids = cur_em->em_nullable_relids;
- if (bms_overlap(new_nullable_relids, top_parent_relids))
- new_nullable_relids =
- adjust_child_relids_multilevel(root,
- new_nullable_relids,
- child_joinrel,
- child_joinrel->top_parent);
-
- (void) add_eq_member(cur_ec, child_expr,
- new_relids, new_nullable_relids,
+ (void) add_eq_member(cur_ec, child_expr, new_relids,
cur_em, cur_em->em_datatype);
}
}
diff --git a/src/backend/optimizer/path/pathkeys.c b/src/backend/optimizer/path/pathkeys.c
index a9943cd6e0..bf919ca97f 100644
--- a/src/backend/optimizer/path/pathkeys.c
+++ b/src/backend/optimizer/path/pathkeys.c
@@ -180,9 +180,6 @@ pathkey_is_redundant(PathKey *new_pathkey, List *pathkeys)
* Given an expression and sort-order information, create a PathKey.
* The result is always a "canonical" PathKey, but it might be redundant.
*
- * expr is the expression, and nullable_relids is the set of base relids
- * that are potentially nullable below it.
- *
* If the PathKey is being generated from a SortGroupClause, sortref should be
* the SortGroupClause's SortGroupRef; otherwise zero.
*
@@ -198,7 +195,6 @@ pathkey_is_redundant(PathKey *new_pathkey, List *pathkeys)
static PathKey *
make_pathkey_from_sortinfo(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
Oid opfamily,
Oid opcintype,
Oid collation,
@@ -234,7 +230,7 @@ make_pathkey_from_sortinfo(PlannerInfo *root,
equality_op);
/* Now find or (optionally) create a matching EquivalenceClass */
- eclass = get_eclass_for_sort_expr(root, expr, nullable_relids,
+ eclass = get_eclass_for_sort_expr(root, expr,
opfamilies, opcintype, collation,
sortref, rel, create_it);
@@ -257,7 +253,6 @@ make_pathkey_from_sortinfo(PlannerInfo *root,
static PathKey *
make_pathkey_from_sortop(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
Oid ordering_op,
bool nulls_first,
Index sortref,
@@ -279,7 +274,6 @@ make_pathkey_from_sortop(PlannerInfo *root,
return make_pathkey_from_sortinfo(root,
expr,
- nullable_relids,
opfamily,
opcintype,
collation,
@@ -584,12 +578,10 @@ build_index_pathkeys(PlannerInfo *root,
}
/*
- * OK, try to make a canonical pathkey for this sort key. Note we're
- * underneath any outer joins, so nullable_relids should be NULL.
+ * OK, try to make a canonical pathkey for this sort key.
*/
cpathkey = make_pathkey_from_sortinfo(root,
indexkey,
- NULL,
index->sortopfamily[i],
index->opcintype[i],
index->indexcollations[i],
@@ -743,14 +735,12 @@ build_partition_pathkeys(PlannerInfo *root, RelOptInfo *partrel,
/*
* Try to make a canonical pathkey for this partkey.
*
- * We're considering a baserel scan, so nullable_relids should be
- * NULL. Also, we assume the PartitionDesc lists any NULL partition
- * last, so we treat the scan like a NULLS LAST index: we have
- * nulls_first for backwards scan only.
+ * We assume the PartitionDesc lists any NULL partition last, so we
+ * treat the scan like a NULLS LAST index: we have nulls_first for
+ * backwards scan only.
*/
cpathkey = make_pathkey_from_sortinfo(root,
keyCol,
- NULL,
partscheme->partopfamily[i],
partscheme->partopcintype[i],
partscheme->partcollation[i],
@@ -799,7 +789,7 @@ build_partition_pathkeys(PlannerInfo *root, RelOptInfo *partrel,
* Build a pathkeys list that describes an ordering by a single expression
* using the given sort operator.
*
- * expr, nullable_relids, and rel are as for make_pathkey_from_sortinfo.
+ * expr and rel are as for make_pathkey_from_sortinfo.
* We induce the other arguments assuming default sort order for the operator.
*
* Similarly to make_pathkey_from_sortinfo, the result is NIL if create_it
@@ -808,7 +798,6 @@ build_partition_pathkeys(PlannerInfo *root, RelOptInfo *partrel,
List *
build_expression_pathkey(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
Oid opno,
Relids rel,
bool create_it)
@@ -827,7 +816,6 @@ build_expression_pathkey(PlannerInfo *root,
cpathkey = make_pathkey_from_sortinfo(root,
expr,
- nullable_relids,
opfamily,
opcintype,
exprCollation((Node *) expr),
@@ -908,14 +896,11 @@ convert_subquery_pathkeys(PlannerInfo *root, RelOptInfo *rel,
* expression is *not* volatile in the outer query: it's just
* a Var referencing whatever the subquery emitted. (IOW, the
* outer query isn't going to re-execute the volatile
- * expression itself.) So this is okay. Likewise, it's
- * correct to pass nullable_relids = NULL, because we're
- * underneath any outer joins appearing in the outer query.
+ * expression itself.) So this is okay.
*/
outer_ec =
get_eclass_for_sort_expr(root,
(Expr *) outer_var,
- NULL,
sub_eclass->ec_opfamilies,
sub_member->em_datatype,
sub_eclass->ec_collation,
@@ -997,7 +982,6 @@ convert_subquery_pathkeys(PlannerInfo *root, RelOptInfo *rel,
/* See if we have a matching EC for the TLE */
outer_ec = get_eclass_for_sort_expr(root,
(Expr *) outer_var,
- NULL,
sub_eclass->ec_opfamilies,
sub_expr_type,
sub_expr_coll,
@@ -1138,13 +1122,6 @@ build_join_pathkeys(PlannerInfo *root,
* The resulting PathKeys are always in canonical form. (Actually, there
* is no longer any code anywhere that creates non-canonical PathKeys.)
*
- * We assume that root->nullable_baserels is the set of base relids that could
- * have gone to NULL below the SortGroupClause expressions. This is okay if
- * the expressions came from the query's top level (ORDER BY, DISTINCT, etc)
- * and if this function is only invoked after deconstruct_jointree. In the
- * future we might have to make callers pass in the appropriate
- * nullable-relids set, but for now it seems unnecessary.
- *
* 'sortclauses' is a list of SortGroupClause nodes
* 'tlist' is the targetlist to find the referenced tlist entries in
*/
@@ -1166,7 +1143,6 @@ make_pathkeys_for_sortclauses(PlannerInfo *root,
Assert(OidIsValid(sortcl->sortop));
pathkey = make_pathkey_from_sortop(root,
sortkey,
- root->nullable_baserels,
sortcl->sortop,
sortcl->nulls_first,
sortcl->tleSortGroupRef,
@@ -1222,7 +1198,6 @@ initialize_mergeclause_eclasses(PlannerInfo *root, RestrictInfo *restrictinfo)
restrictinfo->left_ec =
get_eclass_for_sort_expr(root,
(Expr *) get_leftop(clause),
- restrictinfo->nullable_relids,
restrictinfo->mergeopfamilies,
lefttype,
((OpExpr *) clause)->inputcollid,
@@ -1232,7 +1207,6 @@ initialize_mergeclause_eclasses(PlannerInfo *root, RestrictInfo *restrictinfo)
restrictinfo->right_ec =
get_eclass_for_sort_expr(root,
(Expr *) get_rightop(clause),
- restrictinfo->nullable_relids,
restrictinfo->mergeopfamilies,
righttype,
((OpExpr *) clause)->inputcollid,
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 364c26badf..4b554967d4 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -92,7 +92,7 @@ static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool is_clone,
List **postponed_qual_list);
static bool check_outerjoin_delay(PlannerInfo *root, Relids *relids_p,
- Relids *nullable_relids_p, bool is_pushed_down);
+ bool is_pushed_down);
static bool check_equivalence_delay(PlannerInfo *root,
RestrictInfo *restrictinfo);
static bool check_redundant_nullability_qual(PlannerInfo *root, Node *clause);
@@ -748,9 +748,8 @@ deconstruct_jointree(PlannerInfo *root)
Assert(root->parse->jointree != NULL &&
IsA(root->parse->jointree, FromExpr));
- /* These are filled as we scan the jointree */
+ /* This is filled as we scan the jointree */
root->outer_join_rels = NULL;
- root->nullable_baserels = NULL;
result = deconstruct_recurse(root, (Node *) root->parse->jointree, false,
&qualscope, &inner_join_rels,
@@ -909,7 +908,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
left_inners,
right_inners,
nonnullable_rels,
- nullable_rels,
ojscope;
List *leftjoinlist,
*rightjoinlist;
@@ -945,8 +943,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
*inner_join_rels = *qualscope;
/* Inner join adds no restrictions for quals */
nonnullable_rels = NULL;
- /* and it doesn't force anything to null, either */
- nullable_rels = NULL;
break;
case JOIN_LEFT:
case JOIN_ANTI:
@@ -969,7 +965,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
}
*inner_join_rels = bms_union(left_inners, right_inners);
nonnullable_rels = leftids;
- nullable_rels = rightids;
break;
case JOIN_SEMI:
leftjoinlist = deconstruct_recurse(root, j->larg,
@@ -986,13 +981,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
*inner_join_rels = bms_union(left_inners, right_inners);
/* Semi join adds no restrictions for quals */
nonnullable_rels = NULL;
-
- /*
- * Theoretically, a semijoin would null the RHS; but since the
- * RHS can't be accessed above the join, this is immaterial
- * and we needn't account for it.
- */
- nullable_rels = NULL;
break;
case JOIN_FULL:
leftjoinlist = deconstruct_recurse(root, j->larg,
@@ -1013,22 +1001,16 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
*inner_join_rels = bms_union(left_inners, right_inners);
/* each side is both outer and inner */
nonnullable_rels = *qualscope;
- nullable_rels = *qualscope;
break;
default:
/* JOIN_RIGHT was eliminated during reduce_outer_joins() */
elog(ERROR, "unrecognized join type: %d",
(int) j->jointype);
nonnullable_rels = NULL; /* keep compiler quiet */
- nullable_rels = NULL;
leftjoinlist = rightjoinlist = NIL;
break;
}
- /* Report all rels that will be nulled anywhere in the jointree */
- root->nullable_baserels = bms_add_members(root->nullable_baserels,
- nullable_rels);
-
/*
* Try to process any quals postponed by children. If they need
* further postponement, add them to my output postponed_qual_list.
@@ -2105,7 +2087,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool pseudoconstant = false;
bool maybe_equivalence;
bool maybe_outer_join;
- Relids nullable_relids;
RestrictInfo *restrictinfo;
/*
@@ -2259,7 +2240,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
/* Check to see if must be delayed by lower outer join */
outerjoin_delayed = check_outerjoin_delay(root,
&relids,
- &nullable_relids,
false);
/*
@@ -2287,7 +2267,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
/* Check to see if must be delayed by lower outer join */
outerjoin_delayed = check_outerjoin_delay(root,
&relids,
- &nullable_relids,
true);
if (outerjoin_delayed)
@@ -2347,8 +2326,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
pseudoconstant,
security_level,
relids,
- outerjoin_nonnullable,
- nullable_relids);
+ outerjoin_nonnullable);
/* Apply appropriate clone marking, too */
restrictinfo->has_clone = has_clone;
@@ -2485,9 +2463,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* If the qual must be delayed, add relids to *relids_p to reflect the lowest
* safe level for evaluating the qual, and return true. Any extra delay for
* higher-level joins is reflected by setting delay_upper_joins to true in
- * SpecialJoinInfo structs. We also compute nullable_relids, the set of
- * referenced relids that are nullable by lower outer joins (note that this
- * can be nonempty even for a non-delayed qual).
+ * SpecialJoinInfo structs.
*
* For an is_pushed_down qual, we can evaluate the qual as soon as (1) we have
* all the rels it mentions, and (2) we are at or above any outer joins that
@@ -2510,8 +2486,8 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* mentioning only C cannot be applied below the join to A.
*
* For a non-pushed-down qual, this isn't going to determine where we place the
- * qual, but we need to determine outerjoin_delayed and nullable_relids anyway
- * for use later in the planning process.
+ * qual, but we need to determine outerjoin_delayed anyway for use later in
+ * the planning process.
*
* Lastly, a pushed-down qual that references the nullable side of any current
* join_info_list member and has to be evaluated above that OJ (because its
@@ -2529,24 +2505,18 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
static bool
check_outerjoin_delay(PlannerInfo *root,
Relids *relids_p, /* in/out parameter */
- Relids *nullable_relids_p, /* output parameter */
bool is_pushed_down)
{
Relids relids;
- Relids nullable_relids;
bool outerjoin_delayed;
bool found_some;
/* fast path if no special joins */
if (root->join_info_list == NIL)
- {
- *nullable_relids_p = NULL;
return false;
- }
/* must copy relids because we need the original value at the end */
relids = bms_copy(*relids_p);
- nullable_relids = NULL;
outerjoin_delayed = false;
do
{
@@ -2573,12 +2543,6 @@ check_outerjoin_delay(PlannerInfo *root,
/* we'll need another iteration */
found_some = true;
}
- /* track all the nullable rels of relevant OJs */
- nullable_relids = bms_add_members(nullable_relids,
- sjinfo->min_righthand);
- if (sjinfo->jointype == JOIN_FULL)
- nullable_relids = bms_add_members(nullable_relids,
- sjinfo->min_lefthand);
/* set delay_upper_joins if needed */
if (is_pushed_down && sjinfo->jointype != JOIN_FULL &&
bms_overlap(relids, sjinfo->min_lefthand))
@@ -2587,13 +2551,9 @@ check_outerjoin_delay(PlannerInfo *root,
}
} while (found_some);
- /* identify just the actually-referenced nullable rels */
- nullable_relids = bms_int_members(nullable_relids, *relids_p);
-
- /* replace *relids_p, and return nullable_relids */
+ /* replace *relids_p */
bms_free(*relids_p);
*relids_p = relids;
- *nullable_relids_p = nullable_relids;
return outerjoin_delayed;
}
@@ -2615,7 +2575,6 @@ check_equivalence_delay(PlannerInfo *root,
RestrictInfo *restrictinfo)
{
Relids relids;
- Relids nullable_relids;
/* fast path if no special joins */
if (root->join_info_list == NIL)
@@ -2624,12 +2583,12 @@ check_equivalence_delay(PlannerInfo *root,
/* must copy restrictinfo's relids to avoid changing it */
relids = bms_copy(restrictinfo->left_relids);
/* check left side does not need delay */
- if (check_outerjoin_delay(root, &relids, &nullable_relids, true))
+ if (check_outerjoin_delay(root, &relids, true))
return false;
/* and similarly for the right side */
relids = bms_copy(restrictinfo->right_relids);
- if (check_outerjoin_delay(root, &relids, &nullable_relids, true))
+ if (check_outerjoin_delay(root, &relids, true))
return false;
return true;
@@ -2755,11 +2714,6 @@ distribute_restrictinfo_to_rels(PlannerInfo *root,
* variable-free. Otherwise the qual is applied at the lowest join level
* that provides all its variables.
*
- * "nullable_relids" is the set of relids used in the expressions that are
- * potentially nullable below the expressions. (This has to be supplied by
- * caller because this function is used after deconstruct_jointree, so we
- * don't have knowledge of where the clause items came from.)
- *
* "security_level" is the security level to assign to the new restrictinfo.
*
* "both_const" indicates whether both items are known pseudo-constant;
@@ -2785,7 +2739,6 @@ process_implied_equality(PlannerInfo *root,
Expr *item1,
Expr *item2,
Relids qualscope,
- Relids nullable_relids,
Index security_level,
bool below_outer_join,
bool both_const)
@@ -2869,8 +2822,7 @@ process_implied_equality(PlannerInfo *root,
pseudoconstant,
security_level,
relids,
- NULL, /* outer_relids */
- nullable_relids);
+ NULL); /* outer_relids */
/*
* If it's a join clause, add vars used in the clause to targetlists of
@@ -2935,7 +2887,6 @@ build_implied_join_equality(PlannerInfo *root,
Expr *item1,
Expr *item2,
Relids qualscope,
- Relids nullable_relids,
Index security_level)
{
RestrictInfo *restrictinfo;
@@ -2963,8 +2914,7 @@ build_implied_join_equality(PlannerInfo *root,
false, /* pseudoconstant */
security_level, /* security_level */
qualscope, /* required_relids */
- NULL, /* outer_relids */
- nullable_relids); /* nullable_relids */
+ NULL); /* outer_relids */
/* Set mergejoinability/hashjoinability flags */
check_mergejoinable(restrictinfo);
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index e18d64b6dc..662d2c1f17 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -448,9 +448,6 @@ adjust_appendrel_attrs_mutator(Node *node,
newinfo->outer_relids = adjust_child_relids(oldinfo->outer_relids,
context->nappinfos,
context->appinfos);
- newinfo->nullable_relids = adjust_child_relids(oldinfo->nullable_relids,
- context->nappinfos,
- context->appinfos);
newinfo->left_relids = adjust_child_relids(oldinfo->left_relids,
context->nappinfos,
context->appinfos);
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 3d270e91d6..eb20583f75 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -815,7 +815,7 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
rinfo->outerjoin_delayed,
pseudoconstant,
rinfo->security_level,
- NULL, NULL, NULL));
+ NULL, NULL));
/* track minimum security level among child quals */
cq_min_security = Min(cq_min_security, rinfo->security_level);
}
@@ -850,7 +850,7 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
make_restrictinfo(root, qual,
true, false, false,
security_level,
- NULL, NULL, NULL));
+ NULL, NULL));
cq_min_security = Min(cq_min_security, security_level);
}
security_level++;
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index 9cfde2f790..336a73d3b9 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -275,7 +275,6 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
false,
join_or_rinfo->security_level,
NULL,
- NULL,
NULL);
/*
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index 15f410cf36..1d8912608b 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -29,8 +29,7 @@ static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids);
+ Relids outer_relids);
static Expr *make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
bool is_pushed_down,
@@ -38,8 +37,7 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids);
+ Relids outer_relids);
/*
@@ -49,7 +47,7 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
*
* The is_pushed_down, outerjoin_delayed, and pseudoconstant flags for the
* RestrictInfo must be supplied by the caller, as well as the correct values
- * for security_level, outer_relids, and nullable_relids.
+ * for security_level and outer_relids.
* required_relids can be NULL, in which case it defaults to the actual clause
* contents (i.e., clause_relids).
*
@@ -69,8 +67,7 @@ make_restrictinfo(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids)
+ Relids outer_relids)
{
/*
* If it's an OR clause, build a modified copy with RestrictInfos inserted
@@ -84,8 +81,7 @@ make_restrictinfo(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids);
+ outer_relids);
/* Shouldn't be an AND clause, else AND/OR flattening messed up */
Assert(!is_andclause(clause));
@@ -98,8 +94,7 @@ make_restrictinfo(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids);
+ outer_relids);
}
/*
@@ -116,8 +111,7 @@ make_restrictinfo_internal(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids)
+ Relids outer_relids)
{
RestrictInfo *restrictinfo = makeNode(RestrictInfo);
Relids baserels;
@@ -132,7 +126,6 @@ make_restrictinfo_internal(PlannerInfo *root,
restrictinfo->can_join = false; /* may get set below */
restrictinfo->security_level = security_level;
restrictinfo->outer_relids = outer_relids;
- restrictinfo->nullable_relids = nullable_relids;
/*
* If it's potentially delayable by lower-level security quals, figure out
@@ -260,7 +253,7 @@ make_restrictinfo_internal(PlannerInfo *root,
*
* The same is_pushed_down, outerjoin_delayed, and pseudoconstant flag
* values can be applied to all RestrictInfo nodes in the result. Likewise
- * for security_level, outer_relids, and nullable_relids.
+ * for security_level and outer_relids.
*
* The given required_relids are attached to our top-level output,
* but any OR-clause constituents are allowed to default to just the
@@ -274,8 +267,7 @@ make_sub_restrictinfos(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids)
+ Relids outer_relids)
{
if (is_orclause(clause))
{
@@ -291,8 +283,7 @@ make_sub_restrictinfos(PlannerInfo *root,
pseudoconstant,
security_level,
NULL,
- outer_relids,
- nullable_relids));
+ outer_relids));
return (Expr *) make_restrictinfo_internal(root,
clause,
make_orclause(orlist),
@@ -301,8 +292,7 @@ make_sub_restrictinfos(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids);
+ outer_relids);
}
else if (is_andclause(clause))
{
@@ -318,8 +308,7 @@ make_sub_restrictinfos(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids));
+ outer_relids));
return make_andclause(andlist);
}
else
@@ -331,8 +320,7 @@ make_sub_restrictinfos(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids);
+ outer_relids);
}
/*
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 3f89199e64..ea43313f11 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -263,14 +263,6 @@ struct PlannerInfo
*/
Relids all_query_rels;
- /*
- * nullable_baserels is a Relids set of base relids that are nullable by
- * some outer join in the jointree; these are rels that are potentially
- * nullable below the WHERE clause, SELECT targetlist, etc. This is
- * computed in deconstruct_jointree.
- */
- Relids nullable_baserels;
-
/*
* join_rel_list is a list of all join-relation RelOptInfos we have
* considered in this planning run. For small problems we just scan the
@@ -1358,7 +1350,6 @@ typedef struct EquivalenceMember
Expr *em_expr; /* the expression represented */
Relids em_relids; /* all relids appearing in em_expr */
- Relids em_nullable_relids; /* nullable by lower outer joins */
bool em_is_const; /* expression is pseudoconstant? */
bool em_is_child; /* derived version for a child relation? */
Oid em_datatype; /* the "nominal type" used by the opfamily */
@@ -2384,9 +2375,7 @@ typedef struct LimitPath
* in parameterized scans, since pushing it into the join's outer side would
* lead to wrong answers.)
*
- * There is also a nullable_relids field, which is the set of rels the clause
- * references that can be forced null by some outer join below the clause.
- *
+ * XXX this comment needs work, if we don't remove it completely:
* outerjoin_delayed = true is subtly different from nullable_relids != NULL:
* a clause might reference some nullable rels and yet not be
* outerjoin_delayed because it also references all the other rels of the
@@ -2500,9 +2489,6 @@ typedef struct RestrictInfo
/* If an outer-join clause, the outer-side relations, else NULL: */
Relids outer_relids;
- /* The relids used in the clause that are nullable by lower outer joins: */
- Relids nullable_relids;
-
/*
* Relids in the left/right side of the clause. These fields are set for
* any binary opclause.
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 41f765d342..03866de136 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -128,7 +128,6 @@ extern Expr *canonicalize_ec_expression(Expr *expr,
extern void reconsider_outer_join_clauses(PlannerInfo *root);
extern EquivalenceClass *get_eclass_for_sort_expr(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
List *opfamilies,
Oid opcintype,
Oid collation,
@@ -216,7 +215,7 @@ extern List *build_index_pathkeys(PlannerInfo *root, IndexOptInfo *index,
extern List *build_partition_pathkeys(PlannerInfo *root, RelOptInfo *partrel,
ScanDirection scandir, bool *partialkeys);
extern List *build_expression_pathkey(PlannerInfo *root, Expr *expr,
- Relids nullable_relids, Oid opno,
+ Oid opno,
Relids rel, bool create_it);
extern List *convert_subquery_pathkeys(PlannerInfo *root, RelOptInfo *rel,
List *subquery_pathkeys,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index 9dffdcfd1e..57b963c0f7 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -83,7 +83,6 @@ extern RestrictInfo *process_implied_equality(PlannerInfo *root,
Expr *item1,
Expr *item2,
Relids qualscope,
- Relids nullable_relids,
Index security_level,
bool below_outer_join,
bool both_const);
@@ -93,7 +92,6 @@ extern RestrictInfo *build_implied_join_equality(PlannerInfo *root,
Expr *item1,
Expr *item2,
Relids qualscope,
- Relids nullable_relids,
Index security_level);
extern void match_foreign_keys_to_quals(PlannerInfo *root);
diff --git a/src/include/optimizer/restrictinfo.h b/src/include/optimizer/restrictinfo.h
index 17d3b4ab05..1f092371ea 100644
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -19,7 +19,7 @@
/* Convenience macro for the common case of a valid-everywhere qual */
#define make_simple_restrictinfo(root, clause) \
- make_restrictinfo(root, clause, true, false, false, 0, NULL, NULL, NULL)
+ make_restrictinfo(root, clause, true, false, false, 0, NULL, NULL)
extern RestrictInfo *make_restrictinfo(PlannerInfo *root,
Expr *clause,
@@ -28,8 +28,7 @@ extern RestrictInfo *make_restrictinfo(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids);
+ Relids outer_relids);
extern RestrictInfo *commute_restrictinfo(RestrictInfo *rinfo, Oid comm_op);
extern bool restriction_is_or_clause(RestrictInfo *restrictinfo);
extern bool restriction_is_securely_promotable(RestrictInfo *restrictinfo,
commit f49f9042f40fa46bdc1b5bd3ab5c3be7ac1806e9
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sat Nov 5 17:44:40 2022 -0400
Use constant TRUE for "dummy" clauses when throwing back outer joins.
This improves on a hack I introduced in commit 6a6522529. If we
have a left-join clause l.x = r.y, and a WHERE clause l.x = constant,
we generate r.y = constant and then don't really have a need for the
join clause. Currently we throw the join clause back anyway after
marking it redundant, so that the join search heuristics won't think
this is a clauseless join and avoid it. That was a kluge introduced
under time pressure, and after looking at it I thought of a better
way: let's just introduce constant-TRUE "join clauses" instead,
and get rid of them at the end.
This improves the generated plans for such cases by not having to
test a redundant join clause. We can also get rid of the ugly hack
used to mark such clauses as redundant for selectivity estimation.
The code added here should go away again, once we handle these cases
as ordinary eclasses. But it seemed worth committing separately
so as to get the regression changes and selectivity simplifications
in place.
diff --git a/src/backend/optimizer/path/clausesel.c b/src/backend/optimizer/path/clausesel.c
index c08eb2b1c5..1cf565ee59 100644
--- a/src/backend/optimizer/path/clausesel.c
+++ b/src/backend/optimizer/path/clausesel.c
@@ -715,12 +715,6 @@ clause_selectivity_ext(PlannerInfo *root,
return (Selectivity) 1.0;
}
- /*
- * If the clause is marked redundant, always return 1.0.
- */
- if (rinfo->norm_selec > 1)
- return (Selectivity) 1.0;
-
/*
* If possible, cache the result of the selectivity calculation for
* the clause. We can cache if varRelid is zero or the clause
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 0737cc355f..a9f5db3ef6 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -1954,14 +1954,11 @@ create_join_clause(PlannerInfo *root,
* If we don't find any match for a set-aside outer join clause, we must
* throw it back into the regular joinclause processing by passing it to
* distribute_restrictinfo_to_rels(). If we do generate a derived clause,
- * however, the outer-join clause is redundant. We still throw it back,
- * because otherwise the join will be seen as a clauseless join and avoided
- * during join order searching; but we mark it as redundant to keep from
- * messing up the joinrel's size estimate. (This behavior means that the
- * API for this routine is uselessly complex: we could have just put all
- * the clauses into the regular processing initially. We keep it because
- * someday we might want to do something else, such as inserting "dummy"
- * joinclauses instead of real ones.)
+ * however, the outer-join clause is redundant. We must still put some
+ * clause into the regular processing, because otherwise the join will be
+ * seen as a clauseless join and avoided during join order searching.
+ * We handle this by generating a constant-TRUE clause that is marked with
+ * required_relids that make it a join between the correct relations.
*
* Outer join clauses that are marked outerjoin_delayed are special: this
* condition means that one or both VARs might go to null due to a lower
@@ -1994,10 +1991,15 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* remove it from the list */
root->left_join_clauses =
foreach_delete_current(root->left_join_clauses, cell);
- /* we throw it back anyway (see notes above) */
- /* but the thrown-back clause has no extra selectivity */
- rinfo->norm_selec = 2.0;
- rinfo->outer_selec = 1.0;
+ /* throw back a dummy replacement clause (see notes above) */
+ rinfo = make_restrictinfo(root,
+ (Expr *) makeBoolConst(true, false),
+ true, /* is_pushed_down */
+ false, /* outerjoin_delayed */
+ false, /* pseudoconstant */
+ 0, /* security_level */
+ rinfo->required_relids,
+ rinfo->outer_relids);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
@@ -2013,10 +2015,15 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* remove it from the list */
root->right_join_clauses =
foreach_delete_current(root->right_join_clauses, cell);
- /* we throw it back anyway (see notes above) */
- /* but the thrown-back clause has no extra selectivity */
- rinfo->norm_selec = 2.0;
- rinfo->outer_selec = 1.0;
+ /* throw back a dummy replacement clause (see notes above) */
+ rinfo = make_restrictinfo(root,
+ (Expr *) makeBoolConst(true, false),
+ true, /* is_pushed_down */
+ false, /* outerjoin_delayed */
+ false, /* pseudoconstant */
+ 0, /* security_level */
+ rinfo->required_relids,
+ rinfo->outer_relids);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
@@ -2034,10 +2041,15 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* remove it from the list */
root->full_join_clauses =
foreach_delete_current(root->full_join_clauses, cell);
- /* we throw it back anyway (see notes above) */
- /* but the thrown-back clause has no extra selectivity */
- rinfo->norm_selec = 2.0;
- rinfo->outer_selec = 1.0;
+ /* throw back a dummy replacement clause (see notes above) */
+ rinfo = make_restrictinfo(root,
+ (Expr *) makeBoolConst(true, false),
+ true, /* is_pushed_down */
+ false, /* outerjoin_delayed */
+ false, /* pseudoconstant */
+ 0, /* security_level */
+ rinfo->required_relids,
+ rinfo->outer_relids);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index 336a73d3b9..4b98692189 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -98,18 +98,13 @@ extract_restriction_or_clauses(PlannerInfo *root)
* joinclause that is considered safe to move to this rel by the
* parameterized-path machinery, even though what we are going to do
* with it is not exactly a parameterized path.
- *
- * However, it seems best to ignore clauses that have been marked
- * redundant (by setting norm_selec > 1). That likely can't happen
- * for OR clauses, but let's be safe.
*/
foreach(lc, rel->joininfo)
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
if (restriction_is_or_clause(rinfo) &&
- join_clause_is_movable_to(rinfo, rel) &&
- rinfo->norm_selec <= 1)
+ join_clause_is_movable_to(rinfo, rel))
{
/* Try to extract a qual for this rel only */
Expr *orclause = extract_or_clause(rinfo, rel);
@@ -356,7 +351,7 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
/* And hack cached selectivity so join size remains the same */
join_or_rinfo->norm_selec = orig_selec / or_selec;
- /* ensure result stays in sane range, in particular not "redundant" */
+ /* ensure result stays in sane range */
if (join_or_rinfo->norm_selec > 1)
join_or_rinfo->norm_selec = 1;
/* as explained above, we don't touch outer_selec */
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index 1d8912608b..c3af845acd 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -424,6 +424,21 @@ restriction_is_securely_promotable(RestrictInfo *restrictinfo,
return false;
}
+/*
+ * Detect whether a RestrictInfo's clause is constant TRUE (note that it's
+ * surely of type boolean). No such WHERE clause could survive qual
+ * canonicalization, but equivclass.c may generate such RestrictInfos for
+ * reasons discussed therein. We should drop them again when creating
+ * the finished plan, which is handled by the next few functions.
+ */
+static inline bool
+rinfo_is_constant_true(RestrictInfo *rinfo)
+{
+ return IsA(rinfo->clause, Const) &&
+ !((Const *) rinfo->clause)->constisnull &&
+ DatumGetBool(((Const *) rinfo->clause)->constvalue);
+}
+
/*
* get_actual_clauses
*
@@ -443,6 +458,7 @@ get_actual_clauses(List *restrictinfo_list)
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
Assert(!rinfo->pseudoconstant);
+ Assert(!rinfo_is_constant_true(rinfo));
result = lappend(result, rinfo->clause);
}
@@ -454,6 +470,7 @@ get_actual_clauses(List *restrictinfo_list)
*
* Extract bare clauses from 'restrictinfo_list', returning either the
* regular ones or the pseudoconstant ones per 'pseudoconstant'.
+ * Constant-TRUE clauses are dropped in any case.
*/
List *
extract_actual_clauses(List *restrictinfo_list,
@@ -466,7 +483,8 @@ extract_actual_clauses(List *restrictinfo_list,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (rinfo->pseudoconstant == pseudoconstant)
+ if (rinfo->pseudoconstant == pseudoconstant &&
+ !rinfo_is_constant_true(rinfo))
result = lappend(result, rinfo->clause);
}
return result;
@@ -477,7 +495,7 @@ extract_actual_clauses(List *restrictinfo_list,
*
* Extract bare clauses from 'restrictinfo_list', separating those that
* semantically match the join level from those that were pushed down.
- * Pseudoconstant clauses are excluded from the results.
+ * Pseudoconstant and constant-TRUE clauses are excluded from the results.
*
* This is only used at outer joins, since for plain joins we don't care
* about pushed-down-ness.
@@ -499,13 +517,15 @@ extract_actual_join_clauses(List *restrictinfo_list,
if (RINFO_IS_PUSHED_DOWN(rinfo, joinrelids))
{
- if (!rinfo->pseudoconstant)
+ if (!rinfo->pseudoconstant &&
+ !rinfo_is_constant_true(rinfo))
*otherquals = lappend(*otherquals, rinfo->clause);
}
else
{
/* joinquals shouldn't have been marked pseudoconstant */
Assert(!rinfo->pseudoconstant);
+ Assert(!rinfo_is_constant_true(rinfo));
*joinquals = lappend(*joinquals, rinfo->clause);
}
}
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index ea43313f11..6a5ff13d39 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2534,10 +2534,7 @@ typedef struct RestrictInfo
/* eval cost of clause; -1 if not yet set */
QualCost eval_cost pg_node_attr(equal_ignore);
- /*
- * selectivity for "normal" (JOIN_INNER) semantics; -1 if not yet set; >1
- * means a redundant clause
- */
+ /* selectivity for "normal" (JOIN_INNER) semantics; -1 if not yet set */
Selectivity norm_selec pg_node_attr(equal_ignore);
/* selectivity for outer join semantics; -1 if not yet set */
Selectivity outer_selec pg_node_attr(equal_ignore);
diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out
index 00f4c58238..17c423b763 100644
--- a/src/test/regress/expected/join.out
+++ b/src/test/regress/expected/join.out
@@ -4015,8 +4015,8 @@ explain (costs off)
select a.unique1, b.unique1, c.unique1, coalesce(b.twothousand, a.twothousand)
from tenk1 a left join tenk1 b on b.thousand = a.unique1 left join tenk1 c on c.unique2 =
coalesce(b.twothousand,a.twothousand)
where a.unique2 < 10 and coalesce(b.twothousand, a.twothousand) = 44;
- QUERY PLAN
----------------------------------------------------------------------------------------------
+ QUERY PLAN
+---------------------------------------------------------------
Nested Loop Left Join
-> Nested Loop Left Join
Filter: (COALESCE(b.twothousand, a.twothousand) = 44)
@@ -4027,7 +4027,7 @@ select a.unique1, b.unique1, c.unique1, coalesce(b.twothousand, a.twothousand)
-> Bitmap Index Scan on tenk1_thous_tenthous
Index Cond: (thousand = a.unique1)
-> Index Scan using tenk1_unique2 on tenk1 c
- Index Cond: ((unique2 = COALESCE(b.twothousand, a.twothousand)) AND (unique2 = 44))
+ Index Cond: (unique2 = 44)
(11 rows)
select a.unique1, b.unique1, c.unique1, coalesce(b.twothousand, a.twothousand)
@@ -4458,7 +4458,6 @@ where tt1.f1 = ss1.c0;
Output: tt4.f1
-> Nested Loop Left Join
Output: tt4.f1
- Join Filter: (tt3.f1 = tt4.f1)
-> Seq Scan on public.text_tbl tt3
Output: tt3.f1
Filter: (tt3.f1 = 'foo'::text)
@@ -4476,7 +4475,7 @@ where tt1.f1 = ss1.c0;
Output: (tt4.f1)
-> Seq Scan on public.text_tbl tt5
Output: tt4.f1
-(33 rows)
+(32 rows)
select 1 from
text_tbl as tt1
@@ -4583,24 +4582,22 @@ explain (costs off)
QUERY PLAN
-------------------------------------------------
Nested Loop Left Join
- Join Filter: (a.f1 = b.unique2)
-> Seq Scan on int4_tbl a
Filter: (f1 = 0)
-> Index Scan using tenk1_unique2 on tenk1 b
Index Cond: (unique2 = 0)
-(6 rows)
+(5 rows)
explain (costs off)
select * from tenk1 a full join tenk1 b using(unique2) where unique2 = 42;
QUERY PLAN
-------------------------------------------------
Merge Full Join
- Merge Cond: (a.unique2 = b.unique2)
-> Index Scan using tenk1_unique2 on tenk1 a
Index Cond: (unique2 = 42)
-> Index Scan using tenk1_unique2 on tenk1 b
Index Cond: (unique2 = 42)
-(6 rows)
+(5 rows)
--
-- test that quals attached to an outer join have correct semantics,
On Thu, Aug 25, 2022 at 6:27 PM Richard Guo <guofenglinux@gmail.com> wrote:
I'm not sure if I understand it correctly. If we are given the first
order from the parser, the SpecialJoinInfo for the B/C join would have
min_lefthand as containing both B and the A/B join. And this
SpecialJoinInfo would make the B/C join be invalid, which is not what we
want.
Now I see how this works from v6 patch. Once we notice identity 3
applies, we will remove the lower OJ's ojrelid from the min_lefthand or
min_righthand so that the commutation is allowed. So in this case, the
A/B join would be removed from B/C join's min_lefthand when we build the
SpecialJoinInfo for B/C join, and that makes the B/C join to be legal.
BTW, inner_join_rels can contain base Relids and OJ Relids. Maybe we
can revise the comments a bit for it atop deconstruct_recurse and
make_outerjoininfo. The same for the comments of qualscope, ojscope and
outerjoin_nonnullable atop distribute_qual_to_rels.
The README mentions restriction_is_computable_at(), I think it should be
clause_is_computable_at().
Thanks
Richard
applies, we will remove the lower OJ's ojrelid from the min_lefthand or
min_righthand so that the commutation is allowed. So in this case, the
A/B join would be removed from B/C join's min_lefthand when we build the
SpecialJoinInfo for B/C join, and that makes the B/C join to be legal.
BTW, inner_join_rels can contain base Relids and OJ Relids. Maybe we
can revise the comments a bit for it atop deconstruct_recurse and
make_outerjoininfo. The same for the comments of qualscope, ojscope and
outerjoin_nonnullable atop distribute_qual_to_rels.
The README mentions restriction_is_computable_at(), I think it should be
clause_is_computable_at().
Thanks
Richard
On Sun, Nov 6, 2022 at 5:53 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
I wrote:
> I've been working away at this patch series, and here is an up-to-date
> version.
This needs a rebase after ff8fa0bf7 and b0b72c64a. I also re-ordered
the patches so that the commit messages' claims about when regression
tests start to pass are true again. No interesting changes, though.
I'm reviewing the part about multiple version clauses, and I find a case
that may not work as expected. I tried with some query as below
(A leftjoin (B leftjoin C on (Pbc)) on (Pab)) left join D on (Pcd)
Assume Pbc is strict for B and Pcd is strict for C.
According to identity 3, we know one of its equivalent form is
((A leftjoin B on (Pab)) leftjoin C on (Pbc)) left join D on (Pcd)
For outer join clause Pcd, we would generate two versions from the first
form
Version 1: C Vars with nullingrels as {A/B}
Version 2: C Vars with nullingrels as {B/C, A/B}
I understand version 2 is reasonable as the nullingrels from parser
would be set as that. But it seems version 1 is not applicable in
either form.
Looking at the two forms again, it seems the expected two versions for
Pcd should be
Version 1: C Vars with nullingrels as {B/C}
Version 2: C Vars with nullingrels as {B/C, A/B}
With this we may have another problem that the two versions are both
applicable at the C/D join according to clause_is_computable_at(), in
both forms.
Another thing is I believe we have another equivalent form as
(A left join B on (Pab)) left join (C left join D on (Pcd)) on (Pbc)
Currently this form cannot be generated because of the issue discussed
in [1]. But someday when we can do that, I think we should have a third
version for Pcd
Version 3: C Vars with empty nullingrels
[1] https://www.postgresql.org/message-id/flat/CAMbWs4_8n5ANh_aX2PinRZ9V9mtBguhnRd4DOVt9msPgHmEMOQ%40mail.gmail.com
Thanks
Richard
that may not work as expected. I tried with some query as below
(A leftjoin (B leftjoin C on (Pbc)) on (Pab)) left join D on (Pcd)
Assume Pbc is strict for B and Pcd is strict for C.
According to identity 3, we know one of its equivalent form is
((A leftjoin B on (Pab)) leftjoin C on (Pbc)) left join D on (Pcd)
For outer join clause Pcd, we would generate two versions from the first
form
Version 1: C Vars with nullingrels as {A/B}
Version 2: C Vars with nullingrels as {B/C, A/B}
I understand version 2 is reasonable as the nullingrels from parser
would be set as that. But it seems version 1 is not applicable in
either form.
Looking at the two forms again, it seems the expected two versions for
Pcd should be
Version 1: C Vars with nullingrels as {B/C}
Version 2: C Vars with nullingrels as {B/C, A/B}
With this we may have another problem that the two versions are both
applicable at the C/D join according to clause_is_computable_at(), in
both forms.
Another thing is I believe we have another equivalent form as
(A left join B on (Pab)) left join (C left join D on (Pcd)) on (Pbc)
Currently this form cannot be generated because of the issue discussed
in [1]. But someday when we can do that, I think we should have a third
version for Pcd
Version 3: C Vars with empty nullingrels
[1] https://www.postgresql.org/message-id/flat/CAMbWs4_8n5ANh_aX2PinRZ9V9mtBguhnRd4DOVt9msPgHmEMOQ%40mail.gmail.com
Thanks
Richard
Richard Guo <guofenglinux@gmail.com> writes:
> I'm reviewing the part about multiple version clauses, and I find a case
> that may not work as expected. I tried with some query as below
> (A leftjoin (B leftjoin C on (Pbc)) on (Pab)) left join D on (Pcd)
> Assume Pbc is strict for B and Pcd is strict for C.
> According to identity 3, we know one of its equivalent form is
> ((A leftjoin B on (Pab)) leftjoin C on (Pbc)) left join D on (Pcd)
> For outer join clause Pcd, we would generate two versions from the first
> form
> Version 1: C Vars with nullingrels as {A/B}
> Version 2: C Vars with nullingrels as {B/C, A/B}
> I understand version 2 is reasonable as the nullingrels from parser
> would be set as that. But it seems version 1 is not applicable in
> either form.
Hmm. Looking at the data structures generated for the first form,
we have
B/C join:
{SPECIALJOININFO
:min_lefthand (b 2)
:min_righthand (b 3)
:syn_lefthand (b 2)
:syn_righthand (b 3)
:jointype 1
:ojrelid 4
:commute_above_l (b 7)
:commute_above_r (b 5)
:commute_below (b)
A/B join:
{SPECIALJOININFO
:min_lefthand (b 1)
:min_righthand (b 2)
:syn_lefthand (b 1)
:syn_righthand (b 2 3 4)
:jointype 1
:ojrelid 5
:commute_above_l (b)
:commute_above_r (b)
:commute_below (b 4)
everything-to-D join:
{SPECIALJOININFO
:min_lefthand (b 1 2 3 4 5)
:min_righthand (b 6)
:syn_lefthand (b 1 2 3 4 5)
:syn_righthand (b 6)
:jointype 1
:ojrelid 7
:commute_above_l (b)
:commute_above_r (b)
:commute_below (b 4)
So we've marked the 4 and 7 joins as possibly commuting, but they
cannot commute according to 7's min_lefthand set. I don't think
the extra clone condition is terribly harmful --- it's useless
but shouldn't cause any problems. However, if these joins should be
able to commute then the min_lefthand marking is preventing us
from considering legal join orders (and has been doing so all along,
that's not new in this patch). It looks to me like they should be
able to commute (giving your third form), so this is a pre-existing
planning deficiency.
Without having looked too closely, I suspect this is coming from
the delay_upper_joins/check_outerjoin_delay stuff in initsplan.c.
That's a chunk of logic that I'd like to nuke altogether, and maybe
we will be able to do so once this patchset is a bit further along.
But I've not had time to look at it yet.
I'm not entirely clear on whether the strange selection of clone
clauses for this example is a bug in process_postponed_left_join_quals
or if that function is just getting misled by the bogus min_lefthand
value.
> Looking at the two forms again, it seems the expected two versions for
> Pcd should be
> Version 1: C Vars with nullingrels as {B/C}
> Version 2: C Vars with nullingrels as {B/C, A/B}
> With this we may have another problem that the two versions are both
> applicable at the C/D join according to clause_is_computable_at(), in
> both forms.
At least when I tried it just now, clause_is_computable_at correctly
rejected the first version, because we've already computed A/B when
we are trying to form the C/D join so we expect it to be listed in
varnullingrels.
regards, tom lane
Richard Guo <guofenglinux@gmail.com> writes:
> BTW, inner_join_rels can contain base Relids and OJ Relids. Maybe we
> can revise the comments a bit for it atop deconstruct_recurse and
> make_outerjoininfo. The same for the comments of qualscope, ojscope and
> outerjoin_nonnullable atop distribute_qual_to_rels.
Yeah. I had an XXX comment about whether or not it was okay to
include OJs in inner_join_rels. I took a second look and decided it's
fine, so I removed the XXX and updated these comments.
> The README mentions restriction_is_computable_at(), I think it should be
> clause_is_computable_at().
Right. I think when I wrote that I was imagining that there'd be a
wrapper function specifically concerned with RestrictInfos, but in the
event it didn't seem useful. There's only one place that uses this,
namely subbuild_joinrel_restrictlist.
The cfbot is about to start complaining that this patchset doesn't apply
over e9e26b5e7, so here's a rebase.
regards, tom lane
commit 672d0d38cc5fdc2605b36015e849afb42066b6ab
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Wed Nov 16 14:33:58 2022 -0500
Add overview documentation.
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index 41c120e0cd..95358e814d 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -295,6 +295,239 @@ Therefore, we don't merge FROM-lists if the result would have too many
FROM-items in one list.
+Vars and PlaceHolderVars
+------------------------
+
+A Var node is simply the parse-tree representation of a table column
+reference. However, in the presence of outer joins, that concept is
+more subtle than it might seem. We need to distinguish the values of
+a Var "above" and "below" any outer join that could force the Var to
+null. As an example, consider
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE foo(t2.z)
+
+(Assume foo() is not strict, so that we can't reduce the left join to
+a plain join.) A naive implementation might try to push the foo(t2.z)
+call down to the scan of t2, but that is not correct because
+(a) what foo() should actually see for a null-extended join row is NULL,
+and (b) if foo() returns false, we should suppress the t1 row from the
+join altogether, not emit it with a null-extended t2 row. On the other
+hand, it *would* be correct (and desirable) to push that call down to
+the scan level if the query were
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y AND foo(t2.z))
+
+This motivates considering "t2.z" within the left join's ON clause
+to be a different value from "t2.z" outside the JOIN clause. The
+former can be identified with t2.z as seen at the relation scan level,
+but the latter can't.
+
+Another example occurs in connection with EquivalenceClasses (discussed
+below). Given
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE t1.x = 42
+
+we would like to use the EquivalenceClass mechanisms to derive "t2.y = 42"
+to use as a restriction clause for the scan of t2. (That works, because t2
+rows having y different from 42 cannot affect the query result.) However,
+it'd be wrong to conclude that t2.y will be equal to t1.x in every joined
+row. Part of the solution to this problem is to deem that "t2.y" in the
+ON clause refers to the relation-scan-level value of t2.y, but not to the
+value that y will have in joined rows, where it might be NULL rather than
+equal to t1.x.
+
+Therefore, Var nodes are decorated with "varnullingrels", which are sets
+of the rangetable indexes of outer joins that potentially null the Var
+at the point where it appears in the query. (Using a set, not an ordered
+list, is fine since it doesn't matter which join forced the value to null;
+and that avoids having to change the representation when we consider
+different outer-join orders.) In the examples above, all occurrences of
+t1.x would have empty varnullingrels, since the left join doesn't null t1.
+The t2 references within the JOIN ON clauses would also have empty
+varnullingrels. But outside the JOIN clauses, any Vars referencing t2
+would have varnullingrels containing the index of the JOIN's rangetable
+entry (RTE), so that they'd be understood as potentially different from
+the t2 values seen at scan level. Labeling t2.z in the WHERE clause with
+the JOIN's RT index lets us recognize that that occurrence of foo(t2.z)
+cannot be pushed down to the t2 scan level: we cannot evaluate that value
+at the scan level, but only after the join has been done.
+
+For LEFT and RIGHT outer joins, only Vars coming from the nullable side
+of the join are marked with that join's RT index. For FULL joins, Vars
+from both inputs are marked. (Such marking doesn't let us tell which
+side of the full join a Var came from; but that information can be found
+elsewhere at need.)
+
+Notionally, a Var having nonempty varnullingrels can be thought of as
+ CASE WHEN any-of-these-outer-joins-produced-a-null-extended-row
+ THEN NULL
+ ELSE the-scan-level-value-of-the-column
+ END
+It's only notional, because no such calculation is ever done explicitly.
+In a finished plan, Vars occurring in scan-level plan nodes represent
+the actual table column values, but upper-level Vars are always
+references to outputs of lower-level plan nodes. When a join node emits
+a null-extended row, it just returns nulls for the relevant output
+columns rather than copying up values from its input. Because we don't
+ever have to do this calculation explicitly, it's not necessary to
+distinguish which side of an outer join got null-extended, which'd
+otherwise be essential information for FULL JOIN cases.
+
+Outer join identity 3 (discussed above) complicates this picture
+a bit. In the form
+ A leftjoin (B leftjoin C on (Pbc)) on (Pab)
+all of the Vars in clauses Pbc and Pab will have empty varnullingrels,
+but if we start with
+ (A leftjoin B on (Pab)) leftjoin C on (Pbc)
+then the parser will have marked Pbc's B Vars with the A/B join's
+RT index, making this form artificially different from the first.
+For discussion's sake, let's denote this marking with a star:
+ (A leftjoin B on (Pab)) leftjoin C on (Pb*c)
+To cope with this, once we have detected that commuting these joins
+is legal, we generate both the Pbc and Pb*c forms of that ON clause,
+by either removing or adding the first join's RT index in the B Vars
+that the parser created. While generating paths for a plan step that
+joins B and C, we include as a relevant join qual only the form that
+is appropriate depending on whether A has already been joined to B.
+
+It's also worth noting that identity 3 makes "the left join's RT index"
+itself a bit of a fuzzy concept, since the syntactic scope of each join
+RTE will depend on which form was produced by the parser. We resolve
+this by considering that a left join's identity is determined by its
+minimum set of right-hand-side input relations. In both forms allowed
+by identity 3, we can identify the first join as having minimum RHS B
+and the second join as having minimum RHS C.
+
+Another thing to notice is that C Vars appearing outside the nested
+JOIN clauses will be marked as nulled by both left joins if the
+original parser input was in the first form of identity 3, but if the
+parser input was in the second form, such Vars will only be marked as
+nulled by the second join. This is not really a semantic problem:
+such Vars will be marked the same way throughout the upper part of the
+query, so they will all look equal() which is correct; and they will not
+look equal() to any C Var appearing in the JOIN ON clause or below these
+joins. However, when building Vars representing the outputs of join
+relations, we need to ensure that their varnullingrels are set to
+values consistent with the syntactic join order, so that they will
+appear equal() to pre-existing Vars in the upper part of the query.
+
+Outer joins also complicate handling of subquery pull-up. Consider
+
+ SELECT ..., ss.x FROM tab1
+ LEFT JOIN (SELECT *, 42 AS x FROM tab2) ss ON ...
+
+We want to be able to pull up the subquery as discussed previously,
+but we can't just replace the "ss.x" Var in the top-level SELECT list
+with the constant 42. That'd result in always emitting 42, rather
+than emitting NULL in null-extended join rows.
+
+To solve this, we introduce the concept of PlaceHolderVars.
+A PlaceHolderVar is somewhat like a Var, in that its value originates
+at a relation scan level and can then be forced to null by higher-level
+outer joins; hence PlaceHolderVars carry a set of nulling rel IDs just
+like Vars. Unlike a Var, whose original value comes from a table,
+a PlaceHolderVar's original value is defined by a query-determined
+expression ("42" in this example); so we represent the PlaceHolderVar
+as a node with that expression as child. We insert a PlaceHolderVar
+whenever subquery pullup needs to replace a subquery-referencing Var
+that has nonempty varnullingrels with an expression that is not simply a
+Var. (When the replacement expression is a pulled-up Var, we can just
+add the replaced Var's varnullingrels to its set. Also, if the replaced
+Var has empty varnullingrels, we don't need a PlaceHolderVar: there is
+nothing that'd force the value to null, so the pulled-up expression is
+fine to use as-is.) In a finished plan, a PlaceHolderVar becomes just
+the contained expression at whatever plan level it's supposed to be
+evaluated at, and then upper-level occurrences are replaced by Var
+references to that output column of the lower plan level. That causes
+the value to go to null when appropriate at an outer join, in the same
+way as for normal Vars. Thus, PlaceHolderVars are never seen outside
+the planner.
+
+PlaceHolderVars (PHVs) are more complicated than Vars in another way:
+their original value might need to be calculated at a join, not a
+base-level relation scan. This can happen when a pulled-up subquery
+contains a join. Because of this, a PHV can create a join order
+constraint that wouldn't otherwise exist, to ensure that it can
+be calculated before it is used. A PHV's expression can also contain
+LATERAL references, adding complications that are discussed below.
+
+
+Relation Identification and Qual Clause Placement
+-------------------------------------------------
+
+A qual clause obtained from WHERE or JOIN/ON can be enforced at the lowest
+scan or join level that includes all relations used in the clause. For
+this purpose we consider that outer joins listed in varnullingrels or
+phnullingrels are used in the clause, since we can't compute the qual's
+result correctly until we know whether such Vars have gone to null.
+
+The one exception to this general rule is that a non-degenerate outer
+JOIN/ON qual (one that references the non-nullable side of the join)
+cannot be enforced below that join, even if it doesn't reference the
+nullable side. Pushing it down into the non-nullable side would result
+in rows disappearing from the join's result, rather than appearing as
+null-extended rows. To handle that, when we identify such a qual we
+artificially add the join's minimum input relid set to the set of
+relations it is considered to use, forcing it to be evaluated exactly at
+that join level. The same happens for outer-join quals that mention no
+relations at all.
+
+When attaching a qual clause to a join plan node that is performing an
+outer join, the qual clause is considered a "join clause" (that is, it is
+applied before the join performs null-extension) if it does not reference
+that outer join in any varnullingrels or phnullingrels set, or a "filter
+clause" (applied after null-extension) if it does reference that outer
+join. A qual clause that originally appeared in that outer join's JOIN/ON
+will fall into the first category, since the parser would not have marked
+any of its Vars as referencing the outer join. A qual clause that
+originally came from some upper ON clause or WHERE clause will be seen as
+referencing the outer join if it references any of the nullable side's
+Vars, since those Vars will be so marked by the parser. But, if such a
+qual does not reference any nullable-side Vars, it's okay to push it down
+into the non-nullable side, so it won't get attached to the join node in
+the first place.
+
+These things lead us to identify join relations within the planner
+by the sets of base relation RT indexes plus outer join RT indexes
+that they include. In that way, the sets of relations used by qual
+clauses can be directly compared to join relations' relid sets to
+see where to place the clauses. These identifying sets are unique
+because, for any given collection of base relations, there is only
+one valid set of outer joins to have performed along the way to
+joining that set of base relations (although the order of applying
+them could vary, as discussed above).
+
+SEMI joins do not have RT indexes, because they are artifacts made by
+the planner rather than the parser. (We could create rangetable
+entries for them, but there seems no need at present.) This does not
+cause a problem for qual placement, because the nullable side of a
+semijoin is not referenceable from above the join, so there is never a
+need to cite it in varnullingrels or phnullingrels. It does not cause a
+problem for join relation identification either, since whether a semijoin
+has been completed is again implicit in the set of base relations
+included in the join.
+
+There is one additional complication for qual clause placement, which
+occurs when we have made multiple versions of an outer-join clause as
+described previously (that is, we have both "Pbc" and "Pb*c" forms of
+the same clause seen in outer join identity 3). When forming an outer
+join we only want to apply one of the redundant versions of the clause.
+If we are forming the B/C join without having yet computed the A/B
+join, it's easy to reject the "Pb*c" form since its required relid
+set includes the A/B join relid which is not in the input. However,
+if we form B/C after A/B, then both forms of the clause are applicable
+so far as that test can tell. We have to look more closely to notice
+that the "Pbc" clause form refers to relation B which is no longer
+directly accessible. While this check is straightforward, it's not
+especially cheap (see clause_is_computable_at()). To avoid doing it
+unnecessarily, we mark the variant versions of a redundant clause as
+either "has_clone" or "is_clone". When considering a clone clause,
+we must check clause_is_computable_at() to disentangle which version
+to apply at the current join level. (In debug builds, we also Assert
+that non-clone clauses are validly computable at the current level;
+but that seems too expensive for production usage.)
+
+
Optimizer Functions
-------------------
@@ -437,11 +670,10 @@ inputs.
EquivalenceClasses
------------------
-During the deconstruct_jointree() scan of the query's qual clauses, we look
-for mergejoinable equality clauses A = B whose applicability is not delayed
-by an outer join; these are called "equivalence clauses". When we find
-one, we create an EquivalenceClass containing the expressions A and B to
-record this knowledge. If we later find another equivalence clause B = C,
+During the deconstruct_jointree() scan of the query's qual clauses, we
+look for mergejoinable equality clauses A = B. When we find one, we
+create an EquivalenceClass containing the expressions A and B to record
+that they are equal. If we later find another equivalence clause B = C,
we add C to the existing EquivalenceClass for {A B}; this may require
merging two existing EquivalenceClasses. At the end of the scan, we have
sets of values that are known all transitively equal to each other. We can
@@ -473,15 +705,54 @@ asserts that at any plan node where more than one of its member values
can be computed, output rows in which the values are not all equal may
be discarded without affecting the query result. (We require all levels
of the plan to enforce EquivalenceClasses, hence a join need not recheck
-equality of values that were computable by one of its children.) For an
-ordinary EquivalenceClass that is "valid everywhere", we can further infer
-that the values are all non-null, because all mergejoinable operators are
-strict. However, we also allow equivalence clauses that appear below the
-nullable side of an outer join to form EquivalenceClasses; for these
-classes, the interpretation is that either all the values are equal, or
-all (except pseudo-constants) have gone to null. (This requires a
-limitation that non-constant members be strict, else they might not go
-to null when the other members do.) Consider for example
+equality of values that were computable by one of its children.)
+
+It's tempting to include equality clauses appearing in outer-join
+conditions as sources of EquivalenceClasses, but there's a serious
+difficulty: the resulting deductions are not valid everywhere.
+For example, given
+
+ SELECT * FROM a LEFT JOIN b ON a.x = b.y WHERE a.x = 42;
+
+we could safely derive b.y = 42 and use that in the scan of B,
+because B rows not having b.y = 42 will not contribute to the
+join result. Likewise, given
+
+ SELECT * FROM a LEFT JOIN b ON a.x = b.y AND a.x = b.z;
+
+it's all right to apply b.y = b.z while scanning B, and then only
+one of the two equality conditions need be tested at the join.
+However, if we have
+
+ SELECT * FROM a LEFT JOIN b ON a.x1 = b.y AND a.x2 = b.y;
+
+it'd be completely incorrect to push "a.x1 = a.x2" down to the scan
+of A. Rows where they are different should not be eliminated from
+the join result, but instead produce null-extended join rows.
+
+In general, therefore, we can treat outer-join equalities somewhat like
+real equivalences, but we can only produce derived clauses at that
+outer join and at scans and joins contained within its nullable side.
+(FULL JOIN conditions can't be optimized at all this way, since derived
+clauses couldn't be enforced on either side.)
+
+Another instructive example is:
+
+ SELECT *
+ FROM a LEFT JOIN
+ (SELECT * FROM b JOIN c ON b.y = c.z WHERE b.y = 10) ss
+ ON a.x = ss.y
+ ORDER BY ss.y;
+
+We can form the EquivalenceClass {b.y c.z 10} and thereby apply c.z = 10
+while scanning c. However, this does not tell us anything about the
+ss.y reference appearing in ORDER BY (which is another name for b.y*,
+that is the possibly-nulled form of b.y), so we don't get to conclude
+that sorting for the ORDER BY is unnecessary, as it would be if we could
+prove that b.y* is equal to a constant (see discussion of PathKeys
+below).
+
+Also consider this variant:
SELECT *
FROM a LEFT JOIN
@@ -489,40 +760,62 @@ to null when the other members do.) Consider for example
ON a.x = ss.y
WHERE a.x = 42;
-We can form the below-outer-join EquivalenceClass {b.y c.z 10} and thereby
-apply c.z = 10 while scanning c. (The reason we disallow outerjoin-delayed
-clauses from forming EquivalenceClasses is exactly that we want to be able
-to push any derived clauses as far down as possible.) But once above the
-outer join it's no longer necessarily the case that b.y = 10, and thus we
-cannot use such EquivalenceClasses to conclude that sorting is unnecessary
-(see discussion of PathKeys below).
-
-In this example, notice also that a.x = ss.y (really a.x = b.y) is not an
-equivalence clause because its applicability to b is delayed by the outer
-join; thus we do not try to insert b.y into the equivalence class {a.x 42}.
-But since we see that a.x has been equated to 42 above the outer join, we
-are able to form a below-outer-join class {b.y 42}; this restriction can be
-added because no b/c row not having b.y = 42 can contribute to the result
-of the outer join, and so we need not compute such rows. Now this class
-will get merged with {b.y c.z 10}, leading to the contradiction 10 = 42,
+Here, we have an EquivalenceClass {a.x 42} in addition to {b.y c.z 10},
+and we have an outer-join condition a.x = b.y (not b.y*). That lets us
+derive b.y = 42, but we can only constrain scans/joins below the left join
+that way. Nonetheless, we can still produce the contradiction 10 = 42,
which lets the planner deduce that the b/c join need not be computed at all
because none of its rows can contribute to the outer join. (This gets
implemented as a gating Result filter, since more usually the potential
contradiction involves Param values rather than just Consts, and thus has
to be checked at runtime.)
+To handle outer-join conditions this way, we put their left and right
+operands into EquivalenceClasses in the usual way. (This may result in
+creating single-item equivalence "classes", though of course these are
+still subject to merging if other equivalence clauses are found that
+mention the same Vars.) We do not merge those two EquivalenceClasses
+as would happen with an ordinary equivalence condition. Instead, the
+outer-join condition is recorded in a separate "PartialEquivalence"
+data structure, showing the EquivalenceClasses it connects and the scope
+of the outer join that it is valid within. We can make deductions as
+if the two classes were one, but only when considering a scan or join
+within the scope of the constrained equivalence. A PartialEquivalence
+can link more than two regular EquivalenceClasses, for example if we
+have outer-join clauses like "a.x = b.y AND a.x = c.z".
+
To aid in determining the sort ordering(s) that can work with a mergejoin,
we mark each mergejoinable clause with the EquivalenceClasses of its left
-and right inputs. For an equivalence clause, these are of course the same
-EquivalenceClass. For a non-equivalence mergejoinable clause (such as an
-outer-join qualification), we generate two separate EquivalenceClasses for
-the left and right inputs. This may result in creating single-item
-equivalence "classes", though of course these are still subject to merging
-if other equivalence clauses are later found to bear on the same
-expressions.
+and right inputs. For an ordinary equivalence clause these will be the
+same EquivalenceClass, since processing of the clause itself causes its
+inputs to be put into the same EquivalenceClass. But as described above,
+mergejoinable outer-join clauses will end up with different
+EquivalenceClasses for left and right sides.
+
+There is an additional complication when re-ordering outer joins according
+to identity 3. Recall that the two choices we consider for such joins are
+ A leftjoin (B leftjoin C on (Pbc)) on (Pab)
+ (A leftjoin B on (Pab)) leftjoin C on (Pb*c)
+where the star denotes varnullingrels markers on B's Vars. When Pbc
+is (or includes) a mergejoinable clause, we have something like
+ A leftjoin (B leftjoin C on (b.b = c.c)) on (Pab)
+ (A leftjoin B on (Pab)) leftjoin C on (b.b* = c.c)
+We could generate a PartialEquivalence linking b.b and c.c, but if we
+then also try to link b.b* and c.c, we end with a nonsensical conclusion
+that b.b and b.b* are equal (at least in some parts of the plan tree).
+In any case, the conclusions we could derive from such a thing would be
+largely duplicative. Conditions involving b.b* can't be computed below
+this join nest, while any conditions that can be computed would be
+duplicative of what we'd get from the b.b/c.c combination. Therefore,
+we choose to generate a PartialEquivalence linking b.b and c.c, but
+"b.b* = c.c" is handled as just an ordinary clause.
Another way that we may form a single-item EquivalenceClass is in creation
-of a PathKey to represent a desired sort order (see below). This is a bit
+of a PathKey to represent a desired sort order (see below). This happens
+if an ORDER BY or GROUP BY key is not mentioned in any equivalence
+clause. We need to reason about sort orders in such queries, and our
+representation of sort ordering is a PathKey (see below) which uses an
+EquivalenceClass, so we have to make an EquivalenceClass. This is a bit
different from the above cases because such an EquivalenceClass might
contain an aggregate function or volatile expression. (A clause containing
a volatile function will never be considered mergejoinable, even if its top
@@ -579,7 +872,7 @@ Index scans have Path.pathkeys that represent the chosen index's ordering,
if any. A single-key index would create a single-PathKey list, while a
multi-column index generates a list with one element per key index column.
Non-key columns specified in the INCLUDE clause of covering indexes don't
-have corresponding PathKeys in the list, because the have no influence on
+have corresponding PathKeys in the list, because they have no influence on
index ordering. (Actually, since an index can be scanned either forward or
backward, there are two possible sort orders and two possible PathKey lists
it can generate.)
@@ -655,14 +948,9 @@ redundancy, we save time and improve planning, since the planner will more
easily recognize equivalent orderings as being equivalent.
Another interesting property is that if the underlying EquivalenceClass
-contains a constant and is not below an outer join, then the pathkey is
-completely redundant and need not be sorted by at all! Every row must
-contain the same constant value, so there's no need to sort. (If the EC is
-below an outer join, we still have to sort, since some of the rows might
-have gone to null and others not. In this case we must be careful to pick
-a non-const member to sort by. The assumption that all the non-const
-members go to null at the same plan level is critical here, else they might
-not produce the same sort order.) This might seem pointless because users
+contains a constant, then the pathkey is completely redundant and need not
+be sorted by at all! Every interesting row must contain the same value,
+so there's no need to sort. This might seem pointless because users
are unlikely to write "... WHERE x = 42 ORDER BY x", but it allows us to
recognize when particular index columns are irrelevant to the sort order:
if we have "... WHERE x = 42 ORDER BY y", scanning an index on (x,y)
@@ -670,15 +958,6 @@ produces correctly ordered data without a sort step. We used to have very
ugly ad-hoc code to recognize that in limited contexts, but discarding
constant ECs from pathkeys makes it happen cleanly and automatically.
-You might object that a below-outer-join EquivalenceClass doesn't always
-represent the same values at every level of the join tree, and so using
-it to uniquely identify a sort order is dubious. This is true, but we
-can avoid dealing with the fact explicitly because we always consider that
-an outer join destroys any ordering of its nullable inputs. Thus, even
-if a path was sorted by {a.x} below an outer join, we'll re-sort if that
-sort ordering was important; and so using the same PathKey for both sort
-orderings doesn't create any real problem.
-
Order of processing for EquivalenceClasses and PathKeys
-------------------------------------------------------
commit 20619bbdf756b0bb7a1b514fd2254ed244d4495b
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Wed Nov 16 14:37:15 2022 -0500
Add Var.varnullingrels and PlaceHolderVar.phnullingrels fields.
These fields are always empty as of this commit, so they don't
affect any behavior, even though equal() will compare them.
Update backend/nodes/ and backend/rewrite/ infrastructure as needed.
Also add some rewrite functions we'll need later.
Note this will require a catversion bump when committed.
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index c85d8fe975..cced668f58 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -80,11 +80,13 @@ makeVar(int varno,
var->varlevelsup = varlevelsup;
/*
- * Only a few callers need to make Var nodes with varnosyn/varattnosyn
- * different from varno/varattno. We don't provide separate arguments for
- * them, but just initialize them to the given varno/varattno. This
- * reduces code clutter and chance of error for most callers.
+ * Only a few callers need to make Var nodes with non-null varnullingrels,
+ * or with varnosyn/varattnosyn different from varno/varattno. We don't
+ * provide separate arguments for them, but just initialize them to NULL
+ * and the given varno/varattno. This reduces code clutter and chance of
+ * error for most callers.
*/
+ var->varnullingrels = NULL;
var->varnosyn = (Index) varno;
var->varattnosyn = varattno;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 0a7b22f97e..692f45daa5 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -2663,6 +2663,7 @@ expression_tree_mutator_impl(Node *node,
Var *newnode;
FLATCOPY(newnode, var, Var);
+ /* Assume we need not copy the varnullingrels bitmapset */
return (Node *) newnode;
}
break;
@@ -3257,7 +3258,7 @@ expression_tree_mutator_impl(Node *node,
FLATCOPY(newnode, phv, PlaceHolderVar);
MUTATE(newnode->phexpr, phv->phexpr, Expr *);
- /* Assume we need not copy the relids bitmapset */
+ /* Assume we need not copy the relids bitmapsets */
return (Node *) newnode;
}
break;
diff --git a/src/backend/rewrite/rewriteManip.c b/src/backend/rewrite/rewriteManip.c
index 101c39553a..ab24547e6d 100644
--- a/src/backend/rewrite/rewriteManip.c
+++ b/src/backend/rewrite/rewriteManip.c
@@ -40,6 +40,20 @@ typedef struct
int win_location;
} locate_windowfunc_context;
+typedef struct
+{
+ const Bitmapset *target_relids;
+ const Bitmapset *added_relids;
+ int sublevels_up;
+} add_nulling_relids_context;
+
+typedef struct
+{
+ const Bitmapset *removable_relids;
+ const Bitmapset *except_relids;
+ int sublevels_up;
+} remove_nulling_relids_context;
+
static bool contain_aggs_of_level_walker(Node *node,
contain_aggs_of_level_context *context);
static bool locate_agg_of_level_walker(Node *node,
@@ -50,6 +64,10 @@ static bool locate_windowfunc_walker(Node *node,
static bool checkExprHasSubLink_walker(Node *node, void *context);
static Relids offset_relid_set(Relids relids, int offset);
static Relids adjust_relid_set(Relids relids, int oldrelid, int newrelid);
+static Node *add_nulling_relids_mutator(Node *node,
+ add_nulling_relids_context *context);
+static Node *remove_nulling_relids_mutator(Node *node,
+ remove_nulling_relids_context *context);
/*
@@ -348,6 +366,8 @@ OffsetVarNodes_walker(Node *node, OffsetVarNodes_context *context)
if (var->varlevelsup == context->sublevels_up)
{
var->varno += context->offset;
+ var->varnullingrels = offset_relid_set(var->varnullingrels,
+ context->offset);
if (var->varnosyn > 0)
var->varnosyn += context->offset;
}
@@ -386,6 +406,8 @@ OffsetVarNodes_walker(Node *node, OffsetVarNodes_context *context)
{
phv->phrels = offset_relid_set(phv->phrels,
context->offset);
+ phv->phnullingrels = offset_relid_set(phv->phnullingrels,
+ context->offset);
}
/* fall through to examine children */
}
@@ -510,11 +532,13 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context)
{
Var *var = (Var *) node;
- if (var->varlevelsup == context->sublevels_up &&
- var->varno == context->rt_index)
+ if (var->varlevelsup == context->sublevels_up)
{
- var->varno = context->new_index;
- /* If the syntactic referent is same RTE, fix it too */
+ if (var->varno == context->rt_index)
+ var->varno = context->new_index;
+ var->varnullingrels = adjust_relid_set(var->varnullingrels,
+ context->rt_index,
+ context->new_index);
if (var->varnosyn == context->rt_index)
var->varnosyn = context->new_index;
}
@@ -557,6 +581,9 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context)
phv->phrels = adjust_relid_set(phv->phrels,
context->rt_index,
context->new_index);
+ phv->phnullingrels = adjust_relid_set(phv->phnullingrels,
+ context->rt_index,
+ context->new_index);
}
/* fall through to examine children */
}
@@ -833,7 +860,8 @@ rangeTableEntry_used_walker(Node *node,
Var *var = (Var *) node;
if (var->varlevelsup == context->sublevels_up &&
- var->varno == context->rt_index)
+ (var->varno == context->rt_index ||
+ bms_is_member(context->rt_index, var->varnullingrels)))
return true;
return false;
}
@@ -1061,6 +1089,195 @@ AddInvertedQual(Query *parsetree, Node *qual)
}
+/*
+ * add_nulling_relids() finds Vars and PlaceHolderVars that belong to any
+ * of the target_relids, and adds added_relids to their varnullingrels
+ * and phnullingrels fields.
+ */
+Node *
+add_nulling_relids(Node *node,
+ const Bitmapset *target_relids,
+ const Bitmapset *added_relids)
+{
+ add_nulling_relids_context context;
+
+ context.target_relids = target_relids;
+ context.added_relids = added_relids;
+ context.sublevels_up = 0;
+ return query_or_expression_tree_mutator(node,
+ add_nulling_relids_mutator,
+ &context,
+ 0);
+}
+
+static Node *
+add_nulling_relids_mutator(Node *node,
+ add_nulling_relids_context *context)
+{
+ if (node == NULL)
+ return NULL;
+ if (IsA(node, Var))
+ {
+ Var *var = (Var *) node;
+
+ if (var->varlevelsup == context->sublevels_up &&
+ bms_is_member(var->varno, context->target_relids))
+ {
+ Relids newnullingrels = bms_union(var->varnullingrels,
+ context->added_relids);
+
+ /* Copy the Var ... */
+ var = copyObject(var);
+ /* ... and replace the copy's varnullingrels field */
+ var->varnullingrels = newnullingrels;
+ return (Node *) var;
+ }
+ /* Otherwise fall through to copy the Var normally */
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ if (phv->phlevelsup == context->sublevels_up &&
+ bms_overlap(phv->phrels, context->target_relids))
+ {
+ Relids newnullingrels = bms_union(phv->phnullingrels,
+ context->added_relids);
+
+ /*
+ * We don't modify the contents of the PHV's expression, only add
+ * to phnullingrels. This corresponds to assuming that the PHV
+ * will be evaluated at the same level as before, then perhaps be
+ * nulled as it bubbles up. Hence, just flat-copy the node ...
+ */
+ phv = makeNode(PlaceHolderVar);
+ memcpy(phv, node, sizeof(PlaceHolderVar));
+ /* ... and replace the copy's phnullingrels field */
+ phv->phnullingrels = newnullingrels;
+ return (Node *) phv;
+ }
+ /* Otherwise fall through to copy the PlaceHolderVar normally */
+ }
+ else if (IsA(node, Query))
+ {
+ /* Recurse into RTE or sublink subquery */
+ Query *newnode;
+
+ context->sublevels_up++;
+ newnode = query_tree_mutator((Query *) node,
+ add_nulling_relids_mutator,
+ (void *) context,
+ 0);
+ context->sublevels_up--;
+ return (Node *) newnode;
+ }
+ return expression_tree_mutator(node, add_nulling_relids_mutator,
+ (void *) context);
+}
+
+/*
+ * remove_nulling_relids() removes mentions of the specified RT index(es)
+ * in Var.varnullingrels and PlaceHolderVar.phnullingrels fields within
+ * the given expression, except in nodes belonging to rels listed in
+ * except_relids.
+ */
+Node *
+remove_nulling_relids(Node *node,
+ const Bitmapset *removable_relids,
+ const Bitmapset *except_relids)
+{
+ remove_nulling_relids_context context;
+
+ context.removable_relids = removable_relids;
+ context.except_relids = except_relids;
+ context.sublevels_up = 0;
+ return query_or_expression_tree_mutator(node,
+ remove_nulling_relids_mutator,
+ &context,
+ 0);
+}
+
+static Node *
+remove_nulling_relids_mutator(Node *node,
+ remove_nulling_relids_context *context)
+{
+ if (node == NULL)
+ return NULL;
+ if (IsA(node, Var))
+ {
+ Var *var = (Var *) node;
+
+ if (var->varlevelsup == context->sublevels_up &&
+ !bms_is_member(var->varno, context->except_relids) &&
+ bms_overlap(var->varnullingrels, context->removable_relids))
+ {
+ Relids newnullingrels = bms_difference(var->varnullingrels,
+ context->removable_relids);
+
+ /* Micro-optimization: ensure nullingrels is NULL if empty */
+ if (bms_is_empty(newnullingrels))
+ newnullingrels = NULL;
+ /* Copy the Var ... */
+ var = copyObject(var);
+ /* ... and replace the copy's varnullingrels field */
+ var->varnullingrels = newnullingrels;
+ return (Node *) var;
+ }
+ /* Otherwise fall through to copy the Var normally */
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ if (phv->phlevelsup == context->sublevels_up &&
+ !bms_overlap(phv->phrels, context->except_relids))
+ {
+ Relids newnullingrels = bms_difference(phv->phnullingrels,
+ context->removable_relids);
+
+ /*
+ * Micro-optimization: ensure nullingrels is NULL if empty.
+ *
+ * Note: it might seem desirable to remove the PHV altogether if
+ * phnullingrels goes to empty. Currently we dare not do that
+ * because we use PHVs in some cases to enforce separate identity
+ * of subexpressions; see wrap_non_vars usages in prepjointree.c.
+ */
+ if (bms_is_empty(newnullingrels))
+ newnullingrels = NULL;
+ /* Copy the PlaceHolderVar and mutate what's below ... */
+ phv = (PlaceHolderVar *)
+ expression_tree_mutator(node,
+ remove_nulling_relids_mutator,
+ (void *) context);
+ /* ... and replace the copy's phnullingrels field */
+ phv->phnullingrels = newnullingrels;
+ /* We must also update phrels, if it contains a removable RTI */
+ phv->phrels = bms_difference(phv->phrels,
+ context->removable_relids);
+ Assert(!bms_is_empty(phv->phrels));
+ return (Node *) phv;
+ }
+ /* Otherwise fall through to copy the PlaceHolderVar normally */
+ }
+ else if (IsA(node, Query))
+ {
+ /* Recurse into RTE or sublink subquery */
+ Query *newnode;
+
+ context->sublevels_up++;
+ newnode = query_tree_mutator((Query *) node,
+ remove_nulling_relids_mutator,
+ (void *) context,
+ 0);
+ context->sublevels_up--;
+ return (Node *) newnode;
+ }
+ return expression_tree_mutator(node, remove_nulling_relids_mutator,
+ (void *) context);
+}
+
+
/*
* replace_rte_variables() finds all Vars in an expression tree
* that reference a particular RTE, and replaces them with substitute
diff --git a/src/backend/utils/misc/queryjumble.c b/src/backend/utils/misc/queryjumble.c
index a8508463e7..c0b254fc6e 100644
--- a/src/backend/utils/misc/queryjumble.c
+++ b/src/backend/utils/misc/queryjumble.c
@@ -383,6 +383,11 @@ JumbleExpr(JumbleState *jstate, Node *node)
APP_JUMB(var->varno);
APP_JUMB(var->varattno);
APP_JUMB(var->varlevelsup);
+
+ /*
+ * We can omit varnullingrels, because it's fully determined
+ * by varno/varlevelsup plus the Var's query location.
+ */
}
break;
case T_Const:
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index a544b313d3..3ab6d75bfb 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -874,7 +874,7 @@ typedef struct RelOptInfo
int32 *attr_widths pg_node_attr(read_write_ignore);
/* LATERAL Vars and PHVs referenced by rel */
List *lateral_vars;
- /* rels that reference me laterally */
+ /* rels that reference this baserel laterally */
Relids lateral_referencers;
/* list of IndexOptInfo */
List *indexlist;
@@ -884,10 +884,7 @@ typedef struct RelOptInfo
BlockNumber pages;
Cardinality tuples;
double allvisfrac;
-
- /*
- * Indexes in PlannerInfo's eq_classes list of ECs that mention this rel
- */
+ /* indexes in PlannerInfo's eq_classes list of ECs that mention this rel */
Bitmapset *eclass_indexes;
PlannerInfo *subroot; /* if subquery */
List *subplan_params; /* if subquery */
@@ -2584,10 +2581,15 @@ typedef struct MergeScanSelCache
* of a plan tree. This is used during planning to represent the contained
* expression. At the end of the planning process it is replaced by either
* the contained expression or a Var referring to a lower-level evaluation of
- * the contained expression. Typically the evaluation occurs below an outer
+ * the contained expression. Generally the evaluation occurs below an outer
* join, and Var references above the outer join might thereby yield NULL
* instead of the expression value.
*
+ * phrels and phlevelsup correspond to the varno/varlevelsup fields of a
+ * plain Var, except that phrels has to be a relid set since the evaluation
+ * level of a PlaceHolderVar might be a join rather than a base relation.
+ * Likewise, phnullingrels corresponds to varnullingrels.
+ *
* Although the planner treats this as an expression node type, it is not
* recognized by the parser or executor, so we declare it here rather than
* in primnodes.h.
@@ -2600,8 +2602,10 @@ typedef struct MergeScanSelCache
* PHV. Another way in which it can happen is that initplan sublinks
* could get replaced by differently-numbered Params when sublink folding
* is done. (The end result of such a situation would be some
- * unreferenced initplans, which is annoying but not really a problem.) On
- * the same reasoning, there is no need to examine phrels.
+ * unreferenced initplans, which is annoying but not really a problem.)
+ * On the same reasoning, there is no need to examine phrels. But we do
+ * need to compare phnullingrels, as that represents effects that are
+ * external to the original value of the PHV.
*/
typedef struct PlaceHolderVar
@@ -2611,9 +2615,12 @@ typedef struct PlaceHolderVar
/* the represented expression */
Expr *phexpr pg_node_attr(equal_ignore);
- /* base relids syntactically within expr src */
+ /* base+OJ relids syntactically within expr src */
Relids phrels pg_node_attr(equal_ignore);
+ /* RT indexes of outer joins that can null PHV's value */
+ Relids phnullingrels;
+
/* ID for PHV (unique within planner run) */
Index phid;
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index f71f551782..8f133e12ac 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -180,6 +180,14 @@ typedef struct Expr
* row identity information during UPDATE/DELETE/MERGE. This value should
* never be seen outside the planner.
*
+ * varnullingrels is the set of RT indexes of outer joins that can force
+ * the Var's value to null (at the point where it appears in the query).
+ * See optimizer/README for discussion of that.
+ *
+ * varlevelsup is greater than zero in Vars that represent outer references.
+ * Note that it affects the meaning of all of varno, varnullingrels, and
+ * varnosyn, all of which refer to the range table of that query level.
+ *
* In the parser, varnosyn and varattnosyn are either identical to
* varno/varattno, or they specify the column's position in an aliased JOIN
* RTE that hides the semantic referent RTE's refname. This is a syntactic
@@ -222,6 +230,8 @@ typedef struct Var
int32 vartypmod;
/* OID of collation, or InvalidOid if none */
Oid varcollid;
+ /* RT indexes of outer joins that can replace the Var's value with null */
+ Bitmapset *varnullingrels;
/*
* for subquery variables referencing outer relations; 0 in a normal var,
diff --git a/src/include/rewrite/rewriteManip.h b/src/include/rewrite/rewriteManip.h
index f001ca41bb..351ec15612 100644
--- a/src/include/rewrite/rewriteManip.h
+++ b/src/include/rewrite/rewriteManip.h
@@ -63,6 +63,13 @@ extern bool contain_windowfuncs(Node *node);
extern int locate_windowfunc(Node *node);
extern bool checkExprHasSubLink(Node *node);
+extern Node *add_nulling_relids(Node *node,
+ const Bitmapset *target_relids,
+ const Bitmapset *added_relids);
+extern Node *remove_nulling_relids(Node *node,
+ const Bitmapset *removable_relids,
+ const Bitmapset *except_relids);
+
extern Node *replace_rte_variables(Node *node,
int target_varno, int sublevels_up,
replace_rte_variables_callback callback,
commit c8011baeddb3efc2dc0607d5e659cf269f251ee3
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Wed Nov 16 14:39:44 2022 -0500
Teach the parser to fill Var.varnullingrels correctly.
Vars emitted by the parser are now marked with RT indexes of outer
joins that can null them. (This is done purely according to the
syntax of the query; we don't consider whether an outer join could
be strength-reduced, for example.)
Although the result of this step compiles, it will fail some
regression tests due to the planner not yet knowing what to do.
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 6688c2a865..dff3b1e349 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -670,6 +670,7 @@ transformInsertStmt(ParseState *pstate, InsertStmt *stmt)
*/
sub_pstate->p_rtable = sub_rtable;
sub_pstate->p_joinexprs = NIL; /* sub_rtable has no joins */
+ sub_pstate->p_nullingrels = NIL;
sub_pstate->p_namespace = sub_namespace;
sub_pstate->p_resolve_unknowns = false;
@@ -851,7 +852,7 @@ transformInsertStmt(ParseState *pstate, InsertStmt *stmt)
/*
* Generate list of Vars referencing the RTE
*/
- exprList = expandNSItemVars(nsitem, 0, -1, NULL);
+ exprList = expandNSItemVars(pstate, nsitem, 0, -1, NULL);
/*
* Re-apply any indirection on the target column specs to the Vars
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index e01c0734d1..95590d9ed2 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -52,7 +52,8 @@
#include "utils/syscache.h"
-static int extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
+static int extractRemainingColumns(ParseState *pstate,
+ ParseNamespaceColumn *src_nscolumns,
List *src_colnames,
List **src_colnos,
List **res_colnames, List **res_colvars,
@@ -75,9 +76,11 @@ static ParseNamespaceItem *getNSItemForSpecialRelationTypes(ParseState *pstate,
static Node *transformFromClauseItem(ParseState *pstate, Node *n,
ParseNamespaceItem **top_nsitem,
List **namespace);
-static Var *buildVarFromNSColumn(ParseNamespaceColumn *nscol);
+static Var *buildVarFromNSColumn(ParseState *pstate,
+ ParseNamespaceColumn *nscol);
static Node *buildMergedJoinVar(ParseState *pstate, JoinType jointype,
Var *l_colvar, Var *r_colvar);
+static void markRelsAsNulledBy(ParseState *pstate, Node *n, int jindex);
static void setNamespaceColumnVisibility(List *namespace, bool cols_visible);
static void setNamespaceLateralState(List *namespace,
bool lateral_only, bool lateral_ok);
@@ -251,7 +254,8 @@ setTargetTable(ParseState *pstate, RangeVar *relation,
* Returns the number of columns added.
*/
static int
-extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
+extractRemainingColumns(ParseState *pstate,
+ ParseNamespaceColumn *src_nscolumns,
List *src_colnames,
List **src_colnos,
List **res_colnames, List **res_colvars,
@@ -287,7 +291,8 @@ extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
*src_colnos = lappend_int(*src_colnos, attnum);
*res_colnames = lappend(*res_colnames, lfirst(lc));
*res_colvars = lappend(*res_colvars,
- buildVarFromNSColumn(src_nscolumns + attnum - 1));
+ buildVarFromNSColumn(pstate,
+ src_nscolumns + attnum - 1));
/* Copy the input relation's nscolumn data for this column */
res_nscolumns[colcount] = src_nscolumns[attnum - 1];
colcount++;
@@ -1288,8 +1293,7 @@ transformFromClauseItem(ParseState *pstate, Node *n,
{
/*
* JOIN/USING (or NATURAL JOIN, as transformed above). Transform
- * the list into an explicit ON-condition, and generate a list of
- * merged result columns.
+ * the list into an explicit ON-condition.
*/
List *ucols = j->usingClause;
List *l_usingvars = NIL;
@@ -1307,8 +1311,6 @@ transformFromClauseItem(ParseState *pstate, Node *n,
int r_index = -1;
Var *l_colvar,
*r_colvar;
- Node *u_colvar;
- ParseNamespaceColumn *res_nscolumn;
Assert(u_colname[0] != '\0');
@@ -1372,17 +1374,109 @@ transformFromClauseItem(ParseState *pstate, Node *n,
u_colname)));
r_colnos = lappend_int(r_colnos, r_index + 1);
- l_colvar = buildVarFromNSColumn(l_nscolumns + l_index);
+ /* Build Vars to use in the generated JOIN ON clause */
+ l_colvar = buildVarFromNSColumn(pstate, l_nscolumns + l_index);
l_usingvars = lappend(l_usingvars, l_colvar);
- r_colvar = buildVarFromNSColumn(r_nscolumns + r_index);
+ r_colvar = buildVarFromNSColumn(pstate, r_nscolumns + r_index);
r_usingvars = lappend(r_usingvars, r_colvar);
+ /*
+ * While we're here, add column names to the res_colnames
+ * list. It's a bit ugly to do this here while the
+ * corresponding res_colvars entries are not made till later,
+ * but doing this later would require an additional traversal
+ * of the usingClause list.
+ */
res_colnames = lappend(res_colnames, lfirst(ucol));
+ }
+
+ /* Construct the generated JOIN ON clause */
+ j->quals = transformJoinUsingClause(pstate,
+ l_usingvars,
+ r_usingvars);
+ }
+ else if (j->quals)
+ {
+ /* User-written ON-condition; transform it */
+ j->quals = transformJoinOnClause(pstate, j, my_namespace);
+ }
+ else
+ {
+ /* CROSS JOIN: no quals */
+ }
+
+ /*
+ * If this is an outer join, now mark the appropriate child RTEs as
+ * being nulled by this join. We have finished processing the child
+ * join expressions as well as the current join's quals, which deal in
+ * non-nulled input columns. All future references to those RTEs will
+ * see possibly-nulled values, and we should mark generated Vars to
+ * account for that. In particular, the join alias Vars that we're
+ * about to build should reflect the nulling effects of this join.
+ *
+ * A difficulty with doing this is that we need the join's RT index,
+ * which we don't officially have yet. However, no other RTE can get
+ * made between here and the addRangeTableEntryForJoin call, so we can
+ * predict what the assignment will be. (Alternatively, we could call
+ * addRangeTableEntryForJoin before we have all the data computed, but
+ * this seems less ugly.)
+ */
+ j->rtindex = list_length(pstate->p_rtable) + 1;
+
+ switch (j->jointype)
+ {
+ case JOIN_INNER:
+ break;
+ case JOIN_LEFT:
+ markRelsAsNulledBy(pstate, j->rarg, j->rtindex);
+ break;
+ case JOIN_FULL:
+ markRelsAsNulledBy(pstate, j->larg, j->rtindex);
+ markRelsAsNulledBy(pstate, j->rarg, j->rtindex);
+ break;
+ case JOIN_RIGHT:
+ markRelsAsNulledBy(pstate, j->larg, j->rtindex);
+ break;
+ default:
+ /* shouldn't see any other types here */
+ elog(ERROR, "unrecognized join type: %d",
+ (int) j->jointype);
+ break;
+ }
+
+ /*
+ * Now we can construct join alias expressions for the USING columns.
+ */
+ if (j->usingClause)
+ {
+ ListCell *lc1,
+ *lc2;
+
+ /* Scan the colnos lists to recover info from the previous loop */
+ forboth(lc1, l_colnos, lc2, r_colnos)
+ {
+ int l_index = lfirst_int(lc1) - 1;
+ int r_index = lfirst_int(lc2) - 1;
+ Var *l_colvar,
+ *r_colvar;
+ Node *u_colvar;
+ ParseNamespaceColumn *res_nscolumn;
+
+ /*
+ * Note we re-build these Vars: they might have different
+ * varnullingrels than the ones made in the previous loop.
+ */
+ l_colvar = buildVarFromNSColumn(pstate, l_nscolumns + l_index);
+ r_colvar = buildVarFromNSColumn(pstate, r_nscolumns + r_index);
+
+ /* Construct the join alias Var for this column */
u_colvar = buildMergedJoinVar(pstate,
j->jointype,
l_colvar,
r_colvar);
res_colvars = lappend(res_colvars, u_colvar);
+
+ /* Construct column's res_nscolumns[] entry */
res_nscolumn = res_nscolumns + res_colindex;
res_colindex++;
if (u_colvar == (Node *) l_colvar)
@@ -1400,47 +1494,45 @@ transformFromClauseItem(ParseState *pstate, Node *n,
/*
* Merged column is not semantically equivalent to either
* input, so it needs to be referenced as the join output
- * column. We don't know the join's varno yet, so we'll
- * replace these zeroes below.
+ * column.
*/
- res_nscolumn->p_varno = 0;
+ res_nscolumn->p_varno = j->rtindex;
res_nscolumn->p_varattno = res_colindex;
res_nscolumn->p_vartype = exprType(u_colvar);
res_nscolumn->p_vartypmod = exprTypmod(u_colvar);
res_nscolumn->p_varcollid = exprCollation(u_colvar);
- res_nscolumn->p_varnosyn = 0;
+ res_nscolumn->p_varnosyn = j->rtindex;
res_nscolumn->p_varattnosyn = res_colindex;
}
}
-
- j->quals = transformJoinUsingClause(pstate,
- l_usingvars,
- r_usingvars);
- }
- else if (j->quals)
- {
- /* User-written ON-condition; transform it */
- j->quals = transformJoinOnClause(pstate, j, my_namespace);
- }
- else
- {
- /* CROSS JOIN: no quals */
}
/* Add remaining columns from each side to the output columns */
res_colindex +=
- extractRemainingColumns(l_nscolumns, l_colnames, &l_colnos,
+ extractRemainingColumns(pstate,
+ l_nscolumns, l_colnames, &l_colnos,
&res_colnames, &res_colvars,
res_nscolumns + res_colindex);
res_colindex +=
- extractRemainingColumns(r_nscolumns, r_colnames, &r_colnos,
+ extractRemainingColumns(pstate,
+ r_nscolumns, r_colnames, &r_colnos,
&res_colnames, &res_colvars,
res_nscolumns + res_colindex);
+ /* If join has an alias, it syntactically hides all inputs */
+ if (j->alias)
+ {
+ for (k = 0; k < res_colindex; k++)
+ {
+ ParseNamespaceColumn *nscol = res_nscolumns + k;
+
+ nscol->p_varnosyn = j->rtindex;
+ nscol->p_varattnosyn = k + 1;
+ }
+ }
+
/*
* Now build an RTE and nsitem for the result of the join.
- * res_nscolumns isn't totally done yet, but that's OK because
- * addRangeTableEntryForJoin doesn't examine it, only store a pointer.
*/
nsitem = addRangeTableEntryForJoin(pstate,
res_colnames,
@@ -1454,31 +1546,16 @@ transformFromClauseItem(ParseState *pstate, Node *n,
j->alias,
true);
- j->rtindex = nsitem->p_rtindex;
+ /* Verify that we correctly predicted the join's RT index */
+ Assert(j->rtindex == nsitem->p_rtindex);
+ /* Cross-check number of columns, too */
+ Assert(res_colindex == list_length(nsitem->p_names->colnames));
/*
- * Now that we know the join RTE's rangetable index, we can fix up the
- * res_nscolumns data in places where it should contain that.
+ * Save a link to the JoinExpr in the proper element of p_joinexprs.
+ * Since we maintain that list lazily, it may be necessary to fill in
+ * empty entries before we can add the JoinExpr in the right place.
*/
- Assert(res_colindex == list_length(nsitem->p_names->colnames));
- for (k = 0; k < res_colindex; k++)
- {
- ParseNamespaceColumn *nscol = res_nscolumns + k;
-
- /* fill in join RTI for merged columns */
- if (nscol->p_varno == 0)
- nscol->p_varno = j->rtindex;
- if (nscol->p_varnosyn == 0)
- nscol->p_varnosyn = j->rtindex;
- /* if join has an alias, it syntactically hides all inputs */
- if (j->alias)
- {
- nscol->p_varnosyn = j->rtindex;
- nscol->p_varattnosyn = k + 1;
- }
- }
-
- /* make a matching link to the JoinExpr for later use */
for (k = list_length(pstate->p_joinexprs) + 1; k < j->rtindex; k++)
pstate->p_joinexprs = lappend(pstate->p_joinexprs, NULL);
pstate->p_joinexprs = lappend(pstate->p_joinexprs, j);
@@ -1547,10 +1624,13 @@ transformFromClauseItem(ParseState *pstate, Node *n,
* buildVarFromNSColumn -
* build a Var node using ParseNamespaceColumn data
*
- * We assume varlevelsup should be 0, and no location is specified
+ * This is used to construct joinaliasvars entries.
+ * We can assume varlevelsup should be 0, and no location is specified.
+ * Note also that no column SELECT privilege is requested here; that would
+ * happen only if the column is actually referenced in the query.
*/
static Var *
-buildVarFromNSColumn(ParseNamespaceColumn *nscol)
+buildVarFromNSColumn(ParseState *pstate, ParseNamespaceColumn *nscol)
{
Var *var;
@@ -1564,6 +1644,10 @@ buildVarFromNSColumn(ParseNamespaceColumn *nscol)
/* makeVar doesn't offer parameters for these, so set by hand: */
var->varnosyn = nscol->p_varnosyn;
var->varattnosyn = nscol->p_varattnosyn;
+
+ /* ... and update varnullingrels */
+ markNullableIfNeeded(pstate, var);
+
return var;
}
@@ -1675,6 +1759,47 @@ buildMergedJoinVar(ParseState *pstate, JoinType jointype,
return res_node;
}
+/*
+ * markRelsAsNulledBy -
+ * Mark the given jointree node and its children as nulled by join jindex
+ */
+static void
+markRelsAsNulledBy(ParseState *pstate, Node *n, int jindex)
+{
+ int varno;
+ ListCell *lc;
+
+ /* Note: we can't see FromExpr here */
+ if (IsA(n, RangeTblRef))
+ {
+ varno = ((RangeTblRef *) n)->rtindex;
+ }
+ else if (IsA(n, JoinExpr))
+ {
+ JoinExpr *j = (JoinExpr *) n;
+
+ /* recurse to children */
+ markRelsAsNulledBy(pstate, j->larg, jindex);
+ markRelsAsNulledBy(pstate, j->rarg, jindex);
+ varno = j->rtindex;
+ }
+ else
+ {
+ elog(ERROR, "unrecognized node type: %d", (int) nodeTag(n));
+ varno = 0; /* keep compiler quiet */
+ }
+
+ /*
+ * Now add jindex to the p_nullingrels set for relation varno. Since we
+ * maintain the p_nullingrels list lazily, we might need to extend it to
+ * make the varno'th entry exist.
+ */
+ while (list_length(pstate->p_nullingrels) < varno)
+ pstate->p_nullingrels = lappend(pstate->p_nullingrels, NULL);
+ lc = list_nth_cell(pstate->p_nullingrels, varno - 1);
+ lfirst(lc) = bms_add_member((Bitmapset *) lfirst(lc), jindex);
+}
+
/*
* setNamespaceColumnVisibility -
* Convenience subroutine to update cols_visible flags in a namespace list.
diff --git a/src/backend/parser/parse_coerce.c b/src/backend/parser/parse_coerce.c
index 60908111c8..606491bd66 100644
--- a/src/backend/parser/parse_coerce.c
+++ b/src/backend/parser/parse_coerce.c
@@ -1042,7 +1042,7 @@ coerce_record_to_complex(ParseState *pstate, Node *node,
ParseNamespaceItem *nsitem;
nsitem = GetNSItemByRangeTablePosn(pstate, rtindex, sublevels_up);
- args = expandNSItemVars(nsitem, sublevels_up, vlocation, NULL);
+ args = expandNSItemVars(pstate, nsitem, sublevels_up, vlocation, NULL);
}
else
ereport(ERROR,
diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c
index e5fc708c8a..3fce9c5b62 100644
--- a/src/backend/parser/parse_expr.c
+++ b/src/backend/parser/parse_expr.c
@@ -2538,6 +2538,9 @@ transformWholeRowRef(ParseState *pstate, ParseNamespaceItem *nsitem,
/* location is not filled in by makeWholeRowVar */
result->location = location;
+ /* mark Var if it's nulled by any outer joins */
+ markNullableIfNeeded(pstate, result);
+
/* mark relation as requiring whole-row SELECT access */
markVarForSelectPriv(pstate, result);
@@ -2565,6 +2568,8 @@ transformWholeRowRef(ParseState *pstate, ParseNamespaceItem *nsitem,
rowexpr->colnames = copyObject(nsitem->p_names->colnames);
rowexpr->location = location;
+ /* XXX we ought to mark the row as possibly nullable */
+
return (Node *) rowexpr;
}
}
diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c
index 81f9ae2f02..fd1631fe75 100644
--- a/src/backend/parser/parse_relation.c
+++ b/src/backend/parser/parse_relation.c
@@ -751,6 +751,9 @@ scanNSItemForColumn(ParseState *pstate, ParseNamespaceItem *nsitem,
}
var->location = location;
+ /* Mark Var if it's nulled by any outer joins */
+ markNullableIfNeeded(pstate, var);
+
/* Require read access to the column */
markVarForSelectPriv(pstate, var);
@@ -1007,6 +1010,35 @@ searchRangeTableForCol(ParseState *pstate, const char *alias, const char *colnam
return fuzzystate;
}
+/*
+ * markNullableIfNeeded
+ * If the RTE referenced by the Var is nullable by outer join(s)
+ * at this point in the query, set var->varnullingrels to show that.
+ */
+void
+markNullableIfNeeded(ParseState *pstate, Var *var)
+{
+ int rtindex = var->varno;
+ Bitmapset *relids;
+
+ /* Find the appropriate pstate */
+ for (int lv = 0; lv < var->varlevelsup; lv++)
+ pstate = pstate->parentParseState;
+
+ /* Find currently-relevant join relids for the Var's rel */
+ if (rtindex > 0 && rtindex <= list_length(pstate->p_nullingrels))
+ relids = (Bitmapset *) list_nth(pstate->p_nullingrels, rtindex - 1);
+ else
+ relids = NULL;
+
+ /*
+ * Merge with any already-declared nulling rels. (Typically there won't
+ * be any, but let's get it right if there are.)
+ */
+ if (relids != NULL)
+ var->varnullingrels = bms_union(var->varnullingrels, relids);
+}
+
/*
* markRTEForSelectPriv
* Mark the specified column of the RTE with index rtindex
@@ -3109,7 +3141,7 @@ expandTupleDesc(TupleDesc tupdesc, Alias *eref, int count, int offset,
* the list elements mustn't be modified.
*/
List *
-expandNSItemVars(ParseNamespaceItem *nsitem,
+expandNSItemVars(ParseState *pstate, ParseNamespaceItem *nsitem,
int sublevels_up, int location,
List **colnames)
{
@@ -3145,6 +3177,10 @@ expandNSItemVars(ParseNamespaceItem *nsitem,
var->varnosyn = nscol->p_varnosyn;
var->varattnosyn = nscol->p_varattnosyn;
var->location = location;
+
+ /* ... and update varnullingrels */
+ markNullableIfNeeded(pstate, var);
+
result = lappend(result, var);
if (colnames)
*colnames = lappend(*colnames, colnameval);
@@ -3179,7 +3215,7 @@ expandNSItemAttrs(ParseState *pstate, ParseNamespaceItem *nsitem,
*var;
List *te_list = NIL;
- vars = expandNSItemVars(nsitem, sublevels_up, location, &names);
+ vars = expandNSItemVars(pstate, nsitem, sublevels_up, location, &names);
/*
* Require read access to the table. This is normally redundant with the
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index bd8057bc3e..4f5dd2e99f 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -1370,7 +1370,7 @@ ExpandSingleTable(ParseState *pstate, ParseNamespaceItem *nsitem,
List *vars;
ListCell *l;
- vars = expandNSItemVars(nsitem, sublevels_up, location, NULL);
+ vars = expandNSItemVars(pstate, nsitem, sublevels_up, location, NULL);
/*
* Require read access to the table. This is normally redundant with
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 7caff62af7..63725c8322 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1080,6 +1080,14 @@ typedef struct RangeTblEntry
* alias Vars are generated only for merged columns). We keep these
* entries only because they're needed in expandRTE() and similar code.
*
+ * Vars appearing within joinaliasvars are marked with varnullingrels sets
+ * that describe the nulling effects of this join and lower ones. This is
+ * essential for FULL JOIN cases, because the COALESCE expression only
+ * describes the semantics correctly if its inputs have been nulled by the
+ * join. For other cases, it allows expandRTE() to generate a valid
+ * representation of the join's output without consulting additional
+ * parser state.
+ *
* Within a Query loaded from a stored rule, it is possible for non-merged
* joinaliasvars items to be null pointers, which are placeholders for
* (necessarily unreferenced) columns dropped since the rule was made.
diff --git a/src/include/parser/parse_node.h b/src/include/parser/parse_node.h
index 962ebf65de..636d3231cd 100644
--- a/src/include/parser/parse_node.h
+++ b/src/include/parser/parse_node.h
@@ -115,6 +115,13 @@ typedef Node *(*CoerceParamHook) (ParseState *pstate, Param *param,
* This is one-for-one with p_rtable, but contains NULLs for non-join
* RTEs, and may be shorter than p_rtable if the last RTE(s) aren't joins.
*
+ * p_nullingrels: list of Bitmapsets associated with p_rtable entries, each
+ * containing the set of outer-join RTE indexes that can null that relation
+ * at the current point in the parse tree. This is one-for-one with p_rtable,
+ * but may be shorter than p_rtable, in which case the missing entries are
+ * implicitly empty (NULL). That rule allows us to save work when the query
+ * contains no outer joins.
+ *
* p_joinlist: list of join items (RangeTblRef and JoinExpr nodes) that
* will become the fromlist of the query's top-level FromExpr node.
*
@@ -182,6 +189,7 @@ struct ParseState
const char *p_sourcetext; /* source text, or NULL if not available */
List *p_rtable; /* range table so far */
List *p_joinexprs; /* JoinExprs for RTE_JOIN p_rtable entries */
+ List *p_nullingrels; /* Bitmapsets showing nulling outer joins */
List *p_joinlist; /* join items so far (will become FromExpr
* node's fromlist) */
List *p_namespace; /* currently-referenceable RTEs (List of
diff --git a/src/include/parser/parse_relation.h b/src/include/parser/parse_relation.h
index 484db165db..e7e72d6f3e 100644
--- a/src/include/parser/parse_relation.h
+++ b/src/include/parser/parse_relation.h
@@ -41,6 +41,7 @@ extern Node *scanNSItemForColumn(ParseState *pstate, ParseNamespaceItem *nsitem,
int location);
extern Node *colNameToVar(ParseState *pstate, const char *colname, bool localonly,
int location);
+extern void markNullableIfNeeded(ParseState *pstate, Var *var);
extern void markVarForSelectPriv(ParseState *pstate, Var *var);
extern Relation parserOpenTable(ParseState *pstate, const RangeVar *relation,
int lockmode);
@@ -109,7 +110,7 @@ extern void errorMissingColumn(ParseState *pstate,
extern void expandRTE(RangeTblEntry *rte, int rtindex, int sublevels_up,
int location, bool include_dropped,
List **colnames, List **colvars);
-extern List *expandNSItemVars(ParseNamespaceItem *nsitem,
+extern List *expandNSItemVars(ParseState *pstate, ParseNamespaceItem *nsitem,
int sublevels_up, int location,
List **colnames);
extern List *expandNSItemAttrs(ParseState *pstate, ParseNamespaceItem *nsitem,
commit 0daaa8cd09525a0d3e73545478cf4c5d015aa76f
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Wed Nov 16 15:47:48 2022 -0500
Teach the planner to cope with Vars bearing nullingrels.
The core idea of this step is to include varnullingrels in the
relid sets that qual clauses are considered to depend on.
So that we can still easily compare quals' relids to RelOptInfos'
relids, that means also adding outer join relids to the identifying
relids of join relations. Much of the bulk of this step is concerned
with fallout from the latter change.
I've resolved the previous squishiness entailed by outer join identity 3
by generating multiple versions of outer-join quals that could get moved
to a join level where they need to contain different nullingrels sets.
Now we have versions of such quals with the correct nullingrels for
each level where they could appear.
This requires a bit of new mechanism (RestrictInfo.has_clone/is_clone)
to prevent multiple versions of the same qual from getting used in the
plan. My worry about how that could work with EquivalenceClasses is
resolved by creating EquivalenceClasses only from the least-marked
version of a qual. (This doesn't really lose anything, since versions
with more nullingrels bits don't correspond to any equalities available
outside the nest of commuting outer joins.)
These extra versions of quals would also result in generating multiple
parameterized paths that differ only in what nullingrels they expect
for the Vars from the parameterization rel(s). That seems like it'd
be very wasteful, so I've arranged to generate such paths only from
the least-marked version of a qual (the has_clone version).
Unlike in the previous version of this patch, setrefs.c is able to
cross-check the nullingrel sets of most Vars and PlaceHolderVars to
ensure that they match up with what the previous plan step produces.
But there are three cases that I've so far punted on:
1. The targetlist and qpqual of an outer join node will contain
nullingrels bits for the outer join itself. To check exact matching to
the input, we'd need to know the OJ's relid as well as which input(s)
got nulled, neither of which is cheaply available in setrefs.c. For
now, it's just checking that such Vars have a superset of the input's
nullingrels bits.
2. Parameterized paths will generally refer to the least-marked version
of whichever outer-side Vars they use, which may not be what's actually
available from the outside of the nestloop. (We're relying on the join
ordering rules for that to be sensible.) Again, setrefs.c is in no
position to pass judgment on correctness, so it's just checking that
the parameter expression has a subset of the outer-side marking.
3. Row identity variables are not marked with any nullingrels, which
may not correspond to reality. I've punted on this by skipping the
checks when varattno <= 0.
Point 1 could be addressed if we were willing to add informational
fields to join plan nodes, which might be worth doing, but I'm not sure.
The other two points seem like the extra mechanisms needed for a
bulletproof check would be considerably more trouble than they'd be
worth.
There is still some confusion about which versions of a cloned qual
are actually necessary to check, which results in some extra filter
conditions showing up in a couple of regression test plans. There are
also some failure cases involving full joins that remain to be fixed.
This patch is already mighty big, so I'll address those failures
separately.
This step removes some low-hanging fruit from the old implementation,
such as the need to track lowest_nulling_outer_join during subquery
pullup. There's much more to do in that line, though.
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index 004481d608..1c921879a9 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -273,7 +273,7 @@ merge_clump(PlannerInfo *root, List *clumps, Clump *new_clump, int num_gene,
* rel once we know the final targetlist (see
* grouping_planner).
*/
- if (!bms_equal(joinrel->relids, root->all_baserels))
+ if (!bms_equal(joinrel->relids, root->all_query_rels))
generate_useful_gather_paths(root, joinrel, false);
/* Find and save the cheapest paths for this joinrel */
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 4ddaed31a4..5902c80747 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -159,27 +159,6 @@ make_one_rel(PlannerInfo *root, List *joinlist)
Index rti;
double total_pages;
- /*
- * Construct the all_baserels Relids set.
- */
- root->all_baserels = NULL;
- for (rti = 1; rti < root->simple_rel_array_size; rti++)
- {
- RelOptInfo *brel = root->simple_rel_array[rti];
-
- /* there may be empty slots corresponding to non-baserel RTEs */
- if (brel == NULL)
- continue;
-
- Assert(brel->relid == rti); /* sanity check on array */
-
- /* ignore RTEs that are "other rels" */
- if (brel->reloptkind != RELOPT_BASEREL)
- continue;
-
- root->all_baserels = bms_add_member(root->all_baserels, brel->relid);
- }
-
/* Mark base rels as to whether we care about fast-start plans */
set_base_rel_consider_startup(root);
@@ -207,6 +186,7 @@ make_one_rel(PlannerInfo *root, List *joinlist)
{
RelOptInfo *brel = root->simple_rel_array[rti];
+ /* there may be empty slots corresponding to non-baserel RTEs */
if (brel == NULL)
continue;
@@ -231,9 +211,9 @@ make_one_rel(PlannerInfo *root, List *joinlist)
rel = make_rel_from_joinlist(root, joinlist);
/*
- * The result should join all and only the query's base rels.
+ * The result should join all and only the query's base + outer-join rels.
*/
- Assert(bms_equal(rel->relids, root->all_baserels));
+ Assert(bms_equal(rel->relids, root->all_query_rels));
return rel;
}
@@ -558,7 +538,7 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* the final scan/join targetlist is available (see grouping_planner).
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- !bms_equal(rel->relids, root->all_baserels))
+ !bms_equal(rel->relids, root->all_query_rels))
generate_useful_gather_paths(root, rel, false);
/* Now find the cheapest of the paths for this rel */
@@ -879,7 +859,7 @@ set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *
* to support an uncommon usage of second-rate sampling methods. Instead,
* if there is a risk that the query might perform an unsafe join, just
* wrap the SampleScan in a Materialize node. We can check for joins by
- * counting the membership of all_baserels (note that this correctly
+ * counting the membership of all_query_rels (note that this correctly
* counts inheritance trees as single rels). If we're inside a subquery,
* we can't easily check whether a join might occur in the outer query, so
* just assume one is possible.
@@ -888,7 +868,7 @@ set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *
* so check repeatable_across_scans last, even though that's a bit odd.
*/
if ((root->query_level > 1 ||
- bms_membership(root->all_baserels) != BMS_SINGLETON) &&
+ bms_membership(root->all_query_rels) != BMS_SINGLETON) &&
!(GetTsmRoutine(rte->tablesample->tsmhandler)->repeatable_across_scans))
{
path = (Path *) create_material_path(rel, path);
@@ -970,7 +950,7 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
if (enable_partitionwise_join &&
rel->reloptkind == RELOPT_BASEREL &&
rte->relkind == RELKIND_PARTITIONED_TABLE &&
- rel->attr_needed[InvalidAttrNumber - rel->min_attr] == NULL)
+ bms_is_empty(rel->attr_needed[InvalidAttrNumber - rel->min_attr]))
rel->consider_partitionwise_join = true;
/*
@@ -3435,7 +3415,7 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
* partial paths. We'll do the same for the topmost scan/join rel
* once we know the final targetlist (see grouping_planner).
*/
- if (!bms_equal(rel->relids, root->all_baserels))
+ if (!bms_equal(rel->relids, root->all_query_rels))
generate_useful_gather_paths(root, rel, false);
/* Find and save the cheapest paths for this rel */
diff --git a/src/backend/optimizer/path/clausesel.c b/src/backend/optimizer/path/clausesel.c
index 06f836308d..c08eb2b1c5 100644
--- a/src/backend/optimizer/path/clausesel.c
+++ b/src/backend/optimizer/path/clausesel.c
@@ -218,7 +218,7 @@ clauselist_selectivity_ext(PlannerInfo *root,
if (rinfo)
{
- ok = (bms_membership(rinfo->clause_relids) == BMS_SINGLETON) &&
+ ok = (rinfo->num_base_rels == 1) &&
(is_pseudo_constant_clause_relids(lsecond(expr->args),
rinfo->right_relids) ||
(varonleft = false,
@@ -579,30 +579,6 @@ find_single_rel_for_clauses(PlannerInfo *root, List *clauses)
return NULL; /* no clauses */
}
-/*
- * bms_is_subset_singleton
- *
- * Same result as bms_is_subset(s, bms_make_singleton(x)),
- * but a little faster and doesn't leak memory.
- *
- * Is this of use anywhere else? If so move to bitmapset.c ...
- */
-static bool
-bms_is_subset_singleton(const Bitmapset *s, int x)
-{
- switch (bms_membership(s))
- {
- case BMS_EMPTY_SET:
- return true;
- case BMS_SINGLETON:
- return bms_is_member(x, s);
- case BMS_MULTIPLE:
- return false;
- }
- /* can't get here... */
- return false;
-}
-
/*
* treat_as_join_clause -
* Decide whether an operator clause is to be handled by the
@@ -631,17 +607,20 @@ treat_as_join_clause(PlannerInfo *root, Node *clause, RestrictInfo *rinfo,
else
{
/*
- * Otherwise, it's a join if there's more than one relation used. We
- * can optimize this calculation if an rinfo was passed.
+ * Otherwise, it's a join if there's more than one base relation used.
+ * We can optimize this calculation if an rinfo was passed.
*
* XXX Since we know the clause is being evaluated at a join, the
* only way it could be single-relation is if it was delayed by outer
- * joins. Although we can make use of the restriction qual estimators
- * anyway, it seems likely that we ought to account for the
- * probability of injected nulls somehow.
+ * joins. We intentionally count only baserels here, not OJs that
+ * might be present in rinfo->clause_relids, so that we direct such
+ * cases to the restriction qual estimators not join estimators.
+ * Eventually some notice should be taken of the possibility of
+ * injected nulls, but we'll likely want to do that in the restriction
+ * estimators rather than starting to treat such cases as join quals.
*/
if (rinfo)
- return (bms_membership(rinfo->clause_relids) == BMS_MULTIPLE);
+ return (rinfo->num_base_rels > 1);
else
return (NumRelids(root, clause) > 1);
}
@@ -754,7 +733,9 @@ clause_selectivity_ext(PlannerInfo *root,
* for all non-JOIN_INNER cases.
*/
if (varRelid == 0 ||
- bms_is_subset_singleton(rinfo->clause_relids, varRelid))
+ rinfo->num_base_rels == 0 ||
+ (rinfo->num_base_rels == 1 &&
+ bms_is_member(varRelid, rinfo->clause_relids)))
{
/* Cacheable --- do we already have the result? */
if (jointype == JOIN_INNER)
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 4c6b1d1f55..30ac8ae721 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4783,6 +4783,11 @@ compute_semi_anti_join_factors(PlannerInfo *root,
norm_sjinfo.syn_lefthand = outerrel->relids;
norm_sjinfo.syn_righthand = innerrel->relids;
norm_sjinfo.jointype = JOIN_INNER;
+ norm_sjinfo.ojrelid = 0;
+ norm_sjinfo.commute_above_l = NULL;
+ norm_sjinfo.commute_above_r = NULL;
+ norm_sjinfo.commute_below = NULL;
+ norm_sjinfo.oj_joinclause = NIL;
/* we don't bother trying to make the remaining fields valid */
norm_sjinfo.lhs_strict = false;
norm_sjinfo.delay_upper_joins = false;
@@ -4948,6 +4953,11 @@ approx_tuple_count(PlannerInfo *root, JoinPath *path, List *quals)
sjinfo.syn_lefthand = path->outerjoinpath->parent->relids;
sjinfo.syn_righthand = path->innerjoinpath->parent->relids;
sjinfo.jointype = JOIN_INNER;
+ sjinfo.ojrelid = 0;
+ sjinfo.commute_above_l = NULL;
+ sjinfo.commute_above_r = NULL;
+ sjinfo.commute_below = NULL;
+ sjinfo.oj_joinclause = NIL;
/* we don't bother trying to make the remaining fields valid */
sjinfo.lhs_strict = false;
sjinfo.delay_upper_joins = false;
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index e65b967b1f..349e183372 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -29,6 +29,7 @@
#include "optimizer/paths.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "rewrite/rewriteManip.h"
#include "utils/lsyscache.h"
@@ -64,7 +65,7 @@ static bool reconsider_outer_join_clause(PlannerInfo *root,
RestrictInfo *rinfo,
bool outer_on_left);
static bool reconsider_full_join_clause(PlannerInfo *root,
- RestrictInfo *rinfo);
+ FullJoinClauseInfo *fjinfo);
static Bitmapset *get_eclass_indexes_for_relids(PlannerInfo *root,
Relids relids);
static Bitmapset *get_common_eclass_indexes(PlannerInfo *root, Relids relids1,
@@ -757,6 +758,12 @@ get_eclass_for_sort_expr(PlannerInfo *root,
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL) /* must be an outer join */
+ {
+ Assert(bms_is_member(i, root->outer_join_rels));
+ continue;
+ }
+
Assert(rel->reloptkind == RELOPT_BASEREL ||
rel->reloptkind == RELOPT_DEADREL);
@@ -1113,6 +1120,12 @@ generate_base_implied_equalities(PlannerInfo *root)
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL) /* must be an outer join */
+ {
+ Assert(bms_is_member(i, root->outer_join_rels));
+ continue;
+ }
+
Assert(rel->reloptkind == RELOPT_BASEREL);
rel->eclass_indexes = bms_add_member(rel->eclass_indexes,
@@ -2015,10 +2028,12 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* Process the FULL JOIN clauses */
foreach(cell, root->full_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ FullJoinClauseInfo *fjinfo = (FullJoinClauseInfo *) lfirst(cell);
- if (reconsider_full_join_clause(root, rinfo))
+ if (reconsider_full_join_clause(root, fjinfo))
{
+ RestrictInfo *rinfo = fjinfo->rinfo;
+
found = true;
/* remove it from the list */
root->full_join_clauses =
@@ -2047,9 +2062,9 @@ reconsider_outer_join_clauses(PlannerInfo *root)
}
foreach(cell, root->full_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ FullJoinClauseInfo *fjinfo = (FullJoinClauseInfo *) lfirst(cell);
- distribute_restrictinfo_to_rels(root, rinfo);
+ distribute_restrictinfo_to_rels(root, fjinfo->rinfo);
}
}
@@ -2185,8 +2200,11 @@ reconsider_outer_join_clause(PlannerInfo *root, RestrictInfo *rinfo,
* Returns true if we were able to propagate a constant through the clause.
*/
static bool
-reconsider_full_join_clause(PlannerInfo *root, RestrictInfo *rinfo)
+reconsider_full_join_clause(PlannerInfo *root, FullJoinClauseInfo *fjinfo)
{
+ RestrictInfo *rinfo = fjinfo->rinfo;
+ SpecialJoinInfo *sjinfo = fjinfo->sjinfo;
+ Relids fjrelids = bms_make_singleton(sjinfo->ojrelid);
Expr *leftvar;
Expr *rightvar;
Oid opno,
@@ -2268,6 +2286,18 @@ reconsider_full_join_clause(PlannerInfo *root, RestrictInfo *rinfo)
cfirst = (Node *) linitial(cexpr->args);
csecond = (Node *) lsecond(cexpr->args);
+ /*
+ * The COALESCE arguments will be marked as possibly nulled by
+ * the full join, while we wish to generate clauses that apply
+ * to the join's inputs. So we must strip the join from the
+ * nullingrels fields of cfirst/csecond before comparing them
+ * to leftvar/rightvar. (Perhaps with a less hokey
+ * representation for FULL JOIN USING output columns, this
+ * wouldn't be needed?)
+ */
+ cfirst = remove_nulling_relids(cfirst, fjrelids, NULL);
+ csecond = remove_nulling_relids(csecond, fjrelids, NULL);
+
if (equal(leftvar, cfirst) && equal(rightvar, csecond))
{
coal_idx = foreach_current_index(lc2);
@@ -3204,6 +3234,12 @@ get_eclass_indexes_for_relids(PlannerInfo *root, Relids relids)
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL) /* must be an outer join */
+ {
+ Assert(bms_is_member(i, root->outer_join_rels));
+ continue;
+ }
+
ec_indexes = bms_add_members(ec_indexes, rel->eclass_indexes);
}
return ec_indexes;
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index 914bfd90bc..e24a9c14a9 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -3352,13 +3352,13 @@ check_index_predicates(PlannerInfo *root, RelOptInfo *rel)
* Add on any equivalence-derivable join clauses. Computing the correct
* relid sets for generate_join_implied_equalities is slightly tricky
* because the rel could be a child rel rather than a true baserel, and in
- * that case we must remove its parents' relid(s) from all_baserels.
+ * that case we must subtract its parents' relid(s) from all_query_rels.
*/
if (rel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- otherrels = bms_difference(root->all_baserels,
+ otherrels = bms_difference(root->all_query_rels,
find_childrel_parents(root, rel));
else
- otherrels = bms_difference(root->all_baserels, rel->relids);
+ otherrels = bms_difference(root->all_query_rels, rel->relids);
if (!bms_is_empty(otherrels))
clauselist =
@@ -3736,7 +3736,8 @@ match_index_to_operand(Node *operand,
*/
if (operand && IsA(operand, Var) &&
index->rel->relid == ((Var *) operand)->varno &&
- indkey == ((Var *) operand)->varattno)
+ indkey == ((Var *) operand)->varattno &&
+ ((Var *) operand)->varnullingrels == NULL)
return true;
}
else
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 2a3f0ab7bf..cd3f9fa0af 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -234,7 +234,9 @@ add_paths_to_joinrel(PlannerInfo *root,
* reduces the number of parameterized paths we have to deal with at
* higher join levels, without compromising the quality of the resulting
* plan. We express the restriction as a Relids set that must overlap the
- * parameterization of any proposed join path.
+ * parameterization of any proposed join path. Note: param_source_rels
+ * should contain only baserels, not OJ relids, so starting from
+ * all_baserels not all_query_rels is correct.
*/
foreach(lc, root->join_info_list)
{
@@ -365,6 +367,47 @@ allow_star_schema_join(PlannerInfo *root,
bms_nonempty_difference(inner_paramrels, outerrelids));
}
+/*
+ * If the parameterization is only partly satisfied by the outer rel,
+ * the unsatisfied part can't include any outer-join relids that could
+ * null rels of the satisfied part. That would imply that we're trying
+ * to use a clause involving a Var with nonempty varnullingrels at
+ * a join level where that value isn't yet computable.
+ */
+static inline bool
+have_unsafe_outer_join_ref(PlannerInfo *root,
+ Relids outerrelids,
+ Relids inner_paramrels)
+{
+ bool result = false;
+ Relids unsatisfied = bms_difference(inner_paramrels, outerrelids);
+
+ if (bms_overlap(unsatisfied, root->outer_join_rels))
+ {
+ ListCell *lc;
+
+ foreach(lc, root->join_info_list)
+ {
+ SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+
+ if (!bms_is_member(sjinfo->ojrelid, unsatisfied))
+ continue; /* not relevant */
+ if (bms_overlap(inner_paramrels, sjinfo->min_righthand) ||
+ (sjinfo->jointype == JOIN_FULL &&
+ bms_overlap(inner_paramrels, sjinfo->min_lefthand)))
+ {
+ result = true; /* doesn't work */
+ break;
+ }
+ }
+ }
+
+ /* Waste no memory when we reject a path here */
+ bms_free(unsatisfied);
+
+ return result;
+}
+
/*
* paraminfo_get_equal_hashops
* Determine if param_info and innerrel's lateral_vars can be hashed.
@@ -656,15 +699,16 @@ try_nestloop_path(PlannerInfo *root,
/*
* Check to see if proposed path is still parameterized, and reject if the
* parameterization wouldn't be sensible --- unless allow_star_schema_join
- * says to allow it anyway. Also, we must reject if have_dangerous_phv
- * doesn't like the look of it, which could only happen if the nestloop is
- * still parameterized.
+ * says to allow it anyway. Also, we must reject if either
+ * have_unsafe_outer_join_ref or have_dangerous_phv don't like the look of
+ * it, which could only happen if the nestloop is still parameterized.
*/
required_outer = calc_nestloop_required_outer(outerrelids, outer_paramrels,
innerrelids, inner_paramrels);
if (required_outer &&
((!bms_overlap(required_outer, extra->param_source_rels) &&
!allow_star_schema_join(root, outerrelids, inner_paramrels)) ||
+ have_unsafe_outer_join_ref(root, outerrelids, inner_paramrels) ||
have_dangerous_phv(root, outerrelids, inner_paramrels)))
{
/* Waste no memory when we reject a path here */
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 9da3ff2f9a..6c28b0a057 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -353,7 +353,10 @@ make_rels_by_clauseless_joins(PlannerInfo *root,
*
* Caller must supply not only the two rels, but the union of their relids.
* (We could simplify the API by computing joinrelids locally, but this
- * would be redundant work in the normal path through make_join_rel.)
+ * would be redundant work in the normal path through make_join_rel.
+ * Note that this value does NOT include the RT index of any outer join that
+ * might need to be performed here, so it's not the canonical identifier
+ * of the join relation.)
*
* On success, *sjinfo_p is set to NULL if this is to be a plain inner join,
* else it's set to point to the associated SpecialJoinInfo node. Also,
@@ -695,7 +698,7 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(rel1->relids, rel2->relids));
- /* Construct Relids set that identifies the joinrel. */
+ /* Construct Relids set that identifies the joinrel (without OJ as yet). */
joinrelids = bms_union(rel1->relids, rel2->relids);
/* Check validity and determine join type. */
@@ -707,6 +710,10 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
return NULL;
}
+ /* If we have an outer join, add its RTI to form the canonical relids. */
+ if (sjinfo && sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
+
/* Swap rels if needed to match the join info. */
if (reversed)
{
@@ -730,6 +737,11 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
sjinfo->syn_lefthand = rel1->relids;
sjinfo->syn_righthand = rel2->relids;
sjinfo->jointype = JOIN_INNER;
+ sjinfo->ojrelid = 0;
+ sjinfo->commute_above_l = NULL;
+ sjinfo->commute_above_r = NULL;
+ sjinfo->commute_below = NULL;
+ sjinfo->oj_joinclause = NIL;
/* we don't bother trying to make the remaining fields valid */
sjinfo->lhs_strict = false;
sjinfo->delay_upper_joins = false;
@@ -1510,8 +1522,6 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(child_rel1->relids, child_rel2->relids));
- child_joinrelids = bms_union(child_rel1->relids, child_rel2->relids);
- appinfos = find_appinfos_by_relids(root, child_joinrelids, &nappinfos);
/*
* Construct SpecialJoinInfo from parent join relations's
@@ -1521,6 +1531,15 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
child_rel1->relids,
child_rel2->relids);
+ /* Build correct join relids for child join */
+ child_joinrelids = bms_union(child_rel1->relids, child_rel2->relids);
+ if (child_sjinfo->ojrelid != 0)
+ child_joinrelids = bms_add_member(child_joinrelids,
+ child_sjinfo->ojrelid);
+
+ /* Find the AppendRelInfo structures */
+ appinfos = find_appinfos_by_relids(root, child_joinrelids, &nappinfos);
+
/*
* Construct restrictions applicable to the child join from those
* applicable to the parent join.
@@ -1536,8 +1555,7 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
{
child_joinrel = build_child_join_rel(root, child_rel1, child_rel2,
joinrel, child_restrictlist,
- child_sjinfo,
- child_sjinfo->jointype);
+ child_sjinfo);
joinrel->part_rels[cnt_parts] = child_joinrel;
joinrel->live_parts = bms_add_member(joinrel->live_parts, cnt_parts);
joinrel->all_partrels = bms_add_members(joinrel->all_partrels,
@@ -1583,6 +1601,8 @@ build_child_join_sjinfo(PlannerInfo *root, SpecialJoinInfo *parent_sjinfo,
sjinfo->syn_righthand = adjust_child_relids(sjinfo->syn_righthand,
right_nappinfos,
right_appinfos);
+ /* outer-join relids need no adjustment */
+ Assert(sjinfo->oj_joinclause == NIL); /* should be empty now */
sjinfo->semi_rhs_exprs = (List *) adjust_appendrel_attrs(root,
(Node *) sjinfo->semi_rhs_exprs,
right_nappinfos,
diff --git a/src/backend/optimizer/path/tidpath.c b/src/backend/optimizer/path/tidpath.c
index c4e035b049..71488cec00 100644
--- a/src/backend/optimizer/path/tidpath.c
+++ b/src/backend/optimizer/path/tidpath.c
@@ -59,6 +59,7 @@ IsCTIDVar(Var *var, RelOptInfo *rel)
if (var->varattno == SelfItemPointerAttributeNumber &&
var->vartype == TIDOID &&
var->varno == rel->relid &&
+ var->varnullingrels == NULL &&
var->varlevelsup == 0)
return true;
return false;
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index bbeca9a9ab..0652b3200a 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -34,7 +34,7 @@
/* local functions */
static bool join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo);
-static void remove_rel_from_query(PlannerInfo *root, int relid,
+static void remove_rel_from_query(PlannerInfo *root, int relid, int ojrelid,
Relids joinrelids);
static List *remove_rel_from_joinlist(List *joinlist, int relid, int *nremoved);
static bool rel_supports_distinctness(PlannerInfo *root, RelOptInfo *rel);
@@ -70,6 +70,7 @@ restart:
foreach(lc, root->join_info_list)
{
SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+ Relids joinrelids;
int innerrelid;
int nremoved;
@@ -84,9 +85,12 @@ restart:
*/
innerrelid = bms_singleton_member(sjinfo->min_righthand);
- remove_rel_from_query(root, innerrelid,
- bms_union(sjinfo->min_lefthand,
- sjinfo->min_righthand));
+ /* Compute the relid set for the join we are considering */
+ joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ if (sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
+
+ remove_rel_from_query(root, innerrelid, sjinfo->ojrelid, joinrelids);
/* We verify that exactly one reference gets removed from joinlist */
nremoved = 0;
@@ -188,6 +192,8 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
/* Compute the relid set for the join we are considering */
joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ if (sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
/*
* We can't remove the join if any inner-rel attributes are used above the
@@ -247,6 +253,17 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
{
RestrictInfo *restrictinfo = (RestrictInfo *) lfirst(l);
+ /*
+ * If the current join commutes with some other outer join(s) via
+ * outer join identity 3, there will be multiple clones of its join
+ * clauses in the joininfo list. We want to consider only the
+ * has_clone form of such clauses. Processing more than one form
+ * would be wasteful, and also some of the others would confuse the
+ * RINFO_IS_PUSHED_DOWN test below.
+ */
+ if (restrictinfo->is_clone)
+ continue; /* ignore it */
+
/*
* If it's not a join clause for this outer join, we can't use it.
* Note that if the clause is pushed-down, then it is logically from
@@ -306,10 +323,12 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
* no longer treated as a baserel, and that attributes of other baserels
* are no longer marked as being needed at joins involving this rel.
* Also, join quals involving the rel have to be removed from the joininfo
- * lists, but only if they belong to the outer join identified by joinrelids.
+ * lists, but only if they belong to the outer join identified by ojrelid
+ * and joinrelids.
*/
static void
-remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
+remove_rel_from_query(PlannerInfo *root, int relid, int ojrelid,
+ Relids joinrelids)
{
RelOptInfo *rel = find_base_rel(root, relid);
List *joininfos;
@@ -349,6 +368,14 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
}
}
+ /*
+ * Update all_baserels and related relid sets.
+ */
+ root->all_baserels = bms_del_member(root->all_baserels, relid);
+ root->outer_join_rels = bms_del_member(root->outer_join_rels, ojrelid);
+ root->all_query_rels = bms_del_member(root->all_query_rels, relid);
+ root->all_query_rels = bms_del_member(root->all_query_rels, ojrelid);
+
/*
* Likewise remove references from SpecialJoinInfo data structures.
*
@@ -365,6 +392,15 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
sjinfo->min_righthand = bms_del_member(sjinfo->min_righthand, relid);
sjinfo->syn_lefthand = bms_del_member(sjinfo->syn_lefthand, relid);
sjinfo->syn_righthand = bms_del_member(sjinfo->syn_righthand, relid);
+ sjinfo->min_lefthand = bms_del_member(sjinfo->min_lefthand, ojrelid);
+ sjinfo->min_righthand = bms_del_member(sjinfo->min_righthand, ojrelid);
+ sjinfo->syn_lefthand = bms_del_member(sjinfo->syn_lefthand, ojrelid);
+ sjinfo->syn_righthand = bms_del_member(sjinfo->syn_righthand, ojrelid);
+ /* relid cannot appear in these fields, but ojrelid can: */
+ sjinfo->commute_above_l = bms_del_member(sjinfo->commute_above_l, ojrelid);
+ sjinfo->commute_above_r = bms_del_member(sjinfo->commute_above_r, ojrelid);
+ sjinfo->commute_below = bms_del_member(sjinfo->commute_below, ojrelid);
+ Assert(sjinfo->oj_joinclause == NIL); /* should be empty now */
}
/*
@@ -396,8 +432,10 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
else
{
phinfo->ph_eval_at = bms_del_member(phinfo->ph_eval_at, relid);
+ phinfo->ph_eval_at = bms_del_member(phinfo->ph_eval_at, ojrelid);
Assert(!bms_is_empty(phinfo->ph_eval_at));
phinfo->ph_needed = bms_del_member(phinfo->ph_needed, relid);
+ phinfo->ph_needed = bms_del_member(phinfo->ph_needed, ojrelid);
}
}
@@ -422,7 +460,12 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
remove_join_clause_from_rels(root, rinfo, rinfo->required_relids);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrelids))
+ /*
+ * If the qual lists ojrelid in its required_relids, it must have come
+ * from above the outer join we're removing (so we need to keep it);
+ * if it does not, then it didn't and we can discard it.
+ */
+ if (bms_is_member(ojrelid, rinfo->required_relids))
{
/* Recheck that qual doesn't actually reference the target rel */
Assert(!bms_is_member(relid, rinfo->clause_relids));
@@ -434,6 +477,8 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
rinfo->required_relids = bms_copy(rinfo->required_relids);
rinfo->required_relids = bms_del_member(rinfo->required_relids,
relid);
+ rinfo->required_relids = bms_del_member(rinfo->required_relids,
+ ojrelid);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
@@ -548,6 +593,7 @@ reduce_unique_semijoins(PlannerInfo *root)
/* Compute the relid set for the join we are considering */
joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ Assert(sjinfo->ojrelid == 0); /* SEMI joins don't have RT indexes */
/*
* Since we're only considering a single-rel RHS, any join clauses it
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index fd8cbb1dc7..516984c655 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -60,16 +60,34 @@ static void process_security_barrier_quals(PlannerInfo *root,
static SpecialJoinInfo *make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
- JoinType jointype, List *clause);
+ JoinType jointype, Index ojrelid,
+ List *clause);
static void compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo,
List *clause);
+static void process_postponed_left_join_quals(PlannerInfo *root);
+static void distribute_quals_to_rels(PlannerInfo *root, List *clauses,
+ bool below_outer_join,
+ SpecialJoinInfo *sjinfo,
+ Index security_level,
+ Relids qualscope,
+ Relids ojscope,
+ Relids outerjoin_nonnullable,
+ bool allow_equivalence,
+ bool postpone_nondegenerate_clauses,
+ bool has_clone,
+ bool is_clone,
+ List **postponed_qual_list);
static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool below_outer_join,
- JoinType jointype,
+ SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
Relids ojscope,
Relids outerjoin_nonnullable,
+ bool allow_equivalence,
+ bool postpone_nondegenerate_clauses,
+ bool has_clone,
+ bool is_clone,
List **postponed_qual_list);
static bool check_outerjoin_delay(PlannerInfo *root, Relids *relids_p,
Relids *nullable_relids_p, bool is_pushed_down);
@@ -92,7 +110,7 @@ static void check_memoizable(RestrictInfo *restrictinfo);
*
* Scan the query's jointree and create baserel RelOptInfos for all
* the base relations (e.g., table, subquery, and function RTEs)
- * appearing in the jointree.
+ * appearing in the jointree. Also add their relids to all_baserels.
*
* The initial invocation must pass root->parse->jointree as the value of
* jtnode. Internally, the function recurses through the jointree.
@@ -112,6 +130,7 @@ add_base_rels_to_query(PlannerInfo *root, Node *jtnode)
int varno = ((RangeTblRef *) jtnode)->rtindex;
(void) build_simple_rel(root, varno, NULL);
+ root->all_baserels = bms_add_member(root->all_baserels, varno);
}
else if (IsA(jtnode, FromExpr))
{
@@ -230,6 +249,23 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
{
ListCell *temp;
+ /*
+ * By convention, attr_needed and ph_needed values contain only baserel
+ * relids (and of course "relation 0"), not outer-join relids. It's
+ * sufficient to keep track of this at baserel granularity, since whether
+ * an outer join has been computed at a particular join level is fully
+ * determined by the set of baserels in the join. If we included outer
+ * joins then we'd get confused by the varying sets of outer-join relids
+ * appearing in different versions of commutable outer-join clauses, and
+ * think that vars need to propagate higher than they really do. However,
+ * the presented value of where_needed will be a join relid set that may
+ * contain OJ relids, so we gotta mask here. This code assumes that
+ * "relation 0" is always presented alone, not along with other bits.
+ */
+ if (!bms_is_member(0, where_needed))
+ where_needed = bms_intersect(where_needed, root->all_baserels);
+
+ /* Should (still) have a nonempty set */
Assert(!bms_is_empty(where_needed));
foreach(temp, vars)
@@ -248,10 +284,16 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
attno -= rel->min_attr;
if (rel->attr_needed[attno] == NULL)
{
- /* Variable not yet requested, so add to rel's targetlist */
- /* XXX is copyObject necessary here? */
- rel->reltarget->exprs = lappend(rel->reltarget->exprs,
- copyObject(var));
+ /*
+ * Variable not yet requested, so add to rel's targetlist.
+ *
+ * The value available at the rel's scan level has not been
+ * nulled by any outer join, so drop its varnullingrels.
+ * (We'll put those back as we climb up the join tree.)
+ */
+ var = copyObject(var);
+ var->varnullingrels = NULL;
+ rel->reltarget->exprs = lappend(rel->reltarget->exprs, var);
/* reltarget cost and width will be computed later */
}
rel->attr_needed[attno] = bms_add_members(rel->attr_needed[attno],
@@ -547,8 +589,10 @@ create_lateral_join_info(PlannerInfo *root)
varno = -1;
while ((varno = bms_next_member(eval_at, varno)) >= 0)
{
- RelOptInfo *brel = find_base_rel(root, varno);
+ RelOptInfo *brel = find_base_rel_ignore_join(root, varno);
+ if (brel == NULL)
+ continue; /* ignore outer joins in eval_at */
brel->lateral_relids = bms_add_members(brel->lateral_relids,
phinfo->ph_lateral);
}
@@ -639,7 +683,10 @@ create_lateral_join_info(PlannerInfo *root)
{
RelOptInfo *brel2 = root->simple_rel_array[rti2];
- Assert(brel2 != NULL && brel2->reloptkind == RELOPT_BASEREL);
+ if (brel2 == NULL)
+ continue; /* must be an OJ */
+
+ Assert(brel2->reloptkind == RELOPT_BASEREL);
brel2->lateral_referencers =
bms_add_member(brel2->lateral_referencers, rti);
}
@@ -699,16 +746,27 @@ deconstruct_jointree(PlannerInfo *root)
Assert(root->parse->jointree != NULL &&
IsA(root->parse->jointree, FromExpr));
- /* this is filled as we scan the jointree */
+ /* These are filled as we scan the jointree */
+ root->outer_join_rels = NULL;
root->nullable_baserels = NULL;
result = deconstruct_recurse(root, (Node *) root->parse->jointree, false,
&qualscope, &inner_join_rels,
&postponed_qual_list);
- /* Shouldn't be any leftover quals */
+ /* Now we can form the value of all_query_rels, too */
+ root->all_query_rels = bms_union(root->all_baserels, root->outer_join_rels);
+
+ /* Shouldn't be any leftover postponed quals */
Assert(postponed_qual_list == NIL);
+ /*
+ * However, if there were any special joins then we may well have some
+ * postponed LEFT JOIN clauses to deal with.
+ */
+ if (root->join_info_list)
+ process_postponed_left_join_quals(root);
+
return result;
}
@@ -721,10 +779,10 @@ deconstruct_jointree(PlannerInfo *root)
* below_outer_join is true if this node is within the nullable side of a
* higher-level outer join
* Outputs:
- * *qualscope gets the set of base Relids syntactically included in this
+ * *qualscope gets the set of base+OJ Relids syntactically included in this
* jointree node (do not modify or free this, as it may also be pointed
* to by RestrictInfo and SpecialJoinInfo nodes)
- * *inner_join_rels gets the set of base Relids syntactically included in
+ * *inner_join_rels gets the set of base+OJ Relids syntactically included in
* inner joins appearing at or below this jointree node (do not modify
* or free this, either)
* *postponed_qual_list is a list of PostponedQual structs, which we can
@@ -820,10 +878,10 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
if (bms_is_subset(pq->relids, *qualscope))
distribute_qual_to_rels(root, pq->qual,
- below_outer_join, JOIN_INNER,
+ below_outer_join, NULL,
root->qual_security_level,
*qualscope, NULL, NULL,
- NULL);
+ true, false, false, false, NULL);
else
*postponed_qual_list = lappend(*postponed_qual_list, pq);
}
@@ -831,16 +889,12 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
/*
* Now process the top-level quals.
*/
- foreach(l, (List *) f->quals)
- {
- Node *qual = (Node *) lfirst(l);
-
- distribute_qual_to_rels(root, qual,
- below_outer_join, JOIN_INNER,
- root->qual_security_level,
- *qualscope, NULL, NULL,
- postponed_qual_list);
- }
+ distribute_quals_to_rels(root, (List *) f->quals,
+ below_outer_join, NULL,
+ root->qual_security_level,
+ *qualscope, NULL, NULL,
+ true, false, false, false,
+ postponed_qual_list);
}
else if (IsA(jtnode, JoinExpr))
{
@@ -857,6 +911,7 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
*rightjoinlist;
List *my_quals;
SpecialJoinInfo *sjinfo;
+ bool postpone_nondegenerate_clauses;
ListCell *l;
/*
@@ -900,6 +955,13 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ /* caution: ANTI join derived from SEMI will lack rtindex */
+ if (j->rtindex != 0)
+ {
+ *qualscope = bms_add_member(*qualscope, j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
+ j->rtindex);
+ }
*inner_join_rels = bms_union(left_inners, right_inners);
nonnullable_rels = leftids;
nullable_rels = rightids;
@@ -914,6 +976,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ /* SEMI join never has rtindex, so don't add to qualscope */
+ Assert(j->rtindex == 0);
*inner_join_rels = bms_union(left_inners, right_inners);
/* Semi join adds no restrictions for quals */
nonnullable_rels = NULL;
@@ -935,6 +999,10 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
&rightids, &right_inners,
&child_postponed_quals);
*qualscope = bms_union(leftids, rightids);
+ Assert(j->rtindex != 0);
+ *qualscope = bms_add_member(*qualscope, j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
+ j->rtindex);
*inner_join_rels = bms_union(left_inners, right_inners);
/* each side is both outer and inner */
nonnullable_rels = *qualscope;
@@ -994,12 +1062,28 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
leftids, rightids,
*inner_join_rels,
j->jointype,
+ j->rtindex,
my_quals);
if (j->jointype == JOIN_SEMI)
ojscope = NULL;
else
+ {
ojscope = bms_union(sjinfo->min_lefthand,
sjinfo->min_righthand);
+
+ /*
+ * Add back any commutable lower OJ relids that were removed
+ * from min_lefthand or min_righthand, else the ojscope
+ * cross-check in distribute_qual_to_rels will complain. If
+ * any such OJs were removed, we will postpone processing of
+ * non-degenerate clauses, so this addition doesn't affect
+ * anything except that cross-check and some Asserts. Real
+ * clause positioning decisions will be made later, when we
+ * revisit the postponed clauses.
+ */
+ if (sjinfo->commute_below)
+ ojscope = bms_add_members(ojscope, sjinfo->commute_below);
+ }
}
else
{
@@ -1007,18 +1091,26 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
ojscope = NULL;
}
+ /*
+ * If it's a left join with a join clause that is strict for the LHS,
+ * then we need to postpone handling of any non-degenerate join
+ * clauses, in case the join is able to commute with another left join
+ * per identity 3. (Degenerate clauses need not be postponed, since
+ * they will drop down below this join anyway.)
+ */
+ postpone_nondegenerate_clauses = (j->jointype == JOIN_LEFT &&
+ sjinfo->lhs_strict);
+
/* Process the JOIN's qual clauses */
- foreach(l, my_quals)
- {
- Node *qual = (Node *) lfirst(l);
-
- distribute_qual_to_rels(root, qual,
- below_outer_join, j->jointype,
- root->qual_security_level,
- *qualscope,
- ojscope, nonnullable_rels,
- postponed_qual_list);
- }
+ distribute_quals_to_rels(root, my_quals,
+ below_outer_join, sjinfo,
+ root->qual_security_level,
+ *qualscope,
+ ojscope, nonnullable_rels,
+ true, /* allow_equivalence */
+ postpone_nondegenerate_clauses,
+ false, false, /* not clones */
+ postponed_qual_list);
/* Now we can add the SpecialJoinInfo to join_info_list */
if (sjinfo)
@@ -1102,27 +1194,24 @@ process_security_barrier_quals(PlannerInfo *root,
foreach(lc, rte->securityQuals)
{
List *qualset = (List *) lfirst(lc);
- ListCell *lc2;
- foreach(lc2, qualset)
- {
- Node *qual = (Node *) lfirst(lc2);
-
- /*
- * We cheat to the extent of passing ojscope = qualscope rather
- * than its more logical value of NULL. The only effect this has
- * is to force a Var-free qual to be evaluated at the rel rather
- * than being pushed up to top of tree, which we don't want.
- */
- distribute_qual_to_rels(root, qual,
- below_outer_join,
- JOIN_INNER,
- security_level,
- qualscope,
- qualscope,
- NULL,
- NULL);
- }
+ /*
+ * We cheat to the extent of passing ojscope = qualscope rather than
+ * its more logical value of NULL. The only effect this has is to
+ * force a Var-free qual to be evaluated at the rel rather than being
+ * pushed up to top of tree, which we don't want.
+ */
+ distribute_quals_to_rels(root, qualset,
+ below_outer_join,
+ NULL,
+ security_level,
+ qualscope,
+ qualscope,
+ NULL,
+ true,
+ false,
+ false, false, /* not clones */
+ NULL);
security_level++;
}
@@ -1135,10 +1224,11 @@ process_security_barrier_quals(PlannerInfo *root,
* Build a SpecialJoinInfo for the current outer join
*
* Inputs:
- * left_rels: the base Relids syntactically on outer side of join
- * right_rels: the base Relids syntactically on inner side of join
- * inner_join_rels: base Relids participating in inner joins below this one
+ * left_rels: the base+OJ Relids syntactically on outer side of join
+ * right_rels: the base+OJ Relids syntactically on inner side of join
+ * inner_join_rels: base+OJ Relids participating in inner joins below this one
* jointype: what it says (must always be LEFT, FULL, SEMI, or ANTI)
+ * ojrelid: RT index of the join RTE (0 for SEMI, which isn't in the RT list)
* clause: the outer join's join condition (in implicit-AND format)
*
* The node should eventually be appended to root->join_info_list, but we
@@ -1152,7 +1242,8 @@ static SpecialJoinInfo *
make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
- JoinType jointype, List *clause)
+ JoinType jointype, Index ojrelid,
+ List *clause)
{
SpecialJoinInfo *sjinfo = makeNode(SpecialJoinInfo);
Relids clause_relids;
@@ -1200,6 +1291,12 @@ make_outerjoininfo(PlannerInfo *root,
sjinfo->syn_lefthand = left_rels;
sjinfo->syn_righthand = right_rels;
sjinfo->jointype = jointype;
+ sjinfo->ojrelid = ojrelid;
+ /* these fields may get added to later: */
+ sjinfo->commute_above_l = NULL;
+ sjinfo->commute_above_r = NULL;
+ sjinfo->commute_below = NULL;
+ sjinfo->oj_joinclause = NIL;
/* this always starts out false */
sjinfo->delay_upper_joins = false;
@@ -1247,6 +1344,7 @@ make_outerjoininfo(PlannerInfo *root,
foreach(l, root->join_info_list)
{
SpecialJoinInfo *otherinfo = (SpecialJoinInfo *) lfirst(l);
+ bool have_unsafe_phvs;
/*
* A full join is an optimization barrier: we can't associate into or
@@ -1262,6 +1360,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_lefthand = bms_add_member(min_lefthand,
+ otherinfo->ojrelid);
}
if (bms_overlap(right_rels, otherinfo->syn_lefthand) ||
bms_overlap(right_rels, otherinfo->syn_righthand))
@@ -1270,11 +1371,26 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_righthand = bms_add_member(min_righthand,
+ otherinfo->ojrelid);
}
/* Needn't do anything else with the full join */
continue;
}
+ /*
+ * If our join condition contains any PlaceHolderVars that need to be
+ * evaluated above the lower OJ, then we can't commute with it.
+ */
+ if (otherinfo->ojrelid != 0)
+ have_unsafe_phvs =
+ contain_placeholder_references_to(root,
+ (Node *) clause,
+ otherinfo->ojrelid);
+ else
+ have_unsafe_phvs = false;
+
/*
* For a lower OJ in our LHS, if our join condition uses the lower
* join's RHS and is not strict for that rel, we must preserve the
@@ -1282,23 +1398,44 @@ make_outerjoininfo(PlannerInfo *root,
* min_lefthand. (We must use its full syntactic relset, not just its
* min_lefthand + min_righthand. This is because there might be other
* OJs below this one that this one can commute with, but we cannot
- * commute with them if we don't with this one.) Also, if the current
- * join is a semijoin or antijoin, we must preserve ordering
- * regardless of strictness.
+ * commute with them if we don't with this one.) Also, if we have
+ * unsafe PHVs or the current join is a semijoin or antijoin, we must
+ * preserve ordering regardless of strictness.
*
* Note: I believe we have to insist on being strict for at least one
* rel in the lower OJ's min_righthand, not its whole syn_righthand.
+ *
+ * When we don't need to preserve ordering, check to see if outer join
+ * identity 3 applies, and if so, remove the lower OJ's ojrelid from
+ * our min_lefthand so that commutation is allowed.
*/
if (bms_overlap(left_rels, otherinfo->syn_righthand))
{
if (bms_overlap(clause_relids, otherinfo->syn_righthand) &&
- (jointype == JOIN_SEMI || jointype == JOIN_ANTI ||
+ (have_unsafe_phvs ||
+ jointype == JOIN_SEMI || jointype == JOIN_ANTI ||
!bms_overlap(strict_relids, otherinfo->min_righthand)))
{
+ /* Preserve ordering */
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_lefthand);
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_lefthand = bms_add_member(min_lefthand,
+ otherinfo->ojrelid);
+ }
+ else if (jointype == JOIN_LEFT &&
+ otherinfo->jointype == JOIN_LEFT &&
+ bms_overlap(strict_relids, otherinfo->min_righthand))
+ {
+ /* Identity 3 applies, so remove the ordering restriction */
+ min_lefthand = bms_del_member(min_lefthand, otherinfo->ojrelid);
+ /* Add commutability markers to both SpecialJoinInfos */
+ otherinfo->commute_above_l =
+ bms_add_member(otherinfo->commute_above_l, ojrelid);
+ sjinfo->commute_below =
+ bms_add_member(sjinfo->commute_below, otherinfo->ojrelid);
}
}
@@ -1313,8 +1450,8 @@ make_outerjoininfo(PlannerInfo *root,
* up with SpecialJoinInfos with identical min_righthands, which can
* confuse join_is_legal (see discussion in backend/optimizer/README).
*
- * Also, we must preserve ordering anyway if either the current join
- * or the lower OJ is either a semijoin or an antijoin.
+ * Also, we must preserve ordering anyway if we have unsafe PHVs, or
+ * if either this join or the lower OJ is a semijoin or antijoin.
*
* Here, we have to consider that "our join condition" includes any
* clauses that syntactically appeared above the lower OJ and below
@@ -1326,21 +1463,43 @@ make_outerjoininfo(PlannerInfo *root,
* join condition are not affected by them. The net effect is
* therefore sufficiently represented by the delay_upper_joins flag
* saved for us by check_outerjoin_delay.
+ *
+ * When we don't need to preserve ordering, check to see if outer join
+ * identity 3 applies, and if so, remove the lower OJ's ojrelid from
+ * our min_righthand so that commutation is allowed.
*/
if (bms_overlap(right_rels, otherinfo->syn_righthand))
{
if (bms_overlap(clause_relids, otherinfo->syn_righthand) ||
!bms_overlap(clause_relids, otherinfo->min_lefthand) ||
+ have_unsafe_phvs ||
jointype == JOIN_SEMI ||
jointype == JOIN_ANTI ||
otherinfo->jointype == JOIN_SEMI ||
otherinfo->jointype == JOIN_ANTI ||
!otherinfo->lhs_strict || otherinfo->delay_upper_joins)
{
+ /* Preserve ordering */
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_lefthand);
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_righthand = bms_add_member(min_righthand,
+ otherinfo->ojrelid);
+ }
+ else if (jointype == JOIN_LEFT &&
+ otherinfo->jointype == JOIN_LEFT &&
+ otherinfo->lhs_strict)
+ {
+ /* Identity 3 applies, so remove the ordering restriction */
+ min_righthand = bms_del_member(min_righthand,
+ otherinfo->ojrelid);
+ /* Add commutability markers to both SpecialJoinInfos */
+ otherinfo->commute_above_r =
+ bms_add_member(otherinfo->commute_above_r, ojrelid);
+ sjinfo->commute_below =
+ bms_add_member(sjinfo->commute_below, otherinfo->ojrelid);
}
}
}
@@ -1565,6 +1724,231 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
sjinfo->semi_rhs_exprs = semi_rhs_exprs;
}
+/*
+ * process_postponed_left_join_quals
+ * Adjust LEFT JOIN quals to be suitable for commuted-left-join cases,
+ * then push them into the joinqual lists and EquivalenceClass structures.
+ *
+ * This runs immediately after we've completed the deconstruct_recurse scan.
+ */
+static void
+process_postponed_left_join_quals(PlannerInfo *root)
+{
+ List *join_info_list_orig = root->join_info_list;
+ ListCell *lc;
+
+ /*
+ * XXX hack: when we call distribute_qual_to_rels to process one of these
+ * quals, neither the owning SpecialJoinInfo nor any later ones can appear
+ * in root->join_info_list, else the wrong things will happen. Fake it
+ * out by emptying join_info_list and rebuilding it as we go. This works
+ * because join_info_list is only appended to during deconstruct_recurse,
+ * so we know we are examining SpecialJoinInfos bottom-up, just like the
+ * first time. Maybe we can get rid of this hack later, if we can fix
+ * things so that distribute_qual_to_rels doesn't consult join_info_list.
+ */
+ root->join_info_list = NIL;
+
+ foreach(lc, join_info_list_orig)
+ {
+ SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+ Relids qualscope,
+ ojscope,
+ nonnullable_rels;
+
+ if (sjinfo->oj_joinclause == NIL) /* nothing to do here */
+ {
+ root->join_info_list = lappend(root->join_info_list, sjinfo);
+ continue;
+ }
+
+ /* Recompute syntactic and semantic scopes of this left join */
+ qualscope = bms_union(sjinfo->syn_lefthand, sjinfo->syn_righthand);
+ qualscope = bms_add_member(qualscope, sjinfo->ojrelid);
+ ojscope = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ nonnullable_rels = sjinfo->syn_lefthand;
+
+ /*
+ * If this join can commute with any other ones per outer-join
+ * identity 3, and it is the one providing the join clause with
+ * flexible semantics, then we have to generate variants of the join
+ * clause with different nullingrels labeling. Otherwise, just push
+ * out the postponed clause as-is.
+ */
+ Assert(sjinfo->lhs_strict); /* else we shouldn't be here */
+ if (sjinfo->commute_above_r ||
+ bms_overlap(sjinfo->commute_below, sjinfo->syn_lefthand))
+ {
+ Relids joins_above;
+ Relids joins_below;
+ Relids joins_so_far;
+ List *quals;
+ ListCell *lc2;
+
+ /*
+ * Put any OJ relids that were removed from min_righthand back
+ * into ojscope, else distribute_qual_to_rels will complain.
+ */
+ ojscope = bms_join(ojscope, bms_intersect(sjinfo->commute_below,
+ sjinfo->syn_righthand));
+
+ /* Identify the outer joins this one commutes with */
+ joins_above = sjinfo->commute_above_r;
+ joins_below = bms_intersect(sjinfo->commute_below,
+ sjinfo->syn_lefthand);
+
+ /*
+ * Generate qual variants with different sets of nullingrels bits.
+ * We only need bit-sets that correspond to the successively less
+ * deeply syntactically-nested subsets of this join and its
+ * commutators. That's true first because obviously only those
+ * forms of the Vars and PHVs could appear elsewhere in the query,
+ * and second because the outer join identities do not provide a
+ * way to re-order such joins in a way that would require
+ * different marking. (That is, while the current join may
+ * commute with several others, none of those others can commute
+ * with each other.) To visit the interesting SpecialJoinInfos in
+ * syntactic nesting order, we rely on the join_info_list to be
+ * ordered that way.
+ *
+ * We first strip out all the nullingrels bits corresponding to
+ * commutating joins below this one, and then successively put
+ * them back as we crawl up the join stack.
+ */
+ quals = sjinfo->oj_joinclause;
+ if (!bms_is_empty(joins_below))
+ quals = (List *) remove_nulling_relids((Node *) quals,
+ joins_below,
+ NULL);
+
+ joins_so_far = NULL;
+ foreach(lc2, join_info_list_orig)
+ {
+ SpecialJoinInfo *othersj = (SpecialJoinInfo *) lfirst(lc2);
+ bool below_sjinfo = false;
+ bool above_sjinfo = false;
+ Relids this_qualscope;
+ Relids this_ojscope;
+ bool allow_equivalence,
+ has_clone,
+ is_clone;
+
+ if (bms_is_member(othersj->ojrelid, joins_below))
+ {
+ /* othersj commutes with sjinfo from below left */
+ below_sjinfo = true;
+ }
+ else if (othersj == sjinfo)
+ {
+ /* found our join in syntactic order */
+ Assert(bms_equal(joins_so_far, joins_below));
+ }
+ else if (bms_is_member(othersj->ojrelid, joins_above))
+ {
+ /* othersj commutes with sjinfo from above */
+ above_sjinfo = true;
+ }
+ else
+ {
+ /* othersj is not relevant, ignore */
+ continue;
+ }
+
+ /*
+ * When we are looking at joins above sjinfo, we are
+ * envisioning pushing sjinfo to above othersj, so add
+ * othersj's nulling bit before distributing the quals.
+ */
+ if (above_sjinfo)
+ quals = (List *)
+ add_nulling_relids((Node *) quals,
+ othersj->min_righthand,
+ bms_make_singleton(othersj->ojrelid));
+
+ /* Compute qualscope and ojscope for this join level */
+ this_qualscope = bms_union(qualscope, joins_so_far);
+ this_ojscope = bms_union(ojscope, joins_so_far);
+ if (above_sjinfo)
+ {
+ /* othersj is not yet in joins_so_far, but we need it */
+ this_qualscope = bms_add_member(this_qualscope,
+ othersj->ojrelid);
+ this_ojscope = bms_add_member(this_ojscope,
+ othersj->ojrelid);
+ /* sjinfo is in joins_so_far, and we don't want it */
+ this_ojscope = bms_del_member(this_ojscope,
+ sjinfo->ojrelid);
+ }
+
+ /*
+ * We generate EquivalenceClasses only from the first form of
+ * the quals, with the fewest nullingrels bits set. An EC
+ * made from this version of the quals can be useful below the
+ * outer-join nest, whereas versions with some nullingrels
+ * bits set would not be. We cannot generate ECs from more
+ * than one version, or we'll make nonsensical conclusions
+ * that Vars with nullingrels bits set are equal to their
+ * versions without. Fortunately, such ECs wouldn't be very
+ * useful anyway, because they'd equate values not observable
+ * outside the join nest. (See optimizer/README.)
+ *
+ * The first form of the quals is also the only one marked as
+ * has_clone rather than is_clone.
+ */
+ allow_equivalence = (joins_so_far == NULL);
+ has_clone = allow_equivalence;
+ is_clone = !has_clone;
+
+ distribute_quals_to_rels(root, quals,
+ false, /* XXX below_outer_join? */
+ sjinfo,
+ root->qual_security_level,
+ this_qualscope,
+ this_ojscope, nonnullable_rels,
+ allow_equivalence,
+ false, /* no more postponement */
+ has_clone,
+ is_clone,
+ NULL);
+
+ /*
+ * Adjust qual nulling bits for next level up, if needed. We
+ * don't want to put sjinfo's own bit in at all, and if we're
+ * above sjinfo then we did it already.
+ */
+ if (below_sjinfo)
+ quals = (List *)
+ add_nulling_relids((Node *) quals,
+ othersj->min_righthand,
+ bms_make_singleton(othersj->ojrelid));
+
+ /* ... and track joins processed so far */
+ joins_so_far = bms_add_member(joins_so_far, othersj->ojrelid);
+ }
+ }
+ else
+ {
+ /* No commutation possible, just process the postponed clauses */
+ distribute_quals_to_rels(root, sjinfo->oj_joinclause,
+ false, /* XXX below_outer_join? */
+ sjinfo,
+ root->qual_security_level,
+ qualscope,
+ ojscope, nonnullable_rels,
+ true, /* allow_equivalence */
+ false, /* no more postponement */
+ false, false, /* not clones */
+ NULL);
+ }
+
+ /* Clear out the list, just so we don't have multiply-linked trees */
+ sjinfo->oj_joinclause = NIL;
+
+ /* Now add sjinfo to the new join_info_list */
+ root->join_info_list = lappend(root->join_info_list, sjinfo);
+ }
+}
+
/*****************************************************************************
*
@@ -1572,6 +1956,46 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
*
*****************************************************************************/
+/*
+ * distribute_quals_to_rels
+ * Convenience routine to apply distribute_qual_to_rels to each element
+ * of an AND'ed list of clauses.
+ */
+static void
+distribute_quals_to_rels(PlannerInfo *root, List *clauses,
+ bool below_outer_join,
+ SpecialJoinInfo *sjinfo,
+ Index security_level,
+ Relids qualscope,
+ Relids ojscope,
+ Relids outerjoin_nonnullable,
+ bool allow_equivalence,
+ bool postpone_nondegenerate_clauses,
+ bool has_clone,
+ bool is_clone,
+ List **postponed_qual_list)
+{
+ ListCell *lc;
+
+ foreach(lc, clauses)
+ {
+ Node *clause = (Node *) lfirst(lc);
+
+ distribute_qual_to_rels(root, clause,
+ below_outer_join,
+ sjinfo,
+ security_level,
+ qualscope,
+ ojscope,
+ outerjoin_nonnullable,
+ allow_equivalence,
+ postpone_nondegenerate_clauses,
+ has_clone,
+ is_clone,
+ postponed_qual_list);
+ }
+}
+
/*
* distribute_qual_to_rels
* Add clause information to either the baserestrictinfo or joininfo list
@@ -1586,15 +2010,21 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
* 'clause': the qual clause to be distributed
* 'below_outer_join': true if the qual is from a JOIN/ON that is below the
* nullable side of a higher-level outer join
- * 'jointype': type of join the qual is from (JOIN_INNER for a WHERE clause)
+ * 'sjinfo': join's SpecialJoinInfo (NULL for an inner join or WHERE clause)
* 'security_level': security_level to assign to the qual
- * 'qualscope': set of baserels the qual's syntactic scope covers
- * 'ojscope': NULL if not an outer-join qual, else the minimum set of baserels
- * needed to form this join
+ * 'qualscope': set of base+OJ rels the qual's syntactic scope covers
+ * 'ojscope': NULL if not an outer-join qual, else the minimum set of base+OJ
+ * rels needed to form this join
* 'outerjoin_nonnullable': NULL if not an outer-join qual, else the set of
- * baserels appearing on the outer (nonnullable) side of the join
+ * base+OJ rels appearing on the outer (nonnullable) side of the join
* (for FULL JOIN this includes both sides of the join, and must in fact
* equal qualscope)
+ * 'allow_equivalence': true if it's okay to convert clause into an
+ * EquivalenceClass
+ * 'postpone_nondegenerate_clauses': true if non-degenerate outer join clauses
+ * should be added to sjinfo->oj_joinclause instead of being processed
+ * 'has_clone': has_clone property to assign to the qual
+ * 'is_clone': is_clone property to assign to the qual
* 'postponed_qual_list': list of PostponedQual structs, which we can add
* this qual to if it turns out to belong to a higher join level.
* Can be NULL if caller knows postponement is impossible.
@@ -1604,16 +2034,21 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
* level, which will be ojscope not necessarily qualscope.
*
* At the time this is called, root->join_info_list must contain entries for
- * all and only those special joins that are syntactically below this qual.
+ * all and only those special joins that are syntactically below this qual;
+ * in particular, the passed-in SpecialJoinInfo isn't yet in that list.
*/
static void
distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool below_outer_join,
- JoinType jointype,
+ SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
Relids ojscope,
Relids outerjoin_nonnullable,
+ bool allow_equivalence,
+ bool postpone_nondegenerate_clauses,
+ bool has_clone,
+ bool is_clone,
List **postponed_qual_list)
{
Relids relids;
@@ -1646,7 +2081,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
PostponedQual *pq = (PostponedQual *) palloc(sizeof(PostponedQual));
Assert(root->hasLateralRTEs); /* shouldn't happen otherwise */
- Assert(jointype == JOIN_INNER); /* mustn't postpone past outer join */
+ Assert(sjinfo == NULL); /* mustn't postpone past outer join */
pq->qual = clause;
pq->relids = relids;
*postponed_qual_list = lappend(*postponed_qual_list, pq);
@@ -1708,7 +2143,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
{
relids =
get_relids_in_jointree((Node *) root->parse->jointree,
- false);
+ true, false);
qualscope = bms_copy(relids);
}
}
@@ -1751,8 +2186,18 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
{
/*
* The qual is attached to an outer join and mentions (some of the)
- * rels on the nonnullable side, so it's not degenerate.
- *
+ * rels on the nonnullable side, so it's not degenerate. If the
+ * caller wants to postpone handling such clauses, just add it to
+ * sjinfo->oj_joinclause and return. (The work we've done up to here
+ * will have to be redone later, but there's not much of it.)
+ */
+ if (postpone_nondegenerate_clauses)
+ {
+ sjinfo->oj_joinclause = lappend(sjinfo->oj_joinclause, clause);
+ return;
+ }
+
+ /*
* We can't use such a clause to deduce equivalence (the left and
* right sides might be unequal above the join because one of them has
* gone to NULL) ... but we might be able to use it for more limited
@@ -1818,6 +2263,11 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
if (check_redundant_nullability_qual(root, clause))
return;
}
+ else if (!allow_equivalence)
+ {
+ /* Caller says it mustn't become an equivalence class */
+ maybe_equivalence = false;
+ }
else
{
/*
@@ -1852,6 +2302,10 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
outerjoin_nonnullable,
nullable_relids);
+ /* Apply appropriate clone marking, too */
+ restrictinfo->has_clone = has_clone;
+ restrictinfo->is_clone = is_clone;
+
/*
* If it's a join clause (either naturally, or because delayed by
* outer-join rules), add vars used in the clause to targetlists of their
@@ -1950,11 +2404,15 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
restrictinfo);
return;
}
- if (jointype == JOIN_FULL)
+ if (sjinfo && sjinfo->jointype == JOIN_FULL)
{
/* FULL JOIN (above tests cannot match in this case) */
+ FullJoinClauseInfo *fjinfo = makeNode(FullJoinClauseInfo);
+
+ fjinfo->rinfo = restrictinfo;
+ fjinfo->sjinfo = sjinfo;
root->full_join_clauses = lappend(root->full_join_clauses,
- restrictinfo);
+ fjinfo);
return;
}
/* nope, so fall through to distribute_restrictinfo_to_rels */
@@ -2348,7 +2806,7 @@ process_implied_equality(PlannerInfo *root,
{
relids =
get_relids_in_jointree((Node *) root->parse->jointree,
- false);
+ true, false);
}
}
}
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 63deed27c9..69e725d159 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -158,13 +158,15 @@ query_planner(PlannerInfo *root,
/*
* Construct RelOptInfo nodes for all base relations used in the query.
- * Appendrel member relations ("other rels") will be added later.
+ * Appendrel member relations ("other rels") will be added later. We also
+ * construct a bitmapset of all the baserel relids.
*
* Note: the reason we find the baserels by searching the jointree, rather
* than scanning the rangetable, is that the rangetable may contain RTEs
* for rels not actively part of the query, for example views. We don't
* want to make RelOptInfos for them.
*/
+ root->all_baserels = NULL;
add_base_rels_to_query(root, (Node *) parse->jointree);
/*
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 493a3af0fa..e743a5d9fe 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2223,7 +2223,7 @@ preprocess_rowmarks(PlannerInfo *root)
* make a bitmapset of all base rels and then remove the items we don't
* need or have FOR [KEY] UPDATE/SHARE marks for.
*/
- rels = get_relids_in_jointree((Node *) parse->jointree, false);
+ rels = get_relids_in_jointree((Node *) parse->jointree, false, false);
if (parse->resultRelation)
rels = bms_del_member(rels, parse->resultRelation);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 1cb0abdbc1..8fff731756 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -29,11 +29,21 @@
#include "utils/syscache.h"
+typedef enum
+{
+ NRM_EQUAL, /* expect exact match of nullingrels */
+ NRM_SUBSET, /* actual Var may have a subset of input */
+ NRM_SUPERSET /* actual Var may have a superset of input */
+} NullingRelsMatch;
+
typedef struct
{
int varno; /* RT index of Var */
AttrNumber varattno; /* attr number of Var */
AttrNumber resno; /* TLE position of Var */
+#ifdef USE_ASSERT_CHECKING
+ Bitmapset *varnullingrels; /* Var's varnullingrels */
+#endif
} tlist_vinfo;
typedef struct
@@ -59,6 +69,7 @@ typedef struct
indexed_tlist *inner_itlist;
Index acceptable_rel;
int rtoffset;
+ NullingRelsMatch nrm_match;
double num_exec;
} fix_join_expr_context;
@@ -68,6 +79,7 @@ typedef struct
indexed_tlist *subplan_itlist;
int newvarno;
int rtoffset;
+ NullingRelsMatch nrm_match;
double num_exec;
} fix_upper_expr_context;
@@ -150,7 +162,12 @@ static indexed_tlist *build_tlist_index(List *tlist);
static Var *search_indexed_tlist_for_var(Var *var,
indexed_tlist *itlist,
int newvarno,
- int rtoffset);
+ int rtoffset,
+ NullingRelsMatch nrm_match);
+static Var *search_indexed_tlist_for_phv(PlaceHolderVar *phv,
+ indexed_tlist *itlist,
+ int newvarno,
+ NullingRelsMatch nrm_match);
static Var *search_indexed_tlist_for_non_var(Expr *node,
indexed_tlist *itlist,
int newvarno);
@@ -163,14 +180,18 @@ static List *fix_join_expr(PlannerInfo *root,
indexed_tlist *outer_itlist,
indexed_tlist *inner_itlist,
Index acceptable_rel,
- int rtoffset, double num_exec);
+ int rtoffset,
+ NullingRelsMatch nrm_match,
+ double num_exec);
static Node *fix_join_expr_mutator(Node *node,
fix_join_expr_context *context);
static Node *fix_upper_expr(PlannerInfo *root,
Node *node,
indexed_tlist *subplan_itlist,
int newvarno,
- int rtoffset, double num_exec);
+ int rtoffset,
+ NullingRelsMatch nrm_match,
+ double num_exec);
static Node *fix_upper_expr_mutator(Node *node,
fix_upper_expr_context *context);
static List *set_returning_clause_references(PlannerInfo *root,
@@ -1045,13 +1066,13 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
fix_join_expr(root, splan->onConflictSet,
NULL, itlist,
linitial_int(splan->resultRelations),
- rtoffset, NUM_EXEC_QUAL(plan));
+ rtoffset, NRM_EQUAL, NUM_EXEC_QUAL(plan));
splan->onConflictWhere = (Node *)
fix_join_expr(root, (List *) splan->onConflictWhere,
NULL, itlist,
linitial_int(splan->resultRelations),
- rtoffset, NUM_EXEC_QUAL(plan));
+ rtoffset, NRM_EQUAL, NUM_EXEC_QUAL(plan));
pfree(itlist);
@@ -1108,6 +1129,7 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
NULL, itlist,
resultrel,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST(plan));
/* Fix quals too. */
@@ -1116,6 +1138,7 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
NULL, itlist,
resultrel,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL(plan));
}
}
@@ -1261,6 +1284,7 @@ set_indexonlyscan_references(PlannerInfo *root,
index_itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST((Plan *) plan));
plan->scan.plan.qual = (List *)
fix_upper_expr(root,
@@ -1268,6 +1292,7 @@ set_indexonlyscan_references(PlannerInfo *root,
index_itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) plan));
plan->recheckqual = (List *)
fix_upper_expr(root,
@@ -1275,6 +1300,7 @@ set_indexonlyscan_references(PlannerInfo *root,
index_itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) plan));
/* indexqual is already transformed to reference index columns */
plan->indexqual = fix_scan_list(root, plan->indexqual,
@@ -1481,6 +1507,7 @@ set_foreignscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST((Plan *) fscan));
fscan->scan.plan.qual = (List *)
fix_upper_expr(root,
@@ -1488,6 +1515,7 @@ set_foreignscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) fscan));
fscan->fdw_exprs = (List *)
fix_upper_expr(root,
@@ -1495,6 +1523,7 @@ set_foreignscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) fscan));
fscan->fdw_recheck_quals = (List *)
fix_upper_expr(root,
@@ -1502,6 +1531,7 @@ set_foreignscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) fscan));
pfree(itlist);
/* fdw_scan_tlist itself just needs fix_scan_list() adjustments */
@@ -1562,6 +1592,7 @@ set_customscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST((Plan *) cscan));
cscan->scan.plan.qual = (List *)
fix_upper_expr(root,
@@ -1569,6 +1600,7 @@ set_customscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) cscan));
cscan->custom_exprs = (List *)
fix_upper_expr(root,
@@ -1576,6 +1608,7 @@ set_customscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) cscan));
pfree(itlist);
/* custom_scan_tlist itself just needs fix_scan_list() adjustments */
@@ -1780,6 +1813,7 @@ set_hash_references(PlannerInfo *root, Plan *plan, int rtoffset)
outer_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL(plan));
/* Hash doesn't project */
@@ -2115,6 +2149,7 @@ fix_scan_expr_mutator(Node *node, fix_scan_expr_context *context)
/* At scan level, we should always just evaluate the contained expr */
PlaceHolderVar *phv = (PlaceHolderVar *) node;
+ Assert(phv->phnullingrels == NULL);
return fix_scan_expr_mutator((Node *) phv->phexpr, context);
}
if (IsA(node, AlternativeSubPlan))
@@ -2172,6 +2207,7 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
inner_itlist,
(Index) 0,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) join));
/* Now do join-type-specific stuff */
@@ -2184,11 +2220,21 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
{
NestLoopParam *nlp = (NestLoopParam *) lfirst(lc);
+ /*
+ * Because we don't reparameterize parameterized paths to match
+ * the outer-join level at which they are used, Vars seen in the
+ * NestLoopParam expression may have nullingrels that are just a
+ * subset of those in the Vars actually available from the outer
+ * side. Not checking this exactly is a bit grotty, but the work
+ * needed to make things match up perfectly seems well out of
+ * proportion to the value.
+ */
nlp->paramval = (Var *) fix_upper_expr(root,
(Node *) nlp->paramval,
outer_itlist,
OUTER_VAR,
rtoffset,
+ NRM_SUBSET,
NUM_EXEC_TLIST(outer_plan));
/* Check we replaced any PlaceHolderVar with simple Var */
if (!(IsA(nlp->paramval, Var) &&
@@ -2206,6 +2252,7 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
inner_itlist,
(Index) 0,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) join));
}
else if (IsA(join, HashJoin))
@@ -2218,6 +2265,7 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
inner_itlist,
(Index) 0,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) join));
/*
@@ -2229,45 +2277,27 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
outer_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) join));
}
/*
* Now we need to fix up the targetlist and qpqual, which are logically
- * above the join. This means they should not re-use any input expression
- * that was computed in the nullable side of an outer join. Vars and
- * PlaceHolderVars are fine, so we can implement this restriction just by
- * clearing has_non_vars in the indexed_tlist structs.
- *
- * XXX This is a grotty workaround for the fact that we don't clearly
- * distinguish between a Var appearing below an outer join and the "same"
- * Var appearing above it. If we did, we'd not need to hack the matching
- * rules this way.
+ * above the join. This means that, if it's not an inner join, any Vars
+ * and PHVs appearing here should have nullingrels that include the
+ * effects of the outer join, ie they will have nullingrels equal to the
+ * input Vars' nullingrels plus the bit added by the outer join. We don't
+ * currently have enough info available here to identify what that should
+ * be, so we just tell fix_join_expr to accept superset nullingrels
+ * matches instead of exact ones.
*/
- switch (join->jointype)
- {
- case JOIN_LEFT:
- case JOIN_SEMI:
- case JOIN_ANTI:
- inner_itlist->has_non_vars = false;
- break;
- case JOIN_RIGHT:
- outer_itlist->has_non_vars = false;
- break;
- case JOIN_FULL:
- outer_itlist->has_non_vars = false;
- inner_itlist->has_non_vars = false;
- break;
- default:
- break;
- }
-
join->plan.targetlist = fix_join_expr(root,
join->plan.targetlist,
outer_itlist,
inner_itlist,
(Index) 0,
rtoffset,
+ (join->jointype == JOIN_INNER ? NRM_EQUAL : NRM_SUPERSET),
NUM_EXEC_TLIST((Plan *) join));
join->plan.qual = fix_join_expr(root,
join->plan.qual,
@@ -2275,6 +2305,7 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
inner_itlist,
(Index) 0,
rtoffset,
+ (join->jointype == JOIN_INNER ? NRM_EQUAL : NRM_SUPERSET),
NUM_EXEC_QUAL((Plan *) join));
pfree(outer_itlist);
@@ -2329,6 +2360,7 @@ set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset)
subplan_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST(plan));
}
else
@@ -2337,6 +2369,7 @@ set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset)
subplan_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST(plan));
tle = flatCopyTargetEntry(tle);
tle->expr = (Expr *) newexpr;
@@ -2350,6 +2383,7 @@ set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset)
subplan_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL(plan));
pfree(subplan_itlist);
@@ -2550,7 +2584,7 @@ set_dummy_tlist_references(Plan *plan, int rtoffset)
* tlist_member() searches.
*
* The result of this function is an indexed_tlist struct to pass to
- * search_indexed_tlist_for_var() or search_indexed_tlist_for_non_var().
+ * search_indexed_tlist_for_var() and siblings.
* When done, the indexed_tlist may be freed with a single pfree().
*/
static indexed_tlist *
@@ -2582,6 +2616,9 @@ build_tlist_index(List *tlist)
vinfo->varno = var->varno;
vinfo->varattno = var->varattno;
vinfo->resno = tle->resno;
+#ifdef USE_ASSERT_CHECKING
+ vinfo->varnullingrels = var->varnullingrels;
+#endif
vinfo++;
}
else if (tle->expr && IsA(tle->expr, PlaceHolderVar))
@@ -2634,6 +2671,9 @@ build_tlist_index_other_vars(List *tlist, int ignore_rel)
vinfo->varno = var->varno;
vinfo->varattno = var->varattno;
vinfo->resno = tle->resno;
+#ifdef USE_ASSERT_CHECKING
+ vinfo->varnullingrels = var->varnullingrels;
+#endif
vinfo++;
}
}
@@ -2653,10 +2693,17 @@ build_tlist_index_other_vars(List *tlist, int ignore_rel)
* modified varno/varattno (to wit, newvarno and the resno of the TLE entry).
* Also ensure that varnosyn is incremented by rtoffset.
* If no match, return NULL.
+ *
+ * In debugging builds, we cross-check the varnullingrels of the subplan
+ * output Var based on nrm_match. Most call sites should pass NRM_EQUAL
+ * indicating we expect an exact match. However, there are places where
+ * we haven't cleaned things up completely, and we have to settle for
+ * allowing subset or superset matches.
*/
static Var *
search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
- int newvarno, int rtoffset)
+ int newvarno, int rtoffset,
+ NullingRelsMatch nrm_match)
{
int varno = var->varno;
AttrNumber varattno = var->varattno;
@@ -2672,6 +2719,36 @@ search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
/* Found a match */
Var *newvar = copyVar(var);
+ /*
+ * Assert that we kept all the nullingrels machinations straight.
+ *
+ * XXX eventually reduce this to a plain Assert. Right now it's
+ * more useful to warn and keep going.
+ *
+ * XXX skip this check for system columns and whole-row Vars.
+ * That's because such Vars might be row identity Vars, which are
+ * generated without any varnullingrels. It'd be hard to do
+ * otherwise, since they're normally made very early in planning,
+ * when we haven't looked at the jointree yet and don't know which
+ * joins might null such Vars. Doesn't seem worth the expense to
+ * make them fully valid. (While it's slightly annoying that we
+ * thereby lose checking for user-written references to such
+ * columns, it seems unlikely that a bug in nullingrels logic
+ * would affect only system columns.)
+ */
+#ifdef USE_ASSERT_CHECKING
+ if (!(varattno <= 0 ||
+ (nrm_match == NRM_SUBSET ?
+ bms_is_subset(var->varnullingrels, vinfo->varnullingrels) :
+ nrm_match == NRM_SUPERSET ?
+ bms_is_subset(vinfo->varnullingrels, var->varnullingrels) :
+ bms_equal(vinfo->varnullingrels, var->varnullingrels))))
+ elog(WARNING, "bogus varnullingrels for (%d,%d): expected %s, found %s in subplan",
+ varno, varattno,
+ bmsToString(var->varnullingrels),
+ bmsToString(vinfo->varnullingrels));
+#endif
+
newvar->varno = newvarno;
newvar->varattno = vinfo->resno;
if (newvar->varnosyn > 0)
@@ -2684,15 +2761,74 @@ search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
}
/*
- * search_indexed_tlist_for_non_var --- find a non-Var in an indexed tlist
+ * search_indexed_tlist_for_phv --- find a PlaceHolderVar in an indexed tlist
*
* If a match is found, return a Var constructed to reference the tlist item.
* If no match, return NULL.
*
- * NOTE: it is a waste of time to call this unless itlist->has_ph_vars or
- * itlist->has_non_vars. Furthermore, set_join_references() relies on being
- * able to prevent matching of non-Vars by clearing itlist->has_non_vars,
- * so there's a correctness reason not to call it unless that's set.
+ * Cross-check phnullingrels as in search_indexed_tlist_for_var.
+ *
+ * NOTE: it is a waste of time to call this unless itlist->has_ph_vars.
+ */
+static Var *
+search_indexed_tlist_for_phv(PlaceHolderVar *phv,
+ indexed_tlist *itlist, int newvarno,
+ NullingRelsMatch nrm_match)
+{
+ ListCell *lc;
+
+ foreach(lc, itlist->tlist)
+ {
+ TargetEntry *tle = (TargetEntry *) lfirst(lc);
+
+ if (tle->expr && IsA(tle->expr, PlaceHolderVar))
+ {
+ PlaceHolderVar *subphv = (PlaceHolderVar *) tle->expr;
+ Var *newvar;
+
+ /*
+ * Analogously to search_indexed_tlist_for_var, we match on phid
+ * only. We don't use equal(), partially for speed but mostly
+ * because phnullingrels might not be exactly equal.
+ */
+ if (phv->phid != subphv->phid)
+ continue;
+
+ /*
+ * Assert that we kept all the nullingrels machinations straight.
+ *
+ * XXX eventually reduce this to a plain Assert. Right now it's
+ * more useful to warn and keep going.
+ */
+#ifdef USE_ASSERT_CHECKING
+ if (!(nrm_match == NRM_SUBSET ?
+ bms_is_subset(phv->phnullingrels, subphv->phnullingrels) :
+ nrm_match == NRM_SUPERSET ?
+ bms_is_subset(subphv->phnullingrels, phv->phnullingrels) :
+ bms_equal(subphv->phnullingrels, phv->phnullingrels)))
+ elog(WARNING, "bogus phnullingrels for %d: expected %s, found %s in subplan",
+ phv->phid,
+ bmsToString(phv->phnullingrels),
+ bmsToString(subphv->phnullingrels));
+#endif
+
+ /* Found a matching subplan output expression */
+ newvar = makeVarFromTargetEntry(newvarno, tle);
+ newvar->varnosyn = 0; /* wasn't ever a plain Var */
+ newvar->varattnosyn = 0;
+ return newvar;
+ }
+ }
+ return NULL; /* no match */
+}
+
+/*
+ * search_indexed_tlist_for_non_var --- find a non-Var/PHV in an indexed tlist
+ *
+ * If a match is found, return a Var constructed to reference the tlist item.
+ * If no match, return NULL.
+ *
+ * NOTE: it is a waste of time to call this unless itlist->has_non_vars.
*/
static Var *
search_indexed_tlist_for_non_var(Expr *node,
@@ -2799,6 +2935,7 @@ search_indexed_tlist_for_sortgroupref(Expr *node,
* 'acceptable_rel' is either zero or the rangetable index of a relation
* whose Vars may appear in the clause without provoking an error
* 'rtoffset': how much to increment varnos by
+ * 'nrm_match': as for search_indexed_tlist_for_var()
* 'num_exec': estimated number of executions of expression
*
* Returns the new expression tree. The original clause structure is
@@ -2811,6 +2948,7 @@ fix_join_expr(PlannerInfo *root,
indexed_tlist *inner_itlist,
Index acceptable_rel,
int rtoffset,
+ NullingRelsMatch nrm_match,
double num_exec)
{
fix_join_expr_context context;
@@ -2820,6 +2958,7 @@ fix_join_expr(PlannerInfo *root,
context.inner_itlist = inner_itlist;
context.acceptable_rel = acceptable_rel;
context.rtoffset = rtoffset;
+ context.nrm_match = nrm_match;
context.num_exec = num_exec;
return (List *) fix_join_expr_mutator((Node *) clauses, &context);
}
@@ -2841,7 +2980,8 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
newvar = search_indexed_tlist_for_var(var,
context->outer_itlist,
OUTER_VAR,
- context->rtoffset);
+ context->rtoffset,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
@@ -2852,7 +2992,8 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
newvar = search_indexed_tlist_for_var(var,
context->inner_itlist,
INNER_VAR,
- context->rtoffset);
+ context->rtoffset,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
@@ -2877,22 +3018,25 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* See if the PlaceHolderVar has bubbled up from a lower plan node */
if (context->outer_itlist && context->outer_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->outer_itlist,
- OUTER_VAR);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->outer_itlist,
+ OUTER_VAR,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
if (context->inner_itlist && context->inner_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->inner_itlist,
- INNER_VAR);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->inner_itlist,
+ INNER_VAR,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
/* If not supplied by input plans, evaluate the contained expr */
+ /* XXX can we assert something about phnullingrels? */
return fix_join_expr_mutator((Node *) phv->phexpr, context);
}
/* Try matching more complex expressions too, if tlists have any */
@@ -2951,6 +3095,7 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
* 'subplan_itlist': indexed target list for subplan (or index)
* 'newvarno': varno to use for Vars referencing tlist elements
* 'rtoffset': how much to increment varnos by
+ * 'nrm_match': as for search_indexed_tlist_for_var()
* 'num_exec': estimated number of executions of expression
*
* The resulting tree is a copy of the original in which all Var nodes have
@@ -2963,6 +3108,7 @@ fix_upper_expr(PlannerInfo *root,
indexed_tlist *subplan_itlist,
int newvarno,
int rtoffset,
+ NullingRelsMatch nrm_match,
double num_exec)
{
fix_upper_expr_context context;
@@ -2971,6 +3117,7 @@ fix_upper_expr(PlannerInfo *root,
context.subplan_itlist = subplan_itlist;
context.newvarno = newvarno;
context.rtoffset = rtoffset;
+ context.nrm_match = nrm_match;
context.num_exec = num_exec;
return fix_upper_expr_mutator(node, &context);
}
@@ -2989,7 +3136,8 @@ fix_upper_expr_mutator(Node *node, fix_upper_expr_context *context)
newvar = search_indexed_tlist_for_var(var,
context->subplan_itlist,
context->newvarno,
- context->rtoffset);
+ context->rtoffset,
+ context->nrm_match);
if (!newvar)
elog(ERROR, "variable not found in subplan target list");
return (Node *) newvar;
@@ -3001,13 +3149,15 @@ fix_upper_expr_mutator(Node *node, fix_upper_expr_context *context)
/* See if the PlaceHolderVar has bubbled up from a lower plan node */
if (context->subplan_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->subplan_itlist,
- context->newvarno);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->subplan_itlist,
+ context->newvarno,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
/* If not supplied by input plan, evaluate the contained expr */
+ /* XXX can we assert something about phnullingrels? */
return fix_upper_expr_mutator((Node *) phv->phexpr, context);
}
/* Try matching more complex expressions too, if tlist has any */
@@ -3114,6 +3264,7 @@ set_returning_clause_references(PlannerInfo *root,
NULL,
resultRelation,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST(topplan));
pfree(itlist);
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index f4cdb879c2..68fb712472 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -50,17 +50,28 @@ typedef struct pullup_replace_vars_context
* pullup (set only if target_rte->lateral) */
bool *outer_hasSubLinks; /* -> outer query's hasSubLinks */
int varno; /* varno of subquery */
- bool need_phvs; /* do we need PlaceHolderVars? */
- bool wrap_non_vars; /* do we need 'em on *all* non-Vars? */
+ bool wrap_non_vars; /* do we need all non-Var outputs to be PHVs? */
Node **rv_cache; /* cache for results with PHVs */
} pullup_replace_vars_context;
-typedef struct reduce_outer_joins_state
+typedef struct reduce_outer_joins_pass1_state
{
Relids relids; /* base relids within this subtree */
bool contains_outer; /* does subtree contain outer join(s)? */
List *sub_states; /* List of states for subtree components */
-} reduce_outer_joins_state;
+} reduce_outer_joins_pass1_state;
+
+typedef struct reduce_outer_joins_pass2_state
+{
+ Relids inner_reduced; /* OJ relids reduced to plain inner joins */
+ List *partial_reduced; /* List of partially reduced FULL joins */
+} reduce_outer_joins_pass2_state;
+
+typedef struct reduce_outer_joins_partial_state
+{
+ int full_join_rti; /* RT index of a formerly-FULL join */
+ Relids unreduced_side; /* relids in its still-nullable side */
+} reduce_outer_joins_partial_state;
static Node *pull_up_sublinks_jointree_recurse(PlannerInfo *root, Node *jtnode,
Relids *relids);
@@ -69,12 +80,10 @@ static Node *pull_up_sublinks_qual_recurse(PlannerInfo *root, Node *node,
Node **jtlink2, Relids available_rels2);
static Node *pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static Node *pull_up_simple_subquery(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static Node *pull_up_simple_union_all(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte);
@@ -91,7 +100,6 @@ static Node *pull_up_simple_values(PlannerInfo *root, Node *jtnode,
static bool is_simple_values(PlannerInfo *root, RangeTblEntry *rte);
static Node *pull_up_constant_function(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static bool is_simple_union_all(Query *subquery);
static bool is_simple_union_all_recurse(Node *setOp, Query *setOpQuery,
@@ -102,24 +110,26 @@ static bool jointree_contains_lateral_outer_refs(PlannerInfo *root,
Relids safe_upper_varnos);
static void perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars_context *rvcontext,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static void replace_vars_in_jointree(Node *jtnode,
- pullup_replace_vars_context *context,
- JoinExpr *lowest_nulling_outer_join);
+ pullup_replace_vars_context *context);
static Node *pullup_replace_vars(Node *expr,
pullup_replace_vars_context *context);
static Node *pullup_replace_vars_callback(Var *var,
replace_rte_variables_context *context);
static Query *pullup_replace_vars_subquery(Query *query,
pullup_replace_vars_context *context);
-static reduce_outer_joins_state *reduce_outer_joins_pass1(Node *jtnode);
+static reduce_outer_joins_pass1_state *reduce_outer_joins_pass1(Node *jtnode);
static void reduce_outer_joins_pass2(Node *jtnode,
- reduce_outer_joins_state *state,
+ reduce_outer_joins_pass1_state *state1,
+ reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
List *forced_null_vars);
-static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode);
+static void report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
+ int rtindex, Relids relids);
+static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Relids *dropped_outer_joins);
static int get_result_relid(PlannerInfo *root, Node *jtnode);
static void remove_result_refs(PlannerInfo *root, int varno, Node *newjtloc);
static bool find_dependent_phvs(PlannerInfo *root, int varno);
@@ -764,7 +774,7 @@ pull_up_subqueries(PlannerInfo *root)
/* Recursion starts with no containing join nor appendrel */
root->parse->jointree = (FromExpr *)
pull_up_subqueries_recurse(root, (Node *) root->parse->jointree,
- NULL, NULL, NULL);
+ NULL, NULL);
/* We should still have a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
}
@@ -779,12 +789,6 @@ pull_up_subqueries(PlannerInfo *root)
* lowest_outer_join references the lowest such JoinExpr node; otherwise
* it is NULL. We use this to constrain the effects of LATERAL subqueries.
*
- * If this jointree node is within the nullable side of an outer join, then
- * lowest_nulling_outer_join references the lowest such JoinExpr node;
- * otherwise it is NULL. This forces use of the PlaceHolderVar mechanism for
- * references to non-nullable targetlist items, but only for references above
- * that join.
- *
* If we are looking at a member subquery of an append relation,
* containing_appendrel describes that relation; else it is NULL.
* This forces use of the PlaceHolderVar mechanism for all non-Var targetlist
@@ -801,15 +805,14 @@ pull_up_subqueries(PlannerInfo *root)
* Notice also that we can't turn pullup_replace_vars loose on the whole
* jointree, because it'd return a mutated copy of the tree; we have to
* invoke it just on the quals, instead. This behavior is what makes it
- * reasonable to pass lowest_outer_join and lowest_nulling_outer_join as
- * pointers rather than some more-indirect way of identifying the lowest
- * OJs. Likewise, we don't replace append_rel_list members but only their
- * substructure, so the containing_appendrel reference is safe to use.
+ * reasonable to pass lowest_outer_join as a pointer rather than some
+ * more-indirect way of identifying the lowest OJ. Likewise, we don't
+ * replace append_rel_list members but only their substructure, so the
+ * containing_appendrel reference is safe to use.
*/
static Node *
pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Assert(jtnode != NULL);
@@ -831,7 +834,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
is_safe_append_member(rte->subquery)))
return pull_up_simple_subquery(root, jtnode, rte,
lowest_outer_join,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -864,7 +866,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
*/
if (rte->rtekind == RTE_FUNCTION)
return pull_up_constant_function(root, jtnode, rte,
- lowest_nulling_outer_join,
containing_appendrel);
/* Otherwise, do nothing at this node. */
@@ -880,7 +881,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
{
lfirst(l) = pull_up_subqueries_recurse(root, lfirst(l),
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
}
}
@@ -895,11 +895,9 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
case JOIN_INNER:
j->larg = pull_up_subqueries_recurse(root, j->larg,
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
break;
case JOIN_LEFT:
@@ -907,31 +905,25 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
case JOIN_ANTI:
j->larg = pull_up_subqueries_recurse(root, j->larg,
j,
- lowest_nulling_outer_join,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
- j,
j,
NULL);
break;
case JOIN_FULL:
j->larg = pull_up_subqueries_recurse(root, j->larg,
- j,
j,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
- j,
j,
NULL);
break;
case JOIN_RIGHT:
j->larg = pull_up_subqueries_recurse(root, j->larg,
- j,
j,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
j,
- lowest_nulling_outer_join,
NULL);
break;
default:
@@ -961,7 +953,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
static Node *
pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -1108,31 +1099,25 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* The subquery's targetlist items are now in the appropriate form to
* insert into the top query, except that we may need to wrap them in
* PlaceHolderVars. Set up required context data for pullup_replace_vars.
+ * (Note that we should include the subquery's inner joins in relids,
+ * since it may include join alias vars referencing them.)
*/
rvcontext.root = root;
rvcontext.targetlist = subquery->targetList;
rvcontext.target_rte = rte;
if (rte->lateral)
rvcontext.relids = get_relids_in_jointree((Node *) subquery->jointree,
- true);
+ true, true);
else /* won't need relids */
rvcontext.relids = NULL;
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = varno;
- /* these flags will be set below, if needed */
- rvcontext.need_phvs = false;
+ /* this flag will be set below, if needed */
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(subquery->targetList) + 1) *
sizeof(Node *));
- /*
- * If we are under an outer join then non-nullable items and lateral
- * references may have to be turned into PlaceHolderVars.
- */
- if (lowest_nulling_outer_join != NULL)
- rvcontext.need_phvs = true;
-
/*
* If we are dealing with an appendrel member then anything that's not a
* simple Var has to be turned into a PlaceHolderVar. We force this to
@@ -1141,10 +1126,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* expression actually available from the appendrel.
*/
if (containing_appendrel != NULL)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* If the parent query uses grouping sets, we need a PlaceHolderVar for
@@ -1156,10 +1138,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* that pullup_replace_vars hasn't currently got.)
*/
if (parse->groupingSets)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* Replace all of the top query's references to the subquery's outputs
@@ -1167,7 +1146,6 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* replace any of the jointree structure.
*/
perform_pullup_replace_vars(root, &rvcontext,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -1234,7 +1212,8 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
{
Relids subrelids;
- subrelids = get_relids_in_jointree((Node *) subquery->jointree, false);
+ subrelids = get_relids_in_jointree((Node *) subquery->jointree,
+ true, false);
substitute_phv_relids((Node *) parse, varno, subrelids);
fix_append_rel_relids(root->append_rel_list, varno, subrelids);
}
@@ -1425,7 +1404,7 @@ pull_up_union_leaf_queries(Node *setOp, PlannerInfo *root, int parentRTindex,
rtr = makeNode(RangeTblRef);
rtr->rtindex = childRTindex;
(void) pull_up_subqueries_recurse(root, (Node *) rtr,
- NULL, NULL, appinfo);
+ NULL, appinfo);
}
else if (IsA(setOp, SetOperationStmt))
{
@@ -1562,7 +1541,7 @@ is_simple_subquery(PlannerInfo *root, Query *subquery, RangeTblEntry *rte,
{
restricted = true;
safe_upper_varnos = get_relids_in_jointree((Node *) lowest_outer_join,
- true);
+ true, true);
}
else
{
@@ -1674,7 +1653,6 @@ pull_up_simple_values(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte)
rvcontext.relids = NULL;
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = varno;
- rvcontext.need_phvs = false;
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(tlist) + 1) *
@@ -1686,7 +1664,7 @@ pull_up_simple_values(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte)
* any of the jointree structure. We can assume there's no outer joins or
* appendrels in the dummy Query that surrounds a VALUES RTE.
*/
- perform_pullup_replace_vars(root, &rvcontext, NULL, NULL);
+ perform_pullup_replace_vars(root, &rvcontext, NULL);
/*
* There should be no appendrels to fix, nor any outer joins and hence no
@@ -1785,7 +1763,6 @@ is_simple_values(PlannerInfo *root, RangeTblEntry *rte)
static Node *
pull_up_constant_function(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -1837,40 +1814,26 @@ pull_up_constant_function(PlannerInfo *root, Node *jtnode,
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = ((RangeTblRef *) jtnode)->rtindex;
- /* these flags will be set below, if needed */
- rvcontext.need_phvs = false;
+ /* this flag will be set below, if needed */
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(rvcontext.targetlist) + 1) *
sizeof(Node *));
- /*
- * If we are under an outer join then non-nullable items and lateral
- * references may have to be turned into PlaceHolderVars.
- */
- if (lowest_nulling_outer_join != NULL)
- rvcontext.need_phvs = true;
-
/*
* If we are dealing with an appendrel member then anything that's not a
* simple Var has to be turned into a PlaceHolderVar. (See comments in
* pull_up_simple_subquery().)
*/
if (containing_appendrel != NULL)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* If the parent query uses grouping sets, we need a PlaceHolderVar for
* anything that's not a simple Var.
*/
if (parse->groupingSets)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* Replace all of the top query's references to the RTE's output with
@@ -1878,7 +1841,6 @@ pull_up_constant_function(PlannerInfo *root, Node *jtnode,
* jointree structure.
*/
perform_pullup_replace_vars(root, &rvcontext,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -2100,13 +2062,11 @@ jointree_contains_lateral_outer_refs(PlannerInfo *root, Node *jtnode,
*
* Caller has already filled *rvcontext with data describing what to
* substitute for Vars referencing the target subquery. In addition
- * we need the identity of the lowest outer join that can null the
- * target subquery, and its containing appendrel if any.
+ * we need the identity of the containing appendrel if any.
*/
static void
perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars_context *rvcontext,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -2150,38 +2110,31 @@ perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars((Node *) action->targetList, rvcontext);
}
}
- replace_vars_in_jointree((Node *) parse->jointree, rvcontext,
- lowest_nulling_outer_join);
+ replace_vars_in_jointree((Node *) parse->jointree, rvcontext);
Assert(parse->setOperations == NULL);
parse->havingQual = pullup_replace_vars(parse->havingQual, rvcontext);
/*
* Replace references in the translated_vars lists of appendrels. When
- * pulling up an appendrel member, we do not need PHVs in the list of the
- * parent appendrel --- there isn't any outer join between. Elsewhere,
- * use PHVs for safety. (This analysis could be made tighter but it seems
- * unlikely to be worth much trouble.)
+ * pulling up an appendrel member, we do not want to force PHVs in the
+ * list of the parent appendrel --- there isn't any outer join between.
+ * Elsewhere, use PHVs for safety. (This analysis could be made tighter
+ * but it seems unlikely to be worth much trouble.)
*/
foreach(lc, root->append_rel_list)
{
AppendRelInfo *appinfo = (AppendRelInfo *) lfirst(lc);
- bool save_need_phvs = rvcontext->need_phvs;
+ bool save_wrap_non_vars = rvcontext->wrap_non_vars;
if (appinfo == containing_appendrel)
- rvcontext->need_phvs = false;
+ rvcontext->wrap_non_vars = false;
appinfo->translated_vars = (List *)
pullup_replace_vars((Node *) appinfo->translated_vars, rvcontext);
- rvcontext->need_phvs = save_need_phvs;
+ rvcontext->wrap_non_vars = save_wrap_non_vars;
}
/*
* Replace references in the joinaliasvars lists of join RTEs.
- *
- * You might think that we could avoid using PHVs for alias vars of joins
- * below lowest_nulling_outer_join, but that doesn't work because the
- * alias vars could be referenced above that join; we need the PHVs to be
- * present in such references after the alias vars get flattened. (It
- * might be worth trying to be smarter here, someday.)
*/
foreach(lc, parse->rtable)
{
@@ -2198,14 +2151,10 @@ perform_pullup_replace_vars(PlannerInfo *root,
* Helper routine for perform_pullup_replace_vars: do pullup_replace_vars on
* every expression in the jointree, without changing the jointree structure
* itself. Ugly, but there's no other way...
- *
- * If we are at or below lowest_nulling_outer_join, we can suppress use of
- * PlaceHolderVars wrapped around the replacement expressions.
*/
static void
replace_vars_in_jointree(Node *jtnode,
- pullup_replace_vars_context *context,
- JoinExpr *lowest_nulling_outer_join)
+ pullup_replace_vars_context *context)
{
if (jtnode == NULL)
return;
@@ -2215,10 +2164,8 @@ replace_vars_in_jointree(Node *jtnode,
* If the RangeTblRef refers to a LATERAL subquery (that isn't the
* same subquery we're pulling up), it might contain references to the
* target subquery, which we must replace. We drive this from the
- * jointree scan, rather than a scan of the rtable, for a couple of
- * reasons: we can avoid processing no-longer-referenced RTEs, and we
- * can use the appropriate setting of need_phvs depending on whether
- * the RTE is above possibly-nulling outer joins or not.
+ * jointree scan, rather than a scan of the rtable, so that we can
+ * avoid processing no-longer-referenced RTEs.
*/
int varno = ((RangeTblRef *) jtnode)->rtindex;
@@ -2275,42 +2222,30 @@ replace_vars_in_jointree(Node *jtnode,
ListCell *l;
foreach(l, f->fromlist)
- replace_vars_in_jointree(lfirst(l), context,
- lowest_nulling_outer_join);
+ replace_vars_in_jointree(lfirst(l), context);
f->quals = pullup_replace_vars(f->quals, context);
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- bool save_need_phvs = context->need_phvs;
+ bool save_wrap_non_vars = context->wrap_non_vars;
- if (j == lowest_nulling_outer_join)
- {
- /* no more PHVs in or below this join */
- context->need_phvs = false;
- lowest_nulling_outer_join = NULL;
- }
- replace_vars_in_jointree(j->larg, context, lowest_nulling_outer_join);
- replace_vars_in_jointree(j->rarg, context, lowest_nulling_outer_join);
+ replace_vars_in_jointree(j->larg, context);
+ replace_vars_in_jointree(j->rarg, context);
/*
- * Use PHVs within the join quals of a full join, even when it's the
- * lowest nulling outer join. Otherwise, we cannot identify which
- * side of the join a pulled-up var-free expression came from, which
- * can lead to failure to make a plan at all because none of the quals
- * appear to be mergeable or hashable conditions. For this purpose we
- * don't care about the state of wrap_non_vars, so leave it alone.
+ * Use PHVs within the join quals of a full join. Otherwise, we
+ * cannot identify which side of the join a pulled-up var-free
+ * expression came from, which can lead to failure to make a plan at
+ * all because none of the quals appear to be mergeable or hashable
+ * conditions.
*/
if (j->jointype == JOIN_FULL)
- context->need_phvs = true;
+ context->wrap_non_vars = true;
j->quals = pullup_replace_vars(j->quals, context);
- /*
- * We don't bother to update the colvars list, since it won't be used
- * again ...
- */
- context->need_phvs = save_need_phvs;
+ context->wrap_non_vars = save_wrap_non_vars;
}
else
elog(ERROR, "unrecognized node type: %d",
@@ -2339,8 +2274,18 @@ pullup_replace_vars_callback(Var *var,
{
pullup_replace_vars_context *rcon = (pullup_replace_vars_context *) context->callback_arg;
int varattno = var->varattno;
+ bool need_phv;
Node *newnode;
+ /*
+ * We need a PlaceHolderVar if the Var-to-be-replaced has nonempty
+ * varnullingrels (unless we find below that the replacement expression is
+ * a Var or PlaceHolderVar that we can just add the nullingrels to). We
+ * also need one if the caller has instructed us that all non-Var/PHV
+ * replacements need to be wrapped for identification purposes.
+ */
+ need_phv = (var->varnullingrels != NULL) || rcon->wrap_non_vars;
+
/*
* If PlaceHolderVars are needed, we cache the modified expressions in
* rcon->rv_cache[]. This is not in hopes of any material speed gain
@@ -2349,13 +2294,16 @@ pullup_replace_vars_callback(Var *var,
* and possibly prevent optimizations that rely on recognizing different
* references to the same subquery output as being equal(). So it's worth
* a bit of extra effort to avoid it.
+ *
+ * The cached items have phlevelsup = 0 and phnullingrels = NULL; we'll
+ * copy them and adjust those values for this reference site below.
*/
- if (rcon->need_phvs &&
+ if (need_phv &&
varattno >= InvalidAttrNumber &&
varattno <= list_length(rcon->targetlist) &&
rcon->rv_cache[varattno] != NULL)
{
- /* Just copy the entry and fall through to adjust its varlevelsup */
+ /* Just copy the entry and fall through to adjust phlevelsup etc */
newnode = copyObject(rcon->rv_cache[varattno]);
}
else if (varattno == InvalidAttrNumber)
@@ -2364,7 +2312,7 @@ pullup_replace_vars_callback(Var *var,
RowExpr *rowexpr;
List *colnames;
List *fields;
- bool save_need_phvs = rcon->need_phvs;
+ bool save_wrap_non_vars = rcon->wrap_non_vars;
int save_sublevelsup = context->sublevels_up;
/*
@@ -2375,18 +2323,18 @@ pullup_replace_vars_callback(Var *var,
* the RowExpr for use of the executor and ruleutils.c.
*
* In order to be able to cache the results, we always generate the
- * expansion with varlevelsup = 0, and then adjust if needed.
+ * expansion with varlevelsup = 0, and then adjust below if needed.
*/
expandRTE(rcon->target_rte,
var->varno, 0 /* not varlevelsup */ , var->location,
(var->vartype != RECORDOID),
&colnames, &fields);
- /* Adjust the generated per-field Vars, but don't insert PHVs */
- rcon->need_phvs = false;
+ /* Expand the generated per-field Vars, but don't insert PHVs there */
+ rcon->wrap_non_vars = false;
context->sublevels_up = 0; /* to match the expandRTE output */
fields = (List *) replace_rte_variables_mutator((Node *) fields,
context);
- rcon->need_phvs = save_need_phvs;
+ rcon->wrap_non_vars = save_wrap_non_vars;
context->sublevels_up = save_sublevelsup;
rowexpr = makeNode(RowExpr);
@@ -2404,14 +2352,13 @@ pullup_replace_vars_callback(Var *var,
* expression to yield NULL, not ROW(NULL,NULL,...) when it is forced
* to null by an outer join.
*/
- if (rcon->need_phvs)
+ if (need_phv)
{
- /* RowExpr is certainly not strict, so always need PHV */
newnode = (Node *)
make_placeholder_expr(rcon->root,
(Expr *) newnode,
bms_make_singleton(rcon->varno));
- /* cache it with the PHV, and with varlevelsup still zero */
+ /* cache it with the PHV, and with phlevelsup etc not set yet */
rcon->rv_cache[InvalidAttrNumber] = copyObject(newnode);
}
}
@@ -2428,7 +2375,7 @@ pullup_replace_vars_callback(Var *var,
newnode = (Node *) copyObject(tle->expr);
/* Insert PlaceHolderVar if needed */
- if (rcon->need_phvs)
+ if (need_phv)
{
bool wrap;
@@ -2454,69 +2401,61 @@ pullup_replace_vars_callback(Var *var,
/* No need to wrap a PlaceHolderVar with another one, either */
wrap = false;
}
- else if (rcon->wrap_non_vars)
- {
- /* Wrap all non-Vars in a PlaceHolderVar */
- wrap = true;
- }
else
{
/*
- * If it contains a Var of the subquery being pulled up, and
- * does not contain any non-strict constructs, then it's
- * certainly nullable so we don't need to insert a
- * PlaceHolderVar.
- *
- * This analysis could be tighter: in particular, a non-strict
- * construct hidden within a lower-level PlaceHolderVar is not
- * reason to add another PHV. But for now it doesn't seem
- * worth the code to be more exact.
- *
- * Note: in future maybe we should insert a PlaceHolderVar
- * anyway, if the tlist item is expensive to evaluate?
- *
- * For a LATERAL subquery, we have to check the actual var
- * membership of the node, but if it's non-lateral then any
- * level-zero var must belong to the subquery.
+ * Must wrap, either because we need a place to insert
+ * varnullingrels or because caller told us to wrap
+ * everything.
*/
- if ((rcon->target_rte->lateral ?
- bms_overlap(pull_varnos(rcon->root, (Node *) newnode),
- rcon->relids) :
- contain_vars_of_level((Node *) newnode, 0)) &&
- !contain_nonstrict_functions((Node *) newnode))
- {
- /* No wrap needed */
- wrap = false;
- }
- else
- {
- /* Else wrap it in a PlaceHolderVar */
- wrap = true;
- }
+ wrap = true;
}
if (wrap)
+ {
newnode = (Node *)
make_placeholder_expr(rcon->root,
(Expr *) newnode,
bms_make_singleton(rcon->varno));
- /*
- * Cache it if possible (ie, if the attno is in range, which it
- * probably always should be). We can cache the value even if we
- * decided we didn't need a PHV, since this result will be
- * suitable for any request that has need_phvs.
- */
- if (varattno > InvalidAttrNumber &&
- varattno <= list_length(rcon->targetlist))
- rcon->rv_cache[varattno] = copyObject(newnode);
+ /*
+ * Cache it if possible (ie, if the attno is in range, which
+ * it probably always should be).
+ */
+ if (varattno > InvalidAttrNumber &&
+ varattno <= list_length(rcon->targetlist))
+ rcon->rv_cache[varattno] = copyObject(newnode);
+ }
}
}
- /* Must adjust varlevelsup if tlist item is from higher query */
+ /* Must adjust varlevelsup if replaced Var is within a subquery */
if (var->varlevelsup > 0)
IncrementVarSublevelsUp(newnode, var->varlevelsup, 0);
+ /* Propagate any varnullingrels into the replacement Var or PHV */
+ if (var->varnullingrels != NULL)
+ {
+ if (IsA(newnode, Var))
+ {
+ Var *newvar = (Var *) newnode;
+
+ Assert(newvar->varlevelsup == var->varlevelsup);
+ newvar->varnullingrels = bms_add_members(newvar->varnullingrels,
+ var->varnullingrels);
+ }
+ else if (IsA(newnode, PlaceHolderVar))
+ {
+ PlaceHolderVar *newphv = (PlaceHolderVar *) newnode;
+
+ Assert(newphv->phlevelsup == var->varlevelsup);
+ newphv->phnullingrels = bms_add_members(newphv->phnullingrels,
+ var->varnullingrels);
+ }
+ else
+ elog(ERROR, "failed to wrap a non-Var");
+ }
+
return newnode;
}
@@ -2675,7 +2614,9 @@ flatten_simple_union_all(PlannerInfo *root)
void
reduce_outer_joins(PlannerInfo *root)
{
- reduce_outer_joins_state *state;
+ reduce_outer_joins_pass1_state *state1;
+ reduce_outer_joins_pass2_state state2;
+ ListCell *lc;
/*
* To avoid doing strictness checks on more quals than necessary, we want
@@ -2686,14 +2627,56 @@ reduce_outer_joins(PlannerInfo *root)
* join(s) below each side of each join clause. The second pass examines
* qual clauses and changes join types as it descends the tree.
*/
- state = reduce_outer_joins_pass1((Node *) root->parse->jointree);
+ state1 = reduce_outer_joins_pass1((Node *) root->parse->jointree);
/* planner.c shouldn't have called me if no outer joins */
- if (state == NULL || !state->contains_outer)
+ if (state1 == NULL || !state1->contains_outer)
elog(ERROR, "so where are the outer joins?");
+ state2.inner_reduced = NULL;
+ state2.partial_reduced = NIL;
+
reduce_outer_joins_pass2((Node *) root->parse->jointree,
- state, root, NULL, NIL);
+ state1, &state2,
+ root, NULL, NIL);
+
+ /*
+ * If we successfully reduced the strength of any outer joins, we must
+ * remove references to those joins as nulling rels. This is handled as
+ * an additional pass, for simplicity and because we can handle all
+ * fully-reduced joins in a single pass over the parse tree.
+ */
+ if (!bms_is_empty(state2.inner_reduced))
+ {
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ state2.inner_reduced,
+ NULL);
+ /* There could be references in the append_rel_list, too */
+ root->append_rel_list = (List *)
+ remove_nulling_relids((Node *) root->append_rel_list,
+ state2.inner_reduced,
+ NULL);
+ }
+
+ /*
+ * Partially-reduced full joins have to be done one at a time, since
+ * they'll each need a different setting of except_relids.
+ */
+ foreach(lc, state2.partial_reduced)
+ {
+ reduce_outer_joins_partial_state *statep = lfirst(lc);
+ Relids full_join_relids = bms_make_singleton(statep->full_join_rti);
+
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ full_join_relids,
+ statep->unreduced_side);
+ root->append_rel_list = (List *)
+ remove_nulling_relids((Node *) root->append_rel_list,
+ full_join_relids,
+ statep->unreduced_side);
+ }
}
/*
@@ -2701,13 +2684,13 @@ reduce_outer_joins(PlannerInfo *root)
*
* Returns a state node describing the given jointree node.
*/
-static reduce_outer_joins_state *
+static reduce_outer_joins_pass1_state *
reduce_outer_joins_pass1(Node *jtnode)
{
- reduce_outer_joins_state *result;
+ reduce_outer_joins_pass1_state *result;
- result = (reduce_outer_joins_state *)
- palloc(sizeof(reduce_outer_joins_state));
+ result = (reduce_outer_joins_pass1_state *)
+ palloc(sizeof(reduce_outer_joins_pass1_state));
result->relids = NULL;
result->contains_outer = false;
result->sub_states = NIL;
@@ -2727,7 +2710,7 @@ reduce_outer_joins_pass1(Node *jtnode)
foreach(l, f->fromlist)
{
- reduce_outer_joins_state *sub_state;
+ reduce_outer_joins_pass1_state *sub_state;
sub_state = reduce_outer_joins_pass1(lfirst(l));
result->relids = bms_add_members(result->relids,
@@ -2739,7 +2722,7 @@ reduce_outer_joins_pass1(Node *jtnode)
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- reduce_outer_joins_state *sub_state;
+ reduce_outer_joins_pass1_state *sub_state;
/* join's own RT index is not wanted in result->relids */
if (IS_OUTER_JOIN(j->jointype))
@@ -2767,14 +2750,22 @@ reduce_outer_joins_pass1(Node *jtnode)
* reduce_outer_joins_pass2 - phase 2 processing
*
* jtnode: current jointree node
- * state: state data collected by phase 1 for this node
+ * state1: state data collected by phase 1 for this node
+ * state2: where to accumulate info about successfully-reduced joins
* root: toplevel planner state
* nonnullable_rels: set of base relids forced non-null by upper quals
* forced_null_vars: multibitmapset of Vars forced null by upper quals
+ *
+ * Returns info in state2 about outer joins that were successfully simplified.
+ * Joins that were fully reduced to inner joins are all added to
+ * state2->inner_reduced. If a full join is reduced to a left join,
+ * it needs its own entry in state2->partial_reduced, since that will
+ * require custom processing to remove only the correct nullingrel markers.
*/
static void
reduce_outer_joins_pass2(Node *jtnode,
- reduce_outer_joins_state *state,
+ reduce_outer_joins_pass1_state *state1,
+ reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
List *forced_null_vars)
@@ -2803,13 +2794,14 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_forced_null_vars = mbms_add_members(pass_forced_null_vars,
forced_null_vars);
/* And recurse --- but only into interesting subtrees */
- Assert(list_length(f->fromlist) == list_length(state->sub_states));
- forboth(l, f->fromlist, s, state->sub_states)
+ Assert(list_length(f->fromlist) == list_length(state1->sub_states));
+ forboth(l, f->fromlist, s, state1->sub_states)
{
- reduce_outer_joins_state *sub_state = lfirst(s);
+ reduce_outer_joins_pass1_state *sub_state = lfirst(s);
if (sub_state->contains_outer)
- reduce_outer_joins_pass2(lfirst(l), sub_state, root,
+ reduce_outer_joins_pass2(lfirst(l), sub_state,
+ state2, root,
pass_nonnullable_rels,
pass_forced_null_vars);
}
@@ -2821,8 +2813,8 @@ reduce_outer_joins_pass2(Node *jtnode,
JoinExpr *j = (JoinExpr *) jtnode;
int rtindex = j->rtindex;
JoinType jointype = j->jointype;
- reduce_outer_joins_state *left_state = linitial(state->sub_states);
- reduce_outer_joins_state *right_state = lsecond(state->sub_states);
+ reduce_outer_joins_pass1_state *left_state = linitial(state1->sub_states);
+ reduce_outer_joins_pass1_state *right_state = lsecond(state1->sub_states);
/* Can we simplify this join? */
switch (jointype)
@@ -2843,12 +2835,22 @@ reduce_outer_joins_pass2(Node *jtnode,
if (bms_overlap(nonnullable_rels, right_state->relids))
jointype = JOIN_INNER;
else
+ {
jointype = JOIN_LEFT;
+ /* Also report partial reduction in state2 */
+ report_reduced_full_join(state2, rtindex,
+ right_state->relids);
+ }
}
else
{
if (bms_overlap(nonnullable_rels, right_state->relids))
+ {
jointype = JOIN_RIGHT;
+ /* Also report partial reduction in state2 */
+ report_reduced_full_join(state2, rtindex,
+ left_state->relids);
+ }
}
break;
case JOIN_SEMI:
@@ -2881,8 +2883,8 @@ reduce_outer_joins_pass2(Node *jtnode,
j->larg = j->rarg;
j->rarg = tmparg;
jointype = JOIN_LEFT;
- right_state = linitial(state->sub_states);
- left_state = lsecond(state->sub_states);
+ right_state = linitial(state1->sub_states);
+ left_state = lsecond(state1->sub_states);
}
/*
@@ -2913,7 +2915,10 @@ reduce_outer_joins_pass2(Node *jtnode,
jointype = JOIN_ANTI;
}
- /* Apply the jointype change, if any, to both jointree node and RTE */
+ /*
+ * Apply the jointype change, if any, to both jointree node and RTE.
+ * Also, if we changed an RTE to INNER, add its RTI to inner_reduced.
+ */
if (rtindex && jointype != j->jointype)
{
RangeTblEntry *rte = rt_fetch(rtindex, root->parse->rtable);
@@ -2921,6 +2926,9 @@ reduce_outer_joins_pass2(Node *jtnode,
Assert(rte->rtekind == RTE_JOIN);
Assert(rte->jointype == j->jointype);
rte->jointype = jointype;
+ if (jointype == JOIN_INNER)
+ state2->inner_reduced = bms_add_member(state2->inner_reduced,
+ rtindex);
}
j->jointype = jointype;
@@ -2993,7 +3001,8 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_nonnullable_rels = NULL;
pass_forced_null_vars = NIL;
}
- reduce_outer_joins_pass2(j->larg, left_state, root,
+ reduce_outer_joins_pass2(j->larg, left_state,
+ state2, root,
pass_nonnullable_rels,
pass_forced_null_vars);
}
@@ -3012,7 +3021,8 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_nonnullable_rels = NULL;
pass_forced_null_vars = NIL;
}
- reduce_outer_joins_pass2(j->rarg, right_state, root,
+ reduce_outer_joins_pass2(j->rarg, right_state,
+ state2, root,
pass_nonnullable_rels,
pass_forced_null_vars);
}
@@ -3024,6 +3034,19 @@ reduce_outer_joins_pass2(Node *jtnode,
(int) nodeTag(jtnode));
}
+/* Helper for reduce_outer_joins_pass2 */
+static void
+report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
+ int rtindex, Relids relids)
+{
+ reduce_outer_joins_partial_state *statep;
+
+ statep = palloc(sizeof(reduce_outer_joins_partial_state));
+ statep->full_join_rti = rtindex;
+ statep->unreduced_side = relids;
+ state2->partial_reduced = lappend(state2->partial_reduced, statep);
+}
+
/*
* remove_useless_result_rtes
@@ -3065,16 +3088,41 @@ reduce_outer_joins_pass2(Node *jtnode,
void
remove_useless_result_rtes(PlannerInfo *root)
{
+ Relids dropped_outer_joins = NULL;
ListCell *cell;
/* Top level of jointree must always be a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
/* Recurse ... */
root->parse->jointree = (FromExpr *)
- remove_useless_results_recurse(root, (Node *) root->parse->jointree);
+ remove_useless_results_recurse(root,
+ (Node *) root->parse->jointree,
+ &dropped_outer_joins);
/* We should still have a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
+ /*
+ * If we removed any outer-join nodes from the jointree, run around and
+ * remove references to those joins as nulling rels. (There could be such
+ * references in PHVs that we pulled up out of the original subquery that
+ * the RESULT rel replaced. This is kosher on the grounds that we now
+ * know that such an outer join wouldn't really have nulled anything.) We
+ * don't do this during the main recursion, for simplicity and because we
+ * can handle all such joins in a single pass over the parse tree.
+ */
+ if (!bms_is_empty(dropped_outer_joins))
+ {
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ dropped_outer_joins,
+ NULL);
+ /* There could be references in the append_rel_list, too */
+ root->append_rel_list = (List *)
+ remove_nulling_relids((Node *) root->append_rel_list,
+ dropped_outer_joins,
+ NULL);
+ }
+
/*
* Remove any PlanRowMark referencing an RTE_RESULT RTE. We obviously
* must do that for any RTE_RESULT that we just removed. But one for a
@@ -3100,9 +3148,12 @@ remove_useless_result_rtes(PlannerInfo *root)
* Recursive guts of remove_useless_result_rtes.
*
* This recursively processes the jointree and returns a modified jointree.
+ * In addition, the RT indexes of any removed outer-join nodes are added to
+ * *dropped_outer_joins.
*/
static Node *
-remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
+remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Relids *dropped_outer_joins)
{
Assert(jtnode != NULL);
if (IsA(jtnode, RangeTblRef))
@@ -3130,7 +3181,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
int varno;
/* Recursively transform child ... */
- child = remove_useless_results_recurse(root, child);
+ child = remove_useless_results_recurse(root, child,
+ dropped_outer_joins);
/* ... and stick it back into the tree */
lfirst(cell) = child;
@@ -3179,8 +3231,10 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
int varno;
/* First, recurse */
- j->larg = remove_useless_results_recurse(root, j->larg);
- j->rarg = remove_useless_results_recurse(root, j->rarg);
+ j->larg = remove_useless_results_recurse(root, j->larg,
+ dropped_outer_joins);
+ j->rarg = remove_useless_results_recurse(root, j->rarg,
+ dropped_outer_joins);
/* Apply join-type-specific optimization rules */
switch (j->jointype)
@@ -3248,6 +3302,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
!find_dependent_phvs(root, varno)))
{
remove_result_refs(root, varno, j->larg);
+ *dropped_outer_joins = bms_add_member(*dropped_outer_joins,
+ j->rtindex);
jtnode = j->larg;
}
break;
@@ -3258,6 +3314,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
!find_dependent_phvs(root, varno)))
{
remove_result_refs(root, varno, j->rarg);
+ *dropped_outer_joins = bms_add_member(*dropped_outer_joins,
+ j->rtindex);
jtnode = j->rarg;
}
break;
@@ -3272,11 +3330,14 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
* Unlike the LEFT/RIGHT cases, we just Assert that there are
* no PHVs that need to be evaluated at the semijoin's RHS,
* since the rest of the query couldn't reference any outputs
- * of the semijoin's RHS.
+ * of the semijoin's RHS. Also, we don't need to worry about
+ * removing traces of the join's rtindex, since it hasn't got
+ * one.
*/
if ((varno = get_result_relid(root, j->rarg)) != 0)
{
Assert(!find_dependent_phvs(root, varno));
+ Assert(j->rtindex == 0);
remove_result_refs(root, varno, j->larg);
if (j->quals)
jtnode = (Node *)
@@ -3345,7 +3406,7 @@ remove_result_refs(PlannerInfo *root, int varno, Node *newjtloc)
{
Relids subrelids;
- subrelids = get_relids_in_jointree(newjtloc, false);
+ subrelids = get_relids_in_jointree(newjtloc, true, false);
Assert(!bms_is_empty(subrelids));
substitute_phv_relids((Node *) root->parse, varno, subrelids);
fix_append_rel_relids(root->append_rel_list, varno, subrelids);
@@ -3402,9 +3463,8 @@ find_dependent_phvs_walker(Node *node,
context->sublevels_up--;
return result;
}
- /* Shouldn't need to handle planner auxiliary nodes here */
+ /* Shouldn't need to handle most planner auxiliary nodes here */
Assert(!IsA(node, SpecialJoinInfo));
- Assert(!IsA(node, AppendRelInfo));
Assert(!IsA(node, PlaceHolderInfo));
Assert(!IsA(node, MinMaxAggInfo));
@@ -3424,10 +3484,17 @@ find_dependent_phvs(PlannerInfo *root, int varno)
context.relids = bms_make_singleton(varno);
context.sublevels_up = 0;
- return query_tree_walker(root->parse,
- find_dependent_phvs_walker,
- (void *) &context,
- 0);
+ if (query_tree_walker(root->parse,
+ find_dependent_phvs_walker,
+ (void *) &context,
+ 0))
+ return true;
+ /* The append_rel_list could be populated already, so check it too */
+ if (expression_tree_walker((Node *) root->append_rel_list,
+ find_dependent_phvs_walker,
+ (void *) &context))
+ return true;
+ return false;
}
static bool
@@ -3457,7 +3524,7 @@ find_dependent_phvs_in_jointree(PlannerInfo *root, Node *node, int varno)
* are not marked LATERAL, though, since they couldn't possibly contain
* any cross-references to other RTEs.
*/
- subrelids = get_relids_in_jointree(node, false);
+ subrelids = get_relids_in_jointree(node, false, false);
relid = -1;
while ((relid = bms_next_member(subrelids, relid)) >= 0)
{
@@ -3601,11 +3668,17 @@ fix_append_rel_relids(List *append_rel_list, int varno, Relids subrelids)
/*
* get_relids_in_jointree: get set of RT indexes present in a jointree
*
- * If include_joins is true, join RT indexes are included; if false,
- * only base rels are included.
+ * Base-relation relids are always included in the result.
+ * If include_outer_joins is true, outer-join RT indexes are included.
+ * If include_inner_joins is true, inner-join RT indexes are included.
+ *
+ * Note that for most purposes in the planner, outer joins are included
+ * in standard relid sets. Setting include_inner_joins true is only
+ * appropriate for special purposes during subquery flattening.
*/
Relids
-get_relids_in_jointree(Node *jtnode, bool include_joins)
+get_relids_in_jointree(Node *jtnode, bool include_outer_joins,
+ bool include_inner_joins)
{
Relids result = NULL;
@@ -3626,18 +3699,34 @@ get_relids_in_jointree(Node *jtnode, bool include_joins)
{
result = bms_join(result,
get_relids_in_jointree(lfirst(l),
- include_joins));
+ include_outer_joins,
+ include_inner_joins));
}
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- result = get_relids_in_jointree(j->larg, include_joins);
+ result = get_relids_in_jointree(j->larg,
+ include_outer_joins,
+ include_inner_joins);
result = bms_join(result,
- get_relids_in_jointree(j->rarg, include_joins));
- if (include_joins && j->rtindex)
- result = bms_add_member(result, j->rtindex);
+ get_relids_in_jointree(j->rarg,
+ include_outer_joins,
+ include_inner_joins));
+ if (j->rtindex)
+ {
+ if (j->jointype == JOIN_INNER)
+ {
+ if (include_inner_joins)
+ result = bms_add_member(result, j->rtindex);
+ }
+ else
+ {
+ if (include_outer_joins)
+ result = bms_add_member(result, j->rtindex);
+ }
+ }
}
else
elog(ERROR, "unrecognized node type: %d",
@@ -3646,7 +3735,7 @@ get_relids_in_jointree(Node *jtnode, bool include_joins)
}
/*
- * get_relids_for_join: get set of base RT indexes making up a join
+ * get_relids_for_join: get set of base+OJ RT indexes making up a join
*/
Relids
get_relids_for_join(Query *query, int joinrelid)
@@ -3657,7 +3746,7 @@ get_relids_for_join(Query *query, int joinrelid)
joinrelid);
if (!jtnode)
elog(ERROR, "could not find join node %d", joinrelid);
- return get_relids_in_jointree(jtnode, false);
+ return get_relids_in_jointree(jtnode, true, false);
}
/*
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index f6fc62aa5d..11c6bbaba6 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -228,6 +228,14 @@ adjust_appendrel_attrs_mutator(Node *node,
if (var->varlevelsup != 0)
return (Node *) var; /* no changes needed */
+ /*
+ * You might think we need to adjust var->varnullingrels, but that
+ * shouldn't need any changes. It will contain outer-join relids,
+ * while the transformation we are making affects only baserels.
+ * Below, we just propagate var->varnullingrels into the translated
+ * Var. (XXX what to do if translation is not a Var??)
+ */
+
for (cnt = 0; cnt < nappinfos; cnt++)
{
if (var->varno == appinfos[cnt]->parent_relid)
@@ -255,6 +263,8 @@ adjust_appendrel_attrs_mutator(Node *node,
if (newnode == NULL)
elog(ERROR, "attribute %d of relation \"%s\" does not exist",
var->varattno, get_rel_name(appinfo->parent_reloid));
+ if (IsA(newnode, Var))
+ ((Var *) newnode)->varnullingrels = var->varnullingrels;
return newnode;
}
else if (var->varattno == 0)
@@ -348,6 +358,8 @@ adjust_appendrel_attrs_mutator(Node *node,
var = copyObject(ridinfo->rowidvar);
/* ... but use the correct relid */
var->varno = leaf_relid;
+ /* identity vars shouldn't have nulling rels */
+ Assert(var->varnullingrels == NULL);
/* varnosyn in the RowIdentityVarInfo is probably wrong */
var->varnosyn = 0;
var->varattnosyn = 0;
@@ -392,8 +404,11 @@ adjust_appendrel_attrs_mutator(Node *node,
(void *) context);
/* now fix PlaceHolderVar's relid sets */
if (phv->phlevelsup == 0)
- phv->phrels = adjust_child_relids(phv->phrels, context->nappinfos,
- context->appinfos);
+ {
+ phv->phrels = adjust_child_relids(phv->phrels,
+ nappinfos, appinfos);
+ /* as above, we needn't touch phnullingrels */
+ }
return (Node *) phv;
}
/* Shouldn't need to handle planner auxiliary nodes here */
@@ -688,7 +703,11 @@ get_translated_update_targetlist(PlannerInfo *root, Index relid,
/*
* find_appinfos_by_relids
- * Find AppendRelInfo structures for all relations specified by relids.
+ * Find AppendRelInfo structures for base relations listed in relids.
+ *
+ * The relids argument is typically a join relation's relids, which can
+ * include outer-join RT indexes in addition to baserels. We silently
+ * ignore the outer joins.
*
* The AppendRelInfos are returned in an array, which can be pfree'd by the
* caller. *nappinfos is set to the number of entries in the array.
@@ -700,8 +719,9 @@ find_appinfos_by_relids(PlannerInfo *root, Relids relids, int *nappinfos)
int cnt = 0;
int i;
- *nappinfos = bms_num_members(relids);
- appinfos = (AppendRelInfo **) palloc(sizeof(AppendRelInfo *) * *nappinfos);
+ /* Allocate an array that's certainly big enough */
+ appinfos = (AppendRelInfo **)
+ palloc(sizeof(AppendRelInfo *) * bms_num_members(relids));
i = -1;
while ((i = bms_next_member(relids, i)) >= 0)
@@ -709,10 +729,17 @@ find_appinfos_by_relids(PlannerInfo *root, Relids relids, int *nappinfos)
AppendRelInfo *appinfo = root->append_rel_array[i];
if (!appinfo)
+ {
+ /* Probably i is an OJ index, but let's check */
+ if (find_base_rel_ignore_join(root, i) == NULL)
+ continue;
+ /* It's a base rel, but we lack an append_rel_array entry */
elog(ERROR, "child rel %d not found in append_rel_array", i);
+ }
appinfos[cnt++] = appinfo;
}
+ *nappinfos = cnt;
return appinfos;
}
@@ -754,6 +781,7 @@ add_row_identity_var(PlannerInfo *root, Var *orig_var,
Assert(IsA(orig_var, Var));
Assert(orig_var->varno == rtindex);
Assert(orig_var->varlevelsup == 0);
+ Assert(orig_var->varnullingrels == NULL);
/*
* If we're doing non-inherited UPDATE/DELETE/MERGE, there's little need
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 33790a4f46..0fcd26a958 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -2012,14 +2012,16 @@ is_pseudo_constant_clause_relids(Node *clause, Relids relids)
* NumRelids
* (formerly clause_relids)
*
- * Returns the number of different relations referenced in 'clause'.
+ * Returns the number of different base relations referenced in 'clause'.
*/
int
NumRelids(PlannerInfo *root, Node *clause)
{
+ int result;
Relids varnos = pull_varnos(root, clause);
- int result = bms_num_members(varnos);
+ varnos = bms_del_members(varnos, root->outer_join_rels);
+ result = bms_num_members(varnos);
bms_free(varnos);
return result;
}
diff --git a/src/backend/optimizer/util/joininfo.c b/src/backend/optimizer/util/joininfo.c
index d4cffdb198..afd243f5d8 100644
--- a/src/backend/optimizer/util/joininfo.c
+++ b/src/backend/optimizer/util/joininfo.c
@@ -88,8 +88,8 @@ have_relevant_joinclause(PlannerInfo *root,
* not depend on context).
*
* 'restrictinfo' describes the join clause
- * 'join_relids' is the list of relations participating in the join clause
- * (there must be more than one)
+ * 'join_relids' is the set of relations participating in the join clause
+ * (some of these could be outer joins)
*/
void
add_join_clause_to_rels(PlannerInfo *root,
@@ -101,8 +101,11 @@ add_join_clause_to_rels(PlannerInfo *root,
cur_relid = -1;
while ((cur_relid = bms_next_member(join_relids, cur_relid)) >= 0)
{
- RelOptInfo *rel = find_base_rel(root, cur_relid);
+ RelOptInfo *rel = find_base_rel_ignore_join(root, cur_relid);
+ /* We only need to add the clause to baserels */
+ if (rel == NULL)
+ continue;
rel->joininfo = lappend(rel->joininfo, restrictinfo);
}
}
@@ -115,8 +118,8 @@ add_join_clause_to_rels(PlannerInfo *root,
* discover that a relation need not be joined at all.
*
* 'restrictinfo' describes the join clause
- * 'join_relids' is the list of relations participating in the join clause
- * (there must be more than one)
+ * 'join_relids' is the set of relations participating in the join clause
+ * (some of these could be outer joins)
*/
void
remove_join_clause_from_rels(PlannerInfo *root,
@@ -128,7 +131,11 @@ remove_join_clause_from_rels(PlannerInfo *root,
cur_relid = -1;
while ((cur_relid = bms_next_member(join_relids, cur_relid)) >= 0)
{
- RelOptInfo *rel = find_base_rel(root, cur_relid);
+ RelOptInfo *rel = find_base_rel_ignore_join(root, cur_relid);
+
+ /* We would only have added the clause to baserels */
+ if (rel == NULL)
+ continue;
/*
* Remove the restrictinfo from the list. Pointer comparison is
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index b1363df065..9cfde2f790 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -338,6 +338,11 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
sjinfo.syn_lefthand = sjinfo.min_lefthand;
sjinfo.syn_righthand = sjinfo.min_righthand;
sjinfo.jointype = JOIN_INNER;
+ sjinfo.ojrelid = 0;
+ sjinfo.commute_above_l = NULL;
+ sjinfo.commute_above_r = NULL;
+ sjinfo.commute_below = NULL;
+ sjinfo.oj_joinclause = NIL;
/* we don't bother trying to make the remaining fields valid */
sjinfo.lhs_strict = false;
sjinfo.delay_upper_joins = false;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 6dd11329fb..bf35d1989c 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -1307,7 +1307,7 @@ create_append_path(PlannerInfo *root,
* Apply query-wide LIMIT if known and path is for sole base relation.
* (Handling this at this low level is a bit klugy.)
*/
- if (root != NULL && bms_equal(rel->relids, root->all_baserels))
+ if (root != NULL && bms_equal(rel->relids, root->all_query_rels))
pathnode->limit_tuples = root->limit_tuples;
else
pathnode->limit_tuples = -1.0;
@@ -1436,7 +1436,7 @@ create_merge_append_path(PlannerInfo *root,
* Apply query-wide LIMIT if known and path is for sole base relation.
* (Handling this at this low level is a bit klugy.)
*/
- if (bms_equal(rel->relids, root->all_baserels))
+ if (bms_equal(rel->relids, root->all_query_rels))
pathnode->limit_tuples = root->limit_tuples;
else
pathnode->limit_tuples = -1.0;
diff --git a/src/backend/optimizer/util/placeholder.c b/src/backend/optimizer/util/placeholder.c
index c55027377f..b9cc983df7 100644
--- a/src/backend/optimizer/util/placeholder.c
+++ b/src/backend/optimizer/util/placeholder.c
@@ -23,17 +23,32 @@
#include "optimizer/planmain.h"
#include "utils/lsyscache.h"
+
+typedef struct contain_placeholder_references_context
+{
+ int relid;
+ int sublevels_up;
+} contain_placeholder_references_context;
+
/* Local functions */
static void find_placeholders_recurse(PlannerInfo *root, Node *jtnode);
static void find_placeholders_in_expr(PlannerInfo *root, Node *expr);
+static bool contain_placeholder_references_walker(Node *node,
+ contain_placeholder_references_context *context);
/*
* make_placeholder_expr
* Make a PlaceHolderVar for the given expression.
*
- * phrels is the syntactic location (as a set of baserels) to attribute
+ * phrels is the syntactic location (as a set of relids) to attribute
* to the expression.
+ *
+ * The caller is responsible for adjusting phlevelsup and phnullingrels
+ * as needed. Because we do not know here which query level the PHV
+ * will be associated with, it's important that this function touches
+ * only root->glob; messing with other parts of PlannerInfo would be
+ * likely to do the wrong thing.
*/
PlaceHolderVar *
make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
@@ -42,8 +57,9 @@ make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
phv->phexpr = expr;
phv->phrels = phrels;
+ phv->phnullingrels = NULL; /* caller may change this later */
phv->phid = ++(root->glob->lastPHId);
- phv->phlevelsup = 0;
+ phv->phlevelsup = 0; /* caller may change this later */
return phv;
}
@@ -92,6 +108,15 @@ find_placeholder_info(PlannerInfo *root, PlaceHolderVar *phv)
phinfo->phid = phv->phid;
phinfo->ph_var = copyObject(phv);
+ /*
+ * By convention, phinfo->ph_var->phnullingrels is always empty, since the
+ * PlaceHolderInfo represents the initially-calculated state of the
+ * PlaceHolderVar. PlaceHolderVars appearing in the query tree might have
+ * varying values of phnullingrels, reflecting outer joins applied above
+ * the calculation level.
+ */
+ phinfo->ph_var->phnullingrels = NULL;
+
/*
* Any referenced rels that are outside the PHV's syntactic scope are
* LATERAL references, which should be included in ph_lateral but not in
@@ -339,6 +364,8 @@ update_placeholder_eval_levels(PlannerInfo *root, SpecialJoinInfo *new_sjinfo)
sjinfo->min_lefthand);
eval_at = bms_add_members(eval_at,
sjinfo->min_righthand);
+ if (sjinfo->ojrelid)
+ eval_at = bms_add_member(eval_at, sjinfo->ojrelid);
/* we'll need another iteration */
found_some = true;
}
@@ -413,6 +440,14 @@ add_placeholders_to_base_rels(PlannerInfo *root)
{
RelOptInfo *rel = find_base_rel(root, varno);
+ /*
+ * As in add_vars_to_targetlist(), a value computed at scan level
+ * has not yet been nulled by any outer join, so its phnullingrels
+ * should be empty.
+ */
+ Assert(phinfo->ph_var->phnullingrels == NULL);
+
+ /* Copying the PHV might be unnecessary here, but be safe */
rel->reltarget->exprs = lappend(rel->reltarget->exprs,
copyObject(phinfo->ph_var));
/* reltarget's cost and width fields will be updated later */
@@ -435,7 +470,8 @@ add_placeholders_to_base_rels(PlannerInfo *root)
*/
void
add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *outer_rel, RelOptInfo *inner_rel)
+ RelOptInfo *outer_rel, RelOptInfo *inner_rel,
+ SpecialJoinInfo *sjinfo)
{
Relids relids = joinrel->relids;
ListCell *lc;
@@ -466,9 +502,17 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
if (!bms_is_subset(phinfo->ph_eval_at, outer_rel->relids) &&
!bms_is_subset(phinfo->ph_eval_at, inner_rel->relids))
{
- PlaceHolderVar *phv = phinfo->ph_var;
+ /* Copying might be unnecessary here, but be safe */
+ PlaceHolderVar *phv = copyObject(phinfo->ph_var);
QualCost cost;
+ /*
+ * It'll start out not nulled by anything. Joins above
+ * this one might add to its phnullingrels later, in much
+ * the same way as for Vars.
+ */
+ Assert(phv->phnullingrels == NULL);
+
joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
phv);
cost_qual_eval_node(&cost, (Node *) phv->phexpr, root);
@@ -499,3 +543,74 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
}
}
}
+
+/*
+ * contain_placeholder_references_to
+ * Detect whether any PlaceHolderVars in the given clause contain
+ * references to the given relid (typically an OJ relid).
+ *
+ * "Contain" means that there's a use of the relid inside the PHV's
+ * contained expression, so that changing the nullability status of
+ * the rel might change what the PHV computes.
+ *
+ * The code here to cope with upper-level PHVs is likely dead, but keep it
+ * anyway just in case.
+ */
+bool
+contain_placeholder_references_to(PlannerInfo *root, Node *clause,
+ int relid)
+{
+ contain_placeholder_references_context context;
+
+ /* We can answer quickly in the common case that there's no PHVs at all */
+ if (root->glob->lastPHId == 0)
+ return false;
+ /* Else run the recursive search */
+ context.relid = relid;
+ context.sublevels_up = 0;
+ return contain_placeholder_references_walker(clause, &context);
+}
+
+static bool
+contain_placeholder_references_walker(Node *node,
+ contain_placeholder_references_context *context)
+{
+ if (node == NULL)
+ return false;
+ if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ /* We should just look through PHVs of other query levels */
+ if (phv->phlevelsup == context->sublevels_up)
+ {
+ /* If phrels matches, we found what we came for */
+ if (bms_is_member(context->relid, phv->phrels))
+ return true;
+
+ /*
+ * We should not examine phnullingrels: what we are looking for is
+ * references in the contained expression, not OJs that might null
+ * the result afterwards. Also, we don't need to recurse into the
+ * contained expression, because phrels should adequately
+ * summarize what's in there. So we're done here.
+ */
+ return false;
+ }
+ }
+ else if (IsA(node, Query))
+ {
+ /* Recurse into RTE subquery or not-yet-planned sublink subquery */
+ bool result;
+
+ context->sublevels_up++;
+ result = query_tree_walker((Query *) node,
+ contain_placeholder_references_walker,
+ context,
+ 0);
+ context->sublevels_up--;
+ return result;
+ }
+ return expression_tree_walker(node, contain_placeholder_references_walker,
+ context);
+}
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index d7b4434e7f..84e5e8db7b 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -28,6 +28,7 @@
#include "optimizer/plancat.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+#include "rewrite/rewriteManip.h"
#include "utils/hsearch.h"
#include "utils/lsyscache.h"
@@ -39,7 +40,9 @@ typedef struct JoinHashEntry
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *input_rel);
+ RelOptInfo *input_rel,
+ SpecialJoinInfo *sjinfo,
+ bool can_null);
static List *build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outer_rel,
@@ -47,8 +50,10 @@ static List *build_joinrel_restrictlist(PlannerInfo *root,
static void build_joinrel_joinlist(RelOptInfo *joinrel,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel);
-static List *subbuild_joinrel_restrictlist(RelOptInfo *joinrel,
- List *joininfo_list,
+static List *subbuild_joinrel_restrictlist(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *input_rel,
+ Relids both_input_relids,
List *new_restrictlist);
static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
List *joininfo_list,
@@ -56,10 +61,12 @@ static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
static void set_foreign_rel_properties(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel);
static void add_join_rel(PlannerInfo *root, RelOptInfo *joinrel);
-static void build_joinrel_partition_info(RelOptInfo *joinrel,
+static void build_joinrel_partition_info(PlannerInfo *root,
+ RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
- List *restrictlist, JoinType jointype);
-static bool have_partkey_equi_join(RelOptInfo *joinrel,
+ SpecialJoinInfo *sjinfo,
+ List *restrictlist);
+static bool have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *rel1, RelOptInfo *rel2,
JoinType jointype, List *restrictlist);
static int match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel,
@@ -354,7 +361,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
/*
* find_base_rel
- * Find a base or other relation entry, which must already exist.
+ * Find a base or otherrel relation entry, which must already exist.
*/
RelOptInfo *
find_base_rel(PlannerInfo *root, int relid)
@@ -375,6 +382,44 @@ find_base_rel(PlannerInfo *root, int relid)
return NULL; /* keep compiler quiet */
}
+/*
+ * find_base_rel_ignore_join
+ * Find a base or otherrel relation entry, which must already exist.
+ *
+ * Unlike find_base_rel, if relid references an outer join then this
+ * will return NULL rather than raising an error. This is convenient
+ * for callers that must deal with relid sets including both base and
+ * outer joins.
+ */
+RelOptInfo *
+find_base_rel_ignore_join(PlannerInfo *root, int relid)
+{
+ Assert(relid > 0);
+
+ if (relid < root->simple_rel_array_size)
+ {
+ RelOptInfo *rel;
+ RangeTblEntry *rte;
+
+ rel = root->simple_rel_array[relid];
+ if (rel)
+ return rel;
+
+ /*
+ * We could just return NULL here, but for debugging purposes it seems
+ * best to actually verify that the relid is an outer join and not
+ * something weird.
+ */
+ rte = root->simple_rte_array[relid];
+ if (rte && rte->rtekind == RTE_JOIN && rte->jointype != JOIN_INNER)
+ return NULL;
+ }
+
+ elog(ERROR, "no relation entry for relid %d", relid);
+
+ return NULL; /* keep compiler quiet */
+}
+
/*
* build_join_rel_hash
* Construct the auxiliary hash table for join relations.
@@ -674,9 +719,11 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ build_joinrel_tlist(root, joinrel, outer_rel, sjinfo,
+ (sjinfo->jointype == JOIN_FULL));
+ build_joinrel_tlist(root, joinrel, inner_rel, sjinfo,
+ (sjinfo->jointype != JOIN_INNER));
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel, sjinfo);
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -708,8 +755,8 @@ build_join_rel(PlannerInfo *root,
joinrel->has_eclass_joins = has_relevant_eclass_joinclause(root, joinrel);
/* Store the partition information. */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- sjinfo->jointype);
+ build_joinrel_partition_info(root, joinrel, outer_rel, inner_rel, sjinfo,
+ restrictlist);
/*
* Set estimates of the joinrel's size.
@@ -765,16 +812,14 @@ build_join_rel(PlannerInfo *root,
* 'parent_joinrel' is the RelOptInfo representing the join between parent
* relations. Some of the members of new RelOptInfo are produced by
* translating corresponding members of this RelOptInfo
- * 'sjinfo': child-join context info
* 'restrictlist': list of RestrictInfo nodes that apply to this particular
* pair of joinable relations
- * 'jointype' is the join type (inner, left, full, etc)
+ * 'sjinfo': child join's join-type details
*/
RelOptInfo *
build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
RelOptInfo *inner_rel, RelOptInfo *parent_joinrel,
- List *restrictlist, SpecialJoinInfo *sjinfo,
- JoinType jointype)
+ List *restrictlist, SpecialJoinInfo *sjinfo)
{
RelOptInfo *joinrel = makeNode(RelOptInfo);
AppendRelInfo **appinfos;
@@ -788,6 +833,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->reloptkind = RELOPT_OTHER_JOINREL;
joinrel->relids = bms_union(outer_rel->relids, inner_rel->relids);
+ if (sjinfo->ojrelid != 0)
+ joinrel->relids = bms_add_member(joinrel->relids, sjinfo->ojrelid);
joinrel->rows = 0;
/* cheap startup cost is interesting iff not all tuples to be retrieved */
joinrel->consider_startup = (root->tuple_fraction > 0);
@@ -874,8 +921,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->has_eclass_joins = parent_joinrel->has_eclass_joins;
/* Is the join between partitions itself partitioned? */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- jointype);
+ build_joinrel_partition_info(root, joinrel, outer_rel, inner_rel, sjinfo,
+ restrictlist);
/* Child joinrel is parallel safe if parent is parallel safe. */
joinrel->consider_parallel = parent_joinrel->consider_parallel;
@@ -957,10 +1004,41 @@ min_join_parameterization(PlannerInfo *root,
*
* We also compute the expected width of the join's output, making use
* of data that was cached at the baserel level by set_rel_width().
+ *
+ * Pass can_null as true if the join is an outer join that can null Vars
+ * from this input relation. If so, we will (normally) add the join's relid
+ * to the nulling bitmaps of Vars and PHVs bubbled up from the input.
+ *
+ * When forming an outer join's target list, special handling is needed
+ * in case the outer join was commuted with another one per outer join
+ * identity 3 (see optimizer/README). We must take steps to ensure that
+ * the output Vars have the same nulling bitmaps that they would if the
+ * two joins had been done in syntactic order; else they won't match Vars
+ * appearing higher in the query tree. We need to do two things:
+ *
+ * First, sjinfo->commute_above_r is added to the nulling bitmaps of RHS Vars.
+ * This takes care of the case where we implement
+ * A leftjoin (B leftjoin C on (Pbc)) on (Pab)
+ * as
+ * (A leftjoin B on (Pab)) leftjoin C on (Pbc)
+ * The C columns emitted by the B/C join need to be shown as nulled by both
+ * the B/C and A/B joins, even though they've not traversed the A/B join.
+ * (If the joins haven't been commuted, we are adding the nullingrel bits
+ * prematurely; but that's okay because the C columns can't be referenced
+ * between here and the upper join.)
+ *
+ * Second, if a RHS Var has any of the relids in sjinfo->commute_above_l
+ * already set in its nulling bitmap, then we *don't* add sjinfo->ojrelid
+ * to its nulling bitmap (but we do still add commute_above_r). This takes
+ * care of the reverse transformation: if the original syntax was
+ * (A leftjoin B on (Pab)) leftjoin C on (Pbc)
+ * then the now-upper A/B join must not mark C columns as nulled by itself.
*/
static void
build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *input_rel)
+ RelOptInfo *input_rel,
+ SpecialJoinInfo *sjinfo,
+ bool can_null)
{
Relids relids = joinrel->relids;
ListCell *vars;
@@ -980,7 +1058,24 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
/* Is it still needed above this joinrel? */
if (bms_nonempty_difference(phinfo->ph_needed, relids))
{
- /* Yup, add it to the output */
+ /*
+ * Yup, add it to the output. If this join potentially nulls
+ * this input, we have to update the PHV's phnullingrels,
+ * which means making a copy.
+ */
+ if (can_null)
+ {
+ phv = copyObject(phv);
+ /* See comments above to understand this logic */
+ if (sjinfo->ojrelid != 0 &&
+ !bms_overlap(phv->phnullingrels, sjinfo->commute_above_l))
+ phv->phnullingrels = bms_add_member(phv->phnullingrels,
+ sjinfo->ojrelid);
+ if (sjinfo->commute_above_r)
+ phv->phnullingrels = bms_add_members(phv->phnullingrels,
+ sjinfo->commute_above_r);
+ }
+
joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
phv);
/* Bubbling up the precomputed result has cost zero */
@@ -1004,9 +1099,7 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
RowIdentityVarInfo *ridinfo = (RowIdentityVarInfo *)
list_nth(root->row_identity_vars, var->varattno - 1);
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- var);
- /* Vars have cost zero, so no need to adjust reltarget->cost */
+ /* Update reltarget width estimate from RowIdentityVarInfo */
joinrel->reltarget->width += ridinfo->rowidwidth;
}
else
@@ -1019,15 +1112,35 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
/* Is it still needed above this joinrel? */
ndx = var->varattno - baserel->min_attr;
- if (bms_nonempty_difference(baserel->attr_needed[ndx], relids))
- {
- /* Yup, add it to the output */
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- var);
- /* Vars have cost zero, so no need to adjust reltarget->cost */
- joinrel->reltarget->width += baserel->attr_widths[ndx];
- }
+ if (!bms_nonempty_difference(baserel->attr_needed[ndx], relids))
+ continue; /* nope, skip it */
+
+ /* Update reltarget width estimate from baserel's attr_widths */
+ joinrel->reltarget->width += baserel->attr_widths[ndx];
+ }
+
+ /*
+ * Add the Var to the output. If this join potentially nulls this
+ * input, we have to update the Var's varnullingrels, which means
+ * making a copy.
+ */
+ if (can_null)
+ {
+ var = copyObject(var);
+ /* See comments above to understand this logic */
+ if (sjinfo->ojrelid != 0 &&
+ !bms_overlap(var->varnullingrels, sjinfo->commute_above_l))
+ var->varnullingrels = bms_add_member(var->varnullingrels,
+ sjinfo->ojrelid);
+ if (sjinfo->commute_above_r)
+ var->varnullingrels = bms_add_members(var->varnullingrels,
+ sjinfo->commute_above_r);
}
+
+ joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
+ var);
+
+ /* Vars have cost zero, so no need to adjust reltarget->cost */
}
}
@@ -1046,7 +1159,7 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
* is not handled in the sub-relations, so it depends on which
* sub-relations are considered.
*
- * If a join clause from an input relation refers to base rels still not
+ * If a join clause from an input relation refers to base+OJ rels still not
* present in the joinrel, then it is still a join clause for the joinrel;
* we put it into the joininfo list for the joinrel. Otherwise,
* the clause is now a restrict clause for the joined relation, and we
@@ -1080,14 +1193,19 @@ build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *inner_rel)
{
List *result;
+ Relids both_input_relids;
+
+ both_input_relids = bms_union(outer_rel->relids, inner_rel->relids);
/*
* Collect all the clauses that syntactically belong at this level,
* eliminating any duplicates (important since we will see many of the
* same clauses arriving from both input relations).
*/
- result = subbuild_joinrel_restrictlist(joinrel, outer_rel->joininfo, NIL);
- result = subbuild_joinrel_restrictlist(joinrel, inner_rel->joininfo, result);
+ result = subbuild_joinrel_restrictlist(root, joinrel, outer_rel,
+ both_input_relids, NIL);
+ result = subbuild_joinrel_restrictlist(root, joinrel, inner_rel,
+ both_input_relids, result);
/*
* Add on any clauses derived from EquivalenceClasses. These cannot be
@@ -1122,24 +1240,63 @@ build_joinrel_joinlist(RelOptInfo *joinrel,
}
static List *
-subbuild_joinrel_restrictlist(RelOptInfo *joinrel,
- List *joininfo_list,
+subbuild_joinrel_restrictlist(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *input_rel,
+ Relids both_input_relids,
List *new_restrictlist)
{
ListCell *l;
- foreach(l, joininfo_list)
+ foreach(l, input_rel->joininfo)
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(l);
if (bms_is_subset(rinfo->required_relids, joinrel->relids))
{
/*
- * This clause becomes a restriction clause for the joinrel, since
- * it refers to no outside rels. Add it to the list, being
- * careful to eliminate duplicates. (Since RestrictInfo nodes in
- * different joinlists will have been multiply-linked rather than
- * copied, pointer equality should be a sufficient test.)
+ * This clause should become a restriction clause for the joinrel,
+ * since it refers to no outside rels. However, if it's a clone
+ * clause then it might be too late to evaluate it, so we have to
+ * check. (If it is too late, just ignore the clause, taking it
+ * on faith that another clone was or will be selected.) Clone
+ * clauses should always be outer-join clauses, so we compare
+ * against both_input_relids.
+ */
+ if (rinfo->has_clone || rinfo->is_clone)
+ {
+ Assert(!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids));
+ if (!bms_is_subset(rinfo->required_relids, both_input_relids))
+ continue;
+ if (!clause_is_computable_at(root, rinfo->clause_relids,
+ both_input_relids))
+ continue;
+ }
+ else
+ {
+ /*
+ * For non-clone clauses, we just Assert it's OK. These might
+ * be either join or filter clauses.
+ */
+#ifdef USE_ASSERT_CHECKING
+ if (RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ Assert(clause_is_computable_at(root, rinfo->clause_relids,
+ joinrel->relids));
+ else
+ {
+ Assert(bms_is_subset(rinfo->required_relids,
+ both_input_relids));
+ Assert(clause_is_computable_at(root, rinfo->clause_relids,
+ both_input_relids));
+ }
+#endif
+ }
+
+ /*
+ * OK, so add it to the list, being careful to eliminate
+ * duplicates. (Since RestrictInfo nodes in different joinlists
+ * will have been multiply-linked rather than copied, pointer
+ * equality should be a sufficient test.)
*/
new_restrictlist = list_append_unique_ptr(new_restrictlist, rinfo);
}
@@ -1646,9 +1803,10 @@ find_param_path_info(RelOptInfo *rel, Relids required_outer)
* partitioned join relation.
*/
static void
-build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
- RelOptInfo *inner_rel, List *restrictlist,
- JoinType jointype)
+build_joinrel_partition_info(PlannerInfo *root,
+ RelOptInfo *joinrel, RelOptInfo *outer_rel,
+ RelOptInfo *inner_rel, SpecialJoinInfo *sjinfo,
+ List *restrictlist)
{
PartitionScheme part_scheme;
@@ -1674,8 +1832,8 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
!outer_rel->consider_partitionwise_join ||
!inner_rel->consider_partitionwise_join ||
outer_rel->part_scheme != inner_rel->part_scheme ||
- !have_partkey_equi_join(joinrel, outer_rel, inner_rel,
- jointype, restrictlist))
+ !have_partkey_equi_join(root, joinrel, outer_rel, inner_rel,
+ sjinfo->jointype, restrictlist))
{
Assert(!IS_PARTITIONED_REL(joinrel));
return;
@@ -1699,7 +1857,8 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* child-join relations of the join relation in try_partitionwise_join().
*/
joinrel->part_scheme = part_scheme;
- set_joinrel_partition_key_exprs(joinrel, outer_rel, inner_rel, jointype);
+ set_joinrel_partition_key_exprs(joinrel, outer_rel, inner_rel,
+ sjinfo->jointype);
/*
* Set the consider_partitionwise_join flag.
@@ -1717,7 +1876,7 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* partition keys.
*/
static bool
-have_partkey_equi_join(RelOptInfo *joinrel,
+have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *rel1, RelOptInfo *rel2,
JoinType jointype, List *restrictlist)
{
@@ -1782,6 +1941,24 @@ have_partkey_equi_join(RelOptInfo *joinrel,
*/
strict_op = op_strict(opexpr->opno);
+ /*
+ * Vars appearing in the relation's partition keys will not have any
+ * varnullingrels, but those in expr1 and expr2 will if we're above
+ * outer joins that could null the respective rels. It's okay to
+ * match anyway, if the join operator is strict.
+ */
+ if (strict_op)
+ {
+ if (bms_overlap(rel1->relids, root->outer_join_rels))
+ expr1 = (Expr *) remove_nulling_relids((Node *) expr1,
+ root->outer_join_rels,
+ NULL);
+ if (bms_overlap(rel2->relids, root->outer_join_rels))
+ expr2 = (Expr *) remove_nulling_relids((Node *) expr2,
+ root->outer_join_rels,
+ NULL);
+ }
+
/*
* Only clauses referencing the partition keys are useful for
* partitionwise join.
@@ -1994,7 +2171,12 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
* partitionwise nesting of any outer join.) We assume no
* type coercions are needed to make the coalesce expressions,
* since columns of different types won't have gotten
- * classified as the same PartitionScheme.
+ * classified as the same PartitionScheme. Note that we
+ * intentionally leave out the varnullingrels decoration that
+ * would ordinarily appear on the Vars inside these
+ * CoalesceExprs, because have_partkey_equi_join will strip
+ * varnullingrels from the expressions it will compare to the
+ * partexprs.
*/
foreach(lc, list_concat_copy(outer_expr, outer_null_expr))
{
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index ef8df3d098..327c3ba563 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -53,6 +53,10 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
* required_relids can be NULL, in which case it defaults to the actual clause
* contents (i.e., clause_relids).
*
+ * Note that there aren't options to set the has_clone and is_clone flags:
+ * we always initialize those to false. There's just one place that wants
+ * something different, so making all callers pass them seems inconvenient.
+ *
* We initialize fields that depend only on the given subexpression, leaving
* others that depend on context (or may never be needed at all) to be filled
* later.
@@ -116,12 +120,15 @@ make_restrictinfo_internal(PlannerInfo *root,
Relids nullable_relids)
{
RestrictInfo *restrictinfo = makeNode(RestrictInfo);
+ Relids baserels;
restrictinfo->clause = clause;
restrictinfo->orclause = orclause;
restrictinfo->is_pushed_down = is_pushed_down;
restrictinfo->outerjoin_delayed = outerjoin_delayed;
restrictinfo->pseudoconstant = pseudoconstant;
+ restrictinfo->has_clone = false; /* may get set by caller */
+ restrictinfo->is_clone = false; /* may get set by caller */
restrictinfo->can_join = false; /* may get set below */
restrictinfo->security_level = security_level;
restrictinfo->outer_relids = outer_relids;
@@ -187,6 +194,20 @@ make_restrictinfo_internal(PlannerInfo *root,
else
restrictinfo->required_relids = restrictinfo->clause_relids;
+ /*
+ * Count the number of base rels appearing in clause_relids. To do this,
+ * we just delete rels mentioned in root->outer_join_rels and count the
+ * survivors. Because we are called during deconstruct_jointree which is
+ * the same tree walk that populates outer_join_rels, this is a little bit
+ * unsafe-looking; but it should be fine because the recursion in
+ * deconstruct_jointree should already have visited any outer join that
+ * could be mentioned in this clause.
+ */
+ baserels = bms_difference(restrictinfo->clause_relids,
+ root->outer_join_rels);
+ restrictinfo->num_base_rels = bms_num_members(baserels);
+ bms_free(baserels);
+
/*
* Fill in all the cacheable fields with "not yet set" markers. None of
* these will be computed until/unless needed. Note in particular that we
@@ -497,6 +518,58 @@ extract_actual_join_clauses(List *restrictinfo_list,
}
}
+/*
+ * clause_is_computable_at
+ * Test whether a clause is computable at a given evaluation level.
+ *
+ * There are two conditions for whether an expression can actually be
+ * evaluated at a given join level: the evaluation context must include
+ * all the relids (both base and OJ) used by the expression, and we must
+ * not have already evaluated any outer joins that null Vars/PHVs of the
+ * expression and are not listed in their nullingrels.
+ *
+ * This function checks the second condition; we assume the caller already
+ * saw to the first one.
+ *
+ * For speed reasons, we don't individually examine each Var/PHV of the
+ * expression, but just look at the overall clause_relids (the union of the
+ * varnos and varnullingrels). This could give a misleading answer if the
+ * Vars of a given varno don't all have the same varnullingrels; but that
+ * really shouldn't happen within a single scalar expression or RestrictInfo
+ * clause. Despite that, this is still annoyingly expensive :-(
+ */
+bool
+clause_is_computable_at(PlannerInfo *root,
+ Relids clause_relids,
+ Relids eval_relids)
+{
+ ListCell *lc;
+
+ /* Nothing to do if no outer joins have been performed yet. */
+ if (!bms_overlap(eval_relids, root->outer_join_rels))
+ return true;
+
+ foreach(lc, root->join_info_list)
+ {
+ SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+
+ /* Ignore outer joins that are not yet performed. */
+ if (!bms_is_member(sjinfo->ojrelid, eval_relids))
+ continue;
+
+ /* OK if clause lists it (we assume all Vars in it agree). */
+ if (bms_is_member(sjinfo->ojrelid, clause_relids))
+ continue;
+
+ /* Else, trouble if clause mentions any nullable Vars. */
+ if (bms_overlap(clause_relids, sjinfo->min_righthand) ||
+ (sjinfo->jointype == JOIN_FULL &&
+ bms_overlap(clause_relids, sjinfo->min_lefthand)))
+ return false; /* doesn't work */
+ }
+
+ return true; /* OK */
+}
/*
* join_clause_is_movable_to
@@ -522,6 +595,12 @@ extract_actual_join_clauses(List *restrictinfo_list,
* Also, the join clause must not use any relations that have LATERAL
* references to the target relation, since we could not put such rels on
* the outer side of a nestloop with the target relation.
+ *
+ * Also, we reject is_clone versions of outer-join clauses. This has the
+ * effect of preventing us from generating variant parameterized paths
+ * that differ only in which outer joins null the parameterization rel(s).
+ * Generating one path from the minimally-parameterized has_clone version
+ * is sufficient.
*/
bool
join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
@@ -542,6 +621,10 @@ join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
if (bms_overlap(baserel->lateral_referencers, rinfo->clause_relids))
return false;
+ /* Ignore clones, too */
+ if (rinfo->is_clone)
+ return false;
+
return true;
}
@@ -587,6 +670,9 @@ join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
* moved for some valid set of outer rels, so we don't have the benefit of
* relying on prior checks for lateral-reference validity.
*
+ * Likewise, we don't check is_clone here: rejecting the inappropriate
+ * variants of a cloned clause must be handled upstream.
+ *
* Note: if this returns true, it means that the clause could be moved to
* this join relation, but that doesn't mean that this is the lowest join
* it could be moved to. Caller may need to make additional calls to verify
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
index 7db86c39ef..8d8c9136f8 100644
--- a/src/backend/optimizer/util/var.c
+++ b/src/backend/optimizer/util/var.c
@@ -88,6 +88,9 @@ static Relids alias_relid_set(Query *query, Relids relids);
* Create a set of all the distinct varnos present in a parsetree.
* Only varnos that reference level-zero rtable entries are considered.
*
+ * The result includes outer-join relids mentioned in Var.varnullingrels and
+ * PlaceHolderVar.phnullingrels fields in the parsetree.
+ *
* "root" can be passed as NULL if it is not necessary to process
* PlaceHolderVars.
*
@@ -153,7 +156,11 @@ pull_varnos_walker(Node *node, pull_varnos_context *context)
Var *var = (Var *) node;
if (var->varlevelsup == context->sublevels_up)
+ {
context->varnos = bms_add_member(context->varnos, var->varno);
+ context->varnos = bms_add_members(context->varnos,
+ var->varnullingrels);
+ }
return false;
}
if (IsA(node, CurrentOfExpr))
@@ -244,6 +251,14 @@ pull_varnos_walker(Node *node, pull_varnos_context *context)
context->varnos = bms_join(context->varnos,
newevalat);
}
+
+ /*
+ * In all three cases, include phnullingrels in the result. We
+ * don't worry about possibly needing to translate it, because
+ * appendrels only translate varnos of baserels, not outer joins.
+ */
+ context->varnos = bms_add_members(context->varnos,
+ phv->phnullingrels);
return false; /* don't recurse into expression */
}
}
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index d597b7e81f..40129d435e 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -2205,7 +2205,7 @@ rowcomparesel(PlannerInfo *root,
else
{
/*
- * Otherwise, it's a join if there's more than one relation used.
+ * Otherwise, it's a join if there's more than one base relation used.
*/
is_join_clause = (NumRelids(root, (Node *) opargs) > 1);
}
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 3ab6d75bfb..2ddd245992 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -243,13 +243,26 @@ struct PlannerInfo
struct AppendRelInfo **append_rel_array pg_node_attr(read_write_ignore);
/*
- * all_baserels is a Relids set of all base relids (but not "other"
- * relids) in the query; that is, the Relids identifier of the final join
- * we need to form. This is computed in make_one_rel, just before we
- * start making Paths.
+ * all_baserels is a Relids set of all base relids (but not joins or
+ * "other" relids) in the query. This is computed in
+ * add_base_rels_to_query.
*/
Relids all_baserels;
+ /*
+ * outer_join_rels is a Relids set of all outer-join relids in the query.
+ * This is computed in deconstruct_jointree.
+ */
+ Relids outer_join_rels;
+
+ /*
+ * all_query_rels is a Relids set of all base relids and outer join relids
+ * (but not "other" relids) in the query. This is the Relids identifier
+ * of the final join we need to form. This is computed in
+ * deconstruct_jointree.
+ */
+ Relids all_query_rels;
+
/*
* nullable_baserels is a Relids set of base relids that are nullable by
* some outer join in the jointree; these are rels that are potentially
@@ -319,7 +332,7 @@ struct PlannerInfo
List *right_join_clauses;
/*
- * list of RestrictInfos for mergejoinable full join clauses
+ * list of FullJoinClauseInfos for mergejoinable full join clauses
*/
List *full_join_clauses;
@@ -555,9 +568,10 @@ typedef struct PartitionSchemeData *PartitionScheme;
* or the output of a sub-SELECT or function that appears in the range table.
* In either case it is uniquely identified by an RT index. A "joinrel"
* is the joining of two or more base rels. A joinrel is identified by
- * the set of RT indexes for its component baserels. We create RelOptInfo
- * nodes for each baserel and joinrel, and store them in the PlannerInfo's
- * simple_rel_array and join_rel_list respectively.
+ * the set of RT indexes for its component baserels, along with RT indexes
+ * for any outer joins it has computed. We create RelOptInfo nodes for each
+ * baserel and joinrel, and store them in the PlannerInfo's simple_rel_array
+ * and join_rel_list respectively.
*
* Note that there is only one joinrel for any given set of component
* baserels, no matter what order we assemble them in; so an unordered
@@ -596,8 +610,10 @@ typedef struct PartitionSchemeData *PartitionScheme;
* Parts of this data structure are specific to various scan and join
* mechanisms. It didn't seem worth creating new node types for them.
*
- * relids - Set of base-relation identifiers; it is a base relation
- * if there is just one, a join relation if more than one
+ * relids - Set of relation identifiers (RT indexes). This is a base
+ * relation if there is just one, a join relation if more;
+ * in the join case, RT indexes of any outer joins formed
+ * at or below this join are included along with baserels
* rows - estimated number of tuples in the relation after restriction
* clauses have been applied (ie, output rows of a plan for it)
* consider_startup - true if there is any value in keeping plain paths for
@@ -636,7 +652,9 @@ typedef struct PartitionSchemeData *PartitionScheme;
* min_attr, max_attr - range of valid AttrNumbers for rel
* attr_needed - array of bitmapsets indicating the highest joinrel
* in which each attribute is needed; if bit 0 is set then
- * the attribute is needed as part of final targetlist
+ * the attribute is needed as part of final targetlist.
+ * By convention, attr_needed includes only baserels not
+ * outer-join relids.
* attr_widths - cache space for per-attribute width estimates;
* zero means not computed yet
* lateral_vars - lateral cross-references of rel, if any (list of
@@ -809,7 +827,7 @@ typedef struct RelOptInfo
RelOptKind reloptkind;
/*
- * all relations included in this RelOptInfo; set of base relids
+ * all relations included in this RelOptInfo; set of base + OJ relids
* (rangetable indexes)
*/
Relids relids;
@@ -2276,17 +2294,17 @@ typedef struct LimitPath
* If a restriction clause references a single base relation, it will appear
* in the baserestrictinfo list of the RelOptInfo for that base rel.
*
- * If a restriction clause references more than one base rel, it will
+ * If a restriction clause references more than one base+OJ relation, it will
* appear in the joininfo list of every RelOptInfo that describes a strict
- * subset of the base rels mentioned in the clause. The joininfo lists are
+ * subset of the relations mentioned in the clause. The joininfo lists are
* used to drive join tree building by selecting plausible join candidates.
* The clause cannot actually be applied until we have built a join rel
- * containing all the base rels it references, however.
+ * containing all the relations it references, however.
*
- * When we construct a join rel that includes all the base rels referenced
+ * When we construct a join rel that includes all the relations referenced
* in a multi-relation restriction clause, we place that clause into the
* joinrestrictinfo lists of paths for the join rel, if neither left nor
- * right sub-path includes all base rels referenced in the clause. The clause
+ * right sub-path includes all relations referenced in the clause. The clause
* will be applied at that join level, and will not propagate any further up
* the join tree. (Note: the "predicate migration" code was once intended to
* push restriction clauses up and down the plan tree based on evaluation
@@ -2307,12 +2325,15 @@ typedef struct LimitPath
* or join to enforce that all members of each EquivalenceClass are in fact
* equal in all rows emitted by the scan or join.
*
- * When dealing with outer joins we have to be very careful about pushing qual
- * clauses up and down the tree. An outer join's own JOIN/ON conditions must
- * be evaluated exactly at that join node, unless they are "degenerate"
- * conditions that reference only Vars from the nullable side of the join.
- * Quals appearing in WHERE or in a JOIN above the outer join cannot be pushed
- * down below the outer join, if they reference any nullable Vars.
+ * The clause_relids field lists the base plus outer-join RT indexes that
+ * actually appear in the clause. required_relids lists the minimum set of
+ * relids needed to evaluate the clause; while this is often equal to
+ * clause_relids, it can be more. We will add relids to required_relids when
+ * we need to force an outer join ON clause to be evaluated exactly at the
+ * level of the outer join, which is true except when it is a "degenerate"
+ * condition that references only Vars from the nullable side of the join.
+ *
+ * XXX rewrite or remove me:
* RestrictInfo nodes contain a flag to indicate whether a qual has been
* pushed down to a lower level than its original syntactic placement in the
* join tree would suggest. If an outer join prevents us from pushing a qual
@@ -2397,6 +2418,12 @@ typedef struct LimitPath
* or merge or hash join clause, so it's of no interest to large parts of
* the planner.
*
+ * When we generate multiple versions of a clause so as to have versions
+ * that will work after commuting some left joins per outer join identity 3,
+ * we mark the one with the fewest nullingrels bits with has_clone = true,
+ * and the rest with is_clone = true. This allows proper filtering of
+ * these redundant clauses, so that we apply only one version of them.
+ *
* When join clauses are generated from EquivalenceClasses, there may be
* several equally valid ways to enforce join equivalence, of which we need
* apply only one. We mark clauses of this kind by setting parent_ec to
@@ -2431,16 +2458,23 @@ typedef struct RestrictInfo
/* see comment above */
bool pseudoconstant pg_node_attr(equal_ignore);
+ /* see comment above */
+ bool has_clone;
+ bool is_clone;
+
/* true if known to contain no leaked Vars */
bool leakproof pg_node_attr(equal_ignore);
- /* to indicate if clause contains any volatile functions. */
+ /* indicates if clause contains any volatile functions */
VolatileFunctionStatus has_volatile pg_node_attr(equal_ignore);
/* see comment above */
Index security_level;
- /* The set of relids (varnos) actually referenced in the clause: */
+ /* number of base rels in clause_relids */
+ int num_base_rels pg_node_attr(equal_ignore);
+
+ /* The relids (varnos+varnullingrels) actually referenced in the clause: */
Relids clause_relids pg_node_attr(equal_ignore);
/* The set of relids required to evaluate the clause: */
@@ -2542,6 +2576,7 @@ typedef struct RestrictInfo
} RestrictInfo;
/*
+ * XXX this will need work:
* This macro embodies the correct way to test whether a RestrictInfo is
* "pushed down" to a given outer join, that is, should be treated as a filter
* clause rather than a join clause at that outer join. This is certainly so
@@ -2644,7 +2679,7 @@ typedef struct PlaceHolderVar
* We make SpecialJoinInfos for FULL JOINs even though there is no flexibility
* of planning for them, because this simplifies make_join_rel()'s API.
*
- * min_lefthand and min_righthand are the sets of base relids that must be
+ * min_lefthand and min_righthand are the sets of base+OJ relids that must be
* available on each side when performing the special join. lhs_strict is
* true if the special join's condition cannot succeed when the LHS variables
* are all NULL (this means that an outer join can commute with upper-level
@@ -2654,7 +2689,7 @@ typedef struct PlaceHolderVar
* It is not valid for either min_lefthand or min_righthand to be empty sets;
* if they were, this would break the logic that enforces join order.
*
- * syn_lefthand and syn_righthand are the sets of base relids that are
+ * syn_lefthand and syn_righthand are the sets of base+OJ relids that are
* syntactically below this special join. (These are needed to help compute
* min_lefthand and min_righthand for higher joins.)
*
@@ -2676,6 +2711,37 @@ typedef struct PlaceHolderVar
* the inputs to make it a LEFT JOIN. So the allowed values of jointype
* in a join_info_list member are only LEFT, FULL, SEMI, or ANTI.
*
+ * ojrelid is the RT index of the join RTE representing this outer join,
+ * if there is one. It is zero when jointype is INNER or SEMI, and can be
+ * zero for jointype ANTI (if the join was transformed from a SEMI join).
+ * One use for this field is that when constructing the output targetlist of a
+ * join relation that implements this OJ, we add ojrelid to the varnullingrels
+ * and phnullingrels fields of nullable (RHS) output columns, so that the
+ * output Vars and PlaceHolderVars correctly reflect the nulling that has
+ * potentially happened to them.
+ *
+ * commute_above_l is filled with the relids of syntactically-higher outer
+ * joins that have been found to commute with this one per outer join identity
+ * 3 (see optimizer/README), when this join is in the LHS of the upper join
+ * (so, this is the lower join in the first form of the identity).
+ *
+ * commute_above_r is filled with the relids of syntactically-higher outer
+ * joins that have been found to commute with this one per outer join identity
+ * 3, when this join is in the RHS of the upper join (so, this is the lower
+ * join in the second form of the identity).
+ *
+ * commute_below is filled with the relids of syntactically-lower outer joins
+ * that have been found to commute with this one per outer join identity 3.
+ * (We need not record which side they are on, since that can be determined
+ * by seeing whether the lower join's relid appears in syn_lefthand or
+ * syn_righthand.)
+ *
+ * oj_joinclause is used during deconstruct_jointree() to hold the JOIN/ON
+ * quals of a possibly-commutable outer join until the end of the jointree
+ * walk (at which time we'll know whether any other outer joins actually
+ * commute with it, and can decorate the quals properly). These quals do not
+ * have RestrictInfos yet.
+ *
* For purposes of join selectivity estimation, we create transient
* SpecialJoinInfo structures for regular inner joins; so it is possible
* to have jointype == JOIN_INNER in such a structure, even though this is
@@ -2695,11 +2761,16 @@ struct SpecialJoinInfo
pg_node_attr(no_read)
NodeTag type;
- Relids min_lefthand; /* base relids in minimum LHS for join */
- Relids min_righthand; /* base relids in minimum RHS for join */
- Relids syn_lefthand; /* base relids syntactically within LHS */
- Relids syn_righthand; /* base relids syntactically within RHS */
+ Relids min_lefthand; /* base+OJ relids in minimum LHS for join */
+ Relids min_righthand; /* base+OJ relids in minimum RHS for join */
+ Relids syn_lefthand; /* base+OJ relids syntactically within LHS */
+ Relids syn_righthand; /* base+OJ relids syntactically within RHS */
JoinType jointype; /* always INNER, LEFT, FULL, SEMI, or ANTI */
+ Index ojrelid; /* outer join's RT index; 0 if none */
+ Relids commute_above_l; /* commuting OJs above this one, if LHS */
+ Relids commute_above_r; /* commuting OJs above this one, if RHS */
+ Relids commute_below; /* commuting OJs below this one */
+ List *oj_joinclause; /* outer join quals not yet distributed */
bool lhs_strict; /* joinclause is strict for some LHS rel */
bool delay_upper_joins; /* can't commute with upper RHS */
/* Remaining fields are set only for JOIN_SEMI jointype: */
@@ -2709,6 +2780,21 @@ struct SpecialJoinInfo
List *semi_rhs_exprs; /* righthand-side expressions of these ops */
};
+/*
+ * FULL JOIN clause info.
+ *
+ * We set aside every FULL JOIN ON clause that looks mergejoinable, and
+ * process it specially at the end of qual distribution.
+ */
+typedef struct FullJoinClauseInfo
+{
+ pg_node_attr(no_copy_equal, no_read)
+
+ NodeTag type;
+ RestrictInfo *rinfo; /* a mergejoinable FULL JOIN clause */
+ SpecialJoinInfo *sjinfo; /* the FULL JOIN's SpecialJoinInfo */
+} FullJoinClauseInfo;
+
/*
* Append-relation info.
*
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 050f00e79a..197234d44c 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -304,6 +304,7 @@ extern void expand_planner_arrays(PlannerInfo *root, int add_size);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
+extern RelOptInfo *find_base_rel_ignore_join(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
@@ -335,6 +336,6 @@ extern ParamPathInfo *find_param_path_info(RelOptInfo *rel,
extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
- SpecialJoinInfo *sjinfo, JoinType jointype);
+ SpecialJoinInfo *sjinfo);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/placeholder.h b/src/include/optimizer/placeholder.h
index 507dbc6175..3fe9b57415 100644
--- a/src/include/optimizer/placeholder.h
+++ b/src/include/optimizer/placeholder.h
@@ -27,6 +27,9 @@ extern void update_placeholder_eval_levels(PlannerInfo *root,
extern void fix_placeholder_input_needed_levels(PlannerInfo *root);
extern void add_placeholders_to_base_rels(PlannerInfo *root);
extern void add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *outer_rel, RelOptInfo *inner_rel);
+ RelOptInfo *outer_rel, RelOptInfo *inner_rel,
+ SpecialJoinInfo *sjinfo);
+extern bool contain_placeholder_references_to(PlannerInfo *root, Node *clause,
+ int relid);
#endif /* PLACEHOLDER_H */
diff --git a/src/include/optimizer/prep.h b/src/include/optimizer/prep.h
index 5b4f350b33..0847cfd5f4 100644
--- a/src/include/optimizer/prep.h
+++ b/src/include/optimizer/prep.h
@@ -29,7 +29,8 @@ extern void pull_up_subqueries(PlannerInfo *root);
extern void flatten_simple_union_all(PlannerInfo *root);
extern void reduce_outer_joins(PlannerInfo *root);
extern void remove_useless_result_rtes(PlannerInfo *root);
-extern Relids get_relids_in_jointree(Node *jtnode, bool include_joins);
+extern Relids get_relids_in_jointree(Node *jtnode, bool include_outer_joins,
+ bool include_inner_joins);
extern Relids get_relids_for_join(Query *query, int joinrelid);
/*
diff --git a/src/include/optimizer/restrictinfo.h b/src/include/optimizer/restrictinfo.h
index 6d30bd5e9d..17d3b4ab05 100644
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -41,6 +41,9 @@ extern void extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
List **joinquals,
List **otherquals);
+extern bool clause_is_computable_at(PlannerInfo *root,
+ Relids clause_relids,
+ Relids eval_relids);
extern bool join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel);
extern bool join_clause_is_movable_into(RestrictInfo *rinfo,
Relids currentrelids,
commit 347b5fb29ed9448a85bb6ea067718ca0a34c86a6
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Wed Nov 16 15:52:51 2022 -0500
Detect duplicated pushed-down conditions using RestrictInfo ID numbers.
create_nestloop_path needs to identify which candidates for join
restriction quals were already enforced in the parameterized inner
path. Currently we do that by relying on join_clause_is_movable_into
to give consistent answers, but that is not working very well with
variant clauses generated to satisfy outer join identity 3. We may
have a clause that (correctly) shows the outer-side Var as nulled by
a previous outer join, which makes it dependent on the nestloop outer
side having included that join, so that it appears to not be pushable
into a parameterized path that uses the un-nulled version of that Var.
Nonetheless, the cloned clause *is* redundant and we don't want
to check it again.
This patch offers a somewhat brute-force solution, which is to assign
serial numbers to RestrictInfo nodes, then check for redundancy using
serial number match rather than trusting join_clause_is_movable_into.
The variant-clause problem can be solved by allowing clauses to share
a serial number when we know that they are equivalent. Both the
outer-join variant generator and equivclass.c need to be in on that
trick in order to handle all cases that were handled well before.
It'd be nicer if we could continue to trust join_clause_is_movable_into
for this, but on the other hand this mechanism does provide a much more
concrete, harder-to-break way of verifying that we already enforced
(some version of) a qual. Any failure mode would almost certainly
be in the safe direction of enforcing a qual redundantly, which is
not a claim that the existing method can make.
This patch results in two changes to the core regression test outputs:
* One query in join.sql changes to a different join order. Examining
the cost estimates that are normally not shown, the new order is
estimated as very slightly faster, so this seems like an improvement.
I'm not quite sure why the old code did not find this join order.
* Some of the queries in partition_join.sql revert equivalence-clause
ordering back to what it was before a5fc46414. That's probably a
consequence of investigating parameterized paths in a different order
than before. Anyway, it's visibly harmless.
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 349e183372..d4f8b7893d 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -35,7 +35,8 @@
static EquivalenceMember *add_eq_member(EquivalenceClass *ec,
Expr *expr, Relids relids, Relids nullable_relids,
- bool is_child, Oid datatype);
+ EquivalenceMember *parent,
+ Oid datatype);
static bool is_exprlist_member(Expr *node, List *exprs);
static void generate_base_implied_equalities_const(PlannerInfo *root,
EquivalenceClass *ec);
@@ -400,7 +401,7 @@ process_equivalence(PlannerInfo *root,
{
/* Case 3: add item2 to ec1 */
em2 = add_eq_member(ec1, item2, item2_relids, item2_nullable_relids,
- false, item2_type);
+ NULL, item2_type);
ec1->ec_sources = lappend(ec1->ec_sources, restrictinfo);
ec1->ec_below_outer_join |= below_outer_join;
ec1->ec_min_security = Min(ec1->ec_min_security,
@@ -418,7 +419,7 @@ process_equivalence(PlannerInfo *root,
{
/* Case 3: add item1 to ec2 */
em1 = add_eq_member(ec2, item1, item1_relids, item1_nullable_relids,
- false, item1_type);
+ NULL, item1_type);
ec2->ec_sources = lappend(ec2->ec_sources, restrictinfo);
ec2->ec_below_outer_join |= below_outer_join;
ec2->ec_min_security = Min(ec2->ec_min_security,
@@ -452,9 +453,9 @@ process_equivalence(PlannerInfo *root,
ec->ec_max_security = restrictinfo->security_level;
ec->ec_merged = NULL;
em1 = add_eq_member(ec, item1, item1_relids, item1_nullable_relids,
- false, item1_type);
+ NULL, item1_type);
em2 = add_eq_member(ec, item2, item2_relids, item2_nullable_relids,
- false, item2_type);
+ NULL, item2_type);
root->eq_classes = lappend(root->eq_classes, ec);
@@ -544,7 +545,7 @@ canonicalize_ec_expression(Expr *expr, Oid req_type, Oid req_collation)
*/
static EquivalenceMember *
add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
- Relids nullable_relids, bool is_child, Oid datatype)
+ Relids nullable_relids, EquivalenceMember *parent, Oid datatype)
{
EquivalenceMember *em = makeNode(EquivalenceMember);
@@ -552,8 +553,9 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
em->em_relids = relids;
em->em_nullable_relids = nullable_relids;
em->em_is_const = false;
- em->em_is_child = is_child;
+ em->em_is_child = (parent != NULL);
em->em_datatype = datatype;
+ em->em_parent = parent;
if (bms_is_empty(relids))
{
@@ -565,12 +567,12 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
* get_eclass_for_sort_expr() has to work harder. We put the tests
* there not here to save cycles in the equivalence case.
*/
- Assert(!is_child);
+ Assert(!parent);
em->em_is_const = true;
ec->ec_has_const = true;
/* it can't affect ec_relids */
}
- else if (!is_child) /* child members don't add to ec_relids */
+ else if (!parent) /* child members don't add to ec_relids */
{
ec->ec_relids = bms_add_members(ec->ec_relids, relids);
}
@@ -723,7 +725,7 @@ get_eclass_for_sort_expr(PlannerInfo *root,
nullable_relids = bms_intersect(nullable_relids, expr_relids);
newem = add_eq_member(newec, copyObject(expr), expr_relids,
- nullable_relids, false, opcintype);
+ nullable_relids, NULL, opcintype);
/*
* add_eq_member doesn't check for volatile functions, set-returning
@@ -1821,6 +1823,7 @@ create_join_clause(PlannerInfo *root,
EquivalenceClass *parent_ec)
{
RestrictInfo *rinfo;
+ RestrictInfo *parent_rinfo = NULL;
ListCell *lc;
MemoryContext oldcontext;
@@ -1865,6 +1868,20 @@ create_join_clause(PlannerInfo *root,
*/
oldcontext = MemoryContextSwitchTo(root->planner_cxt);
+ /*
+ * If either EM is a child, recursively create the corresponding
+ * parent-to-parent clause, so that we can duplicate its rinfo_serial.
+ */
+ if (leftem->em_is_child || rightem->em_is_child)
+ {
+ EquivalenceMember *leftp = leftem->em_parent ? leftem->em_parent : leftem;
+ EquivalenceMember *rightp = rightem->em_parent ? rightem->em_parent : rightem;
+
+ parent_rinfo = create_join_clause(root, ec, opno,
+ leftp, rightp,
+ parent_ec);
+ }
+
rinfo = build_implied_join_equality(root,
opno,
ec->ec_collation,
@@ -1876,6 +1893,10 @@ create_join_clause(PlannerInfo *root,
rightem->em_nullable_relids),
ec->ec_min_security);
+ /* If it's a child clause, copy the parent's rinfo_serial */
+ if (parent_rinfo)
+ rinfo->rinfo_serial = parent_rinfo->rinfo_serial;
+
/* Mark the clause as redundant, or not */
rinfo->parent_ec = parent_ec;
@@ -2686,7 +2707,7 @@ add_child_rel_equivalences(PlannerInfo *root,
(void) add_eq_member(cur_ec, child_expr,
new_relids, new_nullable_relids,
- true, cur_em->em_datatype);
+ cur_em, cur_em->em_datatype);
/* Record this EC index for the child rel */
child_rel->eclass_indexes = bms_add_member(child_rel->eclass_indexes, i);
@@ -2827,7 +2848,7 @@ add_child_join_rel_equivalences(PlannerInfo *root,
(void) add_eq_member(cur_ec, child_expr,
new_relids, new_nullable_relids,
- true, cur_em->em_datatype);
+ cur_em, cur_em->em_datatype);
}
}
}
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 516984c655..a128780857 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -1783,6 +1783,7 @@ process_postponed_left_join_quals(PlannerInfo *root)
Relids joins_below;
Relids joins_so_far;
List *quals;
+ int save_last_rinfo_serial;
ListCell *lc2;
/*
@@ -1821,6 +1822,16 @@ process_postponed_left_join_quals(PlannerInfo *root)
joins_below,
NULL);
+ /*
+ * Each time we produce RestrictInfo(s) from these quals, reset
+ * the last_rinfo_serial counter, so that the RestrictInfos for
+ * the "same" qual condition get identical serial numbers. (This
+ * relies on the fact that we're not changing the qual list in any
+ * way that'd affect the number of RestrictInfos built from it.)
+ * This'll allow us to detect duplicative qual usage later.
+ */
+ save_last_rinfo_serial = root->last_rinfo_serial;
+
joins_so_far = NULL;
foreach(lc2, join_info_list_orig)
{
@@ -1854,6 +1865,9 @@ process_postponed_left_join_quals(PlannerInfo *root)
continue;
}
+ /* Reset serial counter for this version of the quals */
+ root->last_rinfo_serial = save_last_rinfo_serial;
+
/*
* When we are looking at joins above sjinfo, we are
* envisioning pushing sjinfo to above othersj, so add
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index e743a5d9fe..d52c2a3595 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -620,6 +620,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
root->multiexpr_params = NIL;
root->eq_classes = NIL;
root->ec_merging_done = false;
+ root->last_rinfo_serial = 0;
root->all_result_relids =
parse->resultRelation ? bms_make_singleton(parse->resultRelation) : NULL;
root->leaf_result_relids = NULL; /* we'll find out leaf-ness later */
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 68fb712472..fc56a81be8 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -990,6 +990,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
subroot->multiexpr_params = NIL;
subroot->eq_classes = NIL;
subroot->ec_merging_done = false;
+ subroot->last_rinfo_serial = 0;
subroot->all_result_relids = NULL;
subroot->leaf_result_relids = NULL;
subroot->append_rel_list = NIL;
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index 11c6bbaba6..e18d64b6dc 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -427,7 +427,7 @@ adjust_appendrel_attrs_mutator(Node *node,
RestrictInfo *oldinfo = (RestrictInfo *) node;
RestrictInfo *newinfo = makeNode(RestrictInfo);
- /* Copy all flat-copiable fields */
+ /* Copy all flat-copiable fields, notably including rinfo_serial */
memcpy(newinfo, oldinfo, sizeof(RestrictInfo));
/* Recursively fix the clause itself */
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index bf35d1989c..c77399ca92 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2442,12 +2442,12 @@ create_nestloop_path(PlannerInfo *root,
* restrict_clauses that are due to be moved into the inner path. We have
* to do this now, rather than postpone the work till createplan time,
* because the restrict_clauses list can affect the size and cost
- * estimates for this path.
+ * estimates for this path. We detect such clauses by checking for serial
+ * number match to clauses already enforced in the inner path.
*/
if (bms_overlap(inner_req_outer, outer_path->parent->relids))
{
- Relids inner_and_outer = bms_union(inner_path->parent->relids,
- inner_req_outer);
+ Bitmapset *enforced_serials = get_param_path_clause_serials(inner_path);
List *jclauses = NIL;
ListCell *lc;
@@ -2455,9 +2455,7 @@ create_nestloop_path(PlannerInfo *root,
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
- if (!join_clause_is_movable_into(rinfo,
- inner_path->parent->relids,
- inner_and_outer))
+ if (!bms_is_member(rinfo->rinfo_serial, enforced_serials))
jclauses = lappend(jclauses, rinfo);
}
restrict_clauses = jclauses;
@@ -4268,6 +4266,7 @@ do { \
new_ppi->ppi_rows = old_ppi->ppi_rows;
new_ppi->ppi_clauses = old_ppi->ppi_clauses;
ADJUST_CHILD_ATTRS(new_ppi->ppi_clauses);
+ new_ppi->ppi_serials = bms_copy(old_ppi->ppi_serials);
rel->ppilist = lappend(rel->ppilist, new_ppi);
MemoryContextSwitchTo(oldcontext);
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 84e5e8db7b..38540e6331 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -1458,6 +1458,7 @@ get_baserel_parampathinfo(PlannerInfo *root, RelOptInfo *baserel,
ParamPathInfo *ppi;
Relids joinrelids;
List *pclauses;
+ Bitmapset *pserials;
double rows;
ListCell *lc;
@@ -1500,6 +1501,15 @@ get_baserel_parampathinfo(PlannerInfo *root, RelOptInfo *baserel,
required_outer,
baserel));
+ /* Compute set of serial numbers of the enforced clauses */
+ pserials = NULL;
+ foreach(lc, pclauses)
+ {
+ RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
+
+ pserials = bms_add_member(pserials, rinfo->rinfo_serial);
+ }
+
/* Estimate the number of rows returned by the parameterized scan */
rows = get_parameterized_baserel_size(root, baserel, pclauses);
@@ -1508,6 +1518,7 @@ get_baserel_parampathinfo(PlannerInfo *root, RelOptInfo *baserel,
ppi->ppi_req_outer = required_outer;
ppi->ppi_rows = rows;
ppi->ppi_clauses = pclauses;
+ ppi->ppi_serials = pserials;
baserel->ppilist = lappend(baserel->ppilist, ppi);
return ppi;
@@ -1733,6 +1744,7 @@ get_joinrel_parampathinfo(PlannerInfo *root, RelOptInfo *joinrel,
ppi->ppi_req_outer = required_outer;
ppi->ppi_rows = rows;
ppi->ppi_clauses = NIL;
+ ppi->ppi_serials = NULL;
joinrel->ppilist = lappend(joinrel->ppilist, ppi);
return ppi;
@@ -1771,6 +1783,7 @@ get_appendrel_parampathinfo(RelOptInfo *appendrel, Relids required_outer)
ppi->ppi_req_outer = required_outer;
ppi->ppi_rows = 0;
ppi->ppi_clauses = NIL;
+ ppi->ppi_serials = NULL;
appendrel->ppilist = lappend(appendrel->ppilist, ppi);
return ppi;
@@ -1796,6 +1809,100 @@ find_param_path_info(RelOptInfo *rel, Relids required_outer)
return NULL;
}
+/*
+ * get_param_path_clause_serials
+ * Given a parameterized Path, return the set of pushed-down clauses
+ * (identified by rinfo_serial numbers) enforced within the Path.
+ */
+Bitmapset *
+get_param_path_clause_serials(Path *path)
+{
+ if (path->param_info == NULL)
+ return NULL; /* not parameterized */
+ if (IsA(path, NestPath) ||
+ IsA(path, MergePath) ||
+ IsA(path, HashPath))
+ {
+ /*
+ * For a join path, combine clauses enforced within either input path
+ * with those enforced as joinrestrictinfo in this path. Note that
+ * joinrestrictinfo may include some non-pushed-down clauses, but for
+ * current purposes it's okay if we include those in the result. (To
+ * be more careful, we could check for clause_relids overlapping the
+ * path parameterization, but it's not worth the cycles for now.)
+ */
+ JoinPath *jpath = (JoinPath *) path;
+ Bitmapset *pserials;
+ ListCell *lc;
+
+ pserials = NULL;
+ pserials = bms_add_members(pserials,
+ get_param_path_clause_serials(jpath->outerjoinpath));
+ pserials = bms_add_members(pserials,
+ get_param_path_clause_serials(jpath->innerjoinpath));
+ foreach(lc, jpath->joinrestrictinfo)
+ {
+ RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
+
+ pserials = bms_add_member(pserials, rinfo->rinfo_serial);
+ }
+ return pserials;
+ }
+ else if (IsA(path, AppendPath))
+ {
+ /*
+ * For an appendrel, take the intersection of the sets of clauses
+ * enforced in each input path.
+ */
+ AppendPath *apath = (AppendPath *) path;
+ Bitmapset *pserials;
+ ListCell *lc;
+
+ pserials = NULL;
+ foreach(lc, apath->subpaths)
+ {
+ Path *subpath = (Path *) lfirst(lc);
+ Bitmapset *subserials;
+
+ subserials = get_param_path_clause_serials(subpath);
+ if (lc == list_head(apath->subpaths))
+ pserials = bms_copy(subserials);
+ else
+ pserials = bms_int_members(pserials, subserials);
+ }
+ return pserials;
+ }
+ else if (IsA(path, MergeAppendPath))
+ {
+ /* Same as AppendPath case */
+ MergeAppendPath *apath = (MergeAppendPath *) path;
+ Bitmapset *pserials;
+ ListCell *lc;
+
+ pserials = NULL;
+ foreach(lc, apath->subpaths)
+ {
+ Path *subpath = (Path *) lfirst(lc);
+ Bitmapset *subserials;
+
+ subserials = get_param_path_clause_serials(subpath);
+ if (lc == list_head(apath->subpaths))
+ pserials = bms_copy(subserials);
+ else
+ pserials = bms_int_members(pserials, subserials);
+ }
+ return pserials;
+ }
+ else
+ {
+ /*
+ * Otherwise, it's a baserel path and we can use the
+ * previously-computed set of serial numbers.
+ */
+ return path->param_info->ppi_serials;
+ }
+}
+
/*
* build_joinrel_partition_info
* Checks if the two relations being joined can use partitionwise join
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index 327c3ba563..bcbee8f943 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -208,6 +208,11 @@ make_restrictinfo_internal(PlannerInfo *root,
restrictinfo->num_base_rels = bms_num_members(baserels);
bms_free(baserels);
+ /*
+ * Label this RestrictInfo with a fresh serial number.
+ */
+ restrictinfo->rinfo_serial = ++(root->last_rinfo_serial);
+
/*
* Fill in all the cacheable fields with "not yet set" markers. None of
* these will be computed until/unless needed. Note in particular that we
@@ -371,7 +376,7 @@ commute_restrictinfo(RestrictInfo *rinfo, Oid comm_op)
* ... and adjust those we need to change. Note in particular that we can
* preserve any cached selectivity or cost estimates, since those ought to
* be the same for the new clause. Likewise we can keep the source's
- * parent_ec.
+ * parent_ec. It's also important that we keep the same rinfo_serial.
*/
result->clause = (Expr *) newclause;
result->left_relids = rinfo->right_relids;
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 2ddd245992..c8538bbd67 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -339,6 +339,9 @@ struct PlannerInfo
/* list of SpecialJoinInfos */
List *join_info_list;
+ /* counter for assigning RestrictInfo serial numbers */
+ int last_rinfo_serial;
+
/*
* all_result_relids is empty for SELECT, otherwise it contains at least
* parse->resultRelation. For UPDATE/DELETE/MERGE across an inheritance
@@ -1354,6 +1357,8 @@ typedef struct EquivalenceMember
bool em_is_const; /* expression is pseudoconstant? */
bool em_is_child; /* derived version for a child relation? */
Oid em_datatype; /* the "nominal type" used by the opfamily */
+ /* if em_is_child is true, this links to corresponding EM for top parent */
+ struct EquivalenceMember *em_parent pg_node_attr(read_write_ignore);
} EquivalenceMember;
/*
@@ -1459,7 +1464,13 @@ typedef struct PathTarget
* Note: ppi_clauses is only used in ParamPathInfos for base relation paths;
* in join cases it's NIL because the set of relevant clauses varies depending
* on how the join is formed. The relevant clauses will appear in each
- * parameterized join path's joinrestrictinfo list, instead.
+ * parameterized join path's joinrestrictinfo list, instead. ParamPathInfos
+ * for append relations don't bother with this, either.
+ *
+ * ppi_serials is the set of rinfo_serial numbers for quals that are enforced
+ * by this path. As with ppi_clauses, it's only maintained for baserels.
+ * (We could construct it on-the-fly from ppi_clauses, but it seems better
+ * to materialize a copy.)
*/
typedef struct ParamPathInfo
{
@@ -1470,6 +1481,7 @@ typedef struct ParamPathInfo
Relids ppi_req_outer; /* rels supplying parameters used by path */
Cardinality ppi_rows; /* estimated number of result tuples */
List *ppi_clauses; /* join clauses available from outer rels */
+ Bitmapset *ppi_serials; /* set of rinfo_serial for enforced quals */
} ParamPathInfo;
@@ -2499,6 +2511,25 @@ typedef struct RestrictInfo
*/
Expr *orclause pg_node_attr(equal_ignore);
+ /*----------
+ * Serial number of this RestrictInfo. This is unique within the current
+ * PlannerInfo context, with a few critical exceptions:
+ * 1. When we generate multiple clones of the same qual condition to
+ * cope with outer join identity 3, all the clones get the same serial
+ * number. This reflects that we only want to apply one of them in any
+ * given plan.
+ * 2. If we manufacture a commuted version of a qual to use as an index
+ * condition, it copies the original's rinfo_serial, since it is in
+ * practice the same condition.
+ * 3. RestrictInfos made for a child relation copy their parent's
+ * rinfo_serial. Likewise, when an EquivalenceClass makes a derived
+ * equality clause for a child relation, it copies the rinfo_serial of
+ * the matching equality clause for the parent. This allows detection
+ * of redundant pushed-down equality clauses.
+ *----------
+ */
+ int rinfo_serial;
+
/*
* Generating EquivalenceClass. This field is NULL unless clause is
* potentially redundant.
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 197234d44c..3440455a2e 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -333,6 +333,7 @@ extern ParamPathInfo *get_appendrel_parampathinfo(RelOptInfo *appendrel,
Relids required_outer);
extern ParamPathInfo *find_param_path_info(RelOptInfo *rel,
Relids required_outer);
+extern Bitmapset *get_param_path_clause_serials(Path *path);
extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out
index 9358371072..00f4c58238 100644
--- a/src/test/regress/expected/join.out
+++ b/src/test/regress/expected/join.out
@@ -2335,17 +2335,17 @@ select a.f1, b.f1, t.thousand, t.tenthous from
(select sum(f1)+1 as f1 from int4_tbl i4a) a,
(select sum(f1) as f1 from int4_tbl i4b) b
where b.f1 = t.thousand and a.f1 = b.f1 and (a.f1+b.f1+999) = t.tenthous;
- QUERY PLAN
------------------------------------------------------------------------------------------------------------------------
+ QUERY PLAN
+-----------------------------------------------------------------------------------------------------------------
Nested Loop
- -> Aggregate
- -> Seq Scan on int4_tbl i4b
-> Nested Loop
Join Filter: ((sum(i4b.f1)) = ((sum(i4a.f1) + 1)))
-> Aggregate
-> Seq Scan on int4_tbl i4a
- -> Index Only Scan using tenk1_thous_tenthous on tenk1 t
- Index Cond: ((thousand = (sum(i4b.f1))) AND (tenthous = ((((sum(i4a.f1) + 1)) + (sum(i4b.f1))) + 999)))
+ -> Aggregate
+ -> Seq Scan on int4_tbl i4b
+ -> Index Only Scan using tenk1_thous_tenthous on tenk1 t
+ Index Cond: ((thousand = (sum(i4b.f1))) AND (tenthous = ((((sum(i4a.f1) + 1)) + (sum(i4b.f1))) + 999)))
(9 rows)
select a.f1, b.f1, t.thousand, t.tenthous from
diff --git a/src/test/regress/expected/partition_join.out b/src/test/regress/expected/partition_join.out
index b20facc19f..bb5b7c47a4 100644
--- a/src/test/regress/expected/partition_join.out
+++ b/src/test/regress/expected/partition_join.out
@@ -304,7 +304,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t2.b FROM prt2 t2 WHERE t2.a = 0)
-> Seq Scan on prt2_p2 t2_2
Filter: (a = 0)
-> Nested Loop Semi Join
- Join Filter: (t2_3.b = t1_3.a)
+ Join Filter: (t1_3.a = t2_3.b)
-> Seq Scan on prt1_p3 t1_3
Filter: (b = 0)
-> Materialize
@@ -601,7 +601,7 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM prt1 t1, prt2 t2, prt1_e t
Sort Key: t1.a
-> Append
-> Nested Loop
- Join Filter: (((t3_1.a + t3_1.b) / 2) = t1_1.a)
+ Join Filter: (t1_1.a = ((t3_1.a + t3_1.b) / 2))
-> Hash Join
Hash Cond: (t2_1.b = t1_1.a)
-> Seq Scan on prt2_p1 t2_1
@@ -611,7 +611,7 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM prt1 t1, prt2 t2, prt1_e t
-> Index Scan using iprt1_e_p1_ab2 on prt1_e_p1 t3_1
Index Cond: (((a + b) / 2) = t2_1.b)
-> Nested Loop
- Join Filter: (((t3_2.a + t3_2.b) / 2) = t1_2.a)
+ Join Filter: (t1_2.a = ((t3_2.a + t3_2.b) / 2))
-> Hash Join
Hash Cond: (t2_2.b = t1_2.a)
-> Seq Scan on prt2_p2 t2_2
@@ -621,7 +621,7 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM prt1 t1, prt2 t2, prt1_e t
-> Index Scan using iprt1_e_p2_ab2 on prt1_e_p2 t3_2
Index Cond: (((a + b) / 2) = t2_2.b)
-> Nested Loop
- Join Filter: (((t3_3.a + t3_3.b) / 2) = t1_3.a)
+ Join Filter: (t1_3.a = ((t3_3.a + t3_3.b) / 2))
-> Hash Join
Hash Cond: (t2_3.b = t1_3.a)
-> Seq Scan on prt2_p3 t2_3
@@ -926,7 +926,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t1.b FROM prt2 t1, prt1_e t2 WHER
Sort Key: t1.a
-> Append
-> Nested Loop
- Join Filter: (t1_5.b = t1_2.a)
+ Join Filter: (t1_2.a = t1_5.b)
-> HashAggregate
Group Key: t1_5.b
-> Hash Join
@@ -939,7 +939,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t1.b FROM prt2 t1, prt1_e t2 WHER
Index Cond: (a = ((t2_1.a + t2_1.b) / 2))
Filter: (b = 0)
-> Nested Loop
- Join Filter: (t1_6.b = t1_3.a)
+ Join Filter: (t1_3.a = t1_6.b)
-> HashAggregate
Group Key: t1_6.b
-> Hash Join
@@ -952,7 +952,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t1.b FROM prt2 t1, prt1_e t2 WHER
Index Cond: (a = ((t2_2.a + t2_2.b) / 2))
Filter: (b = 0)
-> Nested Loop
- Join Filter: (t1_7.b = t1_4.a)
+ Join Filter: (t1_4.a = t1_7.b)
-> HashAggregate
Group Key: t1_7.b
-> Nested Loop
commit 353011daeaa17698a326adccdae3fa521313b8df
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Wed Nov 16 15:56:04 2022 -0500
Fix flatten_join_alias_vars() to handle varnullingrels correctly.
The remaining core regression test failures occur because
flatten_join_alias_vars() isn't doing the right thing. The
alias Var it needs to replace may have acquired varnullingrels
bits signifying the effect of upper outer joins, and if so we
must preserve that information in the replacement expression.
The simplest way to do that is to wrap the replacement expression
in a PlaceHolderVar, and that's what we have to do in the general
case where subquery pullup has mutated the replacement joinaliasvars
entry into an arbitrary expression. But in simpler cases, such as
where the joinaliasvars entry is just a Var, we'd prefer to do it
by merging the alias Var's varnullingrels into the replacement Var.
In that way the flattened alias will compare equal() to semantically
equivalent references that didn't use the alias name.
Moreover, the parser also uses this code while checking certain
semantic constraints, and in that context we *must not* generate
PlaceHolderVars. PHVs shouldn't appear in parse-time expressions,
and adding one would certainly cause the parser to decide the
query is invalid (because the result wouldn't compare equal() to
what it needs to). Fortunately, during parsing the set of possible
contents of a joinaliasvars entry is quite constrained, so we can
guarantee to apply the nullingrels info to the Vars therein.
The result of this step passes all core regression tests, but there
are still loose ends for FDWs (so that contrib/postgres_fdw will fail).
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index d52c2a3595..dc089306ae 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -901,7 +901,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
*/
if (rte->lateral && root->hasJoinRTEs)
rte->subquery = (Query *)
- flatten_join_alias_vars(root->parse,
+ flatten_join_alias_vars(root, root->parse,
(Node *) rte->subquery);
}
else if (rte->rtekind == RTE_FUNCTION)
@@ -1102,7 +1102,7 @@ preprocess_expression(PlannerInfo *root, Node *expr, int kind)
kind == EXPRKIND_VALUES ||
kind == EXPRKIND_TABLESAMPLE ||
kind == EXPRKIND_TABLEFUNC))
- expr = flatten_join_alias_vars(root->parse, expr);
+ expr = flatten_join_alias_vars(root, root->parse, expr);
/*
* Simplify constant expressions. For function RTEs, this was already
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index fc56a81be8..db01edf459 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -1078,7 +1078,8 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* maybe even in the rewriter; but for now let's just fix this case here.)
*/
subquery->targetList = (List *)
- flatten_join_alias_vars(subroot->parse, (Node *) subquery->targetList);
+ flatten_join_alias_vars(subroot, subroot->parse,
+ (Node *) subquery->targetList);
/*
* Adjust level-0 varnos in subquery so that we can append its rangetable
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
index 8d8c9136f8..69c2019553 100644
--- a/src/backend/optimizer/util/var.c
+++ b/src/backend/optimizer/util/var.c
@@ -62,6 +62,7 @@ typedef struct
typedef struct
{
+ PlannerInfo *root; /* could be NULL! */
Query *query; /* outer Query */
int sublevels_up;
bool possible_sublink; /* could aliases include a SubLink? */
@@ -80,6 +81,10 @@ static bool pull_var_clause_walker(Node *node,
pull_var_clause_context *context);
static Node *flatten_join_alias_vars_mutator(Node *node,
flatten_join_alias_vars_context *context);
+static Node *add_nullingrels_if_needed(PlannerInfo *root, Node *newnode,
+ Var *oldvar);
+static bool is_standard_join_alias_expression(Node *newnode, Var *oldvar);
+static void adjust_standard_join_alias_expression(Node *newnode, Var *oldvar);
static Relids alias_relid_set(Query *query, Relids relids);
@@ -722,26 +727,42 @@ pull_var_clause_walker(Node *node, pull_var_clause_context *context)
* is the only way that the executor can directly handle whole-row Vars.
*
* This also adjusts relid sets found in some expression node types to
- * substitute the contained base rels for any join relid.
+ * substitute the contained base+OJ rels for any join relid.
*
* If a JOIN contains sub-selects that have been flattened, its join alias
* entries might now be arbitrary expressions, not just Vars. This affects
- * this function in one important way: we might find ourselves inserting
- * SubLink expressions into subqueries, and we must make sure that their
- * Query.hasSubLinks fields get set to true if so. If there are any
+ * this function in two important ways. First, we might find ourselves
+ * inserting SubLink expressions into subqueries, and we must make sure that
+ * their Query.hasSubLinks fields get set to true if so. If there are any
* SubLinks in the join alias lists, the outer Query should already have
* hasSubLinks = true, so this is only relevant to un-flattened subqueries.
+ * Second, we have to preserve any varnullingrels info attached to the
+ * alias Vars we're replacing. If the replacement expression is a Var or
+ * PlaceHolderVar or constructed from those, we can just add the
+ * varnullingrels bits to the existing nullingrels field(s); otherwise
+ * we have to add a PlaceHolderVar wrapper.
*
- * NOTE: this is used on not-yet-planned expressions. We do not expect it
- * to be applied directly to the whole Query, so if we see a Query to start
- * with, we do want to increment sublevels_up (this occurs for LATERAL
- * subqueries).
+ * NOTE: this is also used by the parser, to expand join alias Vars before
+ * checking GROUP BY validity. For that use-case, root will be NULL, which
+ * is why we have to pass the Query separately. We need the root itself only
+ * for making PlaceHolderVars. We can avoid making PlaceHolderVars in the
+ * parser's usage because it won't be dealing with arbitrary expressions:
+ * so long as adjust_standard_join_alias_expression can handle everything
+ * the parser would make as a join alias expression, we're OK.
*/
Node *
-flatten_join_alias_vars(Query *query, Node *node)
+flatten_join_alias_vars(PlannerInfo *root, Query *query, Node *node)
{
flatten_join_alias_vars_context context;
+ /*
+ * We do not expect this to be applied to the whole Query, only to
+ * expressions or LATERAL subqueries. Hence, if the top node is a Query,
+ * it's okay to immediately increment sublevels_up.
+ */
+ Assert(node != (Node *) query);
+
+ context.root = root;
context.query = query;
context.sublevels_up = 0;
/* flag whether join aliases could possibly contain SubLinks */
@@ -812,7 +833,9 @@ flatten_join_alias_vars_mutator(Node *node,
rowexpr->colnames = colnames;
rowexpr->location = var->location;
- return (Node *) rowexpr;
+ /* Lastly, add any varnullingrels to the replacement expression */
+ return add_nullingrels_if_needed(context->root, (Node *) rowexpr,
+ var);
}
/* Expand join alias reference */
@@ -839,7 +862,8 @@ flatten_join_alias_vars_mutator(Node *node,
if (context->possible_sublink && !context->inserted_sublink)
context->inserted_sublink = checkExprHasSubLink(newvar);
- return newvar;
+ /* Lastly, add any varnullingrels to the replacement expression */
+ return add_nullingrels_if_needed(context->root, newvar, var);
}
if (IsA(node, PlaceHolderVar))
{
@@ -854,6 +878,7 @@ flatten_join_alias_vars_mutator(Node *node,
{
phv->phrels = alias_relid_set(context->query,
phv->phrels);
+ /* we *don't* change phnullingrels */
}
return (Node *) phv;
}
@@ -887,9 +912,145 @@ flatten_join_alias_vars_mutator(Node *node,
(void *) context);
}
+/*
+ * Add oldvar's varnullingrels, if any, to a flattened join alias expression.
+ * The newnode has been copied, so we can modify it freely.
+ */
+static Node *
+add_nullingrels_if_needed(PlannerInfo *root, Node *newnode, Var *oldvar)
+{
+ if (oldvar->varnullingrels == NULL)
+ return newnode; /* nothing to do */
+ /* If possible, do it by adding to existing nullingrel fields */
+ if (is_standard_join_alias_expression(newnode, oldvar))
+ adjust_standard_join_alias_expression(newnode, oldvar);
+ else if (root)
+ {
+ /* We can insert a PlaceHolderVar to carry the nullingrels */
+ PlaceHolderVar *newphv;
+ Relids phrels = pull_varnos(root, newnode);
+
+ /* XXX what if phrels is empty? */
+ Assert(!bms_is_empty(phrels)); /* probably wrong */
+ newphv = make_placeholder_expr(root, (Expr *) newnode, phrels);
+ /* newphv has zero phlevelsup and NULL phnullingrels; fix it */
+ newphv->phlevelsup = oldvar->varlevelsup;
+ newphv->phnullingrels = bms_copy(oldvar->varnullingrels);
+ newnode = (Node *) newphv;
+ }
+ else
+ {
+ /* ooops, we're missing support for something the parser can make */
+ elog(ERROR, "unsupported join alias expression");
+ }
+ return newnode;
+}
+
+/*
+ * Check to see if we can insert nullingrels into this join alias expression
+ * without use of a separate PlaceHolderVar.
+ *
+ * This will handle Vars, PlaceHolderVars, and implicit-coercion and COALESCE
+ * expressions built from those. This coverage needs to handle anything
+ * that the parser would put into joinaliasvars.
+ * XXX it's probably incomplete at the moment.
+ */
+static bool
+is_standard_join_alias_expression(Node *newnode, Var *oldvar)
+{
+ if (newnode == NULL)
+ return false;
+ if (IsA(newnode, Var) &&
+ ((Var *) newnode)->varlevelsup == oldvar->varlevelsup)
+ return true;
+ else if (IsA(newnode, PlaceHolderVar) &&
+ ((PlaceHolderVar *) newnode)->phlevelsup == oldvar->varlevelsup)
+ return true;
+ else if (IsA(newnode, FuncExpr))
+ {
+ FuncExpr *fexpr = (FuncExpr *) newnode;
+
+ /*
+ * We need to assume that the function wouldn't produce non-NULL from
+ * NULL, which is reasonable for implicit coercions but otherwise not
+ * so much. (Looking at its strictness is likely overkill, and anyway
+ * it would cause us to fail if someone forgot to mark an implicit
+ * coercion as strict.)
+ */
+ if (fexpr->funcformat != COERCE_IMPLICIT_CAST ||
+ fexpr->args == NIL)
+ return false;
+
+ /*
+ * Examine only the first argument --- coercions might have additional
+ * arguments that are constants.
+ */
+ return is_standard_join_alias_expression(linitial(fexpr->args), oldvar);
+ }
+ else if (IsA(newnode, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) newnode;
+ ListCell *lc;
+
+ Assert(cexpr->args != NIL);
+ foreach(lc, cexpr->args)
+ {
+ if (!is_standard_join_alias_expression(lfirst(lc), oldvar))
+ return false;
+ }
+ return true;
+ }
+ else
+ return false;
+}
+
+/*
+ * Insert nullingrels into an expression accepted by
+ * is_standard_join_alias_expression.
+ */
+static void
+adjust_standard_join_alias_expression(Node *newnode, Var *oldvar)
+{
+ if (IsA(newnode, Var) &&
+ ((Var *) newnode)->varlevelsup == oldvar->varlevelsup)
+ {
+ Var *newvar = (Var *) newnode;
+
+ newvar->varnullingrels = bms_add_members(newvar->varnullingrels,
+ oldvar->varnullingrels);
+ }
+ else if (IsA(newnode, PlaceHolderVar) &&
+ ((PlaceHolderVar *) newnode)->phlevelsup == oldvar->varlevelsup)
+ {
+ PlaceHolderVar *newphv = (PlaceHolderVar *) newnode;
+
+ newphv->phnullingrels = bms_add_members(newphv->phnullingrels,
+ oldvar->varnullingrels);
+ }
+ else if (IsA(newnode, FuncExpr))
+ {
+ FuncExpr *fexpr = (FuncExpr *) newnode;
+
+ adjust_standard_join_alias_expression(linitial(fexpr->args), oldvar);
+ }
+ else if (IsA(newnode, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) newnode;
+ ListCell *lc;
+
+ Assert(cexpr->args != NIL);
+ foreach(lc, cexpr->args)
+ {
+ adjust_standard_join_alias_expression(lfirst(lc), oldvar);
+ }
+ }
+ else
+ Assert(false);
+}
+
/*
* alias_relid_set: in a set of RT indexes, replace joins by their
- * underlying base relids
+ * underlying base+OJ relids
*/
static Relids
alias_relid_set(Query *query, Relids relids)
diff --git a/src/backend/parser/parse_agg.c b/src/backend/parser/parse_agg.c
index 3ef9e8ee5e..c15fab0f68 100644
--- a/src/backend/parser/parse_agg.c
+++ b/src/backend/parser/parse_agg.c
@@ -1162,7 +1162,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
* entries are RTE_JOIN kind.
*/
if (hasJoinRTEs)
- groupClauses = (List *) flatten_join_alias_vars(qry,
+ groupClauses = (List *) flatten_join_alias_vars(NULL, qry,
(Node *) groupClauses);
/*
@@ -1206,7 +1206,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
groupClauses, hasJoinRTEs,
have_non_var_grouping);
if (hasJoinRTEs)
- clause = flatten_join_alias_vars(qry, clause);
+ clause = flatten_join_alias_vars(NULL, qry, clause);
check_ungrouped_columns(clause, pstate, qry,
groupClauses, groupClauseCommonVars,
have_non_var_grouping,
@@ -1217,7 +1217,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
groupClauses, hasJoinRTEs,
have_non_var_grouping);
if (hasJoinRTEs)
- clause = flatten_join_alias_vars(qry, clause);
+ clause = flatten_join_alias_vars(NULL, qry, clause);
check_ungrouped_columns(clause, pstate, qry,
groupClauses, groupClauseCommonVars,
have_non_var_grouping,
@@ -1546,7 +1546,7 @@ finalize_grouping_exprs_walker(Node *node,
Index ref = 0;
if (context->hasJoinRTEs)
- expr = flatten_join_alias_vars(context->qry, expr);
+ expr = flatten_join_alias_vars(NULL, context->qry, expr);
/*
* Each expression must match a grouping entry at the current
diff --git a/src/include/optimizer/optimizer.h b/src/include/optimizer/optimizer.h
index 409005bae9..95f3461a3d 100644
--- a/src/include/optimizer/optimizer.h
+++ b/src/include/optimizer/optimizer.h
@@ -197,6 +197,6 @@ extern bool contain_var_clause(Node *node);
extern bool contain_vars_of_level(Node *node, int levelsup);
extern int locate_var_of_level(Node *node, int levelsup);
extern List *pull_var_clause(Node *node, int flags);
-extern Node *flatten_join_alias_vars(Query *query, Node *node);
+extern Node *flatten_join_alias_vars(PlannerInfo *root, Query *query, Node *node);
#endif /* OPTIMIZER_H */
commit 377bc2b5563c092dcd20a3364147ad22f56024e5
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Wed Nov 16 16:06:29 2022 -0500
Teach FDWs about base-plus-outer-join relids.
Conversion of the planner to include OJ relids in join relids
affects FDWs that want to plan foreign joins. They *must* follow
suit when labeling foreign joins in order to match with the core
planner, but for many purposes (if postgres_fdw is any guide)
they'd prefer to consider only base relations within the join.
To support both requirements, redefine ForeignScan.fs_relids as
base+OJ relids, and add a new field fs_base_relids that's set up
by the core planner.
Another way we could do this is to keep fs_relids as just base
relids and make the new field be the one with OJ relids added.
While that would be more backwards-compatible in some sense,
it would be inconsistent with the naming used in the core planner,
and I think that it might allow some types of bugs to escape
quick detection.
postgres_fdw also has one place where it needs to ignore varnullingrels
while matching Vars. It's not clear whether it's worth trying to
improve that. (This too is probably only an issue for FDWs that do
join planning, since Vars seen in a base relation scan should never
have any varnullingrels.)
As of this step, this patch series passes check-world.
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
index 9524765650..94dd7b2c96 100644
--- a/contrib/postgres_fdw/deparse.c
+++ b/contrib/postgres_fdw/deparse.c
@@ -3950,7 +3950,17 @@ get_relation_column_alias_ids(Var *node, RelOptInfo *foreignrel,
i = 1;
foreach(lc, foreignrel->reltarget->exprs)
{
- if (equal(lfirst(lc), (Node *) node))
+ Var *tlvar = (Var *) lfirst(lc);
+
+ /*
+ * Match reltarget entries only on varno/varattno. Ideally there
+ * would be some cross-check on varnullingrels, but it's unclear what
+ * to do exactly; we don't have enough context to know what that value
+ * should be.
+ */
+ if (IsA(tlvar, Var) &&
+ tlvar->varno == node->varno &&
+ tlvar->varattno == node->varattno)
{
*colno = i;
return;
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 8d7500abfb..39cc37053c 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -1511,13 +1511,13 @@ postgresBeginForeignScan(ForeignScanState *node, int eflags)
/*
* Identify which user to do the remote access as. This should match what
* ExecCheckRTEPerms() does. In case of a join or aggregate, use the
- * lowest-numbered member RTE as a representative; we would get the same
- * result from any.
+ * lowest-numbered member base RTE as a representative; we would get the
+ * same result from any.
*/
if (fsplan->scan.scanrelid > 0)
rtindex = fsplan->scan.scanrelid;
else
- rtindex = bms_next_member(fsplan->fs_relids, -1);
+ rtindex = bms_next_member(fsplan->fs_base_relids, -1);
rte = exec_rt_fetch(rtindex, estate);
userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
@@ -2414,7 +2414,7 @@ find_modifytable_subplan(PlannerInfo *root,
{
ForeignScan *fscan = (ForeignScan *) subplan;
- if (bms_is_member(rtindex, fscan->fs_relids))
+ if (bms_is_member(rtindex, fscan->fs_base_relids))
return fscan;
}
@@ -2840,8 +2840,8 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
* that setrefs.c won't update the string when flattening the
* rangetable. To find out what rtoffset was applied, identify the
* minimum RT index appearing in the string and compare it to the
- * minimum member of plan->fs_relids. (We expect all the relids in
- * the join will have been offset by the same amount; the Asserts
+ * minimum member of plan->fs_base_relids. (We expect all the relids
+ * in the join will have been offset by the same amount; the Asserts
* below should catch it if that ever changes.)
*/
minrti = INT_MAX;
@@ -2858,7 +2858,7 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
else
ptr++;
}
- rtoffset = bms_next_member(plan->fs_relids, -1) - minrti;
+ rtoffset = bms_next_member(plan->fs_base_relids, -1) - minrti;
/* Now we can translate the string */
relations = makeStringInfo();
@@ -2873,7 +2873,7 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
char *refname;
rti += rtoffset;
- Assert(bms_is_member(rti, plan->fs_relids));
+ Assert(bms_is_member(rti, plan->fs_base_relids));
rte = rt_fetch(rti, es->rtable);
Assert(rte->rtekind == RTE_RELATION);
/* This logic should agree with explain.c's ExplainTargetRel */
diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml
index 94263c628f..ac1717bc3c 100644
--- a/doc/src/sgml/fdwhandler.sgml
+++ b/doc/src/sgml/fdwhandler.sgml
@@ -351,6 +351,17 @@ GetForeignJoinPaths(PlannerInfo *root,
it will supply at run time in the tuples it returns.
</para>
+ <note>
+ <para>
+ Beginning with <productname>PostgreSQL</productname> 16,
+ <structfield>fs_relids</structfield> includes the rangetable indexes
+ of outer joins, if any were involved in this join. The new field
+ <structfield>fs_base_relids</structfield> includes only base
+ relation indexes, and thus
+ mimics <structfield>fs_relids</structfield>'s old semantics.
+ </para>
+ </note>
+
<para>
See <xref linkend="fdw-planning"/> for additional information.
</para>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index f86983c660..ed9a118416 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1114,7 +1114,7 @@ ExplainPreScanNode(PlanState *planstate, Bitmapset **rels_used)
break;
case T_ForeignScan:
*rels_used = bms_add_members(*rels_used,
- ((ForeignScan *) plan)->fs_relids);
+ ((ForeignScan *) plan)->fs_base_relids);
break;
case T_CustomScan:
*rels_used = bms_add_members(*rels_used,
diff --git a/src/backend/executor/execScan.c b/src/backend/executor/execScan.c
index 043bb83f55..2b37266b6a 100644
--- a/src/backend/executor/execScan.c
+++ b/src/backend/executor/execScan.c
@@ -325,7 +325,7 @@ ExecScanReScan(ScanState *node)
* all of them.
*/
if (IsA(node->ps.plan, ForeignScan))
- relids = ((ForeignScan *) node->ps.plan)->fs_relids;
+ relids = ((ForeignScan *) node->ps.plan)->fs_base_relids;
else if (IsA(node->ps.plan, CustomScan))
relids = ((CustomScan *) node->ps.plan)->custom_relids;
else
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index ac86ce9003..13f46e4f23 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -4153,14 +4153,22 @@ create_foreignscan_plan(PlannerInfo *root, ForeignPath *best_path,
/*
* Likewise, copy the relids that are represented by this foreign scan. An
- * upper rel doesn't have relids set, but it covers all the base relations
- * participating in the underlying scan, so use root's all_baserels.
+ * upper rel doesn't have relids set, but it covers all the relations
+ * participating in the underlying scan/join, so use root->all_query_rels.
*/
if (rel->reloptkind == RELOPT_UPPER_REL)
- scan_plan->fs_relids = root->all_baserels;
+ scan_plan->fs_relids = root->all_query_rels;
else
scan_plan->fs_relids = best_path->path.parent->relids;
+ /*
+ * Join relid sets include relevant outer joins, but FDWs may need to know
+ * which are the included base rels. That's a bit tedious to get without
+ * access to the plan-time data structures, so compute it here.
+ */
+ scan_plan->fs_base_relids = bms_difference(scan_plan->fs_relids,
+ root->outer_join_rels);
+
/*
* If this is a foreign join, and to make it valid to push down we had to
* assume that the current user is the same as some user explicitly named
@@ -5800,8 +5808,9 @@ make_foreignscan(List *qptlist,
node->fdw_private = fdw_private;
node->fdw_scan_tlist = fdw_scan_tlist;
node->fdw_recheck_quals = fdw_recheck_quals;
- /* fs_relids will be filled in by create_foreignscan_plan */
+ /* fs_relids, fs_base_relids will be filled by create_foreignscan_plan */
node->fs_relids = NULL;
+ node->fs_base_relids = NULL;
/* fsSystemCol will be filled in by create_foreignscan_plan */
node->fsSystemCol = false;
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 8fff731756..6ed6b950a4 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1560,6 +1560,7 @@ set_foreignscan_references(PlannerInfo *root,
}
fscan->fs_relids = offset_relid_set(fscan->fs_relids, rtoffset);
+ fscan->fs_base_relids = offset_relid_set(fscan->fs_base_relids, rtoffset);
/* Adjust resultRelation if it's valid */
if (fscan->resultRelation > 0)
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 5c2ab1b379..25bc3e61eb 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -689,6 +689,7 @@ typedef struct WorkTableScan
* When the plan node represents a foreign join, scan.scanrelid is zero and
* fs_relids must be consulted to identify the join relation. (fs_relids
* is valid for simple scans as well, but will always match scan.scanrelid.)
+ * fs_relids includes outer joins; fs_base_relids does not.
*
* If the FDW's PlanDirectModify() callback decides to repurpose a ForeignScan
* node to perform the UPDATE or DELETE operation directly in the remote
@@ -708,7 +709,8 @@ typedef struct ForeignScan
List *fdw_private; /* private data for FDW */
List *fdw_scan_tlist; /* optional tlist describing scan tuple */
List *fdw_recheck_quals; /* original quals not in scan.plan.qual */
- Bitmapset *fs_relids; /* RTIs generated by this scan */
+ Bitmapset *fs_relids; /* base+OJ RTIs generated by this scan */
+ Bitmapset *fs_base_relids; /* base RTIs generated by this scan */
bool fsSystemCol; /* true if any "system column" is needed */
} ForeignScan;
commit 240921fd65457cf96b9826b983950faf80f60564
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Wed Nov 16 16:16:26 2022 -0500
Don't use RestrictInfo.nullable_relids in join_clause_is_movable_to.
Instead of using per-clause nullable_relids data, compute a
per-baserel set of outer joins that can null each relation, and
check for overlap between that and clause_relids to detect whether
the clause can safely be pushed down to relation scan level.
join_clause_is_movable_into also uses nullable_relids, but it
turns out that that test can just be dropped entirely. Now that
clause_relids includes nulling outer joins, the preceding tests
in the function are sufficient to reject clauses that can't be
pushed down.
This might seem like a net loss given that we have to add a bit
of code to initsplan.c to compute RelOptInfo.nulling_relids.
However, that's not much code at all, and the payoff is this:
we no longer need RestrictInfo.nullable_relids at all.
The next patch, which removes that field and the extensive
infrastructure that maintains it, saves way more code and cycles
than we add here. Also, I think there are likely going to be
other uses for RelOptInfo.nulling_relids.
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index a128780857..794f5fd197 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -57,6 +57,8 @@ static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
static void process_security_barrier_quals(PlannerInfo *root,
int rti, Relids qualscope,
bool below_outer_join);
+static void mark_rels_nulled_by_join(PlannerInfo *root, Index ojrelid,
+ Relids lower_rels);
static SpecialJoinInfo *make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
@@ -961,6 +963,7 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
*qualscope = bms_add_member(*qualscope, j->rtindex);
root->outer_join_rels = bms_add_member(root->outer_join_rels,
j->rtindex);
+ mark_rels_nulled_by_join(root, j->rtindex, rightids);
}
*inner_join_rels = bms_union(left_inners, right_inners);
nonnullable_rels = leftids;
@@ -1003,6 +1006,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
*qualscope = bms_add_member(*qualscope, j->rtindex);
root->outer_join_rels = bms_add_member(root->outer_join_rels,
j->rtindex);
+ mark_rels_nulled_by_join(root, j->rtindex, leftids);
+ mark_rels_nulled_by_join(root, j->rtindex, rightids);
*inner_join_rels = bms_union(left_inners, right_inners);
/* each side is both outer and inner */
nonnullable_rels = *qualscope;
@@ -1219,6 +1224,33 @@ process_security_barrier_quals(PlannerInfo *root,
Assert(security_level <= root->qual_security_level);
}
+/*
+ * mark_rels_nulled_by_join
+ * Fill RelOptInfo.nulling_relids of baserels nulled by this outer join
+ *
+ * Inputs:
+ * ojrelid: RT index of the join RTE (must not be 0)
+ * lower_rels: the base+OJ Relids syntactically below nullable side of join
+ */
+static void
+mark_rels_nulled_by_join(PlannerInfo *root, Index ojrelid,
+ Relids lower_rels)
+{
+ int relid = -1;
+
+ while ((relid = bms_next_member(lower_rels, relid)) > 0)
+ {
+ RelOptInfo *rel = root->simple_rel_array[relid];
+
+ if (rel == NULL) /* must be an outer join */
+ {
+ Assert(bms_is_member(relid, root->outer_join_rels));
+ continue;
+ }
+ rel->nulling_relids = bms_add_member(rel->nulling_relids, ojrelid);
+ }
+}
+
/*
* make_outerjoininfo
* Build a SpecialJoinInfo for the current outer join
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 38540e6331..984b923c1b 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -264,6 +264,12 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
rel->top_parent = parent->top_parent ? parent->top_parent : parent;
rel->top_parent_relids = rel->top_parent->relids;
+ /*
+ * A child rel is below the same outer joins as its parent. (We
+ * presume this info was already calculated for the parent.)
+ */
+ rel->nulling_relids = parent->nulling_relids;
+
/*
* Also propagate lateral-reference information from appendrel parent
* rels to their child rels. We intentionally give each child rel the
@@ -287,6 +293,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
rel->parent = NULL;
rel->top_parent = NULL;
rel->top_parent_relids = NULL;
+ rel->nulling_relids = NULL;
rel->direct_lateral_relids = NULL;
rel->lateral_relids = NULL;
rel->lateral_referencers = NULL;
@@ -667,6 +674,7 @@ build_join_rel(PlannerInfo *root,
joinrel->max_attr = 0;
joinrel->attr_needed = NULL;
joinrel->attr_widths = NULL;
+ joinrel->nulling_relids = NULL;
joinrel->lateral_vars = NIL;
joinrel->lateral_referencers = NULL;
joinrel->indexlist = NIL;
@@ -856,6 +864,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->max_attr = 0;
joinrel->attr_needed = NULL;
joinrel->attr_widths = NULL;
+ joinrel->nulling_relids = NULL;
joinrel->lateral_vars = NIL;
joinrel->lateral_referencers = NULL;
joinrel->indexlist = NIL;
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index bcbee8f943..15f410cf36 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -618,8 +618,17 @@ join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
if (bms_is_member(baserel->relid, rinfo->outer_relids))
return false;
- /* Target rel must not be nullable below the clause */
- if (bms_is_member(baserel->relid, rinfo->nullable_relids))
+ /*
+ * Target rel's Vars must not be nulled by any outer join. We can check
+ * this without groveling through the individual Vars by seeing whether
+ * clause_relids (which includes all such Vars' varnullingrels) includes
+ * any outer join that can null the target rel. You might object that
+ * this could reject the clause on the basis of an OJ relid that came from
+ * some other rel's Var. However, that would still mean that the clause
+ * came from above that outer join and shouldn't be pushed down; so there
+ * should be no false positives.
+ */
+ if (bms_overlap(rinfo->clause_relids, baserel->nulling_relids))
return false;
/* Clause must not use any rels with LATERAL references to this rel */
@@ -651,16 +660,17 @@ join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
* relation plus the outer rels. We also check that it does reference at
* least one current Var, ensuring that the clause will be pushed down to
* a unique place in a parameterized join tree. And we check that we're
- * not pushing the clause into its outer-join outer side, nor down into
- * a lower outer join's inner side.
- *
- * The check about pushing a clause down into a lower outer join's inner side
- * is only approximate; it sometimes returns "false" when actually it would
- * be safe to use the clause here because we're still above the outer join
- * in question. This is okay as long as the answers at different join levels
- * are consistent: it just means we might sometimes fail to push a clause as
- * far down as it could safely be pushed. It's unclear whether it would be
- * worthwhile to do this more precisely. (But if it's ever fixed to be
+ * not pushing the clause into its outer-join outer side.
+ *
+ * We used to need to check that we're not pushing the clause into a lower
+ * outer join's inner side. However, now that clause_relids includes
+ * references to potentially-nulling outer joins, the other tests handle that
+ * concern. If the clause references any Var coming from the inside of a
+ * lower outer join, its clause_relids will mention that outer join, causing
+ * the evaluability check to fail; while if it references no such Vars, the
+ * references-a-target-rel check will fail.
+ *
+ * XXX not clear if we can do this yet: (But if it's ever fixed to be
* exactly accurate, there's an Assert in get_joinrel_parampathinfo() that
* should be re-enabled.)
*
@@ -704,14 +714,5 @@ join_clause_is_movable_into(RestrictInfo *rinfo,
if (bms_overlap(currentrelids, rinfo->outer_relids))
return false;
- /*
- * Target rel(s) must not be nullable below the clause. This is
- * approximate, in the safe direction, because the current join might be
- * above the join where the nulling would happen, in which case the clause
- * would work correctly here. But we don't have enough info to be sure.
- */
- if (bms_overlap(currentrelids, rinfo->nullable_relids))
- return false;
-
return true;
}
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index c8538bbd67..70c37118c1 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -660,6 +660,7 @@ typedef struct PartitionSchemeData *PartitionScheme;
* outer-join relids.
* attr_widths - cache space for per-attribute width estimates;
* zero means not computed yet
+ * nulling_relids - relids of outer joins that can null this rel
* lateral_vars - lateral cross-references of rel, if any (list of
* Vars and PlaceHolderVars)
* lateral_referencers - relids of rels that reference this one laterally
@@ -893,6 +894,8 @@ typedef struct RelOptInfo
Relids *attr_needed pg_node_attr(read_write_ignore);
/* array indexed [min_attr .. max_attr] */
int32 *attr_widths pg_node_attr(read_write_ignore);
+ /* relids of outer joins that can null this baserel */
+ Relids nulling_relids;
/* LATERAL Vars and PHVs referenced by rel */
List *lateral_vars;
/* rels that reference this baserel laterally */
commit d3e9a64ebc89bd2fa9005386e5a7380884d3c512
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Wed Nov 16 16:21:54 2022 -0500
Remove RestrictInfo.nullable_relids and associated infrastructure.
There is no more code using this field, only code computing it,
so just delete all that. We can likewise get rid of
EquivalenceMember.em_nullable_relids and
PlannerInfo.nullable_baserels.
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 39cc37053c..10aa27a78a 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -6313,7 +6313,6 @@ foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
false,
root->qual_security_level,
grouped_rel->relids,
- NULL,
NULL);
if (is_foreign_expr(root, grouped_rel, expr))
fpinfo->remote_conds = lappend(fpinfo->remote_conds, rinfo);
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 5902c80747..8a6a40f672 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -2733,7 +2733,6 @@ set_function_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
if (var)
pathkeys = build_expression_pathkey(root,
(Expr *) var,
- NULL, /* below outer joins */
Int8LessOperator,
rel->relids,
false);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index d4f8b7893d..0737cc355f 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -34,7 +34,7 @@
static EquivalenceMember *add_eq_member(EquivalenceClass *ec,
- Expr *expr, Relids relids, Relids nullable_relids,
+ Expr *expr, Relids relids,
EquivalenceMember *parent,
Oid datatype);
static bool is_exprlist_member(Expr *node, List *exprs);
@@ -131,9 +131,7 @@ process_equivalence(PlannerInfo *root,
Expr *item1;
Expr *item2;
Relids item1_relids,
- item2_relids,
- item1_nullable_relids,
- item2_nullable_relids;
+ item2_relids;
List *opfamilies;
EquivalenceClass *ec1,
*ec2;
@@ -206,8 +204,7 @@ process_equivalence(PlannerInfo *root,
restrictinfo->pseudoconstant,
restrictinfo->security_level,
NULL,
- restrictinfo->outer_relids,
- restrictinfo->nullable_relids);
+ restrictinfo->outer_relids);
}
return false;
}
@@ -225,12 +222,6 @@ process_equivalence(PlannerInfo *root,
return false; /* RHS is non-strict but not constant */
}
- /* Calculate nullable-relid sets for each side of the clause */
- item1_nullable_relids = bms_intersect(item1_relids,
- restrictinfo->nullable_relids);
- item2_nullable_relids = bms_intersect(item2_relids,
- restrictinfo->nullable_relids);
-
/*
* We use the declared input types of the operator, not exprType() of the
* inputs, as the nominal datatypes for opfamily lookup. This presumes
@@ -400,7 +391,7 @@ process_equivalence(PlannerInfo *root,
else if (ec1)
{
/* Case 3: add item2 to ec1 */
- em2 = add_eq_member(ec1, item2, item2_relids, item2_nullable_relids,
+ em2 = add_eq_member(ec1, item2, item2_relids,
NULL, item2_type);
ec1->ec_sources = lappend(ec1->ec_sources, restrictinfo);
ec1->ec_below_outer_join |= below_outer_join;
@@ -418,7 +409,7 @@ process_equivalence(PlannerInfo *root,
else if (ec2)
{
/* Case 3: add item1 to ec2 */
- em1 = add_eq_member(ec2, item1, item1_relids, item1_nullable_relids,
+ em1 = add_eq_member(ec2, item1, item1_relids,
NULL, item1_type);
ec2->ec_sources = lappend(ec2->ec_sources, restrictinfo);
ec2->ec_below_outer_join |= below_outer_join;
@@ -452,9 +443,9 @@ process_equivalence(PlannerInfo *root,
ec->ec_min_security = restrictinfo->security_level;
ec->ec_max_security = restrictinfo->security_level;
ec->ec_merged = NULL;
- em1 = add_eq_member(ec, item1, item1_relids, item1_nullable_relids,
+ em1 = add_eq_member(ec, item1, item1_relids,
NULL, item1_type);
- em2 = add_eq_member(ec, item2, item2_relids, item2_nullable_relids,
+ em2 = add_eq_member(ec, item2, item2_relids,
NULL, item2_type);
root->eq_classes = lappend(root->eq_classes, ec);
@@ -545,13 +536,12 @@ canonicalize_ec_expression(Expr *expr, Oid req_type, Oid req_collation)
*/
static EquivalenceMember *
add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
- Relids nullable_relids, EquivalenceMember *parent, Oid datatype)
+ EquivalenceMember *parent, Oid datatype)
{
EquivalenceMember *em = makeNode(EquivalenceMember);
em->em_expr = expr;
em->em_relids = relids;
- em->em_nullable_relids = nullable_relids;
em->em_is_const = false;
em->em_is_child = (parent != NULL);
em->em_datatype = datatype;
@@ -588,13 +578,6 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
* equivalence class it is a member of; if none, optionally build a new
* single-member EquivalenceClass for it.
*
- * expr is the expression, and nullable_relids is the set of base relids
- * that are potentially nullable below it. We actually only care about
- * the set of such relids that are used in the expression; but for caller
- * convenience, we perform that intersection step here. The caller need
- * only be sure that nullable_relids doesn't omit any nullable rels that
- * might appear in the expr.
- *
* sortref is the SortGroupRef of the originating SortGroupClause, if any,
* or zero if not. (It should never be zero if the expression is volatile!)
*
@@ -623,7 +606,6 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
EquivalenceClass *
get_eclass_for_sort_expr(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
List *opfamilies,
Oid opcintype,
Oid collation,
@@ -719,13 +701,12 @@ get_eclass_for_sort_expr(PlannerInfo *root,
elog(ERROR, "volatile EquivalenceClass has no sortref");
/*
- * Get the precise set of nullable relids appearing in the expression.
+ * Get the precise set of relids appearing in the expression.
*/
expr_relids = pull_varnos(root, (Node *) expr);
- nullable_relids = bms_intersect(nullable_relids, expr_relids);
newem = add_eq_member(newec, copyObject(expr), expr_relids,
- nullable_relids, NULL, opcintype);
+ NULL, opcintype);
/*
* add_eq_member doesn't check for volatile functions, set-returning
@@ -1211,8 +1192,6 @@ generate_base_implied_equalities_const(PlannerInfo *root,
rinfo = process_implied_equality(root, eq_op, ec->ec_collation,
cur_em->em_expr, const_em->em_expr,
bms_copy(ec->ec_relids),
- bms_union(cur_em->em_nullable_relids,
- const_em->em_nullable_relids),
ec->ec_min_security,
ec->ec_below_outer_join,
cur_em->em_is_const);
@@ -1285,8 +1264,6 @@ generate_base_implied_equalities_no_const(PlannerInfo *root,
rinfo = process_implied_equality(root, eq_op, ec->ec_collation,
prev_em->em_expr, cur_em->em_expr,
bms_copy(ec->ec_relids),
- bms_union(prev_em->em_nullable_relids,
- cur_em->em_nullable_relids),
ec->ec_min_security,
ec->ec_below_outer_join,
false);
@@ -1889,8 +1866,6 @@ create_join_clause(PlannerInfo *root,
rightem->em_expr,
bms_union(leftem->em_relids,
rightem->em_relids),
- bms_union(leftem->em_nullable_relids,
- rightem->em_nullable_relids),
ec->ec_min_security);
/* If it's a child clause, copy the parent's rinfo_serial */
@@ -2105,8 +2080,7 @@ reconsider_outer_join_clause(PlannerInfo *root, RestrictInfo *rinfo,
left_type,
right_type,
inner_datatype;
- Relids inner_relids,
- inner_nullable_relids;
+ Relids inner_relids;
ListCell *lc1;
Assert(is_opclause(rinfo->clause));
@@ -2133,8 +2107,6 @@ reconsider_outer_join_clause(PlannerInfo *root, RestrictInfo *rinfo,
inner_datatype = left_type;
inner_relids = rinfo->left_relids;
}
- inner_nullable_relids = bms_intersect(inner_relids,
- rinfo->nullable_relids);
/* Scan EquivalenceClasses for a match to outervar */
foreach(lc1, root->eq_classes)
@@ -2195,7 +2167,6 @@ reconsider_outer_join_clause(PlannerInfo *root, RestrictInfo *rinfo,
innervar,
cur_em->em_expr,
bms_copy(inner_relids),
- bms_copy(inner_nullable_relids),
cur_ec->ec_min_security);
if (process_equivalence(root, &newrinfo, true))
match = true;
@@ -2233,9 +2204,7 @@ reconsider_full_join_clause(PlannerInfo *root, FullJoinClauseInfo *fjinfo)
left_type,
right_type;
Relids left_relids,
- right_relids,
- left_nullable_relids,
- right_nullable_relids;
+ right_relids;
ListCell *lc1;
/* Can't use an outerjoin_delayed clause here */
@@ -2251,10 +2220,6 @@ reconsider_full_join_clause(PlannerInfo *root, FullJoinClauseInfo *fjinfo)
rightvar = (Expr *) get_rightop(rinfo->clause);
left_relids = rinfo->left_relids;
right_relids = rinfo->right_relids;
- left_nullable_relids = bms_intersect(left_relids,
- rinfo->nullable_relids);
- right_nullable_relids = bms_intersect(right_relids,
- rinfo->nullable_relids);
foreach(lc1, root->eq_classes)
{
@@ -2356,7 +2321,6 @@ reconsider_full_join_clause(PlannerInfo *root, FullJoinClauseInfo *fjinfo)
leftvar,
cur_em->em_expr,
bms_copy(left_relids),
- bms_copy(left_nullable_relids),
cur_ec->ec_min_security);
if (process_equivalence(root, &newrinfo, true))
matchleft = true;
@@ -2372,7 +2336,6 @@ reconsider_full_join_clause(PlannerInfo *root, FullJoinClauseInfo *fjinfo)
rightvar,
cur_em->em_expr,
bms_copy(right_relids),
- bms_copy(right_nullable_relids),
cur_ec->ec_min_security);
if (process_equivalence(root, &newrinfo, true))
matchright = true;
@@ -2662,7 +2625,6 @@ add_child_rel_equivalences(PlannerInfo *root,
/* Yes, generate transformed child version */
Expr *child_expr;
Relids new_relids;
- Relids new_nullable_relids;
if (parent_rel->reloptkind == RELOPT_BASEREL)
{
@@ -2692,21 +2654,7 @@ add_child_rel_equivalences(PlannerInfo *root,
top_parent_relids);
new_relids = bms_add_members(new_relids, child_relids);
- /*
- * And likewise for nullable_relids. Note this code assumes
- * parent and child relids are singletons.
- */
- new_nullable_relids = cur_em->em_nullable_relids;
- if (bms_overlap(new_nullable_relids, top_parent_relids))
- {
- new_nullable_relids = bms_difference(new_nullable_relids,
- top_parent_relids);
- new_nullable_relids = bms_add_members(new_nullable_relids,
- child_relids);
- }
-
- (void) add_eq_member(cur_ec, child_expr,
- new_relids, new_nullable_relids,
+ (void) add_eq_member(cur_ec, child_expr, new_relids,
cur_em, cur_em->em_datatype);
/* Record this EC index for the child rel */
@@ -2803,7 +2751,6 @@ add_child_join_rel_equivalences(PlannerInfo *root,
/* Yes, generate transformed child version */
Expr *child_expr;
Relids new_relids;
- Relids new_nullable_relids;
if (parent_joinrel->reloptkind == RELOPT_JOINREL)
{
@@ -2834,20 +2781,7 @@ add_child_join_rel_equivalences(PlannerInfo *root,
top_parent_relids);
new_relids = bms_add_members(new_relids, child_relids);
- /*
- * For nullable_relids, we must selectively replace parent
- * nullable relids with child ones.
- */
- new_nullable_relids = cur_em->em_nullable_relids;
- if (bms_overlap(new_nullable_relids, top_parent_relids))
- new_nullable_relids =
- adjust_child_relids_multilevel(root,
- new_nullable_relids,
- child_joinrel,
- child_joinrel->top_parent);
-
- (void) add_eq_member(cur_ec, child_expr,
- new_relids, new_nullable_relids,
+ (void) add_eq_member(cur_ec, child_expr, new_relids,
cur_em, cur_em->em_datatype);
}
}
diff --git a/src/backend/optimizer/path/pathkeys.c b/src/backend/optimizer/path/pathkeys.c
index a9943cd6e0..bf919ca97f 100644
--- a/src/backend/optimizer/path/pathkeys.c
+++ b/src/backend/optimizer/path/pathkeys.c
@@ -180,9 +180,6 @@ pathkey_is_redundant(PathKey *new_pathkey, List *pathkeys)
* Given an expression and sort-order information, create a PathKey.
* The result is always a "canonical" PathKey, but it might be redundant.
*
- * expr is the expression, and nullable_relids is the set of base relids
- * that are potentially nullable below it.
- *
* If the PathKey is being generated from a SortGroupClause, sortref should be
* the SortGroupClause's SortGroupRef; otherwise zero.
*
@@ -198,7 +195,6 @@ pathkey_is_redundant(PathKey *new_pathkey, List *pathkeys)
static PathKey *
make_pathkey_from_sortinfo(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
Oid opfamily,
Oid opcintype,
Oid collation,
@@ -234,7 +230,7 @@ make_pathkey_from_sortinfo(PlannerInfo *root,
equality_op);
/* Now find or (optionally) create a matching EquivalenceClass */
- eclass = get_eclass_for_sort_expr(root, expr, nullable_relids,
+ eclass = get_eclass_for_sort_expr(root, expr,
opfamilies, opcintype, collation,
sortref, rel, create_it);
@@ -257,7 +253,6 @@ make_pathkey_from_sortinfo(PlannerInfo *root,
static PathKey *
make_pathkey_from_sortop(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
Oid ordering_op,
bool nulls_first,
Index sortref,
@@ -279,7 +274,6 @@ make_pathkey_from_sortop(PlannerInfo *root,
return make_pathkey_from_sortinfo(root,
expr,
- nullable_relids,
opfamily,
opcintype,
collation,
@@ -584,12 +578,10 @@ build_index_pathkeys(PlannerInfo *root,
}
/*
- * OK, try to make a canonical pathkey for this sort key. Note we're
- * underneath any outer joins, so nullable_relids should be NULL.
+ * OK, try to make a canonical pathkey for this sort key.
*/
cpathkey = make_pathkey_from_sortinfo(root,
indexkey,
- NULL,
index->sortopfamily[i],
index->opcintype[i],
index->indexcollations[i],
@@ -743,14 +735,12 @@ build_partition_pathkeys(PlannerInfo *root, RelOptInfo *partrel,
/*
* Try to make a canonical pathkey for this partkey.
*
- * We're considering a baserel scan, so nullable_relids should be
- * NULL. Also, we assume the PartitionDesc lists any NULL partition
- * last, so we treat the scan like a NULLS LAST index: we have
- * nulls_first for backwards scan only.
+ * We assume the PartitionDesc lists any NULL partition last, so we
+ * treat the scan like a NULLS LAST index: we have nulls_first for
+ * backwards scan only.
*/
cpathkey = make_pathkey_from_sortinfo(root,
keyCol,
- NULL,
partscheme->partopfamily[i],
partscheme->partopcintype[i],
partscheme->partcollation[i],
@@ -799,7 +789,7 @@ build_partition_pathkeys(PlannerInfo *root, RelOptInfo *partrel,
* Build a pathkeys list that describes an ordering by a single expression
* using the given sort operator.
*
- * expr, nullable_relids, and rel are as for make_pathkey_from_sortinfo.
+ * expr and rel are as for make_pathkey_from_sortinfo.
* We induce the other arguments assuming default sort order for the operator.
*
* Similarly to make_pathkey_from_sortinfo, the result is NIL if create_it
@@ -808,7 +798,6 @@ build_partition_pathkeys(PlannerInfo *root, RelOptInfo *partrel,
List *
build_expression_pathkey(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
Oid opno,
Relids rel,
bool create_it)
@@ -827,7 +816,6 @@ build_expression_pathkey(PlannerInfo *root,
cpathkey = make_pathkey_from_sortinfo(root,
expr,
- nullable_relids,
opfamily,
opcintype,
exprCollation((Node *) expr),
@@ -908,14 +896,11 @@ convert_subquery_pathkeys(PlannerInfo *root, RelOptInfo *rel,
* expression is *not* volatile in the outer query: it's just
* a Var referencing whatever the subquery emitted. (IOW, the
* outer query isn't going to re-execute the volatile
- * expression itself.) So this is okay. Likewise, it's
- * correct to pass nullable_relids = NULL, because we're
- * underneath any outer joins appearing in the outer query.
+ * expression itself.) So this is okay.
*/
outer_ec =
get_eclass_for_sort_expr(root,
(Expr *) outer_var,
- NULL,
sub_eclass->ec_opfamilies,
sub_member->em_datatype,
sub_eclass->ec_collation,
@@ -997,7 +982,6 @@ convert_subquery_pathkeys(PlannerInfo *root, RelOptInfo *rel,
/* See if we have a matching EC for the TLE */
outer_ec = get_eclass_for_sort_expr(root,
(Expr *) outer_var,
- NULL,
sub_eclass->ec_opfamilies,
sub_expr_type,
sub_expr_coll,
@@ -1138,13 +1122,6 @@ build_join_pathkeys(PlannerInfo *root,
* The resulting PathKeys are always in canonical form. (Actually, there
* is no longer any code anywhere that creates non-canonical PathKeys.)
*
- * We assume that root->nullable_baserels is the set of base relids that could
- * have gone to NULL below the SortGroupClause expressions. This is okay if
- * the expressions came from the query's top level (ORDER BY, DISTINCT, etc)
- * and if this function is only invoked after deconstruct_jointree. In the
- * future we might have to make callers pass in the appropriate
- * nullable-relids set, but for now it seems unnecessary.
- *
* 'sortclauses' is a list of SortGroupClause nodes
* 'tlist' is the targetlist to find the referenced tlist entries in
*/
@@ -1166,7 +1143,6 @@ make_pathkeys_for_sortclauses(PlannerInfo *root,
Assert(OidIsValid(sortcl->sortop));
pathkey = make_pathkey_from_sortop(root,
sortkey,
- root->nullable_baserels,
sortcl->sortop,
sortcl->nulls_first,
sortcl->tleSortGroupRef,
@@ -1222,7 +1198,6 @@ initialize_mergeclause_eclasses(PlannerInfo *root, RestrictInfo *restrictinfo)
restrictinfo->left_ec =
get_eclass_for_sort_expr(root,
(Expr *) get_leftop(clause),
- restrictinfo->nullable_relids,
restrictinfo->mergeopfamilies,
lefttype,
((OpExpr *) clause)->inputcollid,
@@ -1232,7 +1207,6 @@ initialize_mergeclause_eclasses(PlannerInfo *root, RestrictInfo *restrictinfo)
restrictinfo->right_ec =
get_eclass_for_sort_expr(root,
(Expr *) get_rightop(clause),
- restrictinfo->nullable_relids,
restrictinfo->mergeopfamilies,
righttype,
((OpExpr *) clause)->inputcollid,
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 794f5fd197..c3ffc25670 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -92,7 +92,7 @@ static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool is_clone,
List **postponed_qual_list);
static bool check_outerjoin_delay(PlannerInfo *root, Relids *relids_p,
- Relids *nullable_relids_p, bool is_pushed_down);
+ bool is_pushed_down);
static bool check_equivalence_delay(PlannerInfo *root,
RestrictInfo *restrictinfo);
static bool check_redundant_nullability_qual(PlannerInfo *root, Node *clause);
@@ -748,9 +748,8 @@ deconstruct_jointree(PlannerInfo *root)
Assert(root->parse->jointree != NULL &&
IsA(root->parse->jointree, FromExpr));
- /* These are filled as we scan the jointree */
+ /* This is filled as we scan the jointree */
root->outer_join_rels = NULL;
- root->nullable_baserels = NULL;
result = deconstruct_recurse(root, (Node *) root->parse->jointree, false,
&qualscope, &inner_join_rels,
@@ -907,7 +906,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
left_inners,
right_inners,
nonnullable_rels,
- nullable_rels,
ojscope;
List *leftjoinlist,
*rightjoinlist;
@@ -943,8 +941,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
*inner_join_rels = *qualscope;
/* Inner join adds no restrictions for quals */
nonnullable_rels = NULL;
- /* and it doesn't force anything to null, either */
- nullable_rels = NULL;
break;
case JOIN_LEFT:
case JOIN_ANTI:
@@ -967,7 +963,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
}
*inner_join_rels = bms_union(left_inners, right_inners);
nonnullable_rels = leftids;
- nullable_rels = rightids;
break;
case JOIN_SEMI:
leftjoinlist = deconstruct_recurse(root, j->larg,
@@ -984,13 +979,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
*inner_join_rels = bms_union(left_inners, right_inners);
/* Semi join adds no restrictions for quals */
nonnullable_rels = NULL;
-
- /*
- * Theoretically, a semijoin would null the RHS; but since the
- * RHS can't be accessed above the join, this is immaterial
- * and we needn't account for it.
- */
- nullable_rels = NULL;
break;
case JOIN_FULL:
leftjoinlist = deconstruct_recurse(root, j->larg,
@@ -1011,22 +999,16 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
*inner_join_rels = bms_union(left_inners, right_inners);
/* each side is both outer and inner */
nonnullable_rels = *qualscope;
- nullable_rels = *qualscope;
break;
default:
/* JOIN_RIGHT was eliminated during reduce_outer_joins() */
elog(ERROR, "unrecognized join type: %d",
(int) j->jointype);
nonnullable_rels = NULL; /* keep compiler quiet */
- nullable_rels = NULL;
leftjoinlist = rightjoinlist = NIL;
break;
}
- /* Report all rels that will be nulled anywhere in the jointree */
- root->nullable_baserels = bms_add_members(root->nullable_baserels,
- nullable_rels);
-
/*
* Try to process any quals postponed by children. If they need
* further postponement, add them to my output postponed_qual_list.
@@ -2103,7 +2085,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool pseudoconstant = false;
bool maybe_equivalence;
bool maybe_outer_join;
- Relids nullable_relids;
RestrictInfo *restrictinfo;
/*
@@ -2257,7 +2238,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
/* Check to see if must be delayed by lower outer join */
outerjoin_delayed = check_outerjoin_delay(root,
&relids,
- &nullable_relids,
false);
/*
@@ -2285,7 +2265,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
/* Check to see if must be delayed by lower outer join */
outerjoin_delayed = check_outerjoin_delay(root,
&relids,
- &nullable_relids,
true);
if (outerjoin_delayed)
@@ -2345,8 +2324,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
pseudoconstant,
security_level,
relids,
- outerjoin_nonnullable,
- nullable_relids);
+ outerjoin_nonnullable);
/* Apply appropriate clone marking, too */
restrictinfo->has_clone = has_clone;
@@ -2483,9 +2461,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* If the qual must be delayed, add relids to *relids_p to reflect the lowest
* safe level for evaluating the qual, and return true. Any extra delay for
* higher-level joins is reflected by setting delay_upper_joins to true in
- * SpecialJoinInfo structs. We also compute nullable_relids, the set of
- * referenced relids that are nullable by lower outer joins (note that this
- * can be nonempty even for a non-delayed qual).
+ * SpecialJoinInfo structs.
*
* For an is_pushed_down qual, we can evaluate the qual as soon as (1) we have
* all the rels it mentions, and (2) we are at or above any outer joins that
@@ -2508,8 +2484,8 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* mentioning only C cannot be applied below the join to A.
*
* For a non-pushed-down qual, this isn't going to determine where we place the
- * qual, but we need to determine outerjoin_delayed and nullable_relids anyway
- * for use later in the planning process.
+ * qual, but we need to determine outerjoin_delayed anyway for use later in
+ * the planning process.
*
* Lastly, a pushed-down qual that references the nullable side of any current
* join_info_list member and has to be evaluated above that OJ (because its
@@ -2527,24 +2503,18 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
static bool
check_outerjoin_delay(PlannerInfo *root,
Relids *relids_p, /* in/out parameter */
- Relids *nullable_relids_p, /* output parameter */
bool is_pushed_down)
{
Relids relids;
- Relids nullable_relids;
bool outerjoin_delayed;
bool found_some;
/* fast path if no special joins */
if (root->join_info_list == NIL)
- {
- *nullable_relids_p = NULL;
return false;
- }
/* must copy relids because we need the original value at the end */
relids = bms_copy(*relids_p);
- nullable_relids = NULL;
outerjoin_delayed = false;
do
{
@@ -2571,12 +2541,6 @@ check_outerjoin_delay(PlannerInfo *root,
/* we'll need another iteration */
found_some = true;
}
- /* track all the nullable rels of relevant OJs */
- nullable_relids = bms_add_members(nullable_relids,
- sjinfo->min_righthand);
- if (sjinfo->jointype == JOIN_FULL)
- nullable_relids = bms_add_members(nullable_relids,
- sjinfo->min_lefthand);
/* set delay_upper_joins if needed */
if (is_pushed_down && sjinfo->jointype != JOIN_FULL &&
bms_overlap(relids, sjinfo->min_lefthand))
@@ -2585,13 +2549,9 @@ check_outerjoin_delay(PlannerInfo *root,
}
} while (found_some);
- /* identify just the actually-referenced nullable rels */
- nullable_relids = bms_int_members(nullable_relids, *relids_p);
-
- /* replace *relids_p, and return nullable_relids */
+ /* replace *relids_p */
bms_free(*relids_p);
*relids_p = relids;
- *nullable_relids_p = nullable_relids;
return outerjoin_delayed;
}
@@ -2613,7 +2573,6 @@ check_equivalence_delay(PlannerInfo *root,
RestrictInfo *restrictinfo)
{
Relids relids;
- Relids nullable_relids;
/* fast path if no special joins */
if (root->join_info_list == NIL)
@@ -2622,12 +2581,12 @@ check_equivalence_delay(PlannerInfo *root,
/* must copy restrictinfo's relids to avoid changing it */
relids = bms_copy(restrictinfo->left_relids);
/* check left side does not need delay */
- if (check_outerjoin_delay(root, &relids, &nullable_relids, true))
+ if (check_outerjoin_delay(root, &relids, true))
return false;
/* and similarly for the right side */
relids = bms_copy(restrictinfo->right_relids);
- if (check_outerjoin_delay(root, &relids, &nullable_relids, true))
+ if (check_outerjoin_delay(root, &relids, true))
return false;
return true;
@@ -2753,11 +2712,6 @@ distribute_restrictinfo_to_rels(PlannerInfo *root,
* variable-free. Otherwise the qual is applied at the lowest join level
* that provides all its variables.
*
- * "nullable_relids" is the set of relids used in the expressions that are
- * potentially nullable below the expressions. (This has to be supplied by
- * caller because this function is used after deconstruct_jointree, so we
- * don't have knowledge of where the clause items came from.)
- *
* "security_level" is the security level to assign to the new restrictinfo.
*
* "both_const" indicates whether both items are known pseudo-constant;
@@ -2783,7 +2737,6 @@ process_implied_equality(PlannerInfo *root,
Expr *item1,
Expr *item2,
Relids qualscope,
- Relids nullable_relids,
Index security_level,
bool below_outer_join,
bool both_const)
@@ -2867,8 +2820,7 @@ process_implied_equality(PlannerInfo *root,
pseudoconstant,
security_level,
relids,
- NULL, /* outer_relids */
- nullable_relids);
+ NULL); /* outer_relids */
/*
* If it's a join clause, add vars used in the clause to targetlists of
@@ -2933,7 +2885,6 @@ build_implied_join_equality(PlannerInfo *root,
Expr *item1,
Expr *item2,
Relids qualscope,
- Relids nullable_relids,
Index security_level)
{
RestrictInfo *restrictinfo;
@@ -2961,8 +2912,7 @@ build_implied_join_equality(PlannerInfo *root,
false, /* pseudoconstant */
security_level, /* security_level */
qualscope, /* required_relids */
- NULL, /* outer_relids */
- nullable_relids); /* nullable_relids */
+ NULL); /* outer_relids */
/* Set mergejoinability/hashjoinability flags */
check_mergejoinable(restrictinfo);
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index e18d64b6dc..662d2c1f17 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -448,9 +448,6 @@ adjust_appendrel_attrs_mutator(Node *node,
newinfo->outer_relids = adjust_child_relids(oldinfo->outer_relids,
context->nappinfos,
context->appinfos);
- newinfo->nullable_relids = adjust_child_relids(oldinfo->nullable_relids,
- context->nappinfos,
- context->appinfos);
newinfo->left_relids = adjust_child_relids(oldinfo->left_relids,
context->nappinfos,
context->appinfos);
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 3d270e91d6..eb20583f75 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -815,7 +815,7 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
rinfo->outerjoin_delayed,
pseudoconstant,
rinfo->security_level,
- NULL, NULL, NULL));
+ NULL, NULL));
/* track minimum security level among child quals */
cq_min_security = Min(cq_min_security, rinfo->security_level);
}
@@ -850,7 +850,7 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
make_restrictinfo(root, qual,
true, false, false,
security_level,
- NULL, NULL, NULL));
+ NULL, NULL));
cq_min_security = Min(cq_min_security, security_level);
}
security_level++;
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index 9cfde2f790..336a73d3b9 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -275,7 +275,6 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
false,
join_or_rinfo->security_level,
NULL,
- NULL,
NULL);
/*
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index 15f410cf36..1d8912608b 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -29,8 +29,7 @@ static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids);
+ Relids outer_relids);
static Expr *make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
bool is_pushed_down,
@@ -38,8 +37,7 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids);
+ Relids outer_relids);
/*
@@ -49,7 +47,7 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
*
* The is_pushed_down, outerjoin_delayed, and pseudoconstant flags for the
* RestrictInfo must be supplied by the caller, as well as the correct values
- * for security_level, outer_relids, and nullable_relids.
+ * for security_level and outer_relids.
* required_relids can be NULL, in which case it defaults to the actual clause
* contents (i.e., clause_relids).
*
@@ -69,8 +67,7 @@ make_restrictinfo(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids)
+ Relids outer_relids)
{
/*
* If it's an OR clause, build a modified copy with RestrictInfos inserted
@@ -84,8 +81,7 @@ make_restrictinfo(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids);
+ outer_relids);
/* Shouldn't be an AND clause, else AND/OR flattening messed up */
Assert(!is_andclause(clause));
@@ -98,8 +94,7 @@ make_restrictinfo(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids);
+ outer_relids);
}
/*
@@ -116,8 +111,7 @@ make_restrictinfo_internal(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids)
+ Relids outer_relids)
{
RestrictInfo *restrictinfo = makeNode(RestrictInfo);
Relids baserels;
@@ -132,7 +126,6 @@ make_restrictinfo_internal(PlannerInfo *root,
restrictinfo->can_join = false; /* may get set below */
restrictinfo->security_level = security_level;
restrictinfo->outer_relids = outer_relids;
- restrictinfo->nullable_relids = nullable_relids;
/*
* If it's potentially delayable by lower-level security quals, figure out
@@ -260,7 +253,7 @@ make_restrictinfo_internal(PlannerInfo *root,
*
* The same is_pushed_down, outerjoin_delayed, and pseudoconstant flag
* values can be applied to all RestrictInfo nodes in the result. Likewise
- * for security_level, outer_relids, and nullable_relids.
+ * for security_level and outer_relids.
*
* The given required_relids are attached to our top-level output,
* but any OR-clause constituents are allowed to default to just the
@@ -274,8 +267,7 @@ make_sub_restrictinfos(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids)
+ Relids outer_relids)
{
if (is_orclause(clause))
{
@@ -291,8 +283,7 @@ make_sub_restrictinfos(PlannerInfo *root,
pseudoconstant,
security_level,
NULL,
- outer_relids,
- nullable_relids));
+ outer_relids));
return (Expr *) make_restrictinfo_internal(root,
clause,
make_orclause(orlist),
@@ -301,8 +292,7 @@ make_sub_restrictinfos(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids);
+ outer_relids);
}
else if (is_andclause(clause))
{
@@ -318,8 +308,7 @@ make_sub_restrictinfos(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids));
+ outer_relids));
return make_andclause(andlist);
}
else
@@ -331,8 +320,7 @@ make_sub_restrictinfos(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids);
+ outer_relids);
}
/*
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 70c37118c1..c93543bf0e 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -263,14 +263,6 @@ struct PlannerInfo
*/
Relids all_query_rels;
- /*
- * nullable_baserels is a Relids set of base relids that are nullable by
- * some outer join in the jointree; these are rels that are potentially
- * nullable below the WHERE clause, SELECT targetlist, etc. This is
- * computed in deconstruct_jointree.
- */
- Relids nullable_baserels;
-
/*
* join_rel_list is a list of all join-relation RelOptInfos we have
* considered in this planning run. For small problems we just scan the
@@ -1356,7 +1348,6 @@ typedef struct EquivalenceMember
Expr *em_expr; /* the expression represented */
Relids em_relids; /* all relids appearing in em_expr */
- Relids em_nullable_relids; /* nullable by lower outer joins */
bool em_is_const; /* expression is pseudoconstant? */
bool em_is_child; /* derived version for a child relation? */
Oid em_datatype; /* the "nominal type" used by the opfamily */
@@ -2382,9 +2373,7 @@ typedef struct LimitPath
* in parameterized scans, since pushing it into the join's outer side would
* lead to wrong answers.)
*
- * There is also a nullable_relids field, which is the set of rels the clause
- * references that can be forced null by some outer join below the clause.
- *
+ * XXX this comment needs work, if we don't remove it completely:
* outerjoin_delayed = true is subtly different from nullable_relids != NULL:
* a clause might reference some nullable rels and yet not be
* outerjoin_delayed because it also references all the other rels of the
@@ -2498,9 +2487,6 @@ typedef struct RestrictInfo
/* If an outer-join clause, the outer-side relations, else NULL: */
Relids outer_relids;
- /* The relids used in the clause that are nullable by lower outer joins: */
- Relids nullable_relids;
-
/*
* Relids in the left/right side of the clause. These fields are set for
* any binary opclause.
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 41f765d342..03866de136 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -128,7 +128,6 @@ extern Expr *canonicalize_ec_expression(Expr *expr,
extern void reconsider_outer_join_clauses(PlannerInfo *root);
extern EquivalenceClass *get_eclass_for_sort_expr(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
List *opfamilies,
Oid opcintype,
Oid collation,
@@ -216,7 +215,7 @@ extern List *build_index_pathkeys(PlannerInfo *root, IndexOptInfo *index,
extern List *build_partition_pathkeys(PlannerInfo *root, RelOptInfo *partrel,
ScanDirection scandir, bool *partialkeys);
extern List *build_expression_pathkey(PlannerInfo *root, Expr *expr,
- Relids nullable_relids, Oid opno,
+ Oid opno,
Relids rel, bool create_it);
extern List *convert_subquery_pathkeys(PlannerInfo *root, RelOptInfo *rel,
List *subquery_pathkeys,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index 9dffdcfd1e..57b963c0f7 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -83,7 +83,6 @@ extern RestrictInfo *process_implied_equality(PlannerInfo *root,
Expr *item1,
Expr *item2,
Relids qualscope,
- Relids nullable_relids,
Index security_level,
bool below_outer_join,
bool both_const);
@@ -93,7 +92,6 @@ extern RestrictInfo *build_implied_join_equality(PlannerInfo *root,
Expr *item1,
Expr *item2,
Relids qualscope,
- Relids nullable_relids,
Index security_level);
extern void match_foreign_keys_to_quals(PlannerInfo *root);
diff --git a/src/include/optimizer/restrictinfo.h b/src/include/optimizer/restrictinfo.h
index 17d3b4ab05..1f092371ea 100644
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -19,7 +19,7 @@
/* Convenience macro for the common case of a valid-everywhere qual */
#define make_simple_restrictinfo(root, clause) \
- make_restrictinfo(root, clause, true, false, false, 0, NULL, NULL, NULL)
+ make_restrictinfo(root, clause, true, false, false, 0, NULL, NULL)
extern RestrictInfo *make_restrictinfo(PlannerInfo *root,
Expr *clause,
@@ -28,8 +28,7 @@ extern RestrictInfo *make_restrictinfo(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids);
+ Relids outer_relids);
extern RestrictInfo *commute_restrictinfo(RestrictInfo *rinfo, Oid comm_op);
extern bool restriction_is_or_clause(RestrictInfo *restrictinfo);
extern bool restriction_is_securely_promotable(RestrictInfo *restrictinfo,
commit 8524431d51054ca85bd99367c86dff08a4c67e7f
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Wed Nov 16 16:33:15 2022 -0500
Use constant TRUE for "dummy" clauses when throwing back outer joins.
This improves on a hack I introduced in commit 6a6522529. If we
have a left-join clause l.x = r.y, and a WHERE clause l.x = constant,
we generate r.y = constant and then don't really have a need for the
join clause. Currently we throw the join clause back anyway after
marking it redundant, so that the join search heuristics won't think
this is a clauseless join and avoid it. That was a kluge introduced
under time pressure, and after looking at it I thought of a better
way: let's just introduce constant-TRUE "join clauses" instead,
and get rid of them at the end.
This improves the generated plans for such cases by not having to
test a redundant join clause. We can also get rid of the ugly hack
used to mark such clauses as redundant for selectivity estimation.
The code added here should go away again, once we handle these cases
more like real eclasses. But it seemed worth committing separately
so as to get the regression changes and selectivity simplifications
in place.
diff --git a/src/backend/optimizer/path/clausesel.c b/src/backend/optimizer/path/clausesel.c
index c08eb2b1c5..1cf565ee59 100644
--- a/src/backend/optimizer/path/clausesel.c
+++ b/src/backend/optimizer/path/clausesel.c
@@ -715,12 +715,6 @@ clause_selectivity_ext(PlannerInfo *root,
return (Selectivity) 1.0;
}
- /*
- * If the clause is marked redundant, always return 1.0.
- */
- if (rinfo->norm_selec > 1)
- return (Selectivity) 1.0;
-
/*
* If possible, cache the result of the selectivity calculation for
* the clause. We can cache if varRelid is zero or the clause
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 0737cc355f..a9f5db3ef6 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -1954,14 +1954,11 @@ create_join_clause(PlannerInfo *root,
* If we don't find any match for a set-aside outer join clause, we must
* throw it back into the regular joinclause processing by passing it to
* distribute_restrictinfo_to_rels(). If we do generate a derived clause,
- * however, the outer-join clause is redundant. We still throw it back,
- * because otherwise the join will be seen as a clauseless join and avoided
- * during join order searching; but we mark it as redundant to keep from
- * messing up the joinrel's size estimate. (This behavior means that the
- * API for this routine is uselessly complex: we could have just put all
- * the clauses into the regular processing initially. We keep it because
- * someday we might want to do something else, such as inserting "dummy"
- * joinclauses instead of real ones.)
+ * however, the outer-join clause is redundant. We must still put some
+ * clause into the regular processing, because otherwise the join will be
+ * seen as a clauseless join and avoided during join order searching.
+ * We handle this by generating a constant-TRUE clause that is marked with
+ * required_relids that make it a join between the correct relations.
*
* Outer join clauses that are marked outerjoin_delayed are special: this
* condition means that one or both VARs might go to null due to a lower
@@ -1994,10 +1991,15 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* remove it from the list */
root->left_join_clauses =
foreach_delete_current(root->left_join_clauses, cell);
- /* we throw it back anyway (see notes above) */
- /* but the thrown-back clause has no extra selectivity */
- rinfo->norm_selec = 2.0;
- rinfo->outer_selec = 1.0;
+ /* throw back a dummy replacement clause (see notes above) */
+ rinfo = make_restrictinfo(root,
+ (Expr *) makeBoolConst(true, false),
+ true, /* is_pushed_down */
+ false, /* outerjoin_delayed */
+ false, /* pseudoconstant */
+ 0, /* security_level */
+ rinfo->required_relids,
+ rinfo->outer_relids);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
@@ -2013,10 +2015,15 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* remove it from the list */
root->right_join_clauses =
foreach_delete_current(root->right_join_clauses, cell);
- /* we throw it back anyway (see notes above) */
- /* but the thrown-back clause has no extra selectivity */
- rinfo->norm_selec = 2.0;
- rinfo->outer_selec = 1.0;
+ /* throw back a dummy replacement clause (see notes above) */
+ rinfo = make_restrictinfo(root,
+ (Expr *) makeBoolConst(true, false),
+ true, /* is_pushed_down */
+ false, /* outerjoin_delayed */
+ false, /* pseudoconstant */
+ 0, /* security_level */
+ rinfo->required_relids,
+ rinfo->outer_relids);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
@@ -2034,10 +2041,15 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* remove it from the list */
root->full_join_clauses =
foreach_delete_current(root->full_join_clauses, cell);
- /* we throw it back anyway (see notes above) */
- /* but the thrown-back clause has no extra selectivity */
- rinfo->norm_selec = 2.0;
- rinfo->outer_selec = 1.0;
+ /* throw back a dummy replacement clause (see notes above) */
+ rinfo = make_restrictinfo(root,
+ (Expr *) makeBoolConst(true, false),
+ true, /* is_pushed_down */
+ false, /* outerjoin_delayed */
+ false, /* pseudoconstant */
+ 0, /* security_level */
+ rinfo->required_relids,
+ rinfo->outer_relids);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index 336a73d3b9..4b98692189 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -98,18 +98,13 @@ extract_restriction_or_clauses(PlannerInfo *root)
* joinclause that is considered safe to move to this rel by the
* parameterized-path machinery, even though what we are going to do
* with it is not exactly a parameterized path.
- *
- * However, it seems best to ignore clauses that have been marked
- * redundant (by setting norm_selec > 1). That likely can't happen
- * for OR clauses, but let's be safe.
*/
foreach(lc, rel->joininfo)
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
if (restriction_is_or_clause(rinfo) &&
- join_clause_is_movable_to(rinfo, rel) &&
- rinfo->norm_selec <= 1)
+ join_clause_is_movable_to(rinfo, rel))
{
/* Try to extract a qual for this rel only */
Expr *orclause = extract_or_clause(rinfo, rel);
@@ -356,7 +351,7 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
/* And hack cached selectivity so join size remains the same */
join_or_rinfo->norm_selec = orig_selec / or_selec;
- /* ensure result stays in sane range, in particular not "redundant" */
+ /* ensure result stays in sane range */
if (join_or_rinfo->norm_selec > 1)
join_or_rinfo->norm_selec = 1;
/* as explained above, we don't touch outer_selec */
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index 1d8912608b..c3af845acd 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -424,6 +424,21 @@ restriction_is_securely_promotable(RestrictInfo *restrictinfo,
return false;
}
+/*
+ * Detect whether a RestrictInfo's clause is constant TRUE (note that it's
+ * surely of type boolean). No such WHERE clause could survive qual
+ * canonicalization, but equivclass.c may generate such RestrictInfos for
+ * reasons discussed therein. We should drop them again when creating
+ * the finished plan, which is handled by the next few functions.
+ */
+static inline bool
+rinfo_is_constant_true(RestrictInfo *rinfo)
+{
+ return IsA(rinfo->clause, Const) &&
+ !((Const *) rinfo->clause)->constisnull &&
+ DatumGetBool(((Const *) rinfo->clause)->constvalue);
+}
+
/*
* get_actual_clauses
*
@@ -443,6 +458,7 @@ get_actual_clauses(List *restrictinfo_list)
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
Assert(!rinfo->pseudoconstant);
+ Assert(!rinfo_is_constant_true(rinfo));
result = lappend(result, rinfo->clause);
}
@@ -454,6 +470,7 @@ get_actual_clauses(List *restrictinfo_list)
*
* Extract bare clauses from 'restrictinfo_list', returning either the
* regular ones or the pseudoconstant ones per 'pseudoconstant'.
+ * Constant-TRUE clauses are dropped in any case.
*/
List *
extract_actual_clauses(List *restrictinfo_list,
@@ -466,7 +483,8 @@ extract_actual_clauses(List *restrictinfo_list,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (rinfo->pseudoconstant == pseudoconstant)
+ if (rinfo->pseudoconstant == pseudoconstant &&
+ !rinfo_is_constant_true(rinfo))
result = lappend(result, rinfo->clause);
}
return result;
@@ -477,7 +495,7 @@ extract_actual_clauses(List *restrictinfo_list,
*
* Extract bare clauses from 'restrictinfo_list', separating those that
* semantically match the join level from those that were pushed down.
- * Pseudoconstant clauses are excluded from the results.
+ * Pseudoconstant and constant-TRUE clauses are excluded from the results.
*
* This is only used at outer joins, since for plain joins we don't care
* about pushed-down-ness.
@@ -499,13 +517,15 @@ extract_actual_join_clauses(List *restrictinfo_list,
if (RINFO_IS_PUSHED_DOWN(rinfo, joinrelids))
{
- if (!rinfo->pseudoconstant)
+ if (!rinfo->pseudoconstant &&
+ !rinfo_is_constant_true(rinfo))
*otherquals = lappend(*otherquals, rinfo->clause);
}
else
{
/* joinquals shouldn't have been marked pseudoconstant */
Assert(!rinfo->pseudoconstant);
+ Assert(!rinfo_is_constant_true(rinfo));
*joinquals = lappend(*joinquals, rinfo->clause);
}
}
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index c93543bf0e..e26bf8e823 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2532,10 +2532,7 @@ typedef struct RestrictInfo
/* eval cost of clause; -1 if not yet set */
QualCost eval_cost pg_node_attr(equal_ignore);
- /*
- * selectivity for "normal" (JOIN_INNER) semantics; -1 if not yet set; >1
- * means a redundant clause
- */
+ /* selectivity for "normal" (JOIN_INNER) semantics; -1 if not yet set */
Selectivity norm_selec pg_node_attr(equal_ignore);
/* selectivity for outer join semantics; -1 if not yet set */
Selectivity outer_selec pg_node_attr(equal_ignore);
diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out
index 00f4c58238..17c423b763 100644
--- a/src/test/regress/expected/join.out
+++ b/src/test/regress/expected/join.out
@@ -4015,8 +4015,8 @@ explain (costs off)
select a.unique1, b.unique1, c.unique1, coalesce(b.twothousand, a.twothousand)
from tenk1 a left join tenk1 b on b.thousand = a.unique1 left join tenk1 c on c.unique2 =
coalesce(b.twothousand,a.twothousand)
where a.unique2 < 10 and coalesce(b.twothousand, a.twothousand) = 44;
- QUERY PLAN
----------------------------------------------------------------------------------------------
+ QUERY PLAN
+---------------------------------------------------------------
Nested Loop Left Join
-> Nested Loop Left Join
Filter: (COALESCE(b.twothousand, a.twothousand) = 44)
@@ -4027,7 +4027,7 @@ select a.unique1, b.unique1, c.unique1, coalesce(b.twothousand, a.twothousand)
-> Bitmap Index Scan on tenk1_thous_tenthous
Index Cond: (thousand = a.unique1)
-> Index Scan using tenk1_unique2 on tenk1 c
- Index Cond: ((unique2 = COALESCE(b.twothousand, a.twothousand)) AND (unique2 = 44))
+ Index Cond: (unique2 = 44)
(11 rows)
select a.unique1, b.unique1, c.unique1, coalesce(b.twothousand, a.twothousand)
@@ -4458,7 +4458,6 @@ where tt1.f1 = ss1.c0;
Output: tt4.f1
-> Nested Loop Left Join
Output: tt4.f1
- Join Filter: (tt3.f1 = tt4.f1)
-> Seq Scan on public.text_tbl tt3
Output: tt3.f1
Filter: (tt3.f1 = 'foo'::text)
@@ -4476,7 +4475,7 @@ where tt1.f1 = ss1.c0;
Output: (tt4.f1)
-> Seq Scan on public.text_tbl tt5
Output: tt4.f1
-(33 rows)
+(32 rows)
select 1 from
text_tbl as tt1
@@ -4583,24 +4582,22 @@ explain (costs off)
QUERY PLAN
-------------------------------------------------
Nested Loop Left Join
- Join Filter: (a.f1 = b.unique2)
-> Seq Scan on int4_tbl a
Filter: (f1 = 0)
-> Index Scan using tenk1_unique2 on tenk1 b
Index Cond: (unique2 = 0)
-(6 rows)
+(5 rows)
explain (costs off)
select * from tenk1 a full join tenk1 b using(unique2) where unique2 = 42;
QUERY PLAN
-------------------------------------------------
Merge Full Join
- Merge Cond: (a.unique2 = b.unique2)
-> Index Scan using tenk1_unique2 on tenk1 a
Index Cond: (unique2 = 42)
-> Index Scan using tenk1_unique2 on tenk1 b
Index Cond: (unique2 = 42)
-(6 rows)
+(5 rows)
--
-- test that quals attached to an outer join have correct semantics,
On Thu, Nov 17, 2022 at 4:46 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
So we've marked the 4 and 7 joins as possibly commuting, but they
cannot commute according to 7's min_lefthand set. I don't think
the extra clone condition is terribly harmful --- it's useless
but shouldn't cause any problems. However, if these joins should be
able to commute then the min_lefthand marking is preventing us
from considering legal join orders (and has been doing so all along,
that's not new in this patch). It looks to me like they should be
able to commute (giving your third form), so this is a pre-existing
planning deficiency.
Yeah. This is an issue that can also be seen on HEAD and is discussed
in [1]. It happens because when building SpecialJoinInfo for 7, we find
A/B join 5 is in our LHS, and our join condition (Pcd) uses 5's
syn_righthand while is not strict for 5's min_righthand. So we decide
to preserve the ordering of 7 and 5, by adding 5's full syntactic relset
to 7's min_lefthand. As discussed in [1], maybe we should consider 5's
min_righthand rather than syn_righthand when checking if Pcd uses 5's
RHS.
in [1]. It happens because when building SpecialJoinInfo for 7, we find
A/B join 5 is in our LHS, and our join condition (Pcd) uses 5's
syn_righthand while is not strict for 5's min_righthand. So we decide
to preserve the ordering of 7 and 5, by adding 5's full syntactic relset
to 7's min_lefthand. As discussed in [1], maybe we should consider 5's
min_righthand rather than syn_righthand when checking if Pcd uses 5's
RHS.
> Looking at the two forms again, it seems the expected two versions for
> Pcd should be
> Version 1: C Vars with nullingrels as {B/C}
> Version 2: C Vars with nullingrels as {B/C, A/B}
> With this we may have another problem that the two versions are both
> applicable at the C/D join according to clause_is_computable_at(), in
> both forms.
At least when I tried it just now, clause_is_computable_at correctly
rejected the first version, because we've already computed A/B when
we are trying to form the C/D join so we expect it to be listed in
varnullingrels.
For the first version of Pcd, which is C Vars with nullingrels as {B/C}
here, although at the C/D join level A/B join has been computed and
meanwhile is not listed in varnullingrels, but since Pcd does not
mention any nullable Vars in A/B's min_righthand, it seems to me this
version cannot be rejected by clause_is_computable_at(). But maybe I'm
missing something.
[1] https://www.postgresql.org/message-id/flat/CAMbWs4_8n5ANh_aX2PinRZ9V9mtBguhnRd4DOVt9msPgHmEMOQ%40mail.gmail.com
Thanks
Richard
here, although at the C/D join level A/B join has been computed and
meanwhile is not listed in varnullingrels, but since Pcd does not
mention any nullable Vars in A/B's min_righthand, it seems to me this
version cannot be rejected by clause_is_computable_at(). But maybe I'm
missing something.
[1] https://www.postgresql.org/message-id/flat/CAMbWs4_8n5ANh_aX2PinRZ9V9mtBguhnRd4DOVt9msPgHmEMOQ%40mail.gmail.com
Thanks
Richard
Here's a new edition of this patch series.
I shoved some preliminary refactoring into the 0001 patch,
notably splitting deconstruct_jointree into two passes.
0002-0009 cover the same ground as they did before, though
with some differences in detail. 0010-0012 are new work
mostly aimed at removing kluges we no longer need.
There are two big areas that I would still like to improve, but
I think we've run out of time to address them in the v16 cycle:
* It'd be nice to apply the regular EquivalenceClass deduction
mechanisms to outer-join equalities, instead of the
reconsider_outer_join_clauses kluge. I've made several stabs at that
without much success. I think that the "join domain" framework added
in 0012 is likely to provide a workable foundation, but some more
effort is needed.
* I really want to get rid of RestrictInfo.is_pushed_down and
RINFO_IS_PUSHED_DOWN(), because those seem exceedingly awkward
and squishy. I've not gotten far with finding a better
replacement there, either.
Despite the work being unfinished, I feel that this has moved us a
long way towards outer-join handling being less of a jury-rigged
affair.
regards, tom lane
commit 82c2bdb66e749deb8b764163ea14d9a988bc64ec
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu Dec 22 13:06:38 2022 -0500
Add overview documentation.
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index 41c120e0cd..191ad4c457 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -295,6 +295,239 @@ Therefore, we don't merge FROM-lists if the result would have too many
FROM-items in one list.
+Vars and PlaceHolderVars
+------------------------
+
+A Var node is simply the parse-tree representation of a table column
+reference. However, in the presence of outer joins, that concept is
+more subtle than it might seem. We need to distinguish the values of
+a Var "above" and "below" any outer join that could force the Var to
+null. As an example, consider
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE foo(t2.z)
+
+(Assume foo() is not strict, so that we can't reduce the left join to
+a plain join.) A naive implementation might try to push the foo(t2.z)
+call down to the scan of t2, but that is not correct because
+(a) what foo() should actually see for a null-extended join row is NULL,
+and (b) if foo() returns false, we should suppress the t1 row from the
+join altogether, not emit it with a null-extended t2 row. On the other
+hand, it *would* be correct (and desirable) to push that call down to
+the scan level if the query were
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y AND foo(t2.z))
+
+This motivates considering "t2.z" within the left join's ON clause
+to be a different value from "t2.z" outside the JOIN clause. The
+former can be identified with t2.z as seen at the relation scan level,
+but the latter can't.
+
+Another example occurs in connection with EquivalenceClasses (discussed
+below). Given
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE t1.x = 42
+
+we would like to use the EquivalenceClass mechanisms to derive "t2.y = 42"
+to use as a restriction clause for the scan of t2. (That works, because t2
+rows having y different from 42 cannot affect the query result.) However,
+it'd be wrong to conclude that t2.y will be equal to t1.x in every joined
+row. Part of the solution to this problem is to deem that "t2.y" in the
+ON clause refers to the relation-scan-level value of t2.y, but not to the
+value that y will have in joined rows, where it might be NULL rather than
+equal to t1.x.
+
+Therefore, Var nodes are decorated with "varnullingrels", which are sets
+of the rangetable indexes of outer joins that potentially null the Var
+at the point where it appears in the query. (Using a set, not an ordered
+list, is fine since it doesn't matter which join forced the value to null;
+and that avoids having to change the representation when we consider
+different outer-join orders.) In the examples above, all occurrences of
+t1.x would have empty varnullingrels, since the left join doesn't null t1.
+The t2 references within the JOIN ON clauses would also have empty
+varnullingrels. But outside the JOIN clauses, any Vars referencing t2
+would have varnullingrels containing the index of the JOIN's rangetable
+entry (RTE), so that they'd be understood as potentially different from
+the t2 values seen at scan level. Labeling t2.z in the WHERE clause with
+the JOIN's RT index lets us recognize that that occurrence of foo(t2.z)
+cannot be pushed down to the t2 scan level: we cannot evaluate that value
+at the scan level, but only after the join has been done.
+
+For LEFT and RIGHT outer joins, only Vars coming from the nullable side
+of the join are marked with that join's RT index. For FULL joins, Vars
+from both inputs are marked. (Such marking doesn't let us tell which
+side of the full join a Var came from; but that information can be found
+elsewhere at need.)
+
+Notionally, a Var having nonempty varnullingrels can be thought of as
+ CASE WHEN any-of-these-outer-joins-produced-a-null-extended-row
+ THEN NULL
+ ELSE the-scan-level-value-of-the-column
+ END
+It's only notional, because no such calculation is ever done explicitly.
+In a finished plan, Vars occurring in scan-level plan nodes represent
+the actual table column values, but upper-level Vars are always
+references to outputs of lower-level plan nodes. When a join node emits
+a null-extended row, it just returns nulls for the relevant output
+columns rather than copying up values from its input. Because we don't
+ever have to do this calculation explicitly, it's not necessary to
+distinguish which side of an outer join got null-extended, which'd
+otherwise be essential information for FULL JOIN cases.
+
+Outer join identity 3 (discussed above) complicates this picture
+a bit. In the form
+ A leftjoin (B leftjoin C on (Pbc)) on (Pab)
+all of the Vars in clauses Pbc and Pab will have empty varnullingrels,
+but if we start with
+ (A leftjoin B on (Pab)) leftjoin C on (Pbc)
+then the parser will have marked Pbc's B Vars with the A/B join's
+RT index, making this form artificially different from the first.
+For discussion's sake, let's denote this marking with a star:
+ (A leftjoin B on (Pab)) leftjoin C on (Pb*c)
+To cope with this, once we have detected that commuting these joins
+is legal, we generate both the Pbc and Pb*c forms of that ON clause,
+by either removing or adding the first join's RT index in the B Vars
+that the parser created. While generating paths for a plan step that
+joins B and C, we include as a relevant join qual only the form that
+is appropriate depending on whether A has already been joined to B.
+
+It's also worth noting that identity 3 makes "the left join's RT index"
+itself a bit of a fuzzy concept, since the syntactic scope of each join
+RTE will depend on which form was produced by the parser. We resolve
+this by considering that a left join's identity is determined by its
+minimum set of right-hand-side input relations. In both forms allowed
+by identity 3, we can identify the first join as having minimum RHS B
+and the second join as having minimum RHS C.
+
+Another thing to notice is that C Vars appearing outside the nested
+JOIN clauses will be marked as nulled by both left joins if the
+original parser input was in the first form of identity 3, but if the
+parser input was in the second form, such Vars will only be marked as
+nulled by the second join. This is not really a semantic problem:
+such Vars will be marked the same way throughout the upper part of the
+query, so they will all look equal() which is correct; and they will not
+look equal() to any C Var appearing in the JOIN ON clause or below these
+joins. However, when building Vars representing the outputs of join
+relations, we need to ensure that their varnullingrels are set to
+values consistent with the syntactic join order, so that they will
+appear equal() to pre-existing Vars in the upper part of the query.
+
+Outer joins also complicate handling of subquery pull-up. Consider
+
+ SELECT ..., ss.x FROM tab1
+ LEFT JOIN (SELECT *, 42 AS x FROM tab2) ss ON ...
+
+We want to be able to pull up the subquery as discussed previously,
+but we can't just replace the "ss.x" Var in the top-level SELECT list
+with the constant 42. That'd result in always emitting 42, rather
+than emitting NULL in null-extended join rows.
+
+To solve this, we introduce the concept of PlaceHolderVars.
+A PlaceHolderVar is somewhat like a Var, in that its value originates
+at a relation scan level and can then be forced to null by higher-level
+outer joins; hence PlaceHolderVars carry a set of nulling rel IDs just
+like Vars. Unlike a Var, whose original value comes from a table,
+a PlaceHolderVar's original value is defined by a query-determined
+expression ("42" in this example); so we represent the PlaceHolderVar
+as a node with that expression as child. We insert a PlaceHolderVar
+whenever subquery pullup needs to replace a subquery-referencing Var
+that has nonempty varnullingrels with an expression that is not simply a
+Var. (When the replacement expression is a pulled-up Var, we can just
+add the replaced Var's varnullingrels to its set. Also, if the replaced
+Var has empty varnullingrels, we don't need a PlaceHolderVar: there is
+nothing that'd force the value to null, so the pulled-up expression is
+fine to use as-is.) In a finished plan, a PlaceHolderVar becomes just
+the contained expression at whatever plan level it's supposed to be
+evaluated at, and then upper-level occurrences are replaced by Var
+references to that output column of the lower plan level. That causes
+the value to go to null when appropriate at an outer join, in the same
+way as for normal Vars. Thus, PlaceHolderVars are never seen outside
+the planner.
+
+PlaceHolderVars (PHVs) are more complicated than Vars in another way:
+their original value might need to be calculated at a join, not a
+base-level relation scan. This can happen when a pulled-up subquery
+contains a join. Because of this, a PHV can create a join order
+constraint that wouldn't otherwise exist, to ensure that it can
+be calculated before it is used. A PHV's expression can also contain
+LATERAL references, adding complications that are discussed below.
+
+
+Relation Identification and Qual Clause Placement
+-------------------------------------------------
+
+A qual clause obtained from WHERE or JOIN/ON can be enforced at the lowest
+scan or join level that includes all relations used in the clause. For
+this purpose we consider that outer joins listed in varnullingrels or
+phnullingrels are used in the clause, since we can't compute the qual's
+result correctly until we know whether such Vars have gone to null.
+
+The one exception to this general rule is that a non-degenerate outer
+JOIN/ON qual (one that references the non-nullable side of the join)
+cannot be enforced below that join, even if it doesn't reference the
+nullable side. Pushing it down into the non-nullable side would result
+in rows disappearing from the join's result, rather than appearing as
+null-extended rows. To handle that, when we identify such a qual we
+artificially add the join's minimum input relid set to the set of
+relations it is considered to use, forcing it to be evaluated exactly at
+that join level. The same happens for outer-join quals that mention no
+relations at all.
+
+When attaching a qual clause to a join plan node that is performing an
+outer join, the qual clause is considered a "join clause" (that is, it is
+applied before the join performs null-extension) if it does not reference
+that outer join in any varnullingrels or phnullingrels set, or a "filter
+clause" (applied after null-extension) if it does reference that outer
+join. A qual clause that originally appeared in that outer join's JOIN/ON
+will fall into the first category, since the parser would not have marked
+any of its Vars as referencing the outer join. A qual clause that
+originally came from some upper ON clause or WHERE clause will be seen as
+referencing the outer join if it references any of the nullable side's
+Vars, since those Vars will be so marked by the parser. But, if such a
+qual does not reference any nullable-side Vars, it's okay to push it down
+into the non-nullable side, so it won't get attached to the join node in
+the first place.
+
+These things lead us to identify join relations within the planner
+by the sets of base relation RT indexes plus outer join RT indexes
+that they include. In that way, the sets of relations used by qual
+clauses can be directly compared to join relations' relid sets to
+see where to place the clauses. These identifying sets are unique
+because, for any given collection of base relations, there is only
+one valid set of outer joins to have performed along the way to
+joining that set of base relations (although the order of applying
+them could vary, as discussed above).
+
+SEMI joins do not have RT indexes, because they are artifacts made by
+the planner rather than the parser. (We could create rangetable
+entries for them, but there seems no need at present.) This does not
+cause a problem for qual placement, because the nullable side of a
+semijoin is not referenceable from above the join, so there is never a
+need to cite it in varnullingrels or phnullingrels. It does not cause a
+problem for join relation identification either, since whether a semijoin
+has been completed is again implicit in the set of base relations
+included in the join.
+
+There is one additional complication for qual clause placement, which
+occurs when we have made multiple versions of an outer-join clause as
+described previously (that is, we have both "Pbc" and "Pb*c" forms of
+the same clause seen in outer join identity 3). When forming an outer
+join we only want to apply one of the redundant versions of the clause.
+If we are forming the B/C join without having yet computed the A/B
+join, it's easy to reject the "Pb*c" form since its required relid
+set includes the A/B join relid which is not in the input. However,
+if we form B/C after A/B, then both forms of the clause are applicable
+so far as that test can tell. We have to look more closely to notice
+that the "Pbc" clause form refers to relation B which is no longer
+directly accessible. While this check is straightforward, it's not
+especially cheap (see clause_is_computable_at()). To avoid doing it
+unnecessarily, we mark the variant versions of a redundant clause as
+either "has_clone" or "is_clone". When considering a clone clause,
+we must check clause_is_computable_at() to disentangle which version
+to apply at the current join level. (In debug builds, we also Assert
+that non-clone clauses are validly computable at the current level;
+but that seems too expensive for production usage.)
+
+
Optimizer Functions
-------------------
@@ -437,11 +670,10 @@ inputs.
EquivalenceClasses
------------------
-During the deconstruct_jointree() scan of the query's qual clauses, we look
-for mergejoinable equality clauses A = B whose applicability is not delayed
-by an outer join; these are called "equivalence clauses". When we find
-one, we create an EquivalenceClass containing the expressions A and B to
-record this knowledge. If we later find another equivalence clause B = C,
+During the deconstruct_jointree() scan of the query's qual clauses, we
+look for mergejoinable equality clauses A = B. When we find one, we
+create an EquivalenceClass containing the expressions A and B to record
+that they are equal. If we later find another equivalence clause B = C,
we add C to the existing EquivalenceClass for {A B}; this may require
merging two existing EquivalenceClasses. At the end of the scan, we have
sets of values that are known all transitively equal to each other. We can
@@ -473,15 +705,89 @@ asserts that at any plan node where more than one of its member values
can be computed, output rows in which the values are not all equal may
be discarded without affecting the query result. (We require all levels
of the plan to enforce EquivalenceClasses, hence a join need not recheck
-equality of values that were computable by one of its children.) For an
-ordinary EquivalenceClass that is "valid everywhere", we can further infer
-that the values are all non-null, because all mergejoinable operators are
-strict. However, we also allow equivalence clauses that appear below the
-nullable side of an outer join to form EquivalenceClasses; for these
-classes, the interpretation is that either all the values are equal, or
-all (except pseudo-constants) have gone to null. (This requires a
-limitation that non-constant members be strict, else they might not go
-to null when the other members do.) Consider for example
+equality of values that were computable by one of its children.)
+
+Outer joins complicate this picture quite a bit, however. While we could
+theoretically use mergejoinable equality clauses that appear in outer-join
+conditions as sources of EquivalenceClasses, there's a serious difficulty:
+the resulting deductions are not valid everywhere. For example, given
+
+ SELECT * FROM a LEFT JOIN b ON (a.x = b.y AND a.x = 42);
+
+we can safely derive b.y = 42 and use that in the scan of B, because B
+rows not having b.y = 42 will not contribute to the join result. However,
+we cannot apply a.x = 42 at the scan of A, or we will remove rows that
+should appear in the join result. We could apply a.x = 42 as an outer join
+condition (and then it would be unnecessary to also check a.x = b.y).
+This is not yet implemented, however.
+
+A related issue is that constants appearing below an outer join are
+less constant than they appear. Ordinarily, if we find "A = 1" and
+"B = 1", it's okay to put A and B into the same EquivalenceClass.
+But consider
+
+ SELECT * FROM a
+ LEFT JOIN (SELECT * FROM b WHERE b.z = 1) ss ON (a.x = b.y)
+ WHERE a.x = 1;
+
+It would be a serious error to conclude that a.x = b.z, so we cannot
+form a single EquivalenceClass {a.x b.z 1}.
+
+This leads to considering EquivalenceClasses as applying within "join
+domains", which are sets of relations that are inner-joined to each other.
+(We can treat semijoins as if they were inner joins for this purpose.)
+There is a top-level join domain, and then each outer join in the query
+creates a new join domain comprising its nullable side. Full joins create
+two join domains, one for each side. EquivalenceClasses generated from
+WHERE are associated with the top-level join domain. EquivalenceClasses
+generated from the ON clause of an outer join are associated with the
+domain created by that outer join. EquivalenceClasses generated from the
+ON clause of an inner or semi join are associated with the syntactically
+most closely nested join domain.
+
+Having defined these domains, we can fix the not-so-constant-constants
+problem by considering that constants only match EquivalenceClass members
+when they come from clauses within the same join domain. In the above
+example, this means we keep {a.x 1} and {b.z 1} as separate
+EquivalenceClasses and don't erroneously merge them. We don't have to
+worry about this for Vars (or expressions containing Vars), because
+references to the "same" column from different join domains will have
+different varnullingrels and thus won't be equal() anyway.
+
+In the future, the join-domain concept may allow us to treat mergejoinable
+outer-join conditions as sources of EquivalenceClasses. The idea would be
+that conditions derived from such classes could only be enforced at scans
+or joins that are within the appropriate join domain. This is not
+implemented yet, however, as the details are trickier than they appear.
+
+Another instructive example is:
+
+ SELECT *
+ FROM a LEFT JOIN
+ (SELECT * FROM b JOIN c ON b.y = c.z WHERE b.y = 10) ss
+ ON a.x = ss.y
+ ORDER BY ss.y;
+
+We can form the EquivalenceClass {b.y c.z 10} and thereby apply c.z = 10
+while scanning C, as well as b.y = 10 while scanning B, so that no clause
+needs to be checked at the inner join. The left-join clause "a.x = ss.y"
+(really "a.x = b.y") is not considered an equivalence clause, so we do
+not insert a.x into that same EquivalenceClass; if we did, we'd falsely
+conclude a.x = 10. In the future though we might be able to do that,
+if we can keep from applying a.x = 10 at the scan of A, which in principle
+we could do by noting that the EquivalenceClass only applies within the
+{B,C} join domain.
+
+Also notice that ss.y in the ORDER BY is really b.y* (that is, the
+possibly-nulled form of b.y), so we will not confuse it with the b.y member
+of the lower EquivalenceClass. Thus, we won't mistakenly conclude that
+that ss.y is equal to a constant, which if true would lead us to think that
+sorting for the ORDER BY is unnecessary (see discussion of PathKeys below).
+Instead, there will be a separate EquivalenceClass containing only b.y*,
+which will form the basis for the PathKey describing the required sort
+order.
+
+Also consider this variant:
SELECT *
FROM a LEFT JOIN
@@ -489,27 +795,42 @@ to null when the other members do.) Consider for example
ON a.x = ss.y
WHERE a.x = 42;
-We can form the below-outer-join EquivalenceClass {b.y c.z 10} and thereby
-apply c.z = 10 while scanning c. (The reason we disallow outerjoin-delayed
-clauses from forming EquivalenceClasses is exactly that we want to be able
-to push any derived clauses as far down as possible.) But once above the
-outer join it's no longer necessarily the case that b.y = 10, and thus we
-cannot use such EquivalenceClasses to conclude that sorting is unnecessary
-(see discussion of PathKeys below).
-
-In this example, notice also that a.x = ss.y (really a.x = b.y) is not an
-equivalence clause because its applicability to b is delayed by the outer
-join; thus we do not try to insert b.y into the equivalence class {a.x 42}.
-But since we see that a.x has been equated to 42 above the outer join, we
-are able to form a below-outer-join class {b.y 42}; this restriction can be
-added because no b/c row not having b.y = 42 can contribute to the result
-of the outer join, and so we need not compute such rows. Now this class
-will get merged with {b.y c.z 10}, leading to the contradiction 10 = 42,
-which lets the planner deduce that the b/c join need not be computed at all
-because none of its rows can contribute to the outer join. (This gets
-implemented as a gating Result filter, since more usually the potential
-contradiction involves Param values rather than just Consts, and thus has
-to be checked at runtime.)
+We still form the EquivalenceClass {b.y c.z 10}, and additionally
+we have an EquivalenceClass {a.x 42} belonging to a different join domain.
+We cannot use "a.x = b.y" to merge these classes. However, we can compare
+that outer join clause to the existing EquivalenceClasses and form the
+derived clause "b.y = 42", which we can treat as a valid equivalence
+within the lower join domain (since no row of that domain not having
+b.y = 42 can contribute to the outer-join result). That makes the lower
+EquivalenceClass {42 b.y c.z 10}, resulting in the contradiction 10 = 42,
+which lets the planner deduce that the B/C join need not be computed at
+all: the result of that whole join domain can be forced to empty.
+(This gets implemented as a gating Result filter, since more usually the
+potential contradiction involves Param values rather than just Consts, and
+thus it has to be checked at runtime. We can use the join domain to
+determine the join level at which to place the gating condition.)
+
+There is an additional complication when re-ordering outer joins according
+to identity 3. Recall that the two choices we consider for such joins are
+
+ A leftjoin (B leftjoin C on (Pbc)) on (Pab)
+ (A leftjoin B on (Pab)) leftjoin C on (Pb*c)
+
+where the star denotes varnullingrels markers on B's Vars. When Pbc
+is (or includes) a mergejoinable clause, we have something like
+
+ A leftjoin (B leftjoin C on (b.b = c.c)) on (Pab)
+ (A leftjoin B on (Pab)) leftjoin C on (b.b* = c.c)
+
+We could generate an EquivalenceClause linking b.b and c.c, but if we
+then also try to link b.b* and c.c, we end with a nonsensical conclusion
+that b.b and b.b* are equal (at least in some parts of the plan tree).
+In any case, the conclusions we could derive from such a thing would be
+largely duplicative. Conditions involving b.b* can't be computed below
+this join nest, while any conditions that can be computed would be
+duplicative of what we'd get from the b.b/c.c combination. Therefore,
+we choose to generate an EquivalenceClause linking b.b and c.c, but
+"b.b* = c.c" is handled as just an ordinary clause.
To aid in determining the sort ordering(s) that can work with a mergejoin,
we mark each mergejoinable clause with the EquivalenceClasses of its left
@@ -522,7 +843,11 @@ if other equivalence clauses are later found to bear on the same
expressions.
Another way that we may form a single-item EquivalenceClass is in creation
-of a PathKey to represent a desired sort order (see below). This is a bit
+of a PathKey to represent a desired sort order (see below). This happens
+if an ORDER BY or GROUP BY key is not mentioned in any equivalence
+clause. We need to reason about sort orders in such queries, and our
+representation of sort ordering is a PathKey which depends on an
+EquivalenceClass, so we have to make an EquivalenceClass. This is a bit
different from the above cases because such an EquivalenceClass might
contain an aggregate function or volatile expression. (A clause containing
a volatile function will never be considered mergejoinable, even if its top
@@ -544,6 +869,9 @@ it's possible that it belongs to more than one. We keep track of all the
families to ensure that we can make use of an index belonging to any one of
the families for mergejoin purposes.)
+For the same sort of reason, an EquivalenceClass is also associated
+with a particular collation, if its datatype(s) care about collation.
+
An EquivalenceClass can contain "em_is_child" members, which are copies
of members that contain appendrel parent relation Vars, transposed to
contain the equivalent child-relation variables or expressions. These
@@ -579,7 +907,7 @@ Index scans have Path.pathkeys that represent the chosen index's ordering,
if any. A single-key index would create a single-PathKey list, while a
multi-column index generates a list with one element per key index column.
Non-key columns specified in the INCLUDE clause of covering indexes don't
-have corresponding PathKeys in the list, because the have no influence on
+have corresponding PathKeys in the list, because they have no influence on
index ordering. (Actually, since an index can be scanned either forward or
backward, there are two possible sort orders and two possible PathKey lists
it can generate.)
@@ -608,9 +936,14 @@ must now be ordered too. This is true even though we used neither an
explicit sort nor a mergejoin on Y. (Note: hash joins cannot be counted
on to preserve the order of their outer relation, because the executor
might decide to "batch" the join, so we always set pathkeys to NIL for
-a hashjoin path.) Exception: a RIGHT or FULL join doesn't preserve the
-ordering of its outer relation, because it might insert nulls at random
-points in the ordering.
+a hashjoin path.)
+
+An outer join doesn't preserve the ordering of its nullable input
+relation(s), because it might insert nulls at random points in the
+ordering. We don't need to think about this explicitly in the PathKey
+representation, because a PathKey representing a post-join variable
+will contain varnullingrel bits, making it not equal to a PathKey
+representing the pre-join value.
In general, we can justify using EquivalenceClasses as the basis for
pathkeys because, whenever we scan a relation containing multiple
@@ -655,14 +988,9 @@ redundancy, we save time and improve planning, since the planner will more
easily recognize equivalent orderings as being equivalent.
Another interesting property is that if the underlying EquivalenceClass
-contains a constant and is not below an outer join, then the pathkey is
-completely redundant and need not be sorted by at all! Every row must
-contain the same constant value, so there's no need to sort. (If the EC is
-below an outer join, we still have to sort, since some of the rows might
-have gone to null and others not. In this case we must be careful to pick
-a non-const member to sort by. The assumption that all the non-const
-members go to null at the same plan level is critical here, else they might
-not produce the same sort order.) This might seem pointless because users
+contains a constant, then the pathkey is completely redundant and need not
+be sorted by at all! Every interesting row must contain the same value,
+so there's no need to sort. This might seem pointless because users
are unlikely to write "... WHERE x = 42 ORDER BY x", but it allows us to
recognize when particular index columns are irrelevant to the sort order:
if we have "... WHERE x = 42 ORDER BY y", scanning an index on (x,y)
@@ -670,15 +998,6 @@ produces correctly ordered data without a sort step. We used to have very
ugly ad-hoc code to recognize that in limited contexts, but discarding
constant ECs from pathkeys makes it happen cleanly and automatically.
-You might object that a below-outer-join EquivalenceClass doesn't always
-represent the same values at every level of the join tree, and so using
-it to uniquely identify a sort order is dubious. This is true, but we
-can avoid dealing with the fact explicitly because we always consider that
-an outer join destroys any ordering of its nullable inputs. Thus, even
-if a path was sorted by {a.x} below an outer join, we'll re-sort if that
-sort ordering was important; and so using the same PathKey for both sort
-orderings doesn't create any real problem.
-
Order of processing for EquivalenceClasses and PathKeys
-------------------------------------------------------
commit 2d9bef962ac66d24377b456b6d381a05f27168fb
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu Dec 22 16:19:42 2022 -0500
Do assorted preliminary refactoring.
This patch doesn't actually change any behavior, just lay some
fairly boring groundwork:
* Add Var.varnullingrels and PlaceHolderVar.phnullingrels fields.
These fields are always empty as of this commit, so they don't
affect any behavior, even though equal() will compare them.
Update backend/nodes/ and backend/rewrite/ infrastructure as needed.
* Refactor deconstruct_jointree() to split it into two passes;
the first pass computes relid bitmapsets and the second performs
qual distribution. The reason for this is that when we invent
"join domains", several patches later, we'll need to know the
full contents of the domains before distributing quals. It
seems like a good idea to split up the function anyway, as it's
gotten quite large.
* Move calculation of all_baserels to deconstruct_jointree,
because we'll soon need it to be available earlier.
(This means that remove_rel_from_query now has to update it.)
* Refactor to pass the relevant SpecialJoinInfo to
reconsider_outer_join_clause and reconsider_full_join_clause.
We'll need that later.
* Add some rewrite functions we'll need later for adding and
removing nullingrel bits in expression trees.
Note this will require a catversion bump when committed.
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index c85d8fe975..cced668f58 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -80,11 +80,13 @@ makeVar(int varno,
var->varlevelsup = varlevelsup;
/*
- * Only a few callers need to make Var nodes with varnosyn/varattnosyn
- * different from varno/varattno. We don't provide separate arguments for
- * them, but just initialize them to the given varno/varattno. This
- * reduces code clutter and chance of error for most callers.
+ * Only a few callers need to make Var nodes with non-null varnullingrels,
+ * or with varnosyn/varattnosyn different from varno/varattno. We don't
+ * provide separate arguments for them, but just initialize them to NULL
+ * and the given varno/varattno. This reduces code clutter and chance of
+ * error for most callers.
*/
+ var->varnullingrels = NULL;
var->varnosyn = (Index) varno;
var->varattnosyn = varattno;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index af8620ceb7..62245ae945 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -2641,6 +2641,7 @@ expression_tree_mutator_impl(Node *node,
Var *newnode;
FLATCOPY(newnode, var, Var);
+ /* Assume we need not copy the varnullingrels bitmapset */
return (Node *) newnode;
}
break;
@@ -3234,7 +3235,7 @@ expression_tree_mutator_impl(Node *node,
FLATCOPY(newnode, phv, PlaceHolderVar);
MUTATE(newnode->phexpr, phv->phexpr, Expr *);
- /* Assume we need not copy the relids bitmapset */
+ /* Assume we need not copy the relids bitmapsets */
return (Node *) newnode;
}
break;
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 0cfa3a1d6c..10ac01ac36 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -159,27 +159,6 @@ make_one_rel(PlannerInfo *root, List *joinlist)
Index rti;
double total_pages;
- /*
- * Construct the all_baserels Relids set.
- */
- root->all_baserels = NULL;
- for (rti = 1; rti < root->simple_rel_array_size; rti++)
- {
- RelOptInfo *brel = root->simple_rel_array[rti];
-
- /* there may be empty slots corresponding to non-baserel RTEs */
- if (brel == NULL)
- continue;
-
- Assert(brel->relid == rti); /* sanity check on array */
-
- /* ignore RTEs that are "other rels" */
- if (brel->reloptkind != RELOPT_BASEREL)
- continue;
-
- root->all_baserels = bms_add_member(root->all_baserels, brel->relid);
- }
-
/* Mark base rels as to whether we care about fast-start plans */
set_base_rel_consider_startup(root);
@@ -207,6 +186,7 @@ make_one_rel(PlannerInfo *root, List *joinlist)
{
RelOptInfo *brel = root->simple_rel_array[rti];
+ /* there may be empty slots corresponding to non-baserel RTEs */
if (brel == NULL)
continue;
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index e65b967b1f..019972cefc 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -61,10 +61,10 @@ static RestrictInfo *create_join_clause(PlannerInfo *root,
EquivalenceMember *rightem,
EquivalenceClass *parent_ec);
static bool reconsider_outer_join_clause(PlannerInfo *root,
- RestrictInfo *rinfo,
+ OuterJoinClauseInfo *ojcinfo,
bool outer_on_left);
static bool reconsider_full_join_clause(PlannerInfo *root,
- RestrictInfo *rinfo);
+ OuterJoinClauseInfo *ojcinfo);
static Bitmapset *get_eclass_indexes_for_relids(PlannerInfo *root,
Relids relids);
static Bitmapset *get_common_eclass_indexes(PlannerInfo *root, Relids relids1,
@@ -1977,10 +1977,12 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* Process the LEFT JOIN clauses */
foreach(cell, root->left_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ OuterJoinClauseInfo *ojcinfo = (OuterJoinClauseInfo *) lfirst(cell);
- if (reconsider_outer_join_clause(root, rinfo, true))
+ if (reconsider_outer_join_clause(root, ojcinfo, true))
{
+ RestrictInfo *rinfo = ojcinfo->rinfo;
+
found = true;
/* remove it from the list */
root->left_join_clauses =
@@ -1996,10 +1998,12 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* Process the RIGHT JOIN clauses */
foreach(cell, root->right_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ OuterJoinClauseInfo *ojcinfo = (OuterJoinClauseInfo *) lfirst(cell);
- if (reconsider_outer_join_clause(root, rinfo, false))
+ if (reconsider_outer_join_clause(root, ojcinfo, false))
{
+ RestrictInfo *rinfo = ojcinfo->rinfo;
+
found = true;
/* remove it from the list */
root->right_join_clauses =
@@ -2015,10 +2019,12 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* Process the FULL JOIN clauses */
foreach(cell, root->full_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ OuterJoinClauseInfo *ojcinfo = (OuterJoinClauseInfo *) lfirst(cell);
- if (reconsider_full_join_clause(root, rinfo))
+ if (reconsider_full_join_clause(root, ojcinfo))
{
+ RestrictInfo *rinfo = ojcinfo->rinfo;
+
found = true;
/* remove it from the list */
root->full_join_clauses =
@@ -2035,21 +2041,21 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* Now, any remaining clauses have to be thrown back */
foreach(cell, root->left_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ OuterJoinClauseInfo *ojcinfo = (OuterJoinClauseInfo *) lfirst(cell);
- distribute_restrictinfo_to_rels(root, rinfo);
+ distribute_restrictinfo_to_rels(root, ojcinfo->rinfo);
}
foreach(cell, root->right_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ OuterJoinClauseInfo *ojcinfo = (OuterJoinClauseInfo *) lfirst(cell);
- distribute_restrictinfo_to_rels(root, rinfo);
+ distribute_restrictinfo_to_rels(root, ojcinfo->rinfo);
}
foreach(cell, root->full_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ OuterJoinClauseInfo *ojcinfo = (OuterJoinClauseInfo *) lfirst(cell);
- distribute_restrictinfo_to_rels(root, rinfo);
+ distribute_restrictinfo_to_rels(root, ojcinfo->rinfo);
}
}
@@ -2059,9 +2065,10 @@ reconsider_outer_join_clauses(PlannerInfo *root)
* Returns true if we were able to propagate a constant through the clause.
*/
static bool
-reconsider_outer_join_clause(PlannerInfo *root, RestrictInfo *rinfo,
+reconsider_outer_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo,
bool outer_on_left)
{
+ RestrictInfo *rinfo = ojcinfo->rinfo;
Expr *outervar,
*innervar;
Oid opno,
@@ -2185,8 +2192,9 @@ reconsider_outer_join_clause(PlannerInfo *root, RestrictInfo *rinfo,
* Returns true if we were able to propagate a constant through the clause.
*/
static bool
-reconsider_full_join_clause(PlannerInfo *root, RestrictInfo *rinfo)
+reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
{
+ RestrictInfo *rinfo = ojcinfo->rinfo;
Expr *leftvar;
Expr *rightvar;
Oid opno,
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index bbeca9a9ab..dc0f165477 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -349,6 +349,11 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
}
}
+ /*
+ * Update all_baserels and related relid sets.
+ */
+ root->all_baserels = bms_del_member(root->all_baserels, relid);
+
/*
* Likewise remove references from SpecialJoinInfo data structures.
*
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index fd8cbb1dc7..2ee58f0a68 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -40,7 +40,40 @@ int from_collapse_limit;
int join_collapse_limit;
-/* Elements of the postponed_qual_list used during deconstruct_recurse */
+/*
+ * deconstruct_jointree requires multiple passes over the join tree, because we
+ * need to finish computing JoinDomains before we start distributing quals.
+ * As long as we have to do that, other information such as the relevant
+ * qualscopes might as well be computed in the first pass too.
+ *
+ * deconstruct_recurse recursively examines the join tree and builds a List
+ * (in depth-first traversal order) of JoinTreeItem structs, which are then
+ * processed iteratively by deconstruct_distribute.
+ *
+ * The JoinTreeItem structs themselves can be freed at the end of
+ * deconstruct_jointree, but do not modify or free their substructure,
+ * as the relid sets may also be pointed to by RestrictInfo and
+ * SpecialJoinInfo nodes.
+ */
+typedef struct JoinTreeItem
+{
+ /* Fields filled during deconstruct_recurse: */
+ Node *jtnode; /* jointree node to examine */
+ bool below_outer_join; /* is it below an outer join? */
+ Relids qualscope; /* base Relids syntactically included in this
+ * jointree node */
+ Relids inner_join_rels; /* base Relids syntactically included in
+ * inner joins appearing at or below this
+ * jointree node */
+ Relids left_rels; /* if join node, Relids of the left side */
+ Relids right_rels; /* if join node, Relids of the right side */
+ Relids nonnullable_rels; /* if outer join, Relids of the
+ * non-nullable side */
+ /* Fields filled during deconstruct_distribute: */
+ SpecialJoinInfo *sjinfo; /* if outer join, its SpecialJoinInfo */
+} JoinTreeItem;
+
+/* Elements of the postponed_qual_list used during deconstruct_distribute */
typedef struct PostponedQual
{
Node *qual; /* a qual clause waiting to be processed */
@@ -52,8 +85,9 @@ static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
bool below_outer_join,
- Relids *qualscope, Relids *inner_join_rels,
- List **postponed_qual_list);
+ List **item_list);
+static void deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
+ List **postponed_qual_list);
static void process_security_barrier_quals(PlannerInfo *root,
int rti, Relids qualscope,
bool below_outer_join);
@@ -63,9 +97,17 @@ static SpecialJoinInfo *make_outerjoininfo(PlannerInfo *root,
JoinType jointype, List *clause);
static void compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo,
List *clause);
+static void distribute_quals_to_rels(PlannerInfo *root, List *clauses,
+ bool below_outer_join,
+ SpecialJoinInfo *sjinfo,
+ Index security_level,
+ Relids qualscope,
+ Relids ojscope,
+ Relids outerjoin_nonnullable,
+ List **postponed_qual_list);
static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool below_outer_join,
- JoinType jointype,
+ SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
Relids ojscope,
@@ -683,9 +725,9 @@ List *
deconstruct_jointree(PlannerInfo *root)
{
List *result;
- Relids qualscope;
- Relids inner_join_rels;
+ List *item_list = NIL;
List *postponed_qual_list = NIL;
+ ListCell *lc;
/*
* After this point, no more PlaceHolderInfos may be made, because
@@ -699,98 +741,105 @@ deconstruct_jointree(PlannerInfo *root)
Assert(root->parse->jointree != NULL &&
IsA(root->parse->jointree, FromExpr));
- /* this is filled as we scan the jointree */
+ /* These are filled as we scan the jointree */
+ root->all_baserels = NULL;
root->nullable_baserels = NULL;
- result = deconstruct_recurse(root, (Node *) root->parse->jointree, false,
- &qualscope, &inner_join_rels,
- &postponed_qual_list);
+ /* Perform the initial scan of the jointree */
+ result = deconstruct_recurse(root, (Node *) root->parse->jointree,
+ false,
+ &item_list);
- /* Shouldn't be any leftover quals */
+ /* Now scan all the jointree nodes again, and distribute quals */
+ foreach(lc, item_list)
+ {
+ JoinTreeItem *jtitem = (JoinTreeItem *) lfirst(lc);
+
+ deconstruct_distribute(root, jtitem,
+ &postponed_qual_list);
+ }
+
+ /* Shouldn't be any leftover postponed quals */
Assert(postponed_qual_list == NIL);
+ /* Don't need the JoinTreeItems any more */
+ list_free_deep(item_list);
+
return result;
}
/*
* deconstruct_recurse
- * One recursion level of deconstruct_jointree processing.
+ * One recursion level of deconstruct_jointree's initial jointree scan.
*
* Inputs:
* jtnode is the jointree node to examine
* below_outer_join is true if this node is within the nullable side of a
* higher-level outer join
- * Outputs:
- * *qualscope gets the set of base Relids syntactically included in this
- * jointree node (do not modify or free this, as it may also be pointed
- * to by RestrictInfo and SpecialJoinInfo nodes)
- * *inner_join_rels gets the set of base Relids syntactically included in
- * inner joins appearing at or below this jointree node (do not modify
- * or free this, either)
- * *postponed_qual_list is a list of PostponedQual structs, which we can
- * add quals to if they turn out to belong to a higher join level
- * Return value is the appropriate joinlist for this jointree node
*
- * In addition, entries will be added to root->join_info_list for outer joins.
+ * item_list is an in/out parameter: we add a JoinTreeItem struct to
+ * that list for each jointree node, in depth-first traversal order.
+ * (Hence, after each call, the last list item corresponds to its jtnode.)
+ *
+ * Return value is the appropriate joinlist for this jointree node.
*/
static List *
-deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
- Relids *qualscope, Relids *inner_join_rels,
- List **postponed_qual_list)
+deconstruct_recurse(PlannerInfo *root, Node *jtnode,
+ bool below_outer_join,
+ List **item_list)
{
List *joinlist;
+ JoinTreeItem *jtitem;
+
+ Assert(jtnode != NULL);
+
+ /* Make the new JoinTreeItem, but don't add it to item_list yet */
+ jtitem = palloc0_object(JoinTreeItem);
+ jtitem->jtnode = jtnode;
+ jtitem->below_outer_join = below_outer_join;
- if (jtnode == NULL)
- {
- *qualscope = NULL;
- *inner_join_rels = NULL;
- return NIL;
- }
if (IsA(jtnode, RangeTblRef))
{
int varno = ((RangeTblRef *) jtnode)->rtindex;
+ /* Fill all_baserels as we encounter baserel jointree nodes */
+ root->all_baserels = bms_add_member(root->all_baserels, varno);
/* qualscope is just the one RTE */
- *qualscope = bms_make_singleton(varno);
- /* Deal with any securityQuals attached to the RTE */
- if (root->qual_security_level > 0)
- process_security_barrier_quals(root,
- varno,
- *qualscope,
- below_outer_join);
+ jtitem->qualscope = bms_make_singleton(varno);
/* A single baserel does not create an inner join */
- *inner_join_rels = NULL;
+ jtitem->inner_join_rels = NULL;
joinlist = list_make1(jtnode);
}
else if (IsA(jtnode, FromExpr))
{
FromExpr *f = (FromExpr *) jtnode;
- List *child_postponed_quals = NIL;
int remaining;
ListCell *l;
/*
- * First, recurse to handle child joins. We collapse subproblems into
- * a single joinlist whenever the resulting joinlist wouldn't exceed
- * from_collapse_limit members. Also, always collapse one-element
- * subproblems, since that won't lengthen the joinlist anyway.
+ * Recurse to handle child nodes, and compute output joinlist. We
+ * collapse subproblems into a single joinlist whenever the resulting
+ * joinlist wouldn't exceed from_collapse_limit members. Also, always
+ * collapse one-element subproblems, since that won't lengthen the
+ * joinlist anyway.
*/
- *qualscope = NULL;
- *inner_join_rels = NULL;
+ jtitem->qualscope = NULL;
+ jtitem->inner_join_rels = NULL;
joinlist = NIL;
remaining = list_length(f->fromlist);
foreach(l, f->fromlist)
{
- Relids sub_qualscope;
+ JoinTreeItem *sub_item;
List *sub_joinlist;
int sub_members;
sub_joinlist = deconstruct_recurse(root, lfirst(l),
below_outer_join,
- &sub_qualscope,
- inner_join_rels,
- &child_postponed_quals);
- *qualscope = bms_add_members(*qualscope, sub_qualscope);
+ item_list);
+ sub_item = (JoinTreeItem *) llast(*item_list);
+ jtitem->qualscope = bms_add_members(jtitem->qualscope,
+ sub_item->qualscope);
+ jtitem->inner_join_rels = sub_item->inner_join_rels;
sub_members = list_length(sub_joinlist);
remaining--;
if (sub_members <= 1 ||
@@ -808,115 +857,80 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
* that still possible?) the initialization before the loop fixed it.
*/
if (list_length(f->fromlist) > 1)
- *inner_join_rels = *qualscope;
-
- /*
- * Try to process any quals postponed by children. If they need
- * further postponement, add them to my output postponed_qual_list.
- */
- foreach(l, child_postponed_quals)
- {
- PostponedQual *pq = (PostponedQual *) lfirst(l);
-
- if (bms_is_subset(pq->relids, *qualscope))
- distribute_qual_to_rels(root, pq->qual,
- below_outer_join, JOIN_INNER,
- root->qual_security_level,
- *qualscope, NULL, NULL,
- NULL);
- else
- *postponed_qual_list = lappend(*postponed_qual_list, pq);
- }
-
- /*
- * Now process the top-level quals.
- */
- foreach(l, (List *) f->quals)
- {
- Node *qual = (Node *) lfirst(l);
-
- distribute_qual_to_rels(root, qual,
- below_outer_join, JOIN_INNER,
- root->qual_security_level,
- *qualscope, NULL, NULL,
- postponed_qual_list);
- }
+ jtitem->inner_join_rels = jtitem->qualscope;
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- List *child_postponed_quals = NIL;
- Relids leftids,
- rightids,
- left_inners,
- right_inners,
- nonnullable_rels,
- nullable_rels,
- ojscope;
+ Relids nullable_rels;
+ JoinTreeItem *left_item,
+ *right_item;
List *leftjoinlist,
*rightjoinlist;
- List *my_quals;
- SpecialJoinInfo *sjinfo;
- ListCell *l;
- /*
- * Order of operations here is subtle and critical. First we recurse
- * to handle sub-JOINs. Their join quals will be placed without
- * regard for whether this level is an outer join, which is correct.
- * Then we place our own join quals, which are restricted by lower
- * outer joins in any case, and are forced to this level if this is an
- * outer join and they mention the outer side. Finally, if this is an
- * outer join, we create a join_info_list entry for the join. This
- * will prevent quals above us in the join tree that use those rels
- * from being pushed down below this level. (It's okay for upper
- * quals to be pushed down to the outer side, however.)
- */
switch (j->jointype)
{
case JOIN_INNER:
+ /* Recurse */
leftjoinlist = deconstruct_recurse(root, j->larg,
below_outer_join,
- &leftids, &left_inners,
- &child_postponed_quals);
+ item_list);
+ left_item = (JoinTreeItem *) llast(*item_list);
rightjoinlist = deconstruct_recurse(root, j->rarg,
below_outer_join,
- &rightids, &right_inners,
- &child_postponed_quals);
- *qualscope = bms_union(leftids, rightids);
- *inner_join_rels = *qualscope;
+ item_list);
+ right_item = (JoinTreeItem *) llast(*item_list);
+ /* Compute qualscope etc */
+ jtitem->qualscope = bms_union(left_item->qualscope,
+ right_item->qualscope);
+ jtitem->inner_join_rels = jtitem->qualscope;
+ jtitem->left_rels = left_item->qualscope;
+ jtitem->right_rels = right_item->qualscope;
/* Inner join adds no restrictions for quals */
- nonnullable_rels = NULL;
+ jtitem->nonnullable_rels = NULL;
/* and it doesn't force anything to null, either */
nullable_rels = NULL;
break;
case JOIN_LEFT:
case JOIN_ANTI:
+ /* Recurse */
leftjoinlist = deconstruct_recurse(root, j->larg,
below_outer_join,
- &leftids, &left_inners,
- &child_postponed_quals);
+ item_list);
+ left_item = (JoinTreeItem *) llast(*item_list);
rightjoinlist = deconstruct_recurse(root, j->rarg,
true,
- &rightids, &right_inners,
- &child_postponed_quals);
- *qualscope = bms_union(leftids, rightids);
- *inner_join_rels = bms_union(left_inners, right_inners);
- nonnullable_rels = leftids;
- nullable_rels = rightids;
+ item_list);
+ right_item = (JoinTreeItem *) llast(*item_list);
+ /* Compute qualscope etc */
+ jtitem->qualscope = bms_union(left_item->qualscope,
+ right_item->qualscope);
+ jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
+ right_item->inner_join_rels);
+ jtitem->left_rels = left_item->qualscope;
+ jtitem->right_rels = right_item->qualscope;
+ jtitem->nonnullable_rels = left_item->qualscope;
+ nullable_rels = right_item->qualscope;
break;
case JOIN_SEMI:
+ /* Recurse */
leftjoinlist = deconstruct_recurse(root, j->larg,
below_outer_join,
- &leftids, &left_inners,
- &child_postponed_quals);
+ item_list);
+ left_item = (JoinTreeItem *) llast(*item_list);
rightjoinlist = deconstruct_recurse(root, j->rarg,
below_outer_join,
- &rightids, &right_inners,
- &child_postponed_quals);
- *qualscope = bms_union(leftids, rightids);
- *inner_join_rels = bms_union(left_inners, right_inners);
+ item_list);
+ right_item = (JoinTreeItem *) llast(*item_list);
+ /* Compute qualscope etc */
+ jtitem->qualscope = bms_union(left_item->qualscope,
+ right_item->qualscope);
+ jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
+ right_item->inner_join_rels);
+ jtitem->left_rels = left_item->qualscope;
+ jtitem->right_rels = right_item->qualscope;
/* Semi join adds no restrictions for quals */
- nonnullable_rels = NULL;
+ jtitem->nonnullable_rels = NULL;
/*
* Theoretically, a semijoin would null the RHS; but since the
@@ -926,27 +940,32 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
nullable_rels = NULL;
break;
case JOIN_FULL:
+ /* Recurse */
leftjoinlist = deconstruct_recurse(root, j->larg,
true,
- &leftids, &left_inners,
- &child_postponed_quals);
+ item_list);
+ left_item = (JoinTreeItem *) llast(*item_list);
rightjoinlist = deconstruct_recurse(root, j->rarg,
true,
- &rightids, &right_inners,
- &child_postponed_quals);
- *qualscope = bms_union(leftids, rightids);
- *inner_join_rels = bms_union(left_inners, right_inners);
+ item_list);
+ right_item = (JoinTreeItem *) llast(*item_list);
+ /* Compute qualscope etc */
+ jtitem->qualscope = bms_union(left_item->qualscope,
+ right_item->qualscope);
+ jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
+ right_item->inner_join_rels);
+ jtitem->left_rels = left_item->qualscope;
+ jtitem->right_rels = right_item->qualscope;
/* each side is both outer and inner */
- nonnullable_rels = *qualscope;
- nullable_rels = *qualscope;
+ jtitem->nonnullable_rels = jtitem->qualscope;
+ nullable_rels = jtitem->qualscope;
break;
default:
/* JOIN_RIGHT was eliminated during reduce_outer_joins() */
elog(ERROR, "unrecognized join type: %d",
(int) j->jointype);
- nonnullable_rels = NULL; /* keep compiler quiet */
+ leftjoinlist = rightjoinlist = NIL; /* keep compiler quiet */
nullable_rels = NULL;
- leftjoinlist = rightjoinlist = NIL;
break;
}
@@ -954,6 +973,131 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
root->nullable_baserels = bms_add_members(root->nullable_baserels,
nullable_rels);
+ /*
+ * Compute the output joinlist. We fold subproblems together except
+ * at a FULL JOIN or where join_collapse_limit would be exceeded.
+ */
+ if (j->jointype == JOIN_FULL)
+ {
+ /* force the join order exactly at this node */
+ joinlist = list_make1(list_make2(leftjoinlist, rightjoinlist));
+ }
+ else if (list_length(leftjoinlist) + list_length(rightjoinlist) <=
+ join_collapse_limit)
+ {
+ /* OK to combine subproblems */
+ joinlist = list_concat(leftjoinlist, rightjoinlist);
+ }
+ else
+ {
+ /* can't combine, but needn't force join order above here */
+ Node *leftpart,
+ *rightpart;
+
+ /* avoid creating useless 1-element sublists */
+ if (list_length(leftjoinlist) == 1)
+ leftpart = (Node *) linitial(leftjoinlist);
+ else
+ leftpart = (Node *) leftjoinlist;
+ if (list_length(rightjoinlist) == 1)
+ rightpart = (Node *) linitial(rightjoinlist);
+ else
+ rightpart = (Node *) rightjoinlist;
+ joinlist = list_make2(leftpart, rightpart);
+ }
+ }
+ else
+ {
+ elog(ERROR, "unrecognized node type: %d",
+ (int) nodeTag(jtnode));
+ joinlist = NIL; /* keep compiler quiet */
+ }
+
+ /* Finally, we can add the new JoinTreeItem to item_list */
+ *item_list = lappend(*item_list, jtitem);
+
+ return joinlist;
+}
+
+/*
+ * deconstruct_distribute
+ * Process one jointree node in phase 2 of deconstruct_jointree processing.
+ *
+ * Distribute quals of the node to appropriate restriction and join lists.
+ * In addition, entries will be added to root->join_info_list for outer joins.
+ *
+ * Inputs:
+ * jtitem is the JoinTreeItem to examine
+ * Input/Outputs:
+ * *postponed_qual_list is a list of PostponedQual structs
+ *
+ * On entry, *postponed_qual_list contains any quals that had to be postponed
+ * out of lower join levels (because they contain lateral references).
+ * On exit, *postponed_qual_list contains quals that can't be processed yet
+ * (because their lateral references are still unsatisfied).
+ */
+static void
+deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
+ List **postponed_qual_list)
+{
+ Node *jtnode = jtitem->jtnode;
+
+ if (IsA(jtnode, RangeTblRef))
+ {
+ int varno = ((RangeTblRef *) jtnode)->rtindex;
+
+ /* Deal with any securityQuals attached to the RTE */
+ if (root->qual_security_level > 0)
+ process_security_barrier_quals(root,
+ varno,
+ jtitem->qualscope,
+ jtitem->below_outer_join);
+ }
+ else if (IsA(jtnode, FromExpr))
+ {
+ FromExpr *f = (FromExpr *) jtnode;
+ List *new_postponed_quals = NIL;
+ ListCell *l;
+
+ /*
+ * Try to process any quals postponed by children. If they need
+ * further postponement, add them to my output postponed_qual_list.
+ */
+ foreach(l, *postponed_qual_list)
+ {
+ PostponedQual *pq = (PostponedQual *) lfirst(l);
+
+ if (bms_is_subset(pq->relids, jtitem->qualscope))
+ distribute_qual_to_rels(root, pq->qual,
+ jtitem->below_outer_join,
+ NULL,
+ root->qual_security_level,
+ jtitem->qualscope, NULL, NULL,
+ NULL);
+ else
+ new_postponed_quals = lappend(new_postponed_quals, pq);
+ }
+ *postponed_qual_list = new_postponed_quals;
+
+ /*
+ * Now process the top-level quals.
+ */
+ distribute_quals_to_rels(root, (List *) f->quals,
+ jtitem->below_outer_join,
+ NULL,
+ root->qual_security_level,
+ jtitem->qualscope, NULL, NULL,
+ postponed_qual_list);
+ }
+ else if (IsA(jtnode, JoinExpr))
+ {
+ JoinExpr *j = (JoinExpr *) jtnode;
+ List *new_postponed_quals = NIL;
+ Relids ojscope;
+ List *my_quals;
+ SpecialJoinInfo *sjinfo;
+ ListCell *l;
+
/*
* Try to process any quals postponed by children. If they need
* further postponement, add them to my output postponed_qual_list.
@@ -961,11 +1105,11 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
* that they'll be handled properly in make_outerjoininfo.
*/
my_quals = NIL;
- foreach(l, child_postponed_quals)
+ foreach(l, *postponed_qual_list)
{
PostponedQual *pq = (PostponedQual *) lfirst(l);
- if (bms_is_subset(pq->relids, *qualscope))
+ if (bms_is_subset(pq->relids, jtitem->qualscope))
my_quals = lappend(my_quals, pq->qual);
else
{
@@ -974,16 +1118,15 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
* If this Assert fires, pull_up_subqueries() messed up.
*/
Assert(j->jointype == JOIN_INNER);
- *postponed_qual_list = lappend(*postponed_qual_list, pq);
+ new_postponed_quals = lappend(new_postponed_quals, pq);
}
}
+ *postponed_qual_list = new_postponed_quals;
my_quals = list_concat(my_quals, (List *) j->quals);
/*
- * For an OJ, form the SpecialJoinInfo now, because we need the OJ's
- * semantic scope (ojscope) to pass to distribute_qual_to_rels. But
- * we mustn't add it to join_info_list just yet, because we don't want
- * distribute_qual_to_rels to think it is an outer join below us.
+ * For an OJ, form the SpecialJoinInfo now, so that we can pass it to
+ * distribute_qual_to_rels. We must compute its ojscope too.
*
* Semijoins are a bit of a hybrid: we build a SpecialJoinInfo, but we
* want ojscope = NULL for distribute_qual_to_rels.
@@ -991,15 +1134,19 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
if (j->jointype != JOIN_INNER)
{
sjinfo = make_outerjoininfo(root,
- leftids, rightids,
- *inner_join_rels,
+ jtitem->left_rels,
+ jtitem->right_rels,
+ jtitem->inner_join_rels,
j->jointype,
my_quals);
+ jtitem->sjinfo = sjinfo;
if (j->jointype == JOIN_SEMI)
ojscope = NULL;
else
+ {
ojscope = bms_union(sjinfo->min_lefthand,
sjinfo->min_righthand);
+ }
}
else
{
@@ -1008,67 +1155,27 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
}
/* Process the JOIN's qual clauses */
- foreach(l, my_quals)
- {
- Node *qual = (Node *) lfirst(l);
-
- distribute_qual_to_rels(root, qual,
- below_outer_join, j->jointype,
- root->qual_security_level,
- *qualscope,
- ojscope, nonnullable_rels,
- postponed_qual_list);
- }
-
- /* Now we can add the SpecialJoinInfo to join_info_list */
+ distribute_quals_to_rels(root, my_quals,
+ jtitem->below_outer_join,
+ sjinfo,
+ root->qual_security_level,
+ jtitem->qualscope,
+ ojscope, jtitem->nonnullable_rels,
+ postponed_qual_list);
+
+ /* And add the SpecialJoinInfo to join_info_list */
if (sjinfo)
{
root->join_info_list = lappend(root->join_info_list, sjinfo);
/* Each time we do that, recheck placeholder eval levels */
update_placeholder_eval_levels(root, sjinfo);
}
-
- /*
- * Finally, compute the output joinlist. We fold subproblems together
- * except at a FULL JOIN or where join_collapse_limit would be
- * exceeded.
- */
- if (j->jointype == JOIN_FULL)
- {
- /* force the join order exactly at this node */
- joinlist = list_make1(list_make2(leftjoinlist, rightjoinlist));
- }
- else if (list_length(leftjoinlist) + list_length(rightjoinlist) <=
- join_collapse_limit)
- {
- /* OK to combine subproblems */
- joinlist = list_concat(leftjoinlist, rightjoinlist);
- }
- else
- {
- /* can't combine, but needn't force join order above here */
- Node *leftpart,
- *rightpart;
-
- /* avoid creating useless 1-element sublists */
- if (list_length(leftjoinlist) == 1)
- leftpart = (Node *) linitial(leftjoinlist);
- else
- leftpart = (Node *) leftjoinlist;
- if (list_length(rightjoinlist) == 1)
- rightpart = (Node *) linitial(rightjoinlist);
- else
- rightpart = (Node *) rightjoinlist;
- joinlist = list_make2(leftpart, rightpart);
- }
}
else
{
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(jtnode));
- joinlist = NIL; /* keep compiler quiet */
}
- return joinlist;
}
/*
@@ -1102,27 +1209,21 @@ process_security_barrier_quals(PlannerInfo *root,
foreach(lc, rte->securityQuals)
{
List *qualset = (List *) lfirst(lc);
- ListCell *lc2;
-
- foreach(lc2, qualset)
- {
- Node *qual = (Node *) lfirst(lc2);
- /*
- * We cheat to the extent of passing ojscope = qualscope rather
- * than its more logical value of NULL. The only effect this has
- * is to force a Var-free qual to be evaluated at the rel rather
- * than being pushed up to top of tree, which we don't want.
- */
- distribute_qual_to_rels(root, qual,
- below_outer_join,
- JOIN_INNER,
- security_level,
- qualscope,
- qualscope,
- NULL,
- NULL);
- }
+ /*
+ * We cheat to the extent of passing ojscope = qualscope rather than
+ * its more logical value of NULL. The only effect this has is to
+ * force a Var-free qual to be evaluated at the rel rather than being
+ * pushed up to top of tree, which we don't want.
+ */
+ distribute_quals_to_rels(root, qualset,
+ below_outer_join,
+ NULL,
+ security_level,
+ qualscope,
+ qualscope,
+ NULL,
+ NULL);
security_level++;
}
@@ -1572,6 +1673,38 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
*
*****************************************************************************/
+/*
+ * distribute_quals_to_rels
+ * Convenience routine to apply distribute_qual_to_rels to each element
+ * of an AND'ed list of clauses.
+ */
+static void
+distribute_quals_to_rels(PlannerInfo *root, List *clauses,
+ bool below_outer_join,
+ SpecialJoinInfo *sjinfo,
+ Index security_level,
+ Relids qualscope,
+ Relids ojscope,
+ Relids outerjoin_nonnullable,
+ List **postponed_qual_list)
+{
+ ListCell *lc;
+
+ foreach(lc, clauses)
+ {
+ Node *clause = (Node *) lfirst(lc);
+
+ distribute_qual_to_rels(root, clause,
+ below_outer_join,
+ sjinfo,
+ security_level,
+ qualscope,
+ ojscope,
+ outerjoin_nonnullable,
+ postponed_qual_list);
+ }
+}
+
/*
* distribute_qual_to_rels
* Add clause information to either the baserestrictinfo or joininfo list
@@ -1586,7 +1719,7 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
* 'clause': the qual clause to be distributed
* 'below_outer_join': true if the qual is from a JOIN/ON that is below the
* nullable side of a higher-level outer join
- * 'jointype': type of join the qual is from (JOIN_INNER for a WHERE clause)
+ * 'sjinfo': join's SpecialJoinInfo (NULL for an inner join or WHERE clause)
* 'security_level': security_level to assign to the qual
* 'qualscope': set of baserels the qual's syntactic scope covers
* 'ojscope': NULL if not an outer-join qual, else the minimum set of baserels
@@ -1604,12 +1737,13 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
* level, which will be ojscope not necessarily qualscope.
*
* At the time this is called, root->join_info_list must contain entries for
- * all and only those special joins that are syntactically below this qual.
+ * all and only those special joins that are syntactically below this qual;
+ * in particular, the passed-in SpecialJoinInfo isn't yet in that list.
*/
static void
distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool below_outer_join,
- JoinType jointype,
+ SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
Relids ojscope,
@@ -1646,7 +1780,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
PostponedQual *pq = (PostponedQual *) palloc(sizeof(PostponedQual));
Assert(root->hasLateralRTEs); /* shouldn't happen otherwise */
- Assert(jointype == JOIN_INNER); /* mustn't postpone past outer join */
+ Assert(sjinfo == NULL); /* mustn't postpone past outer join */
pq->qual = clause;
pq->relids = relids;
*postponed_qual_list = lappend(*postponed_qual_list, pq);
@@ -1930,14 +2064,19 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
/* we need to set up left_ec/right_ec the hard way */
initialize_mergeclause_eclasses(root, restrictinfo);
/* now see if it should go to any outer-join lists */
+ Assert(sjinfo != NULL);
if (bms_is_subset(restrictinfo->left_relids,
outerjoin_nonnullable) &&
!bms_overlap(restrictinfo->right_relids,
outerjoin_nonnullable))
{
/* we have outervar = innervar */
+ OuterJoinClauseInfo *ojcinfo = makeNode(OuterJoinClauseInfo);
+
+ ojcinfo->rinfo = restrictinfo;
+ ojcinfo->sjinfo = sjinfo;
root->left_join_clauses = lappend(root->left_join_clauses,
- restrictinfo);
+ ojcinfo);
return;
}
if (bms_is_subset(restrictinfo->right_relids,
@@ -1946,15 +2085,23 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
outerjoin_nonnullable))
{
/* we have innervar = outervar */
+ OuterJoinClauseInfo *ojcinfo = makeNode(OuterJoinClauseInfo);
+
+ ojcinfo->rinfo = restrictinfo;
+ ojcinfo->sjinfo = sjinfo;
root->right_join_clauses = lappend(root->right_join_clauses,
- restrictinfo);
+ ojcinfo);
return;
}
- if (jointype == JOIN_FULL)
+ if (sjinfo->jointype == JOIN_FULL)
{
/* FULL JOIN (above tests cannot match in this case) */
+ OuterJoinClauseInfo *ojcinfo = makeNode(OuterJoinClauseInfo);
+
+ ojcinfo->rinfo = restrictinfo;
+ ojcinfo->sjinfo = sjinfo;
root->full_join_clauses = lappend(root->full_join_clauses,
- restrictinfo);
+ ojcinfo);
return;
}
/* nope, so fall through to distribute_restrictinfo_to_rels */
diff --git a/src/backend/rewrite/rewriteManip.c b/src/backend/rewrite/rewriteManip.c
index 2a07502030..59be25675f 100644
--- a/src/backend/rewrite/rewriteManip.c
+++ b/src/backend/rewrite/rewriteManip.c
@@ -40,6 +40,20 @@ typedef struct
int win_location;
} locate_windowfunc_context;
+typedef struct
+{
+ const Bitmapset *target_relids;
+ const Bitmapset *added_relids;
+ int sublevels_up;
+} add_nulling_relids_context;
+
+typedef struct
+{
+ const Bitmapset *removable_relids;
+ const Bitmapset *except_relids;
+ int sublevels_up;
+} remove_nulling_relids_context;
+
static bool contain_aggs_of_level_walker(Node *node,
contain_aggs_of_level_context *context);
static bool locate_agg_of_level_walker(Node *node,
@@ -50,6 +64,10 @@ static bool locate_windowfunc_walker(Node *node,
static bool checkExprHasSubLink_walker(Node *node, void *context);
static Relids offset_relid_set(Relids relids, int offset);
static Relids adjust_relid_set(Relids relids, int oldrelid, int newrelid);
+static Node *add_nulling_relids_mutator(Node *node,
+ add_nulling_relids_context *context);
+static Node *remove_nulling_relids_mutator(Node *node,
+ remove_nulling_relids_context *context);
/*
@@ -381,6 +399,8 @@ OffsetVarNodes_walker(Node *node, OffsetVarNodes_context *context)
if (var->varlevelsup == context->sublevels_up)
{
var->varno += context->offset;
+ var->varnullingrels = offset_relid_set(var->varnullingrels,
+ context->offset);
if (var->varnosyn > 0)
var->varnosyn += context->offset;
}
@@ -419,6 +439,8 @@ OffsetVarNodes_walker(Node *node, OffsetVarNodes_context *context)
{
phv->phrels = offset_relid_set(phv->phrels,
context->offset);
+ phv->phnullingrels = offset_relid_set(phv->phnullingrels,
+ context->offset);
}
/* fall through to examine children */
}
@@ -543,11 +565,13 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context)
{
Var *var = (Var *) node;
- if (var->varlevelsup == context->sublevels_up &&
- var->varno == context->rt_index)
+ if (var->varlevelsup == context->sublevels_up)
{
- var->varno = context->new_index;
- /* If the syntactic referent is same RTE, fix it too */
+ if (var->varno == context->rt_index)
+ var->varno = context->new_index;
+ var->varnullingrels = adjust_relid_set(var->varnullingrels,
+ context->rt_index,
+ context->new_index);
if (var->varnosyn == context->rt_index)
var->varnosyn = context->new_index;
}
@@ -590,6 +614,9 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context)
phv->phrels = adjust_relid_set(phv->phrels,
context->rt_index,
context->new_index);
+ phv->phnullingrels = adjust_relid_set(phv->phnullingrels,
+ context->rt_index,
+ context->new_index);
}
/* fall through to examine children */
}
@@ -866,7 +893,8 @@ rangeTableEntry_used_walker(Node *node,
Var *var = (Var *) node;
if (var->varlevelsup == context->sublevels_up &&
- var->varno == context->rt_index)
+ (var->varno == context->rt_index ||
+ bms_is_member(context->rt_index, var->varnullingrels)))
return true;
return false;
}
@@ -1094,6 +1122,195 @@ AddInvertedQual(Query *parsetree, Node *qual)
}
+/*
+ * add_nulling_relids() finds Vars and PlaceHolderVars that belong to any
+ * of the target_relids, and adds added_relids to their varnullingrels
+ * and phnullingrels fields.
+ */
+Node *
+add_nulling_relids(Node *node,
+ const Bitmapset *target_relids,
+ const Bitmapset *added_relids)
+{
+ add_nulling_relids_context context;
+
+ context.target_relids = target_relids;
+ context.added_relids = added_relids;
+ context.sublevels_up = 0;
+ return query_or_expression_tree_mutator(node,
+ add_nulling_relids_mutator,
+ &context,
+ 0);
+}
+
+static Node *
+add_nulling_relids_mutator(Node *node,
+ add_nulling_relids_context *context)
+{
+ if (node == NULL)
+ return NULL;
+ if (IsA(node, Var))
+ {
+ Var *var = (Var *) node;
+
+ if (var->varlevelsup == context->sublevels_up &&
+ bms_is_member(var->varno, context->target_relids))
+ {
+ Relids newnullingrels = bms_union(var->varnullingrels,
+ context->added_relids);
+
+ /* Copy the Var ... */
+ var = copyObject(var);
+ /* ... and replace the copy's varnullingrels field */
+ var->varnullingrels = newnullingrels;
+ return (Node *) var;
+ }
+ /* Otherwise fall through to copy the Var normally */
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ if (phv->phlevelsup == context->sublevels_up &&
+ bms_overlap(phv->phrels, context->target_relids))
+ {
+ Relids newnullingrels = bms_union(phv->phnullingrels,
+ context->added_relids);
+
+ /*
+ * We don't modify the contents of the PHV's expression, only add
+ * to phnullingrels. This corresponds to assuming that the PHV
+ * will be evaluated at the same level as before, then perhaps be
+ * nulled as it bubbles up. Hence, just flat-copy the node ...
+ */
+ phv = makeNode(PlaceHolderVar);
+ memcpy(phv, node, sizeof(PlaceHolderVar));
+ /* ... and replace the copy's phnullingrels field */
+ phv->phnullingrels = newnullingrels;
+ return (Node *) phv;
+ }
+ /* Otherwise fall through to copy the PlaceHolderVar normally */
+ }
+ else if (IsA(node, Query))
+ {
+ /* Recurse into RTE or sublink subquery */
+ Query *newnode;
+
+ context->sublevels_up++;
+ newnode = query_tree_mutator((Query *) node,
+ add_nulling_relids_mutator,
+ (void *) context,
+ 0);
+ context->sublevels_up--;
+ return (Node *) newnode;
+ }
+ return expression_tree_mutator(node, add_nulling_relids_mutator,
+ (void *) context);
+}
+
+/*
+ * remove_nulling_relids() removes mentions of the specified RT index(es)
+ * in Var.varnullingrels and PlaceHolderVar.phnullingrels fields within
+ * the given expression, except in nodes belonging to rels listed in
+ * except_relids.
+ */
+Node *
+remove_nulling_relids(Node *node,
+ const Bitmapset *removable_relids,
+ const Bitmapset *except_relids)
+{
+ remove_nulling_relids_context context;
+
+ context.removable_relids = removable_relids;
+ context.except_relids = except_relids;
+ context.sublevels_up = 0;
+ return query_or_expression_tree_mutator(node,
+ remove_nulling_relids_mutator,
+ &context,
+ 0);
+}
+
+static Node *
+remove_nulling_relids_mutator(Node *node,
+ remove_nulling_relids_context *context)
+{
+ if (node == NULL)
+ return NULL;
+ if (IsA(node, Var))
+ {
+ Var *var = (Var *) node;
+
+ if (var->varlevelsup == context->sublevels_up &&
+ !bms_is_member(var->varno, context->except_relids) &&
+ bms_overlap(var->varnullingrels, context->removable_relids))
+ {
+ Relids newnullingrels = bms_difference(var->varnullingrels,
+ context->removable_relids);
+
+ /* Micro-optimization: ensure nullingrels is NULL if empty */
+ if (bms_is_empty(newnullingrels))
+ newnullingrels = NULL;
+ /* Copy the Var ... */
+ var = copyObject(var);
+ /* ... and replace the copy's varnullingrels field */
+ var->varnullingrels = newnullingrels;
+ return (Node *) var;
+ }
+ /* Otherwise fall through to copy the Var normally */
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ if (phv->phlevelsup == context->sublevels_up &&
+ !bms_overlap(phv->phrels, context->except_relids))
+ {
+ Relids newnullingrels = bms_difference(phv->phnullingrels,
+ context->removable_relids);
+
+ /*
+ * Micro-optimization: ensure nullingrels is NULL if empty.
+ *
+ * Note: it might seem desirable to remove the PHV altogether if
+ * phnullingrels goes to empty. Currently we dare not do that
+ * because we use PHVs in some cases to enforce separate identity
+ * of subexpressions; see wrap_non_vars usages in prepjointree.c.
+ */
+ if (bms_is_empty(newnullingrels))
+ newnullingrels = NULL;
+ /* Copy the PlaceHolderVar and mutate what's below ... */
+ phv = (PlaceHolderVar *)
+ expression_tree_mutator(node,
+ remove_nulling_relids_mutator,
+ (void *) context);
+ /* ... and replace the copy's phnullingrels field */
+ phv->phnullingrels = newnullingrels;
+ /* We must also update phrels, if it contains a removable RTI */
+ phv->phrels = bms_difference(phv->phrels,
+ context->removable_relids);
+ Assert(!bms_is_empty(phv->phrels));
+ return (Node *) phv;
+ }
+ /* Otherwise fall through to copy the PlaceHolderVar normally */
+ }
+ else if (IsA(node, Query))
+ {
+ /* Recurse into RTE or sublink subquery */
+ Query *newnode;
+
+ context->sublevels_up++;
+ newnode = query_tree_mutator((Query *) node,
+ remove_nulling_relids_mutator,
+ (void *) context,
+ 0);
+ context->sublevels_up--;
+ return (Node *) newnode;
+ }
+ return expression_tree_mutator(node, remove_nulling_relids_mutator,
+ (void *) context);
+}
+
+
/*
* replace_rte_variables() finds all Vars in an expression tree
* that reference a particular RTE, and replaces them with substitute
diff --git a/src/backend/utils/misc/queryjumble.c b/src/backend/utils/misc/queryjumble.c
index 0ace74de78..6ae8f0ece7 100644
--- a/src/backend/utils/misc/queryjumble.c
+++ b/src/backend/utils/misc/queryjumble.c
@@ -383,6 +383,11 @@ JumbleExpr(JumbleState *jstate, Node *node)
APP_JUMB(var->varno);
APP_JUMB(var->varattno);
APP_JUMB(var->varlevelsup);
+
+ /*
+ * We can omit varnullingrels, because it's fully determined
+ * by varno/varlevelsup plus the Var's query location.
+ */
}
break;
case T_Const:
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 654dba61aa..287bd554f6 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -249,10 +249,8 @@ struct PlannerInfo
struct AppendRelInfo **append_rel_array pg_node_attr(read_write_ignore);
/*
- * all_baserels is a Relids set of all base relids (but not "other"
- * relids) in the query; that is, the Relids identifier of the final join
- * we need to form. This is computed in make_one_rel, just before we
- * start making Paths.
+ * all_baserels is a Relids set of all base relids (but not joins or
+ * "other" rels) in the query. This is computed in deconstruct_jointree.
*/
Relids all_baserels;
@@ -313,19 +311,19 @@ struct PlannerInfo
List *canon_pathkeys;
/*
- * list of RestrictInfos for mergejoinable outer join clauses
+ * list of OuterJoinClauseInfos for mergejoinable outer join clauses
* w/nonnullable var on left
*/
List *left_join_clauses;
/*
- * list of RestrictInfos for mergejoinable outer join clauses
+ * list of OuterJoinClauseInfos for mergejoinable outer join clauses
* w/nonnullable var on right
*/
List *right_join_clauses;
/*
- * list of RestrictInfos for mergejoinable full join clauses
+ * list of OuterJoinClauseInfos for mergejoinable full join clauses
*/
List *full_join_clauses;
@@ -883,7 +881,7 @@ typedef struct RelOptInfo
int32 *attr_widths pg_node_attr(read_write_ignore);
/* LATERAL Vars and PHVs referenced by rel */
List *lateral_vars;
- /* rels that reference me laterally */
+ /* rels that reference this baserel laterally */
Relids lateral_referencers;
/* list of IndexOptInfo */
List *indexlist;
@@ -893,10 +891,7 @@ typedef struct RelOptInfo
BlockNumber pages;
Cardinality tuples;
double allvisfrac;
-
- /*
- * Indexes in PlannerInfo's eq_classes list of ECs that mention this rel
- */
+ /* indexes in PlannerInfo's eq_classes list of ECs that mention this rel */
Bitmapset *eclass_indexes;
PlannerInfo *subroot; /* if subquery */
List *subplan_params; /* if subquery */
@@ -2596,10 +2591,15 @@ typedef struct MergeScanSelCache
* of a plan tree. This is used during planning to represent the contained
* expression. At the end of the planning process it is replaced by either
* the contained expression or a Var referring to a lower-level evaluation of
- * the contained expression. Typically the evaluation occurs below an outer
+ * the contained expression. Generally the evaluation occurs below an outer
* join, and Var references above the outer join might thereby yield NULL
* instead of the expression value.
*
+ * phrels and phlevelsup correspond to the varno/varlevelsup fields of a
+ * plain Var, except that phrels has to be a relid set since the evaluation
+ * level of a PlaceHolderVar might be a join rather than a base relation.
+ * Likewise, phnullingrels corresponds to varnullingrels.
+ *
* Although the planner treats this as an expression node type, it is not
* recognized by the parser or executor, so we declare it here rather than
* in primnodes.h.
@@ -2612,8 +2612,10 @@ typedef struct MergeScanSelCache
* PHV. Another way in which it can happen is that initplan sublinks
* could get replaced by differently-numbered Params when sublink folding
* is done. (The end result of such a situation would be some
- * unreferenced initplans, which is annoying but not really a problem.) On
- * the same reasoning, there is no need to examine phrels.
+ * unreferenced initplans, which is annoying but not really a problem.)
+ * On the same reasoning, there is no need to examine phrels. But we do
+ * need to compare phnullingrels, as that represents effects that are
+ * external to the original value of the PHV.
*/
typedef struct PlaceHolderVar
@@ -2623,9 +2625,12 @@ typedef struct PlaceHolderVar
/* the represented expression */
Expr *phexpr pg_node_attr(equal_ignore);
- /* base relids syntactically within expr src */
+ /* base+OJ relids syntactically within expr src */
Relids phrels pg_node_attr(equal_ignore);
+ /* RT indexes of outer joins that can null PHV's value */
+ Relids phnullingrels;
+
/* ID for PHV (unique within planner run) */
Index phid;
@@ -2714,6 +2719,21 @@ struct SpecialJoinInfo
List *semi_rhs_exprs; /* righthand-side expressions of these ops */
};
+/*
+ * Transient outer-join clause info.
+ *
+ * We set aside every outer join ON clause that looks mergejoinable,
+ * and process it specially at the end of qual distribution.
+ */
+typedef struct OuterJoinClauseInfo
+{
+ pg_node_attr(no_copy_equal, no_read)
+
+ NodeTag type;
+ RestrictInfo *rinfo; /* a mergejoinable outer-join clause */
+ SpecialJoinInfo *sjinfo; /* the outer join's SpecialJoinInfo */
+} OuterJoinClauseInfo;
+
/*
* Append-relation info.
*
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index 74f228d959..716d939abf 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -180,6 +180,14 @@ typedef struct Expr
* row identity information during UPDATE/DELETE/MERGE. This value should
* never be seen outside the planner.
*
+ * varnullingrels is the set of RT indexes of outer joins that can force
+ * the Var's value to null (at the point where it appears in the query).
+ * See optimizer/README for discussion of that.
+ *
+ * varlevelsup is greater than zero in Vars that represent outer references.
+ * Note that it affects the meaning of all of varno, varnullingrels, and
+ * varnosyn, all of which refer to the range table of that query level.
+ *
* In the parser, varnosyn and varattnosyn are either identical to
* varno/varattno, or they specify the column's position in an aliased JOIN
* RTE that hides the semantic referent RTE's refname. This is a syntactic
@@ -222,6 +230,8 @@ typedef struct Var
int32 vartypmod;
/* OID of collation, or InvalidOid if none */
Oid varcollid;
+ /* RT indexes of outer joins that can replace the Var's value with null */
+ Bitmapset *varnullingrels;
/*
* for subquery variables referencing outer relations; 0 in a normal var,
diff --git a/src/include/rewrite/rewriteManip.h b/src/include/rewrite/rewriteManip.h
index 05e6fe1f4b..a77e9980ed 100644
--- a/src/include/rewrite/rewriteManip.h
+++ b/src/include/rewrite/rewriteManip.h
@@ -65,6 +65,13 @@ extern bool contain_windowfuncs(Node *node);
extern int locate_windowfunc(Node *node);
extern bool checkExprHasSubLink(Node *node);
+extern Node *add_nulling_relids(Node *node,
+ const Bitmapset *target_relids,
+ const Bitmapset *added_relids);
+extern Node *remove_nulling_relids(Node *node,
+ const Bitmapset *removable_relids,
+ const Bitmapset *except_relids);
+
extern Node *replace_rte_variables(Node *node,
int target_varno, int sublevels_up,
replace_rte_variables_callback callback,
commit e42dd131129eadc638436083e5a82e0124db5c31
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu Dec 22 16:24:06 2022 -0500
Teach the parser to fill Var.varnullingrels correctly.
Vars emitted by the parser are now marked with RT indexes of outer
joins that can null them. (This is done purely according to the
syntax of the query; we don't consider whether an outer join could
be strength-reduced, for example.)
Although the result of this step compiles, it will fail some
regression tests due to the planner not yet knowing what to do.
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 2e593aed2b..a83dce1a8b 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -675,6 +675,7 @@ transformInsertStmt(ParseState *pstate, InsertStmt *stmt)
sub_pstate->p_rtable = sub_rtable;
sub_pstate->p_rteperminfos = sub_rteperminfos;
sub_pstate->p_joinexprs = NIL; /* sub_rtable has no joins */
+ sub_pstate->p_nullingrels = NIL;
sub_pstate->p_namespace = sub_namespace;
sub_pstate->p_resolve_unknowns = false;
@@ -856,7 +857,7 @@ transformInsertStmt(ParseState *pstate, InsertStmt *stmt)
/*
* Generate list of Vars referencing the RTE
*/
- exprList = expandNSItemVars(nsitem, 0, -1, NULL);
+ exprList = expandNSItemVars(pstate, nsitem, 0, -1, NULL);
/*
* Re-apply any indirection on the target column specs to the Vars
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index 856839f379..7f4b34d6b5 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -52,7 +52,8 @@
#include "utils/syscache.h"
-static int extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
+static int extractRemainingColumns(ParseState *pstate,
+ ParseNamespaceColumn *src_nscolumns,
List *src_colnames,
List **src_colnos,
List **res_colnames, List **res_colvars,
@@ -75,9 +76,11 @@ static ParseNamespaceItem *getNSItemForSpecialRelationTypes(ParseState *pstate,
static Node *transformFromClauseItem(ParseState *pstate, Node *n,
ParseNamespaceItem **top_nsitem,
List **namespace);
-static Var *buildVarFromNSColumn(ParseNamespaceColumn *nscol);
+static Var *buildVarFromNSColumn(ParseState *pstate,
+ ParseNamespaceColumn *nscol);
static Node *buildMergedJoinVar(ParseState *pstate, JoinType jointype,
Var *l_colvar, Var *r_colvar);
+static void markRelsAsNulledBy(ParseState *pstate, Node *n, int jindex);
static void setNamespaceColumnVisibility(List *namespace, bool cols_visible);
static void setNamespaceLateralState(List *namespace,
bool lateral_only, bool lateral_ok);
@@ -251,7 +254,8 @@ setTargetTable(ParseState *pstate, RangeVar *relation,
* Returns the number of columns added.
*/
static int
-extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
+extractRemainingColumns(ParseState *pstate,
+ ParseNamespaceColumn *src_nscolumns,
List *src_colnames,
List **src_colnos,
List **res_colnames, List **res_colvars,
@@ -287,7 +291,8 @@ extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
*src_colnos = lappend_int(*src_colnos, attnum);
*res_colnames = lappend(*res_colnames, lfirst(lc));
*res_colvars = lappend(*res_colvars,
- buildVarFromNSColumn(src_nscolumns + attnum - 1));
+ buildVarFromNSColumn(pstate,
+ src_nscolumns + attnum - 1));
/* Copy the input relation's nscolumn data for this column */
res_nscolumns[colcount] = src_nscolumns[attnum - 1];
colcount++;
@@ -1288,8 +1293,7 @@ transformFromClauseItem(ParseState *pstate, Node *n,
{
/*
* JOIN/USING (or NATURAL JOIN, as transformed above). Transform
- * the list into an explicit ON-condition, and generate a list of
- * merged result columns.
+ * the list into an explicit ON-condition.
*/
List *ucols = j->usingClause;
List *l_usingvars = NIL;
@@ -1307,8 +1311,6 @@ transformFromClauseItem(ParseState *pstate, Node *n,
int r_index = -1;
Var *l_colvar,
*r_colvar;
- Node *u_colvar;
- ParseNamespaceColumn *res_nscolumn;
Assert(u_colname[0] != '\0');
@@ -1372,17 +1374,109 @@ transformFromClauseItem(ParseState *pstate, Node *n,
u_colname)));
r_colnos = lappend_int(r_colnos, r_index + 1);
- l_colvar = buildVarFromNSColumn(l_nscolumns + l_index);
+ /* Build Vars to use in the generated JOIN ON clause */
+ l_colvar = buildVarFromNSColumn(pstate, l_nscolumns + l_index);
l_usingvars = lappend(l_usingvars, l_colvar);
- r_colvar = buildVarFromNSColumn(r_nscolumns + r_index);
+ r_colvar = buildVarFromNSColumn(pstate, r_nscolumns + r_index);
r_usingvars = lappend(r_usingvars, r_colvar);
+ /*
+ * While we're here, add column names to the res_colnames
+ * list. It's a bit ugly to do this here while the
+ * corresponding res_colvars entries are not made till later,
+ * but doing this later would require an additional traversal
+ * of the usingClause list.
+ */
res_colnames = lappend(res_colnames, lfirst(ucol));
+ }
+
+ /* Construct the generated JOIN ON clause */
+ j->quals = transformJoinUsingClause(pstate,
+ l_usingvars,
+ r_usingvars);
+ }
+ else if (j->quals)
+ {
+ /* User-written ON-condition; transform it */
+ j->quals = transformJoinOnClause(pstate, j, my_namespace);
+ }
+ else
+ {
+ /* CROSS JOIN: no quals */
+ }
+
+ /*
+ * If this is an outer join, now mark the appropriate child RTEs as
+ * being nulled by this join. We have finished processing the child
+ * join expressions as well as the current join's quals, which deal in
+ * non-nulled input columns. All future references to those RTEs will
+ * see possibly-nulled values, and we should mark generated Vars to
+ * account for that. In particular, the join alias Vars that we're
+ * about to build should reflect the nulling effects of this join.
+ *
+ * A difficulty with doing this is that we need the join's RT index,
+ * which we don't officially have yet. However, no other RTE can get
+ * made between here and the addRangeTableEntryForJoin call, so we can
+ * predict what the assignment will be. (Alternatively, we could call
+ * addRangeTableEntryForJoin before we have all the data computed, but
+ * this seems less ugly.)
+ */
+ j->rtindex = list_length(pstate->p_rtable) + 1;
+
+ switch (j->jointype)
+ {
+ case JOIN_INNER:
+ break;
+ case JOIN_LEFT:
+ markRelsAsNulledBy(pstate, j->rarg, j->rtindex);
+ break;
+ case JOIN_FULL:
+ markRelsAsNulledBy(pstate, j->larg, j->rtindex);
+ markRelsAsNulledBy(pstate, j->rarg, j->rtindex);
+ break;
+ case JOIN_RIGHT:
+ markRelsAsNulledBy(pstate, j->larg, j->rtindex);
+ break;
+ default:
+ /* shouldn't see any other types here */
+ elog(ERROR, "unrecognized join type: %d",
+ (int) j->jointype);
+ break;
+ }
+
+ /*
+ * Now we can construct join alias expressions for the USING columns.
+ */
+ if (j->usingClause)
+ {
+ ListCell *lc1,
+ *lc2;
+
+ /* Scan the colnos lists to recover info from the previous loop */
+ forboth(lc1, l_colnos, lc2, r_colnos)
+ {
+ int l_index = lfirst_int(lc1) - 1;
+ int r_index = lfirst_int(lc2) - 1;
+ Var *l_colvar,
+ *r_colvar;
+ Node *u_colvar;
+ ParseNamespaceColumn *res_nscolumn;
+
+ /*
+ * Note we re-build these Vars: they might have different
+ * varnullingrels than the ones made in the previous loop.
+ */
+ l_colvar = buildVarFromNSColumn(pstate, l_nscolumns + l_index);
+ r_colvar = buildVarFromNSColumn(pstate, r_nscolumns + r_index);
+
+ /* Construct the join alias Var for this column */
u_colvar = buildMergedJoinVar(pstate,
j->jointype,
l_colvar,
r_colvar);
res_colvars = lappend(res_colvars, u_colvar);
+
+ /* Construct column's res_nscolumns[] entry */
res_nscolumn = res_nscolumns + res_colindex;
res_colindex++;
if (u_colvar == (Node *) l_colvar)
@@ -1400,47 +1494,45 @@ transformFromClauseItem(ParseState *pstate, Node *n,
/*
* Merged column is not semantically equivalent to either
* input, so it needs to be referenced as the join output
- * column. We don't know the join's varno yet, so we'll
- * replace these zeroes below.
+ * column.
*/
- res_nscolumn->p_varno = 0;
+ res_nscolumn->p_varno = j->rtindex;
res_nscolumn->p_varattno = res_colindex;
res_nscolumn->p_vartype = exprType(u_colvar);
res_nscolumn->p_vartypmod = exprTypmod(u_colvar);
res_nscolumn->p_varcollid = exprCollation(u_colvar);
- res_nscolumn->p_varnosyn = 0;
+ res_nscolumn->p_varnosyn = j->rtindex;
res_nscolumn->p_varattnosyn = res_colindex;
}
}
-
- j->quals = transformJoinUsingClause(pstate,
- l_usingvars,
- r_usingvars);
- }
- else if (j->quals)
- {
- /* User-written ON-condition; transform it */
- j->quals = transformJoinOnClause(pstate, j, my_namespace);
- }
- else
- {
- /* CROSS JOIN: no quals */
}
/* Add remaining columns from each side to the output columns */
res_colindex +=
- extractRemainingColumns(l_nscolumns, l_colnames, &l_colnos,
+ extractRemainingColumns(pstate,
+ l_nscolumns, l_colnames, &l_colnos,
&res_colnames, &res_colvars,
res_nscolumns + res_colindex);
res_colindex +=
- extractRemainingColumns(r_nscolumns, r_colnames, &r_colnos,
+ extractRemainingColumns(pstate,
+ r_nscolumns, r_colnames, &r_colnos,
&res_colnames, &res_colvars,
res_nscolumns + res_colindex);
+ /* If join has an alias, it syntactically hides all inputs */
+ if (j->alias)
+ {
+ for (k = 0; k < res_colindex; k++)
+ {
+ ParseNamespaceColumn *nscol = res_nscolumns + k;
+
+ nscol->p_varnosyn = j->rtindex;
+ nscol->p_varattnosyn = k + 1;
+ }
+ }
+
/*
* Now build an RTE and nsitem for the result of the join.
- * res_nscolumns isn't totally done yet, but that's OK because
- * addRangeTableEntryForJoin doesn't examine it, only store a pointer.
*/
nsitem = addRangeTableEntryForJoin(pstate,
res_colnames,
@@ -1454,31 +1546,16 @@ transformFromClauseItem(ParseState *pstate, Node *n,
j->alias,
true);
- j->rtindex = nsitem->p_rtindex;
+ /* Verify that we correctly predicted the join's RT index */
+ Assert(j->rtindex == nsitem->p_rtindex);
+ /* Cross-check number of columns, too */
+ Assert(res_colindex == list_length(nsitem->p_names->colnames));
/*
- * Now that we know the join RTE's rangetable index, we can fix up the
- * res_nscolumns data in places where it should contain that.
+ * Save a link to the JoinExpr in the proper element of p_joinexprs.
+ * Since we maintain that list lazily, it may be necessary to fill in
+ * empty entries before we can add the JoinExpr in the right place.
*/
- Assert(res_colindex == list_length(nsitem->p_names->colnames));
- for (k = 0; k < res_colindex; k++)
- {
- ParseNamespaceColumn *nscol = res_nscolumns + k;
-
- /* fill in join RTI for merged columns */
- if (nscol->p_varno == 0)
- nscol->p_varno = j->rtindex;
- if (nscol->p_varnosyn == 0)
- nscol->p_varnosyn = j->rtindex;
- /* if join has an alias, it syntactically hides all inputs */
- if (j->alias)
- {
- nscol->p_varnosyn = j->rtindex;
- nscol->p_varattnosyn = k + 1;
- }
- }
-
- /* make a matching link to the JoinExpr for later use */
for (k = list_length(pstate->p_joinexprs) + 1; k < j->rtindex; k++)
pstate->p_joinexprs = lappend(pstate->p_joinexprs, NULL);
pstate->p_joinexprs = lappend(pstate->p_joinexprs, j);
@@ -1547,10 +1624,13 @@ transformFromClauseItem(ParseState *pstate, Node *n,
* buildVarFromNSColumn -
* build a Var node using ParseNamespaceColumn data
*
- * We assume varlevelsup should be 0, and no location is specified
+ * This is used to construct joinaliasvars entries.
+ * We can assume varlevelsup should be 0, and no location is specified.
+ * Note also that no column SELECT privilege is requested here; that would
+ * happen only if the column is actually referenced in the query.
*/
static Var *
-buildVarFromNSColumn(ParseNamespaceColumn *nscol)
+buildVarFromNSColumn(ParseState *pstate, ParseNamespaceColumn *nscol)
{
Var *var;
@@ -1564,6 +1644,10 @@ buildVarFromNSColumn(ParseNamespaceColumn *nscol)
/* makeVar doesn't offer parameters for these, so set by hand: */
var->varnosyn = nscol->p_varnosyn;
var->varattnosyn = nscol->p_varattnosyn;
+
+ /* ... and update varnullingrels */
+ markNullableIfNeeded(pstate, var);
+
return var;
}
@@ -1675,6 +1759,47 @@ buildMergedJoinVar(ParseState *pstate, JoinType jointype,
return res_node;
}
+/*
+ * markRelsAsNulledBy -
+ * Mark the given jointree node and its children as nulled by join jindex
+ */
+static void
+markRelsAsNulledBy(ParseState *pstate, Node *n, int jindex)
+{
+ int varno;
+ ListCell *lc;
+
+ /* Note: we can't see FromExpr here */
+ if (IsA(n, RangeTblRef))
+ {
+ varno = ((RangeTblRef *) n)->rtindex;
+ }
+ else if (IsA(n, JoinExpr))
+ {
+ JoinExpr *j = (JoinExpr *) n;
+
+ /* recurse to children */
+ markRelsAsNulledBy(pstate, j->larg, jindex);
+ markRelsAsNulledBy(pstate, j->rarg, jindex);
+ varno = j->rtindex;
+ }
+ else
+ {
+ elog(ERROR, "unrecognized node type: %d", (int) nodeTag(n));
+ varno = 0; /* keep compiler quiet */
+ }
+
+ /*
+ * Now add jindex to the p_nullingrels set for relation varno. Since we
+ * maintain the p_nullingrels list lazily, we might need to extend it to
+ * make the varno'th entry exist.
+ */
+ while (list_length(pstate->p_nullingrels) < varno)
+ pstate->p_nullingrels = lappend(pstate->p_nullingrels, NULL);
+ lc = list_nth_cell(pstate->p_nullingrels, varno - 1);
+ lfirst(lc) = bms_add_member((Bitmapset *) lfirst(lc), jindex);
+}
+
/*
* setNamespaceColumnVisibility -
* Convenience subroutine to update cols_visible flags in a namespace list.
diff --git a/src/backend/parser/parse_coerce.c b/src/backend/parser/parse_coerce.c
index 60908111c8..606491bd66 100644
--- a/src/backend/parser/parse_coerce.c
+++ b/src/backend/parser/parse_coerce.c
@@ -1042,7 +1042,7 @@ coerce_record_to_complex(ParseState *pstate, Node *node,
ParseNamespaceItem *nsitem;
nsitem = GetNSItemByRangeTablePosn(pstate, rtindex, sublevels_up);
- args = expandNSItemVars(nsitem, sublevels_up, vlocation, NULL);
+ args = expandNSItemVars(pstate, nsitem, sublevels_up, vlocation, NULL);
}
else
ereport(ERROR,
diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c
index 150a8099c2..107ba6ef81 100644
--- a/src/backend/parser/parse_expr.c
+++ b/src/backend/parser/parse_expr.c
@@ -2478,6 +2478,9 @@ transformWholeRowRef(ParseState *pstate, ParseNamespaceItem *nsitem,
/* location is not filled in by makeWholeRowVar */
result->location = location;
+ /* mark Var if it's nulled by any outer joins */
+ markNullableIfNeeded(pstate, result);
+
/* mark relation as requiring whole-row SELECT access */
markVarForSelectPriv(pstate, result);
@@ -2505,6 +2508,8 @@ transformWholeRowRef(ParseState *pstate, ParseNamespaceItem *nsitem,
rowexpr->colnames = copyObject(nsitem->p_names->colnames);
rowexpr->location = location;
+ /* XXX we ought to mark the row as possibly nullable */
+
return (Node *) rowexpr;
}
}
diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c
index b4878a92ea..40d16e3cd4 100644
--- a/src/backend/parser/parse_relation.c
+++ b/src/backend/parser/parse_relation.c
@@ -763,6 +763,9 @@ scanNSItemForColumn(ParseState *pstate, ParseNamespaceItem *nsitem,
}
var->location = location;
+ /* Mark Var if it's nulled by any outer joins */
+ markNullableIfNeeded(pstate, var);
+
/* Require read access to the column */
markVarForSelectPriv(pstate, var);
@@ -1023,6 +1026,35 @@ searchRangeTableForCol(ParseState *pstate, const char *alias, const char *colnam
return fuzzystate;
}
+/*
+ * markNullableIfNeeded
+ * If the RTE referenced by the Var is nullable by outer join(s)
+ * at this point in the query, set var->varnullingrels to show that.
+ */
+void
+markNullableIfNeeded(ParseState *pstate, Var *var)
+{
+ int rtindex = var->varno;
+ Bitmapset *relids;
+
+ /* Find the appropriate pstate */
+ for (int lv = 0; lv < var->varlevelsup; lv++)
+ pstate = pstate->parentParseState;
+
+ /* Find currently-relevant join relids for the Var's rel */
+ if (rtindex > 0 && rtindex <= list_length(pstate->p_nullingrels))
+ relids = (Bitmapset *) list_nth(pstate->p_nullingrels, rtindex - 1);
+ else
+ relids = NULL;
+
+ /*
+ * Merge with any already-declared nulling rels. (Typically there won't
+ * be any, but let's get it right if there are.)
+ */
+ if (relids != NULL)
+ var->varnullingrels = bms_union(var->varnullingrels, relids);
+}
+
/*
* markRTEForSelectPriv
* Mark the specified column of the RTE with index rtindex
@@ -3087,7 +3119,7 @@ expandTupleDesc(TupleDesc tupdesc, Alias *eref, int count, int offset,
* the list elements mustn't be modified.
*/
List *
-expandNSItemVars(ParseNamespaceItem *nsitem,
+expandNSItemVars(ParseState *pstate, ParseNamespaceItem *nsitem,
int sublevels_up, int location,
List **colnames)
{
@@ -3123,6 +3155,10 @@ expandNSItemVars(ParseNamespaceItem *nsitem,
var->varnosyn = nscol->p_varnosyn;
var->varattnosyn = nscol->p_varattnosyn;
var->location = location;
+
+ /* ... and update varnullingrels */
+ markNullableIfNeeded(pstate, var);
+
result = lappend(result, var);
if (colnames)
*colnames = lappend(*colnames, colnameval);
@@ -3158,7 +3194,7 @@ expandNSItemAttrs(ParseState *pstate, ParseNamespaceItem *nsitem,
*var;
List *te_list = NIL;
- vars = expandNSItemVars(nsitem, sublevels_up, location, &names);
+ vars = expandNSItemVars(pstate, nsitem, sublevels_up, location, &names);
/*
* Require read access to the table. This is normally redundant with the
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index 56d64c8851..75f08de3a1 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -1371,7 +1371,7 @@ ExpandSingleTable(ParseState *pstate, ParseNamespaceItem *nsitem,
List *vars;
ListCell *l;
- vars = expandNSItemVars(nsitem, sublevels_up, location, NULL);
+ vars = expandNSItemVars(pstate, nsitem, sublevels_up, location, NULL);
/*
* Require read access to the table. This is normally redundant with
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 34bc640ff2..36c2894512 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1057,6 +1057,14 @@ typedef struct RangeTblEntry
* alias Vars are generated only for merged columns). We keep these
* entries only because they're needed in expandRTE() and similar code.
*
+ * Vars appearing within joinaliasvars are marked with varnullingrels sets
+ * that describe the nulling effects of this join and lower ones. This is
+ * essential for FULL JOIN cases, because the COALESCE expression only
+ * describes the semantics correctly if its inputs have been nulled by the
+ * join. For other cases, it allows expandRTE() to generate a valid
+ * representation of the join's output without consulting additional
+ * parser state.
+ *
* Within a Query loaded from a stored rule, it is possible for non-merged
* joinaliasvars items to be null pointers, which are placeholders for
* (necessarily unreferenced) columns dropped since the rule was made.
diff --git a/src/include/parser/parse_node.h b/src/include/parser/parse_node.h
index 3fd56ceccd..050774b851 100644
--- a/src/include/parser/parse_node.h
+++ b/src/include/parser/parse_node.h
@@ -118,6 +118,13 @@ typedef Node *(*CoerceParamHook) (ParseState *pstate, Param *param,
* This is one-for-one with p_rtable, but contains NULLs for non-join
* RTEs, and may be shorter than p_rtable if the last RTE(s) aren't joins.
*
+ * p_nullingrels: list of Bitmapsets associated with p_rtable entries, each
+ * containing the set of outer-join RTE indexes that can null that relation
+ * at the current point in the parse tree. This is one-for-one with p_rtable,
+ * but may be shorter than p_rtable, in which case the missing entries are
+ * implicitly empty (NULL). That rule allows us to save work when the query
+ * contains no outer joins.
+ *
* p_joinlist: list of join items (RangeTblRef and JoinExpr nodes) that
* will become the fromlist of the query's top-level FromExpr node.
*
@@ -187,6 +194,7 @@ struct ParseState
List *p_rteperminfos; /* list of RTEPermissionInfo nodes for each
* RTE_RELATION entry in rtable */
List *p_joinexprs; /* JoinExprs for RTE_JOIN p_rtable entries */
+ List *p_nullingrels; /* Bitmapsets showing nulling outer joins */
List *p_joinlist; /* join items so far (will become FromExpr
* node's fromlist) */
List *p_namespace; /* currently-referenceable RTEs (List of
diff --git a/src/include/parser/parse_relation.h b/src/include/parser/parse_relation.h
index 2f8d417709..1a2ab769d8 100644
--- a/src/include/parser/parse_relation.h
+++ b/src/include/parser/parse_relation.h
@@ -41,6 +41,7 @@ extern Node *scanNSItemForColumn(ParseState *pstate, ParseNamespaceItem *nsitem,
int location);
extern Node *colNameToVar(ParseState *pstate, const char *colname, bool localonly,
int location);
+extern void markNullableIfNeeded(ParseState *pstate, Var *var);
extern void markVarForSelectPriv(ParseState *pstate, Var *var);
extern Relation parserOpenTable(ParseState *pstate, const RangeVar *relation,
int lockmode);
@@ -113,7 +114,7 @@ extern void errorMissingColumn(ParseState *pstate,
extern void expandRTE(RangeTblEntry *rte, int rtindex, int sublevels_up,
int location, bool include_dropped,
List **colnames, List **colvars);
-extern List *expandNSItemVars(ParseNamespaceItem *nsitem,
+extern List *expandNSItemVars(ParseState *pstate, ParseNamespaceItem *nsitem,
int sublevels_up, int location,
List **colnames);
extern List *expandNSItemAttrs(ParseState *pstate, ParseNamespaceItem *nsitem,
commit 90559097c8b6397c4457c55838aa6ebbcf0732de
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu Dec 22 18:38:33 2022 -0500
Teach the planner to cope with Vars bearing nullingrels.
The core idea of this step is to include varnullingrels in the
relid sets that qual clauses are considered to depend on.
So that we can still easily compare quals' relids to RelOptInfos'
relids, that means also adding outer join relids to the identifying
relids of join relations. Much of the bulk of this step is concerned
with fallout from the latter change.
I've resolved the previous squishiness entailed by outer join identity 3
by generating multiple versions of outer-join quals that could get moved
to a join level where they need to contain different nullingrels sets.
Now we have versions of such quals with the correct nullingrels for
each level where they could appear.
This requires a bit of new mechanism (RestrictInfo.has_clone/is_clone)
to prevent multiple versions of the same qual from getting used in the
plan. My worry about how that could work with EquivalenceClasses is
resolved by creating EquivalenceClasses only from the least-marked
version of a qual. (This doesn't really lose anything, since versions
with more nullingrels bits don't correspond to any equalities available
outside the nest of commuting outer joins.)
These extra versions of quals would also result in generating multiple
parameterized paths that differ only in what nullingrels they expect
for the Vars from the parameterization rel(s). That seems like it'd
be very wasteful, so I've arranged to generate such paths only from
the least-marked version of a qual (the has_clone version).
Unlike in the previous version of this patch, setrefs.c is able to
cross-check the nullingrel sets of most Vars and PlaceHolderVars to
ensure that they match up with what the previous plan step produces.
But there are three cases that I've so far punted on:
1. The targetlist and qpqual of an outer join node will contain
nullingrels bits for the outer join itself. To check exact matching to
the input, we'd need to know the OJ's relid as well as which input(s)
got nulled, neither of which is cheaply available in setrefs.c. For
now, it's just checking that such Vars have a superset of the input's
nullingrels bits.
2. Parameterized paths will generally refer to the least-marked version
of whichever outer-side Vars they use, which may not be what's actually
available from the outside of the nestloop. (We're relying on the join
ordering rules for that to be sensible.) Again, setrefs.c is in no
position to pass judgment on correctness, so it's just checking that
the parameter expression has a subset of the outer-side marking.
3. Row identity variables are not marked with any nullingrels, which
may not correspond to reality. I've punted on this by skipping the
checks when varattno <= 0.
Point 1 could be addressed if we were willing to add informational
fields to join plan nodes, which might be worth doing, but I'm not sure.
The other two points seem like the extra mechanisms needed for a
bulletproof check would be considerably more trouble than they'd be
worth.
There is still some confusion about which versions of a cloned qual
are actually necessary to check, which results in some extra filter
conditions showing up in a couple of regression test plans. There are
also some failure cases involving full joins that remain to be fixed.
This patch is already mighty big, so I'll address those failures
separately.
This step removes some low-hanging fruit from the old implementation,
such as the need to track lowest_nulling_outer_join during subquery
pullup. There's much more to do in that line, though.
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index 004481d608..1c921879a9 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -273,7 +273,7 @@ merge_clump(PlannerInfo *root, List *clumps, Clump *new_clump, int num_gene,
* rel once we know the final targetlist (see
* grouping_planner).
*/
- if (!bms_equal(joinrel->relids, root->all_baserels))
+ if (!bms_equal(joinrel->relids, root->all_query_rels))
generate_useful_gather_paths(root, joinrel, false);
/* Find and save the cheapest paths for this joinrel */
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 10ac01ac36..d81b09add5 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -211,9 +211,9 @@ make_one_rel(PlannerInfo *root, List *joinlist)
rel = make_rel_from_joinlist(root, joinlist);
/*
- * The result should join all and only the query's base rels.
+ * The result should join all and only the query's base + outer-join rels.
*/
- Assert(bms_equal(rel->relids, root->all_baserels));
+ Assert(bms_equal(rel->relids, root->all_query_rels));
return rel;
}
@@ -538,7 +538,7 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* the final scan/join targetlist is available (see grouping_planner).
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- !bms_equal(rel->relids, root->all_baserels))
+ !bms_equal(rel->relids, root->all_query_rels))
generate_useful_gather_paths(root, rel, false);
/* Now find the cheapest of the paths for this rel */
@@ -859,7 +859,7 @@ set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *
* to support an uncommon usage of second-rate sampling methods. Instead,
* if there is a risk that the query might perform an unsafe join, just
* wrap the SampleScan in a Materialize node. We can check for joins by
- * counting the membership of all_baserels (note that this correctly
+ * counting the membership of all_query_rels (note that this correctly
* counts inheritance trees as single rels). If we're inside a subquery,
* we can't easily check whether a join might occur in the outer query, so
* just assume one is possible.
@@ -868,7 +868,7 @@ set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *
* so check repeatable_across_scans last, even though that's a bit odd.
*/
if ((root->query_level > 1 ||
- bms_membership(root->all_baserels) != BMS_SINGLETON) &&
+ bms_membership(root->all_query_rels) != BMS_SINGLETON) &&
!(GetTsmRoutine(rte->tablesample->tsmhandler)->repeatable_across_scans))
{
path = (Path *) create_material_path(rel, path);
@@ -950,7 +950,7 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
if (enable_partitionwise_join &&
rel->reloptkind == RELOPT_BASEREL &&
rte->relkind == RELKIND_PARTITIONED_TABLE &&
- rel->attr_needed[InvalidAttrNumber - rel->min_attr] == NULL)
+ bms_is_empty(rel->attr_needed[InvalidAttrNumber - rel->min_attr]))
rel->consider_partitionwise_join = true;
/*
@@ -3409,7 +3409,7 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
* partial paths. We'll do the same for the topmost scan/join rel
* once we know the final targetlist (see grouping_planner).
*/
- if (!bms_equal(rel->relids, root->all_baserels))
+ if (!bms_equal(rel->relids, root->all_query_rels))
generate_useful_gather_paths(root, rel, false);
/* Find and save the cheapest paths for this rel */
diff --git a/src/backend/optimizer/path/clausesel.c b/src/backend/optimizer/path/clausesel.c
index 06f836308d..c08eb2b1c5 100644
--- a/src/backend/optimizer/path/clausesel.c
+++ b/src/backend/optimizer/path/clausesel.c
@@ -218,7 +218,7 @@ clauselist_selectivity_ext(PlannerInfo *root,
if (rinfo)
{
- ok = (bms_membership(rinfo->clause_relids) == BMS_SINGLETON) &&
+ ok = (rinfo->num_base_rels == 1) &&
(is_pseudo_constant_clause_relids(lsecond(expr->args),
rinfo->right_relids) ||
(varonleft = false,
@@ -579,30 +579,6 @@ find_single_rel_for_clauses(PlannerInfo *root, List *clauses)
return NULL; /* no clauses */
}
-/*
- * bms_is_subset_singleton
- *
- * Same result as bms_is_subset(s, bms_make_singleton(x)),
- * but a little faster and doesn't leak memory.
- *
- * Is this of use anywhere else? If so move to bitmapset.c ...
- */
-static bool
-bms_is_subset_singleton(const Bitmapset *s, int x)
-{
- switch (bms_membership(s))
- {
- case BMS_EMPTY_SET:
- return true;
- case BMS_SINGLETON:
- return bms_is_member(x, s);
- case BMS_MULTIPLE:
- return false;
- }
- /* can't get here... */
- return false;
-}
-
/*
* treat_as_join_clause -
* Decide whether an operator clause is to be handled by the
@@ -631,17 +607,20 @@ treat_as_join_clause(PlannerInfo *root, Node *clause, RestrictInfo *rinfo,
else
{
/*
- * Otherwise, it's a join if there's more than one relation used. We
- * can optimize this calculation if an rinfo was passed.
+ * Otherwise, it's a join if there's more than one base relation used.
+ * We can optimize this calculation if an rinfo was passed.
*
* XXX Since we know the clause is being evaluated at a join, the
* only way it could be single-relation is if it was delayed by outer
- * joins. Although we can make use of the restriction qual estimators
- * anyway, it seems likely that we ought to account for the
- * probability of injected nulls somehow.
+ * joins. We intentionally count only baserels here, not OJs that
+ * might be present in rinfo->clause_relids, so that we direct such
+ * cases to the restriction qual estimators not join estimators.
+ * Eventually some notice should be taken of the possibility of
+ * injected nulls, but we'll likely want to do that in the restriction
+ * estimators rather than starting to treat such cases as join quals.
*/
if (rinfo)
- return (bms_membership(rinfo->clause_relids) == BMS_MULTIPLE);
+ return (rinfo->num_base_rels > 1);
else
return (NumRelids(root, clause) > 1);
}
@@ -754,7 +733,9 @@ clause_selectivity_ext(PlannerInfo *root,
* for all non-JOIN_INNER cases.
*/
if (varRelid == 0 ||
- bms_is_subset_singleton(rinfo->clause_relids, varRelid))
+ rinfo->num_base_rels == 0 ||
+ (rinfo->num_base_rels == 1 &&
+ bms_is_member(varRelid, rinfo->clause_relids)))
{
/* Cacheable --- do we already have the result? */
if (jointype == JOIN_INNER)
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 89d3c4352c..92c0644d14 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4781,6 +4781,10 @@ compute_semi_anti_join_factors(PlannerInfo *root,
norm_sjinfo.syn_lefthand = outerrel->relids;
norm_sjinfo.syn_righthand = innerrel->relids;
norm_sjinfo.jointype = JOIN_INNER;
+ norm_sjinfo.ojrelid = 0;
+ norm_sjinfo.commute_above_l = NULL;
+ norm_sjinfo.commute_above_r = NULL;
+ norm_sjinfo.commute_below = NULL;
/* we don't bother trying to make the remaining fields valid */
norm_sjinfo.lhs_strict = false;
norm_sjinfo.delay_upper_joins = false;
@@ -4946,6 +4950,10 @@ approx_tuple_count(PlannerInfo *root, JoinPath *path, List *quals)
sjinfo.syn_lefthand = path->outerjoinpath->parent->relids;
sjinfo.syn_righthand = path->innerjoinpath->parent->relids;
sjinfo.jointype = JOIN_INNER;
+ sjinfo.ojrelid = 0;
+ sjinfo.commute_above_l = NULL;
+ sjinfo.commute_above_r = NULL;
+ sjinfo.commute_below = NULL;
/* we don't bother trying to make the remaining fields valid */
sjinfo.lhs_strict = false;
sjinfo.delay_upper_joins = false;
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 019972cefc..055d70b8e3 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -29,6 +29,7 @@
#include "optimizer/paths.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "rewrite/rewriteManip.h"
#include "utils/lsyscache.h"
@@ -757,6 +758,12 @@ get_eclass_for_sort_expr(PlannerInfo *root,
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL) /* must be an outer join */
+ {
+ Assert(bms_is_member(i, root->outer_join_rels));
+ continue;
+ }
+
Assert(rel->reloptkind == RELOPT_BASEREL ||
rel->reloptkind == RELOPT_DEADREL);
@@ -1113,6 +1120,12 @@ generate_base_implied_equalities(PlannerInfo *root)
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL) /* must be an outer join */
+ {
+ Assert(bms_is_member(i, root->outer_join_rels));
+ continue;
+ }
+
Assert(rel->reloptkind == RELOPT_BASEREL);
rel->eclass_indexes = bms_add_member(rel->eclass_indexes,
@@ -2195,6 +2208,8 @@ static bool
reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
{
RestrictInfo *rinfo = ojcinfo->rinfo;
+ SpecialJoinInfo *sjinfo = ojcinfo->sjinfo;
+ Relids fjrelids = bms_make_singleton(sjinfo->ojrelid);
Expr *leftvar;
Expr *rightvar;
Oid opno,
@@ -2276,6 +2291,18 @@ reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
cfirst = (Node *) linitial(cexpr->args);
csecond = (Node *) lsecond(cexpr->args);
+ /*
+ * The COALESCE arguments will be marked as possibly nulled by
+ * the full join, while we wish to generate clauses that apply
+ * to the join's inputs. So we must strip the join from the
+ * nullingrels fields of cfirst/csecond before comparing them
+ * to leftvar/rightvar. (Perhaps with a less hokey
+ * representation for FULL JOIN USING output columns, this
+ * wouldn't be needed?)
+ */
+ cfirst = remove_nulling_relids(cfirst, fjrelids, NULL);
+ csecond = remove_nulling_relids(csecond, fjrelids, NULL);
+
if (equal(leftvar, cfirst) && equal(rightvar, csecond))
{
coal_idx = foreach_current_index(lc2);
@@ -3212,6 +3239,12 @@ get_eclass_indexes_for_relids(PlannerInfo *root, Relids relids)
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL) /* must be an outer join */
+ {
+ Assert(bms_is_member(i, root->outer_join_rels));
+ continue;
+ }
+
ec_indexes = bms_add_members(ec_indexes, rel->eclass_indexes);
}
return ec_indexes;
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index 914bfd90bc..e24a9c14a9 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -3352,13 +3352,13 @@ check_index_predicates(PlannerInfo *root, RelOptInfo *rel)
* Add on any equivalence-derivable join clauses. Computing the correct
* relid sets for generate_join_implied_equalities is slightly tricky
* because the rel could be a child rel rather than a true baserel, and in
- * that case we must remove its parents' relid(s) from all_baserels.
+ * that case we must subtract its parents' relid(s) from all_query_rels.
*/
if (rel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- otherrels = bms_difference(root->all_baserels,
+ otherrels = bms_difference(root->all_query_rels,
find_childrel_parents(root, rel));
else
- otherrels = bms_difference(root->all_baserels, rel->relids);
+ otherrels = bms_difference(root->all_query_rels, rel->relids);
if (!bms_is_empty(otherrels))
clauselist =
@@ -3736,7 +3736,8 @@ match_index_to_operand(Node *operand,
*/
if (operand && IsA(operand, Var) &&
index->rel->relid == ((Var *) operand)->varno &&
- indkey == ((Var *) operand)->varattno)
+ indkey == ((Var *) operand)->varattno &&
+ ((Var *) operand)->varnullingrels == NULL)
return true;
}
else
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 4d09881259..3d7ee7dfe5 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -234,7 +234,9 @@ add_paths_to_joinrel(PlannerInfo *root,
* reduces the number of parameterized paths we have to deal with at
* higher join levels, without compromising the quality of the resulting
* plan. We express the restriction as a Relids set that must overlap the
- * parameterization of any proposed join path.
+ * parameterization of any proposed join path. Note: param_source_rels
+ * should contain only baserels, not OJ relids, so starting from
+ * all_baserels not all_query_rels is correct.
*/
foreach(lc, root->join_info_list)
{
@@ -365,6 +367,47 @@ allow_star_schema_join(PlannerInfo *root,
bms_nonempty_difference(inner_paramrels, outerrelids));
}
+/*
+ * If the parameterization is only partly satisfied by the outer rel,
+ * the unsatisfied part can't include any outer-join relids that could
+ * null rels of the satisfied part. That would imply that we're trying
+ * to use a clause involving a Var with nonempty varnullingrels at
+ * a join level where that value isn't yet computable.
+ */
+static inline bool
+have_unsafe_outer_join_ref(PlannerInfo *root,
+ Relids outerrelids,
+ Relids inner_paramrels)
+{
+ bool result = false;
+ Relids unsatisfied = bms_difference(inner_paramrels, outerrelids);
+
+ if (bms_overlap(unsatisfied, root->outer_join_rels))
+ {
+ ListCell *lc;
+
+ foreach(lc, root->join_info_list)
+ {
+ SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+
+ if (!bms_is_member(sjinfo->ojrelid, unsatisfied))
+ continue; /* not relevant */
+ if (bms_overlap(inner_paramrels, sjinfo->min_righthand) ||
+ (sjinfo->jointype == JOIN_FULL &&
+ bms_overlap(inner_paramrels, sjinfo->min_lefthand)))
+ {
+ result = true; /* doesn't work */
+ break;
+ }
+ }
+ }
+
+ /* Waste no memory when we reject a path here */
+ bms_free(unsatisfied);
+
+ return result;
+}
+
/*
* paraminfo_get_equal_hashops
* Determine if param_info and innerrel's lateral_vars can be hashed.
@@ -657,15 +700,16 @@ try_nestloop_path(PlannerInfo *root,
/*
* Check to see if proposed path is still parameterized, and reject if the
* parameterization wouldn't be sensible --- unless allow_star_schema_join
- * says to allow it anyway. Also, we must reject if have_dangerous_phv
- * doesn't like the look of it, which could only happen if the nestloop is
- * still parameterized.
+ * says to allow it anyway. Also, we must reject if either
+ * have_unsafe_outer_join_ref or have_dangerous_phv don't like the look of
+ * it, which could only happen if the nestloop is still parameterized.
*/
required_outer = calc_nestloop_required_outer(outerrelids, outer_paramrels,
innerrelids, inner_paramrels);
if (required_outer &&
((!bms_overlap(required_outer, extra->param_source_rels) &&
!allow_star_schema_join(root, outerrelids, inner_paramrels)) ||
+ have_unsafe_outer_join_ref(root, outerrelids, inner_paramrels) ||
have_dangerous_phv(root, outerrelids, inner_paramrels)))
{
/* Waste no memory when we reject a path here */
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 9da3ff2f9a..605f466bdd 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -353,7 +353,10 @@ make_rels_by_clauseless_joins(PlannerInfo *root,
*
* Caller must supply not only the two rels, but the union of their relids.
* (We could simplify the API by computing joinrelids locally, but this
- * would be redundant work in the normal path through make_join_rel.)
+ * would be redundant work in the normal path through make_join_rel.
+ * Note that this value does NOT include the RT index of any outer join that
+ * might need to be performed here, so it's not the canonical identifier
+ * of the join relation.)
*
* On success, *sjinfo_p is set to NULL if this is to be a plain inner join,
* else it's set to point to the associated SpecialJoinInfo node. Also,
@@ -695,7 +698,7 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(rel1->relids, rel2->relids));
- /* Construct Relids set that identifies the joinrel. */
+ /* Construct Relids set that identifies the joinrel (without OJ as yet). */
joinrelids = bms_union(rel1->relids, rel2->relids);
/* Check validity and determine join type. */
@@ -707,6 +710,10 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
return NULL;
}
+ /* If we have an outer join, add its RTI to form the canonical relids. */
+ if (sjinfo && sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
+
/* Swap rels if needed to match the join info. */
if (reversed)
{
@@ -730,6 +737,10 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
sjinfo->syn_lefthand = rel1->relids;
sjinfo->syn_righthand = rel2->relids;
sjinfo->jointype = JOIN_INNER;
+ sjinfo->ojrelid = 0;
+ sjinfo->commute_above_l = NULL;
+ sjinfo->commute_above_r = NULL;
+ sjinfo->commute_below = NULL;
/* we don't bother trying to make the remaining fields valid */
sjinfo->lhs_strict = false;
sjinfo->delay_upper_joins = false;
@@ -1510,8 +1521,6 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(child_rel1->relids, child_rel2->relids));
- child_joinrelids = bms_union(child_rel1->relids, child_rel2->relids);
- appinfos = find_appinfos_by_relids(root, child_joinrelids, &nappinfos);
/*
* Construct SpecialJoinInfo from parent join relations's
@@ -1521,6 +1530,15 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
child_rel1->relids,
child_rel2->relids);
+ /* Build correct join relids for child join */
+ child_joinrelids = bms_union(child_rel1->relids, child_rel2->relids);
+ if (child_sjinfo->ojrelid != 0)
+ child_joinrelids = bms_add_member(child_joinrelids,
+ child_sjinfo->ojrelid);
+
+ /* Find the AppendRelInfo structures */
+ appinfos = find_appinfos_by_relids(root, child_joinrelids, &nappinfos);
+
/*
* Construct restrictions applicable to the child join from those
* applicable to the parent join.
@@ -1536,8 +1554,7 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
{
child_joinrel = build_child_join_rel(root, child_rel1, child_rel2,
joinrel, child_restrictlist,
- child_sjinfo,
- child_sjinfo->jointype);
+ child_sjinfo);
joinrel->part_rels[cnt_parts] = child_joinrel;
joinrel->live_parts = bms_add_member(joinrel->live_parts, cnt_parts);
joinrel->all_partrels = bms_add_members(joinrel->all_partrels,
@@ -1583,6 +1600,7 @@ build_child_join_sjinfo(PlannerInfo *root, SpecialJoinInfo *parent_sjinfo,
sjinfo->syn_righthand = adjust_child_relids(sjinfo->syn_righthand,
right_nappinfos,
right_appinfos);
+ /* outer-join relids need no adjustment */
sjinfo->semi_rhs_exprs = (List *) adjust_appendrel_attrs(root,
(Node *) sjinfo->semi_rhs_exprs,
right_nappinfos,
diff --git a/src/backend/optimizer/path/tidpath.c b/src/backend/optimizer/path/tidpath.c
index c4e035b049..71488cec00 100644
--- a/src/backend/optimizer/path/tidpath.c
+++ b/src/backend/optimizer/path/tidpath.c
@@ -59,6 +59,7 @@ IsCTIDVar(Var *var, RelOptInfo *rel)
if (var->varattno == SelfItemPointerAttributeNumber &&
var->vartype == TIDOID &&
var->varno == rel->relid &&
+ var->varnullingrels == NULL &&
var->varlevelsup == 0)
return true;
return false;
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index dc0f165477..79fd240cf3 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -34,7 +34,7 @@
/* local functions */
static bool join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo);
-static void remove_rel_from_query(PlannerInfo *root, int relid,
+static void remove_rel_from_query(PlannerInfo *root, int relid, int ojrelid,
Relids joinrelids);
static List *remove_rel_from_joinlist(List *joinlist, int relid, int *nremoved);
static bool rel_supports_distinctness(PlannerInfo *root, RelOptInfo *rel);
@@ -70,6 +70,7 @@ restart:
foreach(lc, root->join_info_list)
{
SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+ Relids joinrelids;
int innerrelid;
int nremoved;
@@ -84,9 +85,12 @@ restart:
*/
innerrelid = bms_singleton_member(sjinfo->min_righthand);
- remove_rel_from_query(root, innerrelid,
- bms_union(sjinfo->min_lefthand,
- sjinfo->min_righthand));
+ /* Compute the relid set for the join we are considering */
+ joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ if (sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
+
+ remove_rel_from_query(root, innerrelid, sjinfo->ojrelid, joinrelids);
/* We verify that exactly one reference gets removed from joinlist */
nremoved = 0;
@@ -188,6 +192,8 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
/* Compute the relid set for the join we are considering */
joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ if (sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
/*
* We can't remove the join if any inner-rel attributes are used above the
@@ -247,6 +253,17 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
{
RestrictInfo *restrictinfo = (RestrictInfo *) lfirst(l);
+ /*
+ * If the current join commutes with some other outer join(s) via
+ * outer join identity 3, there will be multiple clones of its join
+ * clauses in the joininfo list. We want to consider only the
+ * has_clone form of such clauses. Processing more than one form
+ * would be wasteful, and also some of the others would confuse the
+ * RINFO_IS_PUSHED_DOWN test below.
+ */
+ if (restrictinfo->is_clone)
+ continue; /* ignore it */
+
/*
* If it's not a join clause for this outer join, we can't use it.
* Note that if the clause is pushed-down, then it is logically from
@@ -306,10 +323,12 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
* no longer treated as a baserel, and that attributes of other baserels
* are no longer marked as being needed at joins involving this rel.
* Also, join quals involving the rel have to be removed from the joininfo
- * lists, but only if they belong to the outer join identified by joinrelids.
+ * lists, but only if they belong to the outer join identified by ojrelid
+ * and joinrelids.
*/
static void
-remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
+remove_rel_from_query(PlannerInfo *root, int relid, int ojrelid,
+ Relids joinrelids)
{
RelOptInfo *rel = find_base_rel(root, relid);
List *joininfos;
@@ -346,6 +365,8 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
{
otherrel->attr_needed[attroff] =
bms_del_member(otherrel->attr_needed[attroff], relid);
+ otherrel->attr_needed[attroff] =
+ bms_del_member(otherrel->attr_needed[attroff], ojrelid);
}
}
@@ -353,6 +374,9 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
* Update all_baserels and related relid sets.
*/
root->all_baserels = bms_del_member(root->all_baserels, relid);
+ root->outer_join_rels = bms_del_member(root->outer_join_rels, ojrelid);
+ root->all_query_rels = bms_del_member(root->all_query_rels, relid);
+ root->all_query_rels = bms_del_member(root->all_query_rels, ojrelid);
/*
* Likewise remove references from SpecialJoinInfo data structures.
@@ -370,6 +394,14 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
sjinfo->min_righthand = bms_del_member(sjinfo->min_righthand, relid);
sjinfo->syn_lefthand = bms_del_member(sjinfo->syn_lefthand, relid);
sjinfo->syn_righthand = bms_del_member(sjinfo->syn_righthand, relid);
+ sjinfo->min_lefthand = bms_del_member(sjinfo->min_lefthand, ojrelid);
+ sjinfo->min_righthand = bms_del_member(sjinfo->min_righthand, ojrelid);
+ sjinfo->syn_lefthand = bms_del_member(sjinfo->syn_lefthand, ojrelid);
+ sjinfo->syn_righthand = bms_del_member(sjinfo->syn_righthand, ojrelid);
+ /* relid cannot appear in these fields, but ojrelid can: */
+ sjinfo->commute_above_l = bms_del_member(sjinfo->commute_above_l, ojrelid);
+ sjinfo->commute_above_r = bms_del_member(sjinfo->commute_above_r, ojrelid);
+ sjinfo->commute_below = bms_del_member(sjinfo->commute_below, ojrelid);
}
/*
@@ -401,8 +433,10 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
else
{
phinfo->ph_eval_at = bms_del_member(phinfo->ph_eval_at, relid);
+ phinfo->ph_eval_at = bms_del_member(phinfo->ph_eval_at, ojrelid);
Assert(!bms_is_empty(phinfo->ph_eval_at));
phinfo->ph_needed = bms_del_member(phinfo->ph_needed, relid);
+ phinfo->ph_needed = bms_del_member(phinfo->ph_needed, ojrelid);
}
}
@@ -427,7 +461,12 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
remove_join_clause_from_rels(root, rinfo, rinfo->required_relids);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrelids))
+ /*
+ * If the qual lists ojrelid in its required_relids, it must have come
+ * from above the outer join we're removing (so we need to keep it);
+ * if it does not, then it didn't and we can discard it.
+ */
+ if (bms_is_member(ojrelid, rinfo->required_relids))
{
/* Recheck that qual doesn't actually reference the target rel */
Assert(!bms_is_member(relid, rinfo->clause_relids));
@@ -439,6 +478,8 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
rinfo->required_relids = bms_copy(rinfo->required_relids);
rinfo->required_relids = bms_del_member(rinfo->required_relids,
relid);
+ rinfo->required_relids = bms_del_member(rinfo->required_relids,
+ ojrelid);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
@@ -553,6 +594,7 @@ reduce_unique_semijoins(PlannerInfo *root)
/* Compute the relid set for the join we are considering */
joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ Assert(sjinfo->ojrelid == 0); /* SEMI joins don't have RT indexes */
/*
* Since we're only considering a single-rel RHS, any join clauses it
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 2ee58f0a68..90da123c10 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -48,7 +48,9 @@ int join_collapse_limit;
*
* deconstruct_recurse recursively examines the join tree and builds a List
* (in depth-first traversal order) of JoinTreeItem structs, which are then
- * processed iteratively by deconstruct_distribute.
+ * processed iteratively by deconstruct_distribute. If there are outer
+ * joins, non-degenerate outer join clauses are processed in a third pass
+ * deconstruct_distribute_oj_quals.
*
* The JoinTreeItem structs themselves can be freed at the end of
* deconstruct_jointree, but do not modify or free their substructure,
@@ -60,17 +62,18 @@ typedef struct JoinTreeItem
/* Fields filled during deconstruct_recurse: */
Node *jtnode; /* jointree node to examine */
bool below_outer_join; /* is it below an outer join? */
- Relids qualscope; /* base Relids syntactically included in this
- * jointree node */
- Relids inner_join_rels; /* base Relids syntactically included in
- * inner joins appearing at or below this
- * jointree node */
+ Relids qualscope; /* base+OJ Relids syntactically included in
+ * this jointree node */
+ Relids inner_join_rels; /* base+OJ Relids syntactically included
+ * in inner joins appearing at or below
+ * this jointree node */
Relids left_rels; /* if join node, Relids of the left side */
Relids right_rels; /* if join node, Relids of the right side */
Relids nonnullable_rels; /* if outer join, Relids of the
* non-nullable side */
/* Fields filled during deconstruct_distribute: */
SpecialJoinInfo *sjinfo; /* if outer join, its SpecialJoinInfo */
+ List *oj_joinclauses; /* outer join quals not yet distributed */
} JoinTreeItem;
/* Elements of the postponed_qual_list used during deconstruct_distribute */
@@ -94,9 +97,13 @@ static void process_security_barrier_quals(PlannerInfo *root,
static SpecialJoinInfo *make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
- JoinType jointype, List *clause);
+ JoinType jointype, Index ojrelid,
+ List *clause);
static void compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo,
List *clause);
+static void deconstruct_distribute_oj_quals(PlannerInfo *root,
+ List *jtitems,
+ JoinTreeItem *jtitem);
static void distribute_quals_to_rels(PlannerInfo *root, List *clauses,
bool below_outer_join,
SpecialJoinInfo *sjinfo,
@@ -104,7 +111,11 @@ static void distribute_quals_to_rels(PlannerInfo *root, List *clauses,
Relids qualscope,
Relids ojscope,
Relids outerjoin_nonnullable,
- List **postponed_qual_list);
+ bool allow_equivalence,
+ bool has_clone,
+ bool is_clone,
+ List **postponed_qual_list,
+ List **postponed_oj_qual_list);
static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool below_outer_join,
SpecialJoinInfo *sjinfo,
@@ -112,7 +123,11 @@ static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
Relids qualscope,
Relids ojscope,
Relids outerjoin_nonnullable,
- List **postponed_qual_list);
+ bool allow_equivalence,
+ bool has_clone,
+ bool is_clone,
+ List **postponed_qual_list,
+ List **postponed_oj_qual_list);
static bool check_outerjoin_delay(PlannerInfo *root, Relids *relids_p,
Relids *nullable_relids_p, bool is_pushed_down);
static bool check_equivalence_delay(PlannerInfo *root,
@@ -290,10 +305,16 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
attno -= rel->min_attr;
if (rel->attr_needed[attno] == NULL)
{
- /* Variable not yet requested, so add to rel's targetlist */
- /* XXX is copyObject necessary here? */
- rel->reltarget->exprs = lappend(rel->reltarget->exprs,
- copyObject(var));
+ /*
+ * Variable not yet requested, so add to rel's targetlist.
+ *
+ * The value available at the rel's scan level has not been
+ * nulled by any outer join, so drop its varnullingrels.
+ * (We'll put those back as we climb up the join tree.)
+ */
+ var = copyObject(var);
+ var->varnullingrels = NULL;
+ rel->reltarget->exprs = lappend(rel->reltarget->exprs, var);
/* reltarget cost and width will be computed later */
}
rel->attr_needed[attno] = bms_add_members(rel->attr_needed[attno],
@@ -589,8 +610,10 @@ create_lateral_join_info(PlannerInfo *root)
varno = -1;
while ((varno = bms_next_member(eval_at, varno)) >= 0)
{
- RelOptInfo *brel = find_base_rel(root, varno);
+ RelOptInfo *brel = find_base_rel_ignore_join(root, varno);
+ if (brel == NULL)
+ continue; /* ignore outer joins in eval_at */
brel->lateral_relids = bms_add_members(brel->lateral_relids,
phinfo->ph_lateral);
}
@@ -681,7 +704,10 @@ create_lateral_join_info(PlannerInfo *root)
{
RelOptInfo *brel2 = root->simple_rel_array[rti2];
- Assert(brel2 != NULL && brel2->reloptkind == RELOPT_BASEREL);
+ if (brel2 == NULL)
+ continue; /* must be an OJ */
+
+ Assert(brel2->reloptkind == RELOPT_BASEREL);
brel2->lateral_referencers =
bms_add_member(brel2->lateral_referencers, rti);
}
@@ -743,6 +769,7 @@ deconstruct_jointree(PlannerInfo *root)
/* These are filled as we scan the jointree */
root->all_baserels = NULL;
+ root->outer_join_rels = NULL;
root->nullable_baserels = NULL;
/* Perform the initial scan of the jointree */
@@ -750,6 +777,9 @@ deconstruct_jointree(PlannerInfo *root)
false,
&item_list);
+ /* Now we can form the value of all_query_rels, too */
+ root->all_query_rels = bms_union(root->all_baserels, root->outer_join_rels);
+
/* Now scan all the jointree nodes again, and distribute quals */
foreach(lc, item_list)
{
@@ -762,6 +792,40 @@ deconstruct_jointree(PlannerInfo *root)
/* Shouldn't be any leftover postponed quals */
Assert(postponed_qual_list == NIL);
+ /*
+ * However, if there were any special joins then we may have some
+ * postponed LEFT JOIN clauses to deal with.
+ */
+ if (root->join_info_list)
+ {
+ /*
+ * XXX hack: when we call distribute_qual_to_rels to process one of
+ * these clauses, neither the owning SpecialJoinInfo nor any later
+ * ones can appear in root->join_info_list, else the wrong things will
+ * happen. Fake it out by emptying join_info_list and rebuilding it
+ * as we go. This works because join_info_list is only appended to
+ * during deconstruct_distribute, so we know we are examining
+ * SpecialJoinInfos bottom-up, just like the first time. We can get
+ * rid of this hack later, after fixing things so that
+ * distribute_qual_to_rels doesn't have that requirement about
+ * join_info_list.
+ */
+ root->join_info_list = NIL;
+
+ foreach(lc, item_list)
+ {
+ JoinTreeItem *jtitem = (JoinTreeItem *) lfirst(lc);
+
+ if (jtitem->oj_joinclauses != NIL)
+ deconstruct_distribute_oj_quals(root, item_list, jtitem);
+
+ /* XXX Rest of hack: rebuild join_info_list as we go */
+ if (jtitem->sjinfo)
+ root->join_info_list = lappend(root->join_info_list,
+ jtitem->sjinfo);
+ }
+ }
+
/* Don't need the JoinTreeItems any more */
list_free_deep(item_list);
@@ -905,6 +969,14 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
/* Compute qualscope etc */
jtitem->qualscope = bms_union(left_item->qualscope,
right_item->qualscope);
+ /* caution: ANTI join derived from SEMI will lack rtindex */
+ if (j->rtindex != 0)
+ {
+ jtitem->qualscope = bms_add_member(jtitem->qualscope,
+ j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
+ j->rtindex);
+ }
jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
right_item->inner_join_rels);
jtitem->left_rels = left_item->qualscope;
@@ -925,6 +997,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
/* Compute qualscope etc */
jtitem->qualscope = bms_union(left_item->qualscope,
right_item->qualscope);
+ /* SEMI join never has rtindex, so don't add to anything */
+ Assert(j->rtindex == 0);
jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
right_item->inner_join_rels);
jtitem->left_rels = left_item->qualscope;
@@ -952,6 +1026,11 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
/* Compute qualscope etc */
jtitem->qualscope = bms_union(left_item->qualscope,
right_item->qualscope);
+ Assert(j->rtindex != 0);
+ jtitem->qualscope = bms_add_member(jtitem->qualscope,
+ j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
+ j->rtindex);
jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
right_item->inner_join_rels);
jtitem->left_rels = left_item->qualscope;
@@ -1073,7 +1152,8 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
NULL,
root->qual_security_level,
jtitem->qualscope, NULL, NULL,
- NULL);
+ true, false, false,
+ NULL, NULL);
else
new_postponed_quals = lappend(new_postponed_quals, pq);
}
@@ -1087,7 +1167,8 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
NULL,
root->qual_security_level,
jtitem->qualscope, NULL, NULL,
- postponed_qual_list);
+ true, false, false,
+ postponed_qual_list, NULL);
}
else if (IsA(jtnode, JoinExpr))
{
@@ -1096,6 +1177,7 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
Relids ojscope;
List *my_quals;
SpecialJoinInfo *sjinfo;
+ List **postponed_oj_qual_list;
ListCell *l;
/*
@@ -1138,6 +1220,7 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
jtitem->right_rels,
jtitem->inner_join_rels,
j->jointype,
+ j->rtindex,
my_quals);
jtitem->sjinfo = sjinfo;
if (j->jointype == JOIN_SEMI)
@@ -1146,6 +1229,19 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
{
ojscope = bms_union(sjinfo->min_lefthand,
sjinfo->min_righthand);
+
+ /*
+ * Add back any commutable lower OJ relids that were removed
+ * from min_lefthand or min_righthand, else the ojscope
+ * cross-check in distribute_qual_to_rels will complain. If
+ * any such OJs were removed, we will postpone processing of
+ * non-degenerate clauses, so this addition doesn't affect
+ * anything except that cross-check and some Asserts. Real
+ * clause positioning decisions will be made later, when we
+ * revisit the postponed clauses.
+ */
+ if (sjinfo->commute_below)
+ ojscope = bms_add_members(ojscope, sjinfo->commute_below);
}
}
else
@@ -1154,6 +1250,18 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
ojscope = NULL;
}
+ /*
+ * If it's a left join with a join clause that is strict for the LHS,
+ * then we need to postpone handling of any non-degenerate join
+ * clauses, in case the join is able to commute with another left join
+ * per identity 3. (Degenerate clauses need not be postponed, since
+ * they will drop down below this join anyway.)
+ */
+ if (j->jointype == JOIN_LEFT && sjinfo->lhs_strict)
+ postponed_oj_qual_list = &jtitem->oj_joinclauses;
+ else
+ postponed_oj_qual_list = NULL;
+
/* Process the JOIN's qual clauses */
distribute_quals_to_rels(root, my_quals,
jtitem->below_outer_join,
@@ -1161,7 +1269,10 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
root->qual_security_level,
jtitem->qualscope,
ojscope, jtitem->nonnullable_rels,
- postponed_qual_list);
+ true, /* allow_equivalence */
+ false, false, /* not clones */
+ postponed_qual_list,
+ postponed_oj_qual_list);
/* And add the SpecialJoinInfo to join_info_list */
if (sjinfo)
@@ -1223,6 +1334,9 @@ process_security_barrier_quals(PlannerInfo *root,
qualscope,
qualscope,
NULL,
+ true,
+ false, false, /* not clones */
+ NULL,
NULL);
security_level++;
}
@@ -1236,10 +1350,11 @@ process_security_barrier_quals(PlannerInfo *root,
* Build a SpecialJoinInfo for the current outer join
*
* Inputs:
- * left_rels: the base Relids syntactically on outer side of join
- * right_rels: the base Relids syntactically on inner side of join
- * inner_join_rels: base Relids participating in inner joins below this one
+ * left_rels: the base+OJ Relids syntactically on outer side of join
+ * right_rels: the base+OJ Relids syntactically on inner side of join
+ * inner_join_rels: base+OJ Relids participating in inner joins below this one
* jointype: what it says (must always be LEFT, FULL, SEMI, or ANTI)
+ * ojrelid: RT index of the join RTE (0 for SEMI, which isn't in the RT list)
* clause: the outer join's join condition (in implicit-AND format)
*
* The node should eventually be appended to root->join_info_list, but we
@@ -1253,7 +1368,8 @@ static SpecialJoinInfo *
make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
- JoinType jointype, List *clause)
+ JoinType jointype, Index ojrelid,
+ List *clause)
{
SpecialJoinInfo *sjinfo = makeNode(SpecialJoinInfo);
Relids clause_relids;
@@ -1301,6 +1417,11 @@ make_outerjoininfo(PlannerInfo *root,
sjinfo->syn_lefthand = left_rels;
sjinfo->syn_righthand = right_rels;
sjinfo->jointype = jointype;
+ sjinfo->ojrelid = ojrelid;
+ /* these fields may get added to later: */
+ sjinfo->commute_above_l = NULL;
+ sjinfo->commute_above_r = NULL;
+ sjinfo->commute_below = NULL;
/* this always starts out false */
sjinfo->delay_upper_joins = false;
@@ -1348,6 +1469,7 @@ make_outerjoininfo(PlannerInfo *root,
foreach(l, root->join_info_list)
{
SpecialJoinInfo *otherinfo = (SpecialJoinInfo *) lfirst(l);
+ bool have_unsafe_phvs;
/*
* A full join is an optimization barrier: we can't associate into or
@@ -1363,6 +1485,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_lefthand = bms_add_member(min_lefthand,
+ otherinfo->ojrelid);
}
if (bms_overlap(right_rels, otherinfo->syn_lefthand) ||
bms_overlap(right_rels, otherinfo->syn_righthand))
@@ -1371,11 +1496,26 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_righthand = bms_add_member(min_righthand,
+ otherinfo->ojrelid);
}
/* Needn't do anything else with the full join */
continue;
}
+ /*
+ * If our join condition contains any PlaceHolderVars that need to be
+ * evaluated above the lower OJ, then we can't commute with it.
+ */
+ if (otherinfo->ojrelid != 0)
+ have_unsafe_phvs =
+ contain_placeholder_references_to(root,
+ (Node *) clause,
+ otherinfo->ojrelid);
+ else
+ have_unsafe_phvs = false;
+
/*
* For a lower OJ in our LHS, if our join condition uses the lower
* join's RHS and is not strict for that rel, we must preserve the
@@ -1383,23 +1523,44 @@ make_outerjoininfo(PlannerInfo *root,
* min_lefthand. (We must use its full syntactic relset, not just its
* min_lefthand + min_righthand. This is because there might be other
* OJs below this one that this one can commute with, but we cannot
- * commute with them if we don't with this one.) Also, if the current
- * join is a semijoin or antijoin, we must preserve ordering
- * regardless of strictness.
+ * commute with them if we don't with this one.) Also, if we have
+ * unsafe PHVs or the current join is a semijoin or antijoin, we must
+ * preserve ordering regardless of strictness.
*
* Note: I believe we have to insist on being strict for at least one
* rel in the lower OJ's min_righthand, not its whole syn_righthand.
+ *
+ * When we don't need to preserve ordering, check to see if outer join
+ * identity 3 applies, and if so, remove the lower OJ's ojrelid from
+ * our min_lefthand so that commutation is allowed.
*/
if (bms_overlap(left_rels, otherinfo->syn_righthand))
{
if (bms_overlap(clause_relids, otherinfo->syn_righthand) &&
- (jointype == JOIN_SEMI || jointype == JOIN_ANTI ||
+ (have_unsafe_phvs ||
+ jointype == JOIN_SEMI || jointype == JOIN_ANTI ||
!bms_overlap(strict_relids, otherinfo->min_righthand)))
{
+ /* Preserve ordering */
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_lefthand);
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_lefthand = bms_add_member(min_lefthand,
+ otherinfo->ojrelid);
+ }
+ else if (jointype == JOIN_LEFT &&
+ otherinfo->jointype == JOIN_LEFT &&
+ bms_overlap(strict_relids, otherinfo->min_righthand))
+ {
+ /* Identity 3 applies, so remove the ordering restriction */
+ min_lefthand = bms_del_member(min_lefthand, otherinfo->ojrelid);
+ /* Add commutability markers to both SpecialJoinInfos */
+ otherinfo->commute_above_l =
+ bms_add_member(otherinfo->commute_above_l, ojrelid);
+ sjinfo->commute_below =
+ bms_add_member(sjinfo->commute_below, otherinfo->ojrelid);
}
}
@@ -1414,8 +1575,8 @@ make_outerjoininfo(PlannerInfo *root,
* up with SpecialJoinInfos with identical min_righthands, which can
* confuse join_is_legal (see discussion in backend/optimizer/README).
*
- * Also, we must preserve ordering anyway if either the current join
- * or the lower OJ is either a semijoin or an antijoin.
+ * Also, we must preserve ordering anyway if we have unsafe PHVs, or
+ * if either this join or the lower OJ is a semijoin or antijoin.
*
* Here, we have to consider that "our join condition" includes any
* clauses that syntactically appeared above the lower OJ and below
@@ -1427,21 +1588,43 @@ make_outerjoininfo(PlannerInfo *root,
* join condition are not affected by them. The net effect is
* therefore sufficiently represented by the delay_upper_joins flag
* saved for us by check_outerjoin_delay.
+ *
+ * When we don't need to preserve ordering, check to see if outer join
+ * identity 3 applies, and if so, remove the lower OJ's ojrelid from
+ * our min_righthand so that commutation is allowed.
*/
if (bms_overlap(right_rels, otherinfo->syn_righthand))
{
if (bms_overlap(clause_relids, otherinfo->syn_righthand) ||
!bms_overlap(clause_relids, otherinfo->min_lefthand) ||
+ have_unsafe_phvs ||
jointype == JOIN_SEMI ||
jointype == JOIN_ANTI ||
otherinfo->jointype == JOIN_SEMI ||
otherinfo->jointype == JOIN_ANTI ||
!otherinfo->lhs_strict || otherinfo->delay_upper_joins)
{
+ /* Preserve ordering */
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_lefthand);
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_righthand = bms_add_member(min_righthand,
+ otherinfo->ojrelid);
+ }
+ else if (jointype == JOIN_LEFT &&
+ otherinfo->jointype == JOIN_LEFT &&
+ otherinfo->lhs_strict)
+ {
+ /* Identity 3 applies, so remove the ordering restriction */
+ min_righthand = bms_del_member(min_righthand,
+ otherinfo->ojrelid);
+ /* Add commutability markers to both SpecialJoinInfos */
+ otherinfo->commute_above_r =
+ bms_add_member(otherinfo->commute_above_r, ojrelid);
+ sjinfo->commute_below =
+ bms_add_member(sjinfo->commute_below, otherinfo->ojrelid);
}
}
}
@@ -1666,6 +1849,207 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
sjinfo->semi_rhs_exprs = semi_rhs_exprs;
}
+/*
+ * deconstruct_distribute_oj_quals
+ * Adjust LEFT JOIN quals to be suitable for commuted-left-join cases,
+ * then push them into the joinqual lists and EquivalenceClass structures.
+ *
+ * This runs immediately after we've completed the deconstruct_distribute scan.
+ * jtitems contains all the JoinTreeItems (in depth-first order), and jtitem
+ * is one that has postponed oj_joinclauses to deal with.
+ */
+static void
+deconstruct_distribute_oj_quals(PlannerInfo *root,
+ List *jtitems,
+ JoinTreeItem *jtitem)
+{
+ SpecialJoinInfo *sjinfo = jtitem->sjinfo;
+ Relids qualscope,
+ ojscope,
+ nonnullable_rels;
+
+ /* Recompute syntactic and semantic scopes of this left join */
+ qualscope = bms_union(sjinfo->syn_lefthand, sjinfo->syn_righthand);
+ qualscope = bms_add_member(qualscope, sjinfo->ojrelid);
+ ojscope = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ nonnullable_rels = sjinfo->syn_lefthand;
+
+ /*
+ * If this join can commute with any other ones per outer-join identity 3,
+ * and it is the one providing the join clause with flexible semantics,
+ * then we have to generate variants of the join clause with different
+ * nullingrels labeling. Otherwise, just push out the postponed clause
+ * as-is.
+ */
+ Assert(sjinfo->lhs_strict); /* else we shouldn't be here */
+ if (sjinfo->commute_above_r ||
+ bms_overlap(sjinfo->commute_below, sjinfo->syn_lefthand))
+ {
+ Relids joins_above;
+ Relids joins_below;
+ Relids joins_so_far;
+ List *quals;
+ ListCell *lc;
+
+ /*
+ * Put any OJ relids that were removed from min_righthand back into
+ * ojscope, else distribute_qual_to_rels will complain.
+ */
+ ojscope = bms_join(ojscope, bms_intersect(sjinfo->commute_below,
+ sjinfo->syn_righthand));
+
+ /* Identify the outer joins this one commutes with */
+ joins_above = sjinfo->commute_above_r;
+ joins_below = bms_intersect(sjinfo->commute_below,
+ sjinfo->syn_lefthand);
+
+ /*
+ * Generate qual variants with different sets of nullingrels bits.
+ *
+ * We only need bit-sets that correspond to the successively less
+ * deeply syntactically-nested subsets of this join and its
+ * commutators. That's true first because obviously only those forms
+ * of the Vars and PHVs could appear elsewhere in the query, and
+ * second because the outer join identities do not provide a way to
+ * re-order such joins in a way that would require different marking.
+ * (That is, while the current join may commute with several others,
+ * none of those others can commute with each other.) To visit the
+ * interesting joins in syntactic nesting order, we rely on the
+ * jtitems list to be ordered that way.
+ *
+ * We first strip out all the nullingrels bits corresponding to
+ * commutating joins below this one, and then successively put them
+ * back as we crawl up the join stack.
+ */
+ quals = jtitem->oj_joinclauses;
+ if (!bms_is_empty(joins_below))
+ quals = (List *) remove_nulling_relids((Node *) quals,
+ joins_below,
+ NULL);
+
+ joins_so_far = NULL;
+ foreach(lc, jtitems)
+ {
+ JoinTreeItem *otherjtitem = (JoinTreeItem *) lfirst(lc);
+ SpecialJoinInfo *othersj = otherjtitem->sjinfo;
+ bool below_sjinfo = false;
+ bool above_sjinfo = false;
+ Relids this_qualscope;
+ Relids this_ojscope;
+ bool allow_equivalence,
+ has_clone,
+ is_clone;
+
+ if (othersj == NULL)
+ continue; /* not an outer-join item, ignore */
+
+ if (bms_is_member(othersj->ojrelid, joins_below))
+ {
+ /* othersj commutes with sjinfo from below left */
+ below_sjinfo = true;
+ }
+ else if (othersj == sjinfo)
+ {
+ /* found our join in syntactic order */
+ Assert(bms_equal(joins_so_far, joins_below));
+ }
+ else if (bms_is_member(othersj->ojrelid, joins_above))
+ {
+ /* othersj commutes with sjinfo from above */
+ above_sjinfo = true;
+ }
+ else
+ {
+ /* othersj is not relevant, ignore */
+ continue;
+ }
+
+ /*
+ * When we are looking at joins above sjinfo, we are envisioning
+ * pushing sjinfo to above othersj, so add othersj's nulling bit
+ * before distributing the quals.
+ */
+ if (above_sjinfo)
+ quals = (List *)
+ add_nulling_relids((Node *) quals,
+ othersj->min_righthand,
+ bms_make_singleton(othersj->ojrelid));
+
+ /* Compute qualscope and ojscope for this join level */
+ this_qualscope = bms_union(qualscope, joins_so_far);
+ this_ojscope = bms_union(ojscope, joins_so_far);
+ if (above_sjinfo)
+ {
+ /* othersj is not yet in joins_so_far, but we need it */
+ this_qualscope = bms_add_member(this_qualscope,
+ othersj->ojrelid);
+ this_ojscope = bms_add_member(this_ojscope,
+ othersj->ojrelid);
+ /* sjinfo is in joins_so_far, and we don't want it */
+ this_ojscope = bms_del_member(this_ojscope,
+ sjinfo->ojrelid);
+ }
+
+ /*
+ * We generate EquivalenceClasses only from the first form of the
+ * quals, with the fewest nullingrels bits set. An EC made from
+ * this version of the quals can be useful below the outer-join
+ * nest, whereas versions with some nullingrels bits set would not
+ * be. We cannot generate ECs from more than one version, or
+ * we'll make nonsensical conclusions that Vars with nullingrels
+ * bits set are equal to their versions without. Fortunately,
+ * such ECs wouldn't be very useful anyway, because they'd equate
+ * values not observable outside the join nest. (See
+ * optimizer/README.)
+ *
+ * The first form of the quals is also the only one marked as
+ * has_clone rather than is_clone.
+ */
+ allow_equivalence = (joins_so_far == NULL);
+ has_clone = allow_equivalence;
+ is_clone = !has_clone;
+
+ distribute_quals_to_rels(root, quals,
+ true,
+ sjinfo,
+ root->qual_security_level,
+ this_qualscope,
+ this_ojscope, nonnullable_rels,
+ allow_equivalence,
+ has_clone,
+ is_clone,
+ NULL, NULL); /* no more postponement */
+
+ /*
+ * Adjust qual nulling bits for next level up, if needed. We
+ * don't want to put sjinfo's own bit in at all, and if we're
+ * above sjinfo then we did it already.
+ */
+ if (below_sjinfo)
+ quals = (List *)
+ add_nulling_relids((Node *) quals,
+ othersj->min_righthand,
+ bms_make_singleton(othersj->ojrelid));
+
+ /* ... and track joins processed so far */
+ joins_so_far = bms_add_member(joins_so_far, othersj->ojrelid);
+ }
+ }
+ else
+ {
+ /* No commutation possible, just process the postponed clauses */
+ distribute_quals_to_rels(root, jtitem->oj_joinclauses,
+ true,
+ sjinfo,
+ root->qual_security_level,
+ qualscope,
+ ojscope, nonnullable_rels,
+ true, /* allow_equivalence */
+ false, false, /* not clones */
+ NULL, NULL); /* no more postponement */
+ }
+}
+
/*****************************************************************************
*
@@ -1686,7 +2070,11 @@ distribute_quals_to_rels(PlannerInfo *root, List *clauses,
Relids qualscope,
Relids ojscope,
Relids outerjoin_nonnullable,
- List **postponed_qual_list)
+ bool allow_equivalence,
+ bool has_clone,
+ bool is_clone,
+ List **postponed_qual_list,
+ List **postponed_oj_qual_list)
{
ListCell *lc;
@@ -1701,7 +2089,11 @@ distribute_quals_to_rels(PlannerInfo *root, List *clauses,
qualscope,
ojscope,
outerjoin_nonnullable,
- postponed_qual_list);
+ allow_equivalence,
+ has_clone,
+ is_clone,
+ postponed_qual_list,
+ postponed_oj_qual_list);
}
}
@@ -1711,26 +2103,35 @@ distribute_quals_to_rels(PlannerInfo *root, List *clauses,
* (depending on whether the clause is a join) of each base relation
* mentioned in the clause. A RestrictInfo node is created and added to
* the appropriate list for each rel. Alternatively, if the clause uses a
- * mergejoinable operator and is not delayed by outer-join rules, enter
- * the left- and right-side expressions into the query's list of
- * EquivalenceClasses. Alternatively, if the clause needs to be treated
- * as belonging to a higher join level, just add it to postponed_qual_list.
+ * mergejoinable operator, enter its left- and right-side expressions into
+ * the query's EquivalenceClasses.
+ *
+ * In some cases, quals will be added to postponed_qual_list or
+ * postponed_oj_qual_list instead of being processed right away.
+ * These will be dealt with in later steps of deconstruct_jointree.
*
* 'clause': the qual clause to be distributed
* 'below_outer_join': true if the qual is from a JOIN/ON that is below the
* nullable side of a higher-level outer join
* 'sjinfo': join's SpecialJoinInfo (NULL for an inner join or WHERE clause)
* 'security_level': security_level to assign to the qual
- * 'qualscope': set of baserels the qual's syntactic scope covers
- * 'ojscope': NULL if not an outer-join qual, else the minimum set of baserels
- * needed to form this join
+ * 'qualscope': set of base+OJ rels the qual's syntactic scope covers
+ * 'ojscope': NULL if not an outer-join qual, else the minimum set of base+OJ
+ * rels needed to form this join
* 'outerjoin_nonnullable': NULL if not an outer-join qual, else the set of
- * baserels appearing on the outer (nonnullable) side of the join
+ * base+OJ rels appearing on the outer (nonnullable) side of the join
* (for FULL JOIN this includes both sides of the join, and must in fact
* equal qualscope)
+ * 'allow_equivalence': true if it's okay to convert clause into an
+ * EquivalenceClass
+ * 'has_clone': has_clone property to assign to the qual
+ * 'is_clone': is_clone property to assign to the qual
* 'postponed_qual_list': list of PostponedQual structs, which we can add
* this qual to if it turns out to belong to a higher join level.
* Can be NULL if caller knows postponement is impossible.
+ * 'postponed_oj_qual_list': if not NULL, non-degenerate outer join clauses
+ * should be added to this list instead of being processed (list entries
+ * are just the bare clauses)
*
* 'qualscope' identifies what level of JOIN the qual came from syntactically.
* 'ojscope' is needed if we decide to force the qual up to the outer-join
@@ -1748,7 +2149,11 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
Relids qualscope,
Relids ojscope,
Relids outerjoin_nonnullable,
- List **postponed_qual_list)
+ bool allow_equivalence,
+ bool has_clone,
+ bool is_clone,
+ List **postponed_qual_list,
+ List **postponed_oj_qual_list)
{
Relids relids;
bool is_pushed_down;
@@ -1842,7 +2247,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
{
relids =
get_relids_in_jointree((Node *) root->parse->jointree,
- false);
+ true, false);
qualscope = bms_copy(relids);
}
}
@@ -1885,8 +2290,18 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
{
/*
* The qual is attached to an outer join and mentions (some of the)
- * rels on the nonnullable side, so it's not degenerate.
- *
+ * rels on the nonnullable side, so it's not degenerate. If the
+ * caller wants to postpone handling such clauses, just add it to
+ * postponed_oj_qual_list and return. (The work we've done up to here
+ * will have to be redone later, but there's not much of it.)
+ */
+ if (postponed_oj_qual_list != NULL)
+ {
+ *postponed_oj_qual_list = lappend(*postponed_oj_qual_list, clause);
+ return;
+ }
+
+ /*
* We can't use such a clause to deduce equivalence (the left and
* right sides might be unequal above the join because one of them has
* gone to NULL) ... but we might be able to use it for more limited
@@ -1952,6 +2367,11 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
if (check_redundant_nullability_qual(root, clause))
return;
}
+ else if (!allow_equivalence)
+ {
+ /* Caller says it mustn't become an equivalence class */
+ maybe_equivalence = false;
+ }
else
{
/*
@@ -1986,11 +2406,22 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
outerjoin_nonnullable,
nullable_relids);
+ /* Apply appropriate clone marking, too */
+ restrictinfo->has_clone = has_clone;
+ restrictinfo->is_clone = is_clone;
+
/*
- * If it's a join clause (either naturally, or because delayed by
- * outer-join rules), add vars used in the clause to targetlists of their
- * relations, so that they will be emitted by the plan nodes that scan
- * those relations (else they won't be available at the join node!).
+ * If it's a join clause, add vars used in the clause to targetlists of
+ * their relations, so that they will be emitted by the plan nodes that
+ * scan those relations (else they won't be available at the join node!).
+ *
+ * Normally we mark the vars as needed at the join identified by "relids".
+ * However, if this is a clone clause then ignore the outer-join relids in
+ * that set. Otherwise, vars appearing in a cloned clause would end up
+ * marked as having to propagate to the highest one of the commuting
+ * joins, which would often be an overestimate. For such clauses, correct
+ * var propagation is ensured by making ojscope include input rels from
+ * both sides of the join.
*
* Note: if the clause gets absorbed into an EquivalenceClass then this
* may be unnecessary, but for now we have to do it to cover the case
@@ -2003,8 +2434,13 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
PVC_RECURSE_AGGREGATES |
PVC_RECURSE_WINDOWFUNCS |
PVC_INCLUDE_PLACEHOLDERS);
+ Relids where_needed;
- add_vars_to_targetlist(root, vars, relids);
+ if (is_clone)
+ where_needed = bms_intersect(relids, root->all_baserels);
+ else
+ where_needed = relids;
+ add_vars_to_targetlist(root, vars, where_needed);
list_free(vars);
}
@@ -2495,7 +2931,7 @@ process_implied_equality(PlannerInfo *root,
{
relids =
get_relids_in_jointree((Node *) root->parse->jointree,
- false);
+ true, false);
}
}
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index e21e72eb87..71b4d3ca56 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2231,7 +2231,7 @@ preprocess_rowmarks(PlannerInfo *root)
* make a bitmapset of all base rels and then remove the items we don't
* need or have FOR [KEY] UPDATE/SHARE marks for.
*/
- rels = get_relids_in_jointree((Node *) parse->jointree, false);
+ rels = get_relids_in_jointree((Node *) parse->jointree, false, false);
if (parse->resultRelation)
rels = bms_del_member(rels, parse->resultRelation);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 596f1fbc8e..846ca39269 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -30,11 +30,21 @@
#include "utils/syscache.h"
+typedef enum
+{
+ NRM_EQUAL, /* expect exact match of nullingrels */
+ NRM_SUBSET, /* actual Var may have a subset of input */
+ NRM_SUPERSET /* actual Var may have a superset of input */
+} NullingRelsMatch;
+
typedef struct
{
int varno; /* RT index of Var */
AttrNumber varattno; /* attr number of Var */
AttrNumber resno; /* TLE position of Var */
+#ifdef USE_ASSERT_CHECKING
+ Bitmapset *varnullingrels; /* Var's varnullingrels */
+#endif
} tlist_vinfo;
typedef struct
@@ -60,6 +70,7 @@ typedef struct
indexed_tlist *inner_itlist;
Index acceptable_rel;
int rtoffset;
+ NullingRelsMatch nrm_match;
double num_exec;
} fix_join_expr_context;
@@ -69,6 +80,7 @@ typedef struct
indexed_tlist *subplan_itlist;
int newvarno;
int rtoffset;
+ NullingRelsMatch nrm_match;
double num_exec;
} fix_upper_expr_context;
@@ -159,7 +171,12 @@ static indexed_tlist *build_tlist_index(List *tlist);
static Var *search_indexed_tlist_for_var(Var *var,
indexed_tlist *itlist,
int newvarno,
- int rtoffset);
+ int rtoffset,
+ NullingRelsMatch nrm_match);
+static Var *search_indexed_tlist_for_phv(PlaceHolderVar *phv,
+ indexed_tlist *itlist,
+ int newvarno,
+ NullingRelsMatch nrm_match);
static Var *search_indexed_tlist_for_non_var(Expr *node,
indexed_tlist *itlist,
int newvarno);
@@ -172,14 +189,18 @@ static List *fix_join_expr(PlannerInfo *root,
indexed_tlist *outer_itlist,
indexed_tlist *inner_itlist,
Index acceptable_rel,
- int rtoffset, double num_exec);
+ int rtoffset,
+ NullingRelsMatch nrm_match,
+ double num_exec);
static Node *fix_join_expr_mutator(Node *node,
fix_join_expr_context *context);
static Node *fix_upper_expr(PlannerInfo *root,
Node *node,
indexed_tlist *subplan_itlist,
int newvarno,
- int rtoffset, double num_exec);
+ int rtoffset,
+ NullingRelsMatch nrm_match,
+ double num_exec);
static Node *fix_upper_expr_mutator(Node *node,
fix_upper_expr_context *context);
static List *set_returning_clause_references(PlannerInfo *root,
@@ -1113,13 +1134,13 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
fix_join_expr(root, splan->onConflictSet,
NULL, itlist,
linitial_int(splan->resultRelations),
- rtoffset, NUM_EXEC_QUAL(plan));
+ rtoffset, NRM_EQUAL, NUM_EXEC_QUAL(plan));
splan->onConflictWhere = (Node *)
fix_join_expr(root, (List *) splan->onConflictWhere,
NULL, itlist,
linitial_int(splan->resultRelations),
- rtoffset, NUM_EXEC_QUAL(plan));
+ rtoffset, NRM_EQUAL, NUM_EXEC_QUAL(plan));
pfree(itlist);
@@ -1176,6 +1197,7 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
NULL, itlist,
resultrel,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST(plan));
/* Fix quals too. */
@@ -1184,6 +1206,7 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
NULL, itlist,
resultrel,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL(plan));
}
}
@@ -1329,6 +1352,7 @@ set_indexonlyscan_references(PlannerInfo *root,
index_itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST((Plan *) plan));
plan->scan.plan.qual = (List *)
fix_upper_expr(root,
@@ -1336,6 +1360,7 @@ set_indexonlyscan_references(PlannerInfo *root,
index_itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) plan));
plan->recheckqual = (List *)
fix_upper_expr(root,
@@ -1343,6 +1368,7 @@ set_indexonlyscan_references(PlannerInfo *root,
index_itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) plan));
/* indexqual is already transformed to reference index columns */
plan->indexqual = fix_scan_list(root, plan->indexqual,
@@ -1549,6 +1575,7 @@ set_foreignscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST((Plan *) fscan));
fscan->scan.plan.qual = (List *)
fix_upper_expr(root,
@@ -1556,6 +1583,7 @@ set_foreignscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) fscan));
fscan->fdw_exprs = (List *)
fix_upper_expr(root,
@@ -1563,6 +1591,7 @@ set_foreignscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) fscan));
fscan->fdw_recheck_quals = (List *)
fix_upper_expr(root,
@@ -1570,6 +1599,7 @@ set_foreignscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) fscan));
pfree(itlist);
/* fdw_scan_tlist itself just needs fix_scan_list() adjustments */
@@ -1630,6 +1660,7 @@ set_customscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST((Plan *) cscan));
cscan->scan.plan.qual = (List *)
fix_upper_expr(root,
@@ -1637,6 +1668,7 @@ set_customscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) cscan));
cscan->custom_exprs = (List *)
fix_upper_expr(root,
@@ -1644,6 +1676,7 @@ set_customscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) cscan));
pfree(itlist);
/* custom_scan_tlist itself just needs fix_scan_list() adjustments */
@@ -1830,6 +1863,7 @@ set_hash_references(PlannerInfo *root, Plan *plan, int rtoffset)
outer_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL(plan));
/* Hash doesn't project */
@@ -2165,6 +2199,7 @@ fix_scan_expr_mutator(Node *node, fix_scan_expr_context *context)
/* At scan level, we should always just evaluate the contained expr */
PlaceHolderVar *phv = (PlaceHolderVar *) node;
+ Assert(phv->phnullingrels == NULL);
return fix_scan_expr_mutator((Node *) phv->phexpr, context);
}
if (IsA(node, AlternativeSubPlan))
@@ -2222,6 +2257,7 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
inner_itlist,
(Index) 0,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) join));
/* Now do join-type-specific stuff */
@@ -2234,11 +2270,21 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
{
NestLoopParam *nlp = (NestLoopParam *) lfirst(lc);
+ /*
+ * Because we don't reparameterize parameterized paths to match
+ * the outer-join level at which they are used, Vars seen in the
+ * NestLoopParam expression may have nullingrels that are just a
+ * subset of those in the Vars actually available from the outer
+ * side. Not checking this exactly is a bit grotty, but the work
+ * needed to make things match up perfectly seems well out of
+ * proportion to the value.
+ */
nlp->paramval = (Var *) fix_upper_expr(root,
(Node *) nlp->paramval,
outer_itlist,
OUTER_VAR,
rtoffset,
+ NRM_SUBSET,
NUM_EXEC_TLIST(outer_plan));
/* Check we replaced any PlaceHolderVar with simple Var */
if (!(IsA(nlp->paramval, Var) &&
@@ -2256,6 +2302,7 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
inner_itlist,
(Index) 0,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) join));
}
else if (IsA(join, HashJoin))
@@ -2268,6 +2315,7 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
inner_itlist,
(Index) 0,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) join));
/*
@@ -2279,45 +2327,27 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
outer_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) join));
}
/*
* Now we need to fix up the targetlist and qpqual, which are logically
- * above the join. This means they should not re-use any input expression
- * that was computed in the nullable side of an outer join. Vars and
- * PlaceHolderVars are fine, so we can implement this restriction just by
- * clearing has_non_vars in the indexed_tlist structs.
- *
- * XXX This is a grotty workaround for the fact that we don't clearly
- * distinguish between a Var appearing below an outer join and the "same"
- * Var appearing above it. If we did, we'd not need to hack the matching
- * rules this way.
+ * above the join. This means that, if it's not an inner join, any Vars
+ * and PHVs appearing here should have nullingrels that include the
+ * effects of the outer join, ie they will have nullingrels equal to the
+ * input Vars' nullingrels plus the bit added by the outer join. We don't
+ * currently have enough info available here to identify what that should
+ * be, so we just tell fix_join_expr to accept superset nullingrels
+ * matches instead of exact ones.
*/
- switch (join->jointype)
- {
- case JOIN_LEFT:
- case JOIN_SEMI:
- case JOIN_ANTI:
- inner_itlist->has_non_vars = false;
- break;
- case JOIN_RIGHT:
- outer_itlist->has_non_vars = false;
- break;
- case JOIN_FULL:
- outer_itlist->has_non_vars = false;
- inner_itlist->has_non_vars = false;
- break;
- default:
- break;
- }
-
join->plan.targetlist = fix_join_expr(root,
join->plan.targetlist,
outer_itlist,
inner_itlist,
(Index) 0,
rtoffset,
+ (join->jointype == JOIN_INNER ? NRM_EQUAL : NRM_SUPERSET),
NUM_EXEC_TLIST((Plan *) join));
join->plan.qual = fix_join_expr(root,
join->plan.qual,
@@ -2325,6 +2355,7 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
inner_itlist,
(Index) 0,
rtoffset,
+ (join->jointype == JOIN_INNER ? NRM_EQUAL : NRM_SUPERSET),
NUM_EXEC_QUAL((Plan *) join));
pfree(outer_itlist);
@@ -2379,6 +2410,7 @@ set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset)
subplan_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST(plan));
}
else
@@ -2387,6 +2419,7 @@ set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset)
subplan_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST(plan));
tle = flatCopyTargetEntry(tle);
tle->expr = (Expr *) newexpr;
@@ -2400,6 +2433,7 @@ set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset)
subplan_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL(plan));
pfree(subplan_itlist);
@@ -2600,7 +2634,7 @@ set_dummy_tlist_references(Plan *plan, int rtoffset)
* tlist_member() searches.
*
* The result of this function is an indexed_tlist struct to pass to
- * search_indexed_tlist_for_var() or search_indexed_tlist_for_non_var().
+ * search_indexed_tlist_for_var() and siblings.
* When done, the indexed_tlist may be freed with a single pfree().
*/
static indexed_tlist *
@@ -2632,6 +2666,9 @@ build_tlist_index(List *tlist)
vinfo->varno = var->varno;
vinfo->varattno = var->varattno;
vinfo->resno = tle->resno;
+#ifdef USE_ASSERT_CHECKING
+ vinfo->varnullingrels = var->varnullingrels;
+#endif
vinfo++;
}
else if (tle->expr && IsA(tle->expr, PlaceHolderVar))
@@ -2684,6 +2721,9 @@ build_tlist_index_other_vars(List *tlist, int ignore_rel)
vinfo->varno = var->varno;
vinfo->varattno = var->varattno;
vinfo->resno = tle->resno;
+#ifdef USE_ASSERT_CHECKING
+ vinfo->varnullingrels = var->varnullingrels;
+#endif
vinfo++;
}
}
@@ -2703,10 +2743,17 @@ build_tlist_index_other_vars(List *tlist, int ignore_rel)
* modified varno/varattno (to wit, newvarno and the resno of the TLE entry).
* Also ensure that varnosyn is incremented by rtoffset.
* If no match, return NULL.
+ *
+ * In debugging builds, we cross-check the varnullingrels of the subplan
+ * output Var based on nrm_match. Most call sites should pass NRM_EQUAL
+ * indicating we expect an exact match. However, there are places where
+ * we haven't cleaned things up completely, and we have to settle for
+ * allowing subset or superset matches.
*/
static Var *
search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
- int newvarno, int rtoffset)
+ int newvarno, int rtoffset,
+ NullingRelsMatch nrm_match)
{
int varno = var->varno;
AttrNumber varattno = var->varattno;
@@ -2722,6 +2769,36 @@ search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
/* Found a match */
Var *newvar = copyVar(var);
+ /*
+ * Assert that we kept all the nullingrels machinations straight.
+ *
+ * XXX eventually reduce this to a plain Assert. Right now it's
+ * more useful to warn and keep going.
+ *
+ * XXX skip this check for system columns and whole-row Vars.
+ * That's because such Vars might be row identity Vars, which are
+ * generated without any varnullingrels. It'd be hard to do
+ * otherwise, since they're normally made very early in planning,
+ * when we haven't looked at the jointree yet and don't know which
+ * joins might null such Vars. Doesn't seem worth the expense to
+ * make them fully valid. (While it's slightly annoying that we
+ * thereby lose checking for user-written references to such
+ * columns, it seems unlikely that a bug in nullingrels logic
+ * would affect only system columns.)
+ */
+#ifdef USE_ASSERT_CHECKING
+ if (!(varattno <= 0 ||
+ (nrm_match == NRM_SUBSET ?
+ bms_is_subset(var->varnullingrels, vinfo->varnullingrels) :
+ nrm_match == NRM_SUPERSET ?
+ bms_is_subset(vinfo->varnullingrels, var->varnullingrels) :
+ bms_equal(vinfo->varnullingrels, var->varnullingrels))))
+ elog(WARNING, "bogus varnullingrels for (%d,%d): expected %s, found %s in subplan",
+ varno, varattno,
+ bmsToString(var->varnullingrels),
+ bmsToString(vinfo->varnullingrels));
+#endif
+
newvar->varno = newvarno;
newvar->varattno = vinfo->resno;
if (newvar->varnosyn > 0)
@@ -2734,15 +2811,74 @@ search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
}
/*
- * search_indexed_tlist_for_non_var --- find a non-Var in an indexed tlist
+ * search_indexed_tlist_for_phv --- find a PlaceHolderVar in an indexed tlist
*
* If a match is found, return a Var constructed to reference the tlist item.
* If no match, return NULL.
*
- * NOTE: it is a waste of time to call this unless itlist->has_ph_vars or
- * itlist->has_non_vars. Furthermore, set_join_references() relies on being
- * able to prevent matching of non-Vars by clearing itlist->has_non_vars,
- * so there's a correctness reason not to call it unless that's set.
+ * Cross-check phnullingrels as in search_indexed_tlist_for_var.
+ *
+ * NOTE: it is a waste of time to call this unless itlist->has_ph_vars.
+ */
+static Var *
+search_indexed_tlist_for_phv(PlaceHolderVar *phv,
+ indexed_tlist *itlist, int newvarno,
+ NullingRelsMatch nrm_match)
+{
+ ListCell *lc;
+
+ foreach(lc, itlist->tlist)
+ {
+ TargetEntry *tle = (TargetEntry *) lfirst(lc);
+
+ if (tle->expr && IsA(tle->expr, PlaceHolderVar))
+ {
+ PlaceHolderVar *subphv = (PlaceHolderVar *) tle->expr;
+ Var *newvar;
+
+ /*
+ * Analogously to search_indexed_tlist_for_var, we match on phid
+ * only. We don't use equal(), partially for speed but mostly
+ * because phnullingrels might not be exactly equal.
+ */
+ if (phv->phid != subphv->phid)
+ continue;
+
+ /*
+ * Assert that we kept all the nullingrels machinations straight.
+ *
+ * XXX eventually reduce this to a plain Assert. Right now it's
+ * more useful to warn and keep going.
+ */
+#ifdef USE_ASSERT_CHECKING
+ if (!(nrm_match == NRM_SUBSET ?
+ bms_is_subset(phv->phnullingrels, subphv->phnullingrels) :
+ nrm_match == NRM_SUPERSET ?
+ bms_is_subset(subphv->phnullingrels, phv->phnullingrels) :
+ bms_equal(subphv->phnullingrels, phv->phnullingrels)))
+ elog(WARNING, "bogus phnullingrels for %d: expected %s, found %s in subplan",
+ phv->phid,
+ bmsToString(phv->phnullingrels),
+ bmsToString(subphv->phnullingrels));
+#endif
+
+ /* Found a matching subplan output expression */
+ newvar = makeVarFromTargetEntry(newvarno, tle);
+ newvar->varnosyn = 0; /* wasn't ever a plain Var */
+ newvar->varattnosyn = 0;
+ return newvar;
+ }
+ }
+ return NULL; /* no match */
+}
+
+/*
+ * search_indexed_tlist_for_non_var --- find a non-Var/PHV in an indexed tlist
+ *
+ * If a match is found, return a Var constructed to reference the tlist item.
+ * If no match, return NULL.
+ *
+ * NOTE: it is a waste of time to call this unless itlist->has_non_vars.
*/
static Var *
search_indexed_tlist_for_non_var(Expr *node,
@@ -2849,6 +2985,7 @@ search_indexed_tlist_for_sortgroupref(Expr *node,
* 'acceptable_rel' is either zero or the rangetable index of a relation
* whose Vars may appear in the clause without provoking an error
* 'rtoffset': how much to increment varnos by
+ * 'nrm_match': as for search_indexed_tlist_for_var()
* 'num_exec': estimated number of executions of expression
*
* Returns the new expression tree. The original clause structure is
@@ -2861,6 +2998,7 @@ fix_join_expr(PlannerInfo *root,
indexed_tlist *inner_itlist,
Index acceptable_rel,
int rtoffset,
+ NullingRelsMatch nrm_match,
double num_exec)
{
fix_join_expr_context context;
@@ -2870,6 +3008,7 @@ fix_join_expr(PlannerInfo *root,
context.inner_itlist = inner_itlist;
context.acceptable_rel = acceptable_rel;
context.rtoffset = rtoffset;
+ context.nrm_match = nrm_match;
context.num_exec = num_exec;
return (List *) fix_join_expr_mutator((Node *) clauses, &context);
}
@@ -2891,7 +3030,8 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
newvar = search_indexed_tlist_for_var(var,
context->outer_itlist,
OUTER_VAR,
- context->rtoffset);
+ context->rtoffset,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
@@ -2902,7 +3042,8 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
newvar = search_indexed_tlist_for_var(var,
context->inner_itlist,
INNER_VAR,
- context->rtoffset);
+ context->rtoffset,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
@@ -2927,22 +3068,25 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* See if the PlaceHolderVar has bubbled up from a lower plan node */
if (context->outer_itlist && context->outer_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->outer_itlist,
- OUTER_VAR);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->outer_itlist,
+ OUTER_VAR,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
if (context->inner_itlist && context->inner_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->inner_itlist,
- INNER_VAR);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->inner_itlist,
+ INNER_VAR,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
/* If not supplied by input plans, evaluate the contained expr */
+ /* XXX can we assert something about phnullingrels? */
return fix_join_expr_mutator((Node *) phv->phexpr, context);
}
/* Try matching more complex expressions too, if tlists have any */
@@ -3001,6 +3145,7 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
* 'subplan_itlist': indexed target list for subplan (or index)
* 'newvarno': varno to use for Vars referencing tlist elements
* 'rtoffset': how much to increment varnos by
+ * 'nrm_match': as for search_indexed_tlist_for_var()
* 'num_exec': estimated number of executions of expression
*
* The resulting tree is a copy of the original in which all Var nodes have
@@ -3013,6 +3158,7 @@ fix_upper_expr(PlannerInfo *root,
indexed_tlist *subplan_itlist,
int newvarno,
int rtoffset,
+ NullingRelsMatch nrm_match,
double num_exec)
{
fix_upper_expr_context context;
@@ -3021,6 +3167,7 @@ fix_upper_expr(PlannerInfo *root,
context.subplan_itlist = subplan_itlist;
context.newvarno = newvarno;
context.rtoffset = rtoffset;
+ context.nrm_match = nrm_match;
context.num_exec = num_exec;
return fix_upper_expr_mutator(node, &context);
}
@@ -3039,7 +3186,8 @@ fix_upper_expr_mutator(Node *node, fix_upper_expr_context *context)
newvar = search_indexed_tlist_for_var(var,
context->subplan_itlist,
context->newvarno,
- context->rtoffset);
+ context->rtoffset,
+ context->nrm_match);
if (!newvar)
elog(ERROR, "variable not found in subplan target list");
return (Node *) newvar;
@@ -3051,13 +3199,15 @@ fix_upper_expr_mutator(Node *node, fix_upper_expr_context *context)
/* See if the PlaceHolderVar has bubbled up from a lower plan node */
if (context->subplan_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->subplan_itlist,
- context->newvarno);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->subplan_itlist,
+ context->newvarno,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
/* If not supplied by input plan, evaluate the contained expr */
+ /* XXX can we assert something about phnullingrels? */
return fix_upper_expr_mutator((Node *) phv->phexpr, context);
}
/* Try matching more complex expressions too, if tlist has any */
@@ -3164,6 +3314,7 @@ set_returning_clause_references(PlannerInfo *root,
NULL,
resultRelation,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST(topplan));
pfree(itlist);
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 4cec12ab19..13bbdfd7ba 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -51,17 +51,28 @@ typedef struct pullup_replace_vars_context
* pullup (set only if target_rte->lateral) */
bool *outer_hasSubLinks; /* -> outer query's hasSubLinks */
int varno; /* varno of subquery */
- bool need_phvs; /* do we need PlaceHolderVars? */
- bool wrap_non_vars; /* do we need 'em on *all* non-Vars? */
+ bool wrap_non_vars; /* do we need all non-Var outputs to be PHVs? */
Node **rv_cache; /* cache for results with PHVs */
} pullup_replace_vars_context;
-typedef struct reduce_outer_joins_state
+typedef struct reduce_outer_joins_pass1_state
{
Relids relids; /* base relids within this subtree */
bool contains_outer; /* does subtree contain outer join(s)? */
List *sub_states; /* List of states for subtree components */
-} reduce_outer_joins_state;
+} reduce_outer_joins_pass1_state;
+
+typedef struct reduce_outer_joins_pass2_state
+{
+ Relids inner_reduced; /* OJ relids reduced to plain inner joins */
+ List *partial_reduced; /* List of partially reduced FULL joins */
+} reduce_outer_joins_pass2_state;
+
+typedef struct reduce_outer_joins_partial_state
+{
+ int full_join_rti; /* RT index of a formerly-FULL join */
+ Relids unreduced_side; /* relids in its still-nullable side */
+} reduce_outer_joins_partial_state;
static Node *pull_up_sublinks_jointree_recurse(PlannerInfo *root, Node *jtnode,
Relids *relids);
@@ -70,12 +81,10 @@ static Node *pull_up_sublinks_qual_recurse(PlannerInfo *root, Node *node,
Node **jtlink2, Relids available_rels2);
static Node *pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static Node *pull_up_simple_subquery(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static Node *pull_up_simple_union_all(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte);
@@ -92,7 +101,6 @@ static Node *pull_up_simple_values(PlannerInfo *root, Node *jtnode,
static bool is_simple_values(PlannerInfo *root, RangeTblEntry *rte);
static Node *pull_up_constant_function(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static bool is_simple_union_all(Query *subquery);
static bool is_simple_union_all_recurse(Node *setOp, Query *setOpQuery,
@@ -103,24 +111,26 @@ static bool jointree_contains_lateral_outer_refs(PlannerInfo *root,
Relids safe_upper_varnos);
static void perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars_context *rvcontext,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static void replace_vars_in_jointree(Node *jtnode,
- pullup_replace_vars_context *context,
- JoinExpr *lowest_nulling_outer_join);
+ pullup_replace_vars_context *context);
static Node *pullup_replace_vars(Node *expr,
pullup_replace_vars_context *context);
static Node *pullup_replace_vars_callback(Var *var,
replace_rte_variables_context *context);
static Query *pullup_replace_vars_subquery(Query *query,
pullup_replace_vars_context *context);
-static reduce_outer_joins_state *reduce_outer_joins_pass1(Node *jtnode);
+static reduce_outer_joins_pass1_state *reduce_outer_joins_pass1(Node *jtnode);
static void reduce_outer_joins_pass2(Node *jtnode,
- reduce_outer_joins_state *state,
+ reduce_outer_joins_pass1_state *state1,
+ reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
List *forced_null_vars);
-static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode);
+static void report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
+ int rtindex, Relids relids);
+static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Relids *dropped_outer_joins);
static int get_result_relid(PlannerInfo *root, Node *jtnode);
static void remove_result_refs(PlannerInfo *root, int varno, Node *newjtloc);
static bool find_dependent_phvs(PlannerInfo *root, int varno);
@@ -761,7 +771,7 @@ pull_up_subqueries(PlannerInfo *root)
/* Recursion starts with no containing join nor appendrel */
root->parse->jointree = (FromExpr *)
pull_up_subqueries_recurse(root, (Node *) root->parse->jointree,
- NULL, NULL, NULL);
+ NULL, NULL);
/* We should still have a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
}
@@ -776,12 +786,6 @@ pull_up_subqueries(PlannerInfo *root)
* lowest_outer_join references the lowest such JoinExpr node; otherwise
* it is NULL. We use this to constrain the effects of LATERAL subqueries.
*
- * If this jointree node is within the nullable side of an outer join, then
- * lowest_nulling_outer_join references the lowest such JoinExpr node;
- * otherwise it is NULL. This forces use of the PlaceHolderVar mechanism for
- * references to non-nullable targetlist items, but only for references above
- * that join.
- *
* If we are looking at a member subquery of an append relation,
* containing_appendrel describes that relation; else it is NULL.
* This forces use of the PlaceHolderVar mechanism for all non-Var targetlist
@@ -798,15 +802,14 @@ pull_up_subqueries(PlannerInfo *root)
* Notice also that we can't turn pullup_replace_vars loose on the whole
* jointree, because it'd return a mutated copy of the tree; we have to
* invoke it just on the quals, instead. This behavior is what makes it
- * reasonable to pass lowest_outer_join and lowest_nulling_outer_join as
- * pointers rather than some more-indirect way of identifying the lowest
- * OJs. Likewise, we don't replace append_rel_list members but only their
- * substructure, so the containing_appendrel reference is safe to use.
+ * reasonable to pass lowest_outer_join as a pointer rather than some
+ * more-indirect way of identifying the lowest OJ. Likewise, we don't
+ * replace append_rel_list members but only their substructure, so the
+ * containing_appendrel reference is safe to use.
*/
static Node *
pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
/* Since this function recurses, it could be driven to stack overflow. */
@@ -833,7 +836,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
is_safe_append_member(rte->subquery)))
return pull_up_simple_subquery(root, jtnode, rte,
lowest_outer_join,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -866,7 +868,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
*/
if (rte->rtekind == RTE_FUNCTION)
return pull_up_constant_function(root, jtnode, rte,
- lowest_nulling_outer_join,
containing_appendrel);
/* Otherwise, do nothing at this node. */
@@ -882,7 +883,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
{
lfirst(l) = pull_up_subqueries_recurse(root, lfirst(l),
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
}
}
@@ -897,11 +897,9 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
case JOIN_INNER:
j->larg = pull_up_subqueries_recurse(root, j->larg,
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
break;
case JOIN_LEFT:
@@ -909,31 +907,25 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
case JOIN_ANTI:
j->larg = pull_up_subqueries_recurse(root, j->larg,
j,
- lowest_nulling_outer_join,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
- j,
j,
NULL);
break;
case JOIN_FULL:
j->larg = pull_up_subqueries_recurse(root, j->larg,
- j,
j,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
- j,
j,
NULL);
break;
case JOIN_RIGHT:
j->larg = pull_up_subqueries_recurse(root, j->larg,
- j,
j,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
j,
- lowest_nulling_outer_join,
NULL);
break;
default:
@@ -963,7 +955,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
static Node *
pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -1110,31 +1101,25 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* The subquery's targetlist items are now in the appropriate form to
* insert into the top query, except that we may need to wrap them in
* PlaceHolderVars. Set up required context data for pullup_replace_vars.
+ * (Note that we should include the subquery's inner joins in relids,
+ * since it may include join alias vars referencing them.)
*/
rvcontext.root = root;
rvcontext.targetlist = subquery->targetList;
rvcontext.target_rte = rte;
if (rte->lateral)
rvcontext.relids = get_relids_in_jointree((Node *) subquery->jointree,
- true);
+ true, true);
else /* won't need relids */
rvcontext.relids = NULL;
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = varno;
- /* these flags will be set below, if needed */
- rvcontext.need_phvs = false;
+ /* this flag will be set below, if needed */
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(subquery->targetList) + 1) *
sizeof(Node *));
- /*
- * If we are under an outer join then non-nullable items and lateral
- * references may have to be turned into PlaceHolderVars.
- */
- if (lowest_nulling_outer_join != NULL)
- rvcontext.need_phvs = true;
-
/*
* If we are dealing with an appendrel member then anything that's not a
* simple Var has to be turned into a PlaceHolderVar. We force this to
@@ -1143,10 +1128,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* expression actually available from the appendrel.
*/
if (containing_appendrel != NULL)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* If the parent query uses grouping sets, we need a PlaceHolderVar for
@@ -1158,10 +1140,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* that pullup_replace_vars hasn't currently got.)
*/
if (parse->groupingSets)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* Replace all of the top query's references to the subquery's outputs
@@ -1169,7 +1148,6 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* replace any of the jointree structure.
*/
perform_pullup_replace_vars(root, &rvcontext,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -1236,7 +1214,8 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
{
Relids subrelids;
- subrelids = get_relids_in_jointree((Node *) subquery->jointree, false);
+ subrelids = get_relids_in_jointree((Node *) subquery->jointree,
+ true, false);
if (root->glob->lastPHId != 0)
substitute_phv_relids((Node *) parse, varno, subrelids);
fix_append_rel_relids(root, varno, subrelids);
@@ -1432,7 +1411,7 @@ pull_up_union_leaf_queries(Node *setOp, PlannerInfo *root, int parentRTindex,
rtr = makeNode(RangeTblRef);
rtr->rtindex = childRTindex;
(void) pull_up_subqueries_recurse(root, (Node *) rtr,
- NULL, NULL, appinfo);
+ NULL, appinfo);
}
else if (IsA(setOp, SetOperationStmt))
{
@@ -1569,7 +1548,7 @@ is_simple_subquery(PlannerInfo *root, Query *subquery, RangeTblEntry *rte,
{
restricted = true;
safe_upper_varnos = get_relids_in_jointree((Node *) lowest_outer_join,
- true);
+ true, true);
}
else
{
@@ -1681,7 +1660,6 @@ pull_up_simple_values(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte)
rvcontext.relids = NULL;
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = varno;
- rvcontext.need_phvs = false;
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(tlist) + 1) *
@@ -1693,7 +1671,7 @@ pull_up_simple_values(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte)
* any of the jointree structure. We can assume there's no outer joins or
* appendrels in the dummy Query that surrounds a VALUES RTE.
*/
- perform_pullup_replace_vars(root, &rvcontext, NULL, NULL);
+ perform_pullup_replace_vars(root, &rvcontext, NULL);
/*
* There should be no appendrels to fix, nor any outer joins and hence no
@@ -1792,7 +1770,6 @@ is_simple_values(PlannerInfo *root, RangeTblEntry *rte)
static Node *
pull_up_constant_function(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -1844,40 +1821,26 @@ pull_up_constant_function(PlannerInfo *root, Node *jtnode,
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = ((RangeTblRef *) jtnode)->rtindex;
- /* these flags will be set below, if needed */
- rvcontext.need_phvs = false;
+ /* this flag will be set below, if needed */
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(rvcontext.targetlist) + 1) *
sizeof(Node *));
- /*
- * If we are under an outer join then non-nullable items and lateral
- * references may have to be turned into PlaceHolderVars.
- */
- if (lowest_nulling_outer_join != NULL)
- rvcontext.need_phvs = true;
-
/*
* If we are dealing with an appendrel member then anything that's not a
* simple Var has to be turned into a PlaceHolderVar. (See comments in
* pull_up_simple_subquery().)
*/
if (containing_appendrel != NULL)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* If the parent query uses grouping sets, we need a PlaceHolderVar for
* anything that's not a simple Var.
*/
if (parse->groupingSets)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* Replace all of the top query's references to the RTE's output with
@@ -1885,7 +1848,6 @@ pull_up_constant_function(PlannerInfo *root, Node *jtnode,
* jointree structure.
*/
perform_pullup_replace_vars(root, &rvcontext,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -2110,13 +2072,11 @@ jointree_contains_lateral_outer_refs(PlannerInfo *root, Node *jtnode,
*
* Caller has already filled *rvcontext with data describing what to
* substitute for Vars referencing the target subquery. In addition
- * we need the identity of the lowest outer join that can null the
- * target subquery, and its containing appendrel if any.
+ * we need the identity of the containing appendrel if any.
*/
static void
perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars_context *rvcontext,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -2126,18 +2086,18 @@ perform_pullup_replace_vars(PlannerInfo *root,
* If we are considering an appendrel child subquery (that is, a UNION ALL
* member query that we're pulling up), then the only part of the upper
* query that could reference the child yet is the translated_vars list of
- * the associated AppendRelInfo. Furthermore, we do not need to insert
- * PHVs in the AppendRelInfo --- there isn't any outer join between.
+ * the associated AppendRelInfo. Furthermore, we do not want to force use
+ * of PHVs in the AppendRelInfo --- there isn't any outer join between.
*/
if (containing_appendrel)
{
- bool save_need_phvs = rvcontext->need_phvs;
+ bool save_wrap_non_vars = rvcontext->wrap_non_vars;
- rvcontext->need_phvs = false;
+ rvcontext->wrap_non_vars = false;
containing_appendrel->translated_vars = (List *)
pullup_replace_vars((Node *) containing_appendrel->translated_vars,
rvcontext);
- rvcontext->need_phvs = save_need_phvs;
+ rvcontext->wrap_non_vars = save_wrap_non_vars;
return;
}
@@ -2188,8 +2148,7 @@ perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars((Node *) action->targetList, rvcontext);
}
}
- replace_vars_in_jointree((Node *) parse->jointree, rvcontext,
- lowest_nulling_outer_join);
+ replace_vars_in_jointree((Node *) parse->jointree, rvcontext);
Assert(parse->setOperations == NULL);
parse->havingQual = pullup_replace_vars(parse->havingQual, rvcontext);
@@ -2206,12 +2165,6 @@ perform_pullup_replace_vars(PlannerInfo *root,
/*
* Replace references in the joinaliasvars lists of join RTEs.
- *
- * You might think that we could avoid using PHVs for alias vars of joins
- * below lowest_nulling_outer_join, but that doesn't work because the
- * alias vars could be referenced above that join; we need the PHVs to be
- * present in such references after the alias vars get flattened. (It
- * might be worth trying to be smarter here, someday.)
*/
foreach(lc, parse->rtable)
{
@@ -2228,14 +2181,10 @@ perform_pullup_replace_vars(PlannerInfo *root,
* Helper routine for perform_pullup_replace_vars: do pullup_replace_vars on
* every expression in the jointree, without changing the jointree structure
* itself. Ugly, but there's no other way...
- *
- * If we are at or below lowest_nulling_outer_join, we can suppress use of
- * PlaceHolderVars wrapped around the replacement expressions.
*/
static void
replace_vars_in_jointree(Node *jtnode,
- pullup_replace_vars_context *context,
- JoinExpr *lowest_nulling_outer_join)
+ pullup_replace_vars_context *context)
{
if (jtnode == NULL)
return;
@@ -2245,10 +2194,8 @@ replace_vars_in_jointree(Node *jtnode,
* If the RangeTblRef refers to a LATERAL subquery (that isn't the
* same subquery we're pulling up), it might contain references to the
* target subquery, which we must replace. We drive this from the
- * jointree scan, rather than a scan of the rtable, for a couple of
- * reasons: we can avoid processing no-longer-referenced RTEs, and we
- * can use the appropriate setting of need_phvs depending on whether
- * the RTE is above possibly-nulling outer joins or not.
+ * jointree scan, rather than a scan of the rtable, so that we can
+ * avoid processing no-longer-referenced RTEs.
*/
int varno = ((RangeTblRef *) jtnode)->rtindex;
@@ -2305,42 +2252,30 @@ replace_vars_in_jointree(Node *jtnode,
ListCell *l;
foreach(l, f->fromlist)
- replace_vars_in_jointree(lfirst(l), context,
- lowest_nulling_outer_join);
+ replace_vars_in_jointree(lfirst(l), context);
f->quals = pullup_replace_vars(f->quals, context);
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- bool save_need_phvs = context->need_phvs;
+ bool save_wrap_non_vars = context->wrap_non_vars;
- if (j == lowest_nulling_outer_join)
- {
- /* no more PHVs in or below this join */
- context->need_phvs = false;
- lowest_nulling_outer_join = NULL;
- }
- replace_vars_in_jointree(j->larg, context, lowest_nulling_outer_join);
- replace_vars_in_jointree(j->rarg, context, lowest_nulling_outer_join);
+ replace_vars_in_jointree(j->larg, context);
+ replace_vars_in_jointree(j->rarg, context);
/*
- * Use PHVs within the join quals of a full join, even when it's the
- * lowest nulling outer join. Otherwise, we cannot identify which
- * side of the join a pulled-up var-free expression came from, which
- * can lead to failure to make a plan at all because none of the quals
- * appear to be mergeable or hashable conditions. For this purpose we
- * don't care about the state of wrap_non_vars, so leave it alone.
+ * Use PHVs within the join quals of a full join. Otherwise, we
+ * cannot identify which side of the join a pulled-up var-free
+ * expression came from, which can lead to failure to make a plan at
+ * all because none of the quals appear to be mergeable or hashable
+ * conditions.
*/
if (j->jointype == JOIN_FULL)
- context->need_phvs = true;
+ context->wrap_non_vars = true;
j->quals = pullup_replace_vars(j->quals, context);
- /*
- * We don't bother to update the colvars list, since it won't be used
- * again ...
- */
- context->need_phvs = save_need_phvs;
+ context->wrap_non_vars = save_wrap_non_vars;
}
else
elog(ERROR, "unrecognized node type: %d",
@@ -2369,8 +2304,18 @@ pullup_replace_vars_callback(Var *var,
{
pullup_replace_vars_context *rcon = (pullup_replace_vars_context *) context->callback_arg;
int varattno = var->varattno;
+ bool need_phv;
Node *newnode;
+ /*
+ * We need a PlaceHolderVar if the Var-to-be-replaced has nonempty
+ * varnullingrels (unless we find below that the replacement expression is
+ * a Var or PlaceHolderVar that we can just add the nullingrels to). We
+ * also need one if the caller has instructed us that all non-Var/PHV
+ * replacements need to be wrapped for identification purposes.
+ */
+ need_phv = (var->varnullingrels != NULL) || rcon->wrap_non_vars;
+
/*
* If PlaceHolderVars are needed, we cache the modified expressions in
* rcon->rv_cache[]. This is not in hopes of any material speed gain
@@ -2379,13 +2324,16 @@ pullup_replace_vars_callback(Var *var,
* and possibly prevent optimizations that rely on recognizing different
* references to the same subquery output as being equal(). So it's worth
* a bit of extra effort to avoid it.
+ *
+ * The cached items have phlevelsup = 0 and phnullingrels = NULL; we'll
+ * copy them and adjust those values for this reference site below.
*/
- if (rcon->need_phvs &&
+ if (need_phv &&
varattno >= InvalidAttrNumber &&
varattno <= list_length(rcon->targetlist) &&
rcon->rv_cache[varattno] != NULL)
{
- /* Just copy the entry and fall through to adjust its varlevelsup */
+ /* Just copy the entry and fall through to adjust phlevelsup etc */
newnode = copyObject(rcon->rv_cache[varattno]);
}
else if (varattno == InvalidAttrNumber)
@@ -2394,7 +2342,7 @@ pullup_replace_vars_callback(Var *var,
RowExpr *rowexpr;
List *colnames;
List *fields;
- bool save_need_phvs = rcon->need_phvs;
+ bool save_wrap_non_vars = rcon->wrap_non_vars;
int save_sublevelsup = context->sublevels_up;
/*
@@ -2405,18 +2353,18 @@ pullup_replace_vars_callback(Var *var,
* the RowExpr for use of the executor and ruleutils.c.
*
* In order to be able to cache the results, we always generate the
- * expansion with varlevelsup = 0, and then adjust if needed.
+ * expansion with varlevelsup = 0, and then adjust below if needed.
*/
expandRTE(rcon->target_rte,
var->varno, 0 /* not varlevelsup */ , var->location,
(var->vartype != RECORDOID),
&colnames, &fields);
- /* Adjust the generated per-field Vars, but don't insert PHVs */
- rcon->need_phvs = false;
+ /* Expand the generated per-field Vars, but don't insert PHVs there */
+ rcon->wrap_non_vars = false;
context->sublevels_up = 0; /* to match the expandRTE output */
fields = (List *) replace_rte_variables_mutator((Node *) fields,
context);
- rcon->need_phvs = save_need_phvs;
+ rcon->wrap_non_vars = save_wrap_non_vars;
context->sublevels_up = save_sublevelsup;
rowexpr = makeNode(RowExpr);
@@ -2434,14 +2382,13 @@ pullup_replace_vars_callback(Var *var,
* expression to yield NULL, not ROW(NULL,NULL,...) when it is forced
* to null by an outer join.
*/
- if (rcon->need_phvs)
+ if (need_phv)
{
- /* RowExpr is certainly not strict, so always need PHV */
newnode = (Node *)
make_placeholder_expr(rcon->root,
(Expr *) newnode,
bms_make_singleton(rcon->varno));
- /* cache it with the PHV, and with varlevelsup still zero */
+ /* cache it with the PHV, and with phlevelsup etc not set yet */
rcon->rv_cache[InvalidAttrNumber] = copyObject(newnode);
}
}
@@ -2458,7 +2405,7 @@ pullup_replace_vars_callback(Var *var,
newnode = (Node *) copyObject(tle->expr);
/* Insert PlaceHolderVar if needed */
- if (rcon->need_phvs)
+ if (need_phv)
{
bool wrap;
@@ -2484,69 +2431,61 @@ pullup_replace_vars_callback(Var *var,
/* No need to wrap a PlaceHolderVar with another one, either */
wrap = false;
}
- else if (rcon->wrap_non_vars)
- {
- /* Wrap all non-Vars in a PlaceHolderVar */
- wrap = true;
- }
else
{
/*
- * If it contains a Var of the subquery being pulled up, and
- * does not contain any non-strict constructs, then it's
- * certainly nullable so we don't need to insert a
- * PlaceHolderVar.
- *
- * This analysis could be tighter: in particular, a non-strict
- * construct hidden within a lower-level PlaceHolderVar is not
- * reason to add another PHV. But for now it doesn't seem
- * worth the code to be more exact.
- *
- * Note: in future maybe we should insert a PlaceHolderVar
- * anyway, if the tlist item is expensive to evaluate?
- *
- * For a LATERAL subquery, we have to check the actual var
- * membership of the node, but if it's non-lateral then any
- * level-zero var must belong to the subquery.
+ * Must wrap, either because we need a place to insert
+ * varnullingrels or because caller told us to wrap
+ * everything.
*/
- if ((rcon->target_rte->lateral ?
- bms_overlap(pull_varnos(rcon->root, (Node *) newnode),
- rcon->relids) :
- contain_vars_of_level((Node *) newnode, 0)) &&
- !contain_nonstrict_functions((Node *) newnode))
- {
- /* No wrap needed */
- wrap = false;
- }
- else
- {
- /* Else wrap it in a PlaceHolderVar */
- wrap = true;
- }
+ wrap = true;
}
if (wrap)
+ {
newnode = (Node *)
make_placeholder_expr(rcon->root,
(Expr *) newnode,
bms_make_singleton(rcon->varno));
- /*
- * Cache it if possible (ie, if the attno is in range, which it
- * probably always should be). We can cache the value even if we
- * decided we didn't need a PHV, since this result will be
- * suitable for any request that has need_phvs.
- */
- if (varattno > InvalidAttrNumber &&
- varattno <= list_length(rcon->targetlist))
- rcon->rv_cache[varattno] = copyObject(newnode);
+ /*
+ * Cache it if possible (ie, if the attno is in range, which
+ * it probably always should be).
+ */
+ if (varattno > InvalidAttrNumber &&
+ varattno <= list_length(rcon->targetlist))
+ rcon->rv_cache[varattno] = copyObject(newnode);
+ }
}
}
- /* Must adjust varlevelsup if tlist item is from higher query */
+ /* Must adjust varlevelsup if replaced Var is within a subquery */
if (var->varlevelsup > 0)
IncrementVarSublevelsUp(newnode, var->varlevelsup, 0);
+ /* Propagate any varnullingrels into the replacement Var or PHV */
+ if (var->varnullingrels != NULL)
+ {
+ if (IsA(newnode, Var))
+ {
+ Var *newvar = (Var *) newnode;
+
+ Assert(newvar->varlevelsup == var->varlevelsup);
+ newvar->varnullingrels = bms_add_members(newvar->varnullingrels,
+ var->varnullingrels);
+ }
+ else if (IsA(newnode, PlaceHolderVar))
+ {
+ PlaceHolderVar *newphv = (PlaceHolderVar *) newnode;
+
+ Assert(newphv->phlevelsup == var->varlevelsup);
+ newphv->phnullingrels = bms_add_members(newphv->phnullingrels,
+ var->varnullingrels);
+ }
+ else
+ elog(ERROR, "failed to wrap a non-Var");
+ }
+
return newnode;
}
@@ -2705,7 +2644,9 @@ flatten_simple_union_all(PlannerInfo *root)
void
reduce_outer_joins(PlannerInfo *root)
{
- reduce_outer_joins_state *state;
+ reduce_outer_joins_pass1_state *state1;
+ reduce_outer_joins_pass2_state state2;
+ ListCell *lc;
/*
* To avoid doing strictness checks on more quals than necessary, we want
@@ -2716,14 +2657,56 @@ reduce_outer_joins(PlannerInfo *root)
* join(s) below each side of each join clause. The second pass examines
* qual clauses and changes join types as it descends the tree.
*/
- state = reduce_outer_joins_pass1((Node *) root->parse->jointree);
+ state1 = reduce_outer_joins_pass1((Node *) root->parse->jointree);
/* planner.c shouldn't have called me if no outer joins */
- if (state == NULL || !state->contains_outer)
+ if (state1 == NULL || !state1->contains_outer)
elog(ERROR, "so where are the outer joins?");
+ state2.inner_reduced = NULL;
+ state2.partial_reduced = NIL;
+
reduce_outer_joins_pass2((Node *) root->parse->jointree,
- state, root, NULL, NIL);
+ state1, &state2,
+ root, NULL, NIL);
+
+ /*
+ * If we successfully reduced the strength of any outer joins, we must
+ * remove references to those joins as nulling rels. This is handled as
+ * an additional pass, for simplicity and because we can handle all
+ * fully-reduced joins in a single pass over the parse tree.
+ */
+ if (!bms_is_empty(state2.inner_reduced))
+ {
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ state2.inner_reduced,
+ NULL);
+ /* There could be references in the append_rel_list, too */
+ root->append_rel_list = (List *)
+ remove_nulling_relids((Node *) root->append_rel_list,
+ state2.inner_reduced,
+ NULL);
+ }
+
+ /*
+ * Partially-reduced full joins have to be done one at a time, since
+ * they'll each need a different setting of except_relids.
+ */
+ foreach(lc, state2.partial_reduced)
+ {
+ reduce_outer_joins_partial_state *statep = lfirst(lc);
+ Relids full_join_relids = bms_make_singleton(statep->full_join_rti);
+
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ full_join_relids,
+ statep->unreduced_side);
+ root->append_rel_list = (List *)
+ remove_nulling_relids((Node *) root->append_rel_list,
+ full_join_relids,
+ statep->unreduced_side);
+ }
}
/*
@@ -2731,13 +2714,13 @@ reduce_outer_joins(PlannerInfo *root)
*
* Returns a state node describing the given jointree node.
*/
-static reduce_outer_joins_state *
+static reduce_outer_joins_pass1_state *
reduce_outer_joins_pass1(Node *jtnode)
{
- reduce_outer_joins_state *result;
+ reduce_outer_joins_pass1_state *result;
- result = (reduce_outer_joins_state *)
- palloc(sizeof(reduce_outer_joins_state));
+ result = (reduce_outer_joins_pass1_state *)
+ palloc(sizeof(reduce_outer_joins_pass1_state));
result->relids = NULL;
result->contains_outer = false;
result->sub_states = NIL;
@@ -2757,7 +2740,7 @@ reduce_outer_joins_pass1(Node *jtnode)
foreach(l, f->fromlist)
{
- reduce_outer_joins_state *sub_state;
+ reduce_outer_joins_pass1_state *sub_state;
sub_state = reduce_outer_joins_pass1(lfirst(l));
result->relids = bms_add_members(result->relids,
@@ -2769,7 +2752,7 @@ reduce_outer_joins_pass1(Node *jtnode)
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- reduce_outer_joins_state *sub_state;
+ reduce_outer_joins_pass1_state *sub_state;
/* join's own RT index is not wanted in result->relids */
if (IS_OUTER_JOIN(j->jointype))
@@ -2797,14 +2780,22 @@ reduce_outer_joins_pass1(Node *jtnode)
* reduce_outer_joins_pass2 - phase 2 processing
*
* jtnode: current jointree node
- * state: state data collected by phase 1 for this node
+ * state1: state data collected by phase 1 for this node
+ * state2: where to accumulate info about successfully-reduced joins
* root: toplevel planner state
* nonnullable_rels: set of base relids forced non-null by upper quals
* forced_null_vars: multibitmapset of Vars forced null by upper quals
+ *
+ * Returns info in state2 about outer joins that were successfully simplified.
+ * Joins that were fully reduced to inner joins are all added to
+ * state2->inner_reduced. If a full join is reduced to a left join,
+ * it needs its own entry in state2->partial_reduced, since that will
+ * require custom processing to remove only the correct nullingrel markers.
*/
static void
reduce_outer_joins_pass2(Node *jtnode,
- reduce_outer_joins_state *state,
+ reduce_outer_joins_pass1_state *state1,
+ reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
List *forced_null_vars)
@@ -2833,13 +2824,14 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_forced_null_vars = mbms_add_members(pass_forced_null_vars,
forced_null_vars);
/* And recurse --- but only into interesting subtrees */
- Assert(list_length(f->fromlist) == list_length(state->sub_states));
- forboth(l, f->fromlist, s, state->sub_states)
+ Assert(list_length(f->fromlist) == list_length(state1->sub_states));
+ forboth(l, f->fromlist, s, state1->sub_states)
{
- reduce_outer_joins_state *sub_state = lfirst(s);
+ reduce_outer_joins_pass1_state *sub_state = lfirst(s);
if (sub_state->contains_outer)
- reduce_outer_joins_pass2(lfirst(l), sub_state, root,
+ reduce_outer_joins_pass2(lfirst(l), sub_state,
+ state2, root,
pass_nonnullable_rels,
pass_forced_null_vars);
}
@@ -2851,8 +2843,8 @@ reduce_outer_joins_pass2(Node *jtnode,
JoinExpr *j = (JoinExpr *) jtnode;
int rtindex = j->rtindex;
JoinType jointype = j->jointype;
- reduce_outer_joins_state *left_state = linitial(state->sub_states);
- reduce_outer_joins_state *right_state = lsecond(state->sub_states);
+ reduce_outer_joins_pass1_state *left_state = linitial(state1->sub_states);
+ reduce_outer_joins_pass1_state *right_state = lsecond(state1->sub_states);
/* Can we simplify this join? */
switch (jointype)
@@ -2873,12 +2865,22 @@ reduce_outer_joins_pass2(Node *jtnode,
if (bms_overlap(nonnullable_rels, right_state->relids))
jointype = JOIN_INNER;
else
+ {
jointype = JOIN_LEFT;
+ /* Also report partial reduction in state2 */
+ report_reduced_full_join(state2, rtindex,
+ right_state->relids);
+ }
}
else
{
if (bms_overlap(nonnullable_rels, right_state->relids))
+ {
jointype = JOIN_RIGHT;
+ /* Also report partial reduction in state2 */
+ report_reduced_full_join(state2, rtindex,
+ left_state->relids);
+ }
}
break;
case JOIN_SEMI:
@@ -2911,8 +2913,8 @@ reduce_outer_joins_pass2(Node *jtnode,
j->larg = j->rarg;
j->rarg = tmparg;
jointype = JOIN_LEFT;
- right_state = linitial(state->sub_states);
- left_state = lsecond(state->sub_states);
+ right_state = linitial(state1->sub_states);
+ left_state = lsecond(state1->sub_states);
}
/*
@@ -2943,7 +2945,10 @@ reduce_outer_joins_pass2(Node *jtnode,
jointype = JOIN_ANTI;
}
- /* Apply the jointype change, if any, to both jointree node and RTE */
+ /*
+ * Apply the jointype change, if any, to both jointree node and RTE.
+ * Also, if we changed an RTE to INNER, add its RTI to inner_reduced.
+ */
if (rtindex && jointype != j->jointype)
{
RangeTblEntry *rte = rt_fetch(rtindex, root->parse->rtable);
@@ -2951,6 +2956,9 @@ reduce_outer_joins_pass2(Node *jtnode,
Assert(rte->rtekind == RTE_JOIN);
Assert(rte->jointype == j->jointype);
rte->jointype = jointype;
+ if (jointype == JOIN_INNER)
+ state2->inner_reduced = bms_add_member(state2->inner_reduced,
+ rtindex);
}
j->jointype = jointype;
@@ -3023,7 +3031,8 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_nonnullable_rels = NULL;
pass_forced_null_vars = NIL;
}
- reduce_outer_joins_pass2(j->larg, left_state, root,
+ reduce_outer_joins_pass2(j->larg, left_state,
+ state2, root,
pass_nonnullable_rels,
pass_forced_null_vars);
}
@@ -3042,7 +3051,8 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_nonnullable_rels = NULL;
pass_forced_null_vars = NIL;
}
- reduce_outer_joins_pass2(j->rarg, right_state, root,
+ reduce_outer_joins_pass2(j->rarg, right_state,
+ state2, root,
pass_nonnullable_rels,
pass_forced_null_vars);
}
@@ -3054,6 +3064,19 @@ reduce_outer_joins_pass2(Node *jtnode,
(int) nodeTag(jtnode));
}
+/* Helper for reduce_outer_joins_pass2 */
+static void
+report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
+ int rtindex, Relids relids)
+{
+ reduce_outer_joins_partial_state *statep;
+
+ statep = palloc(sizeof(reduce_outer_joins_partial_state));
+ statep->full_join_rti = rtindex;
+ statep->unreduced_side = relids;
+ state2->partial_reduced = lappend(state2->partial_reduced, statep);
+}
+
/*
* remove_useless_result_rtes
@@ -3095,16 +3118,41 @@ reduce_outer_joins_pass2(Node *jtnode,
void
remove_useless_result_rtes(PlannerInfo *root)
{
+ Relids dropped_outer_joins = NULL;
ListCell *cell;
/* Top level of jointree must always be a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
/* Recurse ... */
root->parse->jointree = (FromExpr *)
- remove_useless_results_recurse(root, (Node *) root->parse->jointree);
+ remove_useless_results_recurse(root,
+ (Node *) root->parse->jointree,
+ &dropped_outer_joins);
/* We should still have a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
+ /*
+ * If we removed any outer-join nodes from the jointree, run around and
+ * remove references to those joins as nulling rels. (There could be such
+ * references in PHVs that we pulled up out of the original subquery that
+ * the RESULT rel replaced. This is kosher on the grounds that we now
+ * know that such an outer join wouldn't really have nulled anything.) We
+ * don't do this during the main recursion, for simplicity and because we
+ * can handle all such joins in a single pass over the parse tree.
+ */
+ if (!bms_is_empty(dropped_outer_joins))
+ {
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ dropped_outer_joins,
+ NULL);
+ /* There could be references in the append_rel_list, too */
+ root->append_rel_list = (List *)
+ remove_nulling_relids((Node *) root->append_rel_list,
+ dropped_outer_joins,
+ NULL);
+ }
+
/*
* Remove any PlanRowMark referencing an RTE_RESULT RTE. We obviously
* must do that for any RTE_RESULT that we just removed. But one for a
@@ -3130,9 +3178,12 @@ remove_useless_result_rtes(PlannerInfo *root)
* Recursive guts of remove_useless_result_rtes.
*
* This recursively processes the jointree and returns a modified jointree.
+ * In addition, the RT indexes of any removed outer-join nodes are added to
+ * *dropped_outer_joins.
*/
static Node *
-remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
+remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Relids *dropped_outer_joins)
{
Assert(jtnode != NULL);
if (IsA(jtnode, RangeTblRef))
@@ -3160,7 +3211,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
int varno;
/* Recursively transform child ... */
- child = remove_useless_results_recurse(root, child);
+ child = remove_useless_results_recurse(root, child,
+ dropped_outer_joins);
/* ... and stick it back into the tree */
lfirst(cell) = child;
@@ -3209,8 +3261,10 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
int varno;
/* First, recurse */
- j->larg = remove_useless_results_recurse(root, j->larg);
- j->rarg = remove_useless_results_recurse(root, j->rarg);
+ j->larg = remove_useless_results_recurse(root, j->larg,
+ dropped_outer_joins);
+ j->rarg = remove_useless_results_recurse(root, j->rarg,
+ dropped_outer_joins);
/* Apply join-type-specific optimization rules */
switch (j->jointype)
@@ -3278,6 +3332,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
!find_dependent_phvs(root, varno)))
{
remove_result_refs(root, varno, j->larg);
+ *dropped_outer_joins = bms_add_member(*dropped_outer_joins,
+ j->rtindex);
jtnode = j->larg;
}
break;
@@ -3297,9 +3353,13 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
* it'd be OK to just remove the PHV wrapping. We don't have
* infrastructure for that, but remove_result_refs() will
* relabel them as to be evaluated at the LHS, which is fine.
+ *
+ * Also, we don't need to worry about removing traces of the
+ * join's rtindex, since it hasn't got one.
*/
if ((varno = get_result_relid(root, j->rarg)) != 0)
{
+ Assert(j->rtindex == 0);
remove_result_refs(root, varno, j->larg);
if (j->quals)
jtnode = (Node *)
@@ -3369,7 +3429,7 @@ remove_result_refs(PlannerInfo *root, int varno, Node *newjtloc)
{
Relids subrelids;
- subrelids = get_relids_in_jointree(newjtloc, false);
+ subrelids = get_relids_in_jointree(newjtloc, true, false);
Assert(!bms_is_empty(subrelids));
substitute_phv_relids((Node *) root->parse, varno, subrelids);
fix_append_rel_relids(root, varno, subrelids);
@@ -3426,9 +3486,8 @@ find_dependent_phvs_walker(Node *node,
context->sublevels_up--;
return result;
}
- /* Shouldn't need to handle planner auxiliary nodes here */
+ /* Shouldn't need to handle most planner auxiliary nodes here */
Assert(!IsA(node, SpecialJoinInfo));
- Assert(!IsA(node, AppendRelInfo));
Assert(!IsA(node, PlaceHolderInfo));
Assert(!IsA(node, MinMaxAggInfo));
@@ -3448,10 +3507,17 @@ find_dependent_phvs(PlannerInfo *root, int varno)
context.relids = bms_make_singleton(varno);
context.sublevels_up = 0;
- return query_tree_walker(root->parse,
- find_dependent_phvs_walker,
- (void *) &context,
- 0);
+ if (query_tree_walker(root->parse,
+ find_dependent_phvs_walker,
+ (void *) &context,
+ 0))
+ return true;
+ /* The append_rel_list could be populated already, so check it too */
+ if (expression_tree_walker((Node *) root->append_rel_list,
+ find_dependent_phvs_walker,
+ (void *) &context))
+ return true;
+ return false;
}
static bool
@@ -3481,7 +3547,7 @@ find_dependent_phvs_in_jointree(PlannerInfo *root, Node *node, int varno)
* are not marked LATERAL, though, since they couldn't possibly contain
* any cross-references to other RTEs.
*/
- subrelids = get_relids_in_jointree(node, false);
+ subrelids = get_relids_in_jointree(node, false, false);
relid = -1;
while ((relid = bms_next_member(subrelids, relid)) >= 0)
{
@@ -3626,11 +3692,17 @@ fix_append_rel_relids(PlannerInfo *root, int varno, Relids subrelids)
/*
* get_relids_in_jointree: get set of RT indexes present in a jointree
*
- * If include_joins is true, join RT indexes are included; if false,
- * only base rels are included.
+ * Base-relation relids are always included in the result.
+ * If include_outer_joins is true, outer-join RT indexes are included.
+ * If include_inner_joins is true, inner-join RT indexes are included.
+ *
+ * Note that for most purposes in the planner, outer joins are included
+ * in standard relid sets. Setting include_inner_joins true is only
+ * appropriate for special purposes during subquery flattening.
*/
Relids
-get_relids_in_jointree(Node *jtnode, bool include_joins)
+get_relids_in_jointree(Node *jtnode, bool include_outer_joins,
+ bool include_inner_joins)
{
Relids result = NULL;
@@ -3651,18 +3723,34 @@ get_relids_in_jointree(Node *jtnode, bool include_joins)
{
result = bms_join(result,
get_relids_in_jointree(lfirst(l),
- include_joins));
+ include_outer_joins,
+ include_inner_joins));
}
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- result = get_relids_in_jointree(j->larg, include_joins);
+ result = get_relids_in_jointree(j->larg,
+ include_outer_joins,
+ include_inner_joins);
result = bms_join(result,
- get_relids_in_jointree(j->rarg, include_joins));
- if (include_joins && j->rtindex)
- result = bms_add_member(result, j->rtindex);
+ get_relids_in_jointree(j->rarg,
+ include_outer_joins,
+ include_inner_joins));
+ if (j->rtindex)
+ {
+ if (j->jointype == JOIN_INNER)
+ {
+ if (include_inner_joins)
+ result = bms_add_member(result, j->rtindex);
+ }
+ else
+ {
+ if (include_outer_joins)
+ result = bms_add_member(result, j->rtindex);
+ }
+ }
}
else
elog(ERROR, "unrecognized node type: %d",
@@ -3671,7 +3759,7 @@ get_relids_in_jointree(Node *jtnode, bool include_joins)
}
/*
- * get_relids_for_join: get set of base RT indexes making up a join
+ * get_relids_for_join: get set of base+OJ RT indexes making up a join
*/
Relids
get_relids_for_join(Query *query, int joinrelid)
@@ -3682,7 +3770,7 @@ get_relids_for_join(Query *query, int joinrelid)
joinrelid);
if (!jtnode)
elog(ERROR, "could not find join node %d", joinrelid);
- return get_relids_in_jointree(jtnode, false);
+ return get_relids_in_jointree(jtnode, true, false);
}
/*
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index f6fc62aa5d..11c6bbaba6 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -228,6 +228,14 @@ adjust_appendrel_attrs_mutator(Node *node,
if (var->varlevelsup != 0)
return (Node *) var; /* no changes needed */
+ /*
+ * You might think we need to adjust var->varnullingrels, but that
+ * shouldn't need any changes. It will contain outer-join relids,
+ * while the transformation we are making affects only baserels.
+ * Below, we just propagate var->varnullingrels into the translated
+ * Var. (XXX what to do if translation is not a Var??)
+ */
+
for (cnt = 0; cnt < nappinfos; cnt++)
{
if (var->varno == appinfos[cnt]->parent_relid)
@@ -255,6 +263,8 @@ adjust_appendrel_attrs_mutator(Node *node,
if (newnode == NULL)
elog(ERROR, "attribute %d of relation \"%s\" does not exist",
var->varattno, get_rel_name(appinfo->parent_reloid));
+ if (IsA(newnode, Var))
+ ((Var *) newnode)->varnullingrels = var->varnullingrels;
return newnode;
}
else if (var->varattno == 0)
@@ -348,6 +358,8 @@ adjust_appendrel_attrs_mutator(Node *node,
var = copyObject(ridinfo->rowidvar);
/* ... but use the correct relid */
var->varno = leaf_relid;
+ /* identity vars shouldn't have nulling rels */
+ Assert(var->varnullingrels == NULL);
/* varnosyn in the RowIdentityVarInfo is probably wrong */
var->varnosyn = 0;
var->varattnosyn = 0;
@@ -392,8 +404,11 @@ adjust_appendrel_attrs_mutator(Node *node,
(void *) context);
/* now fix PlaceHolderVar's relid sets */
if (phv->phlevelsup == 0)
- phv->phrels = adjust_child_relids(phv->phrels, context->nappinfos,
- context->appinfos);
+ {
+ phv->phrels = adjust_child_relids(phv->phrels,
+ nappinfos, appinfos);
+ /* as above, we needn't touch phnullingrels */
+ }
return (Node *) phv;
}
/* Shouldn't need to handle planner auxiliary nodes here */
@@ -688,7 +703,11 @@ get_translated_update_targetlist(PlannerInfo *root, Index relid,
/*
* find_appinfos_by_relids
- * Find AppendRelInfo structures for all relations specified by relids.
+ * Find AppendRelInfo structures for base relations listed in relids.
+ *
+ * The relids argument is typically a join relation's relids, which can
+ * include outer-join RT indexes in addition to baserels. We silently
+ * ignore the outer joins.
*
* The AppendRelInfos are returned in an array, which can be pfree'd by the
* caller. *nappinfos is set to the number of entries in the array.
@@ -700,8 +719,9 @@ find_appinfos_by_relids(PlannerInfo *root, Relids relids, int *nappinfos)
int cnt = 0;
int i;
- *nappinfos = bms_num_members(relids);
- appinfos = (AppendRelInfo **) palloc(sizeof(AppendRelInfo *) * *nappinfos);
+ /* Allocate an array that's certainly big enough */
+ appinfos = (AppendRelInfo **)
+ palloc(sizeof(AppendRelInfo *) * bms_num_members(relids));
i = -1;
while ((i = bms_next_member(relids, i)) >= 0)
@@ -709,10 +729,17 @@ find_appinfos_by_relids(PlannerInfo *root, Relids relids, int *nappinfos)
AppendRelInfo *appinfo = root->append_rel_array[i];
if (!appinfo)
+ {
+ /* Probably i is an OJ index, but let's check */
+ if (find_base_rel_ignore_join(root, i) == NULL)
+ continue;
+ /* It's a base rel, but we lack an append_rel_array entry */
elog(ERROR, "child rel %d not found in append_rel_array", i);
+ }
appinfos[cnt++] = appinfo;
}
+ *nappinfos = cnt;
return appinfos;
}
@@ -754,6 +781,7 @@ add_row_identity_var(PlannerInfo *root, Var *orig_var,
Assert(IsA(orig_var, Var));
Assert(orig_var->varno == rtindex);
Assert(orig_var->varlevelsup == 0);
+ Assert(orig_var->varnullingrels == NULL);
/*
* If we're doing non-inherited UPDATE/DELETE/MERGE, there's little need
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index bffc8112aa..5976862c10 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -2004,14 +2004,16 @@ is_pseudo_constant_clause_relids(Node *clause, Relids relids)
* NumRelids
* (formerly clause_relids)
*
- * Returns the number of different relations referenced in 'clause'.
+ * Returns the number of different base relations referenced in 'clause'.
*/
int
NumRelids(PlannerInfo *root, Node *clause)
{
+ int result;
Relids varnos = pull_varnos(root, clause);
- int result = bms_num_members(varnos);
+ varnos = bms_del_members(varnos, root->outer_join_rels);
+ result = bms_num_members(varnos);
bms_free(varnos);
return result;
}
diff --git a/src/backend/optimizer/util/joininfo.c b/src/backend/optimizer/util/joininfo.c
index d4cffdb198..afd243f5d8 100644
--- a/src/backend/optimizer/util/joininfo.c
+++ b/src/backend/optimizer/util/joininfo.c
@@ -88,8 +88,8 @@ have_relevant_joinclause(PlannerInfo *root,
* not depend on context).
*
* 'restrictinfo' describes the join clause
- * 'join_relids' is the list of relations participating in the join clause
- * (there must be more than one)
+ * 'join_relids' is the set of relations participating in the join clause
+ * (some of these could be outer joins)
*/
void
add_join_clause_to_rels(PlannerInfo *root,
@@ -101,8 +101,11 @@ add_join_clause_to_rels(PlannerInfo *root,
cur_relid = -1;
while ((cur_relid = bms_next_member(join_relids, cur_relid)) >= 0)
{
- RelOptInfo *rel = find_base_rel(root, cur_relid);
+ RelOptInfo *rel = find_base_rel_ignore_join(root, cur_relid);
+ /* We only need to add the clause to baserels */
+ if (rel == NULL)
+ continue;
rel->joininfo = lappend(rel->joininfo, restrictinfo);
}
}
@@ -115,8 +118,8 @@ add_join_clause_to_rels(PlannerInfo *root,
* discover that a relation need not be joined at all.
*
* 'restrictinfo' describes the join clause
- * 'join_relids' is the list of relations participating in the join clause
- * (there must be more than one)
+ * 'join_relids' is the set of relations participating in the join clause
+ * (some of these could be outer joins)
*/
void
remove_join_clause_from_rels(PlannerInfo *root,
@@ -128,7 +131,11 @@ remove_join_clause_from_rels(PlannerInfo *root,
cur_relid = -1;
while ((cur_relid = bms_next_member(join_relids, cur_relid)) >= 0)
{
- RelOptInfo *rel = find_base_rel(root, cur_relid);
+ RelOptInfo *rel = find_base_rel_ignore_join(root, cur_relid);
+
+ /* We would only have added the clause to baserels */
+ if (rel == NULL)
+ continue;
/*
* Remove the restrictinfo from the list. Pointer comparison is
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index b1363df065..e96ef176ad 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -338,6 +338,10 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
sjinfo.syn_lefthand = sjinfo.min_lefthand;
sjinfo.syn_righthand = sjinfo.min_righthand;
sjinfo.jointype = JOIN_INNER;
+ sjinfo.ojrelid = 0;
+ sjinfo.commute_above_l = NULL;
+ sjinfo.commute_above_r = NULL;
+ sjinfo.commute_below = NULL;
/* we don't bother trying to make the remaining fields valid */
sjinfo.lhs_strict = false;
sjinfo.delay_upper_joins = false;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 55deee555a..70a343b108 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -1307,7 +1307,7 @@ create_append_path(PlannerInfo *root,
* Apply query-wide LIMIT if known and path is for sole base relation.
* (Handling this at this low level is a bit klugy.)
*/
- if (root != NULL && bms_equal(rel->relids, root->all_baserels))
+ if (root != NULL && bms_equal(rel->relids, root->all_query_rels))
pathnode->limit_tuples = root->limit_tuples;
else
pathnode->limit_tuples = -1.0;
@@ -1436,7 +1436,7 @@ create_merge_append_path(PlannerInfo *root,
* Apply query-wide LIMIT if known and path is for sole base relation.
* (Handling this at this low level is a bit klugy.)
*/
- if (bms_equal(rel->relids, root->all_baserels))
+ if (bms_equal(rel->relids, root->all_query_rels))
pathnode->limit_tuples = root->limit_tuples;
else
pathnode->limit_tuples = -1.0;
diff --git a/src/backend/optimizer/util/placeholder.c b/src/backend/optimizer/util/placeholder.c
index c55027377f..b9cc983df7 100644
--- a/src/backend/optimizer/util/placeholder.c
+++ b/src/backend/optimizer/util/placeholder.c
@@ -23,17 +23,32 @@
#include "optimizer/planmain.h"
#include "utils/lsyscache.h"
+
+typedef struct contain_placeholder_references_context
+{
+ int relid;
+ int sublevels_up;
+} contain_placeholder_references_context;
+
/* Local functions */
static void find_placeholders_recurse(PlannerInfo *root, Node *jtnode);
static void find_placeholders_in_expr(PlannerInfo *root, Node *expr);
+static bool contain_placeholder_references_walker(Node *node,
+ contain_placeholder_references_context *context);
/*
* make_placeholder_expr
* Make a PlaceHolderVar for the given expression.
*
- * phrels is the syntactic location (as a set of baserels) to attribute
+ * phrels is the syntactic location (as a set of relids) to attribute
* to the expression.
+ *
+ * The caller is responsible for adjusting phlevelsup and phnullingrels
+ * as needed. Because we do not know here which query level the PHV
+ * will be associated with, it's important that this function touches
+ * only root->glob; messing with other parts of PlannerInfo would be
+ * likely to do the wrong thing.
*/
PlaceHolderVar *
make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
@@ -42,8 +57,9 @@ make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
phv->phexpr = expr;
phv->phrels = phrels;
+ phv->phnullingrels = NULL; /* caller may change this later */
phv->phid = ++(root->glob->lastPHId);
- phv->phlevelsup = 0;
+ phv->phlevelsup = 0; /* caller may change this later */
return phv;
}
@@ -92,6 +108,15 @@ find_placeholder_info(PlannerInfo *root, PlaceHolderVar *phv)
phinfo->phid = phv->phid;
phinfo->ph_var = copyObject(phv);
+ /*
+ * By convention, phinfo->ph_var->phnullingrels is always empty, since the
+ * PlaceHolderInfo represents the initially-calculated state of the
+ * PlaceHolderVar. PlaceHolderVars appearing in the query tree might have
+ * varying values of phnullingrels, reflecting outer joins applied above
+ * the calculation level.
+ */
+ phinfo->ph_var->phnullingrels = NULL;
+
/*
* Any referenced rels that are outside the PHV's syntactic scope are
* LATERAL references, which should be included in ph_lateral but not in
@@ -339,6 +364,8 @@ update_placeholder_eval_levels(PlannerInfo *root, SpecialJoinInfo *new_sjinfo)
sjinfo->min_lefthand);
eval_at = bms_add_members(eval_at,
sjinfo->min_righthand);
+ if (sjinfo->ojrelid)
+ eval_at = bms_add_member(eval_at, sjinfo->ojrelid);
/* we'll need another iteration */
found_some = true;
}
@@ -413,6 +440,14 @@ add_placeholders_to_base_rels(PlannerInfo *root)
{
RelOptInfo *rel = find_base_rel(root, varno);
+ /*
+ * As in add_vars_to_targetlist(), a value computed at scan level
+ * has not yet been nulled by any outer join, so its phnullingrels
+ * should be empty.
+ */
+ Assert(phinfo->ph_var->phnullingrels == NULL);
+
+ /* Copying the PHV might be unnecessary here, but be safe */
rel->reltarget->exprs = lappend(rel->reltarget->exprs,
copyObject(phinfo->ph_var));
/* reltarget's cost and width fields will be updated later */
@@ -435,7 +470,8 @@ add_placeholders_to_base_rels(PlannerInfo *root)
*/
void
add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *outer_rel, RelOptInfo *inner_rel)
+ RelOptInfo *outer_rel, RelOptInfo *inner_rel,
+ SpecialJoinInfo *sjinfo)
{
Relids relids = joinrel->relids;
ListCell *lc;
@@ -466,9 +502,17 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
if (!bms_is_subset(phinfo->ph_eval_at, outer_rel->relids) &&
!bms_is_subset(phinfo->ph_eval_at, inner_rel->relids))
{
- PlaceHolderVar *phv = phinfo->ph_var;
+ /* Copying might be unnecessary here, but be safe */
+ PlaceHolderVar *phv = copyObject(phinfo->ph_var);
QualCost cost;
+ /*
+ * It'll start out not nulled by anything. Joins above
+ * this one might add to its phnullingrels later, in much
+ * the same way as for Vars.
+ */
+ Assert(phv->phnullingrels == NULL);
+
joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
phv);
cost_qual_eval_node(&cost, (Node *) phv->phexpr, root);
@@ -499,3 +543,74 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
}
}
}
+
+/*
+ * contain_placeholder_references_to
+ * Detect whether any PlaceHolderVars in the given clause contain
+ * references to the given relid (typically an OJ relid).
+ *
+ * "Contain" means that there's a use of the relid inside the PHV's
+ * contained expression, so that changing the nullability status of
+ * the rel might change what the PHV computes.
+ *
+ * The code here to cope with upper-level PHVs is likely dead, but keep it
+ * anyway just in case.
+ */
+bool
+contain_placeholder_references_to(PlannerInfo *root, Node *clause,
+ int relid)
+{
+ contain_placeholder_references_context context;
+
+ /* We can answer quickly in the common case that there's no PHVs at all */
+ if (root->glob->lastPHId == 0)
+ return false;
+ /* Else run the recursive search */
+ context.relid = relid;
+ context.sublevels_up = 0;
+ return contain_placeholder_references_walker(clause, &context);
+}
+
+static bool
+contain_placeholder_references_walker(Node *node,
+ contain_placeholder_references_context *context)
+{
+ if (node == NULL)
+ return false;
+ if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ /* We should just look through PHVs of other query levels */
+ if (phv->phlevelsup == context->sublevels_up)
+ {
+ /* If phrels matches, we found what we came for */
+ if (bms_is_member(context->relid, phv->phrels))
+ return true;
+
+ /*
+ * We should not examine phnullingrels: what we are looking for is
+ * references in the contained expression, not OJs that might null
+ * the result afterwards. Also, we don't need to recurse into the
+ * contained expression, because phrels should adequately
+ * summarize what's in there. So we're done here.
+ */
+ return false;
+ }
+ }
+ else if (IsA(node, Query))
+ {
+ /* Recurse into RTE subquery or not-yet-planned sublink subquery */
+ bool result;
+
+ context->sublevels_up++;
+ result = query_tree_walker((Query *) node,
+ contain_placeholder_references_walker,
+ context,
+ 0);
+ context->sublevels_up--;
+ return result;
+ }
+ return expression_tree_walker(node, contain_placeholder_references_walker,
+ context);
+}
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 7085cf3c41..acee3e2559 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -28,6 +28,7 @@
#include "optimizer/plancat.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+#include "rewrite/rewriteManip.h"
#include "parser/parse_relation.h"
#include "utils/hsearch.h"
#include "utils/lsyscache.h"
@@ -40,7 +41,9 @@ typedef struct JoinHashEntry
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *input_rel);
+ RelOptInfo *input_rel,
+ SpecialJoinInfo *sjinfo,
+ bool can_null);
static List *build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outer_rel,
@@ -48,8 +51,10 @@ static List *build_joinrel_restrictlist(PlannerInfo *root,
static void build_joinrel_joinlist(RelOptInfo *joinrel,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel);
-static List *subbuild_joinrel_restrictlist(RelOptInfo *joinrel,
- List *joininfo_list,
+static List *subbuild_joinrel_restrictlist(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *input_rel,
+ Relids both_input_relids,
List *new_restrictlist);
static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
List *joininfo_list,
@@ -57,10 +62,12 @@ static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
static void set_foreign_rel_properties(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel);
static void add_join_rel(PlannerInfo *root, RelOptInfo *joinrel);
-static void build_joinrel_partition_info(RelOptInfo *joinrel,
+static void build_joinrel_partition_info(PlannerInfo *root,
+ RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
- List *restrictlist, JoinType jointype);
-static bool have_partkey_equi_join(RelOptInfo *joinrel,
+ SpecialJoinInfo *sjinfo,
+ List *restrictlist);
+static bool have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *rel1, RelOptInfo *rel2,
JoinType jointype, List *restrictlist);
static int match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel,
@@ -373,7 +380,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
/*
* find_base_rel
- * Find a base or other relation entry, which must already exist.
+ * Find a base or otherrel relation entry, which must already exist.
*/
RelOptInfo *
find_base_rel(PlannerInfo *root, int relid)
@@ -394,6 +401,44 @@ find_base_rel(PlannerInfo *root, int relid)
return NULL; /* keep compiler quiet */
}
+/*
+ * find_base_rel_ignore_join
+ * Find a base or otherrel relation entry, which must already exist.
+ *
+ * Unlike find_base_rel, if relid references an outer join then this
+ * will return NULL rather than raising an error. This is convenient
+ * for callers that must deal with relid sets including both base and
+ * outer joins.
+ */
+RelOptInfo *
+find_base_rel_ignore_join(PlannerInfo *root, int relid)
+{
+ Assert(relid > 0);
+
+ if (relid < root->simple_rel_array_size)
+ {
+ RelOptInfo *rel;
+ RangeTblEntry *rte;
+
+ rel = root->simple_rel_array[relid];
+ if (rel)
+ return rel;
+
+ /*
+ * We could just return NULL here, but for debugging purposes it seems
+ * best to actually verify that the relid is an outer join and not
+ * something weird.
+ */
+ rte = root->simple_rte_array[relid];
+ if (rte && rte->rtekind == RTE_JOIN && rte->jointype != JOIN_INNER)
+ return NULL;
+ }
+
+ elog(ERROR, "no relation entry for relid %d", relid);
+
+ return NULL; /* keep compiler quiet */
+}
+
/*
* build_join_rel_hash
* Construct the auxiliary hash table for join relations.
@@ -693,9 +738,11 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ build_joinrel_tlist(root, joinrel, outer_rel, sjinfo,
+ (sjinfo->jointype == JOIN_FULL));
+ build_joinrel_tlist(root, joinrel, inner_rel, sjinfo,
+ (sjinfo->jointype != JOIN_INNER));
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel, sjinfo);
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -727,8 +774,8 @@ build_join_rel(PlannerInfo *root,
joinrel->has_eclass_joins = has_relevant_eclass_joinclause(root, joinrel);
/* Store the partition information. */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- sjinfo->jointype);
+ build_joinrel_partition_info(root, joinrel, outer_rel, inner_rel, sjinfo,
+ restrictlist);
/*
* Set estimates of the joinrel's size.
@@ -784,16 +831,14 @@ build_join_rel(PlannerInfo *root,
* 'parent_joinrel' is the RelOptInfo representing the join between parent
* relations. Some of the members of new RelOptInfo are produced by
* translating corresponding members of this RelOptInfo
- * 'sjinfo': child-join context info
* 'restrictlist': list of RestrictInfo nodes that apply to this particular
* pair of joinable relations
- * 'jointype' is the join type (inner, left, full, etc)
+ * 'sjinfo': child join's join-type details
*/
RelOptInfo *
build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
RelOptInfo *inner_rel, RelOptInfo *parent_joinrel,
- List *restrictlist, SpecialJoinInfo *sjinfo,
- JoinType jointype)
+ List *restrictlist, SpecialJoinInfo *sjinfo)
{
RelOptInfo *joinrel = makeNode(RelOptInfo);
AppendRelInfo **appinfos;
@@ -807,6 +852,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->reloptkind = RELOPT_OTHER_JOINREL;
joinrel->relids = bms_union(outer_rel->relids, inner_rel->relids);
+ if (sjinfo->ojrelid != 0)
+ joinrel->relids = bms_add_member(joinrel->relids, sjinfo->ojrelid);
joinrel->rows = 0;
/* cheap startup cost is interesting iff not all tuples to be retrieved */
joinrel->consider_startup = (root->tuple_fraction > 0);
@@ -893,8 +940,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->has_eclass_joins = parent_joinrel->has_eclass_joins;
/* Is the join between partitions itself partitioned? */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- jointype);
+ build_joinrel_partition_info(root, joinrel, outer_rel, inner_rel, sjinfo,
+ restrictlist);
/* Child joinrel is parallel safe if parent is parallel safe. */
joinrel->consider_parallel = parent_joinrel->consider_parallel;
@@ -976,10 +1023,41 @@ min_join_parameterization(PlannerInfo *root,
*
* We also compute the expected width of the join's output, making use
* of data that was cached at the baserel level by set_rel_width().
+ *
+ * Pass can_null as true if the join is an outer join that can null Vars
+ * from this input relation. If so, we will (normally) add the join's relid
+ * to the nulling bitmaps of Vars and PHVs bubbled up from the input.
+ *
+ * When forming an outer join's target list, special handling is needed
+ * in case the outer join was commuted with another one per outer join
+ * identity 3 (see optimizer/README). We must take steps to ensure that
+ * the output Vars have the same nulling bitmaps that they would if the
+ * two joins had been done in syntactic order; else they won't match Vars
+ * appearing higher in the query tree. We need to do two things:
+ *
+ * First, sjinfo->commute_above_r is added to the nulling bitmaps of RHS Vars.
+ * This takes care of the case where we implement
+ * A leftjoin (B leftjoin C on (Pbc)) on (Pab)
+ * as
+ * (A leftjoin B on (Pab)) leftjoin C on (Pbc)
+ * The C columns emitted by the B/C join need to be shown as nulled by both
+ * the B/C and A/B joins, even though they've not traversed the A/B join.
+ * (If the joins haven't been commuted, we are adding the nullingrel bits
+ * prematurely; but that's okay because the C columns can't be referenced
+ * between here and the upper join.)
+ *
+ * Second, if a RHS Var has any of the relids in sjinfo->commute_above_l
+ * already set in its nulling bitmap, then we *don't* add sjinfo->ojrelid
+ * to its nulling bitmap (but we do still add commute_above_r). This takes
+ * care of the reverse transformation: if the original syntax was
+ * (A leftjoin B on (Pab)) leftjoin C on (Pbc)
+ * then the now-upper A/B join must not mark C columns as nulled by itself.
*/
static void
build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *input_rel)
+ RelOptInfo *input_rel,
+ SpecialJoinInfo *sjinfo,
+ bool can_null)
{
Relids relids = joinrel->relids;
ListCell *vars;
@@ -999,7 +1077,24 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
/* Is it still needed above this joinrel? */
if (bms_nonempty_difference(phinfo->ph_needed, relids))
{
- /* Yup, add it to the output */
+ /*
+ * Yup, add it to the output. If this join potentially nulls
+ * this input, we have to update the PHV's phnullingrels,
+ * which means making a copy.
+ */
+ if (can_null)
+ {
+ phv = copyObject(phv);
+ /* See comments above to understand this logic */
+ if (sjinfo->ojrelid != 0 &&
+ !bms_overlap(phv->phnullingrels, sjinfo->commute_above_l))
+ phv->phnullingrels = bms_add_member(phv->phnullingrels,
+ sjinfo->ojrelid);
+ if (sjinfo->commute_above_r)
+ phv->phnullingrels = bms_add_members(phv->phnullingrels,
+ sjinfo->commute_above_r);
+ }
+
joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
phv);
/* Bubbling up the precomputed result has cost zero */
@@ -1023,9 +1118,7 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
RowIdentityVarInfo *ridinfo = (RowIdentityVarInfo *)
list_nth(root->row_identity_vars, var->varattno - 1);
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- var);
- /* Vars have cost zero, so no need to adjust reltarget->cost */
+ /* Update reltarget width estimate from RowIdentityVarInfo */
joinrel->reltarget->width += ridinfo->rowidwidth;
}
else
@@ -1038,15 +1131,35 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
/* Is it still needed above this joinrel? */
ndx = var->varattno - baserel->min_attr;
- if (bms_nonempty_difference(baserel->attr_needed[ndx], relids))
- {
- /* Yup, add it to the output */
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- var);
- /* Vars have cost zero, so no need to adjust reltarget->cost */
- joinrel->reltarget->width += baserel->attr_widths[ndx];
- }
+ if (!bms_nonempty_difference(baserel->attr_needed[ndx], relids))
+ continue; /* nope, skip it */
+
+ /* Update reltarget width estimate from baserel's attr_widths */
+ joinrel->reltarget->width += baserel->attr_widths[ndx];
+ }
+
+ /*
+ * Add the Var to the output. If this join potentially nulls this
+ * input, we have to update the Var's varnullingrels, which means
+ * making a copy.
+ */
+ if (can_null)
+ {
+ var = copyObject(var);
+ /* See comments above to understand this logic */
+ if (sjinfo->ojrelid != 0 &&
+ !bms_overlap(var->varnullingrels, sjinfo->commute_above_l))
+ var->varnullingrels = bms_add_member(var->varnullingrels,
+ sjinfo->ojrelid);
+ if (sjinfo->commute_above_r)
+ var->varnullingrels = bms_add_members(var->varnullingrels,
+ sjinfo->commute_above_r);
}
+
+ joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
+ var);
+
+ /* Vars have cost zero, so no need to adjust reltarget->cost */
}
}
@@ -1065,7 +1178,7 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
* is not handled in the sub-relations, so it depends on which
* sub-relations are considered.
*
- * If a join clause from an input relation refers to base rels still not
+ * If a join clause from an input relation refers to base+OJ rels still not
* present in the joinrel, then it is still a join clause for the joinrel;
* we put it into the joininfo list for the joinrel. Otherwise,
* the clause is now a restrict clause for the joined relation, and we
@@ -1099,14 +1212,19 @@ build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *inner_rel)
{
List *result;
+ Relids both_input_relids;
+
+ both_input_relids = bms_union(outer_rel->relids, inner_rel->relids);
/*
* Collect all the clauses that syntactically belong at this level,
* eliminating any duplicates (important since we will see many of the
* same clauses arriving from both input relations).
*/
- result = subbuild_joinrel_restrictlist(joinrel, outer_rel->joininfo, NIL);
- result = subbuild_joinrel_restrictlist(joinrel, inner_rel->joininfo, result);
+ result = subbuild_joinrel_restrictlist(root, joinrel, outer_rel,
+ both_input_relids, NIL);
+ result = subbuild_joinrel_restrictlist(root, joinrel, inner_rel,
+ both_input_relids, result);
/*
* Add on any clauses derived from EquivalenceClasses. These cannot be
@@ -1141,24 +1259,63 @@ build_joinrel_joinlist(RelOptInfo *joinrel,
}
static List *
-subbuild_joinrel_restrictlist(RelOptInfo *joinrel,
- List *joininfo_list,
+subbuild_joinrel_restrictlist(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *input_rel,
+ Relids both_input_relids,
List *new_restrictlist)
{
ListCell *l;
- foreach(l, joininfo_list)
+ foreach(l, input_rel->joininfo)
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(l);
if (bms_is_subset(rinfo->required_relids, joinrel->relids))
{
/*
- * This clause becomes a restriction clause for the joinrel, since
- * it refers to no outside rels. Add it to the list, being
- * careful to eliminate duplicates. (Since RestrictInfo nodes in
- * different joinlists will have been multiply-linked rather than
- * copied, pointer equality should be a sufficient test.)
+ * This clause should become a restriction clause for the joinrel,
+ * since it refers to no outside rels. However, if it's a clone
+ * clause then it might be too late to evaluate it, so we have to
+ * check. (If it is too late, just ignore the clause, taking it
+ * on faith that another clone was or will be selected.) Clone
+ * clauses should always be outer-join clauses, so we compare
+ * against both_input_relids.
+ */
+ if (rinfo->has_clone || rinfo->is_clone)
+ {
+ Assert(!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids));
+ if (!bms_is_subset(rinfo->required_relids, both_input_relids))
+ continue;
+ if (!clause_is_computable_at(root, rinfo->clause_relids,
+ both_input_relids))
+ continue;
+ }
+ else
+ {
+ /*
+ * For non-clone clauses, we just Assert it's OK. These might
+ * be either join or filter clauses.
+ */
+#ifdef USE_ASSERT_CHECKING
+ if (RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ Assert(clause_is_computable_at(root, rinfo->clause_relids,
+ joinrel->relids));
+ else
+ {
+ Assert(bms_is_subset(rinfo->required_relids,
+ both_input_relids));
+ Assert(clause_is_computable_at(root, rinfo->clause_relids,
+ both_input_relids));
+ }
+#endif
+ }
+
+ /*
+ * OK, so add it to the list, being careful to eliminate
+ * duplicates. (Since RestrictInfo nodes in different joinlists
+ * will have been multiply-linked rather than copied, pointer
+ * equality should be a sufficient test.)
*/
new_restrictlist = list_append_unique_ptr(new_restrictlist, rinfo);
}
@@ -1665,9 +1822,10 @@ find_param_path_info(RelOptInfo *rel, Relids required_outer)
* partitioned join relation.
*/
static void
-build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
- RelOptInfo *inner_rel, List *restrictlist,
- JoinType jointype)
+build_joinrel_partition_info(PlannerInfo *root,
+ RelOptInfo *joinrel, RelOptInfo *outer_rel,
+ RelOptInfo *inner_rel, SpecialJoinInfo *sjinfo,
+ List *restrictlist)
{
PartitionScheme part_scheme;
@@ -1693,8 +1851,8 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
!outer_rel->consider_partitionwise_join ||
!inner_rel->consider_partitionwise_join ||
outer_rel->part_scheme != inner_rel->part_scheme ||
- !have_partkey_equi_join(joinrel, outer_rel, inner_rel,
- jointype, restrictlist))
+ !have_partkey_equi_join(root, joinrel, outer_rel, inner_rel,
+ sjinfo->jointype, restrictlist))
{
Assert(!IS_PARTITIONED_REL(joinrel));
return;
@@ -1718,7 +1876,8 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* child-join relations of the join relation in try_partitionwise_join().
*/
joinrel->part_scheme = part_scheme;
- set_joinrel_partition_key_exprs(joinrel, outer_rel, inner_rel, jointype);
+ set_joinrel_partition_key_exprs(joinrel, outer_rel, inner_rel,
+ sjinfo->jointype);
/*
* Set the consider_partitionwise_join flag.
@@ -1736,7 +1895,7 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* partition keys.
*/
static bool
-have_partkey_equi_join(RelOptInfo *joinrel,
+have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *rel1, RelOptInfo *rel2,
JoinType jointype, List *restrictlist)
{
@@ -1801,6 +1960,24 @@ have_partkey_equi_join(RelOptInfo *joinrel,
*/
strict_op = op_strict(opexpr->opno);
+ /*
+ * Vars appearing in the relation's partition keys will not have any
+ * varnullingrels, but those in expr1 and expr2 will if we're above
+ * outer joins that could null the respective rels. It's okay to
+ * match anyway, if the join operator is strict.
+ */
+ if (strict_op)
+ {
+ if (bms_overlap(rel1->relids, root->outer_join_rels))
+ expr1 = (Expr *) remove_nulling_relids((Node *) expr1,
+ root->outer_join_rels,
+ NULL);
+ if (bms_overlap(rel2->relids, root->outer_join_rels))
+ expr2 = (Expr *) remove_nulling_relids((Node *) expr2,
+ root->outer_join_rels,
+ NULL);
+ }
+
/*
* Only clauses referencing the partition keys are useful for
* partitionwise join.
@@ -2013,7 +2190,12 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
* partitionwise nesting of any outer join.) We assume no
* type coercions are needed to make the coalesce expressions,
* since columns of different types won't have gotten
- * classified as the same PartitionScheme.
+ * classified as the same PartitionScheme. Note that we
+ * intentionally leave out the varnullingrels decoration that
+ * would ordinarily appear on the Vars inside these
+ * CoalesceExprs, because have_partkey_equi_join will strip
+ * varnullingrels from the expressions it will compare to the
+ * partexprs.
*/
foreach(lc, list_concat_copy(outer_expr, outer_null_expr))
{
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index ef8df3d098..327c3ba563 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -53,6 +53,10 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
* required_relids can be NULL, in which case it defaults to the actual clause
* contents (i.e., clause_relids).
*
+ * Note that there aren't options to set the has_clone and is_clone flags:
+ * we always initialize those to false. There's just one place that wants
+ * something different, so making all callers pass them seems inconvenient.
+ *
* We initialize fields that depend only on the given subexpression, leaving
* others that depend on context (or may never be needed at all) to be filled
* later.
@@ -116,12 +120,15 @@ make_restrictinfo_internal(PlannerInfo *root,
Relids nullable_relids)
{
RestrictInfo *restrictinfo = makeNode(RestrictInfo);
+ Relids baserels;
restrictinfo->clause = clause;
restrictinfo->orclause = orclause;
restrictinfo->is_pushed_down = is_pushed_down;
restrictinfo->outerjoin_delayed = outerjoin_delayed;
restrictinfo->pseudoconstant = pseudoconstant;
+ restrictinfo->has_clone = false; /* may get set by caller */
+ restrictinfo->is_clone = false; /* may get set by caller */
restrictinfo->can_join = false; /* may get set below */
restrictinfo->security_level = security_level;
restrictinfo->outer_relids = outer_relids;
@@ -187,6 +194,20 @@ make_restrictinfo_internal(PlannerInfo *root,
else
restrictinfo->required_relids = restrictinfo->clause_relids;
+ /*
+ * Count the number of base rels appearing in clause_relids. To do this,
+ * we just delete rels mentioned in root->outer_join_rels and count the
+ * survivors. Because we are called during deconstruct_jointree which is
+ * the same tree walk that populates outer_join_rels, this is a little bit
+ * unsafe-looking; but it should be fine because the recursion in
+ * deconstruct_jointree should already have visited any outer join that
+ * could be mentioned in this clause.
+ */
+ baserels = bms_difference(restrictinfo->clause_relids,
+ root->outer_join_rels);
+ restrictinfo->num_base_rels = bms_num_members(baserels);
+ bms_free(baserels);
+
/*
* Fill in all the cacheable fields with "not yet set" markers. None of
* these will be computed until/unless needed. Note in particular that we
@@ -497,6 +518,58 @@ extract_actual_join_clauses(List *restrictinfo_list,
}
}
+/*
+ * clause_is_computable_at
+ * Test whether a clause is computable at a given evaluation level.
+ *
+ * There are two conditions for whether an expression can actually be
+ * evaluated at a given join level: the evaluation context must include
+ * all the relids (both base and OJ) used by the expression, and we must
+ * not have already evaluated any outer joins that null Vars/PHVs of the
+ * expression and are not listed in their nullingrels.
+ *
+ * This function checks the second condition; we assume the caller already
+ * saw to the first one.
+ *
+ * For speed reasons, we don't individually examine each Var/PHV of the
+ * expression, but just look at the overall clause_relids (the union of the
+ * varnos and varnullingrels). This could give a misleading answer if the
+ * Vars of a given varno don't all have the same varnullingrels; but that
+ * really shouldn't happen within a single scalar expression or RestrictInfo
+ * clause. Despite that, this is still annoyingly expensive :-(
+ */
+bool
+clause_is_computable_at(PlannerInfo *root,
+ Relids clause_relids,
+ Relids eval_relids)
+{
+ ListCell *lc;
+
+ /* Nothing to do if no outer joins have been performed yet. */
+ if (!bms_overlap(eval_relids, root->outer_join_rels))
+ return true;
+
+ foreach(lc, root->join_info_list)
+ {
+ SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+
+ /* Ignore outer joins that are not yet performed. */
+ if (!bms_is_member(sjinfo->ojrelid, eval_relids))
+ continue;
+
+ /* OK if clause lists it (we assume all Vars in it agree). */
+ if (bms_is_member(sjinfo->ojrelid, clause_relids))
+ continue;
+
+ /* Else, trouble if clause mentions any nullable Vars. */
+ if (bms_overlap(clause_relids, sjinfo->min_righthand) ||
+ (sjinfo->jointype == JOIN_FULL &&
+ bms_overlap(clause_relids, sjinfo->min_lefthand)))
+ return false; /* doesn't work */
+ }
+
+ return true; /* OK */
+}
/*
* join_clause_is_movable_to
@@ -522,6 +595,12 @@ extract_actual_join_clauses(List *restrictinfo_list,
* Also, the join clause must not use any relations that have LATERAL
* references to the target relation, since we could not put such rels on
* the outer side of a nestloop with the target relation.
+ *
+ * Also, we reject is_clone versions of outer-join clauses. This has the
+ * effect of preventing us from generating variant parameterized paths
+ * that differ only in which outer joins null the parameterization rel(s).
+ * Generating one path from the minimally-parameterized has_clone version
+ * is sufficient.
*/
bool
join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
@@ -542,6 +621,10 @@ join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
if (bms_overlap(baserel->lateral_referencers, rinfo->clause_relids))
return false;
+ /* Ignore clones, too */
+ if (rinfo->is_clone)
+ return false;
+
return true;
}
@@ -587,6 +670,9 @@ join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
* moved for some valid set of outer rels, so we don't have the benefit of
* relying on prior checks for lateral-reference validity.
*
+ * Likewise, we don't check is_clone here: rejecting the inappropriate
+ * variants of a cloned clause must be handled upstream.
+ *
* Note: if this returns true, it means that the clause could be moved to
* this join relation, but that doesn't mean that this is the lowest join
* it could be moved to. Caller may need to make additional calls to verify
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
index 7db86c39ef..8d8c9136f8 100644
--- a/src/backend/optimizer/util/var.c
+++ b/src/backend/optimizer/util/var.c
@@ -88,6 +88,9 @@ static Relids alias_relid_set(Query *query, Relids relids);
* Create a set of all the distinct varnos present in a parsetree.
* Only varnos that reference level-zero rtable entries are considered.
*
+ * The result includes outer-join relids mentioned in Var.varnullingrels and
+ * PlaceHolderVar.phnullingrels fields in the parsetree.
+ *
* "root" can be passed as NULL if it is not necessary to process
* PlaceHolderVars.
*
@@ -153,7 +156,11 @@ pull_varnos_walker(Node *node, pull_varnos_context *context)
Var *var = (Var *) node;
if (var->varlevelsup == context->sublevels_up)
+ {
context->varnos = bms_add_member(context->varnos, var->varno);
+ context->varnos = bms_add_members(context->varnos,
+ var->varnullingrels);
+ }
return false;
}
if (IsA(node, CurrentOfExpr))
@@ -244,6 +251,14 @@ pull_varnos_walker(Node *node, pull_varnos_context *context)
context->varnos = bms_join(context->varnos,
newevalat);
}
+
+ /*
+ * In all three cases, include phnullingrels in the result. We
+ * don't worry about possibly needing to translate it, because
+ * appendrels only translate varnos of baserels, not outer joins.
+ */
+ context->varnos = bms_add_members(context->varnos,
+ phv->phnullingrels);
return false; /* don't recurse into expression */
}
}
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index 48858a871a..32bd70faad 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -2205,7 +2205,7 @@ rowcomparesel(PlannerInfo *root,
else
{
/*
- * Otherwise, it's a join if there's more than one relation used.
+ * Otherwise, it's a join if there's more than one base relation used.
*/
is_join_clause = (NumRelids(root, (Node *) opargs) > 1);
}
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 287bd554f6..8449dd7202 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -254,6 +254,20 @@ struct PlannerInfo
*/
Relids all_baserels;
+ /*
+ * outer_join_rels is a Relids set of all outer-join relids in the query.
+ * This is computed in deconstruct_jointree.
+ */
+ Relids outer_join_rels;
+
+ /*
+ * all_query_rels is a Relids set of all base relids and outer join relids
+ * (but not "other" relids) in the query. This is the Relids identifier
+ * of the final join we need to form. This is computed in
+ * deconstruct_jointree.
+ */
+ Relids all_query_rels;
+
/*
* nullable_baserels is a Relids set of base relids that are nullable by
* some outer join in the jointree; these are rels that are potentially
@@ -562,9 +576,10 @@ typedef struct PartitionSchemeData *PartitionScheme;
* or the output of a sub-SELECT or function that appears in the range table.
* In either case it is uniquely identified by an RT index. A "joinrel"
* is the joining of two or more base rels. A joinrel is identified by
- * the set of RT indexes for its component baserels. We create RelOptInfo
- * nodes for each baserel and joinrel, and store them in the PlannerInfo's
- * simple_rel_array and join_rel_list respectively.
+ * the set of RT indexes for its component baserels, along with RT indexes
+ * for any outer joins it has computed. We create RelOptInfo nodes for each
+ * baserel and joinrel, and store them in the PlannerInfo's simple_rel_array
+ * and join_rel_list respectively.
*
* Note that there is only one joinrel for any given set of component
* baserels, no matter what order we assemble them in; so an unordered
@@ -603,8 +618,10 @@ typedef struct PartitionSchemeData *PartitionScheme;
* Parts of this data structure are specific to various scan and join
* mechanisms. It didn't seem worth creating new node types for them.
*
- * relids - Set of base-relation identifiers; it is a base relation
- * if there is just one, a join relation if more than one
+ * relids - Set of relation identifiers (RT indexes). This is a base
+ * relation if there is just one, a join relation if more;
+ * in the join case, RT indexes of any outer joins formed
+ * at or below this join are included along with baserels
* rows - estimated number of tuples in the relation after restriction
* clauses have been applied (ie, output rows of a plan for it)
* consider_startup - true if there is any value in keeping plain paths for
@@ -816,7 +833,7 @@ typedef struct RelOptInfo
RelOptKind reloptkind;
/*
- * all relations included in this RelOptInfo; set of base relids
+ * all relations included in this RelOptInfo; set of base + OJ relids
* (rangetable indexes)
*/
Relids relids;
@@ -2286,17 +2303,17 @@ typedef struct LimitPath
* If a restriction clause references a single base relation, it will appear
* in the baserestrictinfo list of the RelOptInfo for that base rel.
*
- * If a restriction clause references more than one base rel, it will
+ * If a restriction clause references more than one base+OJ relation, it will
* appear in the joininfo list of every RelOptInfo that describes a strict
- * subset of the base rels mentioned in the clause. The joininfo lists are
+ * subset of the relations mentioned in the clause. The joininfo lists are
* used to drive join tree building by selecting plausible join candidates.
* The clause cannot actually be applied until we have built a join rel
- * containing all the base rels it references, however.
+ * containing all the relations it references, however.
*
- * When we construct a join rel that includes all the base rels referenced
+ * When we construct a join rel that includes all the relations referenced
* in a multi-relation restriction clause, we place that clause into the
* joinrestrictinfo lists of paths for the join rel, if neither left nor
- * right sub-path includes all base rels referenced in the clause. The clause
+ * right sub-path includes all relations referenced in the clause. The clause
* will be applied at that join level, and will not propagate any further up
* the join tree. (Note: the "predicate migration" code was once intended to
* push restriction clauses up and down the plan tree based on evaluation
@@ -2317,12 +2334,14 @@ typedef struct LimitPath
* or join to enforce that all members of each EquivalenceClass are in fact
* equal in all rows emitted by the scan or join.
*
- * When dealing with outer joins we have to be very careful about pushing qual
- * clauses up and down the tree. An outer join's own JOIN/ON conditions must
- * be evaluated exactly at that join node, unless they are "degenerate"
- * conditions that reference only Vars from the nullable side of the join.
- * Quals appearing in WHERE or in a JOIN above the outer join cannot be pushed
- * down below the outer join, if they reference any nullable Vars.
+ * The clause_relids field lists the base plus outer-join RT indexes that
+ * actually appear in the clause. required_relids lists the minimum set of
+ * relids needed to evaluate the clause; while this is often equal to
+ * clause_relids, it can be more. We will add relids to required_relids when
+ * we need to force an outer join ON clause to be evaluated exactly at the
+ * level of the outer join, which is true except when it is a "degenerate"
+ * condition that references only Vars from the nullable side of the join.
+ *
* RestrictInfo nodes contain a flag to indicate whether a qual has been
* pushed down to a lower level than its original syntactic placement in the
* join tree would suggest. If an outer join prevents us from pushing a qual
@@ -2407,6 +2426,12 @@ typedef struct LimitPath
* or merge or hash join clause, so it's of no interest to large parts of
* the planner.
*
+ * When we generate multiple versions of a clause so as to have versions
+ * that will work after commuting some left joins per outer join identity 3,
+ * we mark the one with the fewest nullingrels bits with has_clone = true,
+ * and the rest with is_clone = true. This allows proper filtering of
+ * these redundant clauses, so that we apply only one version of them.
+ *
* When join clauses are generated from EquivalenceClasses, there may be
* several equally valid ways to enforce join equivalence, of which we need
* apply only one. We mark clauses of this kind by setting parent_ec to
@@ -2441,16 +2466,23 @@ typedef struct RestrictInfo
/* see comment above */
bool pseudoconstant pg_node_attr(equal_ignore);
+ /* see comment above */
+ bool has_clone;
+ bool is_clone;
+
/* true if known to contain no leaked Vars */
bool leakproof pg_node_attr(equal_ignore);
- /* to indicate if clause contains any volatile functions. */
+ /* indicates if clause contains any volatile functions */
VolatileFunctionStatus has_volatile pg_node_attr(equal_ignore);
/* see comment above */
Index security_level;
- /* The set of relids (varnos) actually referenced in the clause: */
+ /* number of base rels in clause_relids */
+ int num_base_rels pg_node_attr(equal_ignore);
+
+ /* The relids (varnos+varnullingrels) actually referenced in the clause: */
Relids clause_relids pg_node_attr(equal_ignore);
/* The set of relids required to evaluate the clause: */
@@ -2654,20 +2686,49 @@ typedef struct PlaceHolderVar
* We make SpecialJoinInfos for FULL JOINs even though there is no flexibility
* of planning for them, because this simplifies make_join_rel()'s API.
*
- * min_lefthand and min_righthand are the sets of base relids that must be
- * available on each side when performing the special join. lhs_strict is
- * true if the special join's condition cannot succeed when the LHS variables
- * are all NULL (this means that an outer join can commute with upper-level
- * outer joins even if it appears in their RHS). We don't bother to set
- * lhs_strict for FULL JOINs, however.
- *
+ * min_lefthand and min_righthand are the sets of base+OJ relids that must be
+ * available on each side when performing the special join.
* It is not valid for either min_lefthand or min_righthand to be empty sets;
* if they were, this would break the logic that enforces join order.
*
- * syn_lefthand and syn_righthand are the sets of base relids that are
+ * syn_lefthand and syn_righthand are the sets of base+OJ relids that are
* syntactically below this special join. (These are needed to help compute
* min_lefthand and min_righthand for higher joins.)
*
+ * jointype is never JOIN_RIGHT; a RIGHT JOIN is handled by switching
+ * the inputs to make it a LEFT JOIN. So the allowed values of jointype
+ * in a join_info_list member are only LEFT, FULL, SEMI, or ANTI.
+ *
+ * ojrelid is the RT index of the join RTE representing this outer join,
+ * if there is one. It is zero when jointype is INNER or SEMI, and can be
+ * zero for jointype ANTI (if the join was transformed from a SEMI join).
+ * One use for this field is that when constructing the output targetlist of a
+ * join relation that implements this OJ, we add ojrelid to the varnullingrels
+ * and phnullingrels fields of nullable (RHS) output columns, so that the
+ * output Vars and PlaceHolderVars correctly reflect the nulling that has
+ * potentially happened to them.
+ *
+ * commute_above_l is filled with the relids of syntactically-higher outer
+ * joins that have been found to commute with this one per outer join identity
+ * 3 (see optimizer/README), when this join is in the LHS of the upper join
+ * (so, this is the lower join in the first form of the identity).
+ *
+ * commute_above_r is filled with the relids of syntactically-higher outer
+ * joins that have been found to commute with this one per outer join identity
+ * 3, when this join is in the RHS of the upper join (so, this is the lower
+ * join in the second form of the identity).
+ *
+ * commute_below is filled with the relids of syntactically-lower outer joins
+ * that have been found to commute with this one per outer join identity 3.
+ * (We need not record which side they are on, since that can be determined
+ * by seeing whether the lower join's relid appears in syn_lefthand or
+ * syn_righthand.)
+ *
+ * lhs_strict is true if the special join's condition cannot succeed when the
+ * LHS variables are all NULL (this means that an outer join can commute with
+ * upper-level outer joins even if it appears in their RHS). We don't bother
+ * to set lhs_strict for FULL JOINs, however.
+ *
* delay_upper_joins is set true if we detect a pushed-down clause that has
* to be evaluated after this join is formed (because it references the RHS).
* Any outer joins that have such a clause and this join in their RHS cannot
@@ -2682,10 +2743,6 @@ typedef struct PlaceHolderVar
* join planning; but it's helpful to have it available during planning of
* parameterized table scans, so we store it in the SpecialJoinInfo structs.)
*
- * jointype is never JOIN_RIGHT; a RIGHT JOIN is handled by switching
- * the inputs to make it a LEFT JOIN. So the allowed values of jointype
- * in a join_info_list member are only LEFT, FULL, SEMI, or ANTI.
- *
* For purposes of join selectivity estimation, we create transient
* SpecialJoinInfo structures for regular inner joins; so it is possible
* to have jointype == JOIN_INNER in such a structure, even though this is
@@ -2705,11 +2762,15 @@ struct SpecialJoinInfo
pg_node_attr(no_read)
NodeTag type;
- Relids min_lefthand; /* base relids in minimum LHS for join */
- Relids min_righthand; /* base relids in minimum RHS for join */
- Relids syn_lefthand; /* base relids syntactically within LHS */
- Relids syn_righthand; /* base relids syntactically within RHS */
+ Relids min_lefthand; /* base+OJ relids in minimum LHS for join */
+ Relids min_righthand; /* base+OJ relids in minimum RHS for join */
+ Relids syn_lefthand; /* base+OJ relids syntactically within LHS */
+ Relids syn_righthand; /* base+OJ relids syntactically within RHS */
JoinType jointype; /* always INNER, LEFT, FULL, SEMI, or ANTI */
+ Index ojrelid; /* outer join's RT index; 0 if none */
+ Relids commute_above_l; /* commuting OJs above this one, if LHS */
+ Relids commute_above_r; /* commuting OJs above this one, if RHS */
+ Relids commute_below; /* commuting OJs below this one */
bool lhs_strict; /* joinclause is strict for some LHS rel */
bool delay_upper_joins; /* can't commute with upper RHS */
/* Remaining fields are set only for JOIN_SEMI jointype: */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 050f00e79a..197234d44c 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -304,6 +304,7 @@ extern void expand_planner_arrays(PlannerInfo *root, int add_size);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
+extern RelOptInfo *find_base_rel_ignore_join(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
@@ -335,6 +336,6 @@ extern ParamPathInfo *find_param_path_info(RelOptInfo *rel,
extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
- SpecialJoinInfo *sjinfo, JoinType jointype);
+ SpecialJoinInfo *sjinfo);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/placeholder.h b/src/include/optimizer/placeholder.h
index 507dbc6175..3fe9b57415 100644
--- a/src/include/optimizer/placeholder.h
+++ b/src/include/optimizer/placeholder.h
@@ -27,6 +27,9 @@ extern void update_placeholder_eval_levels(PlannerInfo *root,
extern void fix_placeholder_input_needed_levels(PlannerInfo *root);
extern void add_placeholders_to_base_rels(PlannerInfo *root);
extern void add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *outer_rel, RelOptInfo *inner_rel);
+ RelOptInfo *outer_rel, RelOptInfo *inner_rel,
+ SpecialJoinInfo *sjinfo);
+extern bool contain_placeholder_references_to(PlannerInfo *root, Node *clause,
+ int relid);
#endif /* PLACEHOLDER_H */
diff --git a/src/include/optimizer/prep.h b/src/include/optimizer/prep.h
index 5b4f350b33..0847cfd5f4 100644
--- a/src/include/optimizer/prep.h
+++ b/src/include/optimizer/prep.h
@@ -29,7 +29,8 @@ extern void pull_up_subqueries(PlannerInfo *root);
extern void flatten_simple_union_all(PlannerInfo *root);
extern void reduce_outer_joins(PlannerInfo *root);
extern void remove_useless_result_rtes(PlannerInfo *root);
-extern Relids get_relids_in_jointree(Node *jtnode, bool include_joins);
+extern Relids get_relids_in_jointree(Node *jtnode, bool include_outer_joins,
+ bool include_inner_joins);
extern Relids get_relids_for_join(Query *query, int joinrelid);
/*
diff --git a/src/include/optimizer/restrictinfo.h b/src/include/optimizer/restrictinfo.h
index 6d30bd5e9d..17d3b4ab05 100644
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -41,6 +41,9 @@ extern void extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
List **joinquals,
List **otherquals);
+extern bool clause_is_computable_at(PlannerInfo *root,
+ Relids clause_relids,
+ Relids eval_relids);
extern bool join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel);
extern bool join_clause_is_movable_into(RestrictInfo *rinfo,
Relids currentrelids,
commit 89ed96bb69b832262cda39f6b846df34f220378d
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Fri Dec 23 10:23:05 2022 -0500
Detect duplicated pushed-down conditions using RestrictInfo ID numbers.
create_nestloop_path needs to identify which candidates for join
restriction quals were already enforced in the parameterized inner
path. Currently we do that by relying on join_clause_is_movable_into
to give consistent answers, but that is not working very well with
variant clauses generated to satisfy outer join identity 3. We may
have a clause that (correctly) shows the outer-side Var as nulled by
a previous outer join, which makes it dependent on the nestloop outer
side having included that join, so that it appears to not be pushable
into a parameterized path that uses the un-nulled version of that Var.
Nonetheless, the cloned clause *is* redundant and we don't want
to check it again.
This patch offers a somewhat brute-force solution, which is to assign
serial numbers to RestrictInfo nodes, then check for redundancy using
serial number match rather than trusting join_clause_is_movable_into.
The variant-clause problem can be solved by allowing clauses to share
a serial number when we know that they are equivalent. Both the
outer-join variant generator and equivclass.c need to be in on that
trick in order to handle all cases that were handled well before.
It'd be nicer if we could continue to trust join_clause_is_movable_into
for this, but on the other hand this mechanism does provide a much more
concrete, harder-to-break way of verifying that we already enforced
(some version of) a qual. Any failure mode would almost certainly
be in the safe direction of enforcing a qual redundantly, which is
not a claim that the existing method can make.
This patch results in two changes to the core regression test outputs:
* One query in join.sql changes to a different join order. Examining
the cost estimates that are normally not shown, the new order is
estimated as very slightly faster, so this seems like an improvement.
I'm not quite sure why the old code did not find this join order.
* Some of the queries in partition_join.sql revert equivalence-clause
ordering back to what it was before a5fc46414. That's probably a
consequence of investigating parameterized paths in a different order
than before. Anyway, it's visibly harmless.
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 055d70b8e3..d1e1965479 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -35,7 +35,8 @@
static EquivalenceMember *add_eq_member(EquivalenceClass *ec,
Expr *expr, Relids relids, Relids nullable_relids,
- bool is_child, Oid datatype);
+ EquivalenceMember *parent,
+ Oid datatype);
static bool is_exprlist_member(Expr *node, List *exprs);
static void generate_base_implied_equalities_const(PlannerInfo *root,
EquivalenceClass *ec);
@@ -400,7 +401,7 @@ process_equivalence(PlannerInfo *root,
{
/* Case 3: add item2 to ec1 */
em2 = add_eq_member(ec1, item2, item2_relids, item2_nullable_relids,
- false, item2_type);
+ NULL, item2_type);
ec1->ec_sources = lappend(ec1->ec_sources, restrictinfo);
ec1->ec_below_outer_join |= below_outer_join;
ec1->ec_min_security = Min(ec1->ec_min_security,
@@ -418,7 +419,7 @@ process_equivalence(PlannerInfo *root,
{
/* Case 3: add item1 to ec2 */
em1 = add_eq_member(ec2, item1, item1_relids, item1_nullable_relids,
- false, item1_type);
+ NULL, item1_type);
ec2->ec_sources = lappend(ec2->ec_sources, restrictinfo);
ec2->ec_below_outer_join |= below_outer_join;
ec2->ec_min_security = Min(ec2->ec_min_security,
@@ -452,9 +453,9 @@ process_equivalence(PlannerInfo *root,
ec->ec_max_security = restrictinfo->security_level;
ec->ec_merged = NULL;
em1 = add_eq_member(ec, item1, item1_relids, item1_nullable_relids,
- false, item1_type);
+ NULL, item1_type);
em2 = add_eq_member(ec, item2, item2_relids, item2_nullable_relids,
- false, item2_type);
+ NULL, item2_type);
root->eq_classes = lappend(root->eq_classes, ec);
@@ -544,7 +545,7 @@ canonicalize_ec_expression(Expr *expr, Oid req_type, Oid req_collation)
*/
static EquivalenceMember *
add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
- Relids nullable_relids, bool is_child, Oid datatype)
+ Relids nullable_relids, EquivalenceMember *parent, Oid datatype)
{
EquivalenceMember *em = makeNode(EquivalenceMember);
@@ -552,8 +553,9 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
em->em_relids = relids;
em->em_nullable_relids = nullable_relids;
em->em_is_const = false;
- em->em_is_child = is_child;
+ em->em_is_child = (parent != NULL);
em->em_datatype = datatype;
+ em->em_parent = parent;
if (bms_is_empty(relids))
{
@@ -565,12 +567,12 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
* get_eclass_for_sort_expr() has to work harder. We put the tests
* there not here to save cycles in the equivalence case.
*/
- Assert(!is_child);
+ Assert(!parent);
em->em_is_const = true;
ec->ec_has_const = true;
/* it can't affect ec_relids */
}
- else if (!is_child) /* child members don't add to ec_relids */
+ else if (!parent) /* child members don't add to ec_relids */
{
ec->ec_relids = bms_add_members(ec->ec_relids, relids);
}
@@ -723,7 +725,7 @@ get_eclass_for_sort_expr(PlannerInfo *root,
nullable_relids = bms_intersect(nullable_relids, expr_relids);
newem = add_eq_member(newec, copyObject(expr), expr_relids,
- nullable_relids, false, opcintype);
+ nullable_relids, NULL, opcintype);
/*
* add_eq_member doesn't check for volatile functions, set-returning
@@ -1821,6 +1823,7 @@ create_join_clause(PlannerInfo *root,
EquivalenceClass *parent_ec)
{
RestrictInfo *rinfo;
+ RestrictInfo *parent_rinfo = NULL;
ListCell *lc;
MemoryContext oldcontext;
@@ -1865,6 +1868,20 @@ create_join_clause(PlannerInfo *root,
*/
oldcontext = MemoryContextSwitchTo(root->planner_cxt);
+ /*
+ * If either EM is a child, recursively create the corresponding
+ * parent-to-parent clause, so that we can duplicate its rinfo_serial.
+ */
+ if (leftem->em_is_child || rightem->em_is_child)
+ {
+ EquivalenceMember *leftp = leftem->em_parent ? leftem->em_parent : leftem;
+ EquivalenceMember *rightp = rightem->em_parent ? rightem->em_parent : rightem;
+
+ parent_rinfo = create_join_clause(root, ec, opno,
+ leftp, rightp,
+ parent_ec);
+ }
+
rinfo = build_implied_join_equality(root,
opno,
ec->ec_collation,
@@ -1876,6 +1893,10 @@ create_join_clause(PlannerInfo *root,
rightem->em_nullable_relids),
ec->ec_min_security);
+ /* If it's a child clause, copy the parent's rinfo_serial */
+ if (parent_rinfo)
+ rinfo->rinfo_serial = parent_rinfo->rinfo_serial;
+
/* Mark the clause as redundant, or not */
rinfo->parent_ec = parent_ec;
@@ -2691,7 +2712,7 @@ add_child_rel_equivalences(PlannerInfo *root,
(void) add_eq_member(cur_ec, child_expr,
new_relids, new_nullable_relids,
- true, cur_em->em_datatype);
+ cur_em, cur_em->em_datatype);
/* Record this EC index for the child rel */
child_rel->eclass_indexes = bms_add_member(child_rel->eclass_indexes, i);
@@ -2832,7 +2853,7 @@ add_child_join_rel_equivalences(PlannerInfo *root,
(void) add_eq_member(cur_ec, child_expr,
new_relids, new_nullable_relids,
- true, cur_em->em_datatype);
+ cur_em, cur_em->em_datatype);
}
}
}
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 90da123c10..58d2752a7e 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -1889,6 +1889,7 @@ deconstruct_distribute_oj_quals(PlannerInfo *root,
Relids joins_below;
Relids joins_so_far;
List *quals;
+ int save_last_rinfo_serial;
ListCell *lc;
/*
@@ -1927,6 +1928,16 @@ deconstruct_distribute_oj_quals(PlannerInfo *root,
joins_below,
NULL);
+ /*
+ * Each time we produce RestrictInfo(s) from these quals, reset the
+ * last_rinfo_serial counter, so that the RestrictInfos for the "same"
+ * qual condition get identical serial numbers. (This relies on the
+ * fact that we're not changing the qual list in any way that'd affect
+ * the number of RestrictInfos built from it.) This'll allow us to
+ * detect duplicative qual usage later.
+ */
+ save_last_rinfo_serial = root->last_rinfo_serial;
+
joins_so_far = NULL;
foreach(lc, jtitems)
{
@@ -1964,6 +1975,9 @@ deconstruct_distribute_oj_quals(PlannerInfo *root,
continue;
}
+ /* Reset serial counter for this version of the quals */
+ root->last_rinfo_serial = save_last_rinfo_serial;
+
/*
* When we are looking at joins above sjinfo, we are envisioning
* pushing sjinfo to above othersj, so add othersj's nulling bit
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 71b4d3ca56..ed1be59687 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -625,6 +625,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
root->multiexpr_params = NIL;
root->eq_classes = NIL;
root->ec_merging_done = false;
+ root->last_rinfo_serial = 0;
root->all_result_relids =
parse->resultRelation ? bms_make_singleton(parse->resultRelation) : NULL;
root->leaf_result_relids = NULL; /* we'll find out leaf-ness later */
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 13bbdfd7ba..26c9b91e3a 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -992,6 +992,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
subroot->multiexpr_params = NIL;
subroot->eq_classes = NIL;
subroot->ec_merging_done = false;
+ subroot->last_rinfo_serial = 0;
subroot->all_result_relids = NULL;
subroot->leaf_result_relids = NULL;
subroot->append_rel_list = NIL;
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index 11c6bbaba6..e18d64b6dc 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -427,7 +427,7 @@ adjust_appendrel_attrs_mutator(Node *node,
RestrictInfo *oldinfo = (RestrictInfo *) node;
RestrictInfo *newinfo = makeNode(RestrictInfo);
- /* Copy all flat-copiable fields */
+ /* Copy all flat-copiable fields, notably including rinfo_serial */
memcpy(newinfo, oldinfo, sizeof(RestrictInfo));
/* Recursively fix the clause itself */
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 70a343b108..c7b0a08811 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2442,12 +2442,12 @@ create_nestloop_path(PlannerInfo *root,
* restrict_clauses that are due to be moved into the inner path. We have
* to do this now, rather than postpone the work till createplan time,
* because the restrict_clauses list can affect the size and cost
- * estimates for this path.
+ * estimates for this path. We detect such clauses by checking for serial
+ * number match to clauses already enforced in the inner path.
*/
if (bms_overlap(inner_req_outer, outer_path->parent->relids))
{
- Relids inner_and_outer = bms_union(inner_path->parent->relids,
- inner_req_outer);
+ Bitmapset *enforced_serials = get_param_path_clause_serials(inner_path);
List *jclauses = NIL;
ListCell *lc;
@@ -2455,9 +2455,7 @@ create_nestloop_path(PlannerInfo *root,
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
- if (!join_clause_is_movable_into(rinfo,
- inner_path->parent->relids,
- inner_and_outer))
+ if (!bms_is_member(rinfo->rinfo_serial, enforced_serials))
jclauses = lappend(jclauses, rinfo);
}
restrict_clauses = jclauses;
@@ -4298,6 +4296,7 @@ do { \
new_ppi->ppi_rows = old_ppi->ppi_rows;
new_ppi->ppi_clauses = old_ppi->ppi_clauses;
ADJUST_CHILD_ATTRS(new_ppi->ppi_clauses);
+ new_ppi->ppi_serials = bms_copy(old_ppi->ppi_serials);
rel->ppilist = lappend(rel->ppilist, new_ppi);
MemoryContextSwitchTo(oldcontext);
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index acee3e2559..e57bd0d3dc 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -1477,6 +1477,7 @@ get_baserel_parampathinfo(PlannerInfo *root, RelOptInfo *baserel,
ParamPathInfo *ppi;
Relids joinrelids;
List *pclauses;
+ Bitmapset *pserials;
double rows;
ListCell *lc;
@@ -1519,6 +1520,15 @@ get_baserel_parampathinfo(PlannerInfo *root, RelOptInfo *baserel,
required_outer,
baserel));
+ /* Compute set of serial numbers of the enforced clauses */
+ pserials = NULL;
+ foreach(lc, pclauses)
+ {
+ RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
+
+ pserials = bms_add_member(pserials, rinfo->rinfo_serial);
+ }
+
/* Estimate the number of rows returned by the parameterized scan */
rows = get_parameterized_baserel_size(root, baserel, pclauses);
@@ -1527,6 +1537,7 @@ get_baserel_parampathinfo(PlannerInfo *root, RelOptInfo *baserel,
ppi->ppi_req_outer = required_outer;
ppi->ppi_rows = rows;
ppi->ppi_clauses = pclauses;
+ ppi->ppi_serials = pserials;
baserel->ppilist = lappend(baserel->ppilist, ppi);
return ppi;
@@ -1752,6 +1763,7 @@ get_joinrel_parampathinfo(PlannerInfo *root, RelOptInfo *joinrel,
ppi->ppi_req_outer = required_outer;
ppi->ppi_rows = rows;
ppi->ppi_clauses = NIL;
+ ppi->ppi_serials = NULL;
joinrel->ppilist = lappend(joinrel->ppilist, ppi);
return ppi;
@@ -1790,6 +1802,7 @@ get_appendrel_parampathinfo(RelOptInfo *appendrel, Relids required_outer)
ppi->ppi_req_outer = required_outer;
ppi->ppi_rows = 0;
ppi->ppi_clauses = NIL;
+ ppi->ppi_serials = NULL;
appendrel->ppilist = lappend(appendrel->ppilist, ppi);
return ppi;
@@ -1815,6 +1828,100 @@ find_param_path_info(RelOptInfo *rel, Relids required_outer)
return NULL;
}
+/*
+ * get_param_path_clause_serials
+ * Given a parameterized Path, return the set of pushed-down clauses
+ * (identified by rinfo_serial numbers) enforced within the Path.
+ */
+Bitmapset *
+get_param_path_clause_serials(Path *path)
+{
+ if (path->param_info == NULL)
+ return NULL; /* not parameterized */
+ if (IsA(path, NestPath) ||
+ IsA(path, MergePath) ||
+ IsA(path, HashPath))
+ {
+ /*
+ * For a join path, combine clauses enforced within either input path
+ * with those enforced as joinrestrictinfo in this path. Note that
+ * joinrestrictinfo may include some non-pushed-down clauses, but for
+ * current purposes it's okay if we include those in the result. (To
+ * be more careful, we could check for clause_relids overlapping the
+ * path parameterization, but it's not worth the cycles for now.)
+ */
+ JoinPath *jpath = (JoinPath *) path;
+ Bitmapset *pserials;
+ ListCell *lc;
+
+ pserials = NULL;
+ pserials = bms_add_members(pserials,
+ get_param_path_clause_serials(jpath->outerjoinpath));
+ pserials = bms_add_members(pserials,
+ get_param_path_clause_serials(jpath->innerjoinpath));
+ foreach(lc, jpath->joinrestrictinfo)
+ {
+ RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
+
+ pserials = bms_add_member(pserials, rinfo->rinfo_serial);
+ }
+ return pserials;
+ }
+ else if (IsA(path, AppendPath))
+ {
+ /*
+ * For an appendrel, take the intersection of the sets of clauses
+ * enforced in each input path.
+ */
+ AppendPath *apath = (AppendPath *) path;
+ Bitmapset *pserials;
+ ListCell *lc;
+
+ pserials = NULL;
+ foreach(lc, apath->subpaths)
+ {
+ Path *subpath = (Path *) lfirst(lc);
+ Bitmapset *subserials;
+
+ subserials = get_param_path_clause_serials(subpath);
+ if (lc == list_head(apath->subpaths))
+ pserials = bms_copy(subserials);
+ else
+ pserials = bms_int_members(pserials, subserials);
+ }
+ return pserials;
+ }
+ else if (IsA(path, MergeAppendPath))
+ {
+ /* Same as AppendPath case */
+ MergeAppendPath *apath = (MergeAppendPath *) path;
+ Bitmapset *pserials;
+ ListCell *lc;
+
+ pserials = NULL;
+ foreach(lc, apath->subpaths)
+ {
+ Path *subpath = (Path *) lfirst(lc);
+ Bitmapset *subserials;
+
+ subserials = get_param_path_clause_serials(subpath);
+ if (lc == list_head(apath->subpaths))
+ pserials = bms_copy(subserials);
+ else
+ pserials = bms_int_members(pserials, subserials);
+ }
+ return pserials;
+ }
+ else
+ {
+ /*
+ * Otherwise, it's a baserel path and we can use the
+ * previously-computed set of serial numbers.
+ */
+ return path->param_info->ppi_serials;
+ }
+}
+
/*
* build_joinrel_partition_info
* Checks if the two relations being joined can use partitionwise join
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index 327c3ba563..bcbee8f943 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -208,6 +208,11 @@ make_restrictinfo_internal(PlannerInfo *root,
restrictinfo->num_base_rels = bms_num_members(baserels);
bms_free(baserels);
+ /*
+ * Label this RestrictInfo with a fresh serial number.
+ */
+ restrictinfo->rinfo_serial = ++(root->last_rinfo_serial);
+
/*
* Fill in all the cacheable fields with "not yet set" markers. None of
* these will be computed until/unless needed. Note in particular that we
@@ -371,7 +376,7 @@ commute_restrictinfo(RestrictInfo *rinfo, Oid comm_op)
* ... and adjust those we need to change. Note in particular that we can
* preserve any cached selectivity or cost estimates, since those ought to
* be the same for the new clause. Likewise we can keep the source's
- * parent_ec.
+ * parent_ec. It's also important that we keep the same rinfo_serial.
*/
result->clause = (Expr *) newclause;
result->left_relids = rinfo->right_relids;
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 8449dd7202..088c4a407d 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -344,6 +344,9 @@ struct PlannerInfo
/* list of SpecialJoinInfos */
List *join_info_list;
+ /* counter for assigning RestrictInfo serial numbers */
+ int last_rinfo_serial;
+
/*
* all_result_relids is empty for SELECT, otherwise it contains at least
* parse->resultRelation. For UPDATE/DELETE/MERGE across an inheritance
@@ -1362,6 +1365,8 @@ typedef struct EquivalenceMember
bool em_is_const; /* expression is pseudoconstant? */
bool em_is_child; /* derived version for a child relation? */
Oid em_datatype; /* the "nominal type" used by the opfamily */
+ /* if em_is_child is true, this links to corresponding EM for top parent */
+ struct EquivalenceMember *em_parent pg_node_attr(read_write_ignore);
} EquivalenceMember;
/*
@@ -1468,7 +1473,13 @@ typedef struct PathTarget
* Note: ppi_clauses is only used in ParamPathInfos for base relation paths;
* in join cases it's NIL because the set of relevant clauses varies depending
* on how the join is formed. The relevant clauses will appear in each
- * parameterized join path's joinrestrictinfo list, instead.
+ * parameterized join path's joinrestrictinfo list, instead. ParamPathInfos
+ * for append relations don't bother with this, either.
+ *
+ * ppi_serials is the set of rinfo_serial numbers for quals that are enforced
+ * by this path. As with ppi_clauses, it's only maintained for baserels.
+ * (We could construct it on-the-fly from ppi_clauses, but it seems better
+ * to materialize a copy.)
*/
typedef struct ParamPathInfo
{
@@ -1479,6 +1490,7 @@ typedef struct ParamPathInfo
Relids ppi_req_outer; /* rels supplying parameters used by path */
Cardinality ppi_rows; /* estimated number of result tuples */
List *ppi_clauses; /* join clauses available from outer rels */
+ Bitmapset *ppi_serials; /* set of rinfo_serial for enforced quals */
} ParamPathInfo;
@@ -2507,6 +2519,25 @@ typedef struct RestrictInfo
*/
Expr *orclause pg_node_attr(equal_ignore);
+ /*----------
+ * Serial number of this RestrictInfo. This is unique within the current
+ * PlannerInfo context, with a few critical exceptions:
+ * 1. When we generate multiple clones of the same qual condition to
+ * cope with outer join identity 3, all the clones get the same serial
+ * number. This reflects that we only want to apply one of them in any
+ * given plan.
+ * 2. If we manufacture a commuted version of a qual to use as an index
+ * condition, it copies the original's rinfo_serial, since it is in
+ * practice the same condition.
+ * 3. RestrictInfos made for a child relation copy their parent's
+ * rinfo_serial. Likewise, when an EquivalenceClass makes a derived
+ * equality clause for a child relation, it copies the rinfo_serial of
+ * the matching equality clause for the parent. This allows detection
+ * of redundant pushed-down equality clauses.
+ *----------
+ */
+ int rinfo_serial;
+
/*
* Generating EquivalenceClass. This field is NULL unless clause is
* potentially redundant.
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 197234d44c..3440455a2e 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -333,6 +333,7 @@ extern ParamPathInfo *get_appendrel_parampathinfo(RelOptInfo *appendrel,
Relids required_outer);
extern ParamPathInfo *find_param_path_info(RelOptInfo *rel,
Relids required_outer);
+extern Bitmapset *get_param_path_clause_serials(Path *path);
extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out
index 3ddea3b683..c1adb2679a 100644
--- a/src/test/regress/expected/join.out
+++ b/src/test/regress/expected/join.out
@@ -2335,17 +2335,17 @@ select a.f1, b.f1, t.thousand, t.tenthous from
(select sum(f1)+1 as f1 from int4_tbl i4a) a,
(select sum(f1) as f1 from int4_tbl i4b) b
where b.f1 = t.thousand and a.f1 = b.f1 and (a.f1+b.f1+999) = t.tenthous;
- QUERY PLAN
------------------------------------------------------------------------------------------------------------------------
+ QUERY PLAN
+-----------------------------------------------------------------------------------------------------------------
Nested Loop
- -> Aggregate
- -> Seq Scan on int4_tbl i4b
-> Nested Loop
Join Filter: ((sum(i4b.f1)) = ((sum(i4a.f1) + 1)))
-> Aggregate
-> Seq Scan on int4_tbl i4a
- -> Index Only Scan using tenk1_thous_tenthous on tenk1 t
- Index Cond: ((thousand = (sum(i4b.f1))) AND (tenthous = ((((sum(i4a.f1) + 1)) + (sum(i4b.f1))) + 999)))
+ -> Aggregate
+ -> Seq Scan on int4_tbl i4b
+ -> Index Only Scan using tenk1_thous_tenthous on tenk1 t
+ Index Cond: ((thousand = (sum(i4b.f1))) AND (tenthous = ((((sum(i4a.f1) + 1)) + (sum(i4b.f1))) + 999)))
(9 rows)
select a.f1, b.f1, t.thousand, t.tenthous from
diff --git a/src/test/regress/expected/partition_join.out b/src/test/regress/expected/partition_join.out
index c59caf1cb3..4c11c7be93 100644
--- a/src/test/regress/expected/partition_join.out
+++ b/src/test/regress/expected/partition_join.out
@@ -304,7 +304,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t2.b FROM prt2 t2 WHERE t2.a = 0)
-> Seq Scan on prt2_p2 t2_2
Filter: (a = 0)
-> Nested Loop Semi Join
- Join Filter: (t2_3.b = t1_3.a)
+ Join Filter: (t1_3.a = t2_3.b)
-> Seq Scan on prt1_p3 t1_3
Filter: (b = 0)
-> Materialize
@@ -601,7 +601,7 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM prt1 t1, prt2 t2, prt1_e t
Sort Key: t1.a
-> Append
-> Nested Loop
- Join Filter: (((t3_1.a + t3_1.b) / 2) = t1_1.a)
+ Join Filter: (t1_1.a = ((t3_1.a + t3_1.b) / 2))
-> Hash Join
Hash Cond: (t2_1.b = t1_1.a)
-> Seq Scan on prt2_p1 t2_1
@@ -611,7 +611,7 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM prt1 t1, prt2 t2, prt1_e t
-> Index Scan using iprt1_e_p1_ab2 on prt1_e_p1 t3_1
Index Cond: (((a + b) / 2) = t2_1.b)
-> Nested Loop
- Join Filter: (((t3_2.a + t3_2.b) / 2) = t1_2.a)
+ Join Filter: (t1_2.a = ((t3_2.a + t3_2.b) / 2))
-> Hash Join
Hash Cond: (t2_2.b = t1_2.a)
-> Seq Scan on prt2_p2 t2_2
@@ -621,7 +621,7 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM prt1 t1, prt2 t2, prt1_e t
-> Index Scan using iprt1_e_p2_ab2 on prt1_e_p2 t3_2
Index Cond: (((a + b) / 2) = t2_2.b)
-> Nested Loop
- Join Filter: (((t3_3.a + t3_3.b) / 2) = t1_3.a)
+ Join Filter: (t1_3.a = ((t3_3.a + t3_3.b) / 2))
-> Hash Join
Hash Cond: (t2_3.b = t1_3.a)
-> Seq Scan on prt2_p3 t2_3
@@ -926,7 +926,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t1.b FROM prt2 t1, prt1_e t2 WHER
Sort Key: t1.a
-> Append
-> Nested Loop
- Join Filter: (t1_5.b = t1_2.a)
+ Join Filter: (t1_2.a = t1_5.b)
-> HashAggregate
Group Key: t1_5.b
-> Hash Join
@@ -939,7 +939,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t1.b FROM prt2 t1, prt1_e t2 WHER
Index Cond: (a = ((t2_1.a + t2_1.b) / 2))
Filter: (b = 0)
-> Nested Loop
- Join Filter: (t1_6.b = t1_3.a)
+ Join Filter: (t1_3.a = t1_6.b)
-> HashAggregate
Group Key: t1_6.b
-> Hash Join
@@ -952,7 +952,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t1.b FROM prt2 t1, prt1_e t2 WHER
Index Cond: (a = ((t2_2.a + t2_2.b) / 2))
Filter: (b = 0)
-> Nested Loop
- Join Filter: (t1_7.b = t1_4.a)
+ Join Filter: (t1_4.a = t1_7.b)
-> HashAggregate
Group Key: t1_7.b
-> Nested Loop
commit cd920f677aaee5aa07b2ac2c23519ac2bbbd8885
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Fri Dec 23 10:27:36 2022 -0500
Fix flatten_join_alias_vars() to handle varnullingrels correctly.
The remaining core regression test failures occur because
flatten_join_alias_vars() isn't doing the right thing. The
alias Var it needs to replace may have acquired varnullingrels
bits signifying the effect of upper outer joins, and if so we
must preserve that information in the replacement expression.
The simplest way to do that is to wrap the replacement expression
in a PlaceHolderVar, and that's what we have to do in the general
case where subquery pullup has mutated the replacement joinaliasvars
entry into an arbitrary expression. But in simpler cases, such as
where the joinaliasvars entry is just a Var, we'd prefer to do it
by merging the alias Var's varnullingrels into the replacement Var.
In that way the flattened alias will compare equal() to semantically
equivalent references that didn't use the alias name.
Moreover, the parser also uses this code while checking certain
semantic constraints, and in that context we *must not* generate
PlaceHolderVars. PHVs shouldn't appear in parse-time expressions,
and adding one would certainly cause the parser to decide the
query is invalid (because the result wouldn't compare equal() to
what it needs to). Fortunately, during parsing the set of possible
contents of a joinaliasvars entry is quite constrained, so we can
guarantee to apply the nullingrels info to the Vars therein.
The result of this step passes all core regression tests, but there
are still loose ends for FDWs (so that contrib/postgres_fdw will fail).
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index ed1be59687..a139d94866 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -909,7 +909,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
*/
if (rte->lateral && root->hasJoinRTEs)
rte->subquery = (Query *)
- flatten_join_alias_vars(root->parse,
+ flatten_join_alias_vars(root, root->parse,
(Node *) rte->subquery);
}
else if (rte->rtekind == RTE_FUNCTION)
@@ -1110,7 +1110,7 @@ preprocess_expression(PlannerInfo *root, Node *expr, int kind)
kind == EXPRKIND_VALUES ||
kind == EXPRKIND_TABLESAMPLE ||
kind == EXPRKIND_TABLEFUNC))
- expr = flatten_join_alias_vars(root->parse, expr);
+ expr = flatten_join_alias_vars(root, root->parse, expr);
/*
* Simplify constant expressions. For function RTEs, this was already
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 26c9b91e3a..356d81bfea 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -1080,7 +1080,8 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* maybe even in the rewriter; but for now let's just fix this case here.)
*/
subquery->targetList = (List *)
- flatten_join_alias_vars(subroot->parse, (Node *) subquery->targetList);
+ flatten_join_alias_vars(subroot, subroot->parse,
+ (Node *) subquery->targetList);
/*
* Adjust level-0 varnos in subquery so that we can append its rangetable
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
index 8d8c9136f8..69c2019553 100644
--- a/src/backend/optimizer/util/var.c
+++ b/src/backend/optimizer/util/var.c
@@ -62,6 +62,7 @@ typedef struct
typedef struct
{
+ PlannerInfo *root; /* could be NULL! */
Query *query; /* outer Query */
int sublevels_up;
bool possible_sublink; /* could aliases include a SubLink? */
@@ -80,6 +81,10 @@ static bool pull_var_clause_walker(Node *node,
pull_var_clause_context *context);
static Node *flatten_join_alias_vars_mutator(Node *node,
flatten_join_alias_vars_context *context);
+static Node *add_nullingrels_if_needed(PlannerInfo *root, Node *newnode,
+ Var *oldvar);
+static bool is_standard_join_alias_expression(Node *newnode, Var *oldvar);
+static void adjust_standard_join_alias_expression(Node *newnode, Var *oldvar);
static Relids alias_relid_set(Query *query, Relids relids);
@@ -722,26 +727,42 @@ pull_var_clause_walker(Node *node, pull_var_clause_context *context)
* is the only way that the executor can directly handle whole-row Vars.
*
* This also adjusts relid sets found in some expression node types to
- * substitute the contained base rels for any join relid.
+ * substitute the contained base+OJ rels for any join relid.
*
* If a JOIN contains sub-selects that have been flattened, its join alias
* entries might now be arbitrary expressions, not just Vars. This affects
- * this function in one important way: we might find ourselves inserting
- * SubLink expressions into subqueries, and we must make sure that their
- * Query.hasSubLinks fields get set to true if so. If there are any
+ * this function in two important ways. First, we might find ourselves
+ * inserting SubLink expressions into subqueries, and we must make sure that
+ * their Query.hasSubLinks fields get set to true if so. If there are any
* SubLinks in the join alias lists, the outer Query should already have
* hasSubLinks = true, so this is only relevant to un-flattened subqueries.
+ * Second, we have to preserve any varnullingrels info attached to the
+ * alias Vars we're replacing. If the replacement expression is a Var or
+ * PlaceHolderVar or constructed from those, we can just add the
+ * varnullingrels bits to the existing nullingrels field(s); otherwise
+ * we have to add a PlaceHolderVar wrapper.
*
- * NOTE: this is used on not-yet-planned expressions. We do not expect it
- * to be applied directly to the whole Query, so if we see a Query to start
- * with, we do want to increment sublevels_up (this occurs for LATERAL
- * subqueries).
+ * NOTE: this is also used by the parser, to expand join alias Vars before
+ * checking GROUP BY validity. For that use-case, root will be NULL, which
+ * is why we have to pass the Query separately. We need the root itself only
+ * for making PlaceHolderVars. We can avoid making PlaceHolderVars in the
+ * parser's usage because it won't be dealing with arbitrary expressions:
+ * so long as adjust_standard_join_alias_expression can handle everything
+ * the parser would make as a join alias expression, we're OK.
*/
Node *
-flatten_join_alias_vars(Query *query, Node *node)
+flatten_join_alias_vars(PlannerInfo *root, Query *query, Node *node)
{
flatten_join_alias_vars_context context;
+ /*
+ * We do not expect this to be applied to the whole Query, only to
+ * expressions or LATERAL subqueries. Hence, if the top node is a Query,
+ * it's okay to immediately increment sublevels_up.
+ */
+ Assert(node != (Node *) query);
+
+ context.root = root;
context.query = query;
context.sublevels_up = 0;
/* flag whether join aliases could possibly contain SubLinks */
@@ -812,7 +833,9 @@ flatten_join_alias_vars_mutator(Node *node,
rowexpr->colnames = colnames;
rowexpr->location = var->location;
- return (Node *) rowexpr;
+ /* Lastly, add any varnullingrels to the replacement expression */
+ return add_nullingrels_if_needed(context->root, (Node *) rowexpr,
+ var);
}
/* Expand join alias reference */
@@ -839,7 +862,8 @@ flatten_join_alias_vars_mutator(Node *node,
if (context->possible_sublink && !context->inserted_sublink)
context->inserted_sublink = checkExprHasSubLink(newvar);
- return newvar;
+ /* Lastly, add any varnullingrels to the replacement expression */
+ return add_nullingrels_if_needed(context->root, newvar, var);
}
if (IsA(node, PlaceHolderVar))
{
@@ -854,6 +878,7 @@ flatten_join_alias_vars_mutator(Node *node,
{
phv->phrels = alias_relid_set(context->query,
phv->phrels);
+ /* we *don't* change phnullingrels */
}
return (Node *) phv;
}
@@ -887,9 +912,145 @@ flatten_join_alias_vars_mutator(Node *node,
(void *) context);
}
+/*
+ * Add oldvar's varnullingrels, if any, to a flattened join alias expression.
+ * The newnode has been copied, so we can modify it freely.
+ */
+static Node *
+add_nullingrels_if_needed(PlannerInfo *root, Node *newnode, Var *oldvar)
+{
+ if (oldvar->varnullingrels == NULL)
+ return newnode; /* nothing to do */
+ /* If possible, do it by adding to existing nullingrel fields */
+ if (is_standard_join_alias_expression(newnode, oldvar))
+ adjust_standard_join_alias_expression(newnode, oldvar);
+ else if (root)
+ {
+ /* We can insert a PlaceHolderVar to carry the nullingrels */
+ PlaceHolderVar *newphv;
+ Relids phrels = pull_varnos(root, newnode);
+
+ /* XXX what if phrels is empty? */
+ Assert(!bms_is_empty(phrels)); /* probably wrong */
+ newphv = make_placeholder_expr(root, (Expr *) newnode, phrels);
+ /* newphv has zero phlevelsup and NULL phnullingrels; fix it */
+ newphv->phlevelsup = oldvar->varlevelsup;
+ newphv->phnullingrels = bms_copy(oldvar->varnullingrels);
+ newnode = (Node *) newphv;
+ }
+ else
+ {
+ /* ooops, we're missing support for something the parser can make */
+ elog(ERROR, "unsupported join alias expression");
+ }
+ return newnode;
+}
+
+/*
+ * Check to see if we can insert nullingrels into this join alias expression
+ * without use of a separate PlaceHolderVar.
+ *
+ * This will handle Vars, PlaceHolderVars, and implicit-coercion and COALESCE
+ * expressions built from those. This coverage needs to handle anything
+ * that the parser would put into joinaliasvars.
+ * XXX it's probably incomplete at the moment.
+ */
+static bool
+is_standard_join_alias_expression(Node *newnode, Var *oldvar)
+{
+ if (newnode == NULL)
+ return false;
+ if (IsA(newnode, Var) &&
+ ((Var *) newnode)->varlevelsup == oldvar->varlevelsup)
+ return true;
+ else if (IsA(newnode, PlaceHolderVar) &&
+ ((PlaceHolderVar *) newnode)->phlevelsup == oldvar->varlevelsup)
+ return true;
+ else if (IsA(newnode, FuncExpr))
+ {
+ FuncExpr *fexpr = (FuncExpr *) newnode;
+
+ /*
+ * We need to assume that the function wouldn't produce non-NULL from
+ * NULL, which is reasonable for implicit coercions but otherwise not
+ * so much. (Looking at its strictness is likely overkill, and anyway
+ * it would cause us to fail if someone forgot to mark an implicit
+ * coercion as strict.)
+ */
+ if (fexpr->funcformat != COERCE_IMPLICIT_CAST ||
+ fexpr->args == NIL)
+ return false;
+
+ /*
+ * Examine only the first argument --- coercions might have additional
+ * arguments that are constants.
+ */
+ return is_standard_join_alias_expression(linitial(fexpr->args), oldvar);
+ }
+ else if (IsA(newnode, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) newnode;
+ ListCell *lc;
+
+ Assert(cexpr->args != NIL);
+ foreach(lc, cexpr->args)
+ {
+ if (!is_standard_join_alias_expression(lfirst(lc), oldvar))
+ return false;
+ }
+ return true;
+ }
+ else
+ return false;
+}
+
+/*
+ * Insert nullingrels into an expression accepted by
+ * is_standard_join_alias_expression.
+ */
+static void
+adjust_standard_join_alias_expression(Node *newnode, Var *oldvar)
+{
+ if (IsA(newnode, Var) &&
+ ((Var *) newnode)->varlevelsup == oldvar->varlevelsup)
+ {
+ Var *newvar = (Var *) newnode;
+
+ newvar->varnullingrels = bms_add_members(newvar->varnullingrels,
+ oldvar->varnullingrels);
+ }
+ else if (IsA(newnode, PlaceHolderVar) &&
+ ((PlaceHolderVar *) newnode)->phlevelsup == oldvar->varlevelsup)
+ {
+ PlaceHolderVar *newphv = (PlaceHolderVar *) newnode;
+
+ newphv->phnullingrels = bms_add_members(newphv->phnullingrels,
+ oldvar->varnullingrels);
+ }
+ else if (IsA(newnode, FuncExpr))
+ {
+ FuncExpr *fexpr = (FuncExpr *) newnode;
+
+ adjust_standard_join_alias_expression(linitial(fexpr->args), oldvar);
+ }
+ else if (IsA(newnode, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) newnode;
+ ListCell *lc;
+
+ Assert(cexpr->args != NIL);
+ foreach(lc, cexpr->args)
+ {
+ adjust_standard_join_alias_expression(lfirst(lc), oldvar);
+ }
+ }
+ else
+ Assert(false);
+}
+
/*
* alias_relid_set: in a set of RT indexes, replace joins by their
- * underlying base relids
+ * underlying base+OJ relids
*/
static Relids
alias_relid_set(Query *query, Relids relids)
diff --git a/src/backend/parser/parse_agg.c b/src/backend/parser/parse_agg.c
index 3ef9e8ee5e..c15fab0f68 100644
--- a/src/backend/parser/parse_agg.c
+++ b/src/backend/parser/parse_agg.c
@@ -1162,7 +1162,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
* entries are RTE_JOIN kind.
*/
if (hasJoinRTEs)
- groupClauses = (List *) flatten_join_alias_vars(qry,
+ groupClauses = (List *) flatten_join_alias_vars(NULL, qry,
(Node *) groupClauses);
/*
@@ -1206,7 +1206,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
groupClauses, hasJoinRTEs,
have_non_var_grouping);
if (hasJoinRTEs)
- clause = flatten_join_alias_vars(qry, clause);
+ clause = flatten_join_alias_vars(NULL, qry, clause);
check_ungrouped_columns(clause, pstate, qry,
groupClauses, groupClauseCommonVars,
have_non_var_grouping,
@@ -1217,7 +1217,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
groupClauses, hasJoinRTEs,
have_non_var_grouping);
if (hasJoinRTEs)
- clause = flatten_join_alias_vars(qry, clause);
+ clause = flatten_join_alias_vars(NULL, qry, clause);
check_ungrouped_columns(clause, pstate, qry,
groupClauses, groupClauseCommonVars,
have_non_var_grouping,
@@ -1546,7 +1546,7 @@ finalize_grouping_exprs_walker(Node *node,
Index ref = 0;
if (context->hasJoinRTEs)
- expr = flatten_join_alias_vars(context->qry, expr);
+ expr = flatten_join_alias_vars(NULL, context->qry, expr);
/*
* Each expression must match a grouping entry at the current
diff --git a/src/include/optimizer/optimizer.h b/src/include/optimizer/optimizer.h
index 409005bae9..95f3461a3d 100644
--- a/src/include/optimizer/optimizer.h
+++ b/src/include/optimizer/optimizer.h
@@ -197,6 +197,6 @@ extern bool contain_var_clause(Node *node);
extern bool contain_vars_of_level(Node *node, int levelsup);
extern int locate_var_of_level(Node *node, int levelsup);
extern List *pull_var_clause(Node *node, int flags);
-extern Node *flatten_join_alias_vars(Query *query, Node *node);
+extern Node *flatten_join_alias_vars(PlannerInfo *root, Query *query, Node *node);
#endif /* OPTIMIZER_H */
commit b187b829578334b13285385f924a896f3a1ee50a
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Fri Dec 23 10:33:13 2022 -0500
Teach FDWs about base-plus-outer-join relids.
Conversion of the planner to include OJ relids in join relids
affects FDWs that want to plan foreign joins. They *must* follow
suit when labeling foreign joins in order to match with the core
planner, but for many purposes (if postgres_fdw is any guide)
they'd prefer to consider only base relations within the join.
To support both requirements, redefine ForeignScan.fs_relids as
base+OJ relids, and add a new field fs_base_relids that's set up
by the core planner.
Another way we could do this is to keep fs_relids as just base
relids and make the new field be the one with OJ relids added.
While that would be more backwards-compatible in some sense,
it would be inconsistent with the naming used in the core planner,
and I think that it might allow some types of bugs to escape
quick detection.
postgres_fdw also has one place where it needs to ignore varnullingrels
while matching Vars. It's not clear whether it's worth trying to
improve that. (This too is probably only an issue for FDWs that do
join planning, since Vars seen in a base relation scan should never
have any varnullingrels.)
As of this step, this patch series again passes check-world.
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
index 9524765650..94dd7b2c96 100644
--- a/contrib/postgres_fdw/deparse.c
+++ b/contrib/postgres_fdw/deparse.c
@@ -3950,7 +3950,17 @@ get_relation_column_alias_ids(Var *node, RelOptInfo *foreignrel,
i = 1;
foreach(lc, foreignrel->reltarget->exprs)
{
- if (equal(lfirst(lc), (Node *) node))
+ Var *tlvar = (Var *) lfirst(lc);
+
+ /*
+ * Match reltarget entries only on varno/varattno. Ideally there
+ * would be some cross-check on varnullingrels, but it's unclear what
+ * to do exactly; we don't have enough context to know what that value
+ * should be.
+ */
+ if (IsA(tlvar, Var) &&
+ tlvar->varno == node->varno &&
+ tlvar->varattno == node->varattno)
{
*colno = i;
return;
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index b9268e32dd..4028c5692e 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -1517,7 +1517,7 @@ postgresBeginForeignScan(ForeignScanState *node, int eflags)
if (fsplan->scan.scanrelid > 0)
rtindex = fsplan->scan.scanrelid;
else
- rtindex = bms_next_member(fsplan->fs_relids, -1);
+ rtindex = bms_next_member(fsplan->fs_base_relids, -1);
rte = exec_rt_fetch(rtindex, estate);
/* Get info about foreign table. */
@@ -2414,7 +2414,7 @@ find_modifytable_subplan(PlannerInfo *root,
{
ForeignScan *fscan = (ForeignScan *) subplan;
- if (bms_is_member(rtindex, fscan->fs_relids))
+ if (bms_is_member(rtindex, fscan->fs_base_relids))
return fscan;
}
@@ -2838,8 +2838,8 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
* that setrefs.c won't update the string when flattening the
* rangetable. To find out what rtoffset was applied, identify the
* minimum RT index appearing in the string and compare it to the
- * minimum member of plan->fs_relids. (We expect all the relids in
- * the join will have been offset by the same amount; the Asserts
+ * minimum member of plan->fs_base_relids. (We expect all the relids
+ * in the join will have been offset by the same amount; the Asserts
* below should catch it if that ever changes.)
*/
minrti = INT_MAX;
@@ -2856,7 +2856,7 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
else
ptr++;
}
- rtoffset = bms_next_member(plan->fs_relids, -1) - minrti;
+ rtoffset = bms_next_member(plan->fs_base_relids, -1) - minrti;
/* Now we can translate the string */
relations = makeStringInfo();
@@ -2871,7 +2871,7 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
char *refname;
rti += rtoffset;
- Assert(bms_is_member(rti, plan->fs_relids));
+ Assert(bms_is_member(rti, plan->fs_base_relids));
rte = rt_fetch(rti, es->rtable);
Assert(rte->rtekind == RTE_RELATION);
/* This logic should agree with explain.c's ExplainTargetRel */
diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml
index 94263c628f..ac1717bc3c 100644
--- a/doc/src/sgml/fdwhandler.sgml
+++ b/doc/src/sgml/fdwhandler.sgml
@@ -351,6 +351,17 @@ GetForeignJoinPaths(PlannerInfo *root,
it will supply at run time in the tuples it returns.
</para>
+ <note>
+ <para>
+ Beginning with <productname>PostgreSQL</productname> 16,
+ <structfield>fs_relids</structfield> includes the rangetable indexes
+ of outer joins, if any were involved in this join. The new field
+ <structfield>fs_base_relids</structfield> includes only base
+ relation indexes, and thus
+ mimics <structfield>fs_relids</structfield>'s old semantics.
+ </para>
+ </note>
+
<para>
See <xref linkend="fdw-planning"/> for additional information.
</para>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index f86983c660..ed9a118416 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1114,7 +1114,7 @@ ExplainPreScanNode(PlanState *planstate, Bitmapset **rels_used)
break;
case T_ForeignScan:
*rels_used = bms_add_members(*rels_used,
- ((ForeignScan *) plan)->fs_relids);
+ ((ForeignScan *) plan)->fs_base_relids);
break;
case T_CustomScan:
*rels_used = bms_add_members(*rels_used,
diff --git a/src/backend/executor/execScan.c b/src/backend/executor/execScan.c
index 043bb83f55..2b37266b6a 100644
--- a/src/backend/executor/execScan.c
+++ b/src/backend/executor/execScan.c
@@ -325,7 +325,7 @@ ExecScanReScan(ScanState *node)
* all of them.
*/
if (IsA(node->ps.plan, ForeignScan))
- relids = ((ForeignScan *) node->ps.plan)->fs_relids;
+ relids = ((ForeignScan *) node->ps.plan)->fs_base_relids;
else if (IsA(node->ps.plan, CustomScan))
relids = ((CustomScan *) node->ps.plan)->custom_relids;
else
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 66139928e8..78dc153882 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -4158,14 +4158,22 @@ create_foreignscan_plan(PlannerInfo *root, ForeignPath *best_path,
/*
* Likewise, copy the relids that are represented by this foreign scan. An
- * upper rel doesn't have relids set, but it covers all the base relations
- * participating in the underlying scan, so use root's all_baserels.
+ * upper rel doesn't have relids set, but it covers all the relations
+ * participating in the underlying scan/join, so use root->all_query_rels.
*/
if (rel->reloptkind == RELOPT_UPPER_REL)
- scan_plan->fs_relids = root->all_baserels;
+ scan_plan->fs_relids = root->all_query_rels;
else
scan_plan->fs_relids = best_path->path.parent->relids;
+ /*
+ * Join relid sets include relevant outer joins, but FDWs may need to know
+ * which are the included base rels. That's a bit tedious to get without
+ * access to the plan-time data structures, so compute it here.
+ */
+ scan_plan->fs_base_relids = bms_difference(scan_plan->fs_relids,
+ root->outer_join_rels);
+
/*
* If this is a foreign join, and to make it valid to push down we had to
* assume that the current user is the same as some user explicitly named
@@ -5806,8 +5814,9 @@ make_foreignscan(List *qptlist,
node->fdw_private = fdw_private;
node->fdw_scan_tlist = fdw_scan_tlist;
node->fdw_recheck_quals = fdw_recheck_quals;
- /* fs_relids will be filled in by create_foreignscan_plan */
+ /* fs_relids, fs_base_relids will be filled by create_foreignscan_plan */
node->fs_relids = NULL;
+ node->fs_base_relids = NULL;
/* fsSystemCol will be filled in by create_foreignscan_plan */
node->fsSystemCol = false;
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 846ca39269..eadfbb7218 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1628,6 +1628,7 @@ set_foreignscan_references(PlannerInfo *root,
}
fscan->fs_relids = offset_relid_set(fscan->fs_relids, rtoffset);
+ fscan->fs_base_relids = offset_relid_set(fscan->fs_base_relids, rtoffset);
/* Adjust resultRelation if it's valid */
if (fscan->resultRelation > 0)
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index bddfe86191..4d0ffb6039 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -695,6 +695,7 @@ typedef struct WorkTableScan
* When the plan node represents a foreign join, scan.scanrelid is zero and
* fs_relids must be consulted to identify the join relation. (fs_relids
* is valid for simple scans as well, but will always match scan.scanrelid.)
+ * fs_relids includes outer joins; fs_base_relids does not.
*
* If the FDW's PlanDirectModify() callback decides to repurpose a ForeignScan
* node to perform the UPDATE or DELETE operation directly in the remote
@@ -716,7 +717,8 @@ typedef struct ForeignScan
List *fdw_private; /* private data for FDW */
List *fdw_scan_tlist; /* optional tlist describing scan tuple */
List *fdw_recheck_quals; /* original quals not in scan.plan.qual */
- Bitmapset *fs_relids; /* RTIs generated by this scan */
+ Bitmapset *fs_relids; /* base+OJ RTIs generated by this scan */
+ Bitmapset *fs_base_relids; /* base RTIs generated by this scan */
bool fsSystemCol; /* true if any "system column" is needed */
} ForeignScan;
commit 771ac6255f752640e36512505907ad92afe1c3df
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Fri Dec 23 10:44:10 2022 -0500
Don't use RestrictInfo.nullable_relids in join_clause_is_movable_to.
Instead of using per-clause nullable_relids data, compute a
per-baserel set of outer joins that can null each relation, and
check for overlap between that and clause_relids to detect whether
the clause can safely be pushed down to relation scan level.
join_clause_is_movable_into also uses nullable_relids, but it
turns out that that test can just be dropped entirely. Now that
clause_relids includes nulling outer joins, the preceding tests
in the function are sufficient to reject clauses that can't be
pushed down.
This might seem like a net loss given that we have to add a bit
of code to initsplan.c to compute RelOptInfo.nulling_relids.
However, that's not much code at all, and the payoff is this:
we no longer need RestrictInfo.nullable_relids at all.
The next patch, which removes that field and the extensive
infrastructure that maintains it, saves way more code and cycles
than we add here. Also, I think there are likely going to be
other uses for RelOptInfo.nulling_relids.
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 58d2752a7e..325dae3d45 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -94,6 +94,8 @@ static void deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
static void process_security_barrier_quals(PlannerInfo *root,
int rti, Relids qualscope,
bool below_outer_join);
+static void mark_rels_nulled_by_join(PlannerInfo *root, Index ojrelid,
+ Relids lower_rels);
static SpecialJoinInfo *make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
@@ -976,6 +978,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
j->rtindex);
root->outer_join_rels = bms_add_member(root->outer_join_rels,
j->rtindex);
+ mark_rels_nulled_by_join(root, j->rtindex,
+ right_item->qualscope);
}
jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
right_item->inner_join_rels);
@@ -1031,6 +1035,10 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
j->rtindex);
root->outer_join_rels = bms_add_member(root->outer_join_rels,
j->rtindex);
+ mark_rels_nulled_by_join(root, j->rtindex,
+ left_item->qualscope);
+ mark_rels_nulled_by_join(root, j->rtindex,
+ right_item->qualscope);
jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
right_item->inner_join_rels);
jtitem->left_rels = left_item->qualscope;
@@ -1345,6 +1353,33 @@ process_security_barrier_quals(PlannerInfo *root,
Assert(security_level <= root->qual_security_level);
}
+/*
+ * mark_rels_nulled_by_join
+ * Fill RelOptInfo.nulling_relids of baserels nulled by this outer join
+ *
+ * Inputs:
+ * ojrelid: RT index of the join RTE (must not be 0)
+ * lower_rels: the base+OJ Relids syntactically below nullable side of join
+ */
+static void
+mark_rels_nulled_by_join(PlannerInfo *root, Index ojrelid,
+ Relids lower_rels)
+{
+ int relid = -1;
+
+ while ((relid = bms_next_member(lower_rels, relid)) > 0)
+ {
+ RelOptInfo *rel = root->simple_rel_array[relid];
+
+ if (rel == NULL) /* must be an outer join */
+ {
+ Assert(bms_is_member(relid, root->outer_join_rels));
+ continue;
+ }
+ rel->nulling_relids = bms_add_member(rel->nulling_relids, ojrelid);
+ }
+}
+
/*
* make_outerjoininfo
* Build a SpecialJoinInfo for the current outer join
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index e57bd0d3dc..52cba6414e 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -283,6 +283,12 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
rel->top_parent = parent->top_parent ? parent->top_parent : parent;
rel->top_parent_relids = rel->top_parent->relids;
+ /*
+ * A child rel is below the same outer joins as its parent. (We
+ * presume this info was already calculated for the parent.)
+ */
+ rel->nulling_relids = parent->nulling_relids;
+
/*
* Also propagate lateral-reference information from appendrel parent
* rels to their child rels. We intentionally give each child rel the
@@ -306,6 +312,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
rel->parent = NULL;
rel->top_parent = NULL;
rel->top_parent_relids = NULL;
+ rel->nulling_relids = NULL;
rel->direct_lateral_relids = NULL;
rel->lateral_relids = NULL;
rel->lateral_referencers = NULL;
@@ -686,6 +693,7 @@ build_join_rel(PlannerInfo *root,
joinrel->max_attr = 0;
joinrel->attr_needed = NULL;
joinrel->attr_widths = NULL;
+ joinrel->nulling_relids = NULL;
joinrel->lateral_vars = NIL;
joinrel->lateral_referencers = NULL;
joinrel->indexlist = NIL;
@@ -875,6 +883,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->max_attr = 0;
joinrel->attr_needed = NULL;
joinrel->attr_widths = NULL;
+ joinrel->nulling_relids = NULL;
joinrel->lateral_vars = NIL;
joinrel->lateral_referencers = NULL;
joinrel->indexlist = NIL;
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index bcbee8f943..15f410cf36 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -618,8 +618,17 @@ join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
if (bms_is_member(baserel->relid, rinfo->outer_relids))
return false;
- /* Target rel must not be nullable below the clause */
- if (bms_is_member(baserel->relid, rinfo->nullable_relids))
+ /*
+ * Target rel's Vars must not be nulled by any outer join. We can check
+ * this without groveling through the individual Vars by seeing whether
+ * clause_relids (which includes all such Vars' varnullingrels) includes
+ * any outer join that can null the target rel. You might object that
+ * this could reject the clause on the basis of an OJ relid that came from
+ * some other rel's Var. However, that would still mean that the clause
+ * came from above that outer join and shouldn't be pushed down; so there
+ * should be no false positives.
+ */
+ if (bms_overlap(rinfo->clause_relids, baserel->nulling_relids))
return false;
/* Clause must not use any rels with LATERAL references to this rel */
@@ -651,16 +660,17 @@ join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
* relation plus the outer rels. We also check that it does reference at
* least one current Var, ensuring that the clause will be pushed down to
* a unique place in a parameterized join tree. And we check that we're
- * not pushing the clause into its outer-join outer side, nor down into
- * a lower outer join's inner side.
- *
- * The check about pushing a clause down into a lower outer join's inner side
- * is only approximate; it sometimes returns "false" when actually it would
- * be safe to use the clause here because we're still above the outer join
- * in question. This is okay as long as the answers at different join levels
- * are consistent: it just means we might sometimes fail to push a clause as
- * far down as it could safely be pushed. It's unclear whether it would be
- * worthwhile to do this more precisely. (But if it's ever fixed to be
+ * not pushing the clause into its outer-join outer side.
+ *
+ * We used to need to check that we're not pushing the clause into a lower
+ * outer join's inner side. However, now that clause_relids includes
+ * references to potentially-nulling outer joins, the other tests handle that
+ * concern. If the clause references any Var coming from the inside of a
+ * lower outer join, its clause_relids will mention that outer join, causing
+ * the evaluability check to fail; while if it references no such Vars, the
+ * references-a-target-rel check will fail.
+ *
+ * XXX not clear if we can do this yet: (But if it's ever fixed to be
* exactly accurate, there's an Assert in get_joinrel_parampathinfo() that
* should be re-enabled.)
*
@@ -704,14 +714,5 @@ join_clause_is_movable_into(RestrictInfo *rinfo,
if (bms_overlap(currentrelids, rinfo->outer_relids))
return false;
- /*
- * Target rel(s) must not be nullable below the clause. This is
- * approximate, in the safe direction, because the current join might be
- * above the join where the nulling would happen, in which case the clause
- * would work correctly here. But we don't have enough info to be sure.
- */
- if (bms_overlap(currentrelids, rinfo->nullable_relids))
- return false;
-
return true;
}
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 088c4a407d..b36cb41edf 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -666,6 +666,7 @@ typedef struct PartitionSchemeData *PartitionScheme;
* the attribute is needed as part of final targetlist
* attr_widths - cache space for per-attribute width estimates;
* zero means not computed yet
+ * nulling_relids - relids of outer joins that can null this rel
* lateral_vars - lateral cross-references of rel, if any (list of
* Vars and PlaceHolderVars)
* lateral_referencers - relids of rels that reference this one laterally
@@ -899,6 +900,8 @@ typedef struct RelOptInfo
Relids *attr_needed pg_node_attr(read_write_ignore);
/* array indexed [min_attr .. max_attr] */
int32 *attr_widths pg_node_attr(read_write_ignore);
+ /* relids of outer joins that can null this baserel */
+ Relids nulling_relids;
/* LATERAL Vars and PHVs referenced by rel */
List *lateral_vars;
/* rels that reference this baserel laterally */
commit 5be335129bedbd5fe648f71ea801a0e1b0677fc5
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Fri Dec 23 10:55:27 2022 -0500
Remove RestrictInfo.nullable_relids and associated infrastructure.
There is no more code using this field, only code computing it,
so just delete all that. We can likewise get rid of
EquivalenceMember.em_nullable_relids and
PlannerInfo.nullable_baserels.
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 4028c5692e..12e940a2e6 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -6308,7 +6308,6 @@ foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
false,
root->qual_security_level,
grouped_rel->relids,
- NULL,
NULL);
if (is_foreign_expr(root, grouped_rel, expr))
fpinfo->remote_conds = lappend(fpinfo->remote_conds, rinfo);
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index d81b09add5..05f24b01b1 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -2745,7 +2745,6 @@ set_function_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
if (var)
pathkeys = build_expression_pathkey(root,
(Expr *) var,
- NULL, /* below outer joins */
Int8LessOperator,
rel->relids,
false);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index d1e1965479..a7c84daa74 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -34,7 +34,7 @@
static EquivalenceMember *add_eq_member(EquivalenceClass *ec,
- Expr *expr, Relids relids, Relids nullable_relids,
+ Expr *expr, Relids relids,
EquivalenceMember *parent,
Oid datatype);
static bool is_exprlist_member(Expr *node, List *exprs);
@@ -131,9 +131,7 @@ process_equivalence(PlannerInfo *root,
Expr *item1;
Expr *item2;
Relids item1_relids,
- item2_relids,
- item1_nullable_relids,
- item2_nullable_relids;
+ item2_relids;
List *opfamilies;
EquivalenceClass *ec1,
*ec2;
@@ -206,8 +204,7 @@ process_equivalence(PlannerInfo *root,
restrictinfo->pseudoconstant,
restrictinfo->security_level,
NULL,
- restrictinfo->outer_relids,
- restrictinfo->nullable_relids);
+ restrictinfo->outer_relids);
}
return false;
}
@@ -225,12 +222,6 @@ process_equivalence(PlannerInfo *root,
return false; /* RHS is non-strict but not constant */
}
- /* Calculate nullable-relid sets for each side of the clause */
- item1_nullable_relids = bms_intersect(item1_relids,
- restrictinfo->nullable_relids);
- item2_nullable_relids = bms_intersect(item2_relids,
- restrictinfo->nullable_relids);
-
/*
* We use the declared input types of the operator, not exprType() of the
* inputs, as the nominal datatypes for opfamily lookup. This presumes
@@ -400,7 +391,7 @@ process_equivalence(PlannerInfo *root,
else if (ec1)
{
/* Case 3: add item2 to ec1 */
- em2 = add_eq_member(ec1, item2, item2_relids, item2_nullable_relids,
+ em2 = add_eq_member(ec1, item2, item2_relids,
NULL, item2_type);
ec1->ec_sources = lappend(ec1->ec_sources, restrictinfo);
ec1->ec_below_outer_join |= below_outer_join;
@@ -418,7 +409,7 @@ process_equivalence(PlannerInfo *root,
else if (ec2)
{
/* Case 3: add item1 to ec2 */
- em1 = add_eq_member(ec2, item1, item1_relids, item1_nullable_relids,
+ em1 = add_eq_member(ec2, item1, item1_relids,
NULL, item1_type);
ec2->ec_sources = lappend(ec2->ec_sources, restrictinfo);
ec2->ec_below_outer_join |= below_outer_join;
@@ -452,9 +443,9 @@ process_equivalence(PlannerInfo *root,
ec->ec_min_security = restrictinfo->security_level;
ec->ec_max_security = restrictinfo->security_level;
ec->ec_merged = NULL;
- em1 = add_eq_member(ec, item1, item1_relids, item1_nullable_relids,
+ em1 = add_eq_member(ec, item1, item1_relids,
NULL, item1_type);
- em2 = add_eq_member(ec, item2, item2_relids, item2_nullable_relids,
+ em2 = add_eq_member(ec, item2, item2_relids,
NULL, item2_type);
root->eq_classes = lappend(root->eq_classes, ec);
@@ -545,13 +536,12 @@ canonicalize_ec_expression(Expr *expr, Oid req_type, Oid req_collation)
*/
static EquivalenceMember *
add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
- Relids nullable_relids, EquivalenceMember *parent, Oid datatype)
+ EquivalenceMember *parent, Oid datatype)
{
EquivalenceMember *em = makeNode(EquivalenceMember);
em->em_expr = expr;
em->em_relids = relids;
- em->em_nullable_relids = nullable_relids;
em->em_is_const = false;
em->em_is_child = (parent != NULL);
em->em_datatype = datatype;
@@ -588,13 +578,6 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
* equivalence class it is a member of; if none, optionally build a new
* single-member EquivalenceClass for it.
*
- * expr is the expression, and nullable_relids is the set of base relids
- * that are potentially nullable below it. We actually only care about
- * the set of such relids that are used in the expression; but for caller
- * convenience, we perform that intersection step here. The caller need
- * only be sure that nullable_relids doesn't omit any nullable rels that
- * might appear in the expr.
- *
* sortref is the SortGroupRef of the originating SortGroupClause, if any,
* or zero if not. (It should never be zero if the expression is volatile!)
*
@@ -623,7 +606,6 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
EquivalenceClass *
get_eclass_for_sort_expr(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
List *opfamilies,
Oid opcintype,
Oid collation,
@@ -719,13 +701,12 @@ get_eclass_for_sort_expr(PlannerInfo *root,
elog(ERROR, "volatile EquivalenceClass has no sortref");
/*
- * Get the precise set of nullable relids appearing in the expression.
+ * Get the precise set of relids appearing in the expression.
*/
expr_relids = pull_varnos(root, (Node *) expr);
- nullable_relids = bms_intersect(nullable_relids, expr_relids);
newem = add_eq_member(newec, copyObject(expr), expr_relids,
- nullable_relids, NULL, opcintype);
+ NULL, opcintype);
/*
* add_eq_member doesn't check for volatile functions, set-returning
@@ -1211,8 +1192,6 @@ generate_base_implied_equalities_const(PlannerInfo *root,
rinfo = process_implied_equality(root, eq_op, ec->ec_collation,
cur_em->em_expr, const_em->em_expr,
bms_copy(ec->ec_relids),
- bms_union(cur_em->em_nullable_relids,
- const_em->em_nullable_relids),
ec->ec_min_security,
ec->ec_below_outer_join,
cur_em->em_is_const);
@@ -1285,8 +1264,6 @@ generate_base_implied_equalities_no_const(PlannerInfo *root,
rinfo = process_implied_equality(root, eq_op, ec->ec_collation,
prev_em->em_expr, cur_em->em_expr,
bms_copy(ec->ec_relids),
- bms_union(prev_em->em_nullable_relids,
- cur_em->em_nullable_relids),
ec->ec_min_security,
ec->ec_below_outer_join,
false);
@@ -1889,8 +1866,6 @@ create_join_clause(PlannerInfo *root,
rightem->em_expr,
bms_union(leftem->em_relids,
rightem->em_relids),
- bms_union(leftem->em_nullable_relids,
- rightem->em_nullable_relids),
ec->ec_min_security);
/* If it's a child clause, copy the parent's rinfo_serial */
@@ -2110,8 +2085,7 @@ reconsider_outer_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo,
left_type,
right_type,
inner_datatype;
- Relids inner_relids,
- inner_nullable_relids;
+ Relids inner_relids;
ListCell *lc1;
Assert(is_opclause(rinfo->clause));
@@ -2138,8 +2112,6 @@ reconsider_outer_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo,
inner_datatype = left_type;
inner_relids = rinfo->left_relids;
}
- inner_nullable_relids = bms_intersect(inner_relids,
- rinfo->nullable_relids);
/* Scan EquivalenceClasses for a match to outervar */
foreach(lc1, root->eq_classes)
@@ -2200,7 +2172,6 @@ reconsider_outer_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo,
innervar,
cur_em->em_expr,
bms_copy(inner_relids),
- bms_copy(inner_nullable_relids),
cur_ec->ec_min_security);
if (process_equivalence(root, &newrinfo, true))
match = true;
@@ -2238,9 +2209,7 @@ reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
left_type,
right_type;
Relids left_relids,
- right_relids,
- left_nullable_relids,
- right_nullable_relids;
+ right_relids;
ListCell *lc1;
/* Can't use an outerjoin_delayed clause here */
@@ -2256,10 +2225,6 @@ reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
rightvar = (Expr *) get_rightop(rinfo->clause);
left_relids = rinfo->left_relids;
right_relids = rinfo->right_relids;
- left_nullable_relids = bms_intersect(left_relids,
- rinfo->nullable_relids);
- right_nullable_relids = bms_intersect(right_relids,
- rinfo->nullable_relids);
foreach(lc1, root->eq_classes)
{
@@ -2361,7 +2326,6 @@ reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
leftvar,
cur_em->em_expr,
bms_copy(left_relids),
- bms_copy(left_nullable_relids),
cur_ec->ec_min_security);
if (process_equivalence(root, &newrinfo, true))
matchleft = true;
@@ -2377,7 +2341,6 @@ reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
rightvar,
cur_em->em_expr,
bms_copy(right_relids),
- bms_copy(right_nullable_relids),
cur_ec->ec_min_security);
if (process_equivalence(root, &newrinfo, true))
matchright = true;
@@ -2667,7 +2630,6 @@ add_child_rel_equivalences(PlannerInfo *root,
/* Yes, generate transformed child version */
Expr *child_expr;
Relids new_relids;
- Relids new_nullable_relids;
if (parent_rel->reloptkind == RELOPT_BASEREL)
{
@@ -2697,21 +2659,7 @@ add_child_rel_equivalences(PlannerInfo *root,
top_parent_relids);
new_relids = bms_add_members(new_relids, child_relids);
- /*
- * And likewise for nullable_relids. Note this code assumes
- * parent and child relids are singletons.
- */
- new_nullable_relids = cur_em->em_nullable_relids;
- if (bms_overlap(new_nullable_relids, top_parent_relids))
- {
- new_nullable_relids = bms_difference(new_nullable_relids,
- top_parent_relids);
- new_nullable_relids = bms_add_members(new_nullable_relids,
- child_relids);
- }
-
- (void) add_eq_member(cur_ec, child_expr,
- new_relids, new_nullable_relids,
+ (void) add_eq_member(cur_ec, child_expr, new_relids,
cur_em, cur_em->em_datatype);
/* Record this EC index for the child rel */
@@ -2808,7 +2756,6 @@ add_child_join_rel_equivalences(PlannerInfo *root,
/* Yes, generate transformed child version */
Expr *child_expr;
Relids new_relids;
- Relids new_nullable_relids;
if (parent_joinrel->reloptkind == RELOPT_JOINREL)
{
@@ -2839,20 +2786,7 @@ add_child_join_rel_equivalences(PlannerInfo *root,
top_parent_relids);
new_relids = bms_add_members(new_relids, child_relids);
- /*
- * For nullable_relids, we must selectively replace parent
- * nullable relids with child ones.
- */
- new_nullable_relids = cur_em->em_nullable_relids;
- if (bms_overlap(new_nullable_relids, top_parent_relids))
- new_nullable_relids =
- adjust_child_relids_multilevel(root,
- new_nullable_relids,
- child_joinrel,
- child_joinrel->top_parent);
-
- (void) add_eq_member(cur_ec, child_expr,
- new_relids, new_nullable_relids,
+ (void) add_eq_member(cur_ec, child_expr, new_relids,
cur_em, cur_em->em_datatype);
}
}
diff --git a/src/backend/optimizer/path/pathkeys.c b/src/backend/optimizer/path/pathkeys.c
index a9943cd6e0..bf919ca97f 100644
--- a/src/backend/optimizer/path/pathkeys.c
+++ b/src/backend/optimizer/path/pathkeys.c
@@ -180,9 +180,6 @@ pathkey_is_redundant(PathKey *new_pathkey, List *pathkeys)
* Given an expression and sort-order information, create a PathKey.
* The result is always a "canonical" PathKey, but it might be redundant.
*
- * expr is the expression, and nullable_relids is the set of base relids
- * that are potentially nullable below it.
- *
* If the PathKey is being generated from a SortGroupClause, sortref should be
* the SortGroupClause's SortGroupRef; otherwise zero.
*
@@ -198,7 +195,6 @@ pathkey_is_redundant(PathKey *new_pathkey, List *pathkeys)
static PathKey *
make_pathkey_from_sortinfo(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
Oid opfamily,
Oid opcintype,
Oid collation,
@@ -234,7 +230,7 @@ make_pathkey_from_sortinfo(PlannerInfo *root,
equality_op);
/* Now find or (optionally) create a matching EquivalenceClass */
- eclass = get_eclass_for_sort_expr(root, expr, nullable_relids,
+ eclass = get_eclass_for_sort_expr(root, expr,
opfamilies, opcintype, collation,
sortref, rel, create_it);
@@ -257,7 +253,6 @@ make_pathkey_from_sortinfo(PlannerInfo *root,
static PathKey *
make_pathkey_from_sortop(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
Oid ordering_op,
bool nulls_first,
Index sortref,
@@ -279,7 +274,6 @@ make_pathkey_from_sortop(PlannerInfo *root,
return make_pathkey_from_sortinfo(root,
expr,
- nullable_relids,
opfamily,
opcintype,
collation,
@@ -584,12 +578,10 @@ build_index_pathkeys(PlannerInfo *root,
}
/*
- * OK, try to make a canonical pathkey for this sort key. Note we're
- * underneath any outer joins, so nullable_relids should be NULL.
+ * OK, try to make a canonical pathkey for this sort key.
*/
cpathkey = make_pathkey_from_sortinfo(root,
indexkey,
- NULL,
index->sortopfamily[i],
index->opcintype[i],
index->indexcollations[i],
@@ -743,14 +735,12 @@ build_partition_pathkeys(PlannerInfo *root, RelOptInfo *partrel,
/*
* Try to make a canonical pathkey for this partkey.
*
- * We're considering a baserel scan, so nullable_relids should be
- * NULL. Also, we assume the PartitionDesc lists any NULL partition
- * last, so we treat the scan like a NULLS LAST index: we have
- * nulls_first for backwards scan only.
+ * We assume the PartitionDesc lists any NULL partition last, so we
+ * treat the scan like a NULLS LAST index: we have nulls_first for
+ * backwards scan only.
*/
cpathkey = make_pathkey_from_sortinfo(root,
keyCol,
- NULL,
partscheme->partopfamily[i],
partscheme->partopcintype[i],
partscheme->partcollation[i],
@@ -799,7 +789,7 @@ build_partition_pathkeys(PlannerInfo *root, RelOptInfo *partrel,
* Build a pathkeys list that describes an ordering by a single expression
* using the given sort operator.
*
- * expr, nullable_relids, and rel are as for make_pathkey_from_sortinfo.
+ * expr and rel are as for make_pathkey_from_sortinfo.
* We induce the other arguments assuming default sort order for the operator.
*
* Similarly to make_pathkey_from_sortinfo, the result is NIL if create_it
@@ -808,7 +798,6 @@ build_partition_pathkeys(PlannerInfo *root, RelOptInfo *partrel,
List *
build_expression_pathkey(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
Oid opno,
Relids rel,
bool create_it)
@@ -827,7 +816,6 @@ build_expression_pathkey(PlannerInfo *root,
cpathkey = make_pathkey_from_sortinfo(root,
expr,
- nullable_relids,
opfamily,
opcintype,
exprCollation((Node *) expr),
@@ -908,14 +896,11 @@ convert_subquery_pathkeys(PlannerInfo *root, RelOptInfo *rel,
* expression is *not* volatile in the outer query: it's just
* a Var referencing whatever the subquery emitted. (IOW, the
* outer query isn't going to re-execute the volatile
- * expression itself.) So this is okay. Likewise, it's
- * correct to pass nullable_relids = NULL, because we're
- * underneath any outer joins appearing in the outer query.
+ * expression itself.) So this is okay.
*/
outer_ec =
get_eclass_for_sort_expr(root,
(Expr *) outer_var,
- NULL,
sub_eclass->ec_opfamilies,
sub_member->em_datatype,
sub_eclass->ec_collation,
@@ -997,7 +982,6 @@ convert_subquery_pathkeys(PlannerInfo *root, RelOptInfo *rel,
/* See if we have a matching EC for the TLE */
outer_ec = get_eclass_for_sort_expr(root,
(Expr *) outer_var,
- NULL,
sub_eclass->ec_opfamilies,
sub_expr_type,
sub_expr_coll,
@@ -1138,13 +1122,6 @@ build_join_pathkeys(PlannerInfo *root,
* The resulting PathKeys are always in canonical form. (Actually, there
* is no longer any code anywhere that creates non-canonical PathKeys.)
*
- * We assume that root->nullable_baserels is the set of base relids that could
- * have gone to NULL below the SortGroupClause expressions. This is okay if
- * the expressions came from the query's top level (ORDER BY, DISTINCT, etc)
- * and if this function is only invoked after deconstruct_jointree. In the
- * future we might have to make callers pass in the appropriate
- * nullable-relids set, but for now it seems unnecessary.
- *
* 'sortclauses' is a list of SortGroupClause nodes
* 'tlist' is the targetlist to find the referenced tlist entries in
*/
@@ -1166,7 +1143,6 @@ make_pathkeys_for_sortclauses(PlannerInfo *root,
Assert(OidIsValid(sortcl->sortop));
pathkey = make_pathkey_from_sortop(root,
sortkey,
- root->nullable_baserels,
sortcl->sortop,
sortcl->nulls_first,
sortcl->tleSortGroupRef,
@@ -1222,7 +1198,6 @@ initialize_mergeclause_eclasses(PlannerInfo *root, RestrictInfo *restrictinfo)
restrictinfo->left_ec =
get_eclass_for_sort_expr(root,
(Expr *) get_leftop(clause),
- restrictinfo->nullable_relids,
restrictinfo->mergeopfamilies,
lefttype,
((OpExpr *) clause)->inputcollid,
@@ -1232,7 +1207,6 @@ initialize_mergeclause_eclasses(PlannerInfo *root, RestrictInfo *restrictinfo)
restrictinfo->right_ec =
get_eclass_for_sort_expr(root,
(Expr *) get_rightop(clause),
- restrictinfo->nullable_relids,
restrictinfo->mergeopfamilies,
righttype,
((OpExpr *) clause)->inputcollid,
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 325dae3d45..0432df29fe 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -131,7 +131,7 @@ static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
List **postponed_qual_list,
List **postponed_oj_qual_list);
static bool check_outerjoin_delay(PlannerInfo *root, Relids *relids_p,
- Relids *nullable_relids_p, bool is_pushed_down);
+ bool is_pushed_down);
static bool check_equivalence_delay(PlannerInfo *root,
RestrictInfo *restrictinfo);
static bool check_redundant_nullability_qual(PlannerInfo *root, Node *clause);
@@ -772,7 +772,6 @@ deconstruct_jointree(PlannerInfo *root)
/* These are filled as we scan the jointree */
root->all_baserels = NULL;
root->outer_join_rels = NULL;
- root->nullable_baserels = NULL;
/* Perform the initial scan of the jointree */
result = deconstruct_recurse(root, (Node *) root->parse->jointree,
@@ -928,7 +927,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- Relids nullable_rels;
JoinTreeItem *left_item,
*right_item;
List *leftjoinlist,
@@ -954,8 +952,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
jtitem->right_rels = right_item->qualscope;
/* Inner join adds no restrictions for quals */
jtitem->nonnullable_rels = NULL;
- /* and it doesn't force anything to null, either */
- nullable_rels = NULL;
break;
case JOIN_LEFT:
case JOIN_ANTI:
@@ -986,7 +982,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
jtitem->left_rels = left_item->qualscope;
jtitem->right_rels = right_item->qualscope;
jtitem->nonnullable_rels = left_item->qualscope;
- nullable_rels = right_item->qualscope;
break;
case JOIN_SEMI:
/* Recurse */
@@ -1009,13 +1004,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
jtitem->right_rels = right_item->qualscope;
/* Semi join adds no restrictions for quals */
jtitem->nonnullable_rels = NULL;
-
- /*
- * Theoretically, a semijoin would null the RHS; but since the
- * RHS can't be accessed above the join, this is immaterial
- * and we needn't account for it.
- */
- nullable_rels = NULL;
break;
case JOIN_FULL:
/* Recurse */
@@ -1045,21 +1033,15 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
jtitem->right_rels = right_item->qualscope;
/* each side is both outer and inner */
jtitem->nonnullable_rels = jtitem->qualscope;
- nullable_rels = jtitem->qualscope;
break;
default:
/* JOIN_RIGHT was eliminated during reduce_outer_joins() */
elog(ERROR, "unrecognized join type: %d",
(int) j->jointype);
leftjoinlist = rightjoinlist = NIL; /* keep compiler quiet */
- nullable_rels = NULL;
break;
}
- /* Report all rels that will be nulled anywhere in the jointree */
- root->nullable_baserels = bms_add_members(root->nullable_baserels,
- nullable_rels);
-
/*
* Compute the output joinlist. We fold subproblems together except
* at a FULL JOIN or where join_collapse_limit would be exceeded.
@@ -2210,7 +2192,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool pseudoconstant = false;
bool maybe_equivalence;
bool maybe_outer_join;
- Relids nullable_relids;
RestrictInfo *restrictinfo;
/*
@@ -2364,7 +2345,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
/* Check to see if must be delayed by lower outer join */
outerjoin_delayed = check_outerjoin_delay(root,
&relids,
- &nullable_relids,
false);
/*
@@ -2392,7 +2372,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
/* Check to see if must be delayed by lower outer join */
outerjoin_delayed = check_outerjoin_delay(root,
&relids,
- &nullable_relids,
true);
if (outerjoin_delayed)
@@ -2452,8 +2431,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
pseudoconstant,
security_level,
relids,
- outerjoin_nonnullable,
- nullable_relids);
+ outerjoin_nonnullable);
/* Apply appropriate clone marking, too */
restrictinfo->has_clone = has_clone;
@@ -2611,9 +2589,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* If the qual must be delayed, add relids to *relids_p to reflect the lowest
* safe level for evaluating the qual, and return true. Any extra delay for
* higher-level joins is reflected by setting delay_upper_joins to true in
- * SpecialJoinInfo structs. We also compute nullable_relids, the set of
- * referenced relids that are nullable by lower outer joins (note that this
- * can be nonempty even for a non-delayed qual).
+ * SpecialJoinInfo structs.
*
* For an is_pushed_down qual, we can evaluate the qual as soon as (1) we have
* all the rels it mentions, and (2) we are at or above any outer joins that
@@ -2636,8 +2612,8 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* mentioning only C cannot be applied below the join to A.
*
* For a non-pushed-down qual, this isn't going to determine where we place the
- * qual, but we need to determine outerjoin_delayed and nullable_relids anyway
- * for use later in the planning process.
+ * qual, but we need to determine outerjoin_delayed anyway for use later in
+ * the planning process.
*
* Lastly, a pushed-down qual that references the nullable side of any current
* join_info_list member and has to be evaluated above that OJ (because its
@@ -2655,24 +2631,18 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
static bool
check_outerjoin_delay(PlannerInfo *root,
Relids *relids_p, /* in/out parameter */
- Relids *nullable_relids_p, /* output parameter */
bool is_pushed_down)
{
Relids relids;
- Relids nullable_relids;
bool outerjoin_delayed;
bool found_some;
/* fast path if no special joins */
if (root->join_info_list == NIL)
- {
- *nullable_relids_p = NULL;
return false;
- }
/* must copy relids because we need the original value at the end */
relids = bms_copy(*relids_p);
- nullable_relids = NULL;
outerjoin_delayed = false;
do
{
@@ -2699,12 +2669,6 @@ check_outerjoin_delay(PlannerInfo *root,
/* we'll need another iteration */
found_some = true;
}
- /* track all the nullable rels of relevant OJs */
- nullable_relids = bms_add_members(nullable_relids,
- sjinfo->min_righthand);
- if (sjinfo->jointype == JOIN_FULL)
- nullable_relids = bms_add_members(nullable_relids,
- sjinfo->min_lefthand);
/* set delay_upper_joins if needed */
if (is_pushed_down && sjinfo->jointype != JOIN_FULL &&
bms_overlap(relids, sjinfo->min_lefthand))
@@ -2713,13 +2677,9 @@ check_outerjoin_delay(PlannerInfo *root,
}
} while (found_some);
- /* identify just the actually-referenced nullable rels */
- nullable_relids = bms_int_members(nullable_relids, *relids_p);
-
- /* replace *relids_p, and return nullable_relids */
+ /* replace *relids_p */
bms_free(*relids_p);
*relids_p = relids;
- *nullable_relids_p = nullable_relids;
return outerjoin_delayed;
}
@@ -2741,7 +2701,6 @@ check_equivalence_delay(PlannerInfo *root,
RestrictInfo *restrictinfo)
{
Relids relids;
- Relids nullable_relids;
/* fast path if no special joins */
if (root->join_info_list == NIL)
@@ -2750,12 +2709,12 @@ check_equivalence_delay(PlannerInfo *root,
/* must copy restrictinfo's relids to avoid changing it */
relids = bms_copy(restrictinfo->left_relids);
/* check left side does not need delay */
- if (check_outerjoin_delay(root, &relids, &nullable_relids, true))
+ if (check_outerjoin_delay(root, &relids, true))
return false;
/* and similarly for the right side */
relids = bms_copy(restrictinfo->right_relids);
- if (check_outerjoin_delay(root, &relids, &nullable_relids, true))
+ if (check_outerjoin_delay(root, &relids, true))
return false;
return true;
@@ -2881,11 +2840,6 @@ distribute_restrictinfo_to_rels(PlannerInfo *root,
* variable-free. Otherwise the qual is applied at the lowest join level
* that provides all its variables.
*
- * "nullable_relids" is the set of relids used in the expressions that are
- * potentially nullable below the expressions. (This has to be supplied by
- * caller because this function is used after deconstruct_jointree, so we
- * don't have knowledge of where the clause items came from.)
- *
* "security_level" is the security level to assign to the new restrictinfo.
*
* "both_const" indicates whether both items are known pseudo-constant;
@@ -2911,7 +2865,6 @@ process_implied_equality(PlannerInfo *root,
Expr *item1,
Expr *item2,
Relids qualscope,
- Relids nullable_relids,
Index security_level,
bool below_outer_join,
bool both_const)
@@ -2995,8 +2948,7 @@ process_implied_equality(PlannerInfo *root,
pseudoconstant,
security_level,
relids,
- NULL, /* outer_relids */
- nullable_relids);
+ NULL); /* outer_relids */
/*
* If it's a join clause, add vars used in the clause to targetlists of
@@ -3061,7 +3013,6 @@ build_implied_join_equality(PlannerInfo *root,
Expr *item1,
Expr *item2,
Relids qualscope,
- Relids nullable_relids,
Index security_level)
{
RestrictInfo *restrictinfo;
@@ -3089,8 +3040,7 @@ build_implied_join_equality(PlannerInfo *root,
false, /* pseudoconstant */
security_level, /* security_level */
qualscope, /* required_relids */
- NULL, /* outer_relids */
- nullable_relids); /* nullable_relids */
+ NULL); /* outer_relids */
/* Set mergejoinability/hashjoinability flags */
check_mergejoinable(restrictinfo);
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index e18d64b6dc..662d2c1f17 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -448,9 +448,6 @@ adjust_appendrel_attrs_mutator(Node *node,
newinfo->outer_relids = adjust_child_relids(oldinfo->outer_relids,
context->nappinfos,
context->appinfos);
- newinfo->nullable_relids = adjust_child_relids(oldinfo->nullable_relids,
- context->nappinfos,
- context->appinfos);
newinfo->left_relids = adjust_child_relids(oldinfo->left_relids,
context->nappinfos,
context->appinfos);
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index f51ce45cd3..6814377599 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -869,7 +869,7 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
rinfo->outerjoin_delayed,
pseudoconstant,
rinfo->security_level,
- NULL, NULL, NULL));
+ NULL, NULL));
/* track minimum security level among child quals */
cq_min_security = Min(cq_min_security, rinfo->security_level);
}
@@ -904,7 +904,7 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
make_restrictinfo(root, qual,
true, false, false,
security_level,
- NULL, NULL, NULL));
+ NULL, NULL));
cq_min_security = Min(cq_min_security, security_level);
}
security_level++;
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index e96ef176ad..17ca0073c5 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -275,7 +275,6 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
false,
join_or_rinfo->security_level,
NULL,
- NULL,
NULL);
/*
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index 15f410cf36..1d8912608b 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -29,8 +29,7 @@ static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids);
+ Relids outer_relids);
static Expr *make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
bool is_pushed_down,
@@ -38,8 +37,7 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids);
+ Relids outer_relids);
/*
@@ -49,7 +47,7 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
*
* The is_pushed_down, outerjoin_delayed, and pseudoconstant flags for the
* RestrictInfo must be supplied by the caller, as well as the correct values
- * for security_level, outer_relids, and nullable_relids.
+ * for security_level and outer_relids.
* required_relids can be NULL, in which case it defaults to the actual clause
* contents (i.e., clause_relids).
*
@@ -69,8 +67,7 @@ make_restrictinfo(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids)
+ Relids outer_relids)
{
/*
* If it's an OR clause, build a modified copy with RestrictInfos inserted
@@ -84,8 +81,7 @@ make_restrictinfo(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids);
+ outer_relids);
/* Shouldn't be an AND clause, else AND/OR flattening messed up */
Assert(!is_andclause(clause));
@@ -98,8 +94,7 @@ make_restrictinfo(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids);
+ outer_relids);
}
/*
@@ -116,8 +111,7 @@ make_restrictinfo_internal(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids)
+ Relids outer_relids)
{
RestrictInfo *restrictinfo = makeNode(RestrictInfo);
Relids baserels;
@@ -132,7 +126,6 @@ make_restrictinfo_internal(PlannerInfo *root,
restrictinfo->can_join = false; /* may get set below */
restrictinfo->security_level = security_level;
restrictinfo->outer_relids = outer_relids;
- restrictinfo->nullable_relids = nullable_relids;
/*
* If it's potentially delayable by lower-level security quals, figure out
@@ -260,7 +253,7 @@ make_restrictinfo_internal(PlannerInfo *root,
*
* The same is_pushed_down, outerjoin_delayed, and pseudoconstant flag
* values can be applied to all RestrictInfo nodes in the result. Likewise
- * for security_level, outer_relids, and nullable_relids.
+ * for security_level and outer_relids.
*
* The given required_relids are attached to our top-level output,
* but any OR-clause constituents are allowed to default to just the
@@ -274,8 +267,7 @@ make_sub_restrictinfos(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids)
+ Relids outer_relids)
{
if (is_orclause(clause))
{
@@ -291,8 +283,7 @@ make_sub_restrictinfos(PlannerInfo *root,
pseudoconstant,
security_level,
NULL,
- outer_relids,
- nullable_relids));
+ outer_relids));
return (Expr *) make_restrictinfo_internal(root,
clause,
make_orclause(orlist),
@@ -301,8 +292,7 @@ make_sub_restrictinfos(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids);
+ outer_relids);
}
else if (is_andclause(clause))
{
@@ -318,8 +308,7 @@ make_sub_restrictinfos(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids));
+ outer_relids));
return make_andclause(andlist);
}
else
@@ -331,8 +320,7 @@ make_sub_restrictinfos(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids);
+ outer_relids);
}
/*
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index b36cb41edf..40880a44f0 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -268,14 +268,6 @@ struct PlannerInfo
*/
Relids all_query_rels;
- /*
- * nullable_baserels is a Relids set of base relids that are nullable by
- * some outer join in the jointree; these are rels that are potentially
- * nullable below the WHERE clause, SELECT targetlist, etc. This is
- * computed in deconstruct_jointree.
- */
- Relids nullable_baserels;
-
/*
* join_rel_list is a list of all join-relation RelOptInfos we have
* considered in this planning run. For small problems we just scan the
@@ -1364,7 +1356,6 @@ typedef struct EquivalenceMember
Expr *em_expr; /* the expression represented */
Relids em_relids; /* all relids appearing in em_expr */
- Relids em_nullable_relids; /* nullable by lower outer joins */
bool em_is_const; /* expression is pseudoconstant? */
bool em_is_child; /* derived version for a child relation? */
Oid em_datatype; /* the "nominal type" used by the opfamily */
@@ -2390,9 +2381,7 @@ typedef struct LimitPath
* in parameterized scans, since pushing it into the join's outer side would
* lead to wrong answers.)
*
- * There is also a nullable_relids field, which is the set of rels the clause
- * references that can be forced null by some outer join below the clause.
- *
+ * XXX this comment needs work, if we don't remove it completely:
* outerjoin_delayed = true is subtly different from nullable_relids != NULL:
* a clause might reference some nullable rels and yet not be
* outerjoin_delayed because it also references all the other rels of the
@@ -2506,9 +2495,6 @@ typedef struct RestrictInfo
/* If an outer-join clause, the outer-side relations, else NULL: */
Relids outer_relids;
- /* The relids used in the clause that are nullable by lower outer joins: */
- Relids nullable_relids;
-
/*
* Relids in the left/right side of the clause. These fields are set for
* any binary opclause.
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 41f765d342..03866de136 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -128,7 +128,6 @@ extern Expr *canonicalize_ec_expression(Expr *expr,
extern void reconsider_outer_join_clauses(PlannerInfo *root);
extern EquivalenceClass *get_eclass_for_sort_expr(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
List *opfamilies,
Oid opcintype,
Oid collation,
@@ -216,7 +215,7 @@ extern List *build_index_pathkeys(PlannerInfo *root, IndexOptInfo *index,
extern List *build_partition_pathkeys(PlannerInfo *root, RelOptInfo *partrel,
ScanDirection scandir, bool *partialkeys);
extern List *build_expression_pathkey(PlannerInfo *root, Expr *expr,
- Relids nullable_relids, Oid opno,
+ Oid opno,
Relids rel, bool create_it);
extern List *convert_subquery_pathkeys(PlannerInfo *root, RelOptInfo *rel,
List *subquery_pathkeys,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index 9dffdcfd1e..57b963c0f7 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -83,7 +83,6 @@ extern RestrictInfo *process_implied_equality(PlannerInfo *root,
Expr *item1,
Expr *item2,
Relids qualscope,
- Relids nullable_relids,
Index security_level,
bool below_outer_join,
bool both_const);
@@ -93,7 +92,6 @@ extern RestrictInfo *build_implied_join_equality(PlannerInfo *root,
Expr *item1,
Expr *item2,
Relids qualscope,
- Relids nullable_relids,
Index security_level);
extern void match_foreign_keys_to_quals(PlannerInfo *root);
diff --git a/src/include/optimizer/restrictinfo.h b/src/include/optimizer/restrictinfo.h
index 17d3b4ab05..1f092371ea 100644
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -19,7 +19,7 @@
/* Convenience macro for the common case of a valid-everywhere qual */
#define make_simple_restrictinfo(root, clause) \
- make_restrictinfo(root, clause, true, false, false, 0, NULL, NULL, NULL)
+ make_restrictinfo(root, clause, true, false, false, 0, NULL, NULL)
extern RestrictInfo *make_restrictinfo(PlannerInfo *root,
Expr *clause,
@@ -28,8 +28,7 @@ extern RestrictInfo *make_restrictinfo(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids);
+ Relids outer_relids);
extern RestrictInfo *commute_restrictinfo(RestrictInfo *rinfo, Oid comm_op);
extern bool restriction_is_or_clause(RestrictInfo *restrictinfo);
extern bool restriction_is_securely_promotable(RestrictInfo *restrictinfo,
commit 4a7852a6808132734ea42d7dfb6310e643c044f0
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Fri Dec 23 11:01:25 2022 -0500
Use constant TRUE for "dummy" clauses when throwing back outer joins.
This improves on a hack I introduced in commit 6a6522529. If we
have a left-join clause l.x = r.y, and a WHERE clause l.x = constant,
we generate r.y = constant and then don't really have a need for the
join clause. Currently we throw the join clause back anyway after
marking it redundant, so that the join search heuristics won't think
this is a clauseless join and avoid it. That was a kluge introduced
under time pressure, and after looking at it I thought of a better
way: let's just introduce constant-TRUE "join clauses" instead,
and get rid of them at the end.
This improves the generated plans for such cases by not having to
test a redundant join clause. We can also get rid of the ugly hack
used to mark such clauses as redundant for selectivity estimation.
The code added here should go away someday, if we can ever handle
outer-join clauses as normal eclasses. But the selectivity
simplifications are good cleanup in any case.
diff --git a/src/backend/optimizer/path/clausesel.c b/src/backend/optimizer/path/clausesel.c
index c08eb2b1c5..1cf565ee59 100644
--- a/src/backend/optimizer/path/clausesel.c
+++ b/src/backend/optimizer/path/clausesel.c
@@ -715,12 +715,6 @@ clause_selectivity_ext(PlannerInfo *root,
return (Selectivity) 1.0;
}
- /*
- * If the clause is marked redundant, always return 1.0.
- */
- if (rinfo->norm_selec > 1)
- return (Selectivity) 1.0;
-
/*
* If possible, cache the result of the selectivity calculation for
* the clause. We can cache if varRelid is zero or the clause
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index a7c84daa74..d9b76ecab4 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -1954,14 +1954,11 @@ create_join_clause(PlannerInfo *root,
* If we don't find any match for a set-aside outer join clause, we must
* throw it back into the regular joinclause processing by passing it to
* distribute_restrictinfo_to_rels(). If we do generate a derived clause,
- * however, the outer-join clause is redundant. We still throw it back,
- * because otherwise the join will be seen as a clauseless join and avoided
- * during join order searching; but we mark it as redundant to keep from
- * messing up the joinrel's size estimate. (This behavior means that the
- * API for this routine is uselessly complex: we could have just put all
- * the clauses into the regular processing initially. We keep it because
- * someday we might want to do something else, such as inserting "dummy"
- * joinclauses instead of real ones.)
+ * however, the outer-join clause is redundant. We must still put some
+ * clause into the regular processing, because otherwise the join will be
+ * seen as a clauseless join and avoided during join order searching.
+ * We handle this by generating a constant-TRUE clause that is marked with
+ * required_relids that make it a join between the correct relations.
*
* Outer join clauses that are marked outerjoin_delayed are special: this
* condition means that one or both VARs might go to null due to a lower
@@ -1996,10 +1993,15 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* remove it from the list */
root->left_join_clauses =
foreach_delete_current(root->left_join_clauses, cell);
- /* we throw it back anyway (see notes above) */
- /* but the thrown-back clause has no extra selectivity */
- rinfo->norm_selec = 2.0;
- rinfo->outer_selec = 1.0;
+ /* throw back a dummy replacement clause (see notes above) */
+ rinfo = make_restrictinfo(root,
+ (Expr *) makeBoolConst(true, false),
+ true, /* is_pushed_down */
+ false, /* outerjoin_delayed */
+ false, /* pseudoconstant */
+ 0, /* security_level */
+ rinfo->required_relids,
+ rinfo->outer_relids);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
@@ -2017,10 +2019,15 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* remove it from the list */
root->right_join_clauses =
foreach_delete_current(root->right_join_clauses, cell);
- /* we throw it back anyway (see notes above) */
- /* but the thrown-back clause has no extra selectivity */
- rinfo->norm_selec = 2.0;
- rinfo->outer_selec = 1.0;
+ /* throw back a dummy replacement clause (see notes above) */
+ rinfo = make_restrictinfo(root,
+ (Expr *) makeBoolConst(true, false),
+ true, /* is_pushed_down */
+ false, /* outerjoin_delayed */
+ false, /* pseudoconstant */
+ 0, /* security_level */
+ rinfo->required_relids,
+ rinfo->outer_relids);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
@@ -2038,10 +2045,15 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* remove it from the list */
root->full_join_clauses =
foreach_delete_current(root->full_join_clauses, cell);
- /* we throw it back anyway (see notes above) */
- /* but the thrown-back clause has no extra selectivity */
- rinfo->norm_selec = 2.0;
- rinfo->outer_selec = 1.0;
+ /* throw back a dummy replacement clause (see notes above) */
+ rinfo = make_restrictinfo(root,
+ (Expr *) makeBoolConst(true, false),
+ true, /* is_pushed_down */
+ false, /* outerjoin_delayed */
+ false, /* pseudoconstant */
+ 0, /* security_level */
+ rinfo->required_relids,
+ rinfo->outer_relids);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index 17ca0073c5..7ae058f4c8 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -98,18 +98,13 @@ extract_restriction_or_clauses(PlannerInfo *root)
* joinclause that is considered safe to move to this rel by the
* parameterized-path machinery, even though what we are going to do
* with it is not exactly a parameterized path.
- *
- * However, it seems best to ignore clauses that have been marked
- * redundant (by setting norm_selec > 1). That likely can't happen
- * for OR clauses, but let's be safe.
*/
foreach(lc, rel->joininfo)
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
if (restriction_is_or_clause(rinfo) &&
- join_clause_is_movable_to(rinfo, rel) &&
- rinfo->norm_selec <= 1)
+ join_clause_is_movable_to(rinfo, rel))
{
/* Try to extract a qual for this rel only */
Expr *orclause = extract_or_clause(rinfo, rel);
@@ -355,7 +350,7 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
/* And hack cached selectivity so join size remains the same */
join_or_rinfo->norm_selec = orig_selec / or_selec;
- /* ensure result stays in sane range, in particular not "redundant" */
+ /* ensure result stays in sane range */
if (join_or_rinfo->norm_selec > 1)
join_or_rinfo->norm_selec = 1;
/* as explained above, we don't touch outer_selec */
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index 1d8912608b..c3af845acd 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -424,6 +424,21 @@ restriction_is_securely_promotable(RestrictInfo *restrictinfo,
return false;
}
+/*
+ * Detect whether a RestrictInfo's clause is constant TRUE (note that it's
+ * surely of type boolean). No such WHERE clause could survive qual
+ * canonicalization, but equivclass.c may generate such RestrictInfos for
+ * reasons discussed therein. We should drop them again when creating
+ * the finished plan, which is handled by the next few functions.
+ */
+static inline bool
+rinfo_is_constant_true(RestrictInfo *rinfo)
+{
+ return IsA(rinfo->clause, Const) &&
+ !((Const *) rinfo->clause)->constisnull &&
+ DatumGetBool(((Const *) rinfo->clause)->constvalue);
+}
+
/*
* get_actual_clauses
*
@@ -443,6 +458,7 @@ get_actual_clauses(List *restrictinfo_list)
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
Assert(!rinfo->pseudoconstant);
+ Assert(!rinfo_is_constant_true(rinfo));
result = lappend(result, rinfo->clause);
}
@@ -454,6 +470,7 @@ get_actual_clauses(List *restrictinfo_list)
*
* Extract bare clauses from 'restrictinfo_list', returning either the
* regular ones or the pseudoconstant ones per 'pseudoconstant'.
+ * Constant-TRUE clauses are dropped in any case.
*/
List *
extract_actual_clauses(List *restrictinfo_list,
@@ -466,7 +483,8 @@ extract_actual_clauses(List *restrictinfo_list,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (rinfo->pseudoconstant == pseudoconstant)
+ if (rinfo->pseudoconstant == pseudoconstant &&
+ !rinfo_is_constant_true(rinfo))
result = lappend(result, rinfo->clause);
}
return result;
@@ -477,7 +495,7 @@ extract_actual_clauses(List *restrictinfo_list,
*
* Extract bare clauses from 'restrictinfo_list', separating those that
* semantically match the join level from those that were pushed down.
- * Pseudoconstant clauses are excluded from the results.
+ * Pseudoconstant and constant-TRUE clauses are excluded from the results.
*
* This is only used at outer joins, since for plain joins we don't care
* about pushed-down-ness.
@@ -499,13 +517,15 @@ extract_actual_join_clauses(List *restrictinfo_list,
if (RINFO_IS_PUSHED_DOWN(rinfo, joinrelids))
{
- if (!rinfo->pseudoconstant)
+ if (!rinfo->pseudoconstant &&
+ !rinfo_is_constant_true(rinfo))
*otherquals = lappend(*otherquals, rinfo->clause);
}
else
{
/* joinquals shouldn't have been marked pseudoconstant */
Assert(!rinfo->pseudoconstant);
+ Assert(!rinfo_is_constant_true(rinfo));
*joinquals = lappend(*joinquals, rinfo->clause);
}
}
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 40880a44f0..2a2c5076df 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2540,10 +2540,7 @@ typedef struct RestrictInfo
/* eval cost of clause; -1 if not yet set */
QualCost eval_cost pg_node_attr(equal_ignore);
- /*
- * selectivity for "normal" (JOIN_INNER) semantics; -1 if not yet set; >1
- * means a redundant clause
- */
+ /* selectivity for "normal" (JOIN_INNER) semantics; -1 if not yet set */
Selectivity norm_selec pg_node_attr(equal_ignore);
/* selectivity for outer join semantics; -1 if not yet set */
Selectivity outer_selec pg_node_attr(equal_ignore);
diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out
index c1adb2679a..4185672a12 100644
--- a/src/test/regress/expected/join.out
+++ b/src/test/regress/expected/join.out
@@ -4064,8 +4064,8 @@ explain (costs off)
select a.unique1, b.unique1, c.unique1, coalesce(b.twothousand, a.twothousand)
from tenk1 a left join tenk1 b on b.thousand = a.unique1 left join tenk1 c on c.unique2 =
coalesce(b.twothousand,a.twothousand)
where a.unique2 < 10 and coalesce(b.twothousand, a.twothousand) = 44;
- QUERY PLAN
----------------------------------------------------------------------------------------------
+ QUERY PLAN
+---------------------------------------------------------------
Nested Loop Left Join
-> Nested Loop Left Join
Filter: (COALESCE(b.twothousand, a.twothousand) = 44)
@@ -4076,7 +4076,7 @@ select a.unique1, b.unique1, c.unique1, coalesce(b.twothousand, a.twothousand)
-> Bitmap Index Scan on tenk1_thous_tenthous
Index Cond: (thousand = a.unique1)
-> Index Scan using tenk1_unique2 on tenk1 c
- Index Cond: ((unique2 = COALESCE(b.twothousand, a.twothousand)) AND (unique2 = 44))
+ Index Cond: (unique2 = 44)
(11 rows)
select a.unique1, b.unique1, c.unique1, coalesce(b.twothousand, a.twothousand)
@@ -4507,7 +4507,6 @@ where tt1.f1 = ss1.c0;
Output: tt4.f1
-> Nested Loop Left Join
Output: tt4.f1
- Join Filter: (tt3.f1 = tt4.f1)
-> Seq Scan on public.text_tbl tt3
Output: tt3.f1
Filter: (tt3.f1 = 'foo'::text)
@@ -4525,7 +4524,7 @@ where tt1.f1 = ss1.c0;
Output: (tt4.f1)
-> Seq Scan on public.text_tbl tt5
Output: tt4.f1
-(33 rows)
+(32 rows)
select 1 from
text_tbl as tt1
@@ -4632,24 +4631,22 @@ explain (costs off)
QUERY PLAN
-------------------------------------------------
Nested Loop Left Join
- Join Filter: (a.f1 = b.unique2)
-> Seq Scan on int4_tbl a
Filter: (f1 = 0)
-> Index Scan using tenk1_unique2 on tenk1 b
Index Cond: (unique2 = 0)
-(6 rows)
+(5 rows)
explain (costs off)
select * from tenk1 a full join tenk1 b using(unique2) where unique2 = 42;
QUERY PLAN
-------------------------------------------------
Merge Full Join
- Merge Cond: (a.unique2 = b.unique2)
-> Index Scan using tenk1_unique2 on tenk1 a
Index Cond: (unique2 = 42)
-> Index Scan using tenk1_unique2 on tenk1 b
Index Cond: (unique2 = 42)
-(6 rows)
+(5 rows)
--
-- test that quals attached to an outer join have correct semantics,
commit d8f1b8b5ea948d564dd5225a42c5f97719019327
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Fri Dec 23 11:11:59 2022 -0500
Teach remove_useless_result_rtes to also remove useless FromExprs.
If a FromExpr has but one child, we can get rid of it even if it
has some quals, if we can move those quals up to the parent jointree
node. In particular, this works for a FromExpr just below the RHS
of a LEFT JOIN, because the quals of such a FromExpr are equivalent
to degenerate quals of the left join. By recursively applying this
rule, we can guarantee that the RHS child of a left join is either
not a FromExpr or has more than one child.
The point of this transformation is not so much to simplify the join
tree as it is to eliminate the problem for which commit 11086f2f2
invented the delay_upper_joins mechanism. The idea of that was
to ensure that quals in such a FromExpr would block outer join
rearrangement as if they were degenerate quals of the upper join.
But after this transformation, they *are* degenerate quals of the
upper join, and we don't need an indirect mechanism any more.
(Note that a multi-child FromExpr isn't a problem, because that must
represent an inner join, which can't commute with the outer join
anyway.)
This results in one visible change in regression outputs: in one test
case, there is now an explicit constant-false join filter condition,
which got there by being pushed up from a removed FromExpr. We still
detect that the child plan is dummy, but now that happens because
populate_joinrel_with_paths pushes the knowledge back down after
noticing that the parent left join has a constant-false join
condition. Since it's such a hokey case, I'm not concerned about
the extra qual condition, and see no point in making an effort to
eliminate it.
Having done that, the delay_upper_joins flag serves no purpose any
more and we can remove it, largely reverting 11086f2f2. (The end
game here is to get rid of check_outerjoin_delay altogether, but
first we must get rid of its side effects.) Although this patch
doesn't make any net code savings, I think it's still an improvement
because it replaces a fuzzily-defined action-at-a-distance flag
with a simple and provably correct jointree transformation.
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 92c0644d14..846335043e 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4787,7 +4787,6 @@ compute_semi_anti_join_factors(PlannerInfo *root,
norm_sjinfo.commute_below = NULL;
/* we don't bother trying to make the remaining fields valid */
norm_sjinfo.lhs_strict = false;
- norm_sjinfo.delay_upper_joins = false;
norm_sjinfo.semi_can_btree = false;
norm_sjinfo.semi_can_hash = false;
norm_sjinfo.semi_operators = NIL;
@@ -4956,7 +4955,6 @@ approx_tuple_count(PlannerInfo *root, JoinPath *path, List *quals)
sjinfo.commute_below = NULL;
/* we don't bother trying to make the remaining fields valid */
sjinfo.lhs_strict = false;
- sjinfo.delay_upper_joins = false;
sjinfo.semi_can_btree = false;
sjinfo.semi_can_hash = false;
sjinfo.semi_operators = NIL;
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 605f466bdd..4c0b81a8d0 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -743,7 +743,6 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
sjinfo->commute_below = NULL;
/* we don't bother trying to make the remaining fields valid */
sjinfo->lhs_strict = false;
- sjinfo->delay_upper_joins = false;
sjinfo->semi_can_btree = false;
sjinfo->semi_can_hash = false;
sjinfo->semi_operators = NIL;
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index 79fd240cf3..3174d77554 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -170,11 +170,10 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
int attroff;
/*
- * Must be a non-delaying left join to a single baserel, else we aren't
- * going to be able to do anything with it.
+ * Must be a left join to a single baserel, else we aren't going to be
+ * able to do anything with it.
*/
- if (sjinfo->jointype != JOIN_LEFT ||
- sjinfo->delay_upper_joins)
+ if (sjinfo->jointype != JOIN_LEFT)
return false;
if (!bms_get_singleton_member(sjinfo->min_righthand, &innerrelid))
@@ -570,13 +569,10 @@ reduce_unique_semijoins(PlannerInfo *root)
List *restrictlist;
/*
- * Must be a non-delaying semijoin to a single baserel, else we aren't
- * going to be able to do anything with it. (It's probably not
- * possible for delay_upper_joins to be set on a semijoin, but we
- * might as well check.)
+ * Must be a semijoin to a single baserel, else we aren't going to be
+ * able to do anything with it.
*/
- if (sjinfo->jointype != JOIN_SEMI ||
- sjinfo->delay_upper_joins)
+ if (sjinfo->jointype != JOIN_SEMI)
continue;
if (!bms_get_singleton_member(sjinfo->min_righthand, &innerrelid))
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 0432df29fe..c9b9cc3f74 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -130,8 +130,7 @@ static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool is_clone,
List **postponed_qual_list,
List **postponed_oj_qual_list);
-static bool check_outerjoin_delay(PlannerInfo *root, Relids *relids_p,
- bool is_pushed_down);
+static bool check_outerjoin_delay(PlannerInfo *root, Relids *relids_p);
static bool check_equivalence_delay(PlannerInfo *root,
RestrictInfo *restrictinfo);
static bool check_redundant_nullability_qual(PlannerInfo *root, Node *clause);
@@ -1439,8 +1438,6 @@ make_outerjoininfo(PlannerInfo *root,
sjinfo->commute_above_l = NULL;
sjinfo->commute_above_r = NULL;
sjinfo->commute_below = NULL;
- /* this always starts out false */
- sjinfo->delay_upper_joins = false;
compute_semijoin_info(root, sjinfo, clause);
@@ -1595,17 +1592,6 @@ make_outerjoininfo(PlannerInfo *root,
* Also, we must preserve ordering anyway if we have unsafe PHVs, or
* if either this join or the lower OJ is a semijoin or antijoin.
*
- * Here, we have to consider that "our join condition" includes any
- * clauses that syntactically appeared above the lower OJ and below
- * ours; those are equivalent to degenerate clauses in our OJ and must
- * be treated as such. Such clauses obviously can't reference our
- * LHS, and they must be non-strict for the lower OJ's RHS (else
- * reduce_outer_joins would have reduced the lower OJ to a plain
- * join). Hence the other ways in which we handle clauses within our
- * join condition are not affected by them. The net effect is
- * therefore sufficiently represented by the delay_upper_joins flag
- * saved for us by check_outerjoin_delay.
- *
* When we don't need to preserve ordering, check to see if outer join
* identity 3 applies, and if so, remove the lower OJ's ojrelid from
* our min_righthand so that commutation is allowed.
@@ -1619,7 +1605,7 @@ make_outerjoininfo(PlannerInfo *root,
jointype == JOIN_ANTI ||
otherinfo->jointype == JOIN_SEMI ||
otherinfo->jointype == JOIN_ANTI ||
- !otherinfo->lhs_strict || otherinfo->delay_upper_joins)
+ !otherinfo->lhs_strict)
{
/* Preserve ordering */
min_righthand = bms_add_members(min_righthand,
@@ -2344,8 +2330,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
/* Check to see if must be delayed by lower outer join */
outerjoin_delayed = check_outerjoin_delay(root,
- &relids,
- false);
+ &relids);
/*
* Now force the qual to be evaluated exactly at the level of joining
@@ -2371,8 +2356,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
/* Check to see if must be delayed by lower outer join */
outerjoin_delayed = check_outerjoin_delay(root,
- &relids,
- true);
+ &relids);
if (outerjoin_delayed)
{
@@ -2583,13 +2567,10 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
/*
* check_outerjoin_delay
* Detect whether a qual referencing the given relids must be delayed
- * in application due to the presence of a lower outer join, and/or
- * may force extra delay of higher-level outer joins.
+ * in application due to the presence of a lower outer join.
*
* If the qual must be delayed, add relids to *relids_p to reflect the lowest
- * safe level for evaluating the qual, and return true. Any extra delay for
- * higher-level joins is reflected by setting delay_upper_joins to true in
- * SpecialJoinInfo structs.
+ * safe level for evaluating the qual, and return true.
*
* For an is_pushed_down qual, we can evaluate the qual as soon as (1) we have
* all the rels it mentions, and (2) we are at or above any outer joins that
@@ -2614,24 +2595,10 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* For a non-pushed-down qual, this isn't going to determine where we place the
* qual, but we need to determine outerjoin_delayed anyway for use later in
* the planning process.
- *
- * Lastly, a pushed-down qual that references the nullable side of any current
- * join_info_list member and has to be evaluated above that OJ (because its
- * required relids overlap the LHS too) causes that OJ's delay_upper_joins
- * flag to be set true. This will prevent any higher-level OJs from
- * being interchanged with that OJ, which would result in not having any
- * correct place to evaluate the qual. (The case we care about here is a
- * sub-select WHERE clause within the RHS of some outer join. The WHERE
- * clause must effectively be treated as a degenerate clause of that outer
- * join's condition. Rather than trying to match such clauses with joins
- * directly, we set delay_upper_joins here, and when the upper outer join
- * is processed by make_outerjoininfo, it will refrain from allowing the
- * two OJs to commute.)
*/
static bool
check_outerjoin_delay(PlannerInfo *root,
- Relids *relids_p, /* in/out parameter */
- bool is_pushed_down)
+ Relids *relids_p) /* in/out parameter */
{
Relids relids;
bool outerjoin_delayed;
@@ -2669,10 +2636,6 @@ check_outerjoin_delay(PlannerInfo *root,
/* we'll need another iteration */
found_some = true;
}
- /* set delay_upper_joins if needed */
- if (is_pushed_down && sjinfo->jointype != JOIN_FULL &&
- bms_overlap(relids, sjinfo->min_lefthand))
- sjinfo->delay_upper_joins = true;
}
}
} while (found_some);
@@ -2709,12 +2672,12 @@ check_equivalence_delay(PlannerInfo *root,
/* must copy restrictinfo's relids to avoid changing it */
relids = bms_copy(restrictinfo->left_relids);
/* check left side does not need delay */
- if (check_outerjoin_delay(root, &relids, true))
+ if (check_outerjoin_delay(root, &relids))
return false;
/* and similarly for the right side */
relids = bms_copy(restrictinfo->right_relids);
- if (check_outerjoin_delay(root, &relids, true))
+ if (check_outerjoin_delay(root, &relids))
return false;
return true;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index a139d94866..00b02d4e91 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -1043,10 +1043,11 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
/*
* If we have any RTE_RESULT relations, see if they can be deleted from
- * the jointree. This step is most effectively done after we've done
- * expression preprocessing and outer join reduction.
+ * the jointree. We also rely on this processing to flatten single-child
+ * FromExprs underneath outer joins. This step is most effectively done
+ * after we've done expression preprocessing and outer join reduction.
*/
- if (hasResultRTEs)
+ if (hasResultRTEs || hasOuterJoins)
remove_useless_result_rtes(root);
/*
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 356d81bfea..50018d0e85 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -130,6 +130,7 @@ static void reduce_outer_joins_pass2(Node *jtnode,
static void report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
int rtindex, Relids relids);
static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Node **parent_quals,
Relids *dropped_outer_joins);
static int get_result_relid(PlannerInfo *root, Node *jtnode);
static void remove_result_refs(PlannerInfo *root, int varno, Node *newjtloc);
@@ -3083,12 +3084,31 @@ report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
/*
* remove_useless_result_rtes
* Attempt to remove RTE_RESULT RTEs from the join tree.
+ * Also, elide single-child FromExprs where possible.
*
* We can remove RTE_RESULT entries from the join tree using the knowledge
* that RTE_RESULT returns exactly one row and has no output columns. Hence,
* if one is inner-joined to anything else, we can delete it. Optimizations
* are also possible for some outer-join cases, as detailed below.
*
+ * This pass also replaces single-child FromExprs with their child node
+ * where possible. It's appropriate to do that here and not earlier because
+ * RTE_RESULT removal might reduce a multiple-child FromExpr to have only one
+ * child. We can remove such a FromExpr if its quals are empty, or if it's
+ * semantically valid to merge the quals into those of the parent node.
+ * While removing unnecessary join tree nodes has some micro-efficiency value,
+ * the real reason to do this is to eliminate cases where the nullable side of
+ * an outer join node is a FromExpr whose single child is another outer join.
+ * To correctly determine whether the two outer joins can commute,
+ * deconstruct_jointree() must treat any quals of such a FromExpr as being
+ * degenerate quals of the upper outer join. The best way to do that is to
+ * make them actually *be* quals of the upper join, by dropping the FromExpr
+ * and hoisting the quals up into the upper join's quals. (Note that there is
+ * no hazard when the intermediate FromExpr has multiple children, since then
+ * it represents an inner join that cannot commute with the upper outer join.)
+ * As long as we have to do that, we might as well elide such FromExprs
+ * everywhere.
+ *
* Some of these optimizations depend on recognizing empty (constant-true)
* quals for FromExprs and JoinExprs. That makes it useful to apply this
* optimization pass after expression preprocessing, since that will have
@@ -3129,6 +3149,7 @@ remove_useless_result_rtes(PlannerInfo *root)
root->parse->jointree = (FromExpr *)
remove_useless_results_recurse(root,
(Node *) root->parse->jointree,
+ NULL,
&dropped_outer_joins);
/* We should still have a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
@@ -3182,9 +3203,14 @@ remove_useless_result_rtes(PlannerInfo *root)
* This recursively processes the jointree and returns a modified jointree.
* In addition, the RT indexes of any removed outer-join nodes are added to
* *dropped_outer_joins.
+ *
+ * jtnode is the current jointree node. If it could be valid to merge
+ * its quals into those of the parent node, parent_quals should point to
+ * the parent's quals list; otherwise, pass NULL for parent_quals.
*/
static Node *
remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Node **parent_quals,
Relids *dropped_outer_joins)
{
Assert(jtnode != NULL);
@@ -3212,8 +3238,9 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
Node *child = (Node *) lfirst(cell);
int varno;
- /* Recursively transform child ... */
+ /* Recursively transform child, allowing it to push up quals ... */
child = remove_useless_results_recurse(root, child,
+ &f->quals,
dropped_outer_joins);
/* ... and stick it back into the tree */
lfirst(cell) = child;
@@ -3247,25 +3274,54 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
}
/*
- * If we're not at the top of the jointree, it's valid to simplify a
- * degenerate FromExpr into its single child. (At the top, we must
- * keep the FromExpr since Query.jointree is required to point to a
- * FromExpr.)
+ * If the FromExpr now has only one child, see if we can elide it.
+ * This is always valid if there are no quals, except at the top of
+ * the jointree (since Query.jointree is required to point to a
+ * FromExpr). Otherwise, we can do it if we can push the quals up to
+ * the parent node.
+ *
+ * Note: while it would not be terribly hard to generalize this
+ * transformation to merge multi-child FromExprs into their parent
+ * FromExpr, that risks making the parent join too expensive to plan.
+ * We leave it to later processing to decide heuristically whether
+ * that's a good idea. Pulling up a single child is always OK,
+ * however.
*/
- if (f != root->parse->jointree &&
- f->quals == NULL &&
- list_length(f->fromlist) == 1)
+ if (list_length(f->fromlist) == 1 &&
+ f != root->parse->jointree &&
+ (f->quals == NULL || parent_quals != NULL))
+ {
+ /*
+ * Merge any quals up to parent. They should be in implicit-AND
+ * format by now, so we just need to concatenate lists. Put the
+ * child quals at the front, on the grounds that they should
+ * nominally be evaluated earlier.
+ */
+ if (f->quals != NULL)
+ *parent_quals = (Node *)
+ list_concat(castNode(List, f->quals),
+ castNode(List, *parent_quals));
return (Node *) linitial(f->fromlist);
+ }
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
int varno;
- /* First, recurse */
+ /*
+ * First, recurse. We can accept pushed-up FromExpr quals from either
+ * child if the jointype is INNER, and we can accept them from the RHS
+ * child if the jointype is LEFT.
+ */
j->larg = remove_useless_results_recurse(root, j->larg,
+ (j->jointype == JOIN_INNER) ?
+ &j->quals : NULL,
dropped_outer_joins);
j->rarg = remove_useless_results_recurse(root, j->rarg,
+ (j->jointype == JOIN_INNER ||
+ j->jointype == JOIN_LEFT) ?
+ &j->quals : NULL,
dropped_outer_joins);
/* Apply join-type-specific optimization rules */
@@ -3276,9 +3332,9 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
/*
* An inner join is equivalent to a FromExpr, so if either
* side was simplified to an RTE_RESULT rel, we can replace
- * the join with a FromExpr with just the other side; and if
- * the qual is empty (JOIN ON TRUE) then we can omit the
- * FromExpr as well.
+ * the join with a FromExpr with just the other side.
+ * Furthermore, we can elide that FromExpr according to the
+ * same rules as above.
*
* Just as in the FromExpr case, we can't simplify if the
* other input rel references any PHVs that are marked as to
@@ -3293,20 +3349,34 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
!find_dependent_phvs_in_jointree(root, j->rarg, varno))
{
remove_result_refs(root, varno, j->rarg);
- if (j->quals)
+ if (j->quals != NULL && parent_quals == NULL)
jtnode = (Node *)
makeFromExpr(list_make1(j->rarg), j->quals);
else
+ {
+ /* Merge any quals up to parent */
+ if (j->quals != NULL)
+ *parent_quals = (Node *)
+ list_concat(castNode(List, j->quals),
+ castNode(List, *parent_quals));
jtnode = j->rarg;
+ }
}
else if ((varno = get_result_relid(root, j->rarg)) != 0)
{
remove_result_refs(root, varno, j->larg);
- if (j->quals)
+ if (j->quals != NULL && parent_quals == NULL)
jtnode = (Node *)
makeFromExpr(list_make1(j->larg), j->quals);
else
+ {
+ /* Merge any quals up to parent */
+ if (j->quals != NULL)
+ *parent_quals = (Node *)
+ list_concat(castNode(List, j->quals),
+ castNode(List, *parent_quals));
jtnode = j->larg;
+ }
}
break;
case JOIN_LEFT:
@@ -3344,8 +3414,9 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
/*
* We may simplify this case if the RHS is an RTE_RESULT; the
* join qual becomes effectively just a filter qual for the
- * LHS, since we should either return the LHS row or not. For
- * simplicity we inject the filter qual into a new FromExpr.
+ * LHS, since we should either return the LHS row or not. The
+ * filter clause must go into a new FromExpr if we can't push
+ * it up to the parent.
*
* There is a fine point about PHVs that are supposed to be
* evaluated at the RHS. Such PHVs could only appear in the
@@ -3363,11 +3434,18 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
{
Assert(j->rtindex == 0);
remove_result_refs(root, varno, j->larg);
- if (j->quals)
+ if (j->quals != NULL && parent_quals == NULL)
jtnode = (Node *)
makeFromExpr(list_make1(j->larg), j->quals);
else
+ {
+ /* Merge any quals up to parent */
+ if (j->quals != NULL)
+ *parent_quals = (Node *)
+ list_concat(castNode(List, j->quals),
+ castNode(List, *parent_quals));
jtnode = j->larg;
+ }
}
break;
case JOIN_FULL:
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index 7ae058f4c8..e02dabe2ee 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -338,7 +338,6 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
sjinfo.commute_below = NULL;
/* we don't bother trying to make the remaining fields valid */
sjinfo.lhs_strict = false;
- sjinfo.delay_upper_joins = false;
sjinfo.semi_can_btree = false;
sjinfo.semi_can_hash = false;
sjinfo.semi_operators = NIL;
diff --git a/src/backend/optimizer/util/placeholder.c b/src/backend/optimizer/util/placeholder.c
index b9cc983df7..25d4c54400 100644
--- a/src/backend/optimizer/util/placeholder.c
+++ b/src/backend/optimizer/util/placeholder.c
@@ -331,10 +331,7 @@ update_placeholder_eval_levels(PlannerInfo *root, SpecialJoinInfo *new_sjinfo)
/*
* Check for delays due to lower outer joins. This is the same logic
- * as in check_outerjoin_delay in initsplan.c, except that we don't
- * have anything to do with the delay_upper_joins flags; delay of
- * upper outer joins will be handled later, based on the eval_at
- * values we compute now.
+ * as in check_outerjoin_delay in initsplan.c.
*/
eval_at = phinfo->ph_eval_at;
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 2a2c5076df..6a6fbf3fd4 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2746,12 +2746,6 @@ typedef struct PlaceHolderVar
* upper-level outer joins even if it appears in their RHS). We don't bother
* to set lhs_strict for FULL JOINs, however.
*
- * delay_upper_joins is set true if we detect a pushed-down clause that has
- * to be evaluated after this join is formed (because it references the RHS).
- * Any outer joins that have such a clause and this join in their RHS cannot
- * commute with this join, because that would leave noplace to check the
- * pushed-down clause. (We don't track this for FULL JOINs, either.)
- *
* For a semijoin, we also extract the join operators and their RHS arguments
* and set semi_operators, semi_rhs_exprs, semi_can_btree, and semi_can_hash.
* This is done in support of possibly unique-ifying the RHS, so we don't
@@ -2766,8 +2760,8 @@ typedef struct PlaceHolderVar
* not allowed within join_info_list. We also create transient
* SpecialJoinInfos with jointype == JOIN_INNER for outer joins, since for
* cost estimation purposes it is sometimes useful to know the join size under
- * plain innerjoin semantics. Note that lhs_strict, delay_upper_joins, and
- * of course the semi_xxx fields are not set meaningfully within such structs.
+ * plain innerjoin semantics. Note that lhs_strict and the semi_xxx fields
+ * are not set meaningfully within such structs.
*/
#ifndef HAVE_SPECIALJOININFO_TYPEDEF
typedef struct SpecialJoinInfo SpecialJoinInfo;
@@ -2789,7 +2783,6 @@ struct SpecialJoinInfo
Relids commute_above_r; /* commuting OJs above this one, if RHS */
Relids commute_below; /* commuting OJs below this one */
bool lhs_strict; /* joinclause is strict for some LHS rel */
- bool delay_upper_joins; /* can't commute with upper RHS */
/* Remaining fields are set only for JOIN_SEMI jointype: */
bool semi_can_btree; /* true if semi_operators are all btree */
bool semi_can_hash; /* true if semi_operators are all hash */
diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out
index 4185672a12..a5c09cce01 100644
--- a/src/test/regress/expected/join.out
+++ b/src/test/regress/expected/join.out
@@ -6038,12 +6038,13 @@ select * from int8_tbl i8 left join lateral
--------------------------------------
Nested Loop Left Join
Output: i8.q1, i8.q2, f1, (i8.q2)
+ Join Filter: false
-> Seq Scan on public.int8_tbl i8
Output: i8.q1, i8.q2
-> Result
Output: f1, i8.q2
One-Time Filter: false
-(7 rows)
+(8 rows)
explain (verbose, costs off)
select * from int8_tbl i8 left join lateral
commit d604b10cd3e6e044a551e6a18b0a520d35916e68
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Fri Dec 23 11:28:14 2022 -0500
Remove the outerjoin_delayed mechanism.
We needed this before to prevent quals from getting evaluated below
outer joins that should null some of their vars. Now that we consider
varnullingrels while placing quals, that's taken care of
automatically, so throw the whole thing away.
This results in one cosmetic change in the regression test outputs,
where clauses that were not previously considered to be
EquivalenceClass candidates are now treated as ECs, and the EC
machinery chooses to emit an equivalent but not identical set of
clauses.
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 12e940a2e6..47ce73ff57 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -6305,7 +6305,6 @@ foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
expr,
true,
false,
- false,
root->qual_security_level,
grouped_rel->relids,
NULL);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index d9b76ecab4..bcf7fcf21c 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -200,7 +200,6 @@ process_equivalence(PlannerInfo *root,
make_restrictinfo(root,
(Expr *) ntest,
restrictinfo->is_pushed_down,
- restrictinfo->outerjoin_delayed,
restrictinfo->pseudoconstant,
restrictinfo->security_level,
NULL,
@@ -1144,11 +1143,8 @@ generate_base_implied_equalities_const(PlannerInfo *root,
{
RestrictInfo *restrictinfo = (RestrictInfo *) linitial(ec->ec_sources);
- if (bms_membership(restrictinfo->required_relids) != BMS_MULTIPLE)
- {
- distribute_restrictinfo_to_rels(root, restrictinfo);
- return;
- }
+ distribute_restrictinfo_to_rels(root, restrictinfo);
+ return;
}
/*
@@ -1959,15 +1955,6 @@ create_join_clause(PlannerInfo *root,
* seen as a clauseless join and avoided during join order searching.
* We handle this by generating a constant-TRUE clause that is marked with
* required_relids that make it a join between the correct relations.
- *
- * Outer join clauses that are marked outerjoin_delayed are special: this
- * condition means that one or both VARs might go to null due to a lower
- * outer join. We can still push a constant through the clause, but only
- * if its operator is strict; and we *have to* throw the clause back into
- * regular joinclause processing. By keeping the strict join clause,
- * we ensure that any null-extended rows that are mistakenly generated due
- * to suppressing rows not matching the constant will be rejected at the
- * upper outer join. (This doesn't work for full-join clauses.)
*/
void
reconsider_outer_join_clauses(PlannerInfo *root)
@@ -1997,7 +1984,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
true, /* is_pushed_down */
- false, /* outerjoin_delayed */
false, /* pseudoconstant */
0, /* security_level */
rinfo->required_relids,
@@ -2023,7 +2009,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
true, /* is_pushed_down */
- false, /* outerjoin_delayed */
false, /* pseudoconstant */
0, /* security_level */
rinfo->required_relids,
@@ -2049,7 +2034,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
true, /* is_pushed_down */
- false, /* outerjoin_delayed */
false, /* pseudoconstant */
0, /* security_level */
rinfo->required_relids,
@@ -2104,10 +2088,6 @@ reconsider_outer_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo,
opno = ((OpExpr *) rinfo->clause)->opno;
collation = ((OpExpr *) rinfo->clause)->inputcollid;
- /* If clause is outerjoin_delayed, operator must be strict */
- if (rinfo->outerjoin_delayed && !op_strict(opno))
- return false;
-
/* Extract needed info from the clause */
op_input_types(opno, &left_type, &right_type);
if (outer_on_left)
@@ -2224,10 +2204,6 @@ reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
right_relids;
ListCell *lc1;
- /* Can't use an outerjoin_delayed clause here */
- if (rinfo->outerjoin_delayed)
- return false;
-
/* Extract needed info from the clause */
Assert(is_opclause(rinfo->clause));
opno = ((OpExpr *) rinfo->clause)->opno;
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index c9b9cc3f74..6fe00652b6 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -130,9 +130,6 @@ static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool is_clone,
List **postponed_qual_list,
List **postponed_oj_qual_list);
-static bool check_outerjoin_delay(PlannerInfo *root, Relids *relids_p);
-static bool check_equivalence_delay(PlannerInfo *root,
- RestrictInfo *restrictinfo);
static bool check_redundant_nullability_qual(PlannerInfo *root, Node *clause);
static void check_mergejoinable(RestrictInfo *restrictinfo);
static void check_hashjoinable(RestrictInfo *restrictinfo);
@@ -738,15 +735,6 @@ create_lateral_join_info(PlannerInfo *root)
* A sub-joinlist represents a subproblem to be planned separately. Currently
* sub-joinlists arise only from FULL OUTER JOIN or when collapsing of
* subproblems is stopped by join_collapse_limit or from_collapse_limit.
- *
- * NOTE: when dealing with inner joins, it is appropriate to let a qual clause
- * be evaluated at the lowest level where all the variables it mentions are
- * available. However, we cannot push a qual down into the nullable side(s)
- * of an outer join since the qual might eliminate matching rows and cause a
- * NULL row to be incorrectly emitted by the join. Therefore, we artificially
- * OR the minimum-relids of such an outer join into the required_relids of
- * clauses appearing above it. This forces those clauses to be delayed until
- * application of the outer join (or maybe even higher in the join tree).
*/
List *
deconstruct_jointree(PlannerInfo *root)
@@ -758,9 +746,8 @@ deconstruct_jointree(PlannerInfo *root)
/*
* After this point, no more PlaceHolderInfos may be made, because
- * make_outerjoininfo and update_placeholder_eval_levels require all
- * active placeholders to be present in root->placeholder_list while we
- * crawl up the join tree.
+ * make_outerjoininfo requires all active placeholders to be present in
+ * root->placeholder_list while we crawl up the join tree.
*/
root->placeholdersFrozen = true;
@@ -798,31 +785,12 @@ deconstruct_jointree(PlannerInfo *root)
*/
if (root->join_info_list)
{
- /*
- * XXX hack: when we call distribute_qual_to_rels to process one of
- * these clauses, neither the owning SpecialJoinInfo nor any later
- * ones can appear in root->join_info_list, else the wrong things will
- * happen. Fake it out by emptying join_info_list and rebuilding it
- * as we go. This works because join_info_list is only appended to
- * during deconstruct_distribute, so we know we are examining
- * SpecialJoinInfos bottom-up, just like the first time. We can get
- * rid of this hack later, after fixing things so that
- * distribute_qual_to_rels doesn't have that requirement about
- * join_info_list.
- */
- root->join_info_list = NIL;
-
foreach(lc, item_list)
{
JoinTreeItem *jtitem = (JoinTreeItem *) lfirst(lc);
if (jtitem->oj_joinclauses != NIL)
deconstruct_distribute_oj_quals(root, item_list, jtitem);
-
- /* XXX Rest of hack: rebuild join_info_list as we go */
- if (jtitem->sjinfo)
- root->join_info_list = lappend(root->join_info_list,
- jtitem->sjinfo);
}
}
@@ -1265,11 +1233,7 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
/* And add the SpecialJoinInfo to join_info_list */
if (sjinfo)
- {
root->join_info_list = lappend(root->join_info_list, sjinfo);
- /* Each time we do that, recheck placeholder eval levels */
- update_placeholder_eval_levels(root, sjinfo);
- }
}
else
{
@@ -2155,8 +2119,8 @@ distribute_quals_to_rels(PlannerInfo *root, List *clauses,
* level, which will be ojscope not necessarily qualscope.
*
* At the time this is called, root->join_info_list must contain entries for
- * all and only those special joins that are syntactically below this qual;
- * in particular, the passed-in SpecialJoinInfo isn't yet in that list.
+ * at least those special joins that are syntactically below this qual.
+ * (We now need that only for detection of redundant IS NULL quals.)
*/
static void
distribute_qual_to_rels(PlannerInfo *root, Node *clause,
@@ -2174,7 +2138,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
{
Relids relids;
bool is_pushed_down;
- bool outerjoin_delayed;
bool pseudoconstant = false;
bool maybe_equivalence;
bool maybe_outer_join;
@@ -2328,19 +2291,12 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
maybe_equivalence = false;
maybe_outer_join = true;
- /* Check to see if must be delayed by lower outer join */
- outerjoin_delayed = check_outerjoin_delay(root,
- &relids);
-
/*
* Now force the qual to be evaluated exactly at the level of joining
* corresponding to the outer join. We cannot let it get pushed down
* into the nonnullable side, since then we'd produce no output rows,
* rather than the intended single null-extended row, for any
* nonnullable-side rows failing the qual.
- *
- * (Do this step after calling check_outerjoin_delay, because that
- * trashes relids.)
*/
Assert(ojscope);
relids = ojscope;
@@ -2354,32 +2310,16 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
*/
is_pushed_down = true;
- /* Check to see if must be delayed by lower outer join */
- outerjoin_delayed = check_outerjoin_delay(root,
- &relids);
-
- if (outerjoin_delayed)
- {
- /* Should still be a subset of current scope ... */
- Assert(root->hasLateralRTEs || bms_is_subset(relids, qualscope));
- Assert(ojscope == NULL || bms_is_subset(relids, ojscope));
-
- /*
- * Because application of the qual will be delayed by outer join,
- * we mustn't assume its vars are equal everywhere.
- */
- maybe_equivalence = false;
+ /*
+ * It's possible that this is an IS NULL clause that's redundant with
+ * a lower antijoin; if so we can just discard it. We need not test
+ * in any of the other cases, because this will only be possible for
+ * pushed-down clauses.
+ */
+ if (check_redundant_nullability_qual(root, clause))
+ return;
- /*
- * It's possible that this is an IS NULL clause that's redundant
- * with a lower antijoin; if so we can just discard it. We need
- * not test in any of the other cases, because this will only be
- * possible for pushed-down, delayed clauses.
- */
- if (check_redundant_nullability_qual(root, clause))
- return;
- }
- else if (!allow_equivalence)
+ if (!allow_equivalence)
{
/* Caller says it mustn't become an equivalence class */
maybe_equivalence = false;
@@ -2387,8 +2327,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
else
{
/*
- * Qual is not delayed by any lower outer-join restriction, so we
- * can consider feeding it to the equivalence machinery. However,
+ * Consider feeding qual to the equivalence machinery. However,
* if it's itself within an outer-join clause, treat it as though
* it appeared below that outer join (note that we can only get
* here when the clause references only nullable-side rels).
@@ -2411,7 +2350,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
restrictinfo = make_restrictinfo(root,
(Expr *) clause,
is_pushed_down,
- outerjoin_delayed,
pseudoconstant,
security_level,
relids,
@@ -2463,6 +2401,8 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
check_mergejoinable(restrictinfo);
/*
+ * XXX rewrite:
+ *
* If it is a true equivalence clause, send it to the EquivalenceClass
* machinery. We do *not* attach it directly to any restriction or join
* lists. The EC code will propagate it to the appropriate places later.
@@ -2498,8 +2438,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
{
if (maybe_equivalence)
{
- if (check_equivalence_delay(root, restrictinfo) &&
- process_equivalence(root, &restrictinfo, below_outer_join))
+ if (process_equivalence(root, &restrictinfo, below_outer_join))
return;
/* EC rejected it, so set left_ec/right_ec the hard way ... */
if (restrictinfo->mergeopfamilies) /* EC might have changed this */
@@ -2564,125 +2503,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
distribute_restrictinfo_to_rels(root, restrictinfo);
}
-/*
- * check_outerjoin_delay
- * Detect whether a qual referencing the given relids must be delayed
- * in application due to the presence of a lower outer join.
- *
- * If the qual must be delayed, add relids to *relids_p to reflect the lowest
- * safe level for evaluating the qual, and return true.
- *
- * For an is_pushed_down qual, we can evaluate the qual as soon as (1) we have
- * all the rels it mentions, and (2) we are at or above any outer joins that
- * can null any of these rels and are below the syntactic location of the
- * given qual. We must enforce (2) because pushing down such a clause below
- * the OJ might cause the OJ to emit null-extended rows that should not have
- * been formed, or that should have been rejected by the clause. (This is
- * only an issue for non-strict quals, since if we can prove a qual mentioning
- * only nullable rels is strict, we'd have reduced the outer join to an inner
- * join in reduce_outer_joins().)
- *
- * To enforce (2), scan the join_info_list and merge the required-relid sets of
- * any such OJs into the clause's own reference list. At the time we are
- * called, the join_info_list contains only outer joins below this qual. We
- * have to repeat the scan until no new relids get added; this ensures that
- * the qual is suitably delayed regardless of the order in which OJs get
- * executed. As an example, if we have one OJ with LHS=A, RHS=B, and one with
- * LHS=B, RHS=C, it is implied that these can be done in either order; if the
- * B/C join is done first then the join to A can null C, so a qual actually
- * mentioning only C cannot be applied below the join to A.
- *
- * For a non-pushed-down qual, this isn't going to determine where we place the
- * qual, but we need to determine outerjoin_delayed anyway for use later in
- * the planning process.
- */
-static bool
-check_outerjoin_delay(PlannerInfo *root,
- Relids *relids_p) /* in/out parameter */
-{
- Relids relids;
- bool outerjoin_delayed;
- bool found_some;
-
- /* fast path if no special joins */
- if (root->join_info_list == NIL)
- return false;
-
- /* must copy relids because we need the original value at the end */
- relids = bms_copy(*relids_p);
- outerjoin_delayed = false;
- do
- {
- ListCell *l;
-
- found_some = false;
- foreach(l, root->join_info_list)
- {
- SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(l);
-
- /* do we reference any nullable rels of this OJ? */
- if (bms_overlap(relids, sjinfo->min_righthand) ||
- (sjinfo->jointype == JOIN_FULL &&
- bms_overlap(relids, sjinfo->min_lefthand)))
- {
- /* yes; have we included all its rels in relids? */
- if (!bms_is_subset(sjinfo->min_lefthand, relids) ||
- !bms_is_subset(sjinfo->min_righthand, relids))
- {
- /* no, so add them in */
- relids = bms_add_members(relids, sjinfo->min_lefthand);
- relids = bms_add_members(relids, sjinfo->min_righthand);
- outerjoin_delayed = true;
- /* we'll need another iteration */
- found_some = true;
- }
- }
- }
- } while (found_some);
-
- /* replace *relids_p */
- bms_free(*relids_p);
- *relids_p = relids;
- return outerjoin_delayed;
-}
-
-/*
- * check_equivalence_delay
- * Detect whether a potential equivalence clause is rendered unsafe
- * by outer-join-delay considerations. Return true if it's safe.
- *
- * The initial tests in distribute_qual_to_rels will consider a mergejoinable
- * clause to be a potential equivalence clause if it is not outerjoin_delayed.
- * But since the point of equivalence processing is that we will recombine the
- * two sides of the clause with others, we have to check that each side
- * satisfies the not-outerjoin_delayed condition on its own; otherwise it might
- * not be safe to evaluate everywhere we could place a derived equivalence
- * condition.
- */
-static bool
-check_equivalence_delay(PlannerInfo *root,
- RestrictInfo *restrictinfo)
-{
- Relids relids;
-
- /* fast path if no special joins */
- if (root->join_info_list == NIL)
- return true;
-
- /* must copy restrictinfo's relids to avoid changing it */
- relids = bms_copy(restrictinfo->left_relids);
- /* check left side does not need delay */
- if (check_outerjoin_delay(root, &relids))
- return false;
-
- /* and similarly for the right side */
- relids = bms_copy(restrictinfo->right_relids);
- if (check_outerjoin_delay(root, &relids))
- return false;
-
- return true;
-}
-
/*
* check_redundant_nullability_qual
* Check to see if the qual is an IS NULL qual that is redundant with
@@ -2697,25 +2517,33 @@ static bool
check_redundant_nullability_qual(PlannerInfo *root, Node *clause)
{
Var *forced_null_var;
- Index forced_null_rel;
ListCell *lc;
/* Check for IS NULL, and identify the Var forced to NULL */
forced_null_var = find_forced_null_var(clause);
if (forced_null_var == NULL)
return false;
- forced_null_rel = forced_null_var->varno;
/*
* If the Var comes from the nullable side of a lower antijoin, the IS
- * NULL condition is necessarily true.
+ * NULL condition is necessarily true. If it's not nulled by anything,
+ * there is no point in searching the join_info_list. Otherwise, we need
+ * to find out whether the nulling rel is an antijoin.
*/
+ if (forced_null_var->varnullingrels == NULL)
+ return false;
+
foreach(lc, root->join_info_list)
{
SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
- if (sjinfo->jointype == JOIN_ANTI &&
- bms_is_member(forced_null_rel, sjinfo->syn_righthand))
+ /*
+ * This test will not succeed if sjinfo->ojrelid is zero, which is
+ * possible for an antijoin that was converted from a semijoin; but in
+ * such a case the Var couldn't have come from its nullable side.
+ */
+ if (sjinfo->jointype == JOIN_ANTI && sjinfo->ojrelid != 0 &&
+ bms_is_member(sjinfo->ojrelid, forced_null_var->varnullingrels))
return true;
}
@@ -2907,7 +2735,6 @@ process_implied_equality(PlannerInfo *root,
restrictinfo = make_restrictinfo(root,
(Expr *) clause,
true, /* is_pushed_down */
- false, /* outerjoin_delayed */
pseudoconstant,
security_level,
relids,
@@ -2999,7 +2826,6 @@ build_implied_join_equality(PlannerInfo *root,
restrictinfo = make_restrictinfo(root,
clause,
true, /* is_pushed_down */
- false, /* outerjoin_delayed */
false, /* pseudoconstant */
security_level, /* security_level */
qualscope, /* required_relids */
@@ -3071,8 +2897,7 @@ match_foreign_keys_to_quals(PlannerInfo *root)
* Note: for simple inner joins, any match should be in an eclass.
* "Loose" quals that syntactically match an FK equality must have
* been rejected for EC status because they are outer-join quals or
- * similar. We can still consider them to match the FK if they are
- * not outerjoin_delayed.
+ * similar. We can still consider them to match the FK.
*/
for (colno = 0; colno < fkinfo->nkeys; colno++)
{
@@ -3107,10 +2932,6 @@ match_foreign_keys_to_quals(PlannerInfo *root)
Var *leftvar;
Var *rightvar;
- /* Ignore outerjoin-delayed clauses */
- if (rinfo->outerjoin_delayed)
- continue;
-
/* Only binary OpExprs are useful for consideration */
if (!IsA(clause, OpExpr) ||
list_length(clause->args) != 2)
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 6814377599..0806ac78fa 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -866,7 +866,6 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
make_restrictinfo(root,
(Expr *) onecq,
rinfo->is_pushed_down,
- rinfo->outerjoin_delayed,
pseudoconstant,
rinfo->security_level,
NULL, NULL));
@@ -902,7 +901,7 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
/* not likely that we'd see constants here, so no check */
childquals = lappend(childquals,
make_restrictinfo(root, qual,
- true, false, false,
+ true, false,
security_level,
NULL, NULL));
cq_min_security = Min(cq_min_security, security_level);
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index e02dabe2ee..50e93f0a62 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -267,7 +267,6 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
orclause,
true,
false,
- false,
join_or_rinfo->security_level,
NULL,
NULL);
diff --git a/src/backend/optimizer/util/placeholder.c b/src/backend/optimizer/util/placeholder.c
index 25d4c54400..4435fd9090 100644
--- a/src/backend/optimizer/util/placeholder.c
+++ b/src/backend/optimizer/util/placeholder.c
@@ -134,7 +134,6 @@ find_placeholder_info(PlannerInfo *root, PlaceHolderVar *phv)
phinfo->ph_eval_at = bms_copy(phv->phrels);
Assert(!bms_is_empty(phinfo->ph_eval_at));
}
- /* ph_eval_at may change later, see update_placeholder_eval_levels */
phinfo->ph_needed = NULL; /* initially it's unused */
/* for the moment, estimate width using just the datatype info */
phinfo->ph_width = get_typavgwidth(exprType((Node *) phv->phexpr),
@@ -284,99 +283,6 @@ find_placeholders_in_expr(PlannerInfo *root, Node *expr)
list_free(vars);
}
-/*
- * update_placeholder_eval_levels
- * Adjust the target evaluation levels for placeholders
- *
- * The initial eval_at level set by find_placeholder_info was the set of
- * rels used in the placeholder's expression (or the whole subselect below
- * the placeholder's syntactic location, if the expr is variable-free).
- * If the query contains any outer joins that can null any of those rels,
- * we must delay evaluation to above those joins.
- *
- * We repeat this operation each time we add another outer join to
- * root->join_info_list. It's somewhat annoying to have to do that, but
- * since we don't have very much information on the placeholders' locations,
- * it's hard to avoid. Each placeholder's eval_at level must be correct
- * by the time it starts to figure in outer-join delay decisions for higher
- * outer joins.
- *
- * In future we might want to put additional policy/heuristics here to
- * try to determine an optimal evaluation level. The current rules will
- * result in evaluation at the lowest possible level. However, pushing a
- * placeholder eval up the tree is likely to further constrain evaluation
- * order for outer joins, so it could easily be counterproductive; and we
- * don't have enough information at this point to make an intelligent choice.
- */
-void
-update_placeholder_eval_levels(PlannerInfo *root, SpecialJoinInfo *new_sjinfo)
-{
- ListCell *lc1;
-
- foreach(lc1, root->placeholder_list)
- {
- PlaceHolderInfo *phinfo = (PlaceHolderInfo *) lfirst(lc1);
- Relids syn_level = phinfo->ph_var->phrels;
- Relids eval_at;
- bool found_some;
- ListCell *lc2;
-
- /*
- * We don't need to do any work on this placeholder unless the
- * newly-added outer join is syntactically beneath its location.
- */
- if (!bms_is_subset(new_sjinfo->syn_lefthand, syn_level) ||
- !bms_is_subset(new_sjinfo->syn_righthand, syn_level))
- continue;
-
- /*
- * Check for delays due to lower outer joins. This is the same logic
- * as in check_outerjoin_delay in initsplan.c.
- */
- eval_at = phinfo->ph_eval_at;
-
- do
- {
- found_some = false;
- foreach(lc2, root->join_info_list)
- {
- SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc2);
-
- /* disregard joins not within the PHV's sub-select */
- if (!bms_is_subset(sjinfo->syn_lefthand, syn_level) ||
- !bms_is_subset(sjinfo->syn_righthand, syn_level))
- continue;
-
- /* do we reference any nullable rels of this OJ? */
- if (bms_overlap(eval_at, sjinfo->min_righthand) ||
- (sjinfo->jointype == JOIN_FULL &&
- bms_overlap(eval_at, sjinfo->min_lefthand)))
- {
- /* yes; have we included all its rels in eval_at? */
- if (!bms_is_subset(sjinfo->min_lefthand, eval_at) ||
- !bms_is_subset(sjinfo->min_righthand, eval_at))
- {
- /* no, so add them in */
- eval_at = bms_add_members(eval_at,
- sjinfo->min_lefthand);
- eval_at = bms_add_members(eval_at,
- sjinfo->min_righthand);
- if (sjinfo->ojrelid)
- eval_at = bms_add_member(eval_at, sjinfo->ojrelid);
- /* we'll need another iteration */
- found_some = true;
- }
- }
- }
- } while (found_some);
-
- /* Can't move the PHV's eval_at level to above its syntactic level */
- Assert(bms_is_subset(eval_at, syn_level));
-
- phinfo->ph_eval_at = eval_at;
- }
-}
-
/*
* fix_placeholder_input_needed_levels
* Adjust the "needed at" levels for placeholder inputs
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index c3af845acd..301b41b278 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -25,7 +25,6 @@ static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
Expr *clause,
Expr *orclause,
bool is_pushed_down,
- bool outerjoin_delayed,
bool pseudoconstant,
Index security_level,
Relids required_relids,
@@ -33,7 +32,6 @@ static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
static Expr *make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
bool is_pushed_down,
- bool outerjoin_delayed,
bool pseudoconstant,
Index security_level,
Relids required_relids,
@@ -45,7 +43,7 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
*
* Build a RestrictInfo node containing the given subexpression.
*
- * The is_pushed_down, outerjoin_delayed, and pseudoconstant flags for the
+ * The is_pushed_down and pseudoconstant flags for the
* RestrictInfo must be supplied by the caller, as well as the correct values
* for security_level and outer_relids.
* required_relids can be NULL, in which case it defaults to the actual clause
@@ -63,7 +61,6 @@ RestrictInfo *
make_restrictinfo(PlannerInfo *root,
Expr *clause,
bool is_pushed_down,
- bool outerjoin_delayed,
bool pseudoconstant,
Index security_level,
Relids required_relids,
@@ -77,7 +74,6 @@ make_restrictinfo(PlannerInfo *root,
return (RestrictInfo *) make_sub_restrictinfos(root,
clause,
is_pushed_down,
- outerjoin_delayed,
pseudoconstant,
security_level,
required_relids,
@@ -90,7 +86,6 @@ make_restrictinfo(PlannerInfo *root,
clause,
NULL,
is_pushed_down,
- outerjoin_delayed,
pseudoconstant,
security_level,
required_relids,
@@ -107,7 +102,6 @@ make_restrictinfo_internal(PlannerInfo *root,
Expr *clause,
Expr *orclause,
bool is_pushed_down,
- bool outerjoin_delayed,
bool pseudoconstant,
Index security_level,
Relids required_relids,
@@ -119,7 +113,6 @@ make_restrictinfo_internal(PlannerInfo *root,
restrictinfo->clause = clause;
restrictinfo->orclause = orclause;
restrictinfo->is_pushed_down = is_pushed_down;
- restrictinfo->outerjoin_delayed = outerjoin_delayed;
restrictinfo->pseudoconstant = pseudoconstant;
restrictinfo->has_clone = false; /* may get set by caller */
restrictinfo->is_clone = false; /* may get set by caller */
@@ -251,7 +244,7 @@ make_restrictinfo_internal(PlannerInfo *root,
* implicit-AND lists at top level of RestrictInfo lists. Only ORs and
* simple clauses are valid RestrictInfos.
*
- * The same is_pushed_down, outerjoin_delayed, and pseudoconstant flag
+ * The same is_pushed_down and pseudoconstant flag
* values can be applied to all RestrictInfo nodes in the result. Likewise
* for security_level and outer_relids.
*
@@ -263,7 +256,6 @@ static Expr *
make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
bool is_pushed_down,
- bool outerjoin_delayed,
bool pseudoconstant,
Index security_level,
Relids required_relids,
@@ -279,7 +271,6 @@ make_sub_restrictinfos(PlannerInfo *root,
make_sub_restrictinfos(root,
lfirst(temp),
is_pushed_down,
- outerjoin_delayed,
pseudoconstant,
security_level,
NULL,
@@ -288,7 +279,6 @@ make_sub_restrictinfos(PlannerInfo *root,
clause,
make_orclause(orlist),
is_pushed_down,
- outerjoin_delayed,
pseudoconstant,
security_level,
required_relids,
@@ -304,7 +294,6 @@ make_sub_restrictinfos(PlannerInfo *root,
make_sub_restrictinfos(root,
lfirst(temp),
is_pushed_down,
- outerjoin_delayed,
pseudoconstant,
security_level,
required_relids,
@@ -316,7 +305,6 @@ make_sub_restrictinfos(PlannerInfo *root,
clause,
NULL,
is_pushed_down,
- outerjoin_delayed,
pseudoconstant,
security_level,
required_relids,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 6a6fbf3fd4..e759f99161 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2370,24 +2370,12 @@ typedef struct LimitPath
* conditions. Possibly we should rename it to reflect that meaning? But
* see also the comments for RINFO_IS_PUSHED_DOWN, below.)
*
- * RestrictInfo nodes also contain an outerjoin_delayed flag, which is true
- * if the clause's applicability must be delayed due to any outer joins
- * appearing below it (ie, it has to be postponed to some join level higher
- * than the set of relations it actually references).
- *
* There is also an outer_relids field, which is NULL except for outer join
* clauses; for those, it is the set of relids on the outer side of the
* clause's outer join. (These are rels that the clause cannot be applied to
* in parameterized scans, since pushing it into the join's outer side would
* lead to wrong answers.)
*
- * XXX this comment needs work, if we don't remove it completely:
- * outerjoin_delayed = true is subtly different from nullable_relids != NULL:
- * a clause might reference some nullable rels and yet not be
- * outerjoin_delayed because it also references all the other rels of the
- * outer join(s). A clause that is not outerjoin_delayed can be enforced
- * anywhere it is computable.
- *
* To handle security-barrier conditions efficiently, we mark RestrictInfo
* nodes with a security_level field, in which higher values identify clauses
* coming from less-trusted sources. The exact semantics are that a clause
@@ -2461,9 +2449,6 @@ typedef struct RestrictInfo
/* true if clause was pushed down in level */
bool is_pushed_down;
- /* true if delayed by lower outer join */
- bool outerjoin_delayed;
-
/* see comment above */
bool can_join pg_node_attr(equal_ignore);
diff --git a/src/include/optimizer/placeholder.h b/src/include/optimizer/placeholder.h
index 3fe9b57415..7026d5a104 100644
--- a/src/include/optimizer/placeholder.h
+++ b/src/include/optimizer/placeholder.h
@@ -22,8 +22,6 @@ extern PlaceHolderVar *make_placeholder_expr(PlannerInfo *root, Expr *expr,
extern PlaceHolderInfo *find_placeholder_info(PlannerInfo *root,
PlaceHolderVar *phv);
extern void find_placeholders_in_jointree(PlannerInfo *root);
-extern void update_placeholder_eval_levels(PlannerInfo *root,
- SpecialJoinInfo *new_sjinfo);
extern void fix_placeholder_input_needed_levels(PlannerInfo *root);
extern void add_placeholders_to_base_rels(PlannerInfo *root);
extern void add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
diff --git a/src/include/optimizer/restrictinfo.h b/src/include/optimizer/restrictinfo.h
index 1f092371ea..9e24f8cd40 100644
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -19,12 +19,11 @@
/* Convenience macro for the common case of a valid-everywhere qual */
#define make_simple_restrictinfo(root, clause) \
- make_restrictinfo(root, clause, true, false, false, 0, NULL, NULL)
+ make_restrictinfo(root, clause, true, false, 0, NULL, NULL)
extern RestrictInfo *make_restrictinfo(PlannerInfo *root,
Expr *clause,
bool is_pushed_down,
- bool outerjoin_delayed,
bool pseudoconstant,
Index security_level,
Relids required_relids,
diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out
index a5c09cce01..7f52524b97 100644
--- a/src/test/regress/expected/join.out
+++ b/src/test/regress/expected/join.out
@@ -4020,10 +4020,10 @@ explain (costs off)
select q1, unique2, thousand, hundred
from int8_tbl a left join tenk1 b on q1 = unique2
where coalesce(thousand,123) = q1 and q1 = coalesce(hundred,123);
- QUERY PLAN
---------------------------------------------------------------------------------------
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------
Nested Loop Left Join
- Filter: ((COALESCE(b.thousand, 123) = a.q1) AND (a.q1 = COALESCE(b.hundred, 123)))
+ Filter: ((COALESCE(b.thousand, 123) = COALESCE(b.hundred, 123)) AND (a.q1 = COALESCE(b.hundred, 123)))
-> Seq Scan on int8_tbl a
-> Index Scan using tenk1_unique2 on tenk1 b
Index Cond: (unique2 = a.q1)
commit 7e33486adaec904a96dddc16f51ac0cc543124b7
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Fri Dec 23 12:49:12 2022 -0500
Invent "join domains" to replace the below_outer_join hack.
EquivalenceClasses are now understood as applying within a "join
domain", which is a set of inner-joined relations (possibly underneath
an outer join). We no longer need to treat an EC from below an outer
join as a second-class citizen.
I have hopes of eventually being able to treat outer-join clauses via
EquivalenceClasses, by means of only applying deductions within the
EC's join domain. There are still problems in the way of that, though,
so for now the reconsider_outer_join_clause logic is still here.
I haven't been able to get rid of is_pushed_down either, but I wonder
if that could be recast using JoinDomains.
I had to hack one test case in postgres_fdw.sql to make it still test
what it was meant to, because postgres_fdw is inconsistent about
how it deals with quals containing non-shippable expressions; see
https://postgr.es/m/1691374.1671659838@sss.pgh.pa.us. That should
be improved, but I don't think it's within the scope of this patch
series.
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index bd706d18d1..6094463195 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2513,7 +2513,7 @@ SELECT * FROM local_tbl LEFT JOIN (SELECT ft1.*, COALESCE(ft1.c3 || ft2.c3, 'foo
ALTER SERVER loopback OPTIONS (DROP extensions);
ALTER SERVER loopback OPTIONS (ADD fdw_startup_cost '10000.0');
EXPLAIN (VERBOSE, COSTS OFF)
-SELECT * FROM local_tbl LEFT JOIN (SELECT ft1.* FROM ft1 INNER JOIN ft2 ON (ft1.c1 = ft2.c1 AND ft1.c1 < 100 AND
ft1.c1= postgres_fdw_abs(ft2.c2))) ss ON (local_tbl.c3 = ss.c3) ORDER BY local_tbl.c1 FOR UPDATE OF local_tbl;
+SELECT * FROM local_tbl LEFT JOIN (SELECT ft1.* FROM ft1 INNER JOIN ft2 ON (ft1.c1 = ft2.c1 AND ft1.c1 < 100 AND
(ft1.c1- postgres_fdw_abs(ft2.c2)) = 0)) ss ON (local_tbl.c3 = ss.c3) ORDER BY local_tbl.c1 FOR UPDATE OF local_tbl;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
LockRows
@@ -2527,7 +2527,7 @@ SELECT * FROM local_tbl LEFT JOIN (SELECT ft1.* FROM ft1 INNER JOIN ft2 ON (ft1.
Output: ft1.c1, ft1.c2, ft1.c3, ft1.c4, ft1.c5, ft1.c6, ft1.c7, ft1.c8, ft1.*, ft2.*
-> Foreign Scan
Output: ft1.c1, ft1.c2, ft1.c3, ft1.c4, ft1.c5, ft1.c6, ft1.c7, ft1.c8, ft1.*, ft2.*
- Filter: (ft1.c1 = postgres_fdw_abs(ft2.c2))
+ Filter: ((ft1.c1 - postgres_fdw_abs(ft2.c2)) = 0)
Relations: (public.ft1) INNER JOIN (public.ft2)
Remote SQL: SELECT r4."C 1", r4.c2, r4.c3, r4.c4, r4.c5, r4.c6, r4.c7, r4.c8, CASE WHEN
(r4.*)::textIS NOT NULL THEN ROW(r4."C 1", r4.c2, r4.c3, r4.c4, r4.c5, r4.c6, r4.c7, r4.c8) END, CASE WHEN (r5.*)::text
ISNOT NULL THEN ROW(r5."C 1", r5.c2, r5.c3, r5.c4, r5.c5, r5.c6, r5.c7, r5.c8) END, r5.c2 FROM ("S 1"."T 1" r4 INNER
JOIN"S 1"."T 1" r5 ON (((r5."C 1" = r4."C 1")) AND ((r4."C 1" < 100)))) ORDER BY r4.c3 ASC NULLS LAST
-> Sort
@@ -2535,18 +2535,18 @@ SELECT * FROM local_tbl LEFT JOIN (SELECT ft1.* FROM ft1 INNER JOIN ft2 ON (ft1.
Sort Key: ft1.c3
-> Merge Join
Output: ft1.c1, ft1.c2, ft1.c3, ft1.c4, ft1.c5, ft1.c6, ft1.c7, ft1.c8, ft1.*, ft2.*,
ft2.c2
- Merge Cond: ((ft1.c1 = (postgres_fdw_abs(ft2.c2))) AND (ft1.c1 = ft2.c1))
+ Merge Cond: (ft1.c1 = ft2.c1)
+ Join Filter: ((ft1.c1 - postgres_fdw_abs(ft2.c2)) = 0)
-> Sort
Output: ft1.c1, ft1.c2, ft1.c3, ft1.c4, ft1.c5, ft1.c6, ft1.c7, ft1.c8, ft1.*
Sort Key: ft1.c1
-> Foreign Scan on public.ft1
Output: ft1.c1, ft1.c2, ft1.c3, ft1.c4, ft1.c5, ft1.c6, ft1.c7, ft1.c8,
ft1.*
Remote SQL: SELECT "C 1", c2, c3, c4, c5, c6, c7, c8 FROM "S 1"."T 1"
WHERE(("C 1" < 100))
- -> Sort
- Output: ft2.*, ft2.c1, ft2.c2, (postgres_fdw_abs(ft2.c2))
- Sort Key: (postgres_fdw_abs(ft2.c2)), ft2.c1
+ -> Materialize
+ Output: ft2.*, ft2.c1, ft2.c2
-> Foreign Scan on public.ft2
- Output: ft2.*, ft2.c1, ft2.c2, postgres_fdw_abs(ft2.c2)
+ Output: ft2.*, ft2.c1, ft2.c2
Remote SQL: SELECT "C 1", c2, c3, c4, c5, c6, c7, c8 FROM "S 1"."T 1"
ORDERBY "C 1" ASC NULLS LAST
(32 rows)
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 2e6f7f4852..b239442cc4 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -681,7 +681,7 @@ SELECT * FROM local_tbl LEFT JOIN (SELECT ft1.*, COALESCE(ft1.c3 || ft2.c3, 'foo
ALTER SERVER loopback OPTIONS (DROP extensions);
ALTER SERVER loopback OPTIONS (ADD fdw_startup_cost '10000.0');
EXPLAIN (VERBOSE, COSTS OFF)
-SELECT * FROM local_tbl LEFT JOIN (SELECT ft1.* FROM ft1 INNER JOIN ft2 ON (ft1.c1 = ft2.c1 AND ft1.c1 < 100 AND
ft1.c1= postgres_fdw_abs(ft2.c2))) ss ON (local_tbl.c3 = ss.c3) ORDER BY local_tbl.c1 FOR UPDATE OF local_tbl;
+SELECT * FROM local_tbl LEFT JOIN (SELECT ft1.* FROM ft1 INNER JOIN ft2 ON (ft1.c1 = ft2.c1 AND ft1.c1 < 100 AND
(ft1.c1- postgres_fdw_abs(ft2.c2)) = 0)) ss ON (local_tbl.c3 = ss.c3) ORDER BY local_tbl.c1 FOR UPDATE OF local_tbl;
ALTER SERVER loopback OPTIONS (DROP fdw_startup_cost);
ALTER SERVER loopback OPTIONS (ADD extensions 'postgres_fdw');
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 59b0fdeb62..06f6d9127b 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -468,7 +468,6 @@ _outEquivalenceClass(StringInfo str, const EquivalenceClass *node)
WRITE_BITMAPSET_FIELD(ec_relids);
WRITE_BOOL_FIELD(ec_has_const);
WRITE_BOOL_FIELD(ec_has_volatile);
- WRITE_BOOL_FIELD(ec_below_outer_join);
WRITE_BOOL_FIELD(ec_broken);
WRITE_UINT_FIELD(ec_sortref);
WRITE_UINT_FIELD(ec_min_security);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index bcf7fcf21c..e4b1e8fa14 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -35,6 +35,7 @@
static EquivalenceMember *add_eq_member(EquivalenceClass *ec,
Expr *expr, Relids relids,
+ JoinDomain *jdomain,
EquivalenceMember *parent,
Oid datatype);
static bool is_exprlist_member(Expr *node, List *exprs);
@@ -67,6 +68,7 @@ static bool reconsider_outer_join_clause(PlannerInfo *root,
bool outer_on_left);
static bool reconsider_full_join_clause(PlannerInfo *root,
OuterJoinClauseInfo *ojcinfo);
+static JoinDomain *find_join_domain(PlannerInfo *root, Relids relids);
static Bitmapset *get_eclass_indexes_for_relids(PlannerInfo *root,
Relids relids);
static Bitmapset *get_common_eclass_indexes(PlannerInfo *root, Relids relids1,
@@ -75,8 +77,8 @@ static Bitmapset *get_common_eclass_indexes(PlannerInfo *root, Relids relids1,
/*
* process_equivalence
- * The given clause has a mergejoinable operator and can be applied without
- * any delay by an outer join, so its two sides can be considered equal
+ * The given clause has a mergejoinable operator and is not an outer-join
+ * qualification, so its two sides can be considered equal
* anywhere they are both computable; moreover that equality can be
* extended transitively. Record this knowledge in the EquivalenceClass
* data structure, if applicable. Returns true if successful, false if not
@@ -88,16 +90,11 @@ static Bitmapset *get_common_eclass_indexes(PlannerInfo *root, Relids relids1,
* Then, *p_restrictinfo will be replaced by a new RestrictInfo, which is what
* the caller should use for further processing.
*
- * If below_outer_join is true, then the clause was found below the nullable
- * side of an outer join, so its sides might validly be both NULL rather than
- * strictly equal. We can still deduce equalities in such cases, but we take
- * care to mark an EquivalenceClass if it came from any such clauses. Also,
- * we have to check that both sides are either pseudo-constants or strict
- * functions of Vars, else they might not both go to NULL above the outer
- * join. (This is the main reason why we need a failure return. It's more
- * convenient to check this case here than at the call sites...)
+ * jdomain is the join domain within which the given clause was found.
+ * This limits the applicability of deductions from the EquivalenceClass,
+ * as described in optimizer/README.
*
- * We also reject proposed equivalence clauses if they contain leaky functions
+ * We reject proposed equivalence clauses if they contain leaky functions
* and have security_level above zero. The EC evaluation rules require us to
* apply certain tests at certain joining levels, and we can't tolerate
* delaying any test on security_level grounds. By rejecting candidate clauses
@@ -120,7 +117,7 @@ static Bitmapset *get_common_eclass_indexes(PlannerInfo *root, Relids relids1,
bool
process_equivalence(PlannerInfo *root,
RestrictInfo **p_restrictinfo,
- bool below_outer_join)
+ JoinDomain *jdomain)
{
RestrictInfo *restrictinfo = *p_restrictinfo;
Expr *clause = restrictinfo->clause;
@@ -208,19 +205,6 @@ process_equivalence(PlannerInfo *root,
return false;
}
- /*
- * If below outer join, check for strictness, else reject.
- */
- if (below_outer_join)
- {
- if (!bms_is_empty(item1_relids) &&
- contain_nonstrict_functions((Node *) item1))
- return false; /* LHS is non-strict but not constant */
- if (!bms_is_empty(item2_relids) &&
- contain_nonstrict_functions((Node *) item2))
- return false; /* RHS is non-strict but not constant */
- }
-
/*
* We use the declared input types of the operator, not exprType() of the
* inputs, as the nominal datatypes for opfamily lookup. This presumes
@@ -285,11 +269,10 @@ process_equivalence(PlannerInfo *root,
Assert(!cur_em->em_is_child); /* no children yet */
/*
- * If below an outer join, don't match constants: they're not as
- * constant as they look.
+ * Match constants only within the same JoinDomain (see
+ * optimizer/README).
*/
- if ((below_outer_join || cur_ec->ec_below_outer_join) &&
- cur_em->em_is_const)
+ if (cur_em->em_is_const && cur_em->em_jdomain != jdomain)
continue;
if (!ec1 &&
@@ -326,7 +309,6 @@ process_equivalence(PlannerInfo *root,
if (ec1 == ec2)
{
ec1->ec_sources = lappend(ec1->ec_sources, restrictinfo);
- ec1->ec_below_outer_join |= below_outer_join;
ec1->ec_min_security = Min(ec1->ec_min_security,
restrictinfo->security_level);
ec1->ec_max_security = Max(ec1->ec_max_security,
@@ -362,7 +344,6 @@ process_equivalence(PlannerInfo *root,
ec1->ec_relids = bms_join(ec1->ec_relids, ec2->ec_relids);
ec1->ec_has_const |= ec2->ec_has_const;
/* can't need to set has_volatile */
- ec1->ec_below_outer_join |= ec2->ec_below_outer_join;
ec1->ec_min_security = Min(ec1->ec_min_security,
ec2->ec_min_security);
ec1->ec_max_security = Max(ec1->ec_max_security,
@@ -375,7 +356,6 @@ process_equivalence(PlannerInfo *root,
ec2->ec_derives = NIL;
ec2->ec_relids = NULL;
ec1->ec_sources = lappend(ec1->ec_sources, restrictinfo);
- ec1->ec_below_outer_join |= below_outer_join;
ec1->ec_min_security = Min(ec1->ec_min_security,
restrictinfo->security_level);
ec1->ec_max_security = Max(ec1->ec_max_security,
@@ -391,9 +371,8 @@ process_equivalence(PlannerInfo *root,
{
/* Case 3: add item2 to ec1 */
em2 = add_eq_member(ec1, item2, item2_relids,
- NULL, item2_type);
+ jdomain, NULL, item2_type);
ec1->ec_sources = lappend(ec1->ec_sources, restrictinfo);
- ec1->ec_below_outer_join |= below_outer_join;
ec1->ec_min_security = Min(ec1->ec_min_security,
restrictinfo->security_level);
ec1->ec_max_security = Max(ec1->ec_max_security,
@@ -409,9 +388,8 @@ process_equivalence(PlannerInfo *root,
{
/* Case 3: add item1 to ec2 */
em1 = add_eq_member(ec2, item1, item1_relids,
- NULL, item1_type);
+ jdomain, NULL, item1_type);
ec2->ec_sources = lappend(ec2->ec_sources, restrictinfo);
- ec2->ec_below_outer_join |= below_outer_join;
ec2->ec_min_security = Min(ec2->ec_min_security,
restrictinfo->security_level);
ec2->ec_max_security = Max(ec2->ec_max_security,
@@ -436,16 +414,15 @@ process_equivalence(PlannerInfo *root,
ec->ec_relids = NULL;
ec->ec_has_const = false;
ec->ec_has_volatile = false;
- ec->ec_below_outer_join = below_outer_join;
ec->ec_broken = false;
ec->ec_sortref = 0;
ec->ec_min_security = restrictinfo->security_level;
ec->ec_max_security = restrictinfo->security_level;
ec->ec_merged = NULL;
em1 = add_eq_member(ec, item1, item1_relids,
- NULL, item1_type);
+ jdomain, NULL, item1_type);
em2 = add_eq_member(ec, item2, item2_relids,
- NULL, item2_type);
+ jdomain, NULL, item2_type);
root->eq_classes = lappend(root->eq_classes, ec);
@@ -535,7 +512,7 @@ canonicalize_ec_expression(Expr *expr, Oid req_type, Oid req_collation)
*/
static EquivalenceMember *
add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
- EquivalenceMember *parent, Oid datatype)
+ JoinDomain *jdomain, EquivalenceMember *parent, Oid datatype)
{
EquivalenceMember *em = makeNode(EquivalenceMember);
@@ -544,6 +521,7 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
em->em_is_const = false;
em->em_is_child = (parent != NULL);
em->em_datatype = datatype;
+ em->em_jdomain = jdomain;
em->em_parent = parent;
if (bms_is_empty(relids))
@@ -612,6 +590,7 @@ get_eclass_for_sort_expr(PlannerInfo *root,
Relids rel,
bool create_it)
{
+ JoinDomain *jdomain;
Relids expr_relids;
EquivalenceClass *newec;
EquivalenceMember *newem;
@@ -623,6 +602,12 @@ get_eclass_for_sort_expr(PlannerInfo *root,
*/
expr = canonicalize_ec_expression(expr, opcintype, collation);
+ /*
+ * Since SortGroupClause nodes are top-level expressions (GROUP BY, ORDER
+ * BY, etc), they can be presumed to belong to the top JoinDomain.
+ */
+ jdomain = linitial_node(JoinDomain, root->join_domains);
+
/*
* Scan through the existing EquivalenceClasses for a match
*/
@@ -656,11 +641,10 @@ get_eclass_for_sort_expr(PlannerInfo *root,
continue;
/*
- * If below an outer join, don't match constants: they're not as
- * constant as they look.
+ * Match constants only within the same JoinDomain (see
+ * optimizer/README).
*/
- if (cur_ec->ec_below_outer_join &&
- cur_em->em_is_const)
+ if (cur_em->em_is_const && cur_em->em_jdomain != jdomain)
continue;
if (opcintype == cur_em->em_datatype &&
@@ -689,7 +673,6 @@ get_eclass_for_sort_expr(PlannerInfo *root,
newec->ec_relids = NULL;
newec->ec_has_const = false;
newec->ec_has_volatile = contain_volatile_functions((Node *) expr);
- newec->ec_below_outer_join = false;
newec->ec_broken = false;
newec->ec_sortref = sortref;
newec->ec_min_security = UINT_MAX;
@@ -705,7 +688,7 @@ get_eclass_for_sort_expr(PlannerInfo *root,
expr_relids = pull_varnos(root, (Node *) expr);
newem = add_eq_member(newec, copyObject(expr), expr_relids,
- NULL, opcintype);
+ jdomain, NULL, opcintype);
/*
* add_eq_member doesn't check for volatile functions, set-returning
@@ -1185,11 +1168,16 @@ generate_base_implied_equalities_const(PlannerInfo *root,
ec->ec_broken = true;
break;
}
+
+ /*
+ * We use the constant's em_jdomain as qualscope, so that if the
+ * generated clause is variable-free (i.e, both EMs are consts) it
+ * will be enforced at the join domain level.
+ */
rinfo = process_implied_equality(root, eq_op, ec->ec_collation,
cur_em->em_expr, const_em->em_expr,
- bms_copy(ec->ec_relids),
+ const_em->em_jdomain->jd_relids,
ec->ec_min_security,
- ec->ec_below_outer_join,
cur_em->em_is_const);
/*
@@ -1257,11 +1245,16 @@ generate_base_implied_equalities_no_const(PlannerInfo *root,
ec->ec_broken = true;
break;
}
+
+ /*
+ * The expressions aren't constants, so the passed qualscope will
+ * never be used to place the generated clause. We just need to
+ * be sure it covers both expressions, so ec_relids will serve.
+ */
rinfo = process_implied_equality(root, eq_op, ec->ec_collation,
prev_em->em_expr, cur_em->em_expr,
- bms_copy(ec->ec_relids),
+ ec->ec_relids,
ec->ec_min_security,
- ec->ec_below_outer_join,
false);
/*
@@ -2074,6 +2067,7 @@ reconsider_outer_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo,
bool outer_on_left)
{
RestrictInfo *rinfo = ojcinfo->rinfo;
+ SpecialJoinInfo *sjinfo = ojcinfo->sjinfo;
Expr *outervar,
*innervar;
Oid opno,
@@ -2150,6 +2144,7 @@ reconsider_outer_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo,
EquivalenceMember *cur_em = (EquivalenceMember *) lfirst(lc2);
Oid eq_op;
RestrictInfo *newrinfo;
+ JoinDomain *jdomain;
if (!cur_em->em_is_const)
continue; /* ignore non-const members */
@@ -2165,7 +2160,9 @@ reconsider_outer_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo,
cur_em->em_expr,
bms_copy(inner_relids),
cur_ec->ec_min_security);
- if (process_equivalence(root, &newrinfo, true))
+ /* This equality holds within the OJ's child JoinDomain */
+ jdomain = find_join_domain(root, sjinfo->syn_righthand);
+ if (process_equivalence(root, &newrinfo, jdomain))
match = true;
}
@@ -2300,6 +2297,7 @@ reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
EquivalenceMember *cur_em = (EquivalenceMember *) lfirst(lc2);
Oid eq_op;
RestrictInfo *newrinfo;
+ JoinDomain *jdomain;
if (!cur_em->em_is_const)
continue; /* ignore non-const members */
@@ -2315,7 +2313,9 @@ reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
cur_em->em_expr,
bms_copy(left_relids),
cur_ec->ec_min_security);
- if (process_equivalence(root, &newrinfo, true))
+ /* This equality holds within the lefthand child JoinDomain */
+ jdomain = find_join_domain(root, sjinfo->syn_lefthand);
+ if (process_equivalence(root, &newrinfo, jdomain))
matchleft = true;
}
eq_op = select_equality_operator(cur_ec,
@@ -2330,7 +2330,9 @@ reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
cur_em->em_expr,
bms_copy(right_relids),
cur_ec->ec_min_security);
- if (process_equivalence(root, &newrinfo, true))
+ /* This equality holds within the righthand child JoinDomain */
+ jdomain = find_join_domain(root, sjinfo->syn_righthand);
+ if (process_equivalence(root, &newrinfo, jdomain))
matchright = true;
}
}
@@ -2359,6 +2361,29 @@ reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
return false; /* failed to make any deduction */
}
+/*
+ * find_join_domain
+ * Find the highest JoinDomain enclosed within the given relid set.
+ *
+ * (We could avoid this search at the cost of complicating APIs elsewhere,
+ * which doesn't seem worth it.)
+ */
+static JoinDomain *
+find_join_domain(PlannerInfo *root, Relids relids)
+{
+ ListCell *lc;
+
+ foreach(lc, root->join_domains)
+ {
+ JoinDomain *jdomain = (JoinDomain *) lfirst(lc);
+
+ if (bms_is_subset(jdomain->jd_relids, relids))
+ return jdomain;
+ }
+ elog(ERROR, "failed to find appropriate JoinDomain");
+ return NULL; /* keep compiler quiet */
+}
+
/*
* exprs_known_equal
@@ -2648,6 +2673,7 @@ add_child_rel_equivalences(PlannerInfo *root,
new_relids = bms_add_members(new_relids, child_relids);
(void) add_eq_member(cur_ec, child_expr, new_relids,
+ cur_em->em_jdomain,
cur_em, cur_em->em_datatype);
/* Record this EC index for the child rel */
@@ -2775,6 +2801,7 @@ add_child_join_rel_equivalences(PlannerInfo *root,
new_relids = bms_add_members(new_relids, child_relids);
(void) add_eq_member(cur_ec, child_expr, new_relids,
+ cur_em->em_jdomain,
cur_em, cur_em->em_datatype);
}
}
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 3d7ee7dfe5..9018f7dd32 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -2321,18 +2321,6 @@ select_mergejoin_clauses(PlannerInfo *root,
* canonical pathkey list, but redundant eclasses can't appear in
* canonical sort orderings. (XXX it might be worth relaxing this,
* but not enough time to address it for 8.3.)
- *
- * Note: it would be bad if this condition failed for an otherwise
- * mergejoinable FULL JOIN clause, since that would result in
- * undesirable planner failure. I believe that is not possible
- * however; a variable involved in a full join could only appear in
- * below_outer_join eclasses, which aren't considered redundant.
- *
- * This case *can* happen for left/right join clauses: the outer-side
- * variable could be equated to a constant. Because we will propagate
- * that constant across the join clause, the loss of ability to do a
- * mergejoin is not really all that big a deal, and so it's not clear
- * that improving this is important.
*/
update_mergeclause_eclasses(root, restrictinfo);
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 78dc153882..0a1bf3acb7 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -6210,10 +6210,7 @@ prepare_sort_from_pathkeys(Plan *lefttree, List *pathkeys,
* the pathkey's EquivalenceClass. For now, we take the first
* tlist item found in the EC. If there's no match, we'll generate
* a resjunk entry using the first EC member that is an expression
- * in the input's vars. (The non-const restriction only matters
- * if the EC is below_outer_join; but if it isn't, it won't
- * contain consts anyway, else we'd have discarded the pathkey as
- * redundant.)
+ * in the input's vars.
*
* XXX if we have a choice, is there any way of figuring out which
* might be cheapest to execute? (For example, int4lt is likely
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 6fe00652b6..83b2c2676d 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -61,7 +61,7 @@ typedef struct JoinTreeItem
{
/* Fields filled during deconstruct_recurse: */
Node *jtnode; /* jointree node to examine */
- bool below_outer_join; /* is it below an outer join? */
+ JoinDomain *jdomain; /* join domain for its ON/WHERE clauses */
Relids qualscope; /* base+OJ Relids syntactically included in
* this jointree node */
Relids inner_join_rels; /* base+OJ Relids syntactically included
@@ -87,13 +87,13 @@ typedef struct PostponedQual
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
- bool below_outer_join,
+ JoinDomain *parent_domain,
List **item_list);
static void deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
List **postponed_qual_list);
static void process_security_barrier_quals(PlannerInfo *root,
int rti, Relids qualscope,
- bool below_outer_join);
+ JoinDomain *jdomain);
static void mark_rels_nulled_by_join(PlannerInfo *root, Index ojrelid,
Relids lower_rels);
static SpecialJoinInfo *make_outerjoininfo(PlannerInfo *root,
@@ -107,7 +107,7 @@ static void deconstruct_distribute_oj_quals(PlannerInfo *root,
List *jtitems,
JoinTreeItem *jtitem);
static void distribute_quals_to_rels(PlannerInfo *root, List *clauses,
- bool below_outer_join,
+ JoinDomain *jdomain,
SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
@@ -119,7 +119,7 @@ static void distribute_quals_to_rels(PlannerInfo *root, List *clauses,
List **postponed_qual_list,
List **postponed_oj_qual_list);
static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
- bool below_outer_join,
+ JoinDomain *jdomain,
SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
@@ -740,6 +740,7 @@ List *
deconstruct_jointree(PlannerInfo *root)
{
List *result;
+ JoinDomain *top_jdomain;
List *item_list = NIL;
List *postponed_qual_list = NIL;
ListCell *lc;
@@ -751,6 +752,10 @@ deconstruct_jointree(PlannerInfo *root)
*/
root->placeholdersFrozen = true;
+ /* Fetch the already-created top-level join domain for the query */
+ top_jdomain = linitial_node(JoinDomain, root->join_domains);
+ top_jdomain->jd_relids = NULL; /* filled during deconstruct_recurse */
+
/* Start recursion at top of jointree */
Assert(root->parse->jointree != NULL &&
IsA(root->parse->jointree, FromExpr));
@@ -761,12 +766,15 @@ deconstruct_jointree(PlannerInfo *root)
/* Perform the initial scan of the jointree */
result = deconstruct_recurse(root, (Node *) root->parse->jointree,
- false,
+ top_jdomain,
&item_list);
/* Now we can form the value of all_query_rels, too */
root->all_query_rels = bms_union(root->all_baserels, root->outer_join_rels);
+ /* ... which should match what we computed for the top join domain */
+ Assert(bms_equal(root->all_query_rels, top_jdomain->jd_relids));
+
/* Now scan all the jointree nodes again, and distribute quals */
foreach(lc, item_list)
{
@@ -804,10 +812,9 @@ deconstruct_jointree(PlannerInfo *root)
* deconstruct_recurse
* One recursion level of deconstruct_jointree's initial jointree scan.
*
- * Inputs:
- * jtnode is the jointree node to examine
- * below_outer_join is true if this node is within the nullable side of a
- * higher-level outer join
+ * jtnode is the jointree node to examine, and parent_domain is the
+ * enclosing join domain. (We must add all base+OJ relids appearing
+ * here or below to parent_domain.)
*
* item_list is an in/out parameter: we add a JoinTreeItem struct to
* that list for each jointree node, in depth-first traversal order.
@@ -817,7 +824,7 @@ deconstruct_jointree(PlannerInfo *root)
*/
static List *
deconstruct_recurse(PlannerInfo *root, Node *jtnode,
- bool below_outer_join,
+ JoinDomain *parent_domain,
List **item_list)
{
List *joinlist;
@@ -828,7 +835,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
/* Make the new JoinTreeItem, but don't add it to item_list yet */
jtitem = palloc0_object(JoinTreeItem);
jtitem->jtnode = jtnode;
- jtitem->below_outer_join = below_outer_join;
if (IsA(jtnode, RangeTblRef))
{
@@ -836,6 +842,10 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
/* Fill all_baserels as we encounter baserel jointree nodes */
root->all_baserels = bms_add_member(root->all_baserels, varno);
+ /* This node belongs to parent_domain */
+ jtitem->jdomain = parent_domain;
+ parent_domain->jd_relids = bms_add_member(parent_domain->jd_relids,
+ varno);
/* qualscope is just the one RTE */
jtitem->qualscope = bms_make_singleton(varno);
/* A single baserel does not create an inner join */
@@ -848,6 +858,9 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
int remaining;
ListCell *l;
+ /* This node belongs to parent_domain, as do its children */
+ jtitem->jdomain = parent_domain;
+
/*
* Recurse to handle child nodes, and compute output joinlist. We
* collapse subproblems into a single joinlist whenever the resulting
@@ -866,7 +879,7 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
int sub_members;
sub_joinlist = deconstruct_recurse(root, lfirst(l),
- below_outer_join,
+ parent_domain,
item_list);
sub_item = (JoinTreeItem *) llast(*item_list);
jtitem->qualscope = bms_add_members(jtitem->qualscope,
@@ -894,6 +907,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
+ JoinDomain *child_domain,
+ *fj_domain;
JoinTreeItem *left_item,
*right_item;
List *leftjoinlist,
@@ -902,13 +917,15 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
switch (j->jointype)
{
case JOIN_INNER:
+ /* This node belongs to parent_domain, as do its children */
+ jtitem->jdomain = parent_domain;
/* Recurse */
leftjoinlist = deconstruct_recurse(root, j->larg,
- below_outer_join,
+ parent_domain,
item_list);
left_item = (JoinTreeItem *) llast(*item_list);
rightjoinlist = deconstruct_recurse(root, j->rarg,
- below_outer_join,
+ parent_domain,
item_list);
right_item = (JoinTreeItem *) llast(*item_list);
/* Compute qualscope etc */
@@ -922,21 +939,32 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
break;
case JOIN_LEFT:
case JOIN_ANTI:
+ /* Make new join domain for my quals and the RHS */
+ child_domain = makeNode(JoinDomain);
+ child_domain->jd_relids = NULL; /* filled by recursion */
+ root->join_domains = lappend(root->join_domains, child_domain);
+ jtitem->jdomain = child_domain;
/* Recurse */
leftjoinlist = deconstruct_recurse(root, j->larg,
- below_outer_join,
+ parent_domain,
item_list);
left_item = (JoinTreeItem *) llast(*item_list);
rightjoinlist = deconstruct_recurse(root, j->rarg,
- true,
+ child_domain,
item_list);
right_item = (JoinTreeItem *) llast(*item_list);
- /* Compute qualscope etc */
+ /* Compute join domain contents, qualscope etc */
+ parent_domain->jd_relids =
+ bms_add_members(parent_domain->jd_relids,
+ child_domain->jd_relids);
jtitem->qualscope = bms_union(left_item->qualscope,
right_item->qualscope);
/* caution: ANTI join derived from SEMI will lack rtindex */
if (j->rtindex != 0)
{
+ parent_domain->jd_relids =
+ bms_add_member(parent_domain->jd_relids,
+ j->rtindex);
jtitem->qualscope = bms_add_member(jtitem->qualscope,
j->rtindex);
root->outer_join_rels = bms_add_member(root->outer_join_rels,
@@ -951,13 +979,15 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
jtitem->nonnullable_rels = left_item->qualscope;
break;
case JOIN_SEMI:
+ /* This node belongs to parent_domain, as do its children */
+ jtitem->jdomain = parent_domain;
/* Recurse */
leftjoinlist = deconstruct_recurse(root, j->larg,
- below_outer_join,
+ parent_domain,
item_list);
left_item = (JoinTreeItem *) llast(*item_list);
rightjoinlist = deconstruct_recurse(root, j->rarg,
- below_outer_join,
+ parent_domain,
item_list);
right_item = (JoinTreeItem *) llast(*item_list);
/* Compute qualscope etc */
@@ -973,19 +1003,36 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
jtitem->nonnullable_rels = NULL;
break;
case JOIN_FULL:
- /* Recurse */
+ /* The FULL JOIN's quals need their very own domain */
+ fj_domain = makeNode(JoinDomain);
+ root->join_domains = lappend(root->join_domains, fj_domain);
+ jtitem->jdomain = fj_domain;
+ /* Recurse, giving each side its own join domain */
+ child_domain = makeNode(JoinDomain);
+ child_domain->jd_relids = NULL; /* filled by recursion */
+ root->join_domains = lappend(root->join_domains, child_domain);
leftjoinlist = deconstruct_recurse(root, j->larg,
- true,
+ child_domain,
item_list);
left_item = (JoinTreeItem *) llast(*item_list);
+ fj_domain->jd_relids = bms_copy(child_domain->jd_relids);
+ child_domain = makeNode(JoinDomain);
+ child_domain->jd_relids = NULL; /* filled by recursion */
+ root->join_domains = lappend(root->join_domains, child_domain);
rightjoinlist = deconstruct_recurse(root, j->rarg,
- true,
+ child_domain,
item_list);
right_item = (JoinTreeItem *) llast(*item_list);
/* Compute qualscope etc */
+ fj_domain->jd_relids = bms_add_members(fj_domain->jd_relids,
+ child_domain->jd_relids);
+ parent_domain->jd_relids = bms_add_members(parent_domain->jd_relids,
+ fj_domain->jd_relids);
jtitem->qualscope = bms_union(left_item->qualscope,
right_item->qualscope);
Assert(j->rtindex != 0);
+ parent_domain->jd_relids = bms_add_member(parent_domain->jd_relids,
+ j->rtindex);
jtitem->qualscope = bms_add_member(jtitem->qualscope,
j->rtindex);
root->outer_join_rels = bms_add_member(root->outer_join_rels,
@@ -1087,7 +1134,7 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
process_security_barrier_quals(root,
varno,
jtitem->qualscope,
- jtitem->below_outer_join);
+ jtitem->jdomain);
}
else if (IsA(jtnode, FromExpr))
{
@@ -1105,7 +1152,7 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
if (bms_is_subset(pq->relids, jtitem->qualscope))
distribute_qual_to_rels(root, pq->qual,
- jtitem->below_outer_join,
+ jtitem->jdomain,
NULL,
root->qual_security_level,
jtitem->qualscope, NULL, NULL,
@@ -1120,7 +1167,7 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
* Now process the top-level quals.
*/
distribute_quals_to_rels(root, (List *) f->quals,
- jtitem->below_outer_join,
+ jtitem->jdomain,
NULL,
root->qual_security_level,
jtitem->qualscope, NULL, NULL,
@@ -1221,7 +1268,7 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
/* Process the JOIN's qual clauses */
distribute_quals_to_rels(root, my_quals,
- jtitem->below_outer_join,
+ jtitem->jdomain,
sjinfo,
root->qual_security_level,
jtitem->qualscope,
@@ -1258,7 +1305,7 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
static void
process_security_barrier_quals(PlannerInfo *root,
int rti, Relids qualscope,
- bool below_outer_join)
+ JoinDomain *jdomain)
{
RangeTblEntry *rte = root->simple_rte_array[rti];
Index security_level = 0;
@@ -1281,7 +1328,7 @@ process_security_barrier_quals(PlannerInfo *root,
* pushed up to top of tree, which we don't want.
*/
distribute_quals_to_rels(root, qualset,
- below_outer_join,
+ jdomain,
NULL,
security_level,
qualscope,
@@ -1991,7 +2038,7 @@ deconstruct_distribute_oj_quals(PlannerInfo *root,
is_clone = !has_clone;
distribute_quals_to_rels(root, quals,
- true,
+ otherjtitem->jdomain,
sjinfo,
root->qual_security_level,
this_qualscope,
@@ -2020,7 +2067,7 @@ deconstruct_distribute_oj_quals(PlannerInfo *root,
{
/* No commutation possible, just process the postponed clauses */
distribute_quals_to_rels(root, jtitem->oj_joinclauses,
- true,
+ jtitem->jdomain,
sjinfo,
root->qual_security_level,
qualscope,
@@ -2045,7 +2092,7 @@ deconstruct_distribute_oj_quals(PlannerInfo *root,
*/
static void
distribute_quals_to_rels(PlannerInfo *root, List *clauses,
- bool below_outer_join,
+ JoinDomain *jdomain,
SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
@@ -2064,7 +2111,7 @@ distribute_quals_to_rels(PlannerInfo *root, List *clauses,
Node *clause = (Node *) lfirst(lc);
distribute_qual_to_rels(root, clause,
- below_outer_join,
+ jdomain,
sjinfo,
security_level,
qualscope,
@@ -2092,8 +2139,7 @@ distribute_quals_to_rels(PlannerInfo *root, List *clauses,
* These will be dealt with in later steps of deconstruct_jointree.
*
* 'clause': the qual clause to be distributed
- * 'below_outer_join': true if the qual is from a JOIN/ON that is below the
- * nullable side of a higher-level outer join
+ * 'jdomain': the join domain containing the clause
* 'sjinfo': join's SpecialJoinInfo (NULL for an inner join or WHERE clause)
* 'security_level': security_level to assign to the qual
* 'qualscope': set of base+OJ rels the qual's syntactic scope covers
@@ -2124,7 +2170,7 @@ distribute_quals_to_rels(PlannerInfo *root, List *clauses,
*/
static void
distribute_qual_to_rels(PlannerInfo *root, Node *clause,
- bool below_outer_join,
+ JoinDomain *jdomain,
SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
@@ -2196,12 +2242,8 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* RestrictInfo lists for the moment, but eventually createplan.c will
* pull it out and make a gating Result node immediately above whatever
* plan node the pseudoconstant clause is assigned to. It's usually best
- * to put a gating node as high in the plan tree as possible. If we are
- * not below an outer join, we can actually push the pseudoconstant qual
- * all the way to the top of the tree. If we are below an outer join, we
- * leave the qual at its original syntactic level (we could push it up to
- * just below the outer join, but that seems more complex than it's
- * worth).
+ * to put a gating node as high in the plan tree as possible, which we can
+ * do by assigning it the full relid set of the current JoinDomain.
*/
if (bms_is_empty(relids))
{
@@ -2211,25 +2253,20 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
relids = bms_copy(ojscope);
/* mustn't use as gating qual, so don't mark pseudoconstant */
}
- else
+ else if (contain_volatile_functions(clause))
{
/* eval at original syntactic level */
relids = bms_copy(qualscope);
- if (!contain_volatile_functions(clause))
- {
- /* mark as gating qual */
- pseudoconstant = true;
- /* tell createplan.c to check for gating quals */
- root->hasPseudoConstantQuals = true;
- /* if not below outer join, push it to top of tree */
- if (!below_outer_join)
- {
- relids =
- get_relids_in_jointree((Node *) root->parse->jointree,
- true, false);
- qualscope = bms_copy(relids);
- }
- }
+ /* again, can't mark pseudoconstant */
+ }
+ else
+ {
+ /* eval at join domain level */
+ relids = bms_copy(jdomain->jd_relids);
+ /* mark as gating qual */
+ pseudoconstant = true;
+ /* tell createplan.c to check for gating quals */
+ root->hasPseudoConstantQuals = true;
}
}
@@ -2319,23 +2356,8 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
if (check_redundant_nullability_qual(root, clause))
return;
- if (!allow_equivalence)
- {
- /* Caller says it mustn't become an equivalence class */
- maybe_equivalence = false;
- }
- else
- {
- /*
- * Consider feeding qual to the equivalence machinery. However,
- * if it's itself within an outer-join clause, treat it as though
- * it appeared below that outer join (note that we can only get
- * here when the clause references only nullable-side rels).
- */
- maybe_equivalence = true;
- if (outerjoin_nonnullable != NULL)
- below_outer_join = true;
- }
+ /* Feed qual to the equivalence machinery, if allowed by caller */
+ maybe_equivalence = allow_equivalence;
/*
* Since it doesn't mention the LHS, it's certainly not useful as a
@@ -2401,16 +2423,14 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
check_mergejoinable(restrictinfo);
/*
- * XXX rewrite:
- *
* If it is a true equivalence clause, send it to the EquivalenceClass
* machinery. We do *not* attach it directly to any restriction or join
* lists. The EC code will propagate it to the appropriate places later.
*
- * If the clause has a mergejoinable operator and is not
- * outerjoin-delayed, yet isn't an equivalence because it is an outer-join
- * clause, the EC code may yet be able to do something with it. We add it
- * to appropriate lists for further consideration later. Specifically:
+ * If the clause has a mergejoinable operator, yet isn't an equivalence
+ * because it is an outer-join clause, the EC code may still be able to do
+ * something with it. We add it to appropriate lists for further
+ * consideration later. Specifically:
*
* If it is a left or right outer-join qualification that relates the two
* sides of the outer join (no funny business like leftvar1 = leftvar2 +
@@ -2438,7 +2458,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
{
if (maybe_equivalence)
{
- if (process_equivalence(root, &restrictinfo, below_outer_join))
+ if (process_equivalence(root, &restrictinfo, jdomain))
return;
/* EC rejected it, so set left_ec/right_ec the hard way ... */
if (restrictinfo->mergeopfamilies) /* EC might have changed this */
@@ -2628,8 +2648,9 @@ distribute_restrictinfo_to_rels(PlannerInfo *root,
* "qualscope" is the nominal syntactic level to impute to the restrictinfo.
* This must contain at least all the rels used in the expressions, but it
* is used only to set the qual application level when both exprs are
- * variable-free. Otherwise the qual is applied at the lowest join level
- * that provides all its variables.
+ * variable-free. (Hence, it should usually match the join domain in which
+ * the clause applies.) Otherwise the qual is applied at the lowest join
+ * level that provides all its variables.
*
* "security_level" is the security level to assign to the new restrictinfo.
*
@@ -2657,7 +2678,6 @@ process_implied_equality(PlannerInfo *root,
Expr *item2,
Relids qualscope,
Index security_level,
- bool below_outer_join,
bool both_const)
{
RestrictInfo *restrictinfo;
@@ -2706,27 +2726,16 @@ process_implied_equality(PlannerInfo *root,
/*
* If the clause is variable-free, our normal heuristic for pushing it
* down to just the mentioned rels doesn't work, because there are none.
- * Apply at the given qualscope, or at the top of tree if it's nonvolatile
- * (which it very likely is, but we'll check, just to be sure).
+ * Apply it as a gating qual at the given qualscope.
*/
if (bms_is_empty(relids))
{
- /* eval at original syntactic level */
+ /* eval at join domain level */
relids = bms_copy(qualscope);
- if (!contain_volatile_functions(clause))
- {
- /* mark as gating qual */
- pseudoconstant = true;
- /* tell createplan.c to check for gating quals */
- root->hasPseudoConstantQuals = true;
- /* if not below outer join, push it to top of tree */
- if (!below_outer_join)
- {
- relids =
- get_relids_in_jointree((Node *) root->parse->jointree,
- true, false);
- }
- }
+ /* mark as gating qual */
+ pseudoconstant = true;
+ /* tell createplan.c to check for gating quals */
+ root->hasPseudoConstantQuals = true;
}
/*
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 00b02d4e91..c6cd02d4ec 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -623,6 +623,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
root->init_plans = NIL;
root->cte_plan_ids = NIL;
root->multiexpr_params = NIL;
+ root->join_domains = NIL;
root->eq_classes = NIL;
root->ec_merging_done = false;
root->last_rinfo_serial = 0;
@@ -650,6 +651,13 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
root->non_recursive_path = NULL;
root->partColsUpdated = false;
+ /*
+ * Create the top-level join domain. This won't have valid contents until
+ * deconstruct_jointree fills it in, but the node needs to exist before
+ * that so we can build EquivalenceClasses referencing it.
+ */
+ root->join_domains = list_make1(makeNode(JoinDomain));
+
/*
* If there is a WITH list, process each WITH query and either convert it
* to RTE_SUBQUERY RTE(s) or build an initplan SubPlan structure for it.
@@ -6242,6 +6250,7 @@ plan_cluster_use_sort(Oid tableOid, Oid indexOid)
root->query_level = 1;
root->planner_cxt = CurrentMemoryContext;
root->wt_param_id = -1;
+ root->join_domains = list_make1(makeNode(JoinDomain));
/* Build a minimal RTE for the rel */
rte = makeNode(RangeTblEntry);
@@ -6363,6 +6372,7 @@ plan_create_index_workers(Oid tableOid, Oid indexOid)
root->query_level = 1;
root->planner_cxt = CurrentMemoryContext;
root->wt_param_id = -1;
+ root->join_domains = list_make1(makeNode(JoinDomain));
/*
* Build a minimal RTE.
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 50018d0e85..48436fe15c 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -991,6 +991,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
subroot->init_plans = NIL;
subroot->cte_plan_ids = NIL;
subroot->multiexpr_params = NIL;
+ subroot->join_domains = NIL;
subroot->eq_classes = NIL;
subroot->ec_merging_done = false;
subroot->last_rinfo_serial = 0;
@@ -1010,6 +1011,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
subroot->hasRecursion = false;
subroot->wt_param_id = -1;
subroot->non_recursive_path = NULL;
+ /* We don't currently need a top JoinDomain for the subroot */
/* No CTEs to worry about */
Assert(subquery->cteList == NIL);
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index e759f99161..b0abfe1374 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -307,6 +307,9 @@ struct PlannerInfo
/* List of Lists of Params for MULTIEXPR subquery outputs */
List *multiexpr_params;
+ /* list of JoinDomains used in the query (higher ones first) */
+ List *join_domains;
+
/* list of active EquivalenceClasses */
List *eq_classes;
@@ -1250,11 +1253,46 @@ typedef struct StatisticExtInfo
List *exprs;
} StatisticExtInfo;
+/*
+ * JoinDomains
+ *
+ * A "join domain" defines the scope of applicability of deductions made via
+ * the EquivalenceClass mechanism. Roughly speaking, a join domain is a set
+ * of base+OJ relations that are inner-joined together. More precisely, it is
+ * the set of relations at which equalities deduced from an EquivalenceClass
+ * can be enforced or should be expected to hold. The topmost JoinDomain
+ * covers the whole query (so its jd_relids should equal all_query_rels).
+ * An outer join creates a new JoinDomain that includes all base+OJ relids
+ * within its nullable side, but (by convention) not the OJ's own relid.
+ * A FULL join creates two new JoinDomains, one for each side.
+ *
+ * Notice that a rel that is below outer join(s) will thus appear to belong
+ * to multiple join domains. However, any of its Vars that appear in
+ * EquivalenceClasses belonging to higher join domains will have nullingrel
+ * bits preventing them from being evaluated at the rel's scan level, so that
+ * we will not be able to derive enforceable-at-the-rel-scan-level clauses
+ * from such ECs. We define the join domain relid sets this way so that
+ * domains can be said to be "higher" or "lower" when one domain relid set
+ * includes another.
+ *
+ * The JoinDomains for a query are computed in deconstruct_jointree.
+ * We do not copy JoinDomain structs once made, so they can be compared
+ * for equality by simple pointer equality.
+ */
+typedef struct JoinDomain
+{
+ pg_node_attr(no_copy_equal, no_read)
+
+ NodeTag type;
+
+ Relids jd_relids; /* all relids contained within the domain */
+} JoinDomain;
+
/*
* EquivalenceClasses
*
- * Whenever we can determine that a mergejoinable equality clause A = B is
- * not delayed by any outer join, we create an EquivalenceClass containing
+ * Whenever we identify a mergejoinable equality clause A = B that is
+ * not an outer-join clause, we create an EquivalenceClass containing
* the expressions A and B to record this knowledge. If we later find another
* equivalence B = C, we add C to the existing EquivalenceClass; this may
* require merging two existing EquivalenceClasses. At the end of the qual
@@ -1268,6 +1306,18 @@ typedef struct StatisticExtInfo
* that all or none of the input datatypes are collatable, so that a single
* collation value is sufficient.)
*
+ * Strictly speaking, deductions from an EquivalenceClass hold only within
+ * a "join domain", that is a set of relations that are innerjoined together
+ * (see JoinDomain above). For the most part we don't need to account for
+ * this explicitly, because equality clauses from different join domains
+ * will contain Vars that are not equal() because they have different
+ * nullingrel sets, and thus we will never falsely merge ECs from different
+ * join domains. But Var-free (pseudoconstant) expressions lack that safety
+ * feature. We handle that by marking "const" EC members with the JoinDomain
+ * of the clause they came from; two nominally-equal const members will be
+ * considered different if they came from different JoinDomains. This ensures
+ * no false EquivalenceClass merges will occur.
+ *
* We also use EquivalenceClasses as the base structure for PathKeys, letting
* us represent knowledge about different sort orderings being equivalent.
* Since every PathKey must reference an EquivalenceClass, we will end up
@@ -1282,11 +1332,6 @@ typedef struct StatisticExtInfo
* entry: consider SELECT random() AS a, random() AS b ... ORDER BY b,a.
* So we record the SortGroupRef of the originating sort clause.
*
- * We allow equality clauses appearing below the nullable side of an outer join
- * to form EquivalenceClasses, but these have a slightly different meaning:
- * the included values might be all NULL rather than all the same non-null
- * values. See src/backend/optimizer/README for more on that point.
- *
* NB: if ec_merged isn't NULL, this class has been merged into another, and
* should be ignored in favor of using the pointed-to class.
*
@@ -1311,7 +1356,6 @@ typedef struct EquivalenceClass
* for child members (see below) */
bool ec_has_const; /* any pseudoconstants in ec_members? */
bool ec_has_volatile; /* the (sole) member is a volatile expr */
- bool ec_below_outer_join; /* equivalence applies below an OJ */
bool ec_broken; /* failed to generate needed clauses? */
Index ec_sortref; /* originating sortclause label, or 0 */
Index ec_min_security; /* minimum security_level in ec_sources */
@@ -1320,11 +1364,11 @@ typedef struct EquivalenceClass
} EquivalenceClass;
/*
- * If an EC contains a const and isn't below-outer-join, any PathKey depending
- * on it must be redundant, since there's only one possible value of the key.
+ * If an EC contains a constant, any PathKey depending on it must be
+ * redundant, since there's only one possible value of the key.
*/
#define EC_MUST_BE_REDUNDANT(eclass) \
- ((eclass)->ec_has_const && !(eclass)->ec_below_outer_join)
+ ((eclass)->ec_has_const)
/*
* EquivalenceMember - one member expression of an EquivalenceClass
@@ -1359,6 +1403,7 @@ typedef struct EquivalenceMember
bool em_is_const; /* expression is pseudoconstant? */
bool em_is_child; /* derived version for a child relation? */
Oid em_datatype; /* the "nominal type" used by the opfamily */
+ JoinDomain *em_jdomain; /* join domain containing the source clause */
/* if em_is_child is true, this links to corresponding EM for top parent */
struct EquivalenceMember *em_parent pg_node_attr(read_write_ignore);
} EquivalenceMember;
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 03866de136..6c68ca1f47 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -122,7 +122,7 @@ typedef bool (*ec_matches_callback_type) (PlannerInfo *root,
extern bool process_equivalence(PlannerInfo *root,
RestrictInfo **p_restrictinfo,
- bool below_outer_join);
+ JoinDomain *jdomain);
extern Expr *canonicalize_ec_expression(Expr *expr,
Oid req_type, Oid req_collation);
extern void reconsider_outer_join_clauses(PlannerInfo *root);
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index 57b963c0f7..6140dc92b9 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -84,7 +84,6 @@ extern RestrictInfo *process_implied_equality(PlannerInfo *root,
Expr *item2,
Relids qualscope,
Index security_level,
- bool below_outer_join,
bool both_const);
extern RestrictInfo *build_implied_join_equality(PlannerInfo *root,
Oid opno,
On Fri, Dec 23, 2022 at 10:21 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Here's a new edition of this patch series.
I shoved some preliminary refactoring into the 0001 patch,
notably splitting deconstruct_jointree into two passes.
0002-0009 cover the same ground as they did before, though
with some differences in detail. 0010-0012 are new work
mostly aimed at removing kluges we no longer need.
There are two big areas that I would still like to improve, but
I think we've run out of time to address them in the v16 cycle:
* It'd be nice to apply the regular EquivalenceClass deduction
mechanisms to outer-join equalities, instead of the
reconsider_outer_join_clauses kluge. I've made several stabs at that
without much success. I think that the "join domain" framework added
in 0012 is likely to provide a workable foundation, but some more
effort is needed.
* I really want to get rid of RestrictInfo.is_pushed_down and
RINFO_IS_PUSHED_DOWN(), because those seem exceedingly awkward
and squishy. I've not gotten far with finding a better
replacement there, either.
Despite the work being unfinished, I feel that this has moved us a
long way towards outer-join handling being less of a jury-rigged
affair.
regards, tom lane
Hi,
For v8-0012-invent-join-domains.patch, in `distribute_qual_to_rels`, it seems that `pseudoconstant` and `root->hasPseudoConstantQuals` carry the same value.
Can the variable `pseudoconstant` be omitted ?
Cheers
Ted Yu <yuzhihong@gmail.com> writes:
> For v8-0012-invent-join-domains.patch, in `distribute_qual_to_rels`, it
> seems that `pseudoconstant` and `root->hasPseudoConstantQuals` carry the
> same value.
> Can the variable `pseudoconstant` be omitted ?
Surely not. 'pseudoconstant' tells whether the current qual clause
is pseudoconstant. root->hasPseudoConstantQuals remembers whether
we have found any pseudoconstant qual in the query.
regards, tom lane
On Sat, Dec 24, 2022 at 2:20 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
I shoved some preliminary refactoring into the 0001 patch,
notably splitting deconstruct_jointree into two passes.
0002-0009 cover the same ground as they did before, though
with some differences in detail. 0010-0012 are new work
mostly aimed at removing kluges we no longer need.
I'm looking at 0010-0012 and I really like the changes and removals
there. Thanks for the great work!
For 0010, the change seems quite independent. I think maybe we can
apply it to HEAD directly.
For 0011, I found that some clauses that were outerjoin_delayed and thus
not equivalent before might be treated as being equivalent now. For
example
explain (costs off)
select * from a left join b on a.i = b.i where coalesce(b.j, 0) = 0 and coalesce(b.j, 0) = a.j;
QUERY PLAN
----------------------------------
Hash Right Join
Hash Cond: (b.i = a.i)
Filter: (COALESCE(b.j, 0) = 0)
-> Seq Scan on b
-> Hash
-> Seq Scan on a
Filter: (j = 0)
(7 rows)
This is different behavior from HEAD. But I think it's an improvement.
For 0012, I'm still trying to understand JoinDomain. AFAIU all EC
members of the same EC should have the same JoinDomain, because for
constants we match EC members only within the same JoinDomain, and for
Vars if they come from different join domains they will have different
nullingrels and thus will not match. So I wonder if we can have the
JoinDomain kept in EquivalenceClass rather than in each
EquivalenceMembers.
Thanks
Richard
there. Thanks for the great work!
For 0010, the change seems quite independent. I think maybe we can
apply it to HEAD directly.
For 0011, I found that some clauses that were outerjoin_delayed and thus
not equivalent before might be treated as being equivalent now. For
example
explain (costs off)
select * from a left join b on a.i = b.i where coalesce(b.j, 0) = 0 and coalesce(b.j, 0) = a.j;
QUERY PLAN
----------------------------------
Hash Right Join
Hash Cond: (b.i = a.i)
Filter: (COALESCE(b.j, 0) = 0)
-> Seq Scan on b
-> Hash
-> Seq Scan on a
Filter: (j = 0)
(7 rows)
This is different behavior from HEAD. But I think it's an improvement.
For 0012, I'm still trying to understand JoinDomain. AFAIU all EC
members of the same EC should have the same JoinDomain, because for
constants we match EC members only within the same JoinDomain, and for
Vars if they come from different join domains they will have different
nullingrels and thus will not match. So I wonder if we can have the
JoinDomain kept in EquivalenceClass rather than in each
EquivalenceMembers.
Thanks
Richard
Richard Guo <guofenglinux@gmail.com> writes:
> For 0012, I'm still trying to understand JoinDomain. AFAIU all EC
> members of the same EC should have the same JoinDomain, because for
> constants we match EC members only within the same JoinDomain, and for
> Vars if they come from different join domains they will have different
> nullingrels and thus will not match. So I wonder if we can have the
> JoinDomain kept in EquivalenceClass rather than in each
> EquivalenceMembers.
Yeah, I tried to do it like that at first, and failed. There is
some sort of association between ECs and join domains, for sure,
but exactly what it is seems to need more elucidation.
The thing that I couldn't get around before is that if you have,
say, a mergejoinable equality clause in an outer join:
select ... from a left join b on a.x = b.y;
that equality clause can only be associated with the join domain
for B, because it certainly can't be enforced against A. However,
you'd still wish to be able to do a mergejoin using indexes on
a.x and b.y, and this means that we have to understand the ordering
induced by a PathKey based on this EC as applicable to A, even
though that relation is not in the same join domain. So there are
situations where sort orderings apply across domain boundaries even
though equalities don't. We might have to split the notion of
EquivalenceClass into two sorts of objects, and somewhere right
about here is where I realized that this wasn't getting finished
for v16 :-(.
So the next pass at this is likely going to involve some more
refactoring, and maybe we'll end up saying that an EquivalenceClass
is tightly bound to a join domain or maybe we won't. For the moment
it seemed to work better to associate domains with only the const
members of ECs. (As written, the patch does fill em_jdomain even
for non-const members, but that was just for simplicity. I'd
originally meant to make it NULL for non-const members, but that
turned out to be a bit too tedious because the responsibility for
marking a member as const or not is split among several places.)
Another part of the motivation for doing it like that is that
I've been thinking about having just a single common pool of
EquivalenceMember objects, and turning EquivalenceClasses into
bitmapsets of indexes into the shared EquivalenceMember list.
This would support having ECs that share some member(s) without
being exactly the same thing, which I think might be necessary
to get to the point of treating outer-join clauses as creating
EC equalities.
BTW, I can't escape the suspicion that I've reinvented an idea
that's already well known in the literature. Has anyone seen
something like this "join domain" concept before, and if so
what was it called?
regards, tom lane
On Tue, Dec 27, 2022 at 11:31 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
The thing that I couldn't get around before is that if you have,
say, a mergejoinable equality clause in an outer join:
select ... from a left join b on a.x = b.y;
that equality clause can only be associated with the join domain
for B, because it certainly can't be enforced against A. However,
you'd still wish to be able to do a mergejoin using indexes on
a.x and b.y, and this means that we have to understand the ordering
induced by a PathKey based on this EC as applicable to A, even
though that relation is not in the same join domain. So there are
situations where sort orderings apply across domain boundaries even
though equalities don't. We might have to split the notion of
EquivalenceClass into two sorts of objects, and somewhere right
about here is where I realized that this wasn't getting finished
for v16 :-(.
I think I see where the problem is. And I can see currently in
get_eclass_for_sort_expr we always use the top JoinDomain. So although
the equality clause 'a.x = b.y' belongs to JoinDomain {B}, we set up ECs
for 'a.x' and 'b.y' that belong to the top JoinDomain {A, B, A/B}.
But doing so would lead to a situation where the "same" Vars from
different join domains might have the same varnullingrels and thus would
match by equal(). As an example, consider
select ... from a left join b on a.x = b.y where a.x = 1;
As said we would set up EC for 'b.y' as belonging to the top JoinDomain.
Then when reconsider_outer_join_clause generates the equality clause
'b.y = 1', we figure out that the new clause belongs to JoinDomain {B}.
Note that the two 'b.y' here belong to different join domains but they
have the same varnullingrels (empty varnullingrels actually). As a
result, the equality 'b.y = 1' would be merged into the existing EC for
'b.y', because the two 'b.y' matches by equal() and we do not check
JoinDomain for non-const EC members. So we would end up with an EC
containing EC members of different join domains.
And it seems this would make the following statement in README not hold
any more.
We don't have to worry about this for Vars (or expressions
containing Vars), because references to the "same" column from
different join domains will have different varnullingrels and thus
won't be equal() anyway.
Thanks
Richard
get_eclass_for_sort_expr we always use the top JoinDomain. So although
the equality clause 'a.x = b.y' belongs to JoinDomain {B}, we set up ECs
for 'a.x' and 'b.y' that belong to the top JoinDomain {A, B, A/B}.
But doing so would lead to a situation where the "same" Vars from
different join domains might have the same varnullingrels and thus would
match by equal(). As an example, consider
select ... from a left join b on a.x = b.y where a.x = 1;
As said we would set up EC for 'b.y' as belonging to the top JoinDomain.
Then when reconsider_outer_join_clause generates the equality clause
'b.y = 1', we figure out that the new clause belongs to JoinDomain {B}.
Note that the two 'b.y' here belong to different join domains but they
have the same varnullingrels (empty varnullingrels actually). As a
result, the equality 'b.y = 1' would be merged into the existing EC for
'b.y', because the two 'b.y' matches by equal() and we do not check
JoinDomain for non-const EC members. So we would end up with an EC
containing EC members of different join domains.
And it seems this would make the following statement in README not hold
any more.
We don't have to worry about this for Vars (or expressions
containing Vars), because references to the "same" column from
different join domains will have different varnullingrels and thus
won't be equal() anyway.
Thanks
Richard
Richard Guo <guofenglinux@gmail.com> writes:
> I think I see where the problem is. And I can see currently in
> get_eclass_for_sort_expr we always use the top JoinDomain. So although
> the equality clause 'a.x = b.y' belongs to JoinDomain {B}, we set up ECs
> for 'a.x' and 'b.y' that belong to the top JoinDomain {A, B, A/B}.
Yeah, that's a pretty squishy point, and likely wrong in detail.
If we want to associate an EC with the sort order of an index on
b.y, we almost certainly want that EC to belong to join domain {B}.
I had tried to do that in an earlier iteration of 0012, by dint of
adding a JoinDomain argument to get_eclass_for_sort_expr and then
having build_index_pathkeys specify the lowest join domain containing
the index's relation. It did not work very well: it couldn't generate
mergejoins on full-join clauses, IIRC.
Maybe some variant on that plan can be made to fly, but I'm not at
all clear on what needs to be adjusted. Anyway, that's part of why
I backed off on the notion of explicitly associating ECs with join
domains. As long as we only pay attention to the join domain in
connection with constants, get_eclass_for_sort_expr can get away with
just using the top join domain, because we'd never apply it to a
constant unless perhaps somebody manages to ORDER BY or GROUP BY a
constant, and in those cases the top domain really is the right one.
(It's syntactically difficult to write such a thing anyway, but not
impossible.)
I think this is sort of an intermediate state, and hopefully a
year from now we'll have a better idea of how to manage all that.
What I mainly settled for doing in 0012 was getting rid of the
"below outer join" concept for ECs, because having to identify
a value for that had been giving me fits in several previous
attempts at extending ECs to cover outer-join equalities.
I think that the JoinDomain concept will offer a better-defined
replacement.
regards, tom lane
The cfbot shows that this needs to be rebased over 8eba3e3f0.
(Just code motion, no interesting changes.)
Richard, are you planning to review this any more? I'm getting
a little antsy to get it committed. For such a large patch,
it's surprising it's had so few conflicts to date.
regards, tom lane
commit a02e22407fc3215896efe7b6e5063ba272ca02ee
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Jan 23 14:09:39 2023 -0500
Add overview documentation.
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index 41c120e0cd..191ad4c457 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -295,6 +295,239 @@ Therefore, we don't merge FROM-lists if the result would have too many
FROM-items in one list.
+Vars and PlaceHolderVars
+------------------------
+
+A Var node is simply the parse-tree representation of a table column
+reference. However, in the presence of outer joins, that concept is
+more subtle than it might seem. We need to distinguish the values of
+a Var "above" and "below" any outer join that could force the Var to
+null. As an example, consider
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE foo(t2.z)
+
+(Assume foo() is not strict, so that we can't reduce the left join to
+a plain join.) A naive implementation might try to push the foo(t2.z)
+call down to the scan of t2, but that is not correct because
+(a) what foo() should actually see for a null-extended join row is NULL,
+and (b) if foo() returns false, we should suppress the t1 row from the
+join altogether, not emit it with a null-extended t2 row. On the other
+hand, it *would* be correct (and desirable) to push that call down to
+the scan level if the query were
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y AND foo(t2.z))
+
+This motivates considering "t2.z" within the left join's ON clause
+to be a different value from "t2.z" outside the JOIN clause. The
+former can be identified with t2.z as seen at the relation scan level,
+but the latter can't.
+
+Another example occurs in connection with EquivalenceClasses (discussed
+below). Given
+
+ SELECT * FROM t1 LEFT JOIN t2 ON (t1.x = t2.y) WHERE t1.x = 42
+
+we would like to use the EquivalenceClass mechanisms to derive "t2.y = 42"
+to use as a restriction clause for the scan of t2. (That works, because t2
+rows having y different from 42 cannot affect the query result.) However,
+it'd be wrong to conclude that t2.y will be equal to t1.x in every joined
+row. Part of the solution to this problem is to deem that "t2.y" in the
+ON clause refers to the relation-scan-level value of t2.y, but not to the
+value that y will have in joined rows, where it might be NULL rather than
+equal to t1.x.
+
+Therefore, Var nodes are decorated with "varnullingrels", which are sets
+of the rangetable indexes of outer joins that potentially null the Var
+at the point where it appears in the query. (Using a set, not an ordered
+list, is fine since it doesn't matter which join forced the value to null;
+and that avoids having to change the representation when we consider
+different outer-join orders.) In the examples above, all occurrences of
+t1.x would have empty varnullingrels, since the left join doesn't null t1.
+The t2 references within the JOIN ON clauses would also have empty
+varnullingrels. But outside the JOIN clauses, any Vars referencing t2
+would have varnullingrels containing the index of the JOIN's rangetable
+entry (RTE), so that they'd be understood as potentially different from
+the t2 values seen at scan level. Labeling t2.z in the WHERE clause with
+the JOIN's RT index lets us recognize that that occurrence of foo(t2.z)
+cannot be pushed down to the t2 scan level: we cannot evaluate that value
+at the scan level, but only after the join has been done.
+
+For LEFT and RIGHT outer joins, only Vars coming from the nullable side
+of the join are marked with that join's RT index. For FULL joins, Vars
+from both inputs are marked. (Such marking doesn't let us tell which
+side of the full join a Var came from; but that information can be found
+elsewhere at need.)
+
+Notionally, a Var having nonempty varnullingrels can be thought of as
+ CASE WHEN any-of-these-outer-joins-produced-a-null-extended-row
+ THEN NULL
+ ELSE the-scan-level-value-of-the-column
+ END
+It's only notional, because no such calculation is ever done explicitly.
+In a finished plan, Vars occurring in scan-level plan nodes represent
+the actual table column values, but upper-level Vars are always
+references to outputs of lower-level plan nodes. When a join node emits
+a null-extended row, it just returns nulls for the relevant output
+columns rather than copying up values from its input. Because we don't
+ever have to do this calculation explicitly, it's not necessary to
+distinguish which side of an outer join got null-extended, which'd
+otherwise be essential information for FULL JOIN cases.
+
+Outer join identity 3 (discussed above) complicates this picture
+a bit. In the form
+ A leftjoin (B leftjoin C on (Pbc)) on (Pab)
+all of the Vars in clauses Pbc and Pab will have empty varnullingrels,
+but if we start with
+ (A leftjoin B on (Pab)) leftjoin C on (Pbc)
+then the parser will have marked Pbc's B Vars with the A/B join's
+RT index, making this form artificially different from the first.
+For discussion's sake, let's denote this marking with a star:
+ (A leftjoin B on (Pab)) leftjoin C on (Pb*c)
+To cope with this, once we have detected that commuting these joins
+is legal, we generate both the Pbc and Pb*c forms of that ON clause,
+by either removing or adding the first join's RT index in the B Vars
+that the parser created. While generating paths for a plan step that
+joins B and C, we include as a relevant join qual only the form that
+is appropriate depending on whether A has already been joined to B.
+
+It's also worth noting that identity 3 makes "the left join's RT index"
+itself a bit of a fuzzy concept, since the syntactic scope of each join
+RTE will depend on which form was produced by the parser. We resolve
+this by considering that a left join's identity is determined by its
+minimum set of right-hand-side input relations. In both forms allowed
+by identity 3, we can identify the first join as having minimum RHS B
+and the second join as having minimum RHS C.
+
+Another thing to notice is that C Vars appearing outside the nested
+JOIN clauses will be marked as nulled by both left joins if the
+original parser input was in the first form of identity 3, but if the
+parser input was in the second form, such Vars will only be marked as
+nulled by the second join. This is not really a semantic problem:
+such Vars will be marked the same way throughout the upper part of the
+query, so they will all look equal() which is correct; and they will not
+look equal() to any C Var appearing in the JOIN ON clause or below these
+joins. However, when building Vars representing the outputs of join
+relations, we need to ensure that their varnullingrels are set to
+values consistent with the syntactic join order, so that they will
+appear equal() to pre-existing Vars in the upper part of the query.
+
+Outer joins also complicate handling of subquery pull-up. Consider
+
+ SELECT ..., ss.x FROM tab1
+ LEFT JOIN (SELECT *, 42 AS x FROM tab2) ss ON ...
+
+We want to be able to pull up the subquery as discussed previously,
+but we can't just replace the "ss.x" Var in the top-level SELECT list
+with the constant 42. That'd result in always emitting 42, rather
+than emitting NULL in null-extended join rows.
+
+To solve this, we introduce the concept of PlaceHolderVars.
+A PlaceHolderVar is somewhat like a Var, in that its value originates
+at a relation scan level and can then be forced to null by higher-level
+outer joins; hence PlaceHolderVars carry a set of nulling rel IDs just
+like Vars. Unlike a Var, whose original value comes from a table,
+a PlaceHolderVar's original value is defined by a query-determined
+expression ("42" in this example); so we represent the PlaceHolderVar
+as a node with that expression as child. We insert a PlaceHolderVar
+whenever subquery pullup needs to replace a subquery-referencing Var
+that has nonempty varnullingrels with an expression that is not simply a
+Var. (When the replacement expression is a pulled-up Var, we can just
+add the replaced Var's varnullingrels to its set. Also, if the replaced
+Var has empty varnullingrels, we don't need a PlaceHolderVar: there is
+nothing that'd force the value to null, so the pulled-up expression is
+fine to use as-is.) In a finished plan, a PlaceHolderVar becomes just
+the contained expression at whatever plan level it's supposed to be
+evaluated at, and then upper-level occurrences are replaced by Var
+references to that output column of the lower plan level. That causes
+the value to go to null when appropriate at an outer join, in the same
+way as for normal Vars. Thus, PlaceHolderVars are never seen outside
+the planner.
+
+PlaceHolderVars (PHVs) are more complicated than Vars in another way:
+their original value might need to be calculated at a join, not a
+base-level relation scan. This can happen when a pulled-up subquery
+contains a join. Because of this, a PHV can create a join order
+constraint that wouldn't otherwise exist, to ensure that it can
+be calculated before it is used. A PHV's expression can also contain
+LATERAL references, adding complications that are discussed below.
+
+
+Relation Identification and Qual Clause Placement
+-------------------------------------------------
+
+A qual clause obtained from WHERE or JOIN/ON can be enforced at the lowest
+scan or join level that includes all relations used in the clause. For
+this purpose we consider that outer joins listed in varnullingrels or
+phnullingrels are used in the clause, since we can't compute the qual's
+result correctly until we know whether such Vars have gone to null.
+
+The one exception to this general rule is that a non-degenerate outer
+JOIN/ON qual (one that references the non-nullable side of the join)
+cannot be enforced below that join, even if it doesn't reference the
+nullable side. Pushing it down into the non-nullable side would result
+in rows disappearing from the join's result, rather than appearing as
+null-extended rows. To handle that, when we identify such a qual we
+artificially add the join's minimum input relid set to the set of
+relations it is considered to use, forcing it to be evaluated exactly at
+that join level. The same happens for outer-join quals that mention no
+relations at all.
+
+When attaching a qual clause to a join plan node that is performing an
+outer join, the qual clause is considered a "join clause" (that is, it is
+applied before the join performs null-extension) if it does not reference
+that outer join in any varnullingrels or phnullingrels set, or a "filter
+clause" (applied after null-extension) if it does reference that outer
+join. A qual clause that originally appeared in that outer join's JOIN/ON
+will fall into the first category, since the parser would not have marked
+any of its Vars as referencing the outer join. A qual clause that
+originally came from some upper ON clause or WHERE clause will be seen as
+referencing the outer join if it references any of the nullable side's
+Vars, since those Vars will be so marked by the parser. But, if such a
+qual does not reference any nullable-side Vars, it's okay to push it down
+into the non-nullable side, so it won't get attached to the join node in
+the first place.
+
+These things lead us to identify join relations within the planner
+by the sets of base relation RT indexes plus outer join RT indexes
+that they include. In that way, the sets of relations used by qual
+clauses can be directly compared to join relations' relid sets to
+see where to place the clauses. These identifying sets are unique
+because, for any given collection of base relations, there is only
+one valid set of outer joins to have performed along the way to
+joining that set of base relations (although the order of applying
+them could vary, as discussed above).
+
+SEMI joins do not have RT indexes, because they are artifacts made by
+the planner rather than the parser. (We could create rangetable
+entries for them, but there seems no need at present.) This does not
+cause a problem for qual placement, because the nullable side of a
+semijoin is not referenceable from above the join, so there is never a
+need to cite it in varnullingrels or phnullingrels. It does not cause a
+problem for join relation identification either, since whether a semijoin
+has been completed is again implicit in the set of base relations
+included in the join.
+
+There is one additional complication for qual clause placement, which
+occurs when we have made multiple versions of an outer-join clause as
+described previously (that is, we have both "Pbc" and "Pb*c" forms of
+the same clause seen in outer join identity 3). When forming an outer
+join we only want to apply one of the redundant versions of the clause.
+If we are forming the B/C join without having yet computed the A/B
+join, it's easy to reject the "Pb*c" form since its required relid
+set includes the A/B join relid which is not in the input. However,
+if we form B/C after A/B, then both forms of the clause are applicable
+so far as that test can tell. We have to look more closely to notice
+that the "Pbc" clause form refers to relation B which is no longer
+directly accessible. While this check is straightforward, it's not
+especially cheap (see clause_is_computable_at()). To avoid doing it
+unnecessarily, we mark the variant versions of a redundant clause as
+either "has_clone" or "is_clone". When considering a clone clause,
+we must check clause_is_computable_at() to disentangle which version
+to apply at the current join level. (In debug builds, we also Assert
+that non-clone clauses are validly computable at the current level;
+but that seems too expensive for production usage.)
+
+
Optimizer Functions
-------------------
@@ -437,11 +670,10 @@ inputs.
EquivalenceClasses
------------------
-During the deconstruct_jointree() scan of the query's qual clauses, we look
-for mergejoinable equality clauses A = B whose applicability is not delayed
-by an outer join; these are called "equivalence clauses". When we find
-one, we create an EquivalenceClass containing the expressions A and B to
-record this knowledge. If we later find another equivalence clause B = C,
+During the deconstruct_jointree() scan of the query's qual clauses, we
+look for mergejoinable equality clauses A = B. When we find one, we
+create an EquivalenceClass containing the expressions A and B to record
+that they are equal. If we later find another equivalence clause B = C,
we add C to the existing EquivalenceClass for {A B}; this may require
merging two existing EquivalenceClasses. At the end of the scan, we have
sets of values that are known all transitively equal to each other. We can
@@ -473,15 +705,89 @@ asserts that at any plan node where more than one of its member values
can be computed, output rows in which the values are not all equal may
be discarded without affecting the query result. (We require all levels
of the plan to enforce EquivalenceClasses, hence a join need not recheck
-equality of values that were computable by one of its children.) For an
-ordinary EquivalenceClass that is "valid everywhere", we can further infer
-that the values are all non-null, because all mergejoinable operators are
-strict. However, we also allow equivalence clauses that appear below the
-nullable side of an outer join to form EquivalenceClasses; for these
-classes, the interpretation is that either all the values are equal, or
-all (except pseudo-constants) have gone to null. (This requires a
-limitation that non-constant members be strict, else they might not go
-to null when the other members do.) Consider for example
+equality of values that were computable by one of its children.)
+
+Outer joins complicate this picture quite a bit, however. While we could
+theoretically use mergejoinable equality clauses that appear in outer-join
+conditions as sources of EquivalenceClasses, there's a serious difficulty:
+the resulting deductions are not valid everywhere. For example, given
+
+ SELECT * FROM a LEFT JOIN b ON (a.x = b.y AND a.x = 42);
+
+we can safely derive b.y = 42 and use that in the scan of B, because B
+rows not having b.y = 42 will not contribute to the join result. However,
+we cannot apply a.x = 42 at the scan of A, or we will remove rows that
+should appear in the join result. We could apply a.x = 42 as an outer join
+condition (and then it would be unnecessary to also check a.x = b.y).
+This is not yet implemented, however.
+
+A related issue is that constants appearing below an outer join are
+less constant than they appear. Ordinarily, if we find "A = 1" and
+"B = 1", it's okay to put A and B into the same EquivalenceClass.
+But consider
+
+ SELECT * FROM a
+ LEFT JOIN (SELECT * FROM b WHERE b.z = 1) ss ON (a.x = b.y)
+ WHERE a.x = 1;
+
+It would be a serious error to conclude that a.x = b.z, so we cannot
+form a single EquivalenceClass {a.x b.z 1}.
+
+This leads to considering EquivalenceClasses as applying within "join
+domains", which are sets of relations that are inner-joined to each other.
+(We can treat semijoins as if they were inner joins for this purpose.)
+There is a top-level join domain, and then each outer join in the query
+creates a new join domain comprising its nullable side. Full joins create
+two join domains, one for each side. EquivalenceClasses generated from
+WHERE are associated with the top-level join domain. EquivalenceClasses
+generated from the ON clause of an outer join are associated with the
+domain created by that outer join. EquivalenceClasses generated from the
+ON clause of an inner or semi join are associated with the syntactically
+most closely nested join domain.
+
+Having defined these domains, we can fix the not-so-constant-constants
+problem by considering that constants only match EquivalenceClass members
+when they come from clauses within the same join domain. In the above
+example, this means we keep {a.x 1} and {b.z 1} as separate
+EquivalenceClasses and don't erroneously merge them. We don't have to
+worry about this for Vars (or expressions containing Vars), because
+references to the "same" column from different join domains will have
+different varnullingrels and thus won't be equal() anyway.
+
+In the future, the join-domain concept may allow us to treat mergejoinable
+outer-join conditions as sources of EquivalenceClasses. The idea would be
+that conditions derived from such classes could only be enforced at scans
+or joins that are within the appropriate join domain. This is not
+implemented yet, however, as the details are trickier than they appear.
+
+Another instructive example is:
+
+ SELECT *
+ FROM a LEFT JOIN
+ (SELECT * FROM b JOIN c ON b.y = c.z WHERE b.y = 10) ss
+ ON a.x = ss.y
+ ORDER BY ss.y;
+
+We can form the EquivalenceClass {b.y c.z 10} and thereby apply c.z = 10
+while scanning C, as well as b.y = 10 while scanning B, so that no clause
+needs to be checked at the inner join. The left-join clause "a.x = ss.y"
+(really "a.x = b.y") is not considered an equivalence clause, so we do
+not insert a.x into that same EquivalenceClass; if we did, we'd falsely
+conclude a.x = 10. In the future though we might be able to do that,
+if we can keep from applying a.x = 10 at the scan of A, which in principle
+we could do by noting that the EquivalenceClass only applies within the
+{B,C} join domain.
+
+Also notice that ss.y in the ORDER BY is really b.y* (that is, the
+possibly-nulled form of b.y), so we will not confuse it with the b.y member
+of the lower EquivalenceClass. Thus, we won't mistakenly conclude that
+that ss.y is equal to a constant, which if true would lead us to think that
+sorting for the ORDER BY is unnecessary (see discussion of PathKeys below).
+Instead, there will be a separate EquivalenceClass containing only b.y*,
+which will form the basis for the PathKey describing the required sort
+order.
+
+Also consider this variant:
SELECT *
FROM a LEFT JOIN
@@ -489,27 +795,42 @@ to null when the other members do.) Consider for example
ON a.x = ss.y
WHERE a.x = 42;
-We can form the below-outer-join EquivalenceClass {b.y c.z 10} and thereby
-apply c.z = 10 while scanning c. (The reason we disallow outerjoin-delayed
-clauses from forming EquivalenceClasses is exactly that we want to be able
-to push any derived clauses as far down as possible.) But once above the
-outer join it's no longer necessarily the case that b.y = 10, and thus we
-cannot use such EquivalenceClasses to conclude that sorting is unnecessary
-(see discussion of PathKeys below).
-
-In this example, notice also that a.x = ss.y (really a.x = b.y) is not an
-equivalence clause because its applicability to b is delayed by the outer
-join; thus we do not try to insert b.y into the equivalence class {a.x 42}.
-But since we see that a.x has been equated to 42 above the outer join, we
-are able to form a below-outer-join class {b.y 42}; this restriction can be
-added because no b/c row not having b.y = 42 can contribute to the result
-of the outer join, and so we need not compute such rows. Now this class
-will get merged with {b.y c.z 10}, leading to the contradiction 10 = 42,
-which lets the planner deduce that the b/c join need not be computed at all
-because none of its rows can contribute to the outer join. (This gets
-implemented as a gating Result filter, since more usually the potential
-contradiction involves Param values rather than just Consts, and thus has
-to be checked at runtime.)
+We still form the EquivalenceClass {b.y c.z 10}, and additionally
+we have an EquivalenceClass {a.x 42} belonging to a different join domain.
+We cannot use "a.x = b.y" to merge these classes. However, we can compare
+that outer join clause to the existing EquivalenceClasses and form the
+derived clause "b.y = 42", which we can treat as a valid equivalence
+within the lower join domain (since no row of that domain not having
+b.y = 42 can contribute to the outer-join result). That makes the lower
+EquivalenceClass {42 b.y c.z 10}, resulting in the contradiction 10 = 42,
+which lets the planner deduce that the B/C join need not be computed at
+all: the result of that whole join domain can be forced to empty.
+(This gets implemented as a gating Result filter, since more usually the
+potential contradiction involves Param values rather than just Consts, and
+thus it has to be checked at runtime. We can use the join domain to
+determine the join level at which to place the gating condition.)
+
+There is an additional complication when re-ordering outer joins according
+to identity 3. Recall that the two choices we consider for such joins are
+
+ A leftjoin (B leftjoin C on (Pbc)) on (Pab)
+ (A leftjoin B on (Pab)) leftjoin C on (Pb*c)
+
+where the star denotes varnullingrels markers on B's Vars. When Pbc
+is (or includes) a mergejoinable clause, we have something like
+
+ A leftjoin (B leftjoin C on (b.b = c.c)) on (Pab)
+ (A leftjoin B on (Pab)) leftjoin C on (b.b* = c.c)
+
+We could generate an EquivalenceClause linking b.b and c.c, but if we
+then also try to link b.b* and c.c, we end with a nonsensical conclusion
+that b.b and b.b* are equal (at least in some parts of the plan tree).
+In any case, the conclusions we could derive from such a thing would be
+largely duplicative. Conditions involving b.b* can't be computed below
+this join nest, while any conditions that can be computed would be
+duplicative of what we'd get from the b.b/c.c combination. Therefore,
+we choose to generate an EquivalenceClause linking b.b and c.c, but
+"b.b* = c.c" is handled as just an ordinary clause.
To aid in determining the sort ordering(s) that can work with a mergejoin,
we mark each mergejoinable clause with the EquivalenceClasses of its left
@@ -522,7 +843,11 @@ if other equivalence clauses are later found to bear on the same
expressions.
Another way that we may form a single-item EquivalenceClass is in creation
-of a PathKey to represent a desired sort order (see below). This is a bit
+of a PathKey to represent a desired sort order (see below). This happens
+if an ORDER BY or GROUP BY key is not mentioned in any equivalence
+clause. We need to reason about sort orders in such queries, and our
+representation of sort ordering is a PathKey which depends on an
+EquivalenceClass, so we have to make an EquivalenceClass. This is a bit
different from the above cases because such an EquivalenceClass might
contain an aggregate function or volatile expression. (A clause containing
a volatile function will never be considered mergejoinable, even if its top
@@ -544,6 +869,9 @@ it's possible that it belongs to more than one. We keep track of all the
families to ensure that we can make use of an index belonging to any one of
the families for mergejoin purposes.)
+For the same sort of reason, an EquivalenceClass is also associated
+with a particular collation, if its datatype(s) care about collation.
+
An EquivalenceClass can contain "em_is_child" members, which are copies
of members that contain appendrel parent relation Vars, transposed to
contain the equivalent child-relation variables or expressions. These
@@ -579,7 +907,7 @@ Index scans have Path.pathkeys that represent the chosen index's ordering,
if any. A single-key index would create a single-PathKey list, while a
multi-column index generates a list with one element per key index column.
Non-key columns specified in the INCLUDE clause of covering indexes don't
-have corresponding PathKeys in the list, because the have no influence on
+have corresponding PathKeys in the list, because they have no influence on
index ordering. (Actually, since an index can be scanned either forward or
backward, there are two possible sort orders and two possible PathKey lists
it can generate.)
@@ -608,9 +936,14 @@ must now be ordered too. This is true even though we used neither an
explicit sort nor a mergejoin on Y. (Note: hash joins cannot be counted
on to preserve the order of their outer relation, because the executor
might decide to "batch" the join, so we always set pathkeys to NIL for
-a hashjoin path.) Exception: a RIGHT or FULL join doesn't preserve the
-ordering of its outer relation, because it might insert nulls at random
-points in the ordering.
+a hashjoin path.)
+
+An outer join doesn't preserve the ordering of its nullable input
+relation(s), because it might insert nulls at random points in the
+ordering. We don't need to think about this explicitly in the PathKey
+representation, because a PathKey representing a post-join variable
+will contain varnullingrel bits, making it not equal to a PathKey
+representing the pre-join value.
In general, we can justify using EquivalenceClasses as the basis for
pathkeys because, whenever we scan a relation containing multiple
@@ -655,14 +988,9 @@ redundancy, we save time and improve planning, since the planner will more
easily recognize equivalent orderings as being equivalent.
Another interesting property is that if the underlying EquivalenceClass
-contains a constant and is not below an outer join, then the pathkey is
-completely redundant and need not be sorted by at all! Every row must
-contain the same constant value, so there's no need to sort. (If the EC is
-below an outer join, we still have to sort, since some of the rows might
-have gone to null and others not. In this case we must be careful to pick
-a non-const member to sort by. The assumption that all the non-const
-members go to null at the same plan level is critical here, else they might
-not produce the same sort order.) This might seem pointless because users
+contains a constant, then the pathkey is completely redundant and need not
+be sorted by at all! Every interesting row must contain the same value,
+so there's no need to sort. This might seem pointless because users
are unlikely to write "... WHERE x = 42 ORDER BY x", but it allows us to
recognize when particular index columns are irrelevant to the sort order:
if we have "... WHERE x = 42 ORDER BY y", scanning an index on (x,y)
@@ -670,15 +998,6 @@ produces correctly ordered data without a sort step. We used to have very
ugly ad-hoc code to recognize that in limited contexts, but discarding
constant ECs from pathkeys makes it happen cleanly and automatically.
-You might object that a below-outer-join EquivalenceClass doesn't always
-represent the same values at every level of the join tree, and so using
-it to uniquely identify a sort order is dubious. This is true, but we
-can avoid dealing with the fact explicitly because we always consider that
-an outer join destroys any ordering of its nullable inputs. Thus, even
-if a path was sorted by {a.x} below an outer join, we'll re-sort if that
-sort ordering was important; and so using the same PathKey for both sort
-orderings doesn't create any real problem.
-
Order of processing for EquivalenceClasses and PathKeys
-------------------------------------------------------
commit 6d24e50c60cc3d9e2a4f0c8ea58f0fdfbadf313d
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Jan 23 14:20:40 2023 -0500
Do assorted preliminary refactoring.
This patch doesn't actually change any behavior, just lay some
fairly boring groundwork:
* Add Var.varnullingrels and PlaceHolderVar.phnullingrels fields.
These fields are always empty as of this commit, so they don't
affect any behavior, even though equal() will compare them.
Update backend/nodes/ and backend/rewrite/ infrastructure as needed.
* Refactor deconstruct_jointree() to split it into two passes;
the first pass computes relid bitmapsets and the second performs
qual distribution. The reason for this is that when we invent
"join domains", several patches later, we'll need to know the
full contents of the domains before distributing quals. It
seems like a good idea to split up the function anyway, as it's
gotten quite large.
* Move calculation of all_baserels to deconstruct_jointree,
because we'll soon need it to be available earlier.
(This means that remove_rel_from_query now has to update it.)
* Refactor to pass the relevant SpecialJoinInfo to
reconsider_outer_join_clause and reconsider_full_join_clause.
We'll need that later.
* Add some rewrite functions we'll need later for adding and
removing nullingrel bits in expression trees.
Note this will require a catversion bump when committed.
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index cad9f28ef5..fe67baf142 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -80,11 +80,13 @@ makeVar(int varno,
var->varlevelsup = varlevelsup;
/*
- * Only a few callers need to make Var nodes with varnosyn/varattnosyn
- * different from varno/varattno. We don't provide separate arguments for
- * them, but just initialize them to the given varno/varattno. This
- * reduces code clutter and chance of error for most callers.
+ * Only a few callers need to make Var nodes with non-null varnullingrels,
+ * or with varnosyn/varattnosyn different from varno/varattno. We don't
+ * provide separate arguments for them, but just initialize them to NULL
+ * and the given varno/varattno. This reduces code clutter and chance of
+ * error for most callers.
*/
+ var->varnullingrels = NULL;
var->varnosyn = (Index) varno;
var->varattnosyn = varattno;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 8fc24c882b..dc8415a693 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -2641,6 +2641,7 @@ expression_tree_mutator_impl(Node *node,
Var *newnode;
FLATCOPY(newnode, var, Var);
+ /* Assume we need not copy the varnullingrels bitmapset */
return (Node *) newnode;
}
break;
@@ -3234,7 +3235,7 @@ expression_tree_mutator_impl(Node *node,
FLATCOPY(newnode, phv, PlaceHolderVar);
MUTATE(newnode->phexpr, phv->phexpr, Expr *);
- /* Assume we need not copy the relids bitmapset */
+ /* Assume we need not copy the relids bitmapsets */
return (Node *) newnode;
}
break;
diff --git a/src/backend/nodes/queryjumblefuncs.c b/src/backend/nodes/queryjumblefuncs.c
index 16084842a3..8b101fc336 100644
--- a/src/backend/nodes/queryjumblefuncs.c
+++ b/src/backend/nodes/queryjumblefuncs.c
@@ -383,6 +383,11 @@ JumbleExpr(JumbleState *jstate, Node *node)
APP_JUMB(var->varno);
APP_JUMB(var->varattno);
APP_JUMB(var->varlevelsup);
+
+ /*
+ * We can omit varnullingrels, because it's fully determined
+ * by varno/varlevelsup plus the Var's query location.
+ */
}
break;
case T_Const:
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index c2fc568dc8..c56d3f926d 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -159,27 +159,6 @@ make_one_rel(PlannerInfo *root, List *joinlist)
Index rti;
double total_pages;
- /*
- * Construct the all_baserels Relids set.
- */
- root->all_baserels = NULL;
- for (rti = 1; rti < root->simple_rel_array_size; rti++)
- {
- RelOptInfo *brel = root->simple_rel_array[rti];
-
- /* there may be empty slots corresponding to non-baserel RTEs */
- if (brel == NULL)
- continue;
-
- Assert(brel->relid == rti); /* sanity check on array */
-
- /* ignore RTEs that are "other rels" */
- if (brel->reloptkind != RELOPT_BASEREL)
- continue;
-
- root->all_baserels = bms_add_member(root->all_baserels, brel->relid);
- }
-
/* Mark base rels as to whether we care about fast-start plans */
set_base_rel_consider_startup(root);
@@ -207,6 +186,7 @@ make_one_rel(PlannerInfo *root, List *joinlist)
{
RelOptInfo *brel = root->simple_rel_array[rti];
+ /* there may be empty slots corresponding to non-baserel RTEs */
if (brel == NULL)
continue;
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 7d7e6facdf..783ef5aa29 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -61,10 +61,10 @@ static RestrictInfo *create_join_clause(PlannerInfo *root,
EquivalenceMember *rightem,
EquivalenceClass *parent_ec);
static bool reconsider_outer_join_clause(PlannerInfo *root,
- RestrictInfo *rinfo,
+ OuterJoinClauseInfo *ojcinfo,
bool outer_on_left);
static bool reconsider_full_join_clause(PlannerInfo *root,
- RestrictInfo *rinfo);
+ OuterJoinClauseInfo *ojcinfo);
static Bitmapset *get_eclass_indexes_for_relids(PlannerInfo *root,
Relids relids);
static Bitmapset *get_common_eclass_indexes(PlannerInfo *root, Relids relids1,
@@ -1977,10 +1977,12 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* Process the LEFT JOIN clauses */
foreach(cell, root->left_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ OuterJoinClauseInfo *ojcinfo = (OuterJoinClauseInfo *) lfirst(cell);
- if (reconsider_outer_join_clause(root, rinfo, true))
+ if (reconsider_outer_join_clause(root, ojcinfo, true))
{
+ RestrictInfo *rinfo = ojcinfo->rinfo;
+
found = true;
/* remove it from the list */
root->left_join_clauses =
@@ -1996,10 +1998,12 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* Process the RIGHT JOIN clauses */
foreach(cell, root->right_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ OuterJoinClauseInfo *ojcinfo = (OuterJoinClauseInfo *) lfirst(cell);
- if (reconsider_outer_join_clause(root, rinfo, false))
+ if (reconsider_outer_join_clause(root, ojcinfo, false))
{
+ RestrictInfo *rinfo = ojcinfo->rinfo;
+
found = true;
/* remove it from the list */
root->right_join_clauses =
@@ -2015,10 +2019,12 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* Process the FULL JOIN clauses */
foreach(cell, root->full_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ OuterJoinClauseInfo *ojcinfo = (OuterJoinClauseInfo *) lfirst(cell);
- if (reconsider_full_join_clause(root, rinfo))
+ if (reconsider_full_join_clause(root, ojcinfo))
{
+ RestrictInfo *rinfo = ojcinfo->rinfo;
+
found = true;
/* remove it from the list */
root->full_join_clauses =
@@ -2035,21 +2041,21 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* Now, any remaining clauses have to be thrown back */
foreach(cell, root->left_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ OuterJoinClauseInfo *ojcinfo = (OuterJoinClauseInfo *) lfirst(cell);
- distribute_restrictinfo_to_rels(root, rinfo);
+ distribute_restrictinfo_to_rels(root, ojcinfo->rinfo);
}
foreach(cell, root->right_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ OuterJoinClauseInfo *ojcinfo = (OuterJoinClauseInfo *) lfirst(cell);
- distribute_restrictinfo_to_rels(root, rinfo);
+ distribute_restrictinfo_to_rels(root, ojcinfo->rinfo);
}
foreach(cell, root->full_join_clauses)
{
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(cell);
+ OuterJoinClauseInfo *ojcinfo = (OuterJoinClauseInfo *) lfirst(cell);
- distribute_restrictinfo_to_rels(root, rinfo);
+ distribute_restrictinfo_to_rels(root, ojcinfo->rinfo);
}
}
@@ -2059,9 +2065,10 @@ reconsider_outer_join_clauses(PlannerInfo *root)
* Returns true if we were able to propagate a constant through the clause.
*/
static bool
-reconsider_outer_join_clause(PlannerInfo *root, RestrictInfo *rinfo,
+reconsider_outer_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo,
bool outer_on_left)
{
+ RestrictInfo *rinfo = ojcinfo->rinfo;
Expr *outervar,
*innervar;
Oid opno,
@@ -2185,8 +2192,9 @@ reconsider_outer_join_clause(PlannerInfo *root, RestrictInfo *rinfo,
* Returns true if we were able to propagate a constant through the clause.
*/
static bool
-reconsider_full_join_clause(PlannerInfo *root, RestrictInfo *rinfo)
+reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
{
+ RestrictInfo *rinfo = ojcinfo->rinfo;
Expr *leftvar;
Expr *rightvar;
Oid opno,
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index d657e8f601..51b653fed4 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -349,6 +349,11 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
}
}
+ /*
+ * Update all_baserels and related relid sets.
+ */
+ root->all_baserels = bms_del_member(root->all_baserels, relid);
+
/*
* Likewise remove references from SpecialJoinInfo data structures.
*
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index d60398f1c6..4acbd3df32 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -40,7 +40,40 @@ int from_collapse_limit;
int join_collapse_limit;
-/* Elements of the postponed_qual_list used during deconstruct_recurse */
+/*
+ * deconstruct_jointree requires multiple passes over the join tree, because we
+ * need to finish computing JoinDomains before we start distributing quals.
+ * As long as we have to do that, other information such as the relevant
+ * qualscopes might as well be computed in the first pass too.
+ *
+ * deconstruct_recurse recursively examines the join tree and builds a List
+ * (in depth-first traversal order) of JoinTreeItem structs, which are then
+ * processed iteratively by deconstruct_distribute.
+ *
+ * The JoinTreeItem structs themselves can be freed at the end of
+ * deconstruct_jointree, but do not modify or free their substructure,
+ * as the relid sets may also be pointed to by RestrictInfo and
+ * SpecialJoinInfo nodes.
+ */
+typedef struct JoinTreeItem
+{
+ /* Fields filled during deconstruct_recurse: */
+ Node *jtnode; /* jointree node to examine */
+ bool below_outer_join; /* is it below an outer join? */
+ Relids qualscope; /* base Relids syntactically included in this
+ * jointree node */
+ Relids inner_join_rels; /* base Relids syntactically included in
+ * inner joins appearing at or below this
+ * jointree node */
+ Relids left_rels; /* if join node, Relids of the left side */
+ Relids right_rels; /* if join node, Relids of the right side */
+ Relids nonnullable_rels; /* if outer join, Relids of the
+ * non-nullable side */
+ /* Fields filled during deconstruct_distribute: */
+ SpecialJoinInfo *sjinfo; /* if outer join, its SpecialJoinInfo */
+} JoinTreeItem;
+
+/* Elements of the postponed_qual_list used during deconstruct_distribute */
typedef struct PostponedQual
{
Node *qual; /* a qual clause waiting to be processed */
@@ -52,8 +85,9 @@ static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
bool below_outer_join,
- Relids *qualscope, Relids *inner_join_rels,
- List **postponed_qual_list);
+ List **item_list);
+static void deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
+ List **postponed_qual_list);
static void process_security_barrier_quals(PlannerInfo *root,
int rti, Relids qualscope,
bool below_outer_join);
@@ -63,9 +97,17 @@ static SpecialJoinInfo *make_outerjoininfo(PlannerInfo *root,
JoinType jointype, List *clause);
static void compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo,
List *clause);
+static void distribute_quals_to_rels(PlannerInfo *root, List *clauses,
+ bool below_outer_join,
+ SpecialJoinInfo *sjinfo,
+ Index security_level,
+ Relids qualscope,
+ Relids ojscope,
+ Relids outerjoin_nonnullable,
+ List **postponed_qual_list);
static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool below_outer_join,
- JoinType jointype,
+ SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
Relids ojscope,
@@ -683,9 +725,9 @@ List *
deconstruct_jointree(PlannerInfo *root)
{
List *result;
- Relids qualscope;
- Relids inner_join_rels;
+ List *item_list = NIL;
List *postponed_qual_list = NIL;
+ ListCell *lc;
/*
* After this point, no more PlaceHolderInfos may be made, because
@@ -699,98 +741,105 @@ deconstruct_jointree(PlannerInfo *root)
Assert(root->parse->jointree != NULL &&
IsA(root->parse->jointree, FromExpr));
- /* this is filled as we scan the jointree */
+ /* These are filled as we scan the jointree */
+ root->all_baserels = NULL;
root->nullable_baserels = NULL;
- result = deconstruct_recurse(root, (Node *) root->parse->jointree, false,
- &qualscope, &inner_join_rels,
- &postponed_qual_list);
+ /* Perform the initial scan of the jointree */
+ result = deconstruct_recurse(root, (Node *) root->parse->jointree,
+ false,
+ &item_list);
- /* Shouldn't be any leftover quals */
+ /* Now scan all the jointree nodes again, and distribute quals */
+ foreach(lc, item_list)
+ {
+ JoinTreeItem *jtitem = (JoinTreeItem *) lfirst(lc);
+
+ deconstruct_distribute(root, jtitem,
+ &postponed_qual_list);
+ }
+
+ /* Shouldn't be any leftover postponed quals */
Assert(postponed_qual_list == NIL);
+ /* Don't need the JoinTreeItems any more */
+ list_free_deep(item_list);
+
return result;
}
/*
* deconstruct_recurse
- * One recursion level of deconstruct_jointree processing.
+ * One recursion level of deconstruct_jointree's initial jointree scan.
*
* Inputs:
* jtnode is the jointree node to examine
* below_outer_join is true if this node is within the nullable side of a
* higher-level outer join
- * Outputs:
- * *qualscope gets the set of base Relids syntactically included in this
- * jointree node (do not modify or free this, as it may also be pointed
- * to by RestrictInfo and SpecialJoinInfo nodes)
- * *inner_join_rels gets the set of base Relids syntactically included in
- * inner joins appearing at or below this jointree node (do not modify
- * or free this, either)
- * *postponed_qual_list is a list of PostponedQual structs, which we can
- * add quals to if they turn out to belong to a higher join level
- * Return value is the appropriate joinlist for this jointree node
*
- * In addition, entries will be added to root->join_info_list for outer joins.
+ * item_list is an in/out parameter: we add a JoinTreeItem struct to
+ * that list for each jointree node, in depth-first traversal order.
+ * (Hence, after each call, the last list item corresponds to its jtnode.)
+ *
+ * Return value is the appropriate joinlist for this jointree node.
*/
static List *
-deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
- Relids *qualscope, Relids *inner_join_rels,
- List **postponed_qual_list)
+deconstruct_recurse(PlannerInfo *root, Node *jtnode,
+ bool below_outer_join,
+ List **item_list)
{
List *joinlist;
+ JoinTreeItem *jtitem;
+
+ Assert(jtnode != NULL);
+
+ /* Make the new JoinTreeItem, but don't add it to item_list yet */
+ jtitem = palloc0_object(JoinTreeItem);
+ jtitem->jtnode = jtnode;
+ jtitem->below_outer_join = below_outer_join;
- if (jtnode == NULL)
- {
- *qualscope = NULL;
- *inner_join_rels = NULL;
- return NIL;
- }
if (IsA(jtnode, RangeTblRef))
{
int varno = ((RangeTblRef *) jtnode)->rtindex;
+ /* Fill all_baserels as we encounter baserel jointree nodes */
+ root->all_baserels = bms_add_member(root->all_baserels, varno);
/* qualscope is just the one RTE */
- *qualscope = bms_make_singleton(varno);
- /* Deal with any securityQuals attached to the RTE */
- if (root->qual_security_level > 0)
- process_security_barrier_quals(root,
- varno,
- *qualscope,
- below_outer_join);
+ jtitem->qualscope = bms_make_singleton(varno);
/* A single baserel does not create an inner join */
- *inner_join_rels = NULL;
+ jtitem->inner_join_rels = NULL;
joinlist = list_make1(jtnode);
}
else if (IsA(jtnode, FromExpr))
{
FromExpr *f = (FromExpr *) jtnode;
- List *child_postponed_quals = NIL;
int remaining;
ListCell *l;
/*
- * First, recurse to handle child joins. We collapse subproblems into
- * a single joinlist whenever the resulting joinlist wouldn't exceed
- * from_collapse_limit members. Also, always collapse one-element
- * subproblems, since that won't lengthen the joinlist anyway.
+ * Recurse to handle child nodes, and compute output joinlist. We
+ * collapse subproblems into a single joinlist whenever the resulting
+ * joinlist wouldn't exceed from_collapse_limit members. Also, always
+ * collapse one-element subproblems, since that won't lengthen the
+ * joinlist anyway.
*/
- *qualscope = NULL;
- *inner_join_rels = NULL;
+ jtitem->qualscope = NULL;
+ jtitem->inner_join_rels = NULL;
joinlist = NIL;
remaining = list_length(f->fromlist);
foreach(l, f->fromlist)
{
- Relids sub_qualscope;
+ JoinTreeItem *sub_item;
List *sub_joinlist;
int sub_members;
sub_joinlist = deconstruct_recurse(root, lfirst(l),
below_outer_join,
- &sub_qualscope,
- inner_join_rels,
- &child_postponed_quals);
- *qualscope = bms_add_members(*qualscope, sub_qualscope);
+ item_list);
+ sub_item = (JoinTreeItem *) llast(*item_list);
+ jtitem->qualscope = bms_add_members(jtitem->qualscope,
+ sub_item->qualscope);
+ jtitem->inner_join_rels = sub_item->inner_join_rels;
sub_members = list_length(sub_joinlist);
remaining--;
if (sub_members <= 1 ||
@@ -808,115 +857,80 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
* that still possible?) the initialization before the loop fixed it.
*/
if (list_length(f->fromlist) > 1)
- *inner_join_rels = *qualscope;
-
- /*
- * Try to process any quals postponed by children. If they need
- * further postponement, add them to my output postponed_qual_list.
- */
- foreach(l, child_postponed_quals)
- {
- PostponedQual *pq = (PostponedQual *) lfirst(l);
-
- if (bms_is_subset(pq->relids, *qualscope))
- distribute_qual_to_rels(root, pq->qual,
- below_outer_join, JOIN_INNER,
- root->qual_security_level,
- *qualscope, NULL, NULL,
- NULL);
- else
- *postponed_qual_list = lappend(*postponed_qual_list, pq);
- }
-
- /*
- * Now process the top-level quals.
- */
- foreach(l, (List *) f->quals)
- {
- Node *qual = (Node *) lfirst(l);
-
- distribute_qual_to_rels(root, qual,
- below_outer_join, JOIN_INNER,
- root->qual_security_level,
- *qualscope, NULL, NULL,
- postponed_qual_list);
- }
+ jtitem->inner_join_rels = jtitem->qualscope;
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- List *child_postponed_quals = NIL;
- Relids leftids,
- rightids,
- left_inners,
- right_inners,
- nonnullable_rels,
- nullable_rels,
- ojscope;
+ Relids nullable_rels;
+ JoinTreeItem *left_item,
+ *right_item;
List *leftjoinlist,
*rightjoinlist;
- List *my_quals;
- SpecialJoinInfo *sjinfo;
- ListCell *l;
- /*
- * Order of operations here is subtle and critical. First we recurse
- * to handle sub-JOINs. Their join quals will be placed without
- * regard for whether this level is an outer join, which is correct.
- * Then we place our own join quals, which are restricted by lower
- * outer joins in any case, and are forced to this level if this is an
- * outer join and they mention the outer side. Finally, if this is an
- * outer join, we create a join_info_list entry for the join. This
- * will prevent quals above us in the join tree that use those rels
- * from being pushed down below this level. (It's okay for upper
- * quals to be pushed down to the outer side, however.)
- */
switch (j->jointype)
{
case JOIN_INNER:
+ /* Recurse */
leftjoinlist = deconstruct_recurse(root, j->larg,
below_outer_join,
- &leftids, &left_inners,
- &child_postponed_quals);
+ item_list);
+ left_item = (JoinTreeItem *) llast(*item_list);
rightjoinlist = deconstruct_recurse(root, j->rarg,
below_outer_join,
- &rightids, &right_inners,
- &child_postponed_quals);
- *qualscope = bms_union(leftids, rightids);
- *inner_join_rels = *qualscope;
+ item_list);
+ right_item = (JoinTreeItem *) llast(*item_list);
+ /* Compute qualscope etc */
+ jtitem->qualscope = bms_union(left_item->qualscope,
+ right_item->qualscope);
+ jtitem->inner_join_rels = jtitem->qualscope;
+ jtitem->left_rels = left_item->qualscope;
+ jtitem->right_rels = right_item->qualscope;
/* Inner join adds no restrictions for quals */
- nonnullable_rels = NULL;
+ jtitem->nonnullable_rels = NULL;
/* and it doesn't force anything to null, either */
nullable_rels = NULL;
break;
case JOIN_LEFT:
case JOIN_ANTI:
+ /* Recurse */
leftjoinlist = deconstruct_recurse(root, j->larg,
below_outer_join,
- &leftids, &left_inners,
- &child_postponed_quals);
+ item_list);
+ left_item = (JoinTreeItem *) llast(*item_list);
rightjoinlist = deconstruct_recurse(root, j->rarg,
true,
- &rightids, &right_inners,
- &child_postponed_quals);
- *qualscope = bms_union(leftids, rightids);
- *inner_join_rels = bms_union(left_inners, right_inners);
- nonnullable_rels = leftids;
- nullable_rels = rightids;
+ item_list);
+ right_item = (JoinTreeItem *) llast(*item_list);
+ /* Compute qualscope etc */
+ jtitem->qualscope = bms_union(left_item->qualscope,
+ right_item->qualscope);
+ jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
+ right_item->inner_join_rels);
+ jtitem->left_rels = left_item->qualscope;
+ jtitem->right_rels = right_item->qualscope;
+ jtitem->nonnullable_rels = left_item->qualscope;
+ nullable_rels = right_item->qualscope;
break;
case JOIN_SEMI:
+ /* Recurse */
leftjoinlist = deconstruct_recurse(root, j->larg,
below_outer_join,
- &leftids, &left_inners,
- &child_postponed_quals);
+ item_list);
+ left_item = (JoinTreeItem *) llast(*item_list);
rightjoinlist = deconstruct_recurse(root, j->rarg,
below_outer_join,
- &rightids, &right_inners,
- &child_postponed_quals);
- *qualscope = bms_union(leftids, rightids);
- *inner_join_rels = bms_union(left_inners, right_inners);
+ item_list);
+ right_item = (JoinTreeItem *) llast(*item_list);
+ /* Compute qualscope etc */
+ jtitem->qualscope = bms_union(left_item->qualscope,
+ right_item->qualscope);
+ jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
+ right_item->inner_join_rels);
+ jtitem->left_rels = left_item->qualscope;
+ jtitem->right_rels = right_item->qualscope;
/* Semi join adds no restrictions for quals */
- nonnullable_rels = NULL;
+ jtitem->nonnullable_rels = NULL;
/*
* Theoretically, a semijoin would null the RHS; but since the
@@ -926,27 +940,32 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
nullable_rels = NULL;
break;
case JOIN_FULL:
+ /* Recurse */
leftjoinlist = deconstruct_recurse(root, j->larg,
true,
- &leftids, &left_inners,
- &child_postponed_quals);
+ item_list);
+ left_item = (JoinTreeItem *) llast(*item_list);
rightjoinlist = deconstruct_recurse(root, j->rarg,
true,
- &rightids, &right_inners,
- &child_postponed_quals);
- *qualscope = bms_union(leftids, rightids);
- *inner_join_rels = bms_union(left_inners, right_inners);
+ item_list);
+ right_item = (JoinTreeItem *) llast(*item_list);
+ /* Compute qualscope etc */
+ jtitem->qualscope = bms_union(left_item->qualscope,
+ right_item->qualscope);
+ jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
+ right_item->inner_join_rels);
+ jtitem->left_rels = left_item->qualscope;
+ jtitem->right_rels = right_item->qualscope;
/* each side is both outer and inner */
- nonnullable_rels = *qualscope;
- nullable_rels = *qualscope;
+ jtitem->nonnullable_rels = jtitem->qualscope;
+ nullable_rels = jtitem->qualscope;
break;
default:
/* JOIN_RIGHT was eliminated during reduce_outer_joins() */
elog(ERROR, "unrecognized join type: %d",
(int) j->jointype);
- nonnullable_rels = NULL; /* keep compiler quiet */
+ leftjoinlist = rightjoinlist = NIL; /* keep compiler quiet */
nullable_rels = NULL;
- leftjoinlist = rightjoinlist = NIL;
break;
}
@@ -954,6 +973,131 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
root->nullable_baserels = bms_add_members(root->nullable_baserels,
nullable_rels);
+ /*
+ * Compute the output joinlist. We fold subproblems together except
+ * at a FULL JOIN or where join_collapse_limit would be exceeded.
+ */
+ if (j->jointype == JOIN_FULL)
+ {
+ /* force the join order exactly at this node */
+ joinlist = list_make1(list_make2(leftjoinlist, rightjoinlist));
+ }
+ else if (list_length(leftjoinlist) + list_length(rightjoinlist) <=
+ join_collapse_limit)
+ {
+ /* OK to combine subproblems */
+ joinlist = list_concat(leftjoinlist, rightjoinlist);
+ }
+ else
+ {
+ /* can't combine, but needn't force join order above here */
+ Node *leftpart,
+ *rightpart;
+
+ /* avoid creating useless 1-element sublists */
+ if (list_length(leftjoinlist) == 1)
+ leftpart = (Node *) linitial(leftjoinlist);
+ else
+ leftpart = (Node *) leftjoinlist;
+ if (list_length(rightjoinlist) == 1)
+ rightpart = (Node *) linitial(rightjoinlist);
+ else
+ rightpart = (Node *) rightjoinlist;
+ joinlist = list_make2(leftpart, rightpart);
+ }
+ }
+ else
+ {
+ elog(ERROR, "unrecognized node type: %d",
+ (int) nodeTag(jtnode));
+ joinlist = NIL; /* keep compiler quiet */
+ }
+
+ /* Finally, we can add the new JoinTreeItem to item_list */
+ *item_list = lappend(*item_list, jtitem);
+
+ return joinlist;
+}
+
+/*
+ * deconstruct_distribute
+ * Process one jointree node in phase 2 of deconstruct_jointree processing.
+ *
+ * Distribute quals of the node to appropriate restriction and join lists.
+ * In addition, entries will be added to root->join_info_list for outer joins.
+ *
+ * Inputs:
+ * jtitem is the JoinTreeItem to examine
+ * Input/Outputs:
+ * *postponed_qual_list is a list of PostponedQual structs
+ *
+ * On entry, *postponed_qual_list contains any quals that had to be postponed
+ * out of lower join levels (because they contain lateral references).
+ * On exit, *postponed_qual_list contains quals that can't be processed yet
+ * (because their lateral references are still unsatisfied).
+ */
+static void
+deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
+ List **postponed_qual_list)
+{
+ Node *jtnode = jtitem->jtnode;
+
+ if (IsA(jtnode, RangeTblRef))
+ {
+ int varno = ((RangeTblRef *) jtnode)->rtindex;
+
+ /* Deal with any securityQuals attached to the RTE */
+ if (root->qual_security_level > 0)
+ process_security_barrier_quals(root,
+ varno,
+ jtitem->qualscope,
+ jtitem->below_outer_join);
+ }
+ else if (IsA(jtnode, FromExpr))
+ {
+ FromExpr *f = (FromExpr *) jtnode;
+ List *new_postponed_quals = NIL;
+ ListCell *l;
+
+ /*
+ * Try to process any quals postponed by children. If they need
+ * further postponement, add them to my output postponed_qual_list.
+ */
+ foreach(l, *postponed_qual_list)
+ {
+ PostponedQual *pq = (PostponedQual *) lfirst(l);
+
+ if (bms_is_subset(pq->relids, jtitem->qualscope))
+ distribute_qual_to_rels(root, pq->qual,
+ jtitem->below_outer_join,
+ NULL,
+ root->qual_security_level,
+ jtitem->qualscope, NULL, NULL,
+ NULL);
+ else
+ new_postponed_quals = lappend(new_postponed_quals, pq);
+ }
+ *postponed_qual_list = new_postponed_quals;
+
+ /*
+ * Now process the top-level quals.
+ */
+ distribute_quals_to_rels(root, (List *) f->quals,
+ jtitem->below_outer_join,
+ NULL,
+ root->qual_security_level,
+ jtitem->qualscope, NULL, NULL,
+ postponed_qual_list);
+ }
+ else if (IsA(jtnode, JoinExpr))
+ {
+ JoinExpr *j = (JoinExpr *) jtnode;
+ List *new_postponed_quals = NIL;
+ Relids ojscope;
+ List *my_quals;
+ SpecialJoinInfo *sjinfo;
+ ListCell *l;
+
/*
* Try to process any quals postponed by children. If they need
* further postponement, add them to my output postponed_qual_list.
@@ -961,11 +1105,11 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
* that they'll be handled properly in make_outerjoininfo.
*/
my_quals = NIL;
- foreach(l, child_postponed_quals)
+ foreach(l, *postponed_qual_list)
{
PostponedQual *pq = (PostponedQual *) lfirst(l);
- if (bms_is_subset(pq->relids, *qualscope))
+ if (bms_is_subset(pq->relids, jtitem->qualscope))
my_quals = lappend(my_quals, pq->qual);
else
{
@@ -974,16 +1118,15 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
* If this Assert fires, pull_up_subqueries() messed up.
*/
Assert(j->jointype == JOIN_INNER);
- *postponed_qual_list = lappend(*postponed_qual_list, pq);
+ new_postponed_quals = lappend(new_postponed_quals, pq);
}
}
+ *postponed_qual_list = new_postponed_quals;
my_quals = list_concat(my_quals, (List *) j->quals);
/*
- * For an OJ, form the SpecialJoinInfo now, because we need the OJ's
- * semantic scope (ojscope) to pass to distribute_qual_to_rels. But
- * we mustn't add it to join_info_list just yet, because we don't want
- * distribute_qual_to_rels to think it is an outer join below us.
+ * For an OJ, form the SpecialJoinInfo now, so that we can pass it to
+ * distribute_qual_to_rels. We must compute its ojscope too.
*
* Semijoins are a bit of a hybrid: we build a SpecialJoinInfo, but we
* want ojscope = NULL for distribute_qual_to_rels.
@@ -991,15 +1134,19 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
if (j->jointype != JOIN_INNER)
{
sjinfo = make_outerjoininfo(root,
- leftids, rightids,
- *inner_join_rels,
+ jtitem->left_rels,
+ jtitem->right_rels,
+ jtitem->inner_join_rels,
j->jointype,
my_quals);
+ jtitem->sjinfo = sjinfo;
if (j->jointype == JOIN_SEMI)
ojscope = NULL;
else
+ {
ojscope = bms_union(sjinfo->min_lefthand,
sjinfo->min_righthand);
+ }
}
else
{
@@ -1008,67 +1155,27 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode, bool below_outer_join,
}
/* Process the JOIN's qual clauses */
- foreach(l, my_quals)
- {
- Node *qual = (Node *) lfirst(l);
-
- distribute_qual_to_rels(root, qual,
- below_outer_join, j->jointype,
- root->qual_security_level,
- *qualscope,
- ojscope, nonnullable_rels,
- postponed_qual_list);
- }
-
- /* Now we can add the SpecialJoinInfo to join_info_list */
+ distribute_quals_to_rels(root, my_quals,
+ jtitem->below_outer_join,
+ sjinfo,
+ root->qual_security_level,
+ jtitem->qualscope,
+ ojscope, jtitem->nonnullable_rels,
+ postponed_qual_list);
+
+ /* And add the SpecialJoinInfo to join_info_list */
if (sjinfo)
{
root->join_info_list = lappend(root->join_info_list, sjinfo);
/* Each time we do that, recheck placeholder eval levels */
update_placeholder_eval_levels(root, sjinfo);
}
-
- /*
- * Finally, compute the output joinlist. We fold subproblems together
- * except at a FULL JOIN or where join_collapse_limit would be
- * exceeded.
- */
- if (j->jointype == JOIN_FULL)
- {
- /* force the join order exactly at this node */
- joinlist = list_make1(list_make2(leftjoinlist, rightjoinlist));
- }
- else if (list_length(leftjoinlist) + list_length(rightjoinlist) <=
- join_collapse_limit)
- {
- /* OK to combine subproblems */
- joinlist = list_concat(leftjoinlist, rightjoinlist);
- }
- else
- {
- /* can't combine, but needn't force join order above here */
- Node *leftpart,
- *rightpart;
-
- /* avoid creating useless 1-element sublists */
- if (list_length(leftjoinlist) == 1)
- leftpart = (Node *) linitial(leftjoinlist);
- else
- leftpart = (Node *) leftjoinlist;
- if (list_length(rightjoinlist) == 1)
- rightpart = (Node *) linitial(rightjoinlist);
- else
- rightpart = (Node *) rightjoinlist;
- joinlist = list_make2(leftpart, rightpart);
- }
}
else
{
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(jtnode));
- joinlist = NIL; /* keep compiler quiet */
}
- return joinlist;
}
/*
@@ -1102,27 +1209,21 @@ process_security_barrier_quals(PlannerInfo *root,
foreach(lc, rte->securityQuals)
{
List *qualset = (List *) lfirst(lc);
- ListCell *lc2;
-
- foreach(lc2, qualset)
- {
- Node *qual = (Node *) lfirst(lc2);
- /*
- * We cheat to the extent of passing ojscope = qualscope rather
- * than its more logical value of NULL. The only effect this has
- * is to force a Var-free qual to be evaluated at the rel rather
- * than being pushed up to top of tree, which we don't want.
- */
- distribute_qual_to_rels(root, qual,
- below_outer_join,
- JOIN_INNER,
- security_level,
- qualscope,
- qualscope,
- NULL,
- NULL);
- }
+ /*
+ * We cheat to the extent of passing ojscope = qualscope rather than
+ * its more logical value of NULL. The only effect this has is to
+ * force a Var-free qual to be evaluated at the rel rather than being
+ * pushed up to top of tree, which we don't want.
+ */
+ distribute_quals_to_rels(root, qualset,
+ below_outer_join,
+ NULL,
+ security_level,
+ qualscope,
+ qualscope,
+ NULL,
+ NULL);
security_level++;
}
@@ -1572,6 +1673,38 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
*
*****************************************************************************/
+/*
+ * distribute_quals_to_rels
+ * Convenience routine to apply distribute_qual_to_rels to each element
+ * of an AND'ed list of clauses.
+ */
+static void
+distribute_quals_to_rels(PlannerInfo *root, List *clauses,
+ bool below_outer_join,
+ SpecialJoinInfo *sjinfo,
+ Index security_level,
+ Relids qualscope,
+ Relids ojscope,
+ Relids outerjoin_nonnullable,
+ List **postponed_qual_list)
+{
+ ListCell *lc;
+
+ foreach(lc, clauses)
+ {
+ Node *clause = (Node *) lfirst(lc);
+
+ distribute_qual_to_rels(root, clause,
+ below_outer_join,
+ sjinfo,
+ security_level,
+ qualscope,
+ ojscope,
+ outerjoin_nonnullable,
+ postponed_qual_list);
+ }
+}
+
/*
* distribute_qual_to_rels
* Add clause information to either the baserestrictinfo or joininfo list
@@ -1586,7 +1719,7 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
* 'clause': the qual clause to be distributed
* 'below_outer_join': true if the qual is from a JOIN/ON that is below the
* nullable side of a higher-level outer join
- * 'jointype': type of join the qual is from (JOIN_INNER for a WHERE clause)
+ * 'sjinfo': join's SpecialJoinInfo (NULL for an inner join or WHERE clause)
* 'security_level': security_level to assign to the qual
* 'qualscope': set of baserels the qual's syntactic scope covers
* 'ojscope': NULL if not an outer-join qual, else the minimum set of baserels
@@ -1604,12 +1737,13 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
* level, which will be ojscope not necessarily qualscope.
*
* At the time this is called, root->join_info_list must contain entries for
- * all and only those special joins that are syntactically below this qual.
+ * all and only those special joins that are syntactically below this qual;
+ * in particular, the passed-in SpecialJoinInfo isn't yet in that list.
*/
static void
distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool below_outer_join,
- JoinType jointype,
+ SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
Relids ojscope,
@@ -1646,7 +1780,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
PostponedQual *pq = (PostponedQual *) palloc(sizeof(PostponedQual));
Assert(root->hasLateralRTEs); /* shouldn't happen otherwise */
- Assert(jointype == JOIN_INNER); /* mustn't postpone past outer join */
+ Assert(sjinfo == NULL); /* mustn't postpone past outer join */
pq->qual = clause;
pq->relids = relids;
*postponed_qual_list = lappend(*postponed_qual_list, pq);
@@ -1930,14 +2064,19 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
/* we need to set up left_ec/right_ec the hard way */
initialize_mergeclause_eclasses(root, restrictinfo);
/* now see if it should go to any outer-join lists */
+ Assert(sjinfo != NULL);
if (bms_is_subset(restrictinfo->left_relids,
outerjoin_nonnullable) &&
!bms_overlap(restrictinfo->right_relids,
outerjoin_nonnullable))
{
/* we have outervar = innervar */
+ OuterJoinClauseInfo *ojcinfo = makeNode(OuterJoinClauseInfo);
+
+ ojcinfo->rinfo = restrictinfo;
+ ojcinfo->sjinfo = sjinfo;
root->left_join_clauses = lappend(root->left_join_clauses,
- restrictinfo);
+ ojcinfo);
return;
}
if (bms_is_subset(restrictinfo->right_relids,
@@ -1946,15 +2085,23 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
outerjoin_nonnullable))
{
/* we have innervar = outervar */
+ OuterJoinClauseInfo *ojcinfo = makeNode(OuterJoinClauseInfo);
+
+ ojcinfo->rinfo = restrictinfo;
+ ojcinfo->sjinfo = sjinfo;
root->right_join_clauses = lappend(root->right_join_clauses,
- restrictinfo);
+ ojcinfo);
return;
}
- if (jointype == JOIN_FULL)
+ if (sjinfo->jointype == JOIN_FULL)
{
/* FULL JOIN (above tests cannot match in this case) */
+ OuterJoinClauseInfo *ojcinfo = makeNode(OuterJoinClauseInfo);
+
+ ojcinfo->rinfo = restrictinfo;
+ ojcinfo->sjinfo = sjinfo;
root->full_join_clauses = lappend(root->full_join_clauses,
- restrictinfo);
+ ojcinfo);
return;
}
/* nope, so fall through to distribute_restrictinfo_to_rels */
diff --git a/src/backend/rewrite/rewriteManip.c b/src/backend/rewrite/rewriteManip.c
index 980f583915..6f51c7ee75 100644
--- a/src/backend/rewrite/rewriteManip.c
+++ b/src/backend/rewrite/rewriteManip.c
@@ -40,6 +40,20 @@ typedef struct
int win_location;
} locate_windowfunc_context;
+typedef struct
+{
+ const Bitmapset *target_relids;
+ const Bitmapset *added_relids;
+ int sublevels_up;
+} add_nulling_relids_context;
+
+typedef struct
+{
+ const Bitmapset *removable_relids;
+ const Bitmapset *except_relids;
+ int sublevels_up;
+} remove_nulling_relids_context;
+
static bool contain_aggs_of_level_walker(Node *node,
contain_aggs_of_level_context *context);
static bool locate_agg_of_level_walker(Node *node,
@@ -50,6 +64,10 @@ static bool locate_windowfunc_walker(Node *node,
static bool checkExprHasSubLink_walker(Node *node, void *context);
static Relids offset_relid_set(Relids relids, int offset);
static Relids adjust_relid_set(Relids relids, int oldrelid, int newrelid);
+static Node *add_nulling_relids_mutator(Node *node,
+ add_nulling_relids_context *context);
+static Node *remove_nulling_relids_mutator(Node *node,
+ remove_nulling_relids_context *context);
/*
@@ -381,6 +399,8 @@ OffsetVarNodes_walker(Node *node, OffsetVarNodes_context *context)
if (var->varlevelsup == context->sublevels_up)
{
var->varno += context->offset;
+ var->varnullingrels = offset_relid_set(var->varnullingrels,
+ context->offset);
if (var->varnosyn > 0)
var->varnosyn += context->offset;
}
@@ -419,6 +439,8 @@ OffsetVarNodes_walker(Node *node, OffsetVarNodes_context *context)
{
phv->phrels = offset_relid_set(phv->phrels,
context->offset);
+ phv->phnullingrels = offset_relid_set(phv->phnullingrels,
+ context->offset);
}
/* fall through to examine children */
}
@@ -543,11 +565,13 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context)
{
Var *var = (Var *) node;
- if (var->varlevelsup == context->sublevels_up &&
- var->varno == context->rt_index)
+ if (var->varlevelsup == context->sublevels_up)
{
- var->varno = context->new_index;
- /* If the syntactic referent is same RTE, fix it too */
+ if (var->varno == context->rt_index)
+ var->varno = context->new_index;
+ var->varnullingrels = adjust_relid_set(var->varnullingrels,
+ context->rt_index,
+ context->new_index);
if (var->varnosyn == context->rt_index)
var->varnosyn = context->new_index;
}
@@ -590,6 +614,9 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context)
phv->phrels = adjust_relid_set(phv->phrels,
context->rt_index,
context->new_index);
+ phv->phnullingrels = adjust_relid_set(phv->phnullingrels,
+ context->rt_index,
+ context->new_index);
}
/* fall through to examine children */
}
@@ -866,7 +893,8 @@ rangeTableEntry_used_walker(Node *node,
Var *var = (Var *) node;
if (var->varlevelsup == context->sublevels_up &&
- var->varno == context->rt_index)
+ (var->varno == context->rt_index ||
+ bms_is_member(context->rt_index, var->varnullingrels)))
return true;
return false;
}
@@ -1094,6 +1122,195 @@ AddInvertedQual(Query *parsetree, Node *qual)
}
+/*
+ * add_nulling_relids() finds Vars and PlaceHolderVars that belong to any
+ * of the target_relids, and adds added_relids to their varnullingrels
+ * and phnullingrels fields.
+ */
+Node *
+add_nulling_relids(Node *node,
+ const Bitmapset *target_relids,
+ const Bitmapset *added_relids)
+{
+ add_nulling_relids_context context;
+
+ context.target_relids = target_relids;
+ context.added_relids = added_relids;
+ context.sublevels_up = 0;
+ return query_or_expression_tree_mutator(node,
+ add_nulling_relids_mutator,
+ &context,
+ 0);
+}
+
+static Node *
+add_nulling_relids_mutator(Node *node,
+ add_nulling_relids_context *context)
+{
+ if (node == NULL)
+ return NULL;
+ if (IsA(node, Var))
+ {
+ Var *var = (Var *) node;
+
+ if (var->varlevelsup == context->sublevels_up &&
+ bms_is_member(var->varno, context->target_relids))
+ {
+ Relids newnullingrels = bms_union(var->varnullingrels,
+ context->added_relids);
+
+ /* Copy the Var ... */
+ var = copyObject(var);
+ /* ... and replace the copy's varnullingrels field */
+ var->varnullingrels = newnullingrels;
+ return (Node *) var;
+ }
+ /* Otherwise fall through to copy the Var normally */
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ if (phv->phlevelsup == context->sublevels_up &&
+ bms_overlap(phv->phrels, context->target_relids))
+ {
+ Relids newnullingrels = bms_union(phv->phnullingrels,
+ context->added_relids);
+
+ /*
+ * We don't modify the contents of the PHV's expression, only add
+ * to phnullingrels. This corresponds to assuming that the PHV
+ * will be evaluated at the same level as before, then perhaps be
+ * nulled as it bubbles up. Hence, just flat-copy the node ...
+ */
+ phv = makeNode(PlaceHolderVar);
+ memcpy(phv, node, sizeof(PlaceHolderVar));
+ /* ... and replace the copy's phnullingrels field */
+ phv->phnullingrels = newnullingrels;
+ return (Node *) phv;
+ }
+ /* Otherwise fall through to copy the PlaceHolderVar normally */
+ }
+ else if (IsA(node, Query))
+ {
+ /* Recurse into RTE or sublink subquery */
+ Query *newnode;
+
+ context->sublevels_up++;
+ newnode = query_tree_mutator((Query *) node,
+ add_nulling_relids_mutator,
+ (void *) context,
+ 0);
+ context->sublevels_up--;
+ return (Node *) newnode;
+ }
+ return expression_tree_mutator(node, add_nulling_relids_mutator,
+ (void *) context);
+}
+
+/*
+ * remove_nulling_relids() removes mentions of the specified RT index(es)
+ * in Var.varnullingrels and PlaceHolderVar.phnullingrels fields within
+ * the given expression, except in nodes belonging to rels listed in
+ * except_relids.
+ */
+Node *
+remove_nulling_relids(Node *node,
+ const Bitmapset *removable_relids,
+ const Bitmapset *except_relids)
+{
+ remove_nulling_relids_context context;
+
+ context.removable_relids = removable_relids;
+ context.except_relids = except_relids;
+ context.sublevels_up = 0;
+ return query_or_expression_tree_mutator(node,
+ remove_nulling_relids_mutator,
+ &context,
+ 0);
+}
+
+static Node *
+remove_nulling_relids_mutator(Node *node,
+ remove_nulling_relids_context *context)
+{
+ if (node == NULL)
+ return NULL;
+ if (IsA(node, Var))
+ {
+ Var *var = (Var *) node;
+
+ if (var->varlevelsup == context->sublevels_up &&
+ !bms_is_member(var->varno, context->except_relids) &&
+ bms_overlap(var->varnullingrels, context->removable_relids))
+ {
+ Relids newnullingrels = bms_difference(var->varnullingrels,
+ context->removable_relids);
+
+ /* Micro-optimization: ensure nullingrels is NULL if empty */
+ if (bms_is_empty(newnullingrels))
+ newnullingrels = NULL;
+ /* Copy the Var ... */
+ var = copyObject(var);
+ /* ... and replace the copy's varnullingrels field */
+ var->varnullingrels = newnullingrels;
+ return (Node *) var;
+ }
+ /* Otherwise fall through to copy the Var normally */
+ }
+ else if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ if (phv->phlevelsup == context->sublevels_up &&
+ !bms_overlap(phv->phrels, context->except_relids))
+ {
+ Relids newnullingrels = bms_difference(phv->phnullingrels,
+ context->removable_relids);
+
+ /*
+ * Micro-optimization: ensure nullingrels is NULL if empty.
+ *
+ * Note: it might seem desirable to remove the PHV altogether if
+ * phnullingrels goes to empty. Currently we dare not do that
+ * because we use PHVs in some cases to enforce separate identity
+ * of subexpressions; see wrap_non_vars usages in prepjointree.c.
+ */
+ if (bms_is_empty(newnullingrels))
+ newnullingrels = NULL;
+ /* Copy the PlaceHolderVar and mutate what's below ... */
+ phv = (PlaceHolderVar *)
+ expression_tree_mutator(node,
+ remove_nulling_relids_mutator,
+ (void *) context);
+ /* ... and replace the copy's phnullingrels field */
+ phv->phnullingrels = newnullingrels;
+ /* We must also update phrels, if it contains a removable RTI */
+ phv->phrels = bms_difference(phv->phrels,
+ context->removable_relids);
+ Assert(!bms_is_empty(phv->phrels));
+ return (Node *) phv;
+ }
+ /* Otherwise fall through to copy the PlaceHolderVar normally */
+ }
+ else if (IsA(node, Query))
+ {
+ /* Recurse into RTE or sublink subquery */
+ Query *newnode;
+
+ context->sublevels_up++;
+ newnode = query_tree_mutator((Query *) node,
+ remove_nulling_relids_mutator,
+ (void *) context,
+ 0);
+ context->sublevels_up--;
+ return (Node *) newnode;
+ }
+ return expression_tree_mutator(node, remove_nulling_relids_mutator,
+ (void *) context);
+}
+
+
/*
* replace_rte_variables() finds all Vars in an expression tree
* that reference a particular RTE, and replaces them with substitute
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 2d1d8f4bcd..0909420615 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -249,10 +249,8 @@ struct PlannerInfo
struct AppendRelInfo **append_rel_array pg_node_attr(read_write_ignore);
/*
- * all_baserels is a Relids set of all base relids (but not "other"
- * relids) in the query; that is, the Relids identifier of the final join
- * we need to form. This is computed in make_one_rel, just before we
- * start making Paths.
+ * all_baserels is a Relids set of all base relids (but not joins or
+ * "other" rels) in the query. This is computed in deconstruct_jointree.
*/
Relids all_baserels;
@@ -313,19 +311,19 @@ struct PlannerInfo
List *canon_pathkeys;
/*
- * list of RestrictInfos for mergejoinable outer join clauses
+ * list of OuterJoinClauseInfos for mergejoinable outer join clauses
* w/nonnullable var on left
*/
List *left_join_clauses;
/*
- * list of RestrictInfos for mergejoinable outer join clauses
+ * list of OuterJoinClauseInfos for mergejoinable outer join clauses
* w/nonnullable var on right
*/
List *right_join_clauses;
/*
- * list of RestrictInfos for mergejoinable full join clauses
+ * list of OuterJoinClauseInfos for mergejoinable full join clauses
*/
List *full_join_clauses;
@@ -911,7 +909,7 @@ typedef struct RelOptInfo
int32 *attr_widths pg_node_attr(read_write_ignore);
/* LATERAL Vars and PHVs referenced by rel */
List *lateral_vars;
- /* rels that reference me laterally */
+ /* rels that reference this baserel laterally */
Relids lateral_referencers;
/* list of IndexOptInfo */
List *indexlist;
@@ -921,10 +919,7 @@ typedef struct RelOptInfo
BlockNumber pages;
Cardinality tuples;
double allvisfrac;
-
- /*
- * Indexes in PlannerInfo's eq_classes list of ECs that mention this rel
- */
+ /* indexes in PlannerInfo's eq_classes list of ECs that mention this rel */
Bitmapset *eclass_indexes;
PlannerInfo *subroot; /* if subquery */
List *subplan_params; /* if subquery */
@@ -2624,10 +2619,15 @@ typedef struct MergeScanSelCache
* of a plan tree. This is used during planning to represent the contained
* expression. At the end of the planning process it is replaced by either
* the contained expression or a Var referring to a lower-level evaluation of
- * the contained expression. Typically the evaluation occurs below an outer
+ * the contained expression. Generally the evaluation occurs below an outer
* join, and Var references above the outer join might thereby yield NULL
* instead of the expression value.
*
+ * phrels and phlevelsup correspond to the varno/varlevelsup fields of a
+ * plain Var, except that phrels has to be a relid set since the evaluation
+ * level of a PlaceHolderVar might be a join rather than a base relation.
+ * Likewise, phnullingrels corresponds to varnullingrels.
+ *
* Although the planner treats this as an expression node type, it is not
* recognized by the parser or executor, so we declare it here rather than
* in primnodes.h.
@@ -2640,8 +2640,10 @@ typedef struct MergeScanSelCache
* PHV. Another way in which it can happen is that initplan sublinks
* could get replaced by differently-numbered Params when sublink folding
* is done. (The end result of such a situation would be some
- * unreferenced initplans, which is annoying but not really a problem.) On
- * the same reasoning, there is no need to examine phrels.
+ * unreferenced initplans, which is annoying but not really a problem.)
+ * On the same reasoning, there is no need to examine phrels. But we do
+ * need to compare phnullingrels, as that represents effects that are
+ * external to the original value of the PHV.
*/
typedef struct PlaceHolderVar
@@ -2651,9 +2653,12 @@ typedef struct PlaceHolderVar
/* the represented expression */
Expr *phexpr pg_node_attr(equal_ignore);
- /* base relids syntactically within expr src */
+ /* base+OJ relids syntactically within expr src */
Relids phrels pg_node_attr(equal_ignore);
+ /* RT indexes of outer joins that can null PHV's value */
+ Relids phnullingrels;
+
/* ID for PHV (unique within planner run) */
Index phid;
@@ -2742,6 +2747,21 @@ struct SpecialJoinInfo
List *semi_rhs_exprs; /* righthand-side expressions of these ops */
};
+/*
+ * Transient outer-join clause info.
+ *
+ * We set aside every outer join ON clause that looks mergejoinable,
+ * and process it specially at the end of qual distribution.
+ */
+typedef struct OuterJoinClauseInfo
+{
+ pg_node_attr(no_copy_equal, no_read)
+
+ NodeTag type;
+ RestrictInfo *rinfo; /* a mergejoinable outer-join clause */
+ SpecialJoinInfo *sjinfo; /* the outer join's SpecialJoinInfo */
+} OuterJoinClauseInfo;
+
/*
* Append-relation info.
*
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index dec7d5c775..6c96fa2f51 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -190,6 +190,14 @@ typedef struct Expr
* row identity information during UPDATE/DELETE/MERGE. This value should
* never be seen outside the planner.
*
+ * varnullingrels is the set of RT indexes of outer joins that can force
+ * the Var's value to null (at the point where it appears in the query).
+ * See optimizer/README for discussion of that.
+ *
+ * varlevelsup is greater than zero in Vars that represent outer references.
+ * Note that it affects the meaning of all of varno, varnullingrels, and
+ * varnosyn, all of which refer to the range table of that query level.
+ *
* In the parser, varnosyn and varattnosyn are either identical to
* varno/varattno, or they specify the column's position in an aliased JOIN
* RTE that hides the semantic referent RTE's refname. This is a syntactic
@@ -232,6 +240,8 @@ typedef struct Var
int32 vartypmod;
/* OID of collation, or InvalidOid if none */
Oid varcollid;
+ /* RT indexes of outer joins that can replace the Var's value with null */
+ Bitmapset *varnullingrels;
/*
* for subquery variables referencing outer relations; 0 in a normal var,
diff --git a/src/include/rewrite/rewriteManip.h b/src/include/rewrite/rewriteManip.h
index 02d130cc26..365061fff4 100644
--- a/src/include/rewrite/rewriteManip.h
+++ b/src/include/rewrite/rewriteManip.h
@@ -65,6 +65,13 @@ extern bool contain_windowfuncs(Node *node);
extern int locate_windowfunc(Node *node);
extern bool checkExprHasSubLink(Node *node);
+extern Node *add_nulling_relids(Node *node,
+ const Bitmapset *target_relids,
+ const Bitmapset *added_relids);
+extern Node *remove_nulling_relids(Node *node,
+ const Bitmapset *removable_relids,
+ const Bitmapset *except_relids);
+
extern Node *replace_rte_variables(Node *node,
int target_varno, int sublevels_up,
replace_rte_variables_callback callback,
commit ec26da36b009833381f94baffb4bd649ad89fc71
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Jan 23 14:26:56 2023 -0500
Teach the parser to fill Var.varnullingrels correctly.
Vars emitted by the parser are now marked with RT indexes of outer
joins that can null them. (This is done purely according to the
syntax of the query; we don't consider whether an outer join could
be strength-reduced, for example.)
Although the result of this step compiles, it will fail some
regression tests due to the planner not yet knowing what to do.
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 4a817b75ad..e892df9819 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -676,6 +676,7 @@ transformInsertStmt(ParseState *pstate, InsertStmt *stmt)
sub_pstate->p_rtable = sub_rtable;
sub_pstate->p_rteperminfos = sub_rteperminfos;
sub_pstate->p_joinexprs = NIL; /* sub_rtable has no joins */
+ sub_pstate->p_nullingrels = NIL;
sub_pstate->p_namespace = sub_namespace;
sub_pstate->p_resolve_unknowns = false;
@@ -857,7 +858,7 @@ transformInsertStmt(ParseState *pstate, InsertStmt *stmt)
/*
* Generate list of Vars referencing the RTE
*/
- exprList = expandNSItemVars(nsitem, 0, -1, NULL);
+ exprList = expandNSItemVars(pstate, nsitem, 0, -1, NULL);
/*
* Re-apply any indirection on the target column specs to the Vars
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index bafa5bb381..f61f794755 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -52,7 +52,8 @@
#include "utils/syscache.h"
-static int extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
+static int extractRemainingColumns(ParseState *pstate,
+ ParseNamespaceColumn *src_nscolumns,
List *src_colnames,
List **src_colnos,
List **res_colnames, List **res_colvars,
@@ -75,9 +76,11 @@ static ParseNamespaceItem *getNSItemForSpecialRelationTypes(ParseState *pstate,
static Node *transformFromClauseItem(ParseState *pstate, Node *n,
ParseNamespaceItem **top_nsitem,
List **namespace);
-static Var *buildVarFromNSColumn(ParseNamespaceColumn *nscol);
+static Var *buildVarFromNSColumn(ParseState *pstate,
+ ParseNamespaceColumn *nscol);
static Node *buildMergedJoinVar(ParseState *pstate, JoinType jointype,
Var *l_colvar, Var *r_colvar);
+static void markRelsAsNulledBy(ParseState *pstate, Node *n, int jindex);
static void setNamespaceColumnVisibility(List *namespace, bool cols_visible);
static void setNamespaceLateralState(List *namespace,
bool lateral_only, bool lateral_ok);
@@ -251,7 +254,8 @@ setTargetTable(ParseState *pstate, RangeVar *relation,
* Returns the number of columns added.
*/
static int
-extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
+extractRemainingColumns(ParseState *pstate,
+ ParseNamespaceColumn *src_nscolumns,
List *src_colnames,
List **src_colnos,
List **res_colnames, List **res_colvars,
@@ -287,7 +291,8 @@ extractRemainingColumns(ParseNamespaceColumn *src_nscolumns,
*src_colnos = lappend_int(*src_colnos, attnum);
*res_colnames = lappend(*res_colnames, lfirst(lc));
*res_colvars = lappend(*res_colvars,
- buildVarFromNSColumn(src_nscolumns + attnum - 1));
+ buildVarFromNSColumn(pstate,
+ src_nscolumns + attnum - 1));
/* Copy the input relation's nscolumn data for this column */
res_nscolumns[colcount] = src_nscolumns[attnum - 1];
colcount++;
@@ -1288,8 +1293,7 @@ transformFromClauseItem(ParseState *pstate, Node *n,
{
/*
* JOIN/USING (or NATURAL JOIN, as transformed above). Transform
- * the list into an explicit ON-condition, and generate a list of
- * merged result columns.
+ * the list into an explicit ON-condition.
*/
List *ucols = j->usingClause;
List *l_usingvars = NIL;
@@ -1307,8 +1311,6 @@ transformFromClauseItem(ParseState *pstate, Node *n,
int r_index = -1;
Var *l_colvar,
*r_colvar;
- Node *u_colvar;
- ParseNamespaceColumn *res_nscolumn;
Assert(u_colname[0] != '\0');
@@ -1372,17 +1374,109 @@ transformFromClauseItem(ParseState *pstate, Node *n,
u_colname)));
r_colnos = lappend_int(r_colnos, r_index + 1);
- l_colvar = buildVarFromNSColumn(l_nscolumns + l_index);
+ /* Build Vars to use in the generated JOIN ON clause */
+ l_colvar = buildVarFromNSColumn(pstate, l_nscolumns + l_index);
l_usingvars = lappend(l_usingvars, l_colvar);
- r_colvar = buildVarFromNSColumn(r_nscolumns + r_index);
+ r_colvar = buildVarFromNSColumn(pstate, r_nscolumns + r_index);
r_usingvars = lappend(r_usingvars, r_colvar);
+ /*
+ * While we're here, add column names to the res_colnames
+ * list. It's a bit ugly to do this here while the
+ * corresponding res_colvars entries are not made till later,
+ * but doing this later would require an additional traversal
+ * of the usingClause list.
+ */
res_colnames = lappend(res_colnames, lfirst(ucol));
+ }
+
+ /* Construct the generated JOIN ON clause */
+ j->quals = transformJoinUsingClause(pstate,
+ l_usingvars,
+ r_usingvars);
+ }
+ else if (j->quals)
+ {
+ /* User-written ON-condition; transform it */
+ j->quals = transformJoinOnClause(pstate, j, my_namespace);
+ }
+ else
+ {
+ /* CROSS JOIN: no quals */
+ }
+
+ /*
+ * If this is an outer join, now mark the appropriate child RTEs as
+ * being nulled by this join. We have finished processing the child
+ * join expressions as well as the current join's quals, which deal in
+ * non-nulled input columns. All future references to those RTEs will
+ * see possibly-nulled values, and we should mark generated Vars to
+ * account for that. In particular, the join alias Vars that we're
+ * about to build should reflect the nulling effects of this join.
+ *
+ * A difficulty with doing this is that we need the join's RT index,
+ * which we don't officially have yet. However, no other RTE can get
+ * made between here and the addRangeTableEntryForJoin call, so we can
+ * predict what the assignment will be. (Alternatively, we could call
+ * addRangeTableEntryForJoin before we have all the data computed, but
+ * this seems less ugly.)
+ */
+ j->rtindex = list_length(pstate->p_rtable) + 1;
+
+ switch (j->jointype)
+ {
+ case JOIN_INNER:
+ break;
+ case JOIN_LEFT:
+ markRelsAsNulledBy(pstate, j->rarg, j->rtindex);
+ break;
+ case JOIN_FULL:
+ markRelsAsNulledBy(pstate, j->larg, j->rtindex);
+ markRelsAsNulledBy(pstate, j->rarg, j->rtindex);
+ break;
+ case JOIN_RIGHT:
+ markRelsAsNulledBy(pstate, j->larg, j->rtindex);
+ break;
+ default:
+ /* shouldn't see any other types here */
+ elog(ERROR, "unrecognized join type: %d",
+ (int) j->jointype);
+ break;
+ }
+
+ /*
+ * Now we can construct join alias expressions for the USING columns.
+ */
+ if (j->usingClause)
+ {
+ ListCell *lc1,
+ *lc2;
+
+ /* Scan the colnos lists to recover info from the previous loop */
+ forboth(lc1, l_colnos, lc2, r_colnos)
+ {
+ int l_index = lfirst_int(lc1) - 1;
+ int r_index = lfirst_int(lc2) - 1;
+ Var *l_colvar,
+ *r_colvar;
+ Node *u_colvar;
+ ParseNamespaceColumn *res_nscolumn;
+
+ /*
+ * Note we re-build these Vars: they might have different
+ * varnullingrels than the ones made in the previous loop.
+ */
+ l_colvar = buildVarFromNSColumn(pstate, l_nscolumns + l_index);
+ r_colvar = buildVarFromNSColumn(pstate, r_nscolumns + r_index);
+
+ /* Construct the join alias Var for this column */
u_colvar = buildMergedJoinVar(pstate,
j->jointype,
l_colvar,
r_colvar);
res_colvars = lappend(res_colvars, u_colvar);
+
+ /* Construct column's res_nscolumns[] entry */
res_nscolumn = res_nscolumns + res_colindex;
res_colindex++;
if (u_colvar == (Node *) l_colvar)
@@ -1400,47 +1494,45 @@ transformFromClauseItem(ParseState *pstate, Node *n,
/*
* Merged column is not semantically equivalent to either
* input, so it needs to be referenced as the join output
- * column. We don't know the join's varno yet, so we'll
- * replace these zeroes below.
+ * column.
*/
- res_nscolumn->p_varno = 0;
+ res_nscolumn->p_varno = j->rtindex;
res_nscolumn->p_varattno = res_colindex;
res_nscolumn->p_vartype = exprType(u_colvar);
res_nscolumn->p_vartypmod = exprTypmod(u_colvar);
res_nscolumn->p_varcollid = exprCollation(u_colvar);
- res_nscolumn->p_varnosyn = 0;
+ res_nscolumn->p_varnosyn = j->rtindex;
res_nscolumn->p_varattnosyn = res_colindex;
}
}
-
- j->quals = transformJoinUsingClause(pstate,
- l_usingvars,
- r_usingvars);
- }
- else if (j->quals)
- {
- /* User-written ON-condition; transform it */
- j->quals = transformJoinOnClause(pstate, j, my_namespace);
- }
- else
- {
- /* CROSS JOIN: no quals */
}
/* Add remaining columns from each side to the output columns */
res_colindex +=
- extractRemainingColumns(l_nscolumns, l_colnames, &l_colnos,
+ extractRemainingColumns(pstate,
+ l_nscolumns, l_colnames, &l_colnos,
&res_colnames, &res_colvars,
res_nscolumns + res_colindex);
res_colindex +=
- extractRemainingColumns(r_nscolumns, r_colnames, &r_colnos,
+ extractRemainingColumns(pstate,
+ r_nscolumns, r_colnames, &r_colnos,
&res_colnames, &res_colvars,
res_nscolumns + res_colindex);
+ /* If join has an alias, it syntactically hides all inputs */
+ if (j->alias)
+ {
+ for (k = 0; k < res_colindex; k++)
+ {
+ ParseNamespaceColumn *nscol = res_nscolumns + k;
+
+ nscol->p_varnosyn = j->rtindex;
+ nscol->p_varattnosyn = k + 1;
+ }
+ }
+
/*
* Now build an RTE and nsitem for the result of the join.
- * res_nscolumns isn't totally done yet, but that's OK because
- * addRangeTableEntryForJoin doesn't examine it, only store a pointer.
*/
nsitem = addRangeTableEntryForJoin(pstate,
res_colnames,
@@ -1454,31 +1546,16 @@ transformFromClauseItem(ParseState *pstate, Node *n,
j->alias,
true);
- j->rtindex = nsitem->p_rtindex;
+ /* Verify that we correctly predicted the join's RT index */
+ Assert(j->rtindex == nsitem->p_rtindex);
+ /* Cross-check number of columns, too */
+ Assert(res_colindex == list_length(nsitem->p_names->colnames));
/*
- * Now that we know the join RTE's rangetable index, we can fix up the
- * res_nscolumns data in places where it should contain that.
+ * Save a link to the JoinExpr in the proper element of p_joinexprs.
+ * Since we maintain that list lazily, it may be necessary to fill in
+ * empty entries before we can add the JoinExpr in the right place.
*/
- Assert(res_colindex == list_length(nsitem->p_names->colnames));
- for (k = 0; k < res_colindex; k++)
- {
- ParseNamespaceColumn *nscol = res_nscolumns + k;
-
- /* fill in join RTI for merged columns */
- if (nscol->p_varno == 0)
- nscol->p_varno = j->rtindex;
- if (nscol->p_varnosyn == 0)
- nscol->p_varnosyn = j->rtindex;
- /* if join has an alias, it syntactically hides all inputs */
- if (j->alias)
- {
- nscol->p_varnosyn = j->rtindex;
- nscol->p_varattnosyn = k + 1;
- }
- }
-
- /* make a matching link to the JoinExpr for later use */
for (k = list_length(pstate->p_joinexprs) + 1; k < j->rtindex; k++)
pstate->p_joinexprs = lappend(pstate->p_joinexprs, NULL);
pstate->p_joinexprs = lappend(pstate->p_joinexprs, j);
@@ -1547,10 +1624,13 @@ transformFromClauseItem(ParseState *pstate, Node *n,
* buildVarFromNSColumn -
* build a Var node using ParseNamespaceColumn data
*
- * We assume varlevelsup should be 0, and no location is specified
+ * This is used to construct joinaliasvars entries.
+ * We can assume varlevelsup should be 0, and no location is specified.
+ * Note also that no column SELECT privilege is requested here; that would
+ * happen only if the column is actually referenced in the query.
*/
static Var *
-buildVarFromNSColumn(ParseNamespaceColumn *nscol)
+buildVarFromNSColumn(ParseState *pstate, ParseNamespaceColumn *nscol)
{
Var *var;
@@ -1564,6 +1644,10 @@ buildVarFromNSColumn(ParseNamespaceColumn *nscol)
/* makeVar doesn't offer parameters for these, so set by hand: */
var->varnosyn = nscol->p_varnosyn;
var->varattnosyn = nscol->p_varattnosyn;
+
+ /* ... and update varnullingrels */
+ markNullableIfNeeded(pstate, var);
+
return var;
}
@@ -1675,6 +1759,47 @@ buildMergedJoinVar(ParseState *pstate, JoinType jointype,
return res_node;
}
+/*
+ * markRelsAsNulledBy -
+ * Mark the given jointree node and its children as nulled by join jindex
+ */
+static void
+markRelsAsNulledBy(ParseState *pstate, Node *n, int jindex)
+{
+ int varno;
+ ListCell *lc;
+
+ /* Note: we can't see FromExpr here */
+ if (IsA(n, RangeTblRef))
+ {
+ varno = ((RangeTblRef *) n)->rtindex;
+ }
+ else if (IsA(n, JoinExpr))
+ {
+ JoinExpr *j = (JoinExpr *) n;
+
+ /* recurse to children */
+ markRelsAsNulledBy(pstate, j->larg, jindex);
+ markRelsAsNulledBy(pstate, j->rarg, jindex);
+ varno = j->rtindex;
+ }
+ else
+ {
+ elog(ERROR, "unrecognized node type: %d", (int) nodeTag(n));
+ varno = 0; /* keep compiler quiet */
+ }
+
+ /*
+ * Now add jindex to the p_nullingrels set for relation varno. Since we
+ * maintain the p_nullingrels list lazily, we might need to extend it to
+ * make the varno'th entry exist.
+ */
+ while (list_length(pstate->p_nullingrels) < varno)
+ pstate->p_nullingrels = lappend(pstate->p_nullingrels, NULL);
+ lc = list_nth_cell(pstate->p_nullingrels, varno - 1);
+ lfirst(lc) = bms_add_member((Bitmapset *) lfirst(lc), jindex);
+}
+
/*
* setNamespaceColumnVisibility -
* Convenience subroutine to update cols_visible flags in a namespace list.
diff --git a/src/backend/parser/parse_coerce.c b/src/backend/parser/parse_coerce.c
index 34757da0fa..52787b6794 100644
--- a/src/backend/parser/parse_coerce.c
+++ b/src/backend/parser/parse_coerce.c
@@ -1042,7 +1042,7 @@ coerce_record_to_complex(ParseState *pstate, Node *node,
ParseNamespaceItem *nsitem;
nsitem = GetNSItemByRangeTablePosn(pstate, rtindex, sublevels_up);
- args = expandNSItemVars(nsitem, sublevels_up, vlocation, NULL);
+ args = expandNSItemVars(pstate, nsitem, sublevels_up, vlocation, NULL);
}
else
ereport(ERROR,
diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c
index 53e904ca6d..7ff41acb84 100644
--- a/src/backend/parser/parse_expr.c
+++ b/src/backend/parser/parse_expr.c
@@ -2478,6 +2478,9 @@ transformWholeRowRef(ParseState *pstate, ParseNamespaceItem *nsitem,
/* location is not filled in by makeWholeRowVar */
result->location = location;
+ /* mark Var if it's nulled by any outer joins */
+ markNullableIfNeeded(pstate, result);
+
/* mark relation as requiring whole-row SELECT access */
markVarForSelectPriv(pstate, result);
@@ -2505,6 +2508,8 @@ transformWholeRowRef(ParseState *pstate, ParseNamespaceItem *nsitem,
rowexpr->colnames = copyObject(nsitem->p_names->colnames);
rowexpr->location = location;
+ /* XXX we ought to mark the row as possibly nullable */
+
return (Node *) rowexpr;
}
}
diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c
index b490541f03..52b4a6e89d 100644
--- a/src/backend/parser/parse_relation.c
+++ b/src/backend/parser/parse_relation.c
@@ -763,6 +763,9 @@ scanNSItemForColumn(ParseState *pstate, ParseNamespaceItem *nsitem,
}
var->location = location;
+ /* Mark Var if it's nulled by any outer joins */
+ markNullableIfNeeded(pstate, var);
+
/* Require read access to the column */
markVarForSelectPriv(pstate, var);
@@ -1023,6 +1026,35 @@ searchRangeTableForCol(ParseState *pstate, const char *alias, const char *colnam
return fuzzystate;
}
+/*
+ * markNullableIfNeeded
+ * If the RTE referenced by the Var is nullable by outer join(s)
+ * at this point in the query, set var->varnullingrels to show that.
+ */
+void
+markNullableIfNeeded(ParseState *pstate, Var *var)
+{
+ int rtindex = var->varno;
+ Bitmapset *relids;
+
+ /* Find the appropriate pstate */
+ for (int lv = 0; lv < var->varlevelsup; lv++)
+ pstate = pstate->parentParseState;
+
+ /* Find currently-relevant join relids for the Var's rel */
+ if (rtindex > 0 && rtindex <= list_length(pstate->p_nullingrels))
+ relids = (Bitmapset *) list_nth(pstate->p_nullingrels, rtindex - 1);
+ else
+ relids = NULL;
+
+ /*
+ * Merge with any already-declared nulling rels. (Typically there won't
+ * be any, but let's get it right if there are.)
+ */
+ if (relids != NULL)
+ var->varnullingrels = bms_union(var->varnullingrels, relids);
+}
+
/*
* markRTEForSelectPriv
* Mark the specified column of the RTE with index rtindex
@@ -3087,7 +3119,7 @@ expandTupleDesc(TupleDesc tupdesc, Alias *eref, int count, int offset,
* the list elements mustn't be modified.
*/
List *
-expandNSItemVars(ParseNamespaceItem *nsitem,
+expandNSItemVars(ParseState *pstate, ParseNamespaceItem *nsitem,
int sublevels_up, int location,
List **colnames)
{
@@ -3123,6 +3155,10 @@ expandNSItemVars(ParseNamespaceItem *nsitem,
var->varnosyn = nscol->p_varnosyn;
var->varattnosyn = nscol->p_varattnosyn;
var->location = location;
+
+ /* ... and update varnullingrels */
+ markNullableIfNeeded(pstate, var);
+
result = lappend(result, var);
if (colnames)
*colnames = lappend(*colnames, colnameval);
@@ -3158,7 +3194,7 @@ expandNSItemAttrs(ParseState *pstate, ParseNamespaceItem *nsitem,
*var;
List *te_list = NIL;
- vars = expandNSItemVars(nsitem, sublevels_up, location, &names);
+ vars = expandNSItemVars(pstate, nsitem, sublevels_up, location, &names);
/*
* Require read access to the table. This is normally redundant with the
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index 0ca6beccb8..25781db5c1 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -1371,7 +1371,7 @@ ExpandSingleTable(ParseState *pstate, ParseNamespaceItem *nsitem,
List *vars;
ListCell *l;
- vars = expandNSItemVars(nsitem, sublevels_up, location, NULL);
+ vars = expandNSItemVars(pstate, nsitem, sublevels_up, location, NULL);
/*
* Require read access to the table. This is normally redundant with
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 89335d95e7..fbbbe647a4 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1090,6 +1090,14 @@ typedef struct RangeTblEntry
* alias Vars are generated only for merged columns). We keep these
* entries only because they're needed in expandRTE() and similar code.
*
+ * Vars appearing within joinaliasvars are marked with varnullingrels sets
+ * that describe the nulling effects of this join and lower ones. This is
+ * essential for FULL JOIN cases, because the COALESCE expression only
+ * describes the semantics correctly if its inputs have been nulled by the
+ * join. For other cases, it allows expandRTE() to generate a valid
+ * representation of the join's output without consulting additional
+ * parser state.
+ *
* Within a Query loaded from a stored rule, it is possible for non-merged
* joinaliasvars items to be null pointers, which are placeholders for
* (necessarily unreferenced) columns dropped since the rule was made.
diff --git a/src/include/parser/parse_node.h b/src/include/parser/parse_node.h
index 1a3792236a..f589112d5e 100644
--- a/src/include/parser/parse_node.h
+++ b/src/include/parser/parse_node.h
@@ -118,6 +118,13 @@ typedef Node *(*CoerceParamHook) (ParseState *pstate, Param *param,
* This is one-for-one with p_rtable, but contains NULLs for non-join
* RTEs, and may be shorter than p_rtable if the last RTE(s) aren't joins.
*
+ * p_nullingrels: list of Bitmapsets associated with p_rtable entries, each
+ * containing the set of outer-join RTE indexes that can null that relation
+ * at the current point in the parse tree. This is one-for-one with p_rtable,
+ * but may be shorter than p_rtable, in which case the missing entries are
+ * implicitly empty (NULL). That rule allows us to save work when the query
+ * contains no outer joins.
+ *
* p_joinlist: list of join items (RangeTblRef and JoinExpr nodes) that
* will become the fromlist of the query's top-level FromExpr node.
*
@@ -187,6 +194,7 @@ struct ParseState
List *p_rteperminfos; /* list of RTEPermissionInfo nodes for each
* RTE_RELATION entry in rtable */
List *p_joinexprs; /* JoinExprs for RTE_JOIN p_rtable entries */
+ List *p_nullingrels; /* Bitmapsets showing nulling outer joins */
List *p_joinlist; /* join items so far (will become FromExpr
* node's fromlist) */
List *p_namespace; /* currently-referenceable RTEs (List of
diff --git a/src/include/parser/parse_relation.h b/src/include/parser/parse_relation.h
index dfa584b6ac..67d9b1e412 100644
--- a/src/include/parser/parse_relation.h
+++ b/src/include/parser/parse_relation.h
@@ -41,6 +41,7 @@ extern Node *scanNSItemForColumn(ParseState *pstate, ParseNamespaceItem *nsitem,
int location);
extern Node *colNameToVar(ParseState *pstate, const char *colname, bool localonly,
int location);
+extern void markNullableIfNeeded(ParseState *pstate, Var *var);
extern void markVarForSelectPriv(ParseState *pstate, Var *var);
extern Relation parserOpenTable(ParseState *pstate, const RangeVar *relation,
int lockmode);
@@ -113,7 +114,7 @@ extern void errorMissingColumn(ParseState *pstate,
extern void expandRTE(RangeTblEntry *rte, int rtindex, int sublevels_up,
int location, bool include_dropped,
List **colnames, List **colvars);
-extern List *expandNSItemVars(ParseNamespaceItem *nsitem,
+extern List *expandNSItemVars(ParseState *pstate, ParseNamespaceItem *nsitem,
int sublevels_up, int location,
List **colnames);
extern List *expandNSItemAttrs(ParseState *pstate, ParseNamespaceItem *nsitem,
commit 2f8421adf64e6ec4d3701ded2f6cb99cdd47eaec
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Jan 23 14:31:10 2023 -0500
Teach the planner to cope with Vars bearing nullingrels.
The core idea of this step is to include varnullingrels in the
relid sets that qual clauses are considered to depend on.
So that we can still easily compare quals' relids to RelOptInfos'
relids, that means also adding outer join relids to the identifying
relids of join relations. Much of the bulk of this step is concerned
with fallout from the latter change.
I've resolved the previous squishiness entailed by outer join identity 3
by generating multiple versions of outer-join quals that could get moved
to a join level where they need to contain different nullingrels sets.
Now we have versions of such quals with the correct nullingrels for
each level where they could appear.
This requires a bit of new mechanism (RestrictInfo.has_clone/is_clone)
to prevent multiple versions of the same qual from getting used in the
plan. My worry about how that could work with EquivalenceClasses is
resolved by creating EquivalenceClasses only from the least-marked
version of a qual. (This doesn't really lose anything, since versions
with more nullingrels bits don't correspond to any equalities available
outside the nest of commuting outer joins.)
These extra versions of quals would also result in generating multiple
parameterized paths that differ only in what nullingrels they expect
for the Vars from the parameterization rel(s). That seems like it'd
be very wasteful, so I've arranged to generate such paths only from
the least-marked version of a qual (the has_clone version).
Unlike in the previous version of this patch, setrefs.c is able to
cross-check the nullingrel sets of most Vars and PlaceHolderVars to
ensure that they match up with what the previous plan step produces.
But there are three cases that I've so far punted on:
1. The targetlist and qpqual of an outer join node will contain
nullingrels bits for the outer join itself. To check exact matching to
the input, we'd need to know the OJ's relid as well as which input(s)
got nulled, neither of which is cheaply available in setrefs.c. For
now, it's just checking that such Vars have a superset of the input's
nullingrels bits.
2. Parameterized paths will generally refer to the least-marked version
of whichever outer-side Vars they use, which may not be what's actually
available from the outside of the nestloop. (We're relying on the join
ordering rules for that to be sensible.) Again, setrefs.c is in no
position to pass judgment on correctness, so it's just checking that
the parameter expression has a subset of the outer-side marking.
3. Row identity variables are not marked with any nullingrels, which
may not correspond to reality. I've punted on this by skipping the
checks when varattno <= 0.
Point 1 could be addressed if we were willing to add informational
fields to join plan nodes, which might be worth doing, but I'm not sure.
The other two points seem like the extra mechanisms needed for a
bulletproof check would be considerably more trouble than they'd be
worth.
There is still some confusion about which versions of a cloned qual
are actually necessary to check, which results in some extra filter
conditions showing up in a couple of regression test plans. There are
also some failure cases involving full joins that remain to be fixed.
This patch is already mighty big, so I'll address those failures
separately.
This step removes some low-hanging fruit from the old implementation,
such as the need to track lowest_nulling_outer_join during subquery
pullup. There's much more to do in that line, though.
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index 6d5d1a7eb2..a694ac4a13 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -273,7 +273,7 @@ merge_clump(PlannerInfo *root, List *clumps, Clump *new_clump, int num_gene,
* rel once we know the final targetlist (see
* grouping_planner).
*/
- if (!bms_equal(joinrel->relids, root->all_baserels))
+ if (!bms_equal(joinrel->relids, root->all_query_rels))
generate_useful_gather_paths(root, joinrel, false);
/* Find and save the cheapest paths for this joinrel */
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index c56d3f926d..26b294d5d0 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -211,9 +211,9 @@ make_one_rel(PlannerInfo *root, List *joinlist)
rel = make_rel_from_joinlist(root, joinlist);
/*
- * The result should join all and only the query's base rels.
+ * The result should join all and only the query's base + outer-join rels.
*/
- Assert(bms_equal(rel->relids, root->all_baserels));
+ Assert(bms_equal(rel->relids, root->all_query_rels));
return rel;
}
@@ -538,7 +538,7 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* the final scan/join targetlist is available (see grouping_planner).
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- !bms_equal(rel->relids, root->all_baserels))
+ !bms_equal(rel->relids, root->all_query_rels))
generate_useful_gather_paths(root, rel, false);
/* Now find the cheapest of the paths for this rel */
@@ -859,7 +859,7 @@ set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *
* to support an uncommon usage of second-rate sampling methods. Instead,
* if there is a risk that the query might perform an unsafe join, just
* wrap the SampleScan in a Materialize node. We can check for joins by
- * counting the membership of all_baserels (note that this correctly
+ * counting the membership of all_query_rels (note that this correctly
* counts inheritance trees as single rels). If we're inside a subquery,
* we can't easily check whether a join might occur in the outer query, so
* just assume one is possible.
@@ -868,7 +868,7 @@ set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *
* so check repeatable_across_scans last, even though that's a bit odd.
*/
if ((root->query_level > 1 ||
- bms_membership(root->all_baserels) != BMS_SINGLETON) &&
+ bms_membership(root->all_query_rels) != BMS_SINGLETON) &&
!(GetTsmRoutine(rte->tablesample->tsmhandler)->repeatable_across_scans))
{
path = (Path *) create_material_path(rel, path);
@@ -950,7 +950,7 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
if (enable_partitionwise_join &&
rel->reloptkind == RELOPT_BASEREL &&
rte->relkind == RELKIND_PARTITIONED_TABLE &&
- rel->attr_needed[InvalidAttrNumber - rel->min_attr] == NULL)
+ bms_is_empty(rel->attr_needed[InvalidAttrNumber - rel->min_attr]))
rel->consider_partitionwise_join = true;
/*
@@ -3409,7 +3409,7 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
* partial paths. We'll do the same for the topmost scan/join rel
* once we know the final targetlist (see grouping_planner).
*/
- if (!bms_equal(rel->relids, root->all_baserels))
+ if (!bms_equal(rel->relids, root->all_query_rels))
generate_useful_gather_paths(root, rel, false);
/* Find and save the cheapest paths for this rel */
diff --git a/src/backend/optimizer/path/clausesel.c b/src/backend/optimizer/path/clausesel.c
index 929a231112..61db6ad951 100644
--- a/src/backend/optimizer/path/clausesel.c
+++ b/src/backend/optimizer/path/clausesel.c
@@ -218,7 +218,7 @@ clauselist_selectivity_ext(PlannerInfo *root,
if (rinfo)
{
- ok = (bms_membership(rinfo->clause_relids) == BMS_SINGLETON) &&
+ ok = (rinfo->num_base_rels == 1) &&
(is_pseudo_constant_clause_relids(lsecond(expr->args),
rinfo->right_relids) ||
(varonleft = false,
@@ -579,30 +579,6 @@ find_single_rel_for_clauses(PlannerInfo *root, List *clauses)
return NULL; /* no clauses */
}
-/*
- * bms_is_subset_singleton
- *
- * Same result as bms_is_subset(s, bms_make_singleton(x)),
- * but a little faster and doesn't leak memory.
- *
- * Is this of use anywhere else? If so move to bitmapset.c ...
- */
-static bool
-bms_is_subset_singleton(const Bitmapset *s, int x)
-{
- switch (bms_membership(s))
- {
- case BMS_EMPTY_SET:
- return true;
- case BMS_SINGLETON:
- return bms_is_member(x, s);
- case BMS_MULTIPLE:
- return false;
- }
- /* can't get here... */
- return false;
-}
-
/*
* treat_as_join_clause -
* Decide whether an operator clause is to be handled by the
@@ -631,17 +607,20 @@ treat_as_join_clause(PlannerInfo *root, Node *clause, RestrictInfo *rinfo,
else
{
/*
- * Otherwise, it's a join if there's more than one relation used. We
- * can optimize this calculation if an rinfo was passed.
+ * Otherwise, it's a join if there's more than one base relation used.
+ * We can optimize this calculation if an rinfo was passed.
*
* XXX Since we know the clause is being evaluated at a join, the
* only way it could be single-relation is if it was delayed by outer
- * joins. Although we can make use of the restriction qual estimators
- * anyway, it seems likely that we ought to account for the
- * probability of injected nulls somehow.
+ * joins. We intentionally count only baserels here, not OJs that
+ * might be present in rinfo->clause_relids, so that we direct such
+ * cases to the restriction qual estimators not join estimators.
+ * Eventually some notice should be taken of the possibility of
+ * injected nulls, but we'll likely want to do that in the restriction
+ * estimators rather than starting to treat such cases as join quals.
*/
if (rinfo)
- return (bms_membership(rinfo->clause_relids) == BMS_MULTIPLE);
+ return (rinfo->num_base_rels > 1);
else
return (NumRelids(root, clause) > 1);
}
@@ -754,7 +733,9 @@ clause_selectivity_ext(PlannerInfo *root,
* for all non-JOIN_INNER cases.
*/
if (varRelid == 0 ||
- bms_is_subset_singleton(rinfo->clause_relids, varRelid))
+ rinfo->num_base_rels == 0 ||
+ (rinfo->num_base_rels == 1 &&
+ bms_is_member(varRelid, rinfo->clause_relids)))
{
/* Cacheable --- do we already have the result? */
if (jointype == JOIN_INNER)
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 29ae32d960..7d957a47a4 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4781,6 +4781,10 @@ compute_semi_anti_join_factors(PlannerInfo *root,
norm_sjinfo.syn_lefthand = outerrel->relids;
norm_sjinfo.syn_righthand = innerrel->relids;
norm_sjinfo.jointype = JOIN_INNER;
+ norm_sjinfo.ojrelid = 0;
+ norm_sjinfo.commute_above_l = NULL;
+ norm_sjinfo.commute_above_r = NULL;
+ norm_sjinfo.commute_below = NULL;
/* we don't bother trying to make the remaining fields valid */
norm_sjinfo.lhs_strict = false;
norm_sjinfo.delay_upper_joins = false;
@@ -4946,6 +4950,10 @@ approx_tuple_count(PlannerInfo *root, JoinPath *path, List *quals)
sjinfo.syn_lefthand = path->outerjoinpath->parent->relids;
sjinfo.syn_righthand = path->innerjoinpath->parent->relids;
sjinfo.jointype = JOIN_INNER;
+ sjinfo.ojrelid = 0;
+ sjinfo.commute_above_l = NULL;
+ sjinfo.commute_above_r = NULL;
+ sjinfo.commute_below = NULL;
/* we don't bother trying to make the remaining fields valid */
sjinfo.lhs_strict = false;
sjinfo.delay_upper_joins = false;
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 783ef5aa29..bfde80e8a4 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -29,6 +29,7 @@
#include "optimizer/paths.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "rewrite/rewriteManip.h"
#include "utils/lsyscache.h"
@@ -757,6 +758,12 @@ get_eclass_for_sort_expr(PlannerInfo *root,
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL) /* must be an outer join */
+ {
+ Assert(bms_is_member(i, root->outer_join_rels));
+ continue;
+ }
+
Assert(rel->reloptkind == RELOPT_BASEREL ||
rel->reloptkind == RELOPT_DEADREL);
@@ -1113,6 +1120,12 @@ generate_base_implied_equalities(PlannerInfo *root)
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL) /* must be an outer join */
+ {
+ Assert(bms_is_member(i, root->outer_join_rels));
+ continue;
+ }
+
Assert(rel->reloptkind == RELOPT_BASEREL);
rel->eclass_indexes = bms_add_member(rel->eclass_indexes,
@@ -2195,6 +2208,8 @@ static bool
reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
{
RestrictInfo *rinfo = ojcinfo->rinfo;
+ SpecialJoinInfo *sjinfo = ojcinfo->sjinfo;
+ Relids fjrelids = bms_make_singleton(sjinfo->ojrelid);
Expr *leftvar;
Expr *rightvar;
Oid opno,
@@ -2276,6 +2291,18 @@ reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
cfirst = (Node *) linitial(cexpr->args);
csecond = (Node *) lsecond(cexpr->args);
+ /*
+ * The COALESCE arguments will be marked as possibly nulled by
+ * the full join, while we wish to generate clauses that apply
+ * to the join's inputs. So we must strip the join from the
+ * nullingrels fields of cfirst/csecond before comparing them
+ * to leftvar/rightvar. (Perhaps with a less hokey
+ * representation for FULL JOIN USING output columns, this
+ * wouldn't be needed?)
+ */
+ cfirst = remove_nulling_relids(cfirst, fjrelids, NULL);
+ csecond = remove_nulling_relids(csecond, fjrelids, NULL);
+
if (equal(leftvar, cfirst) && equal(rightvar, csecond))
{
coal_idx = foreach_current_index(lc2);
@@ -3212,6 +3239,12 @@ get_eclass_indexes_for_relids(PlannerInfo *root, Relids relids)
{
RelOptInfo *rel = root->simple_rel_array[i];
+ if (rel == NULL) /* must be an outer join */
+ {
+ Assert(bms_is_member(i, root->outer_join_rels));
+ continue;
+ }
+
ec_indexes = bms_add_members(ec_indexes, rel->eclass_indexes);
}
return ec_indexes;
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index e13c8f1914..e9b784bcab 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -3352,13 +3352,13 @@ check_index_predicates(PlannerInfo *root, RelOptInfo *rel)
* Add on any equivalence-derivable join clauses. Computing the correct
* relid sets for generate_join_implied_equalities is slightly tricky
* because the rel could be a child rel rather than a true baserel, and in
- * that case we must remove its parents' relid(s) from all_baserels.
+ * that case we must subtract its parents' relid(s) from all_query_rels.
*/
if (rel->reloptkind == RELOPT_OTHER_MEMBER_REL)
- otherrels = bms_difference(root->all_baserels,
+ otherrels = bms_difference(root->all_query_rels,
find_childrel_parents(root, rel));
else
- otherrels = bms_difference(root->all_baserels, rel->relids);
+ otherrels = bms_difference(root->all_query_rels, rel->relids);
if (!bms_is_empty(otherrels))
clauselist =
@@ -3736,7 +3736,8 @@ match_index_to_operand(Node *operand,
*/
if (operand && IsA(operand, Var) &&
index->rel->relid == ((Var *) operand)->varno &&
- indkey == ((Var *) operand)->varattno)
+ indkey == ((Var *) operand)->varattno &&
+ ((Var *) operand)->varnullingrels == NULL)
return true;
}
else
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index d345c0437a..7591a7d81d 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -234,7 +234,9 @@ add_paths_to_joinrel(PlannerInfo *root,
* reduces the number of parameterized paths we have to deal with at
* higher join levels, without compromising the quality of the resulting
* plan. We express the restriction as a Relids set that must overlap the
- * parameterization of any proposed join path.
+ * parameterization of any proposed join path. Note: param_source_rels
+ * should contain only baserels, not OJ relids, so starting from
+ * all_baserels not all_query_rels is correct.
*/
foreach(lc, root->join_info_list)
{
@@ -365,6 +367,47 @@ allow_star_schema_join(PlannerInfo *root,
bms_nonempty_difference(inner_paramrels, outerrelids));
}
+/*
+ * If the parameterization is only partly satisfied by the outer rel,
+ * the unsatisfied part can't include any outer-join relids that could
+ * null rels of the satisfied part. That would imply that we're trying
+ * to use a clause involving a Var with nonempty varnullingrels at
+ * a join level where that value isn't yet computable.
+ */
+static inline bool
+have_unsafe_outer_join_ref(PlannerInfo *root,
+ Relids outerrelids,
+ Relids inner_paramrels)
+{
+ bool result = false;
+ Relids unsatisfied = bms_difference(inner_paramrels, outerrelids);
+
+ if (bms_overlap(unsatisfied, root->outer_join_rels))
+ {
+ ListCell *lc;
+
+ foreach(lc, root->join_info_list)
+ {
+ SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+
+ if (!bms_is_member(sjinfo->ojrelid, unsatisfied))
+ continue; /* not relevant */
+ if (bms_overlap(inner_paramrels, sjinfo->min_righthand) ||
+ (sjinfo->jointype == JOIN_FULL &&
+ bms_overlap(inner_paramrels, sjinfo->min_lefthand)))
+ {
+ result = true; /* doesn't work */
+ break;
+ }
+ }
+ }
+
+ /* Waste no memory when we reject a path here */
+ bms_free(unsatisfied);
+
+ return result;
+}
+
/*
* paraminfo_get_equal_hashops
* Determine if param_info and innerrel's lateral_vars can be hashed.
@@ -657,15 +700,16 @@ try_nestloop_path(PlannerInfo *root,
/*
* Check to see if proposed path is still parameterized, and reject if the
* parameterization wouldn't be sensible --- unless allow_star_schema_join
- * says to allow it anyway. Also, we must reject if have_dangerous_phv
- * doesn't like the look of it, which could only happen if the nestloop is
- * still parameterized.
+ * says to allow it anyway. Also, we must reject if either
+ * have_unsafe_outer_join_ref or have_dangerous_phv don't like the look of
+ * it, which could only happen if the nestloop is still parameterized.
*/
required_outer = calc_nestloop_required_outer(outerrelids, outer_paramrels,
innerrelids, inner_paramrels);
if (required_outer &&
((!bms_overlap(required_outer, extra->param_source_rels) &&
!allow_star_schema_join(root, outerrelids, inner_paramrels)) ||
+ have_unsafe_outer_join_ref(root, outerrelids, inner_paramrels) ||
have_dangerous_phv(root, outerrelids, inner_paramrels)))
{
/* Waste no memory when we reject a path here */
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 9a5930ce86..56dd1073c5 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -353,7 +353,10 @@ make_rels_by_clauseless_joins(PlannerInfo *root,
*
* Caller must supply not only the two rels, but the union of their relids.
* (We could simplify the API by computing joinrelids locally, but this
- * would be redundant work in the normal path through make_join_rel.)
+ * would be redundant work in the normal path through make_join_rel.
+ * Note that this value does NOT include the RT index of any outer join that
+ * might need to be performed here, so it's not the canonical identifier
+ * of the join relation.)
*
* On success, *sjinfo_p is set to NULL if this is to be a plain inner join,
* else it's set to point to the associated SpecialJoinInfo node. Also,
@@ -695,7 +698,7 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(rel1->relids, rel2->relids));
- /* Construct Relids set that identifies the joinrel. */
+ /* Construct Relids set that identifies the joinrel (without OJ as yet). */
joinrelids = bms_union(rel1->relids, rel2->relids);
/* Check validity and determine join type. */
@@ -707,6 +710,10 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
return NULL;
}
+ /* If we have an outer join, add its RTI to form the canonical relids. */
+ if (sjinfo && sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
+
/* Swap rels if needed to match the join info. */
if (reversed)
{
@@ -730,6 +737,10 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
sjinfo->syn_lefthand = rel1->relids;
sjinfo->syn_righthand = rel2->relids;
sjinfo->jointype = JOIN_INNER;
+ sjinfo->ojrelid = 0;
+ sjinfo->commute_above_l = NULL;
+ sjinfo->commute_above_r = NULL;
+ sjinfo->commute_below = NULL;
/* we don't bother trying to make the remaining fields valid */
sjinfo->lhs_strict = false;
sjinfo->delay_upper_joins = false;
@@ -1510,8 +1521,6 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(child_rel1->relids, child_rel2->relids));
- child_joinrelids = bms_union(child_rel1->relids, child_rel2->relids);
- appinfos = find_appinfos_by_relids(root, child_joinrelids, &nappinfos);
/*
* Construct SpecialJoinInfo from parent join relations's
@@ -1521,6 +1530,15 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
child_rel1->relids,
child_rel2->relids);
+ /* Build correct join relids for child join */
+ child_joinrelids = bms_union(child_rel1->relids, child_rel2->relids);
+ if (child_sjinfo->ojrelid != 0)
+ child_joinrelids = bms_add_member(child_joinrelids,
+ child_sjinfo->ojrelid);
+
+ /* Find the AppendRelInfo structures */
+ appinfos = find_appinfos_by_relids(root, child_joinrelids, &nappinfos);
+
/*
* Construct restrictions applicable to the child join from those
* applicable to the parent join.
@@ -1536,8 +1554,7 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
{
child_joinrel = build_child_join_rel(root, child_rel1, child_rel2,
joinrel, child_restrictlist,
- child_sjinfo,
- child_sjinfo->jointype);
+ child_sjinfo);
joinrel->part_rels[cnt_parts] = child_joinrel;
joinrel->live_parts = bms_add_member(joinrel->live_parts, cnt_parts);
joinrel->all_partrels = bms_add_members(joinrel->all_partrels,
@@ -1583,6 +1600,7 @@ build_child_join_sjinfo(PlannerInfo *root, SpecialJoinInfo *parent_sjinfo,
sjinfo->syn_righthand = adjust_child_relids(sjinfo->syn_righthand,
right_nappinfos,
right_appinfos);
+ /* outer-join relids need no adjustment */
sjinfo->semi_rhs_exprs = (List *) adjust_appendrel_attrs(root,
(Node *) sjinfo->semi_rhs_exprs,
right_nappinfos,
diff --git a/src/backend/optimizer/path/tidpath.c b/src/backend/optimizer/path/tidpath.c
index 6de994480b..05ad585a8f 100644
--- a/src/backend/optimizer/path/tidpath.c
+++ b/src/backend/optimizer/path/tidpath.c
@@ -59,6 +59,7 @@ IsCTIDVar(Var *var, RelOptInfo *rel)
if (var->varattno == SelfItemPointerAttributeNumber &&
var->vartype == TIDOID &&
var->varno == rel->relid &&
+ var->varnullingrels == NULL &&
var->varlevelsup == 0)
return true;
return false;
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index 51b653fed4..fbb652e7b0 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -34,7 +34,7 @@
/* local functions */
static bool join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo);
-static void remove_rel_from_query(PlannerInfo *root, int relid,
+static void remove_rel_from_query(PlannerInfo *root, int relid, int ojrelid,
Relids joinrelids);
static List *remove_rel_from_joinlist(List *joinlist, int relid, int *nremoved);
static bool rel_supports_distinctness(PlannerInfo *root, RelOptInfo *rel);
@@ -70,6 +70,7 @@ restart:
foreach(lc, root->join_info_list)
{
SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+ Relids joinrelids;
int innerrelid;
int nremoved;
@@ -84,9 +85,12 @@ restart:
*/
innerrelid = bms_singleton_member(sjinfo->min_righthand);
- remove_rel_from_query(root, innerrelid,
- bms_union(sjinfo->min_lefthand,
- sjinfo->min_righthand));
+ /* Compute the relid set for the join we are considering */
+ joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ if (sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
+
+ remove_rel_from_query(root, innerrelid, sjinfo->ojrelid, joinrelids);
/* We verify that exactly one reference gets removed from joinlist */
nremoved = 0;
@@ -188,6 +192,8 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
/* Compute the relid set for the join we are considering */
joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ if (sjinfo->ojrelid != 0)
+ joinrelids = bms_add_member(joinrelids, sjinfo->ojrelid);
/*
* We can't remove the join if any inner-rel attributes are used above the
@@ -247,6 +253,17 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
{
RestrictInfo *restrictinfo = (RestrictInfo *) lfirst(l);
+ /*
+ * If the current join commutes with some other outer join(s) via
+ * outer join identity 3, there will be multiple clones of its join
+ * clauses in the joininfo list. We want to consider only the
+ * has_clone form of such clauses. Processing more than one form
+ * would be wasteful, and also some of the others would confuse the
+ * RINFO_IS_PUSHED_DOWN test below.
+ */
+ if (restrictinfo->is_clone)
+ continue; /* ignore it */
+
/*
* If it's not a join clause for this outer join, we can't use it.
* Note that if the clause is pushed-down, then it is logically from
@@ -306,10 +323,12 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
* no longer treated as a baserel, and that attributes of other baserels
* are no longer marked as being needed at joins involving this rel.
* Also, join quals involving the rel have to be removed from the joininfo
- * lists, but only if they belong to the outer join identified by joinrelids.
+ * lists, but only if they belong to the outer join identified by ojrelid
+ * and joinrelids.
*/
static void
-remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
+remove_rel_from_query(PlannerInfo *root, int relid, int ojrelid,
+ Relids joinrelids)
{
RelOptInfo *rel = find_base_rel(root, relid);
List *joininfos;
@@ -346,6 +365,8 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
{
otherrel->attr_needed[attroff] =
bms_del_member(otherrel->attr_needed[attroff], relid);
+ otherrel->attr_needed[attroff] =
+ bms_del_member(otherrel->attr_needed[attroff], ojrelid);
}
}
@@ -353,6 +374,9 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
* Update all_baserels and related relid sets.
*/
root->all_baserels = bms_del_member(root->all_baserels, relid);
+ root->outer_join_rels = bms_del_member(root->outer_join_rels, ojrelid);
+ root->all_query_rels = bms_del_member(root->all_query_rels, relid);
+ root->all_query_rels = bms_del_member(root->all_query_rels, ojrelid);
/*
* Likewise remove references from SpecialJoinInfo data structures.
@@ -370,6 +394,14 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
sjinfo->min_righthand = bms_del_member(sjinfo->min_righthand, relid);
sjinfo->syn_lefthand = bms_del_member(sjinfo->syn_lefthand, relid);
sjinfo->syn_righthand = bms_del_member(sjinfo->syn_righthand, relid);
+ sjinfo->min_lefthand = bms_del_member(sjinfo->min_lefthand, ojrelid);
+ sjinfo->min_righthand = bms_del_member(sjinfo->min_righthand, ojrelid);
+ sjinfo->syn_lefthand = bms_del_member(sjinfo->syn_lefthand, ojrelid);
+ sjinfo->syn_righthand = bms_del_member(sjinfo->syn_righthand, ojrelid);
+ /* relid cannot appear in these fields, but ojrelid can: */
+ sjinfo->commute_above_l = bms_del_member(sjinfo->commute_above_l, ojrelid);
+ sjinfo->commute_above_r = bms_del_member(sjinfo->commute_above_r, ojrelid);
+ sjinfo->commute_below = bms_del_member(sjinfo->commute_below, ojrelid);
}
/*
@@ -401,8 +433,10 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
else
{
phinfo->ph_eval_at = bms_del_member(phinfo->ph_eval_at, relid);
+ phinfo->ph_eval_at = bms_del_member(phinfo->ph_eval_at, ojrelid);
Assert(!bms_is_empty(phinfo->ph_eval_at));
phinfo->ph_needed = bms_del_member(phinfo->ph_needed, relid);
+ phinfo->ph_needed = bms_del_member(phinfo->ph_needed, ojrelid);
}
}
@@ -427,7 +461,12 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
remove_join_clause_from_rels(root, rinfo, rinfo->required_relids);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrelids))
+ /*
+ * If the qual lists ojrelid in its required_relids, it must have come
+ * from above the outer join we're removing (so we need to keep it);
+ * if it does not, then it didn't and we can discard it.
+ */
+ if (bms_is_member(ojrelid, rinfo->required_relids))
{
/* Recheck that qual doesn't actually reference the target rel */
Assert(!bms_is_member(relid, rinfo->clause_relids));
@@ -439,6 +478,8 @@ remove_rel_from_query(PlannerInfo *root, int relid, Relids joinrelids)
rinfo->required_relids = bms_copy(rinfo->required_relids);
rinfo->required_relids = bms_del_member(rinfo->required_relids,
relid);
+ rinfo->required_relids = bms_del_member(rinfo->required_relids,
+ ojrelid);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
@@ -553,6 +594,7 @@ reduce_unique_semijoins(PlannerInfo *root)
/* Compute the relid set for the join we are considering */
joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ Assert(sjinfo->ojrelid == 0); /* SEMI joins don't have RT indexes */
/*
* Since we're only considering a single-rel RHS, any join clauses it
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 4acbd3df32..7bd654a77e 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -48,7 +48,9 @@ int join_collapse_limit;
*
* deconstruct_recurse recursively examines the join tree and builds a List
* (in depth-first traversal order) of JoinTreeItem structs, which are then
- * processed iteratively by deconstruct_distribute.
+ * processed iteratively by deconstruct_distribute. If there are outer
+ * joins, non-degenerate outer join clauses are processed in a third pass
+ * deconstruct_distribute_oj_quals.
*
* The JoinTreeItem structs themselves can be freed at the end of
* deconstruct_jointree, but do not modify or free their substructure,
@@ -60,17 +62,18 @@ typedef struct JoinTreeItem
/* Fields filled during deconstruct_recurse: */
Node *jtnode; /* jointree node to examine */
bool below_outer_join; /* is it below an outer join? */
- Relids qualscope; /* base Relids syntactically included in this
- * jointree node */
- Relids inner_join_rels; /* base Relids syntactically included in
- * inner joins appearing at or below this
- * jointree node */
+ Relids qualscope; /* base+OJ Relids syntactically included in
+ * this jointree node */
+ Relids inner_join_rels; /* base+OJ Relids syntactically included
+ * in inner joins appearing at or below
+ * this jointree node */
Relids left_rels; /* if join node, Relids of the left side */
Relids right_rels; /* if join node, Relids of the right side */
Relids nonnullable_rels; /* if outer join, Relids of the
* non-nullable side */
/* Fields filled during deconstruct_distribute: */
SpecialJoinInfo *sjinfo; /* if outer join, its SpecialJoinInfo */
+ List *oj_joinclauses; /* outer join quals not yet distributed */
} JoinTreeItem;
/* Elements of the postponed_qual_list used during deconstruct_distribute */
@@ -94,9 +97,13 @@ static void process_security_barrier_quals(PlannerInfo *root,
static SpecialJoinInfo *make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
- JoinType jointype, List *clause);
+ JoinType jointype, Index ojrelid,
+ List *clause);
static void compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo,
List *clause);
+static void deconstruct_distribute_oj_quals(PlannerInfo *root,
+ List *jtitems,
+ JoinTreeItem *jtitem);
static void distribute_quals_to_rels(PlannerInfo *root, List *clauses,
bool below_outer_join,
SpecialJoinInfo *sjinfo,
@@ -104,7 +111,11 @@ static void distribute_quals_to_rels(PlannerInfo *root, List *clauses,
Relids qualscope,
Relids ojscope,
Relids outerjoin_nonnullable,
- List **postponed_qual_list);
+ bool allow_equivalence,
+ bool has_clone,
+ bool is_clone,
+ List **postponed_qual_list,
+ List **postponed_oj_qual_list);
static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool below_outer_join,
SpecialJoinInfo *sjinfo,
@@ -112,7 +123,11 @@ static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
Relids qualscope,
Relids ojscope,
Relids outerjoin_nonnullable,
- List **postponed_qual_list);
+ bool allow_equivalence,
+ bool has_clone,
+ bool is_clone,
+ List **postponed_qual_list,
+ List **postponed_oj_qual_list);
static bool check_outerjoin_delay(PlannerInfo *root, Relids *relids_p,
Relids *nullable_relids_p, bool is_pushed_down);
static bool check_equivalence_delay(PlannerInfo *root,
@@ -290,10 +305,16 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
attno -= rel->min_attr;
if (rel->attr_needed[attno] == NULL)
{
- /* Variable not yet requested, so add to rel's targetlist */
- /* XXX is copyObject necessary here? */
- rel->reltarget->exprs = lappend(rel->reltarget->exprs,
- copyObject(var));
+ /*
+ * Variable not yet requested, so add to rel's targetlist.
+ *
+ * The value available at the rel's scan level has not been
+ * nulled by any outer join, so drop its varnullingrels.
+ * (We'll put those back as we climb up the join tree.)
+ */
+ var = copyObject(var);
+ var->varnullingrels = NULL;
+ rel->reltarget->exprs = lappend(rel->reltarget->exprs, var);
/* reltarget cost and width will be computed later */
}
rel->attr_needed[attno] = bms_add_members(rel->attr_needed[attno],
@@ -589,8 +610,10 @@ create_lateral_join_info(PlannerInfo *root)
varno = -1;
while ((varno = bms_next_member(eval_at, varno)) >= 0)
{
- RelOptInfo *brel = find_base_rel(root, varno);
+ RelOptInfo *brel = find_base_rel_ignore_join(root, varno);
+ if (brel == NULL)
+ continue; /* ignore outer joins in eval_at */
brel->lateral_relids = bms_add_members(brel->lateral_relids,
phinfo->ph_lateral);
}
@@ -681,7 +704,10 @@ create_lateral_join_info(PlannerInfo *root)
{
RelOptInfo *brel2 = root->simple_rel_array[rti2];
- Assert(brel2 != NULL && brel2->reloptkind == RELOPT_BASEREL);
+ if (brel2 == NULL)
+ continue; /* must be an OJ */
+
+ Assert(brel2->reloptkind == RELOPT_BASEREL);
brel2->lateral_referencers =
bms_add_member(brel2->lateral_referencers, rti);
}
@@ -743,6 +769,7 @@ deconstruct_jointree(PlannerInfo *root)
/* These are filled as we scan the jointree */
root->all_baserels = NULL;
+ root->outer_join_rels = NULL;
root->nullable_baserels = NULL;
/* Perform the initial scan of the jointree */
@@ -750,6 +777,9 @@ deconstruct_jointree(PlannerInfo *root)
false,
&item_list);
+ /* Now we can form the value of all_query_rels, too */
+ root->all_query_rels = bms_union(root->all_baserels, root->outer_join_rels);
+
/* Now scan all the jointree nodes again, and distribute quals */
foreach(lc, item_list)
{
@@ -762,6 +792,40 @@ deconstruct_jointree(PlannerInfo *root)
/* Shouldn't be any leftover postponed quals */
Assert(postponed_qual_list == NIL);
+ /*
+ * However, if there were any special joins then we may have some
+ * postponed LEFT JOIN clauses to deal with.
+ */
+ if (root->join_info_list)
+ {
+ /*
+ * XXX hack: when we call distribute_qual_to_rels to process one of
+ * these clauses, neither the owning SpecialJoinInfo nor any later
+ * ones can appear in root->join_info_list, else the wrong things will
+ * happen. Fake it out by emptying join_info_list and rebuilding it
+ * as we go. This works because join_info_list is only appended to
+ * during deconstruct_distribute, so we know we are examining
+ * SpecialJoinInfos bottom-up, just like the first time. We can get
+ * rid of this hack later, after fixing things so that
+ * distribute_qual_to_rels doesn't have that requirement about
+ * join_info_list.
+ */
+ root->join_info_list = NIL;
+
+ foreach(lc, item_list)
+ {
+ JoinTreeItem *jtitem = (JoinTreeItem *) lfirst(lc);
+
+ if (jtitem->oj_joinclauses != NIL)
+ deconstruct_distribute_oj_quals(root, item_list, jtitem);
+
+ /* XXX Rest of hack: rebuild join_info_list as we go */
+ if (jtitem->sjinfo)
+ root->join_info_list = lappend(root->join_info_list,
+ jtitem->sjinfo);
+ }
+ }
+
/* Don't need the JoinTreeItems any more */
list_free_deep(item_list);
@@ -905,6 +969,14 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
/* Compute qualscope etc */
jtitem->qualscope = bms_union(left_item->qualscope,
right_item->qualscope);
+ /* caution: ANTI join derived from SEMI will lack rtindex */
+ if (j->rtindex != 0)
+ {
+ jtitem->qualscope = bms_add_member(jtitem->qualscope,
+ j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
+ j->rtindex);
+ }
jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
right_item->inner_join_rels);
jtitem->left_rels = left_item->qualscope;
@@ -925,6 +997,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
/* Compute qualscope etc */
jtitem->qualscope = bms_union(left_item->qualscope,
right_item->qualscope);
+ /* SEMI join never has rtindex, so don't add to anything */
+ Assert(j->rtindex == 0);
jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
right_item->inner_join_rels);
jtitem->left_rels = left_item->qualscope;
@@ -952,6 +1026,11 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
/* Compute qualscope etc */
jtitem->qualscope = bms_union(left_item->qualscope,
right_item->qualscope);
+ Assert(j->rtindex != 0);
+ jtitem->qualscope = bms_add_member(jtitem->qualscope,
+ j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
+ j->rtindex);
jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
right_item->inner_join_rels);
jtitem->left_rels = left_item->qualscope;
@@ -1073,7 +1152,8 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
NULL,
root->qual_security_level,
jtitem->qualscope, NULL, NULL,
- NULL);
+ true, false, false,
+ NULL, NULL);
else
new_postponed_quals = lappend(new_postponed_quals, pq);
}
@@ -1087,7 +1167,8 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
NULL,
root->qual_security_level,
jtitem->qualscope, NULL, NULL,
- postponed_qual_list);
+ true, false, false,
+ postponed_qual_list, NULL);
}
else if (IsA(jtnode, JoinExpr))
{
@@ -1096,6 +1177,7 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
Relids ojscope;
List *my_quals;
SpecialJoinInfo *sjinfo;
+ List **postponed_oj_qual_list;
ListCell *l;
/*
@@ -1138,6 +1220,7 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
jtitem->right_rels,
jtitem->inner_join_rels,
j->jointype,
+ j->rtindex,
my_quals);
jtitem->sjinfo = sjinfo;
if (j->jointype == JOIN_SEMI)
@@ -1146,6 +1229,19 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
{
ojscope = bms_union(sjinfo->min_lefthand,
sjinfo->min_righthand);
+
+ /*
+ * Add back any commutable lower OJ relids that were removed
+ * from min_lefthand or min_righthand, else the ojscope
+ * cross-check in distribute_qual_to_rels will complain. If
+ * any such OJs were removed, we will postpone processing of
+ * non-degenerate clauses, so this addition doesn't affect
+ * anything except that cross-check and some Asserts. Real
+ * clause positioning decisions will be made later, when we
+ * revisit the postponed clauses.
+ */
+ if (sjinfo->commute_below)
+ ojscope = bms_add_members(ojscope, sjinfo->commute_below);
}
}
else
@@ -1154,6 +1250,18 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
ojscope = NULL;
}
+ /*
+ * If it's a left join with a join clause that is strict for the LHS,
+ * then we need to postpone handling of any non-degenerate join
+ * clauses, in case the join is able to commute with another left join
+ * per identity 3. (Degenerate clauses need not be postponed, since
+ * they will drop down below this join anyway.)
+ */
+ if (j->jointype == JOIN_LEFT && sjinfo->lhs_strict)
+ postponed_oj_qual_list = &jtitem->oj_joinclauses;
+ else
+ postponed_oj_qual_list = NULL;
+
/* Process the JOIN's qual clauses */
distribute_quals_to_rels(root, my_quals,
jtitem->below_outer_join,
@@ -1161,7 +1269,10 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
root->qual_security_level,
jtitem->qualscope,
ojscope, jtitem->nonnullable_rels,
- postponed_qual_list);
+ true, /* allow_equivalence */
+ false, false, /* not clones */
+ postponed_qual_list,
+ postponed_oj_qual_list);
/* And add the SpecialJoinInfo to join_info_list */
if (sjinfo)
@@ -1223,6 +1334,9 @@ process_security_barrier_quals(PlannerInfo *root,
qualscope,
qualscope,
NULL,
+ true,
+ false, false, /* not clones */
+ NULL,
NULL);
security_level++;
}
@@ -1236,10 +1350,11 @@ process_security_barrier_quals(PlannerInfo *root,
* Build a SpecialJoinInfo for the current outer join
*
* Inputs:
- * left_rels: the base Relids syntactically on outer side of join
- * right_rels: the base Relids syntactically on inner side of join
- * inner_join_rels: base Relids participating in inner joins below this one
+ * left_rels: the base+OJ Relids syntactically on outer side of join
+ * right_rels: the base+OJ Relids syntactically on inner side of join
+ * inner_join_rels: base+OJ Relids participating in inner joins below this one
* jointype: what it says (must always be LEFT, FULL, SEMI, or ANTI)
+ * ojrelid: RT index of the join RTE (0 for SEMI, which isn't in the RT list)
* clause: the outer join's join condition (in implicit-AND format)
*
* The node should eventually be appended to root->join_info_list, but we
@@ -1253,7 +1368,8 @@ static SpecialJoinInfo *
make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
- JoinType jointype, List *clause)
+ JoinType jointype, Index ojrelid,
+ List *clause)
{
SpecialJoinInfo *sjinfo = makeNode(SpecialJoinInfo);
Relids clause_relids;
@@ -1301,6 +1417,11 @@ make_outerjoininfo(PlannerInfo *root,
sjinfo->syn_lefthand = left_rels;
sjinfo->syn_righthand = right_rels;
sjinfo->jointype = jointype;
+ sjinfo->ojrelid = ojrelid;
+ /* these fields may get added to later: */
+ sjinfo->commute_above_l = NULL;
+ sjinfo->commute_above_r = NULL;
+ sjinfo->commute_below = NULL;
/* this always starts out false */
sjinfo->delay_upper_joins = false;
@@ -1348,6 +1469,7 @@ make_outerjoininfo(PlannerInfo *root,
foreach(l, root->join_info_list)
{
SpecialJoinInfo *otherinfo = (SpecialJoinInfo *) lfirst(l);
+ bool have_unsafe_phvs;
/*
* A full join is an optimization barrier: we can't associate into or
@@ -1363,6 +1485,9 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_lefthand = bms_add_member(min_lefthand,
+ otherinfo->ojrelid);
}
if (bms_overlap(right_rels, otherinfo->syn_lefthand) ||
bms_overlap(right_rels, otherinfo->syn_righthand))
@@ -1371,11 +1496,26 @@ make_outerjoininfo(PlannerInfo *root,
otherinfo->syn_lefthand);
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_righthand = bms_add_member(min_righthand,
+ otherinfo->ojrelid);
}
/* Needn't do anything else with the full join */
continue;
}
+ /*
+ * If our join condition contains any PlaceHolderVars that need to be
+ * evaluated above the lower OJ, then we can't commute with it.
+ */
+ if (otherinfo->ojrelid != 0)
+ have_unsafe_phvs =
+ contain_placeholder_references_to(root,
+ (Node *) clause,
+ otherinfo->ojrelid);
+ else
+ have_unsafe_phvs = false;
+
/*
* For a lower OJ in our LHS, if our join condition uses the lower
* join's RHS and is not strict for that rel, we must preserve the
@@ -1383,23 +1523,44 @@ make_outerjoininfo(PlannerInfo *root,
* min_lefthand. (We must use its full syntactic relset, not just its
* min_lefthand + min_righthand. This is because there might be other
* OJs below this one that this one can commute with, but we cannot
- * commute with them if we don't with this one.) Also, if the current
- * join is a semijoin or antijoin, we must preserve ordering
- * regardless of strictness.
+ * commute with them if we don't with this one.) Also, if we have
+ * unsafe PHVs or the current join is a semijoin or antijoin, we must
+ * preserve ordering regardless of strictness.
*
* Note: I believe we have to insist on being strict for at least one
* rel in the lower OJ's min_righthand, not its whole syn_righthand.
+ *
+ * When we don't need to preserve ordering, check to see if outer join
+ * identity 3 applies, and if so, remove the lower OJ's ojrelid from
+ * our min_lefthand so that commutation is allowed.
*/
if (bms_overlap(left_rels, otherinfo->syn_righthand))
{
if (bms_overlap(clause_relids, otherinfo->syn_righthand) &&
- (jointype == JOIN_SEMI || jointype == JOIN_ANTI ||
+ (have_unsafe_phvs ||
+ jointype == JOIN_SEMI || jointype == JOIN_ANTI ||
!bms_overlap(strict_relids, otherinfo->min_righthand)))
{
+ /* Preserve ordering */
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_lefthand);
min_lefthand = bms_add_members(min_lefthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_lefthand = bms_add_member(min_lefthand,
+ otherinfo->ojrelid);
+ }
+ else if (jointype == JOIN_LEFT &&
+ otherinfo->jointype == JOIN_LEFT &&
+ bms_overlap(strict_relids, otherinfo->min_righthand))
+ {
+ /* Identity 3 applies, so remove the ordering restriction */
+ min_lefthand = bms_del_member(min_lefthand, otherinfo->ojrelid);
+ /* Add commutability markers to both SpecialJoinInfos */
+ otherinfo->commute_above_l =
+ bms_add_member(otherinfo->commute_above_l, ojrelid);
+ sjinfo->commute_below =
+ bms_add_member(sjinfo->commute_below, otherinfo->ojrelid);
}
}
@@ -1414,8 +1575,8 @@ make_outerjoininfo(PlannerInfo *root,
* up with SpecialJoinInfos with identical min_righthands, which can
* confuse join_is_legal (see discussion in backend/optimizer/README).
*
- * Also, we must preserve ordering anyway if either the current join
- * or the lower OJ is either a semijoin or an antijoin.
+ * Also, we must preserve ordering anyway if we have unsafe PHVs, or
+ * if either this join or the lower OJ is a semijoin or antijoin.
*
* Here, we have to consider that "our join condition" includes any
* clauses that syntactically appeared above the lower OJ and below
@@ -1427,21 +1588,43 @@ make_outerjoininfo(PlannerInfo *root,
* join condition are not affected by them. The net effect is
* therefore sufficiently represented by the delay_upper_joins flag
* saved for us by check_outerjoin_delay.
+ *
+ * When we don't need to preserve ordering, check to see if outer join
+ * identity 3 applies, and if so, remove the lower OJ's ojrelid from
+ * our min_righthand so that commutation is allowed.
*/
if (bms_overlap(right_rels, otherinfo->syn_righthand))
{
if (bms_overlap(clause_relids, otherinfo->syn_righthand) ||
!bms_overlap(clause_relids, otherinfo->min_lefthand) ||
+ have_unsafe_phvs ||
jointype == JOIN_SEMI ||
jointype == JOIN_ANTI ||
otherinfo->jointype == JOIN_SEMI ||
otherinfo->jointype == JOIN_ANTI ||
!otherinfo->lhs_strict || otherinfo->delay_upper_joins)
{
+ /* Preserve ordering */
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_lefthand);
min_righthand = bms_add_members(min_righthand,
otherinfo->syn_righthand);
+ if (otherinfo->ojrelid != 0)
+ min_righthand = bms_add_member(min_righthand,
+ otherinfo->ojrelid);
+ }
+ else if (jointype == JOIN_LEFT &&
+ otherinfo->jointype == JOIN_LEFT &&
+ otherinfo->lhs_strict)
+ {
+ /* Identity 3 applies, so remove the ordering restriction */
+ min_righthand = bms_del_member(min_righthand,
+ otherinfo->ojrelid);
+ /* Add commutability markers to both SpecialJoinInfos */
+ otherinfo->commute_above_r =
+ bms_add_member(otherinfo->commute_above_r, ojrelid);
+ sjinfo->commute_below =
+ bms_add_member(sjinfo->commute_below, otherinfo->ojrelid);
}
}
}
@@ -1666,6 +1849,207 @@ compute_semijoin_info(PlannerInfo *root, SpecialJoinInfo *sjinfo, List *clause)
sjinfo->semi_rhs_exprs = semi_rhs_exprs;
}
+/*
+ * deconstruct_distribute_oj_quals
+ * Adjust LEFT JOIN quals to be suitable for commuted-left-join cases,
+ * then push them into the joinqual lists and EquivalenceClass structures.
+ *
+ * This runs immediately after we've completed the deconstruct_distribute scan.
+ * jtitems contains all the JoinTreeItems (in depth-first order), and jtitem
+ * is one that has postponed oj_joinclauses to deal with.
+ */
+static void
+deconstruct_distribute_oj_quals(PlannerInfo *root,
+ List *jtitems,
+ JoinTreeItem *jtitem)
+{
+ SpecialJoinInfo *sjinfo = jtitem->sjinfo;
+ Relids qualscope,
+ ojscope,
+ nonnullable_rels;
+
+ /* Recompute syntactic and semantic scopes of this left join */
+ qualscope = bms_union(sjinfo->syn_lefthand, sjinfo->syn_righthand);
+ qualscope = bms_add_member(qualscope, sjinfo->ojrelid);
+ ojscope = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand);
+ nonnullable_rels = sjinfo->syn_lefthand;
+
+ /*
+ * If this join can commute with any other ones per outer-join identity 3,
+ * and it is the one providing the join clause with flexible semantics,
+ * then we have to generate variants of the join clause with different
+ * nullingrels labeling. Otherwise, just push out the postponed clause
+ * as-is.
+ */
+ Assert(sjinfo->lhs_strict); /* else we shouldn't be here */
+ if (sjinfo->commute_above_r ||
+ bms_overlap(sjinfo->commute_below, sjinfo->syn_lefthand))
+ {
+ Relids joins_above;
+ Relids joins_below;
+ Relids joins_so_far;
+ List *quals;
+ ListCell *lc;
+
+ /*
+ * Put any OJ relids that were removed from min_righthand back into
+ * ojscope, else distribute_qual_to_rels will complain.
+ */
+ ojscope = bms_join(ojscope, bms_intersect(sjinfo->commute_below,
+ sjinfo->syn_righthand));
+
+ /* Identify the outer joins this one commutes with */
+ joins_above = sjinfo->commute_above_r;
+ joins_below = bms_intersect(sjinfo->commute_below,
+ sjinfo->syn_lefthand);
+
+ /*
+ * Generate qual variants with different sets of nullingrels bits.
+ *
+ * We only need bit-sets that correspond to the successively less
+ * deeply syntactically-nested subsets of this join and its
+ * commutators. That's true first because obviously only those forms
+ * of the Vars and PHVs could appear elsewhere in the query, and
+ * second because the outer join identities do not provide a way to
+ * re-order such joins in a way that would require different marking.
+ * (That is, while the current join may commute with several others,
+ * none of those others can commute with each other.) To visit the
+ * interesting joins in syntactic nesting order, we rely on the
+ * jtitems list to be ordered that way.
+ *
+ * We first strip out all the nullingrels bits corresponding to
+ * commutating joins below this one, and then successively put them
+ * back as we crawl up the join stack.
+ */
+ quals = jtitem->oj_joinclauses;
+ if (!bms_is_empty(joins_below))
+ quals = (List *) remove_nulling_relids((Node *) quals,
+ joins_below,
+ NULL);
+
+ joins_so_far = NULL;
+ foreach(lc, jtitems)
+ {
+ JoinTreeItem *otherjtitem = (JoinTreeItem *) lfirst(lc);
+ SpecialJoinInfo *othersj = otherjtitem->sjinfo;
+ bool below_sjinfo = false;
+ bool above_sjinfo = false;
+ Relids this_qualscope;
+ Relids this_ojscope;
+ bool allow_equivalence,
+ has_clone,
+ is_clone;
+
+ if (othersj == NULL)
+ continue; /* not an outer-join item, ignore */
+
+ if (bms_is_member(othersj->ojrelid, joins_below))
+ {
+ /* othersj commutes with sjinfo from below left */
+ below_sjinfo = true;
+ }
+ else if (othersj == sjinfo)
+ {
+ /* found our join in syntactic order */
+ Assert(bms_equal(joins_so_far, joins_below));
+ }
+ else if (bms_is_member(othersj->ojrelid, joins_above))
+ {
+ /* othersj commutes with sjinfo from above */
+ above_sjinfo = true;
+ }
+ else
+ {
+ /* othersj is not relevant, ignore */
+ continue;
+ }
+
+ /*
+ * When we are looking at joins above sjinfo, we are envisioning
+ * pushing sjinfo to above othersj, so add othersj's nulling bit
+ * before distributing the quals.
+ */
+ if (above_sjinfo)
+ quals = (List *)
+ add_nulling_relids((Node *) quals,
+ othersj->min_righthand,
+ bms_make_singleton(othersj->ojrelid));
+
+ /* Compute qualscope and ojscope for this join level */
+ this_qualscope = bms_union(qualscope, joins_so_far);
+ this_ojscope = bms_union(ojscope, joins_so_far);
+ if (above_sjinfo)
+ {
+ /* othersj is not yet in joins_so_far, but we need it */
+ this_qualscope = bms_add_member(this_qualscope,
+ othersj->ojrelid);
+ this_ojscope = bms_add_member(this_ojscope,
+ othersj->ojrelid);
+ /* sjinfo is in joins_so_far, and we don't want it */
+ this_ojscope = bms_del_member(this_ojscope,
+ sjinfo->ojrelid);
+ }
+
+ /*
+ * We generate EquivalenceClasses only from the first form of the
+ * quals, with the fewest nullingrels bits set. An EC made from
+ * this version of the quals can be useful below the outer-join
+ * nest, whereas versions with some nullingrels bits set would not
+ * be. We cannot generate ECs from more than one version, or
+ * we'll make nonsensical conclusions that Vars with nullingrels
+ * bits set are equal to their versions without. Fortunately,
+ * such ECs wouldn't be very useful anyway, because they'd equate
+ * values not observable outside the join nest. (See
+ * optimizer/README.)
+ *
+ * The first form of the quals is also the only one marked as
+ * has_clone rather than is_clone.
+ */
+ allow_equivalence = (joins_so_far == NULL);
+ has_clone = allow_equivalence;
+ is_clone = !has_clone;
+
+ distribute_quals_to_rels(root, quals,
+ true,
+ sjinfo,
+ root->qual_security_level,
+ this_qualscope,
+ this_ojscope, nonnullable_rels,
+ allow_equivalence,
+ has_clone,
+ is_clone,
+ NULL, NULL); /* no more postponement */
+
+ /*
+ * Adjust qual nulling bits for next level up, if needed. We
+ * don't want to put sjinfo's own bit in at all, and if we're
+ * above sjinfo then we did it already.
+ */
+ if (below_sjinfo)
+ quals = (List *)
+ add_nulling_relids((Node *) quals,
+ othersj->min_righthand,
+ bms_make_singleton(othersj->ojrelid));
+
+ /* ... and track joins processed so far */
+ joins_so_far = bms_add_member(joins_so_far, othersj->ojrelid);
+ }
+ }
+ else
+ {
+ /* No commutation possible, just process the postponed clauses */
+ distribute_quals_to_rels(root, jtitem->oj_joinclauses,
+ true,
+ sjinfo,
+ root->qual_security_level,
+ qualscope,
+ ojscope, nonnullable_rels,
+ true, /* allow_equivalence */
+ false, false, /* not clones */
+ NULL, NULL); /* no more postponement */
+ }
+}
+
/*****************************************************************************
*
@@ -1686,7 +2070,11 @@ distribute_quals_to_rels(PlannerInfo *root, List *clauses,
Relids qualscope,
Relids ojscope,
Relids outerjoin_nonnullable,
- List **postponed_qual_list)
+ bool allow_equivalence,
+ bool has_clone,
+ bool is_clone,
+ List **postponed_qual_list,
+ List **postponed_oj_qual_list)
{
ListCell *lc;
@@ -1701,7 +2089,11 @@ distribute_quals_to_rels(PlannerInfo *root, List *clauses,
qualscope,
ojscope,
outerjoin_nonnullable,
- postponed_qual_list);
+ allow_equivalence,
+ has_clone,
+ is_clone,
+ postponed_qual_list,
+ postponed_oj_qual_list);
}
}
@@ -1711,26 +2103,35 @@ distribute_quals_to_rels(PlannerInfo *root, List *clauses,
* (depending on whether the clause is a join) of each base relation
* mentioned in the clause. A RestrictInfo node is created and added to
* the appropriate list for each rel. Alternatively, if the clause uses a
- * mergejoinable operator and is not delayed by outer-join rules, enter
- * the left- and right-side expressions into the query's list of
- * EquivalenceClasses. Alternatively, if the clause needs to be treated
- * as belonging to a higher join level, just add it to postponed_qual_list.
+ * mergejoinable operator, enter its left- and right-side expressions into
+ * the query's EquivalenceClasses.
+ *
+ * In some cases, quals will be added to postponed_qual_list or
+ * postponed_oj_qual_list instead of being processed right away.
+ * These will be dealt with in later steps of deconstruct_jointree.
*
* 'clause': the qual clause to be distributed
* 'below_outer_join': true if the qual is from a JOIN/ON that is below the
* nullable side of a higher-level outer join
* 'sjinfo': join's SpecialJoinInfo (NULL for an inner join or WHERE clause)
* 'security_level': security_level to assign to the qual
- * 'qualscope': set of baserels the qual's syntactic scope covers
- * 'ojscope': NULL if not an outer-join qual, else the minimum set of baserels
- * needed to form this join
+ * 'qualscope': set of base+OJ rels the qual's syntactic scope covers
+ * 'ojscope': NULL if not an outer-join qual, else the minimum set of base+OJ
+ * rels needed to form this join
* 'outerjoin_nonnullable': NULL if not an outer-join qual, else the set of
- * baserels appearing on the outer (nonnullable) side of the join
+ * base+OJ rels appearing on the outer (nonnullable) side of the join
* (for FULL JOIN this includes both sides of the join, and must in fact
* equal qualscope)
+ * 'allow_equivalence': true if it's okay to convert clause into an
+ * EquivalenceClass
+ * 'has_clone': has_clone property to assign to the qual
+ * 'is_clone': is_clone property to assign to the qual
* 'postponed_qual_list': list of PostponedQual structs, which we can add
* this qual to if it turns out to belong to a higher join level.
* Can be NULL if caller knows postponement is impossible.
+ * 'postponed_oj_qual_list': if not NULL, non-degenerate outer join clauses
+ * should be added to this list instead of being processed (list entries
+ * are just the bare clauses)
*
* 'qualscope' identifies what level of JOIN the qual came from syntactically.
* 'ojscope' is needed if we decide to force the qual up to the outer-join
@@ -1748,7 +2149,11 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
Relids qualscope,
Relids ojscope,
Relids outerjoin_nonnullable,
- List **postponed_qual_list)
+ bool allow_equivalence,
+ bool has_clone,
+ bool is_clone,
+ List **postponed_qual_list,
+ List **postponed_oj_qual_list)
{
Relids relids;
bool is_pushed_down;
@@ -1842,7 +2247,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
{
relids =
get_relids_in_jointree((Node *) root->parse->jointree,
- false);
+ true, false);
qualscope = bms_copy(relids);
}
}
@@ -1885,8 +2290,18 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
{
/*
* The qual is attached to an outer join and mentions (some of the)
- * rels on the nonnullable side, so it's not degenerate.
- *
+ * rels on the nonnullable side, so it's not degenerate. If the
+ * caller wants to postpone handling such clauses, just add it to
+ * postponed_oj_qual_list and return. (The work we've done up to here
+ * will have to be redone later, but there's not much of it.)
+ */
+ if (postponed_oj_qual_list != NULL)
+ {
+ *postponed_oj_qual_list = lappend(*postponed_oj_qual_list, clause);
+ return;
+ }
+
+ /*
* We can't use such a clause to deduce equivalence (the left and
* right sides might be unequal above the join because one of them has
* gone to NULL) ... but we might be able to use it for more limited
@@ -1952,6 +2367,11 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
if (check_redundant_nullability_qual(root, clause))
return;
}
+ else if (!allow_equivalence)
+ {
+ /* Caller says it mustn't become an equivalence class */
+ maybe_equivalence = false;
+ }
else
{
/*
@@ -1986,11 +2406,22 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
outerjoin_nonnullable,
nullable_relids);
+ /* Apply appropriate clone marking, too */
+ restrictinfo->has_clone = has_clone;
+ restrictinfo->is_clone = is_clone;
+
/*
- * If it's a join clause (either naturally, or because delayed by
- * outer-join rules), add vars used in the clause to targetlists of their
- * relations, so that they will be emitted by the plan nodes that scan
- * those relations (else they won't be available at the join node!).
+ * If it's a join clause, add vars used in the clause to targetlists of
+ * their relations, so that they will be emitted by the plan nodes that
+ * scan those relations (else they won't be available at the join node!).
+ *
+ * Normally we mark the vars as needed at the join identified by "relids".
+ * However, if this is a clone clause then ignore the outer-join relids in
+ * that set. Otherwise, vars appearing in a cloned clause would end up
+ * marked as having to propagate to the highest one of the commuting
+ * joins, which would often be an overestimate. For such clauses, correct
+ * var propagation is ensured by making ojscope include input rels from
+ * both sides of the join.
*
* Note: if the clause gets absorbed into an EquivalenceClass then this
* may be unnecessary, but for now we have to do it to cover the case
@@ -2003,8 +2434,13 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
PVC_RECURSE_AGGREGATES |
PVC_RECURSE_WINDOWFUNCS |
PVC_INCLUDE_PLACEHOLDERS);
+ Relids where_needed;
- add_vars_to_targetlist(root, vars, relids);
+ if (is_clone)
+ where_needed = bms_intersect(relids, root->all_baserels);
+ else
+ where_needed = relids;
+ add_vars_to_targetlist(root, vars, where_needed);
list_free(vars);
}
@@ -2495,7 +2931,7 @@ process_implied_equality(PlannerInfo *root,
{
relids =
get_relids_in_jointree((Node *) root->parse->jointree,
- false);
+ true, false);
}
}
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 05f44faf6e..8a472bce0c 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2246,7 +2246,7 @@ preprocess_rowmarks(PlannerInfo *root)
* make a bitmapset of all base rels and then remove the items we don't
* need or have FOR [KEY] UPDATE/SHARE marks for.
*/
- rels = get_relids_in_jointree((Node *) parse->jointree, false);
+ rels = get_relids_in_jointree((Node *) parse->jointree, false, false);
if (parse->resultRelation)
rels = bms_del_member(rels, parse->resultRelation);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 85ba9d1ca1..8ea257f492 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -30,11 +30,21 @@
#include "utils/syscache.h"
+typedef enum
+{
+ NRM_EQUAL, /* expect exact match of nullingrels */
+ NRM_SUBSET, /* actual Var may have a subset of input */
+ NRM_SUPERSET /* actual Var may have a superset of input */
+} NullingRelsMatch;
+
typedef struct
{
int varno; /* RT index of Var */
AttrNumber varattno; /* attr number of Var */
AttrNumber resno; /* TLE position of Var */
+#ifdef USE_ASSERT_CHECKING
+ Bitmapset *varnullingrels; /* Var's varnullingrels */
+#endif
} tlist_vinfo;
typedef struct
@@ -60,6 +70,7 @@ typedef struct
indexed_tlist *inner_itlist;
Index acceptable_rel;
int rtoffset;
+ NullingRelsMatch nrm_match;
double num_exec;
} fix_join_expr_context;
@@ -69,6 +80,7 @@ typedef struct
indexed_tlist *subplan_itlist;
int newvarno;
int rtoffset;
+ NullingRelsMatch nrm_match;
double num_exec;
} fix_upper_expr_context;
@@ -159,7 +171,12 @@ static indexed_tlist *build_tlist_index(List *tlist);
static Var *search_indexed_tlist_for_var(Var *var,
indexed_tlist *itlist,
int newvarno,
- int rtoffset);
+ int rtoffset,
+ NullingRelsMatch nrm_match);
+static Var *search_indexed_tlist_for_phv(PlaceHolderVar *phv,
+ indexed_tlist *itlist,
+ int newvarno,
+ NullingRelsMatch nrm_match);
static Var *search_indexed_tlist_for_non_var(Expr *node,
indexed_tlist *itlist,
int newvarno);
@@ -172,14 +189,18 @@ static List *fix_join_expr(PlannerInfo *root,
indexed_tlist *outer_itlist,
indexed_tlist *inner_itlist,
Index acceptable_rel,
- int rtoffset, double num_exec);
+ int rtoffset,
+ NullingRelsMatch nrm_match,
+ double num_exec);
static Node *fix_join_expr_mutator(Node *node,
fix_join_expr_context *context);
static Node *fix_upper_expr(PlannerInfo *root,
Node *node,
indexed_tlist *subplan_itlist,
int newvarno,
- int rtoffset, double num_exec);
+ int rtoffset,
+ NullingRelsMatch nrm_match,
+ double num_exec);
static Node *fix_upper_expr_mutator(Node *node,
fix_upper_expr_context *context);
static List *set_returning_clause_references(PlannerInfo *root,
@@ -1118,13 +1139,13 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
fix_join_expr(root, splan->onConflictSet,
NULL, itlist,
linitial_int(splan->resultRelations),
- rtoffset, NUM_EXEC_QUAL(plan));
+ rtoffset, NRM_EQUAL, NUM_EXEC_QUAL(plan));
splan->onConflictWhere = (Node *)
fix_join_expr(root, (List *) splan->onConflictWhere,
NULL, itlist,
linitial_int(splan->resultRelations),
- rtoffset, NUM_EXEC_QUAL(plan));
+ rtoffset, NRM_EQUAL, NUM_EXEC_QUAL(plan));
pfree(itlist);
@@ -1181,6 +1202,7 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
NULL, itlist,
resultrel,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST(plan));
/* Fix quals too. */
@@ -1189,6 +1211,7 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
NULL, itlist,
resultrel,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL(plan));
}
}
@@ -1334,6 +1357,7 @@ set_indexonlyscan_references(PlannerInfo *root,
index_itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST((Plan *) plan));
plan->scan.plan.qual = (List *)
fix_upper_expr(root,
@@ -1341,6 +1365,7 @@ set_indexonlyscan_references(PlannerInfo *root,
index_itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) plan));
plan->recheckqual = (List *)
fix_upper_expr(root,
@@ -1348,6 +1373,7 @@ set_indexonlyscan_references(PlannerInfo *root,
index_itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) plan));
/* indexqual is already transformed to reference index columns */
plan->indexqual = fix_scan_list(root, plan->indexqual,
@@ -1554,6 +1580,7 @@ set_foreignscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST((Plan *) fscan));
fscan->scan.plan.qual = (List *)
fix_upper_expr(root,
@@ -1561,6 +1588,7 @@ set_foreignscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) fscan));
fscan->fdw_exprs = (List *)
fix_upper_expr(root,
@@ -1568,6 +1596,7 @@ set_foreignscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) fscan));
fscan->fdw_recheck_quals = (List *)
fix_upper_expr(root,
@@ -1575,6 +1604,7 @@ set_foreignscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) fscan));
pfree(itlist);
/* fdw_scan_tlist itself just needs fix_scan_list() adjustments */
@@ -1635,6 +1665,7 @@ set_customscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST((Plan *) cscan));
cscan->scan.plan.qual = (List *)
fix_upper_expr(root,
@@ -1642,6 +1673,7 @@ set_customscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) cscan));
cscan->custom_exprs = (List *)
fix_upper_expr(root,
@@ -1649,6 +1681,7 @@ set_customscan_references(PlannerInfo *root,
itlist,
INDEX_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) cscan));
pfree(itlist);
/* custom_scan_tlist itself just needs fix_scan_list() adjustments */
@@ -1835,6 +1868,7 @@ set_hash_references(PlannerInfo *root, Plan *plan, int rtoffset)
outer_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL(plan));
/* Hash doesn't project */
@@ -2170,6 +2204,7 @@ fix_scan_expr_mutator(Node *node, fix_scan_expr_context *context)
/* At scan level, we should always just evaluate the contained expr */
PlaceHolderVar *phv = (PlaceHolderVar *) node;
+ Assert(phv->phnullingrels == NULL);
return fix_scan_expr_mutator((Node *) phv->phexpr, context);
}
if (IsA(node, AlternativeSubPlan))
@@ -2227,6 +2262,7 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
inner_itlist,
(Index) 0,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) join));
/* Now do join-type-specific stuff */
@@ -2239,11 +2275,21 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
{
NestLoopParam *nlp = (NestLoopParam *) lfirst(lc);
+ /*
+ * Because we don't reparameterize parameterized paths to match
+ * the outer-join level at which they are used, Vars seen in the
+ * NestLoopParam expression may have nullingrels that are just a
+ * subset of those in the Vars actually available from the outer
+ * side. Not checking this exactly is a bit grotty, but the work
+ * needed to make things match up perfectly seems well out of
+ * proportion to the value.
+ */
nlp->paramval = (Var *) fix_upper_expr(root,
(Node *) nlp->paramval,
outer_itlist,
OUTER_VAR,
rtoffset,
+ NRM_SUBSET,
NUM_EXEC_TLIST(outer_plan));
/* Check we replaced any PlaceHolderVar with simple Var */
if (!(IsA(nlp->paramval, Var) &&
@@ -2261,6 +2307,7 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
inner_itlist,
(Index) 0,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) join));
}
else if (IsA(join, HashJoin))
@@ -2273,6 +2320,7 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
inner_itlist,
(Index) 0,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) join));
/*
@@ -2284,45 +2332,27 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
outer_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL((Plan *) join));
}
/*
* Now we need to fix up the targetlist and qpqual, which are logically
- * above the join. This means they should not re-use any input expression
- * that was computed in the nullable side of an outer join. Vars and
- * PlaceHolderVars are fine, so we can implement this restriction just by
- * clearing has_non_vars in the indexed_tlist structs.
- *
- * XXX This is a grotty workaround for the fact that we don't clearly
- * distinguish between a Var appearing below an outer join and the "same"
- * Var appearing above it. If we did, we'd not need to hack the matching
- * rules this way.
+ * above the join. This means that, if it's not an inner join, any Vars
+ * and PHVs appearing here should have nullingrels that include the
+ * effects of the outer join, ie they will have nullingrels equal to the
+ * input Vars' nullingrels plus the bit added by the outer join. We don't
+ * currently have enough info available here to identify what that should
+ * be, so we just tell fix_join_expr to accept superset nullingrels
+ * matches instead of exact ones.
*/
- switch (join->jointype)
- {
- case JOIN_LEFT:
- case JOIN_SEMI:
- case JOIN_ANTI:
- inner_itlist->has_non_vars = false;
- break;
- case JOIN_RIGHT:
- outer_itlist->has_non_vars = false;
- break;
- case JOIN_FULL:
- outer_itlist->has_non_vars = false;
- inner_itlist->has_non_vars = false;
- break;
- default:
- break;
- }
-
join->plan.targetlist = fix_join_expr(root,
join->plan.targetlist,
outer_itlist,
inner_itlist,
(Index) 0,
rtoffset,
+ (join->jointype == JOIN_INNER ? NRM_EQUAL : NRM_SUPERSET),
NUM_EXEC_TLIST((Plan *) join));
join->plan.qual = fix_join_expr(root,
join->plan.qual,
@@ -2330,6 +2360,7 @@ set_join_references(PlannerInfo *root, Join *join, int rtoffset)
inner_itlist,
(Index) 0,
rtoffset,
+ (join->jointype == JOIN_INNER ? NRM_EQUAL : NRM_SUPERSET),
NUM_EXEC_QUAL((Plan *) join));
pfree(outer_itlist);
@@ -2384,6 +2415,7 @@ set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset)
subplan_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST(plan));
}
else
@@ -2392,6 +2424,7 @@ set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset)
subplan_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST(plan));
tle = flatCopyTargetEntry(tle);
tle->expr = (Expr *) newexpr;
@@ -2405,6 +2438,7 @@ set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset)
subplan_itlist,
OUTER_VAR,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_QUAL(plan));
pfree(subplan_itlist);
@@ -2605,7 +2639,7 @@ set_dummy_tlist_references(Plan *plan, int rtoffset)
* tlist_member() searches.
*
* The result of this function is an indexed_tlist struct to pass to
- * search_indexed_tlist_for_var() or search_indexed_tlist_for_non_var().
+ * search_indexed_tlist_for_var() and siblings.
* When done, the indexed_tlist may be freed with a single pfree().
*/
static indexed_tlist *
@@ -2637,6 +2671,9 @@ build_tlist_index(List *tlist)
vinfo->varno = var->varno;
vinfo->varattno = var->varattno;
vinfo->resno = tle->resno;
+#ifdef USE_ASSERT_CHECKING
+ vinfo->varnullingrels = var->varnullingrels;
+#endif
vinfo++;
}
else if (tle->expr && IsA(tle->expr, PlaceHolderVar))
@@ -2689,6 +2726,9 @@ build_tlist_index_other_vars(List *tlist, int ignore_rel)
vinfo->varno = var->varno;
vinfo->varattno = var->varattno;
vinfo->resno = tle->resno;
+#ifdef USE_ASSERT_CHECKING
+ vinfo->varnullingrels = var->varnullingrels;
+#endif
vinfo++;
}
}
@@ -2708,10 +2748,17 @@ build_tlist_index_other_vars(List *tlist, int ignore_rel)
* modified varno/varattno (to wit, newvarno and the resno of the TLE entry).
* Also ensure that varnosyn is incremented by rtoffset.
* If no match, return NULL.
+ *
+ * In debugging builds, we cross-check the varnullingrels of the subplan
+ * output Var based on nrm_match. Most call sites should pass NRM_EQUAL
+ * indicating we expect an exact match. However, there are places where
+ * we haven't cleaned things up completely, and we have to settle for
+ * allowing subset or superset matches.
*/
static Var *
search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
- int newvarno, int rtoffset)
+ int newvarno, int rtoffset,
+ NullingRelsMatch nrm_match)
{
int varno = var->varno;
AttrNumber varattno = var->varattno;
@@ -2727,6 +2774,36 @@ search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
/* Found a match */
Var *newvar = copyVar(var);
+ /*
+ * Assert that we kept all the nullingrels machinations straight.
+ *
+ * XXX eventually reduce this to a plain Assert. Right now it's
+ * more useful to warn and keep going.
+ *
+ * XXX skip this check for system columns and whole-row Vars.
+ * That's because such Vars might be row identity Vars, which are
+ * generated without any varnullingrels. It'd be hard to do
+ * otherwise, since they're normally made very early in planning,
+ * when we haven't looked at the jointree yet and don't know which
+ * joins might null such Vars. Doesn't seem worth the expense to
+ * make them fully valid. (While it's slightly annoying that we
+ * thereby lose checking for user-written references to such
+ * columns, it seems unlikely that a bug in nullingrels logic
+ * would affect only system columns.)
+ */
+#ifdef USE_ASSERT_CHECKING
+ if (!(varattno <= 0 ||
+ (nrm_match == NRM_SUBSET ?
+ bms_is_subset(var->varnullingrels, vinfo->varnullingrels) :
+ nrm_match == NRM_SUPERSET ?
+ bms_is_subset(vinfo->varnullingrels, var->varnullingrels) :
+ bms_equal(vinfo->varnullingrels, var->varnullingrels))))
+ elog(WARNING, "bogus varnullingrels for (%d,%d): expected %s, found %s in subplan",
+ varno, varattno,
+ bmsToString(var->varnullingrels),
+ bmsToString(vinfo->varnullingrels));
+#endif
+
newvar->varno = newvarno;
newvar->varattno = vinfo->resno;
if (newvar->varnosyn > 0)
@@ -2739,15 +2816,74 @@ search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
}
/*
- * search_indexed_tlist_for_non_var --- find a non-Var in an indexed tlist
+ * search_indexed_tlist_for_phv --- find a PlaceHolderVar in an indexed tlist
*
* If a match is found, return a Var constructed to reference the tlist item.
* If no match, return NULL.
*
- * NOTE: it is a waste of time to call this unless itlist->has_ph_vars or
- * itlist->has_non_vars. Furthermore, set_join_references() relies on being
- * able to prevent matching of non-Vars by clearing itlist->has_non_vars,
- * so there's a correctness reason not to call it unless that's set.
+ * Cross-check phnullingrels as in search_indexed_tlist_for_var.
+ *
+ * NOTE: it is a waste of time to call this unless itlist->has_ph_vars.
+ */
+static Var *
+search_indexed_tlist_for_phv(PlaceHolderVar *phv,
+ indexed_tlist *itlist, int newvarno,
+ NullingRelsMatch nrm_match)
+{
+ ListCell *lc;
+
+ foreach(lc, itlist->tlist)
+ {
+ TargetEntry *tle = (TargetEntry *) lfirst(lc);
+
+ if (tle->expr && IsA(tle->expr, PlaceHolderVar))
+ {
+ PlaceHolderVar *subphv = (PlaceHolderVar *) tle->expr;
+ Var *newvar;
+
+ /*
+ * Analogously to search_indexed_tlist_for_var, we match on phid
+ * only. We don't use equal(), partially for speed but mostly
+ * because phnullingrels might not be exactly equal.
+ */
+ if (phv->phid != subphv->phid)
+ continue;
+
+ /*
+ * Assert that we kept all the nullingrels machinations straight.
+ *
+ * XXX eventually reduce this to a plain Assert. Right now it's
+ * more useful to warn and keep going.
+ */
+#ifdef USE_ASSERT_CHECKING
+ if (!(nrm_match == NRM_SUBSET ?
+ bms_is_subset(phv->phnullingrels, subphv->phnullingrels) :
+ nrm_match == NRM_SUPERSET ?
+ bms_is_subset(subphv->phnullingrels, phv->phnullingrels) :
+ bms_equal(subphv->phnullingrels, phv->phnullingrels)))
+ elog(WARNING, "bogus phnullingrels for %d: expected %s, found %s in subplan",
+ phv->phid,
+ bmsToString(phv->phnullingrels),
+ bmsToString(subphv->phnullingrels));
+#endif
+
+ /* Found a matching subplan output expression */
+ newvar = makeVarFromTargetEntry(newvarno, tle);
+ newvar->varnosyn = 0; /* wasn't ever a plain Var */
+ newvar->varattnosyn = 0;
+ return newvar;
+ }
+ }
+ return NULL; /* no match */
+}
+
+/*
+ * search_indexed_tlist_for_non_var --- find a non-Var/PHV in an indexed tlist
+ *
+ * If a match is found, return a Var constructed to reference the tlist item.
+ * If no match, return NULL.
+ *
+ * NOTE: it is a waste of time to call this unless itlist->has_non_vars.
*/
static Var *
search_indexed_tlist_for_non_var(Expr *node,
@@ -2854,6 +2990,7 @@ search_indexed_tlist_for_sortgroupref(Expr *node,
* 'acceptable_rel' is either zero or the rangetable index of a relation
* whose Vars may appear in the clause without provoking an error
* 'rtoffset': how much to increment varnos by
+ * 'nrm_match': as for search_indexed_tlist_for_var()
* 'num_exec': estimated number of executions of expression
*
* Returns the new expression tree. The original clause structure is
@@ -2866,6 +3003,7 @@ fix_join_expr(PlannerInfo *root,
indexed_tlist *inner_itlist,
Index acceptable_rel,
int rtoffset,
+ NullingRelsMatch nrm_match,
double num_exec)
{
fix_join_expr_context context;
@@ -2875,6 +3013,7 @@ fix_join_expr(PlannerInfo *root,
context.inner_itlist = inner_itlist;
context.acceptable_rel = acceptable_rel;
context.rtoffset = rtoffset;
+ context.nrm_match = nrm_match;
context.num_exec = num_exec;
return (List *) fix_join_expr_mutator((Node *) clauses, &context);
}
@@ -2896,7 +3035,8 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
newvar = search_indexed_tlist_for_var(var,
context->outer_itlist,
OUTER_VAR,
- context->rtoffset);
+ context->rtoffset,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
@@ -2907,7 +3047,8 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
newvar = search_indexed_tlist_for_var(var,
context->inner_itlist,
INNER_VAR,
- context->rtoffset);
+ context->rtoffset,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
@@ -2932,22 +3073,25 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* See if the PlaceHolderVar has bubbled up from a lower plan node */
if (context->outer_itlist && context->outer_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->outer_itlist,
- OUTER_VAR);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->outer_itlist,
+ OUTER_VAR,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
if (context->inner_itlist && context->inner_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->inner_itlist,
- INNER_VAR);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->inner_itlist,
+ INNER_VAR,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
/* If not supplied by input plans, evaluate the contained expr */
+ /* XXX can we assert something about phnullingrels? */
return fix_join_expr_mutator((Node *) phv->phexpr, context);
}
/* Try matching more complex expressions too, if tlists have any */
@@ -3006,6 +3150,7 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
* 'subplan_itlist': indexed target list for subplan (or index)
* 'newvarno': varno to use for Vars referencing tlist elements
* 'rtoffset': how much to increment varnos by
+ * 'nrm_match': as for search_indexed_tlist_for_var()
* 'num_exec': estimated number of executions of expression
*
* The resulting tree is a copy of the original in which all Var nodes have
@@ -3018,6 +3163,7 @@ fix_upper_expr(PlannerInfo *root,
indexed_tlist *subplan_itlist,
int newvarno,
int rtoffset,
+ NullingRelsMatch nrm_match,
double num_exec)
{
fix_upper_expr_context context;
@@ -3026,6 +3172,7 @@ fix_upper_expr(PlannerInfo *root,
context.subplan_itlist = subplan_itlist;
context.newvarno = newvarno;
context.rtoffset = rtoffset;
+ context.nrm_match = nrm_match;
context.num_exec = num_exec;
return fix_upper_expr_mutator(node, &context);
}
@@ -3044,7 +3191,8 @@ fix_upper_expr_mutator(Node *node, fix_upper_expr_context *context)
newvar = search_indexed_tlist_for_var(var,
context->subplan_itlist,
context->newvarno,
- context->rtoffset);
+ context->rtoffset,
+ context->nrm_match);
if (!newvar)
elog(ERROR, "variable not found in subplan target list");
return (Node *) newvar;
@@ -3056,13 +3204,15 @@ fix_upper_expr_mutator(Node *node, fix_upper_expr_context *context)
/* See if the PlaceHolderVar has bubbled up from a lower plan node */
if (context->subplan_itlist->has_ph_vars)
{
- newvar = search_indexed_tlist_for_non_var((Expr *) phv,
- context->subplan_itlist,
- context->newvarno);
+ newvar = search_indexed_tlist_for_phv(phv,
+ context->subplan_itlist,
+ context->newvarno,
+ context->nrm_match);
if (newvar)
return (Node *) newvar;
}
/* If not supplied by input plan, evaluate the contained expr */
+ /* XXX can we assert something about phnullingrels? */
return fix_upper_expr_mutator((Node *) phv->phexpr, context);
}
/* Try matching more complex expressions too, if tlist has any */
@@ -3169,6 +3319,7 @@ set_returning_clause_references(PlannerInfo *root,
NULL,
resultRelation,
rtoffset,
+ NRM_EQUAL,
NUM_EXEC_TLIST(topplan));
pfree(itlist);
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 37a7af8c66..029172333e 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -51,17 +51,28 @@ typedef struct pullup_replace_vars_context
* pullup (set only if target_rte->lateral) */
bool *outer_hasSubLinks; /* -> outer query's hasSubLinks */
int varno; /* varno of subquery */
- bool need_phvs; /* do we need PlaceHolderVars? */
- bool wrap_non_vars; /* do we need 'em on *all* non-Vars? */
+ bool wrap_non_vars; /* do we need all non-Var outputs to be PHVs? */
Node **rv_cache; /* cache for results with PHVs */
} pullup_replace_vars_context;
-typedef struct reduce_outer_joins_state
+typedef struct reduce_outer_joins_pass1_state
{
Relids relids; /* base relids within this subtree */
bool contains_outer; /* does subtree contain outer join(s)? */
List *sub_states; /* List of states for subtree components */
-} reduce_outer_joins_state;
+} reduce_outer_joins_pass1_state;
+
+typedef struct reduce_outer_joins_pass2_state
+{
+ Relids inner_reduced; /* OJ relids reduced to plain inner joins */
+ List *partial_reduced; /* List of partially reduced FULL joins */
+} reduce_outer_joins_pass2_state;
+
+typedef struct reduce_outer_joins_partial_state
+{
+ int full_join_rti; /* RT index of a formerly-FULL join */
+ Relids unreduced_side; /* relids in its still-nullable side */
+} reduce_outer_joins_partial_state;
static Node *pull_up_sublinks_jointree_recurse(PlannerInfo *root, Node *jtnode,
Relids *relids);
@@ -70,12 +81,10 @@ static Node *pull_up_sublinks_qual_recurse(PlannerInfo *root, Node *node,
Node **jtlink2, Relids available_rels2);
static Node *pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static Node *pull_up_simple_subquery(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static Node *pull_up_simple_union_all(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte);
@@ -92,7 +101,6 @@ static Node *pull_up_simple_values(PlannerInfo *root, Node *jtnode,
static bool is_simple_values(PlannerInfo *root, RangeTblEntry *rte);
static Node *pull_up_constant_function(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static bool is_simple_union_all(Query *subquery);
static bool is_simple_union_all_recurse(Node *setOp, Query *setOpQuery,
@@ -103,24 +111,26 @@ static bool jointree_contains_lateral_outer_refs(PlannerInfo *root,
Relids safe_upper_varnos);
static void perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars_context *rvcontext,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel);
static void replace_vars_in_jointree(Node *jtnode,
- pullup_replace_vars_context *context,
- JoinExpr *lowest_nulling_outer_join);
+ pullup_replace_vars_context *context);
static Node *pullup_replace_vars(Node *expr,
pullup_replace_vars_context *context);
static Node *pullup_replace_vars_callback(Var *var,
replace_rte_variables_context *context);
static Query *pullup_replace_vars_subquery(Query *query,
pullup_replace_vars_context *context);
-static reduce_outer_joins_state *reduce_outer_joins_pass1(Node *jtnode);
+static reduce_outer_joins_pass1_state *reduce_outer_joins_pass1(Node *jtnode);
static void reduce_outer_joins_pass2(Node *jtnode,
- reduce_outer_joins_state *state,
+ reduce_outer_joins_pass1_state *state1,
+ reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
List *forced_null_vars);
-static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode);
+static void report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
+ int rtindex, Relids relids);
+static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Relids *dropped_outer_joins);
static int get_result_relid(PlannerInfo *root, Node *jtnode);
static void remove_result_refs(PlannerInfo *root, int varno, Node *newjtloc);
static bool find_dependent_phvs(PlannerInfo *root, int varno);
@@ -761,7 +771,7 @@ pull_up_subqueries(PlannerInfo *root)
/* Recursion starts with no containing join nor appendrel */
root->parse->jointree = (FromExpr *)
pull_up_subqueries_recurse(root, (Node *) root->parse->jointree,
- NULL, NULL, NULL);
+ NULL, NULL);
/* We should still have a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
}
@@ -776,12 +786,6 @@ pull_up_subqueries(PlannerInfo *root)
* lowest_outer_join references the lowest such JoinExpr node; otherwise
* it is NULL. We use this to constrain the effects of LATERAL subqueries.
*
- * If this jointree node is within the nullable side of an outer join, then
- * lowest_nulling_outer_join references the lowest such JoinExpr node;
- * otherwise it is NULL. This forces use of the PlaceHolderVar mechanism for
- * references to non-nullable targetlist items, but only for references above
- * that join.
- *
* If we are looking at a member subquery of an append relation,
* containing_appendrel describes that relation; else it is NULL.
* This forces use of the PlaceHolderVar mechanism for all non-Var targetlist
@@ -798,15 +802,14 @@ pull_up_subqueries(PlannerInfo *root)
* Notice also that we can't turn pullup_replace_vars loose on the whole
* jointree, because it'd return a mutated copy of the tree; we have to
* invoke it just on the quals, instead. This behavior is what makes it
- * reasonable to pass lowest_outer_join and lowest_nulling_outer_join as
- * pointers rather than some more-indirect way of identifying the lowest
- * OJs. Likewise, we don't replace append_rel_list members but only their
- * substructure, so the containing_appendrel reference is safe to use.
+ * reasonable to pass lowest_outer_join as a pointer rather than some
+ * more-indirect way of identifying the lowest OJ. Likewise, we don't
+ * replace append_rel_list members but only their substructure, so the
+ * containing_appendrel reference is safe to use.
*/
static Node *
pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
/* Since this function recurses, it could be driven to stack overflow. */
@@ -833,7 +836,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
is_safe_append_member(rte->subquery)))
return pull_up_simple_subquery(root, jtnode, rte,
lowest_outer_join,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -866,7 +868,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
*/
if (rte->rtekind == RTE_FUNCTION)
return pull_up_constant_function(root, jtnode, rte,
- lowest_nulling_outer_join,
containing_appendrel);
/* Otherwise, do nothing at this node. */
@@ -882,7 +883,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
{
lfirst(l) = pull_up_subqueries_recurse(root, lfirst(l),
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
}
}
@@ -897,11 +897,9 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
case JOIN_INNER:
j->larg = pull_up_subqueries_recurse(root, j->larg,
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
lowest_outer_join,
- lowest_nulling_outer_join,
NULL);
break;
case JOIN_LEFT:
@@ -909,31 +907,25 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
case JOIN_ANTI:
j->larg = pull_up_subqueries_recurse(root, j->larg,
j,
- lowest_nulling_outer_join,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
- j,
j,
NULL);
break;
case JOIN_FULL:
j->larg = pull_up_subqueries_recurse(root, j->larg,
- j,
j,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
- j,
j,
NULL);
break;
case JOIN_RIGHT:
j->larg = pull_up_subqueries_recurse(root, j->larg,
- j,
j,
NULL);
j->rarg = pull_up_subqueries_recurse(root, j->rarg,
j,
- lowest_nulling_outer_join,
NULL);
break;
default:
@@ -963,7 +955,6 @@ pull_up_subqueries_recurse(PlannerInfo *root, Node *jtnode,
static Node *
pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
JoinExpr *lowest_outer_join,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -1112,31 +1103,25 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* The subquery's targetlist items are now in the appropriate form to
* insert into the top query, except that we may need to wrap them in
* PlaceHolderVars. Set up required context data for pullup_replace_vars.
+ * (Note that we should include the subquery's inner joins in relids,
+ * since it may include join alias vars referencing them.)
*/
rvcontext.root = root;
rvcontext.targetlist = subquery->targetList;
rvcontext.target_rte = rte;
if (rte->lateral)
rvcontext.relids = get_relids_in_jointree((Node *) subquery->jointree,
- true);
+ true, true);
else /* won't need relids */
rvcontext.relids = NULL;
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = varno;
- /* these flags will be set below, if needed */
- rvcontext.need_phvs = false;
+ /* this flag will be set below, if needed */
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(subquery->targetList) + 1) *
sizeof(Node *));
- /*
- * If we are under an outer join then non-nullable items and lateral
- * references may have to be turned into PlaceHolderVars.
- */
- if (lowest_nulling_outer_join != NULL)
- rvcontext.need_phvs = true;
-
/*
* If we are dealing with an appendrel member then anything that's not a
* simple Var has to be turned into a PlaceHolderVar. We force this to
@@ -1145,10 +1130,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* expression actually available from the appendrel.
*/
if (containing_appendrel != NULL)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* If the parent query uses grouping sets, we need a PlaceHolderVar for
@@ -1160,10 +1142,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* that pullup_replace_vars hasn't currently got.)
*/
if (parse->groupingSets)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* Replace all of the top query's references to the subquery's outputs
@@ -1171,7 +1150,6 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* replace any of the jointree structure.
*/
perform_pullup_replace_vars(root, &rvcontext,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -1238,7 +1216,8 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
{
Relids subrelids;
- subrelids = get_relids_in_jointree((Node *) subquery->jointree, false);
+ subrelids = get_relids_in_jointree((Node *) subquery->jointree,
+ true, false);
if (root->glob->lastPHId != 0)
substitute_phv_relids((Node *) parse, varno, subrelids);
fix_append_rel_relids(root, varno, subrelids);
@@ -1434,7 +1413,7 @@ pull_up_union_leaf_queries(Node *setOp, PlannerInfo *root, int parentRTindex,
rtr = makeNode(RangeTblRef);
rtr->rtindex = childRTindex;
(void) pull_up_subqueries_recurse(root, (Node *) rtr,
- NULL, NULL, appinfo);
+ NULL, appinfo);
}
else if (IsA(setOp, SetOperationStmt))
{
@@ -1571,7 +1550,7 @@ is_simple_subquery(PlannerInfo *root, Query *subquery, RangeTblEntry *rte,
{
restricted = true;
safe_upper_varnos = get_relids_in_jointree((Node *) lowest_outer_join,
- true);
+ true, true);
}
else
{
@@ -1683,7 +1662,6 @@ pull_up_simple_values(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte)
rvcontext.relids = NULL;
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = varno;
- rvcontext.need_phvs = false;
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(tlist) + 1) *
@@ -1695,7 +1673,7 @@ pull_up_simple_values(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte)
* any of the jointree structure. We can assume there's no outer joins or
* appendrels in the dummy Query that surrounds a VALUES RTE.
*/
- perform_pullup_replace_vars(root, &rvcontext, NULL, NULL);
+ perform_pullup_replace_vars(root, &rvcontext, NULL);
/*
* There should be no appendrels to fix, nor any outer joins and hence no
@@ -1794,7 +1772,6 @@ is_simple_values(PlannerInfo *root, RangeTblEntry *rte)
static Node *
pull_up_constant_function(PlannerInfo *root, Node *jtnode,
RangeTblEntry *rte,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -1846,40 +1823,26 @@ pull_up_constant_function(PlannerInfo *root, Node *jtnode,
rvcontext.outer_hasSubLinks = &parse->hasSubLinks;
rvcontext.varno = ((RangeTblRef *) jtnode)->rtindex;
- /* these flags will be set below, if needed */
- rvcontext.need_phvs = false;
+ /* this flag will be set below, if needed */
rvcontext.wrap_non_vars = false;
/* initialize cache array with indexes 0 .. length(tlist) */
rvcontext.rv_cache = palloc0((list_length(rvcontext.targetlist) + 1) *
sizeof(Node *));
- /*
- * If we are under an outer join then non-nullable items and lateral
- * references may have to be turned into PlaceHolderVars.
- */
- if (lowest_nulling_outer_join != NULL)
- rvcontext.need_phvs = true;
-
/*
* If we are dealing with an appendrel member then anything that's not a
* simple Var has to be turned into a PlaceHolderVar. (See comments in
* pull_up_simple_subquery().)
*/
if (containing_appendrel != NULL)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* If the parent query uses grouping sets, we need a PlaceHolderVar for
* anything that's not a simple Var.
*/
if (parse->groupingSets)
- {
- rvcontext.need_phvs = true;
rvcontext.wrap_non_vars = true;
- }
/*
* Replace all of the top query's references to the RTE's output with
@@ -1887,7 +1850,6 @@ pull_up_constant_function(PlannerInfo *root, Node *jtnode,
* jointree structure.
*/
perform_pullup_replace_vars(root, &rvcontext,
- lowest_nulling_outer_join,
containing_appendrel);
/*
@@ -2112,13 +2074,11 @@ jointree_contains_lateral_outer_refs(PlannerInfo *root, Node *jtnode,
*
* Caller has already filled *rvcontext with data describing what to
* substitute for Vars referencing the target subquery. In addition
- * we need the identity of the lowest outer join that can null the
- * target subquery, and its containing appendrel if any.
+ * we need the identity of the containing appendrel if any.
*/
static void
perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars_context *rvcontext,
- JoinExpr *lowest_nulling_outer_join,
AppendRelInfo *containing_appendrel)
{
Query *parse = root->parse;
@@ -2128,18 +2088,18 @@ perform_pullup_replace_vars(PlannerInfo *root,
* If we are considering an appendrel child subquery (that is, a UNION ALL
* member query that we're pulling up), then the only part of the upper
* query that could reference the child yet is the translated_vars list of
- * the associated AppendRelInfo. Furthermore, we do not need to insert
- * PHVs in the AppendRelInfo --- there isn't any outer join between.
+ * the associated AppendRelInfo. Furthermore, we do not want to force use
+ * of PHVs in the AppendRelInfo --- there isn't any outer join between.
*/
if (containing_appendrel)
{
- bool save_need_phvs = rvcontext->need_phvs;
+ bool save_wrap_non_vars = rvcontext->wrap_non_vars;
- rvcontext->need_phvs = false;
+ rvcontext->wrap_non_vars = false;
containing_appendrel->translated_vars = (List *)
pullup_replace_vars((Node *) containing_appendrel->translated_vars,
rvcontext);
- rvcontext->need_phvs = save_need_phvs;
+ rvcontext->wrap_non_vars = save_wrap_non_vars;
return;
}
@@ -2190,8 +2150,7 @@ perform_pullup_replace_vars(PlannerInfo *root,
pullup_replace_vars((Node *) action->targetList, rvcontext);
}
}
- replace_vars_in_jointree((Node *) parse->jointree, rvcontext,
- lowest_nulling_outer_join);
+ replace_vars_in_jointree((Node *) parse->jointree, rvcontext);
Assert(parse->setOperations == NULL);
parse->havingQual = pullup_replace_vars(parse->havingQual, rvcontext);
@@ -2208,12 +2167,6 @@ perform_pullup_replace_vars(PlannerInfo *root,
/*
* Replace references in the joinaliasvars lists of join RTEs.
- *
- * You might think that we could avoid using PHVs for alias vars of joins
- * below lowest_nulling_outer_join, but that doesn't work because the
- * alias vars could be referenced above that join; we need the PHVs to be
- * present in such references after the alias vars get flattened. (It
- * might be worth trying to be smarter here, someday.)
*/
foreach(lc, parse->rtable)
{
@@ -2230,14 +2183,10 @@ perform_pullup_replace_vars(PlannerInfo *root,
* Helper routine for perform_pullup_replace_vars: do pullup_replace_vars on
* every expression in the jointree, without changing the jointree structure
* itself. Ugly, but there's no other way...
- *
- * If we are at or below lowest_nulling_outer_join, we can suppress use of
- * PlaceHolderVars wrapped around the replacement expressions.
*/
static void
replace_vars_in_jointree(Node *jtnode,
- pullup_replace_vars_context *context,
- JoinExpr *lowest_nulling_outer_join)
+ pullup_replace_vars_context *context)
{
if (jtnode == NULL)
return;
@@ -2247,10 +2196,8 @@ replace_vars_in_jointree(Node *jtnode,
* If the RangeTblRef refers to a LATERAL subquery (that isn't the
* same subquery we're pulling up), it might contain references to the
* target subquery, which we must replace. We drive this from the
- * jointree scan, rather than a scan of the rtable, for a couple of
- * reasons: we can avoid processing no-longer-referenced RTEs, and we
- * can use the appropriate setting of need_phvs depending on whether
- * the RTE is above possibly-nulling outer joins or not.
+ * jointree scan, rather than a scan of the rtable, so that we can
+ * avoid processing no-longer-referenced RTEs.
*/
int varno = ((RangeTblRef *) jtnode)->rtindex;
@@ -2307,42 +2254,30 @@ replace_vars_in_jointree(Node *jtnode,
ListCell *l;
foreach(l, f->fromlist)
- replace_vars_in_jointree(lfirst(l), context,
- lowest_nulling_outer_join);
+ replace_vars_in_jointree(lfirst(l), context);
f->quals = pullup_replace_vars(f->quals, context);
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- bool save_need_phvs = context->need_phvs;
+ bool save_wrap_non_vars = context->wrap_non_vars;
- if (j == lowest_nulling_outer_join)
- {
- /* no more PHVs in or below this join */
- context->need_phvs = false;
- lowest_nulling_outer_join = NULL;
- }
- replace_vars_in_jointree(j->larg, context, lowest_nulling_outer_join);
- replace_vars_in_jointree(j->rarg, context, lowest_nulling_outer_join);
+ replace_vars_in_jointree(j->larg, context);
+ replace_vars_in_jointree(j->rarg, context);
/*
- * Use PHVs within the join quals of a full join, even when it's the
- * lowest nulling outer join. Otherwise, we cannot identify which
- * side of the join a pulled-up var-free expression came from, which
- * can lead to failure to make a plan at all because none of the quals
- * appear to be mergeable or hashable conditions. For this purpose we
- * don't care about the state of wrap_non_vars, so leave it alone.
+ * Use PHVs within the join quals of a full join. Otherwise, we
+ * cannot identify which side of the join a pulled-up var-free
+ * expression came from, which can lead to failure to make a plan at
+ * all because none of the quals appear to be mergeable or hashable
+ * conditions.
*/
if (j->jointype == JOIN_FULL)
- context->need_phvs = true;
+ context->wrap_non_vars = true;
j->quals = pullup_replace_vars(j->quals, context);
- /*
- * We don't bother to update the colvars list, since it won't be used
- * again ...
- */
- context->need_phvs = save_need_phvs;
+ context->wrap_non_vars = save_wrap_non_vars;
}
else
elog(ERROR, "unrecognized node type: %d",
@@ -2371,8 +2306,18 @@ pullup_replace_vars_callback(Var *var,
{
pullup_replace_vars_context *rcon = (pullup_replace_vars_context *) context->callback_arg;
int varattno = var->varattno;
+ bool need_phv;
Node *newnode;
+ /*
+ * We need a PlaceHolderVar if the Var-to-be-replaced has nonempty
+ * varnullingrels (unless we find below that the replacement expression is
+ * a Var or PlaceHolderVar that we can just add the nullingrels to). We
+ * also need one if the caller has instructed us that all non-Var/PHV
+ * replacements need to be wrapped for identification purposes.
+ */
+ need_phv = (var->varnullingrels != NULL) || rcon->wrap_non_vars;
+
/*
* If PlaceHolderVars are needed, we cache the modified expressions in
* rcon->rv_cache[]. This is not in hopes of any material speed gain
@@ -2381,13 +2326,16 @@ pullup_replace_vars_callback(Var *var,
* and possibly prevent optimizations that rely on recognizing different
* references to the same subquery output as being equal(). So it's worth
* a bit of extra effort to avoid it.
+ *
+ * The cached items have phlevelsup = 0 and phnullingrels = NULL; we'll
+ * copy them and adjust those values for this reference site below.
*/
- if (rcon->need_phvs &&
+ if (need_phv &&
varattno >= InvalidAttrNumber &&
varattno <= list_length(rcon->targetlist) &&
rcon->rv_cache[varattno] != NULL)
{
- /* Just copy the entry and fall through to adjust its varlevelsup */
+ /* Just copy the entry and fall through to adjust phlevelsup etc */
newnode = copyObject(rcon->rv_cache[varattno]);
}
else if (varattno == InvalidAttrNumber)
@@ -2396,7 +2344,7 @@ pullup_replace_vars_callback(Var *var,
RowExpr *rowexpr;
List *colnames;
List *fields;
- bool save_need_phvs = rcon->need_phvs;
+ bool save_wrap_non_vars = rcon->wrap_non_vars;
int save_sublevelsup = context->sublevels_up;
/*
@@ -2407,18 +2355,18 @@ pullup_replace_vars_callback(Var *var,
* the RowExpr for use of the executor and ruleutils.c.
*
* In order to be able to cache the results, we always generate the
- * expansion with varlevelsup = 0, and then adjust if needed.
+ * expansion with varlevelsup = 0, and then adjust below if needed.
*/
expandRTE(rcon->target_rte,
var->varno, 0 /* not varlevelsup */ , var->location,
(var->vartype != RECORDOID),
&colnames, &fields);
- /* Adjust the generated per-field Vars, but don't insert PHVs */
- rcon->need_phvs = false;
+ /* Expand the generated per-field Vars, but don't insert PHVs there */
+ rcon->wrap_non_vars = false;
context->sublevels_up = 0; /* to match the expandRTE output */
fields = (List *) replace_rte_variables_mutator((Node *) fields,
context);
- rcon->need_phvs = save_need_phvs;
+ rcon->wrap_non_vars = save_wrap_non_vars;
context->sublevels_up = save_sublevelsup;
rowexpr = makeNode(RowExpr);
@@ -2436,14 +2384,13 @@ pullup_replace_vars_callback(Var *var,
* expression to yield NULL, not ROW(NULL,NULL,...) when it is forced
* to null by an outer join.
*/
- if (rcon->need_phvs)
+ if (need_phv)
{
- /* RowExpr is certainly not strict, so always need PHV */
newnode = (Node *)
make_placeholder_expr(rcon->root,
(Expr *) newnode,
bms_make_singleton(rcon->varno));
- /* cache it with the PHV, and with varlevelsup still zero */
+ /* cache it with the PHV, and with phlevelsup etc not set yet */
rcon->rv_cache[InvalidAttrNumber] = copyObject(newnode);
}
}
@@ -2460,7 +2407,7 @@ pullup_replace_vars_callback(Var *var,
newnode = (Node *) copyObject(tle->expr);
/* Insert PlaceHolderVar if needed */
- if (rcon->need_phvs)
+ if (need_phv)
{
bool wrap;
@@ -2486,69 +2433,61 @@ pullup_replace_vars_callback(Var *var,
/* No need to wrap a PlaceHolderVar with another one, either */
wrap = false;
}
- else if (rcon->wrap_non_vars)
- {
- /* Wrap all non-Vars in a PlaceHolderVar */
- wrap = true;
- }
else
{
/*
- * If it contains a Var of the subquery being pulled up, and
- * does not contain any non-strict constructs, then it's
- * certainly nullable so we don't need to insert a
- * PlaceHolderVar.
- *
- * This analysis could be tighter: in particular, a non-strict
- * construct hidden within a lower-level PlaceHolderVar is not
- * reason to add another PHV. But for now it doesn't seem
- * worth the code to be more exact.
- *
- * Note: in future maybe we should insert a PlaceHolderVar
- * anyway, if the tlist item is expensive to evaluate?
- *
- * For a LATERAL subquery, we have to check the actual var
- * membership of the node, but if it's non-lateral then any
- * level-zero var must belong to the subquery.
+ * Must wrap, either because we need a place to insert
+ * varnullingrels or because caller told us to wrap
+ * everything.
*/
- if ((rcon->target_rte->lateral ?
- bms_overlap(pull_varnos(rcon->root, (Node *) newnode),
- rcon->relids) :
- contain_vars_of_level((Node *) newnode, 0)) &&
- !contain_nonstrict_functions((Node *) newnode))
- {
- /* No wrap needed */
- wrap = false;
- }
- else
- {
- /* Else wrap it in a PlaceHolderVar */
- wrap = true;
- }
+ wrap = true;
}
if (wrap)
+ {
newnode = (Node *)
make_placeholder_expr(rcon->root,
(Expr *) newnode,
bms_make_singleton(rcon->varno));
- /*
- * Cache it if possible (ie, if the attno is in range, which it
- * probably always should be). We can cache the value even if we
- * decided we didn't need a PHV, since this result will be
- * suitable for any request that has need_phvs.
- */
- if (varattno > InvalidAttrNumber &&
- varattno <= list_length(rcon->targetlist))
- rcon->rv_cache[varattno] = copyObject(newnode);
+ /*
+ * Cache it if possible (ie, if the attno is in range, which
+ * it probably always should be).
+ */
+ if (varattno > InvalidAttrNumber &&
+ varattno <= list_length(rcon->targetlist))
+ rcon->rv_cache[varattno] = copyObject(newnode);
+ }
}
}
- /* Must adjust varlevelsup if tlist item is from higher query */
+ /* Must adjust varlevelsup if replaced Var is within a subquery */
if (var->varlevelsup > 0)
IncrementVarSublevelsUp(newnode, var->varlevelsup, 0);
+ /* Propagate any varnullingrels into the replacement Var or PHV */
+ if (var->varnullingrels != NULL)
+ {
+ if (IsA(newnode, Var))
+ {
+ Var *newvar = (Var *) newnode;
+
+ Assert(newvar->varlevelsup == var->varlevelsup);
+ newvar->varnullingrels = bms_add_members(newvar->varnullingrels,
+ var->varnullingrels);
+ }
+ else if (IsA(newnode, PlaceHolderVar))
+ {
+ PlaceHolderVar *newphv = (PlaceHolderVar *) newnode;
+
+ Assert(newphv->phlevelsup == var->varlevelsup);
+ newphv->phnullingrels = bms_add_members(newphv->phnullingrels,
+ var->varnullingrels);
+ }
+ else
+ elog(ERROR, "failed to wrap a non-Var");
+ }
+
return newnode;
}
@@ -2707,7 +2646,9 @@ flatten_simple_union_all(PlannerInfo *root)
void
reduce_outer_joins(PlannerInfo *root)
{
- reduce_outer_joins_state *state;
+ reduce_outer_joins_pass1_state *state1;
+ reduce_outer_joins_pass2_state state2;
+ ListCell *lc;
/*
* To avoid doing strictness checks on more quals than necessary, we want
@@ -2718,14 +2659,56 @@ reduce_outer_joins(PlannerInfo *root)
* join(s) below each side of each join clause. The second pass examines
* qual clauses and changes join types as it descends the tree.
*/
- state = reduce_outer_joins_pass1((Node *) root->parse->jointree);
+ state1 = reduce_outer_joins_pass1((Node *) root->parse->jointree);
/* planner.c shouldn't have called me if no outer joins */
- if (state == NULL || !state->contains_outer)
+ if (state1 == NULL || !state1->contains_outer)
elog(ERROR, "so where are the outer joins?");
+ state2.inner_reduced = NULL;
+ state2.partial_reduced = NIL;
+
reduce_outer_joins_pass2((Node *) root->parse->jointree,
- state, root, NULL, NIL);
+ state1, &state2,
+ root, NULL, NIL);
+
+ /*
+ * If we successfully reduced the strength of any outer joins, we must
+ * remove references to those joins as nulling rels. This is handled as
+ * an additional pass, for simplicity and because we can handle all
+ * fully-reduced joins in a single pass over the parse tree.
+ */
+ if (!bms_is_empty(state2.inner_reduced))
+ {
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ state2.inner_reduced,
+ NULL);
+ /* There could be references in the append_rel_list, too */
+ root->append_rel_list = (List *)
+ remove_nulling_relids((Node *) root->append_rel_list,
+ state2.inner_reduced,
+ NULL);
+ }
+
+ /*
+ * Partially-reduced full joins have to be done one at a time, since
+ * they'll each need a different setting of except_relids.
+ */
+ foreach(lc, state2.partial_reduced)
+ {
+ reduce_outer_joins_partial_state *statep = lfirst(lc);
+ Relids full_join_relids = bms_make_singleton(statep->full_join_rti);
+
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ full_join_relids,
+ statep->unreduced_side);
+ root->append_rel_list = (List *)
+ remove_nulling_relids((Node *) root->append_rel_list,
+ full_join_relids,
+ statep->unreduced_side);
+ }
}
/*
@@ -2733,13 +2716,13 @@ reduce_outer_joins(PlannerInfo *root)
*
* Returns a state node describing the given jointree node.
*/
-static reduce_outer_joins_state *
+static reduce_outer_joins_pass1_state *
reduce_outer_joins_pass1(Node *jtnode)
{
- reduce_outer_joins_state *result;
+ reduce_outer_joins_pass1_state *result;
- result = (reduce_outer_joins_state *)
- palloc(sizeof(reduce_outer_joins_state));
+ result = (reduce_outer_joins_pass1_state *)
+ palloc(sizeof(reduce_outer_joins_pass1_state));
result->relids = NULL;
result->contains_outer = false;
result->sub_states = NIL;
@@ -2759,7 +2742,7 @@ reduce_outer_joins_pass1(Node *jtnode)
foreach(l, f->fromlist)
{
- reduce_outer_joins_state *sub_state;
+ reduce_outer_joins_pass1_state *sub_state;
sub_state = reduce_outer_joins_pass1(lfirst(l));
result->relids = bms_add_members(result->relids,
@@ -2771,7 +2754,7 @@ reduce_outer_joins_pass1(Node *jtnode)
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- reduce_outer_joins_state *sub_state;
+ reduce_outer_joins_pass1_state *sub_state;
/* join's own RT index is not wanted in result->relids */
if (IS_OUTER_JOIN(j->jointype))
@@ -2799,14 +2782,22 @@ reduce_outer_joins_pass1(Node *jtnode)
* reduce_outer_joins_pass2 - phase 2 processing
*
* jtnode: current jointree node
- * state: state data collected by phase 1 for this node
+ * state1: state data collected by phase 1 for this node
+ * state2: where to accumulate info about successfully-reduced joins
* root: toplevel planner state
* nonnullable_rels: set of base relids forced non-null by upper quals
* forced_null_vars: multibitmapset of Vars forced null by upper quals
+ *
+ * Returns info in state2 about outer joins that were successfully simplified.
+ * Joins that were fully reduced to inner joins are all added to
+ * state2->inner_reduced. If a full join is reduced to a left join,
+ * it needs its own entry in state2->partial_reduced, since that will
+ * require custom processing to remove only the correct nullingrel markers.
*/
static void
reduce_outer_joins_pass2(Node *jtnode,
- reduce_outer_joins_state *state,
+ reduce_outer_joins_pass1_state *state1,
+ reduce_outer_joins_pass2_state *state2,
PlannerInfo *root,
Relids nonnullable_rels,
List *forced_null_vars)
@@ -2835,13 +2826,14 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_forced_null_vars = mbms_add_members(pass_forced_null_vars,
forced_null_vars);
/* And recurse --- but only into interesting subtrees */
- Assert(list_length(f->fromlist) == list_length(state->sub_states));
- forboth(l, f->fromlist, s, state->sub_states)
+ Assert(list_length(f->fromlist) == list_length(state1->sub_states));
+ forboth(l, f->fromlist, s, state1->sub_states)
{
- reduce_outer_joins_state *sub_state = lfirst(s);
+ reduce_outer_joins_pass1_state *sub_state = lfirst(s);
if (sub_state->contains_outer)
- reduce_outer_joins_pass2(lfirst(l), sub_state, root,
+ reduce_outer_joins_pass2(lfirst(l), sub_state,
+ state2, root,
pass_nonnullable_rels,
pass_forced_null_vars);
}
@@ -2853,8 +2845,8 @@ reduce_outer_joins_pass2(Node *jtnode,
JoinExpr *j = (JoinExpr *) jtnode;
int rtindex = j->rtindex;
JoinType jointype = j->jointype;
- reduce_outer_joins_state *left_state = linitial(state->sub_states);
- reduce_outer_joins_state *right_state = lsecond(state->sub_states);
+ reduce_outer_joins_pass1_state *left_state = linitial(state1->sub_states);
+ reduce_outer_joins_pass1_state *right_state = lsecond(state1->sub_states);
/* Can we simplify this join? */
switch (jointype)
@@ -2875,12 +2867,22 @@ reduce_outer_joins_pass2(Node *jtnode,
if (bms_overlap(nonnullable_rels, right_state->relids))
jointype = JOIN_INNER;
else
+ {
jointype = JOIN_LEFT;
+ /* Also report partial reduction in state2 */
+ report_reduced_full_join(state2, rtindex,
+ right_state->relids);
+ }
}
else
{
if (bms_overlap(nonnullable_rels, right_state->relids))
+ {
jointype = JOIN_RIGHT;
+ /* Also report partial reduction in state2 */
+ report_reduced_full_join(state2, rtindex,
+ left_state->relids);
+ }
}
break;
case JOIN_SEMI:
@@ -2913,8 +2915,8 @@ reduce_outer_joins_pass2(Node *jtnode,
j->larg = j->rarg;
j->rarg = tmparg;
jointype = JOIN_LEFT;
- right_state = linitial(state->sub_states);
- left_state = lsecond(state->sub_states);
+ right_state = linitial(state1->sub_states);
+ left_state = lsecond(state1->sub_states);
}
/*
@@ -2945,7 +2947,10 @@ reduce_outer_joins_pass2(Node *jtnode,
jointype = JOIN_ANTI;
}
- /* Apply the jointype change, if any, to both jointree node and RTE */
+ /*
+ * Apply the jointype change, if any, to both jointree node and RTE.
+ * Also, if we changed an RTE to INNER, add its RTI to inner_reduced.
+ */
if (rtindex && jointype != j->jointype)
{
RangeTblEntry *rte = rt_fetch(rtindex, root->parse->rtable);
@@ -2953,6 +2958,9 @@ reduce_outer_joins_pass2(Node *jtnode,
Assert(rte->rtekind == RTE_JOIN);
Assert(rte->jointype == j->jointype);
rte->jointype = jointype;
+ if (jointype == JOIN_INNER)
+ state2->inner_reduced = bms_add_member(state2->inner_reduced,
+ rtindex);
}
j->jointype = jointype;
@@ -3025,7 +3033,8 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_nonnullable_rels = NULL;
pass_forced_null_vars = NIL;
}
- reduce_outer_joins_pass2(j->larg, left_state, root,
+ reduce_outer_joins_pass2(j->larg, left_state,
+ state2, root,
pass_nonnullable_rels,
pass_forced_null_vars);
}
@@ -3044,7 +3053,8 @@ reduce_outer_joins_pass2(Node *jtnode,
pass_nonnullable_rels = NULL;
pass_forced_null_vars = NIL;
}
- reduce_outer_joins_pass2(j->rarg, right_state, root,
+ reduce_outer_joins_pass2(j->rarg, right_state,
+ state2, root,
pass_nonnullable_rels,
pass_forced_null_vars);
}
@@ -3056,6 +3066,19 @@ reduce_outer_joins_pass2(Node *jtnode,
(int) nodeTag(jtnode));
}
+/* Helper for reduce_outer_joins_pass2 */
+static void
+report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
+ int rtindex, Relids relids)
+{
+ reduce_outer_joins_partial_state *statep;
+
+ statep = palloc(sizeof(reduce_outer_joins_partial_state));
+ statep->full_join_rti = rtindex;
+ statep->unreduced_side = relids;
+ state2->partial_reduced = lappend(state2->partial_reduced, statep);
+}
+
/*
* remove_useless_result_rtes
@@ -3097,16 +3120,41 @@ reduce_outer_joins_pass2(Node *jtnode,
void
remove_useless_result_rtes(PlannerInfo *root)
{
+ Relids dropped_outer_joins = NULL;
ListCell *cell;
/* Top level of jointree must always be a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
/* Recurse ... */
root->parse->jointree = (FromExpr *)
- remove_useless_results_recurse(root, (Node *) root->parse->jointree);
+ remove_useless_results_recurse(root,
+ (Node *) root->parse->jointree,
+ &dropped_outer_joins);
/* We should still have a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
+ /*
+ * If we removed any outer-join nodes from the jointree, run around and
+ * remove references to those joins as nulling rels. (There could be such
+ * references in PHVs that we pulled up out of the original subquery that
+ * the RESULT rel replaced. This is kosher on the grounds that we now
+ * know that such an outer join wouldn't really have nulled anything.) We
+ * don't do this during the main recursion, for simplicity and because we
+ * can handle all such joins in a single pass over the parse tree.
+ */
+ if (!bms_is_empty(dropped_outer_joins))
+ {
+ root->parse = (Query *)
+ remove_nulling_relids((Node *) root->parse,
+ dropped_outer_joins,
+ NULL);
+ /* There could be references in the append_rel_list, too */
+ root->append_rel_list = (List *)
+ remove_nulling_relids((Node *) root->append_rel_list,
+ dropped_outer_joins,
+ NULL);
+ }
+
/*
* Remove any PlanRowMark referencing an RTE_RESULT RTE. We obviously
* must do that for any RTE_RESULT that we just removed. But one for a
@@ -3132,9 +3180,12 @@ remove_useless_result_rtes(PlannerInfo *root)
* Recursive guts of remove_useless_result_rtes.
*
* This recursively processes the jointree and returns a modified jointree.
+ * In addition, the RT indexes of any removed outer-join nodes are added to
+ * *dropped_outer_joins.
*/
static Node *
-remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
+remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Relids *dropped_outer_joins)
{
Assert(jtnode != NULL);
if (IsA(jtnode, RangeTblRef))
@@ -3162,7 +3213,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
int varno;
/* Recursively transform child ... */
- child = remove_useless_results_recurse(root, child);
+ child = remove_useless_results_recurse(root, child,
+ dropped_outer_joins);
/* ... and stick it back into the tree */
lfirst(cell) = child;
@@ -3211,8 +3263,10 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
int varno;
/* First, recurse */
- j->larg = remove_useless_results_recurse(root, j->larg);
- j->rarg = remove_useless_results_recurse(root, j->rarg);
+ j->larg = remove_useless_results_recurse(root, j->larg,
+ dropped_outer_joins);
+ j->rarg = remove_useless_results_recurse(root, j->rarg,
+ dropped_outer_joins);
/* Apply join-type-specific optimization rules */
switch (j->jointype)
@@ -3280,6 +3334,8 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
!find_dependent_phvs(root, varno)))
{
remove_result_refs(root, varno, j->larg);
+ *dropped_outer_joins = bms_add_member(*dropped_outer_joins,
+ j->rtindex);
jtnode = j->larg;
}
break;
@@ -3299,9 +3355,13 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode)
* it'd be OK to just remove the PHV wrapping. We don't have
* infrastructure for that, but remove_result_refs() will
* relabel them as to be evaluated at the LHS, which is fine.
+ *
+ * Also, we don't need to worry about removing traces of the
+ * join's rtindex, since it hasn't got one.
*/
if ((varno = get_result_relid(root, j->rarg)) != 0)
{
+ Assert(j->rtindex == 0);
remove_result_refs(root, varno, j->larg);
if (j->quals)
jtnode = (Node *)
@@ -3371,7 +3431,7 @@ remove_result_refs(PlannerInfo *root, int varno, Node *newjtloc)
{
Relids subrelids;
- subrelids = get_relids_in_jointree(newjtloc, false);
+ subrelids = get_relids_in_jointree(newjtloc, true, false);
Assert(!bms_is_empty(subrelids));
substitute_phv_relids((Node *) root->parse, varno, subrelids);
fix_append_rel_relids(root, varno, subrelids);
@@ -3428,9 +3488,8 @@ find_dependent_phvs_walker(Node *node,
context->sublevels_up--;
return result;
}
- /* Shouldn't need to handle planner auxiliary nodes here */
+ /* Shouldn't need to handle most planner auxiliary nodes here */
Assert(!IsA(node, SpecialJoinInfo));
- Assert(!IsA(node, AppendRelInfo));
Assert(!IsA(node, PlaceHolderInfo));
Assert(!IsA(node, MinMaxAggInfo));
@@ -3450,10 +3509,17 @@ find_dependent_phvs(PlannerInfo *root, int varno)
context.relids = bms_make_singleton(varno);
context.sublevels_up = 0;
- return query_tree_walker(root->parse,
- find_dependent_phvs_walker,
- (void *) &context,
- 0);
+ if (query_tree_walker(root->parse,
+ find_dependent_phvs_walker,
+ (void *) &context,
+ 0))
+ return true;
+ /* The append_rel_list could be populated already, so check it too */
+ if (expression_tree_walker((Node *) root->append_rel_list,
+ find_dependent_phvs_walker,
+ (void *) &context))
+ return true;
+ return false;
}
static bool
@@ -3483,7 +3549,7 @@ find_dependent_phvs_in_jointree(PlannerInfo *root, Node *node, int varno)
* are not marked LATERAL, though, since they couldn't possibly contain
* any cross-references to other RTEs.
*/
- subrelids = get_relids_in_jointree(node, false);
+ subrelids = get_relids_in_jointree(node, false, false);
relid = -1;
while ((relid = bms_next_member(subrelids, relid)) >= 0)
{
@@ -3628,11 +3694,17 @@ fix_append_rel_relids(PlannerInfo *root, int varno, Relids subrelids)
/*
* get_relids_in_jointree: get set of RT indexes present in a jointree
*
- * If include_joins is true, join RT indexes are included; if false,
- * only base rels are included.
+ * Base-relation relids are always included in the result.
+ * If include_outer_joins is true, outer-join RT indexes are included.
+ * If include_inner_joins is true, inner-join RT indexes are included.
+ *
+ * Note that for most purposes in the planner, outer joins are included
+ * in standard relid sets. Setting include_inner_joins true is only
+ * appropriate for special purposes during subquery flattening.
*/
Relids
-get_relids_in_jointree(Node *jtnode, bool include_joins)
+get_relids_in_jointree(Node *jtnode, bool include_outer_joins,
+ bool include_inner_joins)
{
Relids result = NULL;
@@ -3653,18 +3725,34 @@ get_relids_in_jointree(Node *jtnode, bool include_joins)
{
result = bms_join(result,
get_relids_in_jointree(lfirst(l),
- include_joins));
+ include_outer_joins,
+ include_inner_joins));
}
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- result = get_relids_in_jointree(j->larg, include_joins);
+ result = get_relids_in_jointree(j->larg,
+ include_outer_joins,
+ include_inner_joins);
result = bms_join(result,
- get_relids_in_jointree(j->rarg, include_joins));
- if (include_joins && j->rtindex)
- result = bms_add_member(result, j->rtindex);
+ get_relids_in_jointree(j->rarg,
+ include_outer_joins,
+ include_inner_joins));
+ if (j->rtindex)
+ {
+ if (j->jointype == JOIN_INNER)
+ {
+ if (include_inner_joins)
+ result = bms_add_member(result, j->rtindex);
+ }
+ else
+ {
+ if (include_outer_joins)
+ result = bms_add_member(result, j->rtindex);
+ }
+ }
}
else
elog(ERROR, "unrecognized node type: %d",
@@ -3673,7 +3761,7 @@ get_relids_in_jointree(Node *jtnode, bool include_joins)
}
/*
- * get_relids_for_join: get set of base RT indexes making up a join
+ * get_relids_for_join: get set of base+OJ RT indexes making up a join
*/
Relids
get_relids_for_join(Query *query, int joinrelid)
@@ -3684,7 +3772,7 @@ get_relids_for_join(Query *query, int joinrelid)
joinrelid);
if (!jtnode)
elog(ERROR, "could not find join node %d", joinrelid);
- return get_relids_in_jointree(jtnode, false);
+ return get_relids_in_jointree(jtnode, true, false);
}
/*
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index cd45ab4899..61378c8d58 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -228,6 +228,14 @@ adjust_appendrel_attrs_mutator(Node *node,
if (var->varlevelsup != 0)
return (Node *) var; /* no changes needed */
+ /*
+ * You might think we need to adjust var->varnullingrels, but that
+ * shouldn't need any changes. It will contain outer-join relids,
+ * while the transformation we are making affects only baserels.
+ * Below, we just propagate var->varnullingrels into the translated
+ * Var. (XXX what to do if translation is not a Var??)
+ */
+
for (cnt = 0; cnt < nappinfos; cnt++)
{
if (var->varno == appinfos[cnt]->parent_relid)
@@ -255,6 +263,8 @@ adjust_appendrel_attrs_mutator(Node *node,
if (newnode == NULL)
elog(ERROR, "attribute %d of relation \"%s\" does not exist",
var->varattno, get_rel_name(appinfo->parent_reloid));
+ if (IsA(newnode, Var))
+ ((Var *) newnode)->varnullingrels = var->varnullingrels;
return newnode;
}
else if (var->varattno == 0)
@@ -348,6 +358,8 @@ adjust_appendrel_attrs_mutator(Node *node,
var = copyObject(ridinfo->rowidvar);
/* ... but use the correct relid */
var->varno = leaf_relid;
+ /* identity vars shouldn't have nulling rels */
+ Assert(var->varnullingrels == NULL);
/* varnosyn in the RowIdentityVarInfo is probably wrong */
var->varnosyn = 0;
var->varattnosyn = 0;
@@ -392,8 +404,11 @@ adjust_appendrel_attrs_mutator(Node *node,
(void *) context);
/* now fix PlaceHolderVar's relid sets */
if (phv->phlevelsup == 0)
- phv->phrels = adjust_child_relids(phv->phrels, context->nappinfos,
- context->appinfos);
+ {
+ phv->phrels = adjust_child_relids(phv->phrels,
+ nappinfos, appinfos);
+ /* as above, we needn't touch phnullingrels */
+ }
return (Node *) phv;
}
/* Shouldn't need to handle planner auxiliary nodes here */
@@ -688,7 +703,11 @@ get_translated_update_targetlist(PlannerInfo *root, Index relid,
/*
* find_appinfos_by_relids
- * Find AppendRelInfo structures for all relations specified by relids.
+ * Find AppendRelInfo structures for base relations listed in relids.
+ *
+ * The relids argument is typically a join relation's relids, which can
+ * include outer-join RT indexes in addition to baserels. We silently
+ * ignore the outer joins.
*
* The AppendRelInfos are returned in an array, which can be pfree'd by the
* caller. *nappinfos is set to the number of entries in the array.
@@ -700,8 +719,9 @@ find_appinfos_by_relids(PlannerInfo *root, Relids relids, int *nappinfos)
int cnt = 0;
int i;
- *nappinfos = bms_num_members(relids);
- appinfos = (AppendRelInfo **) palloc(sizeof(AppendRelInfo *) * *nappinfos);
+ /* Allocate an array that's certainly big enough */
+ appinfos = (AppendRelInfo **)
+ palloc(sizeof(AppendRelInfo *) * bms_num_members(relids));
i = -1;
while ((i = bms_next_member(relids, i)) >= 0)
@@ -709,10 +729,17 @@ find_appinfos_by_relids(PlannerInfo *root, Relids relids, int *nappinfos)
AppendRelInfo *appinfo = root->append_rel_array[i];
if (!appinfo)
+ {
+ /* Probably i is an OJ index, but let's check */
+ if (find_base_rel_ignore_join(root, i) == NULL)
+ continue;
+ /* It's a base rel, but we lack an append_rel_array entry */
elog(ERROR, "child rel %d not found in append_rel_array", i);
+ }
appinfos[cnt++] = appinfo;
}
+ *nappinfos = cnt;
return appinfos;
}
@@ -754,6 +781,7 @@ add_row_identity_var(PlannerInfo *root, Var *orig_var,
Assert(IsA(orig_var, Var));
Assert(orig_var->varno == rtindex);
Assert(orig_var->varlevelsup == 0);
+ Assert(orig_var->varnullingrels == NULL);
/*
* If we're doing non-inherited UPDATE/DELETE/MERGE, there's little need
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index aa584848cf..76e25118f9 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -2004,14 +2004,16 @@ is_pseudo_constant_clause_relids(Node *clause, Relids relids)
* NumRelids
* (formerly clause_relids)
*
- * Returns the number of different relations referenced in 'clause'.
+ * Returns the number of different base relations referenced in 'clause'.
*/
int
NumRelids(PlannerInfo *root, Node *clause)
{
+ int result;
Relids varnos = pull_varnos(root, clause);
- int result = bms_num_members(varnos);
+ varnos = bms_del_members(varnos, root->outer_join_rels);
+ result = bms_num_members(varnos);
bms_free(varnos);
return result;
}
diff --git a/src/backend/optimizer/util/joininfo.c b/src/backend/optimizer/util/joininfo.c
index 197f20faec..968a5a488e 100644
--- a/src/backend/optimizer/util/joininfo.c
+++ b/src/backend/optimizer/util/joininfo.c
@@ -88,8 +88,8 @@ have_relevant_joinclause(PlannerInfo *root,
* not depend on context).
*
* 'restrictinfo' describes the join clause
- * 'join_relids' is the list of relations participating in the join clause
- * (there must be more than one)
+ * 'join_relids' is the set of relations participating in the join clause
+ * (some of these could be outer joins)
*/
void
add_join_clause_to_rels(PlannerInfo *root,
@@ -101,8 +101,11 @@ add_join_clause_to_rels(PlannerInfo *root,
cur_relid = -1;
while ((cur_relid = bms_next_member(join_relids, cur_relid)) >= 0)
{
- RelOptInfo *rel = find_base_rel(root, cur_relid);
+ RelOptInfo *rel = find_base_rel_ignore_join(root, cur_relid);
+ /* We only need to add the clause to baserels */
+ if (rel == NULL)
+ continue;
rel->joininfo = lappend(rel->joininfo, restrictinfo);
}
}
@@ -115,8 +118,8 @@ add_join_clause_to_rels(PlannerInfo *root,
* discover that a relation need not be joined at all.
*
* 'restrictinfo' describes the join clause
- * 'join_relids' is the list of relations participating in the join clause
- * (there must be more than one)
+ * 'join_relids' is the set of relations participating in the join clause
+ * (some of these could be outer joins)
*/
void
remove_join_clause_from_rels(PlannerInfo *root,
@@ -128,7 +131,11 @@ remove_join_clause_from_rels(PlannerInfo *root,
cur_relid = -1;
while ((cur_relid = bms_next_member(join_relids, cur_relid)) >= 0)
{
- RelOptInfo *rel = find_base_rel(root, cur_relid);
+ RelOptInfo *rel = find_base_rel_ignore_join(root, cur_relid);
+
+ /* We would only have added the clause to baserels */
+ if (rel == NULL)
+ continue;
/*
* Remove the restrictinfo from the list. Pointer comparison is
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index 493aaa7aed..abc994dbf2 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -338,6 +338,10 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
sjinfo.syn_lefthand = sjinfo.min_lefthand;
sjinfo.syn_righthand = sjinfo.min_righthand;
sjinfo.jointype = JOIN_INNER;
+ sjinfo.ojrelid = 0;
+ sjinfo.commute_above_l = NULL;
+ sjinfo.commute_above_r = NULL;
+ sjinfo.commute_below = NULL;
/* we don't bother trying to make the remaining fields valid */
sjinfo.lhs_strict = false;
sjinfo.delay_upper_joins = false;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 4478036bb6..6d76a11c08 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -1307,7 +1307,7 @@ create_append_path(PlannerInfo *root,
* Apply query-wide LIMIT if known and path is for sole base relation.
* (Handling this at this low level is a bit klugy.)
*/
- if (root != NULL && bms_equal(rel->relids, root->all_baserels))
+ if (root != NULL && bms_equal(rel->relids, root->all_query_rels))
pathnode->limit_tuples = root->limit_tuples;
else
pathnode->limit_tuples = -1.0;
@@ -1436,7 +1436,7 @@ create_merge_append_path(PlannerInfo *root,
* Apply query-wide LIMIT if known and path is for sole base relation.
* (Handling this at this low level is a bit klugy.)
*/
- if (bms_equal(rel->relids, root->all_baserels))
+ if (bms_equal(rel->relids, root->all_query_rels))
pathnode->limit_tuples = root->limit_tuples;
else
pathnode->limit_tuples = -1.0;
diff --git a/src/backend/optimizer/util/placeholder.c b/src/backend/optimizer/util/placeholder.c
index 72b9977022..af10dbd124 100644
--- a/src/backend/optimizer/util/placeholder.c
+++ b/src/backend/optimizer/util/placeholder.c
@@ -23,17 +23,32 @@
#include "optimizer/planmain.h"
#include "utils/lsyscache.h"
+
+typedef struct contain_placeholder_references_context
+{
+ int relid;
+ int sublevels_up;
+} contain_placeholder_references_context;
+
/* Local functions */
static void find_placeholders_recurse(PlannerInfo *root, Node *jtnode);
static void find_placeholders_in_expr(PlannerInfo *root, Node *expr);
+static bool contain_placeholder_references_walker(Node *node,
+ contain_placeholder_references_context *context);
/*
* make_placeholder_expr
* Make a PlaceHolderVar for the given expression.
*
- * phrels is the syntactic location (as a set of baserels) to attribute
+ * phrels is the syntactic location (as a set of relids) to attribute
* to the expression.
+ *
+ * The caller is responsible for adjusting phlevelsup and phnullingrels
+ * as needed. Because we do not know here which query level the PHV
+ * will be associated with, it's important that this function touches
+ * only root->glob; messing with other parts of PlannerInfo would be
+ * likely to do the wrong thing.
*/
PlaceHolderVar *
make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
@@ -42,8 +57,9 @@ make_placeholder_expr(PlannerInfo *root, Expr *expr, Relids phrels)
phv->phexpr = expr;
phv->phrels = phrels;
+ phv->phnullingrels = NULL; /* caller may change this later */
phv->phid = ++(root->glob->lastPHId);
- phv->phlevelsup = 0;
+ phv->phlevelsup = 0; /* caller may change this later */
return phv;
}
@@ -92,6 +108,15 @@ find_placeholder_info(PlannerInfo *root, PlaceHolderVar *phv)
phinfo->phid = phv->phid;
phinfo->ph_var = copyObject(phv);
+ /*
+ * By convention, phinfo->ph_var->phnullingrels is always empty, since the
+ * PlaceHolderInfo represents the initially-calculated state of the
+ * PlaceHolderVar. PlaceHolderVars appearing in the query tree might have
+ * varying values of phnullingrels, reflecting outer joins applied above
+ * the calculation level.
+ */
+ phinfo->ph_var->phnullingrels = NULL;
+
/*
* Any referenced rels that are outside the PHV's syntactic scope are
* LATERAL references, which should be included in ph_lateral but not in
@@ -339,6 +364,8 @@ update_placeholder_eval_levels(PlannerInfo *root, SpecialJoinInfo *new_sjinfo)
sjinfo->min_lefthand);
eval_at = bms_add_members(eval_at,
sjinfo->min_righthand);
+ if (sjinfo->ojrelid)
+ eval_at = bms_add_member(eval_at, sjinfo->ojrelid);
/* we'll need another iteration */
found_some = true;
}
@@ -413,6 +440,14 @@ add_placeholders_to_base_rels(PlannerInfo *root)
{
RelOptInfo *rel = find_base_rel(root, varno);
+ /*
+ * As in add_vars_to_targetlist(), a value computed at scan level
+ * has not yet been nulled by any outer join, so its phnullingrels
+ * should be empty.
+ */
+ Assert(phinfo->ph_var->phnullingrels == NULL);
+
+ /* Copying the PHV might be unnecessary here, but be safe */
rel->reltarget->exprs = lappend(rel->reltarget->exprs,
copyObject(phinfo->ph_var));
/* reltarget's cost and width fields will be updated later */
@@ -435,7 +470,8 @@ add_placeholders_to_base_rels(PlannerInfo *root)
*/
void
add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *outer_rel, RelOptInfo *inner_rel)
+ RelOptInfo *outer_rel, RelOptInfo *inner_rel,
+ SpecialJoinInfo *sjinfo)
{
Relids relids = joinrel->relids;
ListCell *lc;
@@ -466,9 +502,17 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
if (!bms_is_subset(phinfo->ph_eval_at, outer_rel->relids) &&
!bms_is_subset(phinfo->ph_eval_at, inner_rel->relids))
{
- PlaceHolderVar *phv = phinfo->ph_var;
+ /* Copying might be unnecessary here, but be safe */
+ PlaceHolderVar *phv = copyObject(phinfo->ph_var);
QualCost cost;
+ /*
+ * It'll start out not nulled by anything. Joins above
+ * this one might add to its phnullingrels later, in much
+ * the same way as for Vars.
+ */
+ Assert(phv->phnullingrels == NULL);
+
joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
phv);
cost_qual_eval_node(&cost, (Node *) phv->phexpr, root);
@@ -499,3 +543,74 @@ add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
}
}
}
+
+/*
+ * contain_placeholder_references_to
+ * Detect whether any PlaceHolderVars in the given clause contain
+ * references to the given relid (typically an OJ relid).
+ *
+ * "Contain" means that there's a use of the relid inside the PHV's
+ * contained expression, so that changing the nullability status of
+ * the rel might change what the PHV computes.
+ *
+ * The code here to cope with upper-level PHVs is likely dead, but keep it
+ * anyway just in case.
+ */
+bool
+contain_placeholder_references_to(PlannerInfo *root, Node *clause,
+ int relid)
+{
+ contain_placeholder_references_context context;
+
+ /* We can answer quickly in the common case that there's no PHVs at all */
+ if (root->glob->lastPHId == 0)
+ return false;
+ /* Else run the recursive search */
+ context.relid = relid;
+ context.sublevels_up = 0;
+ return contain_placeholder_references_walker(clause, &context);
+}
+
+static bool
+contain_placeholder_references_walker(Node *node,
+ contain_placeholder_references_context *context)
+{
+ if (node == NULL)
+ return false;
+ if (IsA(node, PlaceHolderVar))
+ {
+ PlaceHolderVar *phv = (PlaceHolderVar *) node;
+
+ /* We should just look through PHVs of other query levels */
+ if (phv->phlevelsup == context->sublevels_up)
+ {
+ /* If phrels matches, we found what we came for */
+ if (bms_is_member(context->relid, phv->phrels))
+ return true;
+
+ /*
+ * We should not examine phnullingrels: what we are looking for is
+ * references in the contained expression, not OJs that might null
+ * the result afterwards. Also, we don't need to recurse into the
+ * contained expression, because phrels should adequately
+ * summarize what's in there. So we're done here.
+ */
+ return false;
+ }
+ }
+ else if (IsA(node, Query))
+ {
+ /* Recurse into RTE subquery or not-yet-planned sublink subquery */
+ bool result;
+
+ context->sublevels_up++;
+ result = query_tree_walker((Query *) node,
+ contain_placeholder_references_walker,
+ context,
+ 0);
+ context->sublevels_up--;
+ return result;
+ }
+ return expression_tree_walker(node, contain_placeholder_references_walker,
+ context);
+}
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 0a5632699d..34de0029bd 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -28,6 +28,7 @@
#include "optimizer/plancat.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+#include "rewrite/rewriteManip.h"
#include "parser/parse_relation.h"
#include "utils/hsearch.h"
#include "utils/lsyscache.h"
@@ -40,7 +41,9 @@ typedef struct JoinHashEntry
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *input_rel);
+ RelOptInfo *input_rel,
+ SpecialJoinInfo *sjinfo,
+ bool can_null);
static List *build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outer_rel,
@@ -48,8 +51,10 @@ static List *build_joinrel_restrictlist(PlannerInfo *root,
static void build_joinrel_joinlist(RelOptInfo *joinrel,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel);
-static List *subbuild_joinrel_restrictlist(RelOptInfo *joinrel,
- List *joininfo_list,
+static List *subbuild_joinrel_restrictlist(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *input_rel,
+ Relids both_input_relids,
List *new_restrictlist);
static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
List *joininfo_list,
@@ -57,10 +62,12 @@ static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
static void set_foreign_rel_properties(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel);
static void add_join_rel(PlannerInfo *root, RelOptInfo *joinrel);
-static void build_joinrel_partition_info(RelOptInfo *joinrel,
+static void build_joinrel_partition_info(PlannerInfo *root,
+ RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
- List *restrictlist, JoinType jointype);
-static bool have_partkey_equi_join(RelOptInfo *joinrel,
+ SpecialJoinInfo *sjinfo,
+ List *restrictlist);
+static bool have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *rel1, RelOptInfo *rel2,
JoinType jointype, List *restrictlist);
static int match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel,
@@ -373,7 +380,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
/*
* find_base_rel
- * Find a base or other relation entry, which must already exist.
+ * Find a base or otherrel relation entry, which must already exist.
*/
RelOptInfo *
find_base_rel(PlannerInfo *root, int relid)
@@ -394,6 +401,44 @@ find_base_rel(PlannerInfo *root, int relid)
return NULL; /* keep compiler quiet */
}
+/*
+ * find_base_rel_ignore_join
+ * Find a base or otherrel relation entry, which must already exist.
+ *
+ * Unlike find_base_rel, if relid references an outer join then this
+ * will return NULL rather than raising an error. This is convenient
+ * for callers that must deal with relid sets including both base and
+ * outer joins.
+ */
+RelOptInfo *
+find_base_rel_ignore_join(PlannerInfo *root, int relid)
+{
+ Assert(relid > 0);
+
+ if (relid < root->simple_rel_array_size)
+ {
+ RelOptInfo *rel;
+ RangeTblEntry *rte;
+
+ rel = root->simple_rel_array[relid];
+ if (rel)
+ return rel;
+
+ /*
+ * We could just return NULL here, but for debugging purposes it seems
+ * best to actually verify that the relid is an outer join and not
+ * something weird.
+ */
+ rte = root->simple_rte_array[relid];
+ if (rte && rte->rtekind == RTE_JOIN && rte->jointype != JOIN_INNER)
+ return NULL;
+ }
+
+ elog(ERROR, "no relation entry for relid %d", relid);
+
+ return NULL; /* keep compiler quiet */
+}
+
/*
* build_join_rel_hash
* Construct the auxiliary hash table for join relations.
@@ -692,9 +737,11 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ build_joinrel_tlist(root, joinrel, outer_rel, sjinfo,
+ (sjinfo->jointype == JOIN_FULL));
+ build_joinrel_tlist(root, joinrel, inner_rel, sjinfo,
+ (sjinfo->jointype != JOIN_INNER));
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel, sjinfo);
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -726,8 +773,8 @@ build_join_rel(PlannerInfo *root,
joinrel->has_eclass_joins = has_relevant_eclass_joinclause(root, joinrel);
/* Store the partition information. */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- sjinfo->jointype);
+ build_joinrel_partition_info(root, joinrel, outer_rel, inner_rel, sjinfo,
+ restrictlist);
/*
* Set estimates of the joinrel's size.
@@ -783,16 +830,14 @@ build_join_rel(PlannerInfo *root,
* 'parent_joinrel' is the RelOptInfo representing the join between parent
* relations. Some of the members of new RelOptInfo are produced by
* translating corresponding members of this RelOptInfo
- * 'sjinfo': child-join context info
* 'restrictlist': list of RestrictInfo nodes that apply to this particular
* pair of joinable relations
- * 'jointype' is the join type (inner, left, full, etc)
+ * 'sjinfo': child join's join-type details
*/
RelOptInfo *
build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
RelOptInfo *inner_rel, RelOptInfo *parent_joinrel,
- List *restrictlist, SpecialJoinInfo *sjinfo,
- JoinType jointype)
+ List *restrictlist, SpecialJoinInfo *sjinfo)
{
RelOptInfo *joinrel = makeNode(RelOptInfo);
AppendRelInfo **appinfos;
@@ -806,6 +851,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->reloptkind = RELOPT_OTHER_JOINREL;
joinrel->relids = bms_union(outer_rel->relids, inner_rel->relids);
+ if (sjinfo->ojrelid != 0)
+ joinrel->relids = bms_add_member(joinrel->relids, sjinfo->ojrelid);
joinrel->rows = 0;
/* cheap startup cost is interesting iff not all tuples to be retrieved */
joinrel->consider_startup = (root->tuple_fraction > 0);
@@ -892,8 +939,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->has_eclass_joins = parent_joinrel->has_eclass_joins;
/* Is the join between partitions itself partitioned? */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- jointype);
+ build_joinrel_partition_info(root, joinrel, outer_rel, inner_rel, sjinfo,
+ restrictlist);
/* Child joinrel is parallel safe if parent is parallel safe. */
joinrel->consider_parallel = parent_joinrel->consider_parallel;
@@ -975,10 +1022,41 @@ min_join_parameterization(PlannerInfo *root,
*
* We also compute the expected width of the join's output, making use
* of data that was cached at the baserel level by set_rel_width().
+ *
+ * Pass can_null as true if the join is an outer join that can null Vars
+ * from this input relation. If so, we will (normally) add the join's relid
+ * to the nulling bitmaps of Vars and PHVs bubbled up from the input.
+ *
+ * When forming an outer join's target list, special handling is needed
+ * in case the outer join was commuted with another one per outer join
+ * identity 3 (see optimizer/README). We must take steps to ensure that
+ * the output Vars have the same nulling bitmaps that they would if the
+ * two joins had been done in syntactic order; else they won't match Vars
+ * appearing higher in the query tree. We need to do two things:
+ *
+ * First, sjinfo->commute_above_r is added to the nulling bitmaps of RHS Vars.
+ * This takes care of the case where we implement
+ * A leftjoin (B leftjoin C on (Pbc)) on (Pab)
+ * as
+ * (A leftjoin B on (Pab)) leftjoin C on (Pbc)
+ * The C columns emitted by the B/C join need to be shown as nulled by both
+ * the B/C and A/B joins, even though they've not traversed the A/B join.
+ * (If the joins haven't been commuted, we are adding the nullingrel bits
+ * prematurely; but that's okay because the C columns can't be referenced
+ * between here and the upper join.)
+ *
+ * Second, if a RHS Var has any of the relids in sjinfo->commute_above_l
+ * already set in its nulling bitmap, then we *don't* add sjinfo->ojrelid
+ * to its nulling bitmap (but we do still add commute_above_r). This takes
+ * care of the reverse transformation: if the original syntax was
+ * (A leftjoin B on (Pab)) leftjoin C on (Pbc)
+ * then the now-upper A/B join must not mark C columns as nulled by itself.
*/
static void
build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *input_rel)
+ RelOptInfo *input_rel,
+ SpecialJoinInfo *sjinfo,
+ bool can_null)
{
Relids relids = joinrel->relids;
ListCell *vars;
@@ -998,7 +1076,24 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
/* Is it still needed above this joinrel? */
if (bms_nonempty_difference(phinfo->ph_needed, relids))
{
- /* Yup, add it to the output */
+ /*
+ * Yup, add it to the output. If this join potentially nulls
+ * this input, we have to update the PHV's phnullingrels,
+ * which means making a copy.
+ */
+ if (can_null)
+ {
+ phv = copyObject(phv);
+ /* See comments above to understand this logic */
+ if (sjinfo->ojrelid != 0 &&
+ !bms_overlap(phv->phnullingrels, sjinfo->commute_above_l))
+ phv->phnullingrels = bms_add_member(phv->phnullingrels,
+ sjinfo->ojrelid);
+ if (sjinfo->commute_above_r)
+ phv->phnullingrels = bms_add_members(phv->phnullingrels,
+ sjinfo->commute_above_r);
+ }
+
joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
phv);
/* Bubbling up the precomputed result has cost zero */
@@ -1022,9 +1117,7 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
RowIdentityVarInfo *ridinfo = (RowIdentityVarInfo *)
list_nth(root->row_identity_vars, var->varattno - 1);
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- var);
- /* Vars have cost zero, so no need to adjust reltarget->cost */
+ /* Update reltarget width estimate from RowIdentityVarInfo */
joinrel->reltarget->width += ridinfo->rowidwidth;
}
else
@@ -1037,15 +1130,35 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
/* Is it still needed above this joinrel? */
ndx = var->varattno - baserel->min_attr;
- if (bms_nonempty_difference(baserel->attr_needed[ndx], relids))
- {
- /* Yup, add it to the output */
- joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
- var);
- /* Vars have cost zero, so no need to adjust reltarget->cost */
- joinrel->reltarget->width += baserel->attr_widths[ndx];
- }
+ if (!bms_nonempty_difference(baserel->attr_needed[ndx], relids))
+ continue; /* nope, skip it */
+
+ /* Update reltarget width estimate from baserel's attr_widths */
+ joinrel->reltarget->width += baserel->attr_widths[ndx];
+ }
+
+ /*
+ * Add the Var to the output. If this join potentially nulls this
+ * input, we have to update the Var's varnullingrels, which means
+ * making a copy.
+ */
+ if (can_null)
+ {
+ var = copyObject(var);
+ /* See comments above to understand this logic */
+ if (sjinfo->ojrelid != 0 &&
+ !bms_overlap(var->varnullingrels, sjinfo->commute_above_l))
+ var->varnullingrels = bms_add_member(var->varnullingrels,
+ sjinfo->ojrelid);
+ if (sjinfo->commute_above_r)
+ var->varnullingrels = bms_add_members(var->varnullingrels,
+ sjinfo->commute_above_r);
}
+
+ joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs,
+ var);
+
+ /* Vars have cost zero, so no need to adjust reltarget->cost */
}
}
@@ -1064,7 +1177,7 @@ build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
* is not handled in the sub-relations, so it depends on which
* sub-relations are considered.
*
- * If a join clause from an input relation refers to base rels still not
+ * If a join clause from an input relation refers to base+OJ rels still not
* present in the joinrel, then it is still a join clause for the joinrel;
* we put it into the joininfo list for the joinrel. Otherwise,
* the clause is now a restrict clause for the joined relation, and we
@@ -1098,14 +1211,19 @@ build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *inner_rel)
{
List *result;
+ Relids both_input_relids;
+
+ both_input_relids = bms_union(outer_rel->relids, inner_rel->relids);
/*
* Collect all the clauses that syntactically belong at this level,
* eliminating any duplicates (important since we will see many of the
* same clauses arriving from both input relations).
*/
- result = subbuild_joinrel_restrictlist(joinrel, outer_rel->joininfo, NIL);
- result = subbuild_joinrel_restrictlist(joinrel, inner_rel->joininfo, result);
+ result = subbuild_joinrel_restrictlist(root, joinrel, outer_rel,
+ both_input_relids, NIL);
+ result = subbuild_joinrel_restrictlist(root, joinrel, inner_rel,
+ both_input_relids, result);
/*
* Add on any clauses derived from EquivalenceClasses. These cannot be
@@ -1140,24 +1258,63 @@ build_joinrel_joinlist(RelOptInfo *joinrel,
}
static List *
-subbuild_joinrel_restrictlist(RelOptInfo *joinrel,
- List *joininfo_list,
+subbuild_joinrel_restrictlist(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *input_rel,
+ Relids both_input_relids,
List *new_restrictlist)
{
ListCell *l;
- foreach(l, joininfo_list)
+ foreach(l, input_rel->joininfo)
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(l);
if (bms_is_subset(rinfo->required_relids, joinrel->relids))
{
/*
- * This clause becomes a restriction clause for the joinrel, since
- * it refers to no outside rels. Add it to the list, being
- * careful to eliminate duplicates. (Since RestrictInfo nodes in
- * different joinlists will have been multiply-linked rather than
- * copied, pointer equality should be a sufficient test.)
+ * This clause should become a restriction clause for the joinrel,
+ * since it refers to no outside rels. However, if it's a clone
+ * clause then it might be too late to evaluate it, so we have to
+ * check. (If it is too late, just ignore the clause, taking it
+ * on faith that another clone was or will be selected.) Clone
+ * clauses should always be outer-join clauses, so we compare
+ * against both_input_relids.
+ */
+ if (rinfo->has_clone || rinfo->is_clone)
+ {
+ Assert(!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids));
+ if (!bms_is_subset(rinfo->required_relids, both_input_relids))
+ continue;
+ if (!clause_is_computable_at(root, rinfo->clause_relids,
+ both_input_relids))
+ continue;
+ }
+ else
+ {
+ /*
+ * For non-clone clauses, we just Assert it's OK. These might
+ * be either join or filter clauses.
+ */
+#ifdef USE_ASSERT_CHECKING
+ if (RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ Assert(clause_is_computable_at(root, rinfo->clause_relids,
+ joinrel->relids));
+ else
+ {
+ Assert(bms_is_subset(rinfo->required_relids,
+ both_input_relids));
+ Assert(clause_is_computable_at(root, rinfo->clause_relids,
+ both_input_relids));
+ }
+#endif
+ }
+
+ /*
+ * OK, so add it to the list, being careful to eliminate
+ * duplicates. (Since RestrictInfo nodes in different joinlists
+ * will have been multiply-linked rather than copied, pointer
+ * equality should be a sufficient test.)
*/
new_restrictlist = list_append_unique_ptr(new_restrictlist, rinfo);
}
@@ -1664,9 +1821,10 @@ find_param_path_info(RelOptInfo *rel, Relids required_outer)
* partitioned join relation.
*/
static void
-build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
- RelOptInfo *inner_rel, List *restrictlist,
- JoinType jointype)
+build_joinrel_partition_info(PlannerInfo *root,
+ RelOptInfo *joinrel, RelOptInfo *outer_rel,
+ RelOptInfo *inner_rel, SpecialJoinInfo *sjinfo,
+ List *restrictlist)
{
PartitionScheme part_scheme;
@@ -1692,8 +1850,8 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
!outer_rel->consider_partitionwise_join ||
!inner_rel->consider_partitionwise_join ||
outer_rel->part_scheme != inner_rel->part_scheme ||
- !have_partkey_equi_join(joinrel, outer_rel, inner_rel,
- jointype, restrictlist))
+ !have_partkey_equi_join(root, joinrel, outer_rel, inner_rel,
+ sjinfo->jointype, restrictlist))
{
Assert(!IS_PARTITIONED_REL(joinrel));
return;
@@ -1717,7 +1875,8 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* child-join relations of the join relation in try_partitionwise_join().
*/
joinrel->part_scheme = part_scheme;
- set_joinrel_partition_key_exprs(joinrel, outer_rel, inner_rel, jointype);
+ set_joinrel_partition_key_exprs(joinrel, outer_rel, inner_rel,
+ sjinfo->jointype);
/*
* Set the consider_partitionwise_join flag.
@@ -1735,7 +1894,7 @@ build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* partition keys.
*/
static bool
-have_partkey_equi_join(RelOptInfo *joinrel,
+have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *rel1, RelOptInfo *rel2,
JoinType jointype, List *restrictlist)
{
@@ -1800,6 +1959,24 @@ have_partkey_equi_join(RelOptInfo *joinrel,
*/
strict_op = op_strict(opexpr->opno);
+ /*
+ * Vars appearing in the relation's partition keys will not have any
+ * varnullingrels, but those in expr1 and expr2 will if we're above
+ * outer joins that could null the respective rels. It's okay to
+ * match anyway, if the join operator is strict.
+ */
+ if (strict_op)
+ {
+ if (bms_overlap(rel1->relids, root->outer_join_rels))
+ expr1 = (Expr *) remove_nulling_relids((Node *) expr1,
+ root->outer_join_rels,
+ NULL);
+ if (bms_overlap(rel2->relids, root->outer_join_rels))
+ expr2 = (Expr *) remove_nulling_relids((Node *) expr2,
+ root->outer_join_rels,
+ NULL);
+ }
+
/*
* Only clauses referencing the partition keys are useful for
* partitionwise join.
@@ -2012,7 +2189,12 @@ set_joinrel_partition_key_exprs(RelOptInfo *joinrel,
* partitionwise nesting of any outer join.) We assume no
* type coercions are needed to make the coalesce expressions,
* since columns of different types won't have gotten
- * classified as the same PartitionScheme.
+ * classified as the same PartitionScheme. Note that we
+ * intentionally leave out the varnullingrels decoration that
+ * would ordinarily appear on the Vars inside these
+ * CoalesceExprs, because have_partkey_equi_join will strip
+ * varnullingrels from the expressions it will compare to the
+ * partexprs.
*/
foreach(lc, list_concat_copy(outer_expr, outer_null_expr))
{
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index 0c3878a805..d6dd8307ec 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -53,6 +53,10 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
* required_relids can be NULL, in which case it defaults to the actual clause
* contents (i.e., clause_relids).
*
+ * Note that there aren't options to set the has_clone and is_clone flags:
+ * we always initialize those to false. There's just one place that wants
+ * something different, so making all callers pass them seems inconvenient.
+ *
* We initialize fields that depend only on the given subexpression, leaving
* others that depend on context (or may never be needed at all) to be filled
* later.
@@ -116,12 +120,15 @@ make_restrictinfo_internal(PlannerInfo *root,
Relids nullable_relids)
{
RestrictInfo *restrictinfo = makeNode(RestrictInfo);
+ Relids baserels;
restrictinfo->clause = clause;
restrictinfo->orclause = orclause;
restrictinfo->is_pushed_down = is_pushed_down;
restrictinfo->outerjoin_delayed = outerjoin_delayed;
restrictinfo->pseudoconstant = pseudoconstant;
+ restrictinfo->has_clone = false; /* may get set by caller */
+ restrictinfo->is_clone = false; /* may get set by caller */
restrictinfo->can_join = false; /* may get set below */
restrictinfo->security_level = security_level;
restrictinfo->outer_relids = outer_relids;
@@ -187,6 +194,20 @@ make_restrictinfo_internal(PlannerInfo *root,
else
restrictinfo->required_relids = restrictinfo->clause_relids;
+ /*
+ * Count the number of base rels appearing in clause_relids. To do this,
+ * we just delete rels mentioned in root->outer_join_rels and count the
+ * survivors. Because we are called during deconstruct_jointree which is
+ * the same tree walk that populates outer_join_rels, this is a little bit
+ * unsafe-looking; but it should be fine because the recursion in
+ * deconstruct_jointree should already have visited any outer join that
+ * could be mentioned in this clause.
+ */
+ baserels = bms_difference(restrictinfo->clause_relids,
+ root->outer_join_rels);
+ restrictinfo->num_base_rels = bms_num_members(baserels);
+ bms_free(baserels);
+
/*
* Fill in all the cacheable fields with "not yet set" markers. None of
* these will be computed until/unless needed. Note in particular that we
@@ -497,6 +518,58 @@ extract_actual_join_clauses(List *restrictinfo_list,
}
}
+/*
+ * clause_is_computable_at
+ * Test whether a clause is computable at a given evaluation level.
+ *
+ * There are two conditions for whether an expression can actually be
+ * evaluated at a given join level: the evaluation context must include
+ * all the relids (both base and OJ) used by the expression, and we must
+ * not have already evaluated any outer joins that null Vars/PHVs of the
+ * expression and are not listed in their nullingrels.
+ *
+ * This function checks the second condition; we assume the caller already
+ * saw to the first one.
+ *
+ * For speed reasons, we don't individually examine each Var/PHV of the
+ * expression, but just look at the overall clause_relids (the union of the
+ * varnos and varnullingrels). This could give a misleading answer if the
+ * Vars of a given varno don't all have the same varnullingrels; but that
+ * really shouldn't happen within a single scalar expression or RestrictInfo
+ * clause. Despite that, this is still annoyingly expensive :-(
+ */
+bool
+clause_is_computable_at(PlannerInfo *root,
+ Relids clause_relids,
+ Relids eval_relids)
+{
+ ListCell *lc;
+
+ /* Nothing to do if no outer joins have been performed yet. */
+ if (!bms_overlap(eval_relids, root->outer_join_rels))
+ return true;
+
+ foreach(lc, root->join_info_list)
+ {
+ SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
+
+ /* Ignore outer joins that are not yet performed. */
+ if (!bms_is_member(sjinfo->ojrelid, eval_relids))
+ continue;
+
+ /* OK if clause lists it (we assume all Vars in it agree). */
+ if (bms_is_member(sjinfo->ojrelid, clause_relids))
+ continue;
+
+ /* Else, trouble if clause mentions any nullable Vars. */
+ if (bms_overlap(clause_relids, sjinfo->min_righthand) ||
+ (sjinfo->jointype == JOIN_FULL &&
+ bms_overlap(clause_relids, sjinfo->min_lefthand)))
+ return false; /* doesn't work */
+ }
+
+ return true; /* OK */
+}
/*
* join_clause_is_movable_to
@@ -522,6 +595,12 @@ extract_actual_join_clauses(List *restrictinfo_list,
* Also, the join clause must not use any relations that have LATERAL
* references to the target relation, since we could not put such rels on
* the outer side of a nestloop with the target relation.
+ *
+ * Also, we reject is_clone versions of outer-join clauses. This has the
+ * effect of preventing us from generating variant parameterized paths
+ * that differ only in which outer joins null the parameterization rel(s).
+ * Generating one path from the minimally-parameterized has_clone version
+ * is sufficient.
*/
bool
join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
@@ -542,6 +621,10 @@ join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
if (bms_overlap(baserel->lateral_referencers, rinfo->clause_relids))
return false;
+ /* Ignore clones, too */
+ if (rinfo->is_clone)
+ return false;
+
return true;
}
@@ -587,6 +670,9 @@ join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
* moved for some valid set of outer rels, so we don't have the benefit of
* relying on prior checks for lateral-reference validity.
*
+ * Likewise, we don't check is_clone here: rejecting the inappropriate
+ * variants of a cloned clause must be handled upstream.
+ *
* Note: if this returns true, it means that the clause could be moved to
* this join relation, but that doesn't mean that this is the lowest join
* it could be moved to. Caller may need to make additional calls to verify
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
index 09cc7d6f51..b65de54cf8 100644
--- a/src/backend/optimizer/util/var.c
+++ b/src/backend/optimizer/util/var.c
@@ -88,6 +88,9 @@ static Relids alias_relid_set(Query *query, Relids relids);
* Create a set of all the distinct varnos present in a parsetree.
* Only varnos that reference level-zero rtable entries are considered.
*
+ * The result includes outer-join relids mentioned in Var.varnullingrels and
+ * PlaceHolderVar.phnullingrels fields in the parsetree.
+ *
* "root" can be passed as NULL if it is not necessary to process
* PlaceHolderVars.
*
@@ -153,7 +156,11 @@ pull_varnos_walker(Node *node, pull_varnos_context *context)
Var *var = (Var *) node;
if (var->varlevelsup == context->sublevels_up)
+ {
context->varnos = bms_add_member(context->varnos, var->varno);
+ context->varnos = bms_add_members(context->varnos,
+ var->varnullingrels);
+ }
return false;
}
if (IsA(node, CurrentOfExpr))
@@ -244,6 +251,14 @@ pull_varnos_walker(Node *node, pull_varnos_context *context)
context->varnos = bms_join(context->varnos,
newevalat);
}
+
+ /*
+ * In all three cases, include phnullingrels in the result. We
+ * don't worry about possibly needing to translate it, because
+ * appendrels only translate varnos of baserels, not outer joins.
+ */
+ context->varnos = bms_add_members(context->varnos,
+ phv->phnullingrels);
return false; /* don't recurse into expression */
}
}
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index 4e4888dde4..fe37e65af0 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -2206,7 +2206,7 @@ rowcomparesel(PlannerInfo *root,
else
{
/*
- * Otherwise, it's a join if there's more than one relation used.
+ * Otherwise, it's a join if there's more than one base relation used.
*/
is_join_clause = (NumRelids(root, (Node *) opargs) > 1);
}
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 0909420615..583af844d0 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -254,6 +254,20 @@ struct PlannerInfo
*/
Relids all_baserels;
+ /*
+ * outer_join_rels is a Relids set of all outer-join relids in the query.
+ * This is computed in deconstruct_jointree.
+ */
+ Relids outer_join_rels;
+
+ /*
+ * all_query_rels is a Relids set of all base relids and outer join relids
+ * (but not "other" relids) in the query. This is the Relids identifier
+ * of the final join we need to form. This is computed in
+ * deconstruct_jointree.
+ */
+ Relids all_query_rels;
+
/*
* nullable_baserels is a Relids set of base relids that are nullable by
* some outer join in the jointree; these are rels that are potentially
@@ -590,9 +604,10 @@ typedef struct PartitionSchemeData *PartitionScheme;
* or the output of a sub-SELECT or function that appears in the range table.
* In either case it is uniquely identified by an RT index. A "joinrel"
* is the joining of two or more base rels. A joinrel is identified by
- * the set of RT indexes for its component baserels. We create RelOptInfo
- * nodes for each baserel and joinrel, and store them in the PlannerInfo's
- * simple_rel_array and join_rel_list respectively.
+ * the set of RT indexes for its component baserels, along with RT indexes
+ * for any outer joins it has computed. We create RelOptInfo nodes for each
+ * baserel and joinrel, and store them in the PlannerInfo's simple_rel_array
+ * and join_rel_list respectively.
*
* Note that there is only one joinrel for any given set of component
* baserels, no matter what order we assemble them in; so an unordered
@@ -631,8 +646,10 @@ typedef struct PartitionSchemeData *PartitionScheme;
* Parts of this data structure are specific to various scan and join
* mechanisms. It didn't seem worth creating new node types for them.
*
- * relids - Set of base-relation identifiers; it is a base relation
- * if there is just one, a join relation if more than one
+ * relids - Set of relation identifiers (RT indexes). This is a base
+ * relation if there is just one, a join relation if more;
+ * in the join case, RT indexes of any outer joins formed
+ * at or below this join are included along with baserels
* rows - estimated number of tuples in the relation after restriction
* clauses have been applied (ie, output rows of a plan for it)
* consider_startup - true if there is any value in keeping plain paths for
@@ -844,7 +861,7 @@ typedef struct RelOptInfo
RelOptKind reloptkind;
/*
- * all relations included in this RelOptInfo; set of base relids
+ * all relations included in this RelOptInfo; set of base + OJ relids
* (rangetable indexes)
*/
Relids relids;
@@ -2314,17 +2331,17 @@ typedef struct LimitPath
* If a restriction clause references a single base relation, it will appear
* in the baserestrictinfo list of the RelOptInfo for that base rel.
*
- * If a restriction clause references more than one base rel, it will
+ * If a restriction clause references more than one base+OJ relation, it will
* appear in the joininfo list of every RelOptInfo that describes a strict
- * subset of the base rels mentioned in the clause. The joininfo lists are
+ * subset of the relations mentioned in the clause. The joininfo lists are
* used to drive join tree building by selecting plausible join candidates.
* The clause cannot actually be applied until we have built a join rel
- * containing all the base rels it references, however.
+ * containing all the relations it references, however.
*
- * When we construct a join rel that includes all the base rels referenced
+ * When we construct a join rel that includes all the relations referenced
* in a multi-relation restriction clause, we place that clause into the
* joinrestrictinfo lists of paths for the join rel, if neither left nor
- * right sub-path includes all base rels referenced in the clause. The clause
+ * right sub-path includes all relations referenced in the clause. The clause
* will be applied at that join level, and will not propagate any further up
* the join tree. (Note: the "predicate migration" code was once intended to
* push restriction clauses up and down the plan tree based on evaluation
@@ -2345,12 +2362,14 @@ typedef struct LimitPath
* or join to enforce that all members of each EquivalenceClass are in fact
* equal in all rows emitted by the scan or join.
*
- * When dealing with outer joins we have to be very careful about pushing qual
- * clauses up and down the tree. An outer join's own JOIN/ON conditions must
- * be evaluated exactly at that join node, unless they are "degenerate"
- * conditions that reference only Vars from the nullable side of the join.
- * Quals appearing in WHERE or in a JOIN above the outer join cannot be pushed
- * down below the outer join, if they reference any nullable Vars.
+ * The clause_relids field lists the base plus outer-join RT indexes that
+ * actually appear in the clause. required_relids lists the minimum set of
+ * relids needed to evaluate the clause; while this is often equal to
+ * clause_relids, it can be more. We will add relids to required_relids when
+ * we need to force an outer join ON clause to be evaluated exactly at the
+ * level of the outer join, which is true except when it is a "degenerate"
+ * condition that references only Vars from the nullable side of the join.
+ *
* RestrictInfo nodes contain a flag to indicate whether a qual has been
* pushed down to a lower level than its original syntactic placement in the
* join tree would suggest. If an outer join prevents us from pushing a qual
@@ -2435,6 +2454,12 @@ typedef struct LimitPath
* or merge or hash join clause, so it's of no interest to large parts of
* the planner.
*
+ * When we generate multiple versions of a clause so as to have versions
+ * that will work after commuting some left joins per outer join identity 3,
+ * we mark the one with the fewest nullingrels bits with has_clone = true,
+ * and the rest with is_clone = true. This allows proper filtering of
+ * these redundant clauses, so that we apply only one version of them.
+ *
* When join clauses are generated from EquivalenceClasses, there may be
* several equally valid ways to enforce join equivalence, of which we need
* apply only one. We mark clauses of this kind by setting parent_ec to
@@ -2469,16 +2494,23 @@ typedef struct RestrictInfo
/* see comment above */
bool pseudoconstant pg_node_attr(equal_ignore);
+ /* see comment above */
+ bool has_clone;
+ bool is_clone;
+
/* true if known to contain no leaked Vars */
bool leakproof pg_node_attr(equal_ignore);
- /* to indicate if clause contains any volatile functions. */
+ /* indicates if clause contains any volatile functions */
VolatileFunctionStatus has_volatile pg_node_attr(equal_ignore);
/* see comment above */
Index security_level;
- /* The set of relids (varnos) actually referenced in the clause: */
+ /* number of base rels in clause_relids */
+ int num_base_rels pg_node_attr(equal_ignore);
+
+ /* The relids (varnos+varnullingrels) actually referenced in the clause: */
Relids clause_relids pg_node_attr(equal_ignore);
/* The set of relids required to evaluate the clause: */
@@ -2682,20 +2714,49 @@ typedef struct PlaceHolderVar
* We make SpecialJoinInfos for FULL JOINs even though there is no flexibility
* of planning for them, because this simplifies make_join_rel()'s API.
*
- * min_lefthand and min_righthand are the sets of base relids that must be
- * available on each side when performing the special join. lhs_strict is
- * true if the special join's condition cannot succeed when the LHS variables
- * are all NULL (this means that an outer join can commute with upper-level
- * outer joins even if it appears in their RHS). We don't bother to set
- * lhs_strict for FULL JOINs, however.
- *
+ * min_lefthand and min_righthand are the sets of base+OJ relids that must be
+ * available on each side when performing the special join.
* It is not valid for either min_lefthand or min_righthand to be empty sets;
* if they were, this would break the logic that enforces join order.
*
- * syn_lefthand and syn_righthand are the sets of base relids that are
+ * syn_lefthand and syn_righthand are the sets of base+OJ relids that are
* syntactically below this special join. (These are needed to help compute
* min_lefthand and min_righthand for higher joins.)
*
+ * jointype is never JOIN_RIGHT; a RIGHT JOIN is handled by switching
+ * the inputs to make it a LEFT JOIN. So the allowed values of jointype
+ * in a join_info_list member are only LEFT, FULL, SEMI, or ANTI.
+ *
+ * ojrelid is the RT index of the join RTE representing this outer join,
+ * if there is one. It is zero when jointype is INNER or SEMI, and can be
+ * zero for jointype ANTI (if the join was transformed from a SEMI join).
+ * One use for this field is that when constructing the output targetlist of a
+ * join relation that implements this OJ, we add ojrelid to the varnullingrels
+ * and phnullingrels fields of nullable (RHS) output columns, so that the
+ * output Vars and PlaceHolderVars correctly reflect the nulling that has
+ * potentially happened to them.
+ *
+ * commute_above_l is filled with the relids of syntactically-higher outer
+ * joins that have been found to commute with this one per outer join identity
+ * 3 (see optimizer/README), when this join is in the LHS of the upper join
+ * (so, this is the lower join in the first form of the identity).
+ *
+ * commute_above_r is filled with the relids of syntactically-higher outer
+ * joins that have been found to commute with this one per outer join identity
+ * 3, when this join is in the RHS of the upper join (so, this is the lower
+ * join in the second form of the identity).
+ *
+ * commute_below is filled with the relids of syntactically-lower outer joins
+ * that have been found to commute with this one per outer join identity 3.
+ * (We need not record which side they are on, since that can be determined
+ * by seeing whether the lower join's relid appears in syn_lefthand or
+ * syn_righthand.)
+ *
+ * lhs_strict is true if the special join's condition cannot succeed when the
+ * LHS variables are all NULL (this means that an outer join can commute with
+ * upper-level outer joins even if it appears in their RHS). We don't bother
+ * to set lhs_strict for FULL JOINs, however.
+ *
* delay_upper_joins is set true if we detect a pushed-down clause that has
* to be evaluated after this join is formed (because it references the RHS).
* Any outer joins that have such a clause and this join in their RHS cannot
@@ -2710,10 +2771,6 @@ typedef struct PlaceHolderVar
* join planning; but it's helpful to have it available during planning of
* parameterized table scans, so we store it in the SpecialJoinInfo structs.)
*
- * jointype is never JOIN_RIGHT; a RIGHT JOIN is handled by switching
- * the inputs to make it a LEFT JOIN. So the allowed values of jointype
- * in a join_info_list member are only LEFT, FULL, SEMI, or ANTI.
- *
* For purposes of join selectivity estimation, we create transient
* SpecialJoinInfo structures for regular inner joins; so it is possible
* to have jointype == JOIN_INNER in such a structure, even though this is
@@ -2733,11 +2790,15 @@ struct SpecialJoinInfo
pg_node_attr(no_read)
NodeTag type;
- Relids min_lefthand; /* base relids in minimum LHS for join */
- Relids min_righthand; /* base relids in minimum RHS for join */
- Relids syn_lefthand; /* base relids syntactically within LHS */
- Relids syn_righthand; /* base relids syntactically within RHS */
+ Relids min_lefthand; /* base+OJ relids in minimum LHS for join */
+ Relids min_righthand; /* base+OJ relids in minimum RHS for join */
+ Relids syn_lefthand; /* base+OJ relids syntactically within LHS */
+ Relids syn_righthand; /* base+OJ relids syntactically within RHS */
JoinType jointype; /* always INNER, LEFT, FULL, SEMI, or ANTI */
+ Index ojrelid; /* outer join's RT index; 0 if none */
+ Relids commute_above_l; /* commuting OJs above this one, if LHS */
+ Relids commute_above_r; /* commuting OJs above this one, if RHS */
+ Relids commute_below; /* commuting OJs below this one */
bool lhs_strict; /* joinclause is strict for some LHS rel */
bool delay_upper_joins; /* can't commute with upper RHS */
/* Remaining fields are set only for JOIN_SEMI jointype: */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 02305ef902..8c05cafc8a 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -304,6 +304,7 @@ extern void expand_planner_arrays(PlannerInfo *root, int add_size);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
+extern RelOptInfo *find_base_rel_ignore_join(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
@@ -335,6 +336,6 @@ extern ParamPathInfo *find_param_path_info(RelOptInfo *rel,
extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
- SpecialJoinInfo *sjinfo, JoinType jointype);
+ SpecialJoinInfo *sjinfo);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/placeholder.h b/src/include/optimizer/placeholder.h
index 7367dca1e8..31e1578e82 100644
--- a/src/include/optimizer/placeholder.h
+++ b/src/include/optimizer/placeholder.h
@@ -27,6 +27,9 @@ extern void update_placeholder_eval_levels(PlannerInfo *root,
extern void fix_placeholder_input_needed_levels(PlannerInfo *root);
extern void add_placeholders_to_base_rels(PlannerInfo *root);
extern void add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
- RelOptInfo *outer_rel, RelOptInfo *inner_rel);
+ RelOptInfo *outer_rel, RelOptInfo *inner_rel,
+ SpecialJoinInfo *sjinfo);
+extern bool contain_placeholder_references_to(PlannerInfo *root, Node *clause,
+ int relid);
#endif /* PLACEHOLDER_H */
diff --git a/src/include/optimizer/prep.h b/src/include/optimizer/prep.h
index 452b92ad55..54fd61c9c3 100644
--- a/src/include/optimizer/prep.h
+++ b/src/include/optimizer/prep.h
@@ -29,7 +29,8 @@ extern void pull_up_subqueries(PlannerInfo *root);
extern void flatten_simple_union_all(PlannerInfo *root);
extern void reduce_outer_joins(PlannerInfo *root);
extern void remove_useless_result_rtes(PlannerInfo *root);
-extern Relids get_relids_in_jointree(Node *jtnode, bool include_joins);
+extern Relids get_relids_in_jointree(Node *jtnode, bool include_outer_joins,
+ bool include_inner_joins);
extern Relids get_relids_for_join(Query *query, int joinrelid);
/*
diff --git a/src/include/optimizer/restrictinfo.h b/src/include/optimizer/restrictinfo.h
index a47fccc2ed..c79bb420e4 100644
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -41,6 +41,9 @@ extern void extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
List **joinquals,
List **otherquals);
+extern bool clause_is_computable_at(PlannerInfo *root,
+ Relids clause_relids,
+ Relids eval_relids);
extern bool join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel);
extern bool join_clause_is_movable_into(RestrictInfo *rinfo,
Relids currentrelids,
commit 7bcf9dbe7b0ce47701729bc190deeb6fcbad99c4
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Jan 23 14:35:26 2023 -0500
Detect duplicated pushed-down conditions using RestrictInfo ID numbers.
create_nestloop_path needs to identify which candidates for join
restriction quals were already enforced in the parameterized inner
path. Currently we do that by relying on join_clause_is_movable_into
to give consistent answers, but that is not working very well with
variant clauses generated to satisfy outer join identity 3. We may
have a clause that (correctly) shows the outer-side Var as nulled by
a previous outer join, which makes it dependent on the nestloop outer
side having included that join, so that it appears to not be pushable
into a parameterized path that uses the un-nulled version of that Var.
Nonetheless, the cloned clause *is* redundant and we don't want
to check it again.
This patch offers a somewhat brute-force solution, which is to assign
serial numbers to RestrictInfo nodes, then check for redundancy using
serial number match rather than trusting join_clause_is_movable_into.
The variant-clause problem can be solved by allowing clauses to share
a serial number when we know that they are equivalent. Both the
outer-join variant generator and equivclass.c need to be in on that
trick in order to handle all cases that were handled well before.
It'd be nicer if we could continue to trust join_clause_is_movable_into
for this, but on the other hand this mechanism does provide a much more
concrete, harder-to-break way of verifying that we already enforced
(some version of) a qual. Any failure mode would almost certainly
be in the safe direction of enforcing a qual redundantly, which is
not a claim that the existing method can make.
This patch results in two changes to the core regression test outputs:
* One query in join.sql changes to a different join order. Examining
the cost estimates that are normally not shown, the new order is
estimated as very slightly faster, so this seems like an improvement.
I'm not quite sure why the old code did not find this join order.
* Some of the queries in partition_join.sql revert equivalence-clause
ordering back to what it was before a5fc46414. That's probably a
consequence of investigating parameterized paths in a different order
than before. Anyway, it's visibly harmless.
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index bfde80e8a4..6cbb72a672 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -35,7 +35,8 @@
static EquivalenceMember *add_eq_member(EquivalenceClass *ec,
Expr *expr, Relids relids, Relids nullable_relids,
- bool is_child, Oid datatype);
+ EquivalenceMember *parent,
+ Oid datatype);
static bool is_exprlist_member(Expr *node, List *exprs);
static void generate_base_implied_equalities_const(PlannerInfo *root,
EquivalenceClass *ec);
@@ -400,7 +401,7 @@ process_equivalence(PlannerInfo *root,
{
/* Case 3: add item2 to ec1 */
em2 = add_eq_member(ec1, item2, item2_relids, item2_nullable_relids,
- false, item2_type);
+ NULL, item2_type);
ec1->ec_sources = lappend(ec1->ec_sources, restrictinfo);
ec1->ec_below_outer_join |= below_outer_join;
ec1->ec_min_security = Min(ec1->ec_min_security,
@@ -418,7 +419,7 @@ process_equivalence(PlannerInfo *root,
{
/* Case 3: add item1 to ec2 */
em1 = add_eq_member(ec2, item1, item1_relids, item1_nullable_relids,
- false, item1_type);
+ NULL, item1_type);
ec2->ec_sources = lappend(ec2->ec_sources, restrictinfo);
ec2->ec_below_outer_join |= below_outer_join;
ec2->ec_min_security = Min(ec2->ec_min_security,
@@ -452,9 +453,9 @@ process_equivalence(PlannerInfo *root,
ec->ec_max_security = restrictinfo->security_level;
ec->ec_merged = NULL;
em1 = add_eq_member(ec, item1, item1_relids, item1_nullable_relids,
- false, item1_type);
+ NULL, item1_type);
em2 = add_eq_member(ec, item2, item2_relids, item2_nullable_relids,
- false, item2_type);
+ NULL, item2_type);
root->eq_classes = lappend(root->eq_classes, ec);
@@ -544,7 +545,7 @@ canonicalize_ec_expression(Expr *expr, Oid req_type, Oid req_collation)
*/
static EquivalenceMember *
add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
- Relids nullable_relids, bool is_child, Oid datatype)
+ Relids nullable_relids, EquivalenceMember *parent, Oid datatype)
{
EquivalenceMember *em = makeNode(EquivalenceMember);
@@ -552,8 +553,9 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
em->em_relids = relids;
em->em_nullable_relids = nullable_relids;
em->em_is_const = false;
- em->em_is_child = is_child;
+ em->em_is_child = (parent != NULL);
em->em_datatype = datatype;
+ em->em_parent = parent;
if (bms_is_empty(relids))
{
@@ -565,12 +567,12 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
* get_eclass_for_sort_expr() has to work harder. We put the tests
* there not here to save cycles in the equivalence case.
*/
- Assert(!is_child);
+ Assert(!parent);
em->em_is_const = true;
ec->ec_has_const = true;
/* it can't affect ec_relids */
}
- else if (!is_child) /* child members don't add to ec_relids */
+ else if (!parent) /* child members don't add to ec_relids */
{
ec->ec_relids = bms_add_members(ec->ec_relids, relids);
}
@@ -723,7 +725,7 @@ get_eclass_for_sort_expr(PlannerInfo *root,
nullable_relids = bms_intersect(nullable_relids, expr_relids);
newem = add_eq_member(newec, copyObject(expr), expr_relids,
- nullable_relids, false, opcintype);
+ nullable_relids, NULL, opcintype);
/*
* add_eq_member doesn't check for volatile functions, set-returning
@@ -1821,6 +1823,7 @@ create_join_clause(PlannerInfo *root,
EquivalenceClass *parent_ec)
{
RestrictInfo *rinfo;
+ RestrictInfo *parent_rinfo = NULL;
ListCell *lc;
MemoryContext oldcontext;
@@ -1865,6 +1868,20 @@ create_join_clause(PlannerInfo *root,
*/
oldcontext = MemoryContextSwitchTo(root->planner_cxt);
+ /*
+ * If either EM is a child, recursively create the corresponding
+ * parent-to-parent clause, so that we can duplicate its rinfo_serial.
+ */
+ if (leftem->em_is_child || rightem->em_is_child)
+ {
+ EquivalenceMember *leftp = leftem->em_parent ? leftem->em_parent : leftem;
+ EquivalenceMember *rightp = rightem->em_parent ? rightem->em_parent : rightem;
+
+ parent_rinfo = create_join_clause(root, ec, opno,
+ leftp, rightp,
+ parent_ec);
+ }
+
rinfo = build_implied_join_equality(root,
opno,
ec->ec_collation,
@@ -1876,6 +1893,10 @@ create_join_clause(PlannerInfo *root,
rightem->em_nullable_relids),
ec->ec_min_security);
+ /* If it's a child clause, copy the parent's rinfo_serial */
+ if (parent_rinfo)
+ rinfo->rinfo_serial = parent_rinfo->rinfo_serial;
+
/* Mark the clause as redundant, or not */
rinfo->parent_ec = parent_ec;
@@ -2691,7 +2712,7 @@ add_child_rel_equivalences(PlannerInfo *root,
(void) add_eq_member(cur_ec, child_expr,
new_relids, new_nullable_relids,
- true, cur_em->em_datatype);
+ cur_em, cur_em->em_datatype);
/* Record this EC index for the child rel */
child_rel->eclass_indexes = bms_add_member(child_rel->eclass_indexes, i);
@@ -2832,7 +2853,7 @@ add_child_join_rel_equivalences(PlannerInfo *root,
(void) add_eq_member(cur_ec, child_expr,
new_relids, new_nullable_relids,
- true, cur_em->em_datatype);
+ cur_em, cur_em->em_datatype);
}
}
}
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 7bd654a77e..7db8291549 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -1889,6 +1889,7 @@ deconstruct_distribute_oj_quals(PlannerInfo *root,
Relids joins_below;
Relids joins_so_far;
List *quals;
+ int save_last_rinfo_serial;
ListCell *lc;
/*
@@ -1927,6 +1928,16 @@ deconstruct_distribute_oj_quals(PlannerInfo *root,
joins_below,
NULL);
+ /*
+ * Each time we produce RestrictInfo(s) from these quals, reset the
+ * last_rinfo_serial counter, so that the RestrictInfos for the "same"
+ * qual condition get identical serial numbers. (This relies on the
+ * fact that we're not changing the qual list in any way that'd affect
+ * the number of RestrictInfos built from it.) This'll allow us to
+ * detect duplicative qual usage later.
+ */
+ save_last_rinfo_serial = root->last_rinfo_serial;
+
joins_so_far = NULL;
foreach(lc, jtitems)
{
@@ -1964,6 +1975,9 @@ deconstruct_distribute_oj_quals(PlannerInfo *root,
continue;
}
+ /* Reset serial counter for this version of the quals */
+ root->last_rinfo_serial = save_last_rinfo_serial;
+
/*
* When we are looking at joins above sjinfo, we are envisioning
* pushing sjinfo to above othersj, so add othersj's nulling bit
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 8a472bce0c..12fe442980 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -627,6 +627,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
root->multiexpr_params = NIL;
root->eq_classes = NIL;
root->ec_merging_done = false;
+ root->last_rinfo_serial = 0;
root->all_result_relids =
parse->resultRelation ? bms_make_singleton(parse->resultRelation) : NULL;
root->leaf_result_relids = NULL; /* we'll find out leaf-ness later */
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 029172333e..a5380361bf 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -992,6 +992,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
subroot->multiexpr_params = NIL;
subroot->eq_classes = NIL;
subroot->ec_merging_done = false;
+ subroot->last_rinfo_serial = 0;
subroot->all_result_relids = NULL;
subroot->leaf_result_relids = NULL;
subroot->append_rel_list = NIL;
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index 61378c8d58..dba895ccc4 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -427,7 +427,7 @@ adjust_appendrel_attrs_mutator(Node *node,
RestrictInfo *oldinfo = (RestrictInfo *) node;
RestrictInfo *newinfo = makeNode(RestrictInfo);
- /* Copy all flat-copiable fields */
+ /* Copy all flat-copiable fields, notably including rinfo_serial */
memcpy(newinfo, oldinfo, sizeof(RestrictInfo));
/* Recursively fix the clause itself */
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 6d76a11c08..f2bf68d33b 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2442,12 +2442,12 @@ create_nestloop_path(PlannerInfo *root,
* restrict_clauses that are due to be moved into the inner path. We have
* to do this now, rather than postpone the work till createplan time,
* because the restrict_clauses list can affect the size and cost
- * estimates for this path.
+ * estimates for this path. We detect such clauses by checking for serial
+ * number match to clauses already enforced in the inner path.
*/
if (bms_overlap(inner_req_outer, outer_path->parent->relids))
{
- Relids inner_and_outer = bms_union(inner_path->parent->relids,
- inner_req_outer);
+ Bitmapset *enforced_serials = get_param_path_clause_serials(inner_path);
List *jclauses = NIL;
ListCell *lc;
@@ -2455,9 +2455,7 @@ create_nestloop_path(PlannerInfo *root,
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
- if (!join_clause_is_movable_into(rinfo,
- inner_path->parent->relids,
- inner_and_outer))
+ if (!bms_is_member(rinfo->rinfo_serial, enforced_serials))
jclauses = lappend(jclauses, rinfo);
}
restrict_clauses = jclauses;
@@ -4298,6 +4296,7 @@ do { \
new_ppi->ppi_rows = old_ppi->ppi_rows;
new_ppi->ppi_clauses = old_ppi->ppi_clauses;
ADJUST_CHILD_ATTRS(new_ppi->ppi_clauses);
+ new_ppi->ppi_serials = bms_copy(old_ppi->ppi_serials);
rel->ppilist = lappend(rel->ppilist, new_ppi);
MemoryContextSwitchTo(oldcontext);
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 34de0029bd..ebfb4ddd12 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -1476,6 +1476,7 @@ get_baserel_parampathinfo(PlannerInfo *root, RelOptInfo *baserel,
ParamPathInfo *ppi;
Relids joinrelids;
List *pclauses;
+ Bitmapset *pserials;
double rows;
ListCell *lc;
@@ -1518,6 +1519,15 @@ get_baserel_parampathinfo(PlannerInfo *root, RelOptInfo *baserel,
required_outer,
baserel));
+ /* Compute set of serial numbers of the enforced clauses */
+ pserials = NULL;
+ foreach(lc, pclauses)
+ {
+ RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
+
+ pserials = bms_add_member(pserials, rinfo->rinfo_serial);
+ }
+
/* Estimate the number of rows returned by the parameterized scan */
rows = get_parameterized_baserel_size(root, baserel, pclauses);
@@ -1526,6 +1536,7 @@ get_baserel_parampathinfo(PlannerInfo *root, RelOptInfo *baserel,
ppi->ppi_req_outer = required_outer;
ppi->ppi_rows = rows;
ppi->ppi_clauses = pclauses;
+ ppi->ppi_serials = pserials;
baserel->ppilist = lappend(baserel->ppilist, ppi);
return ppi;
@@ -1751,6 +1762,7 @@ get_joinrel_parampathinfo(PlannerInfo *root, RelOptInfo *joinrel,
ppi->ppi_req_outer = required_outer;
ppi->ppi_rows = rows;
ppi->ppi_clauses = NIL;
+ ppi->ppi_serials = NULL;
joinrel->ppilist = lappend(joinrel->ppilist, ppi);
return ppi;
@@ -1789,6 +1801,7 @@ get_appendrel_parampathinfo(RelOptInfo *appendrel, Relids required_outer)
ppi->ppi_req_outer = required_outer;
ppi->ppi_rows = 0;
ppi->ppi_clauses = NIL;
+ ppi->ppi_serials = NULL;
appendrel->ppilist = lappend(appendrel->ppilist, ppi);
return ppi;
@@ -1814,6 +1827,100 @@ find_param_path_info(RelOptInfo *rel, Relids required_outer)
return NULL;
}
+/*
+ * get_param_path_clause_serials
+ * Given a parameterized Path, return the set of pushed-down clauses
+ * (identified by rinfo_serial numbers) enforced within the Path.
+ */
+Bitmapset *
+get_param_path_clause_serials(Path *path)
+{
+ if (path->param_info == NULL)
+ return NULL; /* not parameterized */
+ if (IsA(path, NestPath) ||
+ IsA(path, MergePath) ||
+ IsA(path, HashPath))
+ {
+ /*
+ * For a join path, combine clauses enforced within either input path
+ * with those enforced as joinrestrictinfo in this path. Note that
+ * joinrestrictinfo may include some non-pushed-down clauses, but for
+ * current purposes it's okay if we include those in the result. (To
+ * be more careful, we could check for clause_relids overlapping the
+ * path parameterization, but it's not worth the cycles for now.)
+ */
+ JoinPath *jpath = (JoinPath *) path;
+ Bitmapset *pserials;
+ ListCell *lc;
+
+ pserials = NULL;
+ pserials = bms_add_members(pserials,
+ get_param_path_clause_serials(jpath->outerjoinpath));
+ pserials = bms_add_members(pserials,
+ get_param_path_clause_serials(jpath->innerjoinpath));
+ foreach(lc, jpath->joinrestrictinfo)
+ {
+ RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
+
+ pserials = bms_add_member(pserials, rinfo->rinfo_serial);
+ }
+ return pserials;
+ }
+ else if (IsA(path, AppendPath))
+ {
+ /*
+ * For an appendrel, take the intersection of the sets of clauses
+ * enforced in each input path.
+ */
+ AppendPath *apath = (AppendPath *) path;
+ Bitmapset *pserials;
+ ListCell *lc;
+
+ pserials = NULL;
+ foreach(lc, apath->subpaths)
+ {
+ Path *subpath = (Path *) lfirst(lc);
+ Bitmapset *subserials;
+
+ subserials = get_param_path_clause_serials(subpath);
+ if (lc == list_head(apath->subpaths))
+ pserials = bms_copy(subserials);
+ else
+ pserials = bms_int_members(pserials, subserials);
+ }
+ return pserials;
+ }
+ else if (IsA(path, MergeAppendPath))
+ {
+ /* Same as AppendPath case */
+ MergeAppendPath *apath = (MergeAppendPath *) path;
+ Bitmapset *pserials;
+ ListCell *lc;
+
+ pserials = NULL;
+ foreach(lc, apath->subpaths)
+ {
+ Path *subpath = (Path *) lfirst(lc);
+ Bitmapset *subserials;
+
+ subserials = get_param_path_clause_serials(subpath);
+ if (lc == list_head(apath->subpaths))
+ pserials = bms_copy(subserials);
+ else
+ pserials = bms_int_members(pserials, subserials);
+ }
+ return pserials;
+ }
+ else
+ {
+ /*
+ * Otherwise, it's a baserel path and we can use the
+ * previously-computed set of serial numbers.
+ */
+ return path->param_info->ppi_serials;
+ }
+}
+
/*
* build_joinrel_partition_info
* Checks if the two relations being joined can use partitionwise join
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index d6dd8307ec..1350f011a6 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -208,6 +208,11 @@ make_restrictinfo_internal(PlannerInfo *root,
restrictinfo->num_base_rels = bms_num_members(baserels);
bms_free(baserels);
+ /*
+ * Label this RestrictInfo with a fresh serial number.
+ */
+ restrictinfo->rinfo_serial = ++(root->last_rinfo_serial);
+
/*
* Fill in all the cacheable fields with "not yet set" markers. None of
* these will be computed until/unless needed. Note in particular that we
@@ -371,7 +376,7 @@ commute_restrictinfo(RestrictInfo *rinfo, Oid comm_op)
* ... and adjust those we need to change. Note in particular that we can
* preserve any cached selectivity or cost estimates, since those ought to
* be the same for the new clause. Likewise we can keep the source's
- * parent_ec.
+ * parent_ec. It's also important that we keep the same rinfo_serial.
*/
result->clause = (Expr *) newclause;
result->left_relids = rinfo->right_relids;
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 583af844d0..630a044089 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -344,6 +344,9 @@ struct PlannerInfo
/* list of SpecialJoinInfos */
List *join_info_list;
+ /* counter for assigning RestrictInfo serial numbers */
+ int last_rinfo_serial;
+
/*
* all_result_relids is empty for SELECT, otherwise it contains at least
* parse->resultRelation. For UPDATE/DELETE/MERGE across an inheritance
@@ -1390,6 +1393,8 @@ typedef struct EquivalenceMember
bool em_is_const; /* expression is pseudoconstant? */
bool em_is_child; /* derived version for a child relation? */
Oid em_datatype; /* the "nominal type" used by the opfamily */
+ /* if em_is_child is true, this links to corresponding EM for top parent */
+ struct EquivalenceMember *em_parent pg_node_attr(read_write_ignore);
} EquivalenceMember;
/*
@@ -1496,7 +1501,13 @@ typedef struct PathTarget
* Note: ppi_clauses is only used in ParamPathInfos for base relation paths;
* in join cases it's NIL because the set of relevant clauses varies depending
* on how the join is formed. The relevant clauses will appear in each
- * parameterized join path's joinrestrictinfo list, instead.
+ * parameterized join path's joinrestrictinfo list, instead. ParamPathInfos
+ * for append relations don't bother with this, either.
+ *
+ * ppi_serials is the set of rinfo_serial numbers for quals that are enforced
+ * by this path. As with ppi_clauses, it's only maintained for baserels.
+ * (We could construct it on-the-fly from ppi_clauses, but it seems better
+ * to materialize a copy.)
*/
typedef struct ParamPathInfo
{
@@ -1507,6 +1518,7 @@ typedef struct ParamPathInfo
Relids ppi_req_outer; /* rels supplying parameters used by path */
Cardinality ppi_rows; /* estimated number of result tuples */
List *ppi_clauses; /* join clauses available from outer rels */
+ Bitmapset *ppi_serials; /* set of rinfo_serial for enforced quals */
} ParamPathInfo;
@@ -2535,6 +2547,25 @@ typedef struct RestrictInfo
*/
Expr *orclause pg_node_attr(equal_ignore);
+ /*----------
+ * Serial number of this RestrictInfo. This is unique within the current
+ * PlannerInfo context, with a few critical exceptions:
+ * 1. When we generate multiple clones of the same qual condition to
+ * cope with outer join identity 3, all the clones get the same serial
+ * number. This reflects that we only want to apply one of them in any
+ * given plan.
+ * 2. If we manufacture a commuted version of a qual to use as an index
+ * condition, it copies the original's rinfo_serial, since it is in
+ * practice the same condition.
+ * 3. RestrictInfos made for a child relation copy their parent's
+ * rinfo_serial. Likewise, when an EquivalenceClass makes a derived
+ * equality clause for a child relation, it copies the rinfo_serial of
+ * the matching equality clause for the parent. This allows detection
+ * of redundant pushed-down equality clauses.
+ *----------
+ */
+ int rinfo_serial;
+
/*
* Generating EquivalenceClass. This field is NULL unless clause is
* potentially redundant.
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 8c05cafc8a..69be701b16 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -333,6 +333,7 @@ extern ParamPathInfo *get_appendrel_parampathinfo(RelOptInfo *appendrel,
Relids required_outer);
extern ParamPathInfo *find_param_path_info(RelOptInfo *rel,
Relids required_outer);
+extern Bitmapset *get_param_path_clause_serials(Path *path);
extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out
index c2b85d2795..360d87e1ed 100644
--- a/src/test/regress/expected/join.out
+++ b/src/test/regress/expected/join.out
@@ -2335,17 +2335,17 @@ select a.f1, b.f1, t.thousand, t.tenthous from
(select sum(f1)+1 as f1 from int4_tbl i4a) a,
(select sum(f1) as f1 from int4_tbl i4b) b
where b.f1 = t.thousand and a.f1 = b.f1 and (a.f1+b.f1+999) = t.tenthous;
- QUERY PLAN
------------------------------------------------------------------------------------------------------------------------
+ QUERY PLAN
+-----------------------------------------------------------------------------------------------------------------
Nested Loop
- -> Aggregate
- -> Seq Scan on int4_tbl i4b
-> Nested Loop
Join Filter: ((sum(i4b.f1)) = ((sum(i4a.f1) + 1)))
-> Aggregate
-> Seq Scan on int4_tbl i4a
- -> Index Only Scan using tenk1_thous_tenthous on tenk1 t
- Index Cond: ((thousand = (sum(i4b.f1))) AND (tenthous = ((((sum(i4a.f1) + 1)) + (sum(i4b.f1))) + 999)))
+ -> Aggregate
+ -> Seq Scan on int4_tbl i4b
+ -> Index Only Scan using tenk1_thous_tenthous on tenk1 t
+ Index Cond: ((thousand = (sum(i4b.f1))) AND (tenthous = ((((sum(i4a.f1) + 1)) + (sum(i4b.f1))) + 999)))
(9 rows)
select a.f1, b.f1, t.thousand, t.tenthous from
diff --git a/src/test/regress/expected/partition_join.out b/src/test/regress/expected/partition_join.out
index c0ff13fb82..e18641ab92 100644
--- a/src/test/regress/expected/partition_join.out
+++ b/src/test/regress/expected/partition_join.out
@@ -304,7 +304,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t2.b FROM prt2 t2 WHERE t2.a = 0)
-> Seq Scan on prt2_p2 t2_2
Filter: (a = 0)
-> Nested Loop Semi Join
- Join Filter: (t2_3.b = t1_3.a)
+ Join Filter: (t1_3.a = t2_3.b)
-> Seq Scan on prt1_p3 t1_3
Filter: (b = 0)
-> Materialize
@@ -601,7 +601,7 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM prt1 t1, prt2 t2, prt1_e t
Sort Key: t1.a
-> Append
-> Nested Loop
- Join Filter: (((t3_1.a + t3_1.b) / 2) = t1_1.a)
+ Join Filter: (t1_1.a = ((t3_1.a + t3_1.b) / 2))
-> Hash Join
Hash Cond: (t2_1.b = t1_1.a)
-> Seq Scan on prt2_p1 t2_1
@@ -611,7 +611,7 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM prt1 t1, prt2 t2, prt1_e t
-> Index Scan using iprt1_e_p1_ab2 on prt1_e_p1 t3_1
Index Cond: (((a + b) / 2) = t2_1.b)
-> Nested Loop
- Join Filter: (((t3_2.a + t3_2.b) / 2) = t1_2.a)
+ Join Filter: (t1_2.a = ((t3_2.a + t3_2.b) / 2))
-> Hash Join
Hash Cond: (t2_2.b = t1_2.a)
-> Seq Scan on prt2_p2 t2_2
@@ -621,7 +621,7 @@ SELECT t1.a, t1.c, t2.b, t2.c, t3.a + t3.b, t3.c FROM prt1 t1, prt2 t2, prt1_e t
-> Index Scan using iprt1_e_p2_ab2 on prt1_e_p2 t3_2
Index Cond: (((a + b) / 2) = t2_2.b)
-> Nested Loop
- Join Filter: (((t3_3.a + t3_3.b) / 2) = t1_3.a)
+ Join Filter: (t1_3.a = ((t3_3.a + t3_3.b) / 2))
-> Hash Join
Hash Cond: (t2_3.b = t1_3.a)
-> Seq Scan on prt2_p3 t2_3
@@ -926,7 +926,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t1.b FROM prt2 t1, prt1_e t2 WHER
Sort Key: t1.a
-> Append
-> Nested Loop
- Join Filter: (t1_5.b = t1_2.a)
+ Join Filter: (t1_2.a = t1_5.b)
-> HashAggregate
Group Key: t1_5.b
-> Hash Join
@@ -939,7 +939,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t1.b FROM prt2 t1, prt1_e t2 WHER
Index Cond: (a = ((t2_1.a + t2_1.b) / 2))
Filter: (b = 0)
-> Nested Loop
- Join Filter: (t1_6.b = t1_3.a)
+ Join Filter: (t1_3.a = t1_6.b)
-> HashAggregate
Group Key: t1_6.b
-> Hash Join
@@ -952,7 +952,7 @@ SELECT t1.* FROM prt1 t1 WHERE t1.a IN (SELECT t1.b FROM prt2 t1, prt1_e t2 WHER
Index Cond: (a = ((t2_2.a + t2_2.b) / 2))
Filter: (b = 0)
-> Nested Loop
- Join Filter: (t1_7.b = t1_4.a)
+ Join Filter: (t1_4.a = t1_7.b)
-> HashAggregate
Group Key: t1_7.b
-> Nested Loop
commit 376b24da1de84c7a445af9282de569fb2fbceb1b
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Jan 23 14:38:40 2023 -0500
Fix flatten_join_alias_vars() to handle varnullingrels correctly.
The remaining core regression test failures occur because
flatten_join_alias_vars() isn't doing the right thing. The
alias Var it needs to replace may have acquired varnullingrels
bits signifying the effect of upper outer joins, and if so we
must preserve that information in the replacement expression.
The simplest way to do that is to wrap the replacement expression
in a PlaceHolderVar, and that's what we have to do in the general
case where subquery pullup has mutated the replacement joinaliasvars
entry into an arbitrary expression. But in simpler cases, such as
where the joinaliasvars entry is just a Var, we'd prefer to do it
by merging the alias Var's varnullingrels into the replacement Var.
In that way the flattened alias will compare equal() to semantically
equivalent references that didn't use the alias name.
Moreover, the parser also uses this code while checking certain
semantic constraints, and in that context we *must not* generate
PlaceHolderVars. PHVs shouldn't appear in parse-time expressions,
and adding one would certainly cause the parser to decide the
query is invalid (because the result wouldn't compare equal() to
what it needs to). Fortunately, during parsing the set of possible
contents of a joinaliasvars entry is quite constrained, so we can
guarantee to apply the nullingrels info to the Vars therein.
The result of this step passes all core regression tests, but there
are still loose ends for FDWs (so that contrib/postgres_fdw will fail).
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 12fe442980..320caebd87 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -913,7 +913,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
*/
if (rte->lateral && root->hasJoinRTEs)
rte->subquery = (Query *)
- flatten_join_alias_vars(root->parse,
+ flatten_join_alias_vars(root, root->parse,
(Node *) rte->subquery);
}
else if (rte->rtekind == RTE_FUNCTION)
@@ -1111,7 +1111,7 @@ preprocess_expression(PlannerInfo *root, Node *expr, int kind)
kind == EXPRKIND_VALUES ||
kind == EXPRKIND_TABLESAMPLE ||
kind == EXPRKIND_TABLEFUNC))
- expr = flatten_join_alias_vars(root->parse, expr);
+ expr = flatten_join_alias_vars(root, root->parse, expr);
/*
* Simplify constant expressions. For function RTEs, this was already
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index a5380361bf..9c96a558fc 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -1082,7 +1082,8 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
* maybe even in the rewriter; but for now let's just fix this case here.)
*/
subquery->targetList = (List *)
- flatten_join_alias_vars(subroot->parse, (Node *) subquery->targetList);
+ flatten_join_alias_vars(subroot, subroot->parse,
+ (Node *) subquery->targetList);
/*
* Adjust level-0 varnos in subquery so that we can append its rangetable
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
index b65de54cf8..509f307658 100644
--- a/src/backend/optimizer/util/var.c
+++ b/src/backend/optimizer/util/var.c
@@ -62,6 +62,7 @@ typedef struct
typedef struct
{
+ PlannerInfo *root; /* could be NULL! */
Query *query; /* outer Query */
int sublevels_up;
bool possible_sublink; /* could aliases include a SubLink? */
@@ -80,6 +81,10 @@ static bool pull_var_clause_walker(Node *node,
pull_var_clause_context *context);
static Node *flatten_join_alias_vars_mutator(Node *node,
flatten_join_alias_vars_context *context);
+static Node *add_nullingrels_if_needed(PlannerInfo *root, Node *newnode,
+ Var *oldvar);
+static bool is_standard_join_alias_expression(Node *newnode, Var *oldvar);
+static void adjust_standard_join_alias_expression(Node *newnode, Var *oldvar);
static Relids alias_relid_set(Query *query, Relids relids);
@@ -722,26 +727,42 @@ pull_var_clause_walker(Node *node, pull_var_clause_context *context)
* is the only way that the executor can directly handle whole-row Vars.
*
* This also adjusts relid sets found in some expression node types to
- * substitute the contained base rels for any join relid.
+ * substitute the contained base+OJ rels for any join relid.
*
* If a JOIN contains sub-selects that have been flattened, its join alias
* entries might now be arbitrary expressions, not just Vars. This affects
- * this function in one important way: we might find ourselves inserting
- * SubLink expressions into subqueries, and we must make sure that their
- * Query.hasSubLinks fields get set to true if so. If there are any
+ * this function in two important ways. First, we might find ourselves
+ * inserting SubLink expressions into subqueries, and we must make sure that
+ * their Query.hasSubLinks fields get set to true if so. If there are any
* SubLinks in the join alias lists, the outer Query should already have
* hasSubLinks = true, so this is only relevant to un-flattened subqueries.
+ * Second, we have to preserve any varnullingrels info attached to the
+ * alias Vars we're replacing. If the replacement expression is a Var or
+ * PlaceHolderVar or constructed from those, we can just add the
+ * varnullingrels bits to the existing nullingrels field(s); otherwise
+ * we have to add a PlaceHolderVar wrapper.
*
- * NOTE: this is used on not-yet-planned expressions. We do not expect it
- * to be applied directly to the whole Query, so if we see a Query to start
- * with, we do want to increment sublevels_up (this occurs for LATERAL
- * subqueries).
+ * NOTE: this is also used by the parser, to expand join alias Vars before
+ * checking GROUP BY validity. For that use-case, root will be NULL, which
+ * is why we have to pass the Query separately. We need the root itself only
+ * for making PlaceHolderVars. We can avoid making PlaceHolderVars in the
+ * parser's usage because it won't be dealing with arbitrary expressions:
+ * so long as adjust_standard_join_alias_expression can handle everything
+ * the parser would make as a join alias expression, we're OK.
*/
Node *
-flatten_join_alias_vars(Query *query, Node *node)
+flatten_join_alias_vars(PlannerInfo *root, Query *query, Node *node)
{
flatten_join_alias_vars_context context;
+ /*
+ * We do not expect this to be applied to the whole Query, only to
+ * expressions or LATERAL subqueries. Hence, if the top node is a Query,
+ * it's okay to immediately increment sublevels_up.
+ */
+ Assert(node != (Node *) query);
+
+ context.root = root;
context.query = query;
context.sublevels_up = 0;
/* flag whether join aliases could possibly contain SubLinks */
@@ -812,7 +833,9 @@ flatten_join_alias_vars_mutator(Node *node,
rowexpr->colnames = colnames;
rowexpr->location = var->location;
- return (Node *) rowexpr;
+ /* Lastly, add any varnullingrels to the replacement expression */
+ return add_nullingrels_if_needed(context->root, (Node *) rowexpr,
+ var);
}
/* Expand join alias reference */
@@ -839,7 +862,8 @@ flatten_join_alias_vars_mutator(Node *node,
if (context->possible_sublink && !context->inserted_sublink)
context->inserted_sublink = checkExprHasSubLink(newvar);
- return newvar;
+ /* Lastly, add any varnullingrels to the replacement expression */
+ return add_nullingrels_if_needed(context->root, newvar, var);
}
if (IsA(node, PlaceHolderVar))
{
@@ -854,6 +878,7 @@ flatten_join_alias_vars_mutator(Node *node,
{
phv->phrels = alias_relid_set(context->query,
phv->phrels);
+ /* we *don't* change phnullingrels */
}
return (Node *) phv;
}
@@ -887,9 +912,145 @@ flatten_join_alias_vars_mutator(Node *node,
(void *) context);
}
+/*
+ * Add oldvar's varnullingrels, if any, to a flattened join alias expression.
+ * The newnode has been copied, so we can modify it freely.
+ */
+static Node *
+add_nullingrels_if_needed(PlannerInfo *root, Node *newnode, Var *oldvar)
+{
+ if (oldvar->varnullingrels == NULL)
+ return newnode; /* nothing to do */
+ /* If possible, do it by adding to existing nullingrel fields */
+ if (is_standard_join_alias_expression(newnode, oldvar))
+ adjust_standard_join_alias_expression(newnode, oldvar);
+ else if (root)
+ {
+ /* We can insert a PlaceHolderVar to carry the nullingrels */
+ PlaceHolderVar *newphv;
+ Relids phrels = pull_varnos(root, newnode);
+
+ /* XXX what if phrels is empty? */
+ Assert(!bms_is_empty(phrels)); /* probably wrong */
+ newphv = make_placeholder_expr(root, (Expr *) newnode, phrels);
+ /* newphv has zero phlevelsup and NULL phnullingrels; fix it */
+ newphv->phlevelsup = oldvar->varlevelsup;
+ newphv->phnullingrels = bms_copy(oldvar->varnullingrels);
+ newnode = (Node *) newphv;
+ }
+ else
+ {
+ /* ooops, we're missing support for something the parser can make */
+ elog(ERROR, "unsupported join alias expression");
+ }
+ return newnode;
+}
+
+/*
+ * Check to see if we can insert nullingrels into this join alias expression
+ * without use of a separate PlaceHolderVar.
+ *
+ * This will handle Vars, PlaceHolderVars, and implicit-coercion and COALESCE
+ * expressions built from those. This coverage needs to handle anything
+ * that the parser would put into joinaliasvars.
+ * XXX it's probably incomplete at the moment.
+ */
+static bool
+is_standard_join_alias_expression(Node *newnode, Var *oldvar)
+{
+ if (newnode == NULL)
+ return false;
+ if (IsA(newnode, Var) &&
+ ((Var *) newnode)->varlevelsup == oldvar->varlevelsup)
+ return true;
+ else if (IsA(newnode, PlaceHolderVar) &&
+ ((PlaceHolderVar *) newnode)->phlevelsup == oldvar->varlevelsup)
+ return true;
+ else if (IsA(newnode, FuncExpr))
+ {
+ FuncExpr *fexpr = (FuncExpr *) newnode;
+
+ /*
+ * We need to assume that the function wouldn't produce non-NULL from
+ * NULL, which is reasonable for implicit coercions but otherwise not
+ * so much. (Looking at its strictness is likely overkill, and anyway
+ * it would cause us to fail if someone forgot to mark an implicit
+ * coercion as strict.)
+ */
+ if (fexpr->funcformat != COERCE_IMPLICIT_CAST ||
+ fexpr->args == NIL)
+ return false;
+
+ /*
+ * Examine only the first argument --- coercions might have additional
+ * arguments that are constants.
+ */
+ return is_standard_join_alias_expression(linitial(fexpr->args), oldvar);
+ }
+ else if (IsA(newnode, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) newnode;
+ ListCell *lc;
+
+ Assert(cexpr->args != NIL);
+ foreach(lc, cexpr->args)
+ {
+ if (!is_standard_join_alias_expression(lfirst(lc), oldvar))
+ return false;
+ }
+ return true;
+ }
+ else
+ return false;
+}
+
+/*
+ * Insert nullingrels into an expression accepted by
+ * is_standard_join_alias_expression.
+ */
+static void
+adjust_standard_join_alias_expression(Node *newnode, Var *oldvar)
+{
+ if (IsA(newnode, Var) &&
+ ((Var *) newnode)->varlevelsup == oldvar->varlevelsup)
+ {
+ Var *newvar = (Var *) newnode;
+
+ newvar->varnullingrels = bms_add_members(newvar->varnullingrels,
+ oldvar->varnullingrels);
+ }
+ else if (IsA(newnode, PlaceHolderVar) &&
+ ((PlaceHolderVar *) newnode)->phlevelsup == oldvar->varlevelsup)
+ {
+ PlaceHolderVar *newphv = (PlaceHolderVar *) newnode;
+
+ newphv->phnullingrels = bms_add_members(newphv->phnullingrels,
+ oldvar->varnullingrels);
+ }
+ else if (IsA(newnode, FuncExpr))
+ {
+ FuncExpr *fexpr = (FuncExpr *) newnode;
+
+ adjust_standard_join_alias_expression(linitial(fexpr->args), oldvar);
+ }
+ else if (IsA(newnode, CoalesceExpr))
+ {
+ CoalesceExpr *cexpr = (CoalesceExpr *) newnode;
+ ListCell *lc;
+
+ Assert(cexpr->args != NIL);
+ foreach(lc, cexpr->args)
+ {
+ adjust_standard_join_alias_expression(lfirst(lc), oldvar);
+ }
+ }
+ else
+ Assert(false);
+}
+
/*
* alias_relid_set: in a set of RT indexes, replace joins by their
- * underlying base relids
+ * underlying base+OJ relids
*/
static Relids
alias_relid_set(Query *query, Relids relids)
diff --git a/src/backend/parser/parse_agg.c b/src/backend/parser/parse_agg.c
index caecaaae50..4fbf80c271 100644
--- a/src/backend/parser/parse_agg.c
+++ b/src/backend/parser/parse_agg.c
@@ -1167,7 +1167,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
* entries are RTE_JOIN kind.
*/
if (hasJoinRTEs)
- groupClauses = (List *) flatten_join_alias_vars(qry,
+ groupClauses = (List *) flatten_join_alias_vars(NULL, qry,
(Node *) groupClauses);
/*
@@ -1211,7 +1211,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
groupClauses, hasJoinRTEs,
have_non_var_grouping);
if (hasJoinRTEs)
- clause = flatten_join_alias_vars(qry, clause);
+ clause = flatten_join_alias_vars(NULL, qry, clause);
check_ungrouped_columns(clause, pstate, qry,
groupClauses, groupClauseCommonVars,
have_non_var_grouping,
@@ -1222,7 +1222,7 @@ parseCheckAggregates(ParseState *pstate, Query *qry)
groupClauses, hasJoinRTEs,
have_non_var_grouping);
if (hasJoinRTEs)
- clause = flatten_join_alias_vars(qry, clause);
+ clause = flatten_join_alias_vars(NULL, qry, clause);
check_ungrouped_columns(clause, pstate, qry,
groupClauses, groupClauseCommonVars,
have_non_var_grouping,
@@ -1551,7 +1551,7 @@ finalize_grouping_exprs_walker(Node *node,
Index ref = 0;
if (context->hasJoinRTEs)
- expr = flatten_join_alias_vars(context->qry, expr);
+ expr = flatten_join_alias_vars(NULL, context->qry, expr);
/*
* Each expression must match a grouping entry at the current
diff --git a/src/include/optimizer/optimizer.h b/src/include/optimizer/optimizer.h
index 69ce6ee4d3..b6df013c21 100644
--- a/src/include/optimizer/optimizer.h
+++ b/src/include/optimizer/optimizer.h
@@ -197,6 +197,6 @@ extern bool contain_var_clause(Node *node);
extern bool contain_vars_of_level(Node *node, int levelsup);
extern int locate_var_of_level(Node *node, int levelsup);
extern List *pull_var_clause(Node *node, int flags);
-extern Node *flatten_join_alias_vars(Query *query, Node *node);
+extern Node *flatten_join_alias_vars(PlannerInfo *root, Query *query, Node *node);
#endif /* OPTIMIZER_H */
commit 82ea2fee2d71cd9f32fd9bc591d64780f2d7d6f4
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Jan 23 14:45:35 2023 -0500
Teach FDWs about base-plus-outer-join relids.
Conversion of the planner to include OJ relids in join relids
affects FDWs that want to plan foreign joins. They *must* follow
suit when labeling foreign joins in order to match with the core
planner, but for many purposes (if postgres_fdw is any guide)
they'd prefer to consider only base relations within the join.
To support both requirements, redefine ForeignScan.fs_relids as
base+OJ relids, and add a new field fs_base_relids that's set up
by the core planner.
Another way we could do this is to keep fs_relids as just base
relids and make the new field be the one with OJ relids added.
While that would be more backwards-compatible in some sense,
it would be inconsistent with the naming used in the core planner,
and I think that it might allow some types of bugs to escape
quick detection.
postgres_fdw also has one place where it needs to ignore varnullingrels
while matching Vars. It's not clear whether it's worth trying to
improve that. (This too is probably only an issue for FDWs that do
join planning, since Vars seen in a base relation scan should never
have any varnullingrels.)
As of this step, this patch series again passes check-world.
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
index 473fa45bd4..8a847deb13 100644
--- a/contrib/postgres_fdw/deparse.c
+++ b/contrib/postgres_fdw/deparse.c
@@ -4026,7 +4026,17 @@ get_relation_column_alias_ids(Var *node, RelOptInfo *foreignrel,
i = 1;
foreach(lc, foreignrel->reltarget->exprs)
{
- if (equal(lfirst(lc), (Node *) node))
+ Var *tlvar = (Var *) lfirst(lc);
+
+ /*
+ * Match reltarget entries only on varno/varattno. Ideally there
+ * would be some cross-check on varnullingrels, but it's unclear what
+ * to do exactly; we don't have enough context to know what that value
+ * should be.
+ */
+ if (IsA(tlvar, Var) &&
+ tlvar->varno == node->varno &&
+ tlvar->varattno == node->varattno)
{
*colno = i;
return;
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 50d23f922c..29b2be602c 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -1517,7 +1517,7 @@ postgresBeginForeignScan(ForeignScanState *node, int eflags)
if (fsplan->scan.scanrelid > 0)
rtindex = fsplan->scan.scanrelid;
else
- rtindex = bms_next_member(fsplan->fs_relids, -1);
+ rtindex = bms_next_member(fsplan->fs_base_relids, -1);
rte = exec_rt_fetch(rtindex, estate);
/* Get info about foreign table. */
@@ -2414,7 +2414,7 @@ find_modifytable_subplan(PlannerInfo *root,
{
ForeignScan *fscan = (ForeignScan *) subplan;
- if (bms_is_member(rtindex, fscan->fs_relids))
+ if (bms_is_member(rtindex, fscan->fs_base_relids))
return fscan;
}
@@ -2838,8 +2838,8 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
* that setrefs.c won't update the string when flattening the
* rangetable. To find out what rtoffset was applied, identify the
* minimum RT index appearing in the string and compare it to the
- * minimum member of plan->fs_relids. (We expect all the relids in
- * the join will have been offset by the same amount; the Asserts
+ * minimum member of plan->fs_base_relids. (We expect all the relids
+ * in the join will have been offset by the same amount; the Asserts
* below should catch it if that ever changes.)
*/
minrti = INT_MAX;
@@ -2856,7 +2856,7 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
else
ptr++;
}
- rtoffset = bms_next_member(plan->fs_relids, -1) - minrti;
+ rtoffset = bms_next_member(plan->fs_base_relids, -1) - minrti;
/* Now we can translate the string */
relations = makeStringInfo();
@@ -2871,7 +2871,7 @@ postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
char *refname;
rti += rtoffset;
- Assert(bms_is_member(rti, plan->fs_relids));
+ Assert(bms_is_member(rti, plan->fs_base_relids));
rte = rt_fetch(rti, es->rtable);
Assert(rte->rtekind == RTE_RELATION);
/* This logic should agree with explain.c's ExplainTargetRel */
diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml
index 94263c628f..ac1717bc3c 100644
--- a/doc/src/sgml/fdwhandler.sgml
+++ b/doc/src/sgml/fdwhandler.sgml
@@ -351,6 +351,17 @@ GetForeignJoinPaths(PlannerInfo *root,
it will supply at run time in the tuples it returns.
</para>
+ <note>
+ <para>
+ Beginning with <productname>PostgreSQL</productname> 16,
+ <structfield>fs_relids</structfield> includes the rangetable indexes
+ of outer joins, if any were involved in this join. The new field
+ <structfield>fs_base_relids</structfield> includes only base
+ relation indexes, and thus
+ mimics <structfield>fs_relids</structfield>'s old semantics.
+ </para>
+ </note>
+
<para>
See <xref linkend="fdw-planning"/> for additional information.
</para>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 5212a64b1e..1b0e1bc26a 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1114,7 +1114,7 @@ ExplainPreScanNode(PlanState *planstate, Bitmapset **rels_used)
break;
case T_ForeignScan:
*rels_used = bms_add_members(*rels_used,
- ((ForeignScan *) plan)->fs_relids);
+ ((ForeignScan *) plan)->fs_base_relids);
break;
case T_CustomScan:
*rels_used = bms_add_members(*rels_used,
diff --git a/src/backend/executor/execScan.c b/src/backend/executor/execScan.c
index abe8432853..cf1871b0f5 100644
--- a/src/backend/executor/execScan.c
+++ b/src/backend/executor/execScan.c
@@ -325,7 +325,7 @@ ExecScanReScan(ScanState *node)
* all of them.
*/
if (IsA(node->ps.plan, ForeignScan))
- relids = ((ForeignScan *) node->ps.plan)->fs_relids;
+ relids = ((ForeignScan *) node->ps.plan)->fs_base_relids;
else if (IsA(node->ps.plan, CustomScan))
relids = ((CustomScan *) node->ps.plan)->custom_relids;
else
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index cd68942af0..4c99b28d0a 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -4158,14 +4158,22 @@ create_foreignscan_plan(PlannerInfo *root, ForeignPath *best_path,
/*
* Likewise, copy the relids that are represented by this foreign scan. An
- * upper rel doesn't have relids set, but it covers all the base relations
- * participating in the underlying scan, so use root's all_baserels.
+ * upper rel doesn't have relids set, but it covers all the relations
+ * participating in the underlying scan/join, so use root->all_query_rels.
*/
if (rel->reloptkind == RELOPT_UPPER_REL)
- scan_plan->fs_relids = root->all_baserels;
+ scan_plan->fs_relids = root->all_query_rels;
else
scan_plan->fs_relids = best_path->path.parent->relids;
+ /*
+ * Join relid sets include relevant outer joins, but FDWs may need to know
+ * which are the included base rels. That's a bit tedious to get without
+ * access to the plan-time data structures, so compute it here.
+ */
+ scan_plan->fs_base_relids = bms_difference(scan_plan->fs_relids,
+ root->outer_join_rels);
+
/*
* If this is a foreign join, and to make it valid to push down we had to
* assume that the current user is the same as some user explicitly named
@@ -5806,8 +5814,9 @@ make_foreignscan(List *qptlist,
node->fdw_private = fdw_private;
node->fdw_scan_tlist = fdw_scan_tlist;
node->fdw_recheck_quals = fdw_recheck_quals;
- /* fs_relids will be filled in by create_foreignscan_plan */
+ /* fs_relids, fs_base_relids will be filled by create_foreignscan_plan */
node->fs_relids = NULL;
+ node->fs_base_relids = NULL;
/* fsSystemCol will be filled in by create_foreignscan_plan */
node->fsSystemCol = false;
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 8ea257f492..fb28dead4d 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -1633,6 +1633,7 @@ set_foreignscan_references(PlannerInfo *root,
}
fscan->fs_relids = offset_relid_set(fscan->fs_relids, rtoffset);
+ fscan->fs_base_relids = offset_relid_set(fscan->fs_base_relids, rtoffset);
/* Adjust resultRelation if it's valid */
if (fscan->resultRelation > 0)
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index c1234fcf36..4781a9c632 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -695,6 +695,7 @@ typedef struct WorkTableScan
* When the plan node represents a foreign join, scan.scanrelid is zero and
* fs_relids must be consulted to identify the join relation. (fs_relids
* is valid for simple scans as well, but will always match scan.scanrelid.)
+ * fs_relids includes outer joins; fs_base_relids does not.
*
* If the FDW's PlanDirectModify() callback decides to repurpose a ForeignScan
* node to perform the UPDATE or DELETE operation directly in the remote
@@ -716,7 +717,8 @@ typedef struct ForeignScan
List *fdw_private; /* private data for FDW */
List *fdw_scan_tlist; /* optional tlist describing scan tuple */
List *fdw_recheck_quals; /* original quals not in scan.plan.qual */
- Bitmapset *fs_relids; /* RTIs generated by this scan */
+ Bitmapset *fs_relids; /* base+OJ RTIs generated by this scan */
+ Bitmapset *fs_base_relids; /* base RTIs generated by this scan */
bool fsSystemCol; /* true if any "system column" is needed */
} ForeignScan;
commit f4b47feed8a8a2307121cfbecda7105930143f61
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Jan 23 14:51:02 2023 -0500
Don't use RestrictInfo.nullable_relids in join_clause_is_movable_to.
Instead of using per-clause nullable_relids data, compute a
per-baserel set of outer joins that can null each relation, and
check for overlap between that and clause_relids to detect whether
the clause can safely be pushed down to relation scan level.
join_clause_is_movable_into also uses nullable_relids, but it
turns out that that test can just be dropped entirely. Now that
clause_relids includes nulling outer joins, the preceding tests
in the function are sufficient to reject clauses that can't be
pushed down.
This might seem like a net loss given that we have to add a bit
of code to initsplan.c to compute RelOptInfo.nulling_relids.
However, that's not much code at all, and the payoff is this:
we no longer need RestrictInfo.nullable_relids at all.
The next patch, which removes that field and the extensive
infrastructure that maintains it, saves way more code and cycles
than we add here. Also, I think there are likely going to be
other uses for RelOptInfo.nulling_relids.
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 7db8291549..c285806a81 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -94,6 +94,8 @@ static void deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
static void process_security_barrier_quals(PlannerInfo *root,
int rti, Relids qualscope,
bool below_outer_join);
+static void mark_rels_nulled_by_join(PlannerInfo *root, Index ojrelid,
+ Relids lower_rels);
static SpecialJoinInfo *make_outerjoininfo(PlannerInfo *root,
Relids left_rels, Relids right_rels,
Relids inner_join_rels,
@@ -976,6 +978,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
j->rtindex);
root->outer_join_rels = bms_add_member(root->outer_join_rels,
j->rtindex);
+ mark_rels_nulled_by_join(root, j->rtindex,
+ right_item->qualscope);
}
jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
right_item->inner_join_rels);
@@ -1031,6 +1035,10 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
j->rtindex);
root->outer_join_rels = bms_add_member(root->outer_join_rels,
j->rtindex);
+ mark_rels_nulled_by_join(root, j->rtindex,
+ left_item->qualscope);
+ mark_rels_nulled_by_join(root, j->rtindex,
+ right_item->qualscope);
jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
right_item->inner_join_rels);
jtitem->left_rels = left_item->qualscope;
@@ -1345,6 +1353,33 @@ process_security_barrier_quals(PlannerInfo *root,
Assert(security_level <= root->qual_security_level);
}
+/*
+ * mark_rels_nulled_by_join
+ * Fill RelOptInfo.nulling_relids of baserels nulled by this outer join
+ *
+ * Inputs:
+ * ojrelid: RT index of the join RTE (must not be 0)
+ * lower_rels: the base+OJ Relids syntactically below nullable side of join
+ */
+static void
+mark_rels_nulled_by_join(PlannerInfo *root, Index ojrelid,
+ Relids lower_rels)
+{
+ int relid = -1;
+
+ while ((relid = bms_next_member(lower_rels, relid)) > 0)
+ {
+ RelOptInfo *rel = root->simple_rel_array[relid];
+
+ if (rel == NULL) /* must be an outer join */
+ {
+ Assert(bms_is_member(relid, root->outer_join_rels));
+ continue;
+ }
+ rel->nulling_relids = bms_add_member(rel->nulling_relids, ojrelid);
+ }
+}
+
/*
* make_outerjoininfo
* Build a SpecialJoinInfo for the current outer join
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index ebfb4ddd12..6e0e3a1a58 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -283,6 +283,12 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
rel->top_parent = parent->top_parent ? parent->top_parent : parent;
rel->top_parent_relids = rel->top_parent->relids;
+ /*
+ * A child rel is below the same outer joins as its parent. (We
+ * presume this info was already calculated for the parent.)
+ */
+ rel->nulling_relids = parent->nulling_relids;
+
/*
* Also propagate lateral-reference information from appendrel parent
* rels to their child rels. We intentionally give each child rel the
@@ -306,6 +312,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
rel->parent = NULL;
rel->top_parent = NULL;
rel->top_parent_relids = NULL;
+ rel->nulling_relids = NULL;
rel->direct_lateral_relids = NULL;
rel->lateral_relids = NULL;
rel->lateral_referencers = NULL;
@@ -685,6 +692,7 @@ build_join_rel(PlannerInfo *root,
joinrel->max_attr = 0;
joinrel->attr_needed = NULL;
joinrel->attr_widths = NULL;
+ joinrel->nulling_relids = NULL;
joinrel->lateral_vars = NIL;
joinrel->lateral_referencers = NULL;
joinrel->indexlist = NIL;
@@ -874,6 +882,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->max_attr = 0;
joinrel->attr_needed = NULL;
joinrel->attr_widths = NULL;
+ joinrel->nulling_relids = NULL;
joinrel->lateral_vars = NIL;
joinrel->lateral_referencers = NULL;
joinrel->indexlist = NIL;
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index 1350f011a6..e3b43a7cf3 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -618,8 +618,17 @@ join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
if (bms_is_member(baserel->relid, rinfo->outer_relids))
return false;
- /* Target rel must not be nullable below the clause */
- if (bms_is_member(baserel->relid, rinfo->nullable_relids))
+ /*
+ * Target rel's Vars must not be nulled by any outer join. We can check
+ * this without groveling through the individual Vars by seeing whether
+ * clause_relids (which includes all such Vars' varnullingrels) includes
+ * any outer join that can null the target rel. You might object that
+ * this could reject the clause on the basis of an OJ relid that came from
+ * some other rel's Var. However, that would still mean that the clause
+ * came from above that outer join and shouldn't be pushed down; so there
+ * should be no false positives.
+ */
+ if (bms_overlap(rinfo->clause_relids, baserel->nulling_relids))
return false;
/* Clause must not use any rels with LATERAL references to this rel */
@@ -651,16 +660,17 @@ join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel)
* relation plus the outer rels. We also check that it does reference at
* least one current Var, ensuring that the clause will be pushed down to
* a unique place in a parameterized join tree. And we check that we're
- * not pushing the clause into its outer-join outer side, nor down into
- * a lower outer join's inner side.
- *
- * The check about pushing a clause down into a lower outer join's inner side
- * is only approximate; it sometimes returns "false" when actually it would
- * be safe to use the clause here because we're still above the outer join
- * in question. This is okay as long as the answers at different join levels
- * are consistent: it just means we might sometimes fail to push a clause as
- * far down as it could safely be pushed. It's unclear whether it would be
- * worthwhile to do this more precisely. (But if it's ever fixed to be
+ * not pushing the clause into its outer-join outer side.
+ *
+ * We used to need to check that we're not pushing the clause into a lower
+ * outer join's inner side. However, now that clause_relids includes
+ * references to potentially-nulling outer joins, the other tests handle that
+ * concern. If the clause references any Var coming from the inside of a
+ * lower outer join, its clause_relids will mention that outer join, causing
+ * the evaluability check to fail; while if it references no such Vars, the
+ * references-a-target-rel check will fail.
+ *
+ * XXX not clear if we can do this yet: (But if it's ever fixed to be
* exactly accurate, there's an Assert in get_joinrel_parampathinfo() that
* should be re-enabled.)
*
@@ -704,14 +714,5 @@ join_clause_is_movable_into(RestrictInfo *rinfo,
if (bms_overlap(currentrelids, rinfo->outer_relids))
return false;
- /*
- * Target rel(s) must not be nullable below the clause. This is
- * approximate, in the safe direction, because the current join might be
- * above the join where the nulling would happen, in which case the clause
- * would work correctly here. But we don't have enough info to be sure.
- */
- if (bms_overlap(currentrelids, rinfo->nullable_relids))
- return false;
-
return true;
}
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 630a044089..cbe99ca397 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -694,6 +694,7 @@ typedef struct PartitionSchemeData *PartitionScheme;
* the attribute is needed as part of final targetlist
* attr_widths - cache space for per-attribute width estimates;
* zero means not computed yet
+ * nulling_relids - relids of outer joins that can null this rel
* lateral_vars - lateral cross-references of rel, if any (list of
* Vars and PlaceHolderVars)
* lateral_referencers - relids of rels that reference this one laterally
@@ -927,6 +928,8 @@ typedef struct RelOptInfo
Relids *attr_needed pg_node_attr(read_write_ignore);
/* array indexed [min_attr .. max_attr] */
int32 *attr_widths pg_node_attr(read_write_ignore);
+ /* relids of outer joins that can null this baserel */
+ Relids nulling_relids;
/* LATERAL Vars and PHVs referenced by rel */
List *lateral_vars;
/* rels that reference this baserel laterally */
commit dd1a14bb08b999f17407fbec447ef5e19ceff266
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Jan 23 14:59:37 2023 -0500
Remove RestrictInfo.nullable_relids and associated infrastructure.
There is no more code using this field, only code computing it,
so just delete all that. We can likewise get rid of
EquivalenceMember.em_nullable_relids and
PlannerInfo.nullable_baserels.
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 29b2be602c..d95e70dfce 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -6525,7 +6525,6 @@ foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
false,
root->qual_security_level,
grouped_rel->relids,
- NULL,
NULL);
if (is_foreign_expr(root, grouped_rel, expr))
fpinfo->remote_conds = lappend(fpinfo->remote_conds, rinfo);
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 26b294d5d0..ae0f9bdc8a 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -2745,7 +2745,6 @@ set_function_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
if (var)
pathkeys = build_expression_pathkey(root,
(Expr *) var,
- NULL, /* below outer joins */
Int8LessOperator,
rel->relids,
false);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 6cbb72a672..6f81cc2b08 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -34,7 +34,7 @@
static EquivalenceMember *add_eq_member(EquivalenceClass *ec,
- Expr *expr, Relids relids, Relids nullable_relids,
+ Expr *expr, Relids relids,
EquivalenceMember *parent,
Oid datatype);
static bool is_exprlist_member(Expr *node, List *exprs);
@@ -131,9 +131,7 @@ process_equivalence(PlannerInfo *root,
Expr *item1;
Expr *item2;
Relids item1_relids,
- item2_relids,
- item1_nullable_relids,
- item2_nullable_relids;
+ item2_relids;
List *opfamilies;
EquivalenceClass *ec1,
*ec2;
@@ -206,8 +204,7 @@ process_equivalence(PlannerInfo *root,
restrictinfo->pseudoconstant,
restrictinfo->security_level,
NULL,
- restrictinfo->outer_relids,
- restrictinfo->nullable_relids);
+ restrictinfo->outer_relids);
}
return false;
}
@@ -225,12 +222,6 @@ process_equivalence(PlannerInfo *root,
return false; /* RHS is non-strict but not constant */
}
- /* Calculate nullable-relid sets for each side of the clause */
- item1_nullable_relids = bms_intersect(item1_relids,
- restrictinfo->nullable_relids);
- item2_nullable_relids = bms_intersect(item2_relids,
- restrictinfo->nullable_relids);
-
/*
* We use the declared input types of the operator, not exprType() of the
* inputs, as the nominal datatypes for opfamily lookup. This presumes
@@ -400,7 +391,7 @@ process_equivalence(PlannerInfo *root,
else if (ec1)
{
/* Case 3: add item2 to ec1 */
- em2 = add_eq_member(ec1, item2, item2_relids, item2_nullable_relids,
+ em2 = add_eq_member(ec1, item2, item2_relids,
NULL, item2_type);
ec1->ec_sources = lappend(ec1->ec_sources, restrictinfo);
ec1->ec_below_outer_join |= below_outer_join;
@@ -418,7 +409,7 @@ process_equivalence(PlannerInfo *root,
else if (ec2)
{
/* Case 3: add item1 to ec2 */
- em1 = add_eq_member(ec2, item1, item1_relids, item1_nullable_relids,
+ em1 = add_eq_member(ec2, item1, item1_relids,
NULL, item1_type);
ec2->ec_sources = lappend(ec2->ec_sources, restrictinfo);
ec2->ec_below_outer_join |= below_outer_join;
@@ -452,9 +443,9 @@ process_equivalence(PlannerInfo *root,
ec->ec_min_security = restrictinfo->security_level;
ec->ec_max_security = restrictinfo->security_level;
ec->ec_merged = NULL;
- em1 = add_eq_member(ec, item1, item1_relids, item1_nullable_relids,
+ em1 = add_eq_member(ec, item1, item1_relids,
NULL, item1_type);
- em2 = add_eq_member(ec, item2, item2_relids, item2_nullable_relids,
+ em2 = add_eq_member(ec, item2, item2_relids,
NULL, item2_type);
root->eq_classes = lappend(root->eq_classes, ec);
@@ -545,13 +536,12 @@ canonicalize_ec_expression(Expr *expr, Oid req_type, Oid req_collation)
*/
static EquivalenceMember *
add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
- Relids nullable_relids, EquivalenceMember *parent, Oid datatype)
+ EquivalenceMember *parent, Oid datatype)
{
EquivalenceMember *em = makeNode(EquivalenceMember);
em->em_expr = expr;
em->em_relids = relids;
- em->em_nullable_relids = nullable_relids;
em->em_is_const = false;
em->em_is_child = (parent != NULL);
em->em_datatype = datatype;
@@ -588,13 +578,6 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
* equivalence class it is a member of; if none, optionally build a new
* single-member EquivalenceClass for it.
*
- * expr is the expression, and nullable_relids is the set of base relids
- * that are potentially nullable below it. We actually only care about
- * the set of such relids that are used in the expression; but for caller
- * convenience, we perform that intersection step here. The caller need
- * only be sure that nullable_relids doesn't omit any nullable rels that
- * might appear in the expr.
- *
* sortref is the SortGroupRef of the originating SortGroupClause, if any,
* or zero if not. (It should never be zero if the expression is volatile!)
*
@@ -623,7 +606,6 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
EquivalenceClass *
get_eclass_for_sort_expr(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
List *opfamilies,
Oid opcintype,
Oid collation,
@@ -719,13 +701,12 @@ get_eclass_for_sort_expr(PlannerInfo *root,
elog(ERROR, "volatile EquivalenceClass has no sortref");
/*
- * Get the precise set of nullable relids appearing in the expression.
+ * Get the precise set of relids appearing in the expression.
*/
expr_relids = pull_varnos(root, (Node *) expr);
- nullable_relids = bms_intersect(nullable_relids, expr_relids);
newem = add_eq_member(newec, copyObject(expr), expr_relids,
- nullable_relids, NULL, opcintype);
+ NULL, opcintype);
/*
* add_eq_member doesn't check for volatile functions, set-returning
@@ -1211,8 +1192,6 @@ generate_base_implied_equalities_const(PlannerInfo *root,
rinfo = process_implied_equality(root, eq_op, ec->ec_collation,
cur_em->em_expr, const_em->em_expr,
bms_copy(ec->ec_relids),
- bms_union(cur_em->em_nullable_relids,
- const_em->em_nullable_relids),
ec->ec_min_security,
ec->ec_below_outer_join,
cur_em->em_is_const);
@@ -1285,8 +1264,6 @@ generate_base_implied_equalities_no_const(PlannerInfo *root,
rinfo = process_implied_equality(root, eq_op, ec->ec_collation,
prev_em->em_expr, cur_em->em_expr,
bms_copy(ec->ec_relids),
- bms_union(prev_em->em_nullable_relids,
- cur_em->em_nullable_relids),
ec->ec_min_security,
ec->ec_below_outer_join,
false);
@@ -1889,8 +1866,6 @@ create_join_clause(PlannerInfo *root,
rightem->em_expr,
bms_union(leftem->em_relids,
rightem->em_relids),
- bms_union(leftem->em_nullable_relids,
- rightem->em_nullable_relids),
ec->ec_min_security);
/* If it's a child clause, copy the parent's rinfo_serial */
@@ -2110,8 +2085,7 @@ reconsider_outer_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo,
left_type,
right_type,
inner_datatype;
- Relids inner_relids,
- inner_nullable_relids;
+ Relids inner_relids;
ListCell *lc1;
Assert(is_opclause(rinfo->clause));
@@ -2138,8 +2112,6 @@ reconsider_outer_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo,
inner_datatype = left_type;
inner_relids = rinfo->left_relids;
}
- inner_nullable_relids = bms_intersect(inner_relids,
- rinfo->nullable_relids);
/* Scan EquivalenceClasses for a match to outervar */
foreach(lc1, root->eq_classes)
@@ -2200,7 +2172,6 @@ reconsider_outer_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo,
innervar,
cur_em->em_expr,
bms_copy(inner_relids),
- bms_copy(inner_nullable_relids),
cur_ec->ec_min_security);
if (process_equivalence(root, &newrinfo, true))
match = true;
@@ -2238,9 +2209,7 @@ reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
left_type,
right_type;
Relids left_relids,
- right_relids,
- left_nullable_relids,
- right_nullable_relids;
+ right_relids;
ListCell *lc1;
/* Can't use an outerjoin_delayed clause here */
@@ -2256,10 +2225,6 @@ reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
rightvar = (Expr *) get_rightop(rinfo->clause);
left_relids = rinfo->left_relids;
right_relids = rinfo->right_relids;
- left_nullable_relids = bms_intersect(left_relids,
- rinfo->nullable_relids);
- right_nullable_relids = bms_intersect(right_relids,
- rinfo->nullable_relids);
foreach(lc1, root->eq_classes)
{
@@ -2361,7 +2326,6 @@ reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
leftvar,
cur_em->em_expr,
bms_copy(left_relids),
- bms_copy(left_nullable_relids),
cur_ec->ec_min_security);
if (process_equivalence(root, &newrinfo, true))
matchleft = true;
@@ -2377,7 +2341,6 @@ reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
rightvar,
cur_em->em_expr,
bms_copy(right_relids),
- bms_copy(right_nullable_relids),
cur_ec->ec_min_security);
if (process_equivalence(root, &newrinfo, true))
matchright = true;
@@ -2667,7 +2630,6 @@ add_child_rel_equivalences(PlannerInfo *root,
/* Yes, generate transformed child version */
Expr *child_expr;
Relids new_relids;
- Relids new_nullable_relids;
if (parent_rel->reloptkind == RELOPT_BASEREL)
{
@@ -2697,21 +2659,7 @@ add_child_rel_equivalences(PlannerInfo *root,
top_parent_relids);
new_relids = bms_add_members(new_relids, child_relids);
- /*
- * And likewise for nullable_relids. Note this code assumes
- * parent and child relids are singletons.
- */
- new_nullable_relids = cur_em->em_nullable_relids;
- if (bms_overlap(new_nullable_relids, top_parent_relids))
- {
- new_nullable_relids = bms_difference(new_nullable_relids,
- top_parent_relids);
- new_nullable_relids = bms_add_members(new_nullable_relids,
- child_relids);
- }
-
- (void) add_eq_member(cur_ec, child_expr,
- new_relids, new_nullable_relids,
+ (void) add_eq_member(cur_ec, child_expr, new_relids,
cur_em, cur_em->em_datatype);
/* Record this EC index for the child rel */
@@ -2808,7 +2756,6 @@ add_child_join_rel_equivalences(PlannerInfo *root,
/* Yes, generate transformed child version */
Expr *child_expr;
Relids new_relids;
- Relids new_nullable_relids;
if (parent_joinrel->reloptkind == RELOPT_JOINREL)
{
@@ -2839,20 +2786,7 @@ add_child_join_rel_equivalences(PlannerInfo *root,
top_parent_relids);
new_relids = bms_add_members(new_relids, child_relids);
- /*
- * For nullable_relids, we must selectively replace parent
- * nullable relids with child ones.
- */
- new_nullable_relids = cur_em->em_nullable_relids;
- if (bms_overlap(new_nullable_relids, top_parent_relids))
- new_nullable_relids =
- adjust_child_relids_multilevel(root,
- new_nullable_relids,
- child_joinrel,
- child_joinrel->top_parent);
-
- (void) add_eq_member(cur_ec, child_expr,
- new_relids, new_nullable_relids,
+ (void) add_eq_member(cur_ec, child_expr, new_relids,
cur_em, cur_em->em_datatype);
}
}
diff --git a/src/backend/optimizer/path/pathkeys.c b/src/backend/optimizer/path/pathkeys.c
index d2e241c983..c4e7f97f68 100644
--- a/src/backend/optimizer/path/pathkeys.c
+++ b/src/backend/optimizer/path/pathkeys.c
@@ -180,9 +180,6 @@ pathkey_is_redundant(PathKey *new_pathkey, List *pathkeys)
* Given an expression and sort-order information, create a PathKey.
* The result is always a "canonical" PathKey, but it might be redundant.
*
- * expr is the expression, and nullable_relids is the set of base relids
- * that are potentially nullable below it.
- *
* If the PathKey is being generated from a SortGroupClause, sortref should be
* the SortGroupClause's SortGroupRef; otherwise zero.
*
@@ -198,7 +195,6 @@ pathkey_is_redundant(PathKey *new_pathkey, List *pathkeys)
static PathKey *
make_pathkey_from_sortinfo(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
Oid opfamily,
Oid opcintype,
Oid collation,
@@ -234,7 +230,7 @@ make_pathkey_from_sortinfo(PlannerInfo *root,
equality_op);
/* Now find or (optionally) create a matching EquivalenceClass */
- eclass = get_eclass_for_sort_expr(root, expr, nullable_relids,
+ eclass = get_eclass_for_sort_expr(root, expr,
opfamilies, opcintype, collation,
sortref, rel, create_it);
@@ -257,7 +253,6 @@ make_pathkey_from_sortinfo(PlannerInfo *root,
static PathKey *
make_pathkey_from_sortop(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
Oid ordering_op,
bool nulls_first,
Index sortref,
@@ -279,7 +274,6 @@ make_pathkey_from_sortop(PlannerInfo *root,
return make_pathkey_from_sortinfo(root,
expr,
- nullable_relids,
opfamily,
opcintype,
collation,
@@ -584,12 +578,10 @@ build_index_pathkeys(PlannerInfo *root,
}
/*
- * OK, try to make a canonical pathkey for this sort key. Note we're
- * underneath any outer joins, so nullable_relids should be NULL.
+ * OK, try to make a canonical pathkey for this sort key.
*/
cpathkey = make_pathkey_from_sortinfo(root,
indexkey,
- NULL,
index->sortopfamily[i],
index->opcintype[i],
index->indexcollations[i],
@@ -743,14 +735,12 @@ build_partition_pathkeys(PlannerInfo *root, RelOptInfo *partrel,
/*
* Try to make a canonical pathkey for this partkey.
*
- * We're considering a baserel scan, so nullable_relids should be
- * NULL. Also, we assume the PartitionDesc lists any NULL partition
- * last, so we treat the scan like a NULLS LAST index: we have
- * nulls_first for backwards scan only.
+ * We assume the PartitionDesc lists any NULL partition last, so we
+ * treat the scan like a NULLS LAST index: we have nulls_first for
+ * backwards scan only.
*/
cpathkey = make_pathkey_from_sortinfo(root,
keyCol,
- NULL,
partscheme->partopfamily[i],
partscheme->partopcintype[i],
partscheme->partcollation[i],
@@ -799,7 +789,7 @@ build_partition_pathkeys(PlannerInfo *root, RelOptInfo *partrel,
* Build a pathkeys list that describes an ordering by a single expression
* using the given sort operator.
*
- * expr, nullable_relids, and rel are as for make_pathkey_from_sortinfo.
+ * expr and rel are as for make_pathkey_from_sortinfo.
* We induce the other arguments assuming default sort order for the operator.
*
* Similarly to make_pathkey_from_sortinfo, the result is NIL if create_it
@@ -808,7 +798,6 @@ build_partition_pathkeys(PlannerInfo *root, RelOptInfo *partrel,
List *
build_expression_pathkey(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
Oid opno,
Relids rel,
bool create_it)
@@ -827,7 +816,6 @@ build_expression_pathkey(PlannerInfo *root,
cpathkey = make_pathkey_from_sortinfo(root,
expr,
- nullable_relids,
opfamily,
opcintype,
exprCollation((Node *) expr),
@@ -908,14 +896,11 @@ convert_subquery_pathkeys(PlannerInfo *root, RelOptInfo *rel,
* expression is *not* volatile in the outer query: it's just
* a Var referencing whatever the subquery emitted. (IOW, the
* outer query isn't going to re-execute the volatile
- * expression itself.) So this is okay. Likewise, it's
- * correct to pass nullable_relids = NULL, because we're
- * underneath any outer joins appearing in the outer query.
+ * expression itself.) So this is okay.
*/
outer_ec =
get_eclass_for_sort_expr(root,
(Expr *) outer_var,
- NULL,
sub_eclass->ec_opfamilies,
sub_member->em_datatype,
sub_eclass->ec_collation,
@@ -997,7 +982,6 @@ convert_subquery_pathkeys(PlannerInfo *root, RelOptInfo *rel,
/* See if we have a matching EC for the TLE */
outer_ec = get_eclass_for_sort_expr(root,
(Expr *) outer_var,
- NULL,
sub_eclass->ec_opfamilies,
sub_expr_type,
sub_expr_coll,
@@ -1138,13 +1122,6 @@ build_join_pathkeys(PlannerInfo *root,
* The resulting PathKeys are always in canonical form. (Actually, there
* is no longer any code anywhere that creates non-canonical PathKeys.)
*
- * We assume that root->nullable_baserels is the set of base relids that could
- * have gone to NULL below the SortGroupClause expressions. This is okay if
- * the expressions came from the query's top level (ORDER BY, DISTINCT, etc)
- * and if this function is only invoked after deconstruct_jointree. In the
- * future we might have to make callers pass in the appropriate
- * nullable-relids set, but for now it seems unnecessary.
- *
* 'sortclauses' is a list of SortGroupClause nodes
* 'tlist' is the targetlist to find the referenced tlist entries in
*/
@@ -1210,7 +1187,6 @@ make_pathkeys_for_sortclauses_extended(PlannerInfo *root,
}
pathkey = make_pathkey_from_sortop(root,
sortkey,
- root->nullable_baserels,
sortcl->sortop,
sortcl->nulls_first,
sortcl->tleSortGroupRef,
@@ -1268,7 +1244,6 @@ initialize_mergeclause_eclasses(PlannerInfo *root, RestrictInfo *restrictinfo)
restrictinfo->left_ec =
get_eclass_for_sort_expr(root,
(Expr *) get_leftop(clause),
- restrictinfo->nullable_relids,
restrictinfo->mergeopfamilies,
lefttype,
((OpExpr *) clause)->inputcollid,
@@ -1278,7 +1253,6 @@ initialize_mergeclause_eclasses(PlannerInfo *root, RestrictInfo *restrictinfo)
restrictinfo->right_ec =
get_eclass_for_sort_expr(root,
(Expr *) get_rightop(clause),
- restrictinfo->nullable_relids,
restrictinfo->mergeopfamilies,
righttype,
((OpExpr *) clause)->inputcollid,
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index c285806a81..934432b9c0 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -131,7 +131,7 @@ static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
List **postponed_qual_list,
List **postponed_oj_qual_list);
static bool check_outerjoin_delay(PlannerInfo *root, Relids *relids_p,
- Relids *nullable_relids_p, bool is_pushed_down);
+ bool is_pushed_down);
static bool check_equivalence_delay(PlannerInfo *root,
RestrictInfo *restrictinfo);
static bool check_redundant_nullability_qual(PlannerInfo *root, Node *clause);
@@ -772,7 +772,6 @@ deconstruct_jointree(PlannerInfo *root)
/* These are filled as we scan the jointree */
root->all_baserels = NULL;
root->outer_join_rels = NULL;
- root->nullable_baserels = NULL;
/* Perform the initial scan of the jointree */
result = deconstruct_recurse(root, (Node *) root->parse->jointree,
@@ -928,7 +927,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
- Relids nullable_rels;
JoinTreeItem *left_item,
*right_item;
List *leftjoinlist,
@@ -954,8 +952,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
jtitem->right_rels = right_item->qualscope;
/* Inner join adds no restrictions for quals */
jtitem->nonnullable_rels = NULL;
- /* and it doesn't force anything to null, either */
- nullable_rels = NULL;
break;
case JOIN_LEFT:
case JOIN_ANTI:
@@ -986,7 +982,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
jtitem->left_rels = left_item->qualscope;
jtitem->right_rels = right_item->qualscope;
jtitem->nonnullable_rels = left_item->qualscope;
- nullable_rels = right_item->qualscope;
break;
case JOIN_SEMI:
/* Recurse */
@@ -1009,13 +1004,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
jtitem->right_rels = right_item->qualscope;
/* Semi join adds no restrictions for quals */
jtitem->nonnullable_rels = NULL;
-
- /*
- * Theoretically, a semijoin would null the RHS; but since the
- * RHS can't be accessed above the join, this is immaterial
- * and we needn't account for it.
- */
- nullable_rels = NULL;
break;
case JOIN_FULL:
/* Recurse */
@@ -1045,21 +1033,15 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
jtitem->right_rels = right_item->qualscope;
/* each side is both outer and inner */
jtitem->nonnullable_rels = jtitem->qualscope;
- nullable_rels = jtitem->qualscope;
break;
default:
/* JOIN_RIGHT was eliminated during reduce_outer_joins() */
elog(ERROR, "unrecognized join type: %d",
(int) j->jointype);
leftjoinlist = rightjoinlist = NIL; /* keep compiler quiet */
- nullable_rels = NULL;
break;
}
- /* Report all rels that will be nulled anywhere in the jointree */
- root->nullable_baserels = bms_add_members(root->nullable_baserels,
- nullable_rels);
-
/*
* Compute the output joinlist. We fold subproblems together except
* at a FULL JOIN or where join_collapse_limit would be exceeded.
@@ -2210,7 +2192,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool pseudoconstant = false;
bool maybe_equivalence;
bool maybe_outer_join;
- Relids nullable_relids;
RestrictInfo *restrictinfo;
/*
@@ -2364,7 +2345,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
/* Check to see if must be delayed by lower outer join */
outerjoin_delayed = check_outerjoin_delay(root,
&relids,
- &nullable_relids,
false);
/*
@@ -2392,7 +2372,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
/* Check to see if must be delayed by lower outer join */
outerjoin_delayed = check_outerjoin_delay(root,
&relids,
- &nullable_relids,
true);
if (outerjoin_delayed)
@@ -2452,8 +2431,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
pseudoconstant,
security_level,
relids,
- outerjoin_nonnullable,
- nullable_relids);
+ outerjoin_nonnullable);
/* Apply appropriate clone marking, too */
restrictinfo->has_clone = has_clone;
@@ -2611,9 +2589,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* If the qual must be delayed, add relids to *relids_p to reflect the lowest
* safe level for evaluating the qual, and return true. Any extra delay for
* higher-level joins is reflected by setting delay_upper_joins to true in
- * SpecialJoinInfo structs. We also compute nullable_relids, the set of
- * referenced relids that are nullable by lower outer joins (note that this
- * can be nonempty even for a non-delayed qual).
+ * SpecialJoinInfo structs.
*
* For an is_pushed_down qual, we can evaluate the qual as soon as (1) we have
* all the rels it mentions, and (2) we are at or above any outer joins that
@@ -2636,8 +2612,8 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* mentioning only C cannot be applied below the join to A.
*
* For a non-pushed-down qual, this isn't going to determine where we place the
- * qual, but we need to determine outerjoin_delayed and nullable_relids anyway
- * for use later in the planning process.
+ * qual, but we need to determine outerjoin_delayed anyway for use later in
+ * the planning process.
*
* Lastly, a pushed-down qual that references the nullable side of any current
* join_info_list member and has to be evaluated above that OJ (because its
@@ -2655,24 +2631,18 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
static bool
check_outerjoin_delay(PlannerInfo *root,
Relids *relids_p, /* in/out parameter */
- Relids *nullable_relids_p, /* output parameter */
bool is_pushed_down)
{
Relids relids;
- Relids nullable_relids;
bool outerjoin_delayed;
bool found_some;
/* fast path if no special joins */
if (root->join_info_list == NIL)
- {
- *nullable_relids_p = NULL;
return false;
- }
/* must copy relids because we need the original value at the end */
relids = bms_copy(*relids_p);
- nullable_relids = NULL;
outerjoin_delayed = false;
do
{
@@ -2699,12 +2669,6 @@ check_outerjoin_delay(PlannerInfo *root,
/* we'll need another iteration */
found_some = true;
}
- /* track all the nullable rels of relevant OJs */
- nullable_relids = bms_add_members(nullable_relids,
- sjinfo->min_righthand);
- if (sjinfo->jointype == JOIN_FULL)
- nullable_relids = bms_add_members(nullable_relids,
- sjinfo->min_lefthand);
/* set delay_upper_joins if needed */
if (is_pushed_down && sjinfo->jointype != JOIN_FULL &&
bms_overlap(relids, sjinfo->min_lefthand))
@@ -2713,13 +2677,9 @@ check_outerjoin_delay(PlannerInfo *root,
}
} while (found_some);
- /* identify just the actually-referenced nullable rels */
- nullable_relids = bms_int_members(nullable_relids, *relids_p);
-
- /* replace *relids_p, and return nullable_relids */
+ /* replace *relids_p */
bms_free(*relids_p);
*relids_p = relids;
- *nullable_relids_p = nullable_relids;
return outerjoin_delayed;
}
@@ -2741,7 +2701,6 @@ check_equivalence_delay(PlannerInfo *root,
RestrictInfo *restrictinfo)
{
Relids relids;
- Relids nullable_relids;
/* fast path if no special joins */
if (root->join_info_list == NIL)
@@ -2750,12 +2709,12 @@ check_equivalence_delay(PlannerInfo *root,
/* must copy restrictinfo's relids to avoid changing it */
relids = bms_copy(restrictinfo->left_relids);
/* check left side does not need delay */
- if (check_outerjoin_delay(root, &relids, &nullable_relids, true))
+ if (check_outerjoin_delay(root, &relids, true))
return false;
/* and similarly for the right side */
relids = bms_copy(restrictinfo->right_relids);
- if (check_outerjoin_delay(root, &relids, &nullable_relids, true))
+ if (check_outerjoin_delay(root, &relids, true))
return false;
return true;
@@ -2881,11 +2840,6 @@ distribute_restrictinfo_to_rels(PlannerInfo *root,
* variable-free. Otherwise the qual is applied at the lowest join level
* that provides all its variables.
*
- * "nullable_relids" is the set of relids used in the expressions that are
- * potentially nullable below the expressions. (This has to be supplied by
- * caller because this function is used after deconstruct_jointree, so we
- * don't have knowledge of where the clause items came from.)
- *
* "security_level" is the security level to assign to the new restrictinfo.
*
* "both_const" indicates whether both items are known pseudo-constant;
@@ -2911,7 +2865,6 @@ process_implied_equality(PlannerInfo *root,
Expr *item1,
Expr *item2,
Relids qualscope,
- Relids nullable_relids,
Index security_level,
bool below_outer_join,
bool both_const)
@@ -2995,8 +2948,7 @@ process_implied_equality(PlannerInfo *root,
pseudoconstant,
security_level,
relids,
- NULL, /* outer_relids */
- nullable_relids);
+ NULL); /* outer_relids */
/*
* If it's a join clause, add vars used in the clause to targetlists of
@@ -3061,7 +3013,6 @@ build_implied_join_equality(PlannerInfo *root,
Expr *item1,
Expr *item2,
Relids qualscope,
- Relids nullable_relids,
Index security_level)
{
RestrictInfo *restrictinfo;
@@ -3089,8 +3040,7 @@ build_implied_join_equality(PlannerInfo *root,
false, /* pseudoconstant */
security_level, /* security_level */
qualscope, /* required_relids */
- NULL, /* outer_relids */
- nullable_relids); /* nullable_relids */
+ NULL); /* outer_relids */
/* Set mergejoinability/hashjoinability flags */
check_mergejoinable(restrictinfo);
diff --git a/src/backend/optimizer/util/appendinfo.c b/src/backend/optimizer/util/appendinfo.c
index dba895ccc4..d5869e0c1c 100644
--- a/src/backend/optimizer/util/appendinfo.c
+++ b/src/backend/optimizer/util/appendinfo.c
@@ -448,9 +448,6 @@ adjust_appendrel_attrs_mutator(Node *node,
newinfo->outer_relids = adjust_child_relids(oldinfo->outer_relids,
context->nappinfos,
context->appinfos);
- newinfo->nullable_relids = adjust_child_relids(oldinfo->nullable_relids,
- context->nappinfos,
- context->appinfos);
newinfo->left_relids = adjust_child_relids(oldinfo->left_relids,
context->nappinfos,
context->appinfos);
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index bf3ea26cf4..7842baea49 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -897,7 +897,7 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
rinfo->outerjoin_delayed,
pseudoconstant,
rinfo->security_level,
- NULL, NULL, NULL));
+ NULL, NULL));
/* track minimum security level among child quals */
cq_min_security = Min(cq_min_security, rinfo->security_level);
}
@@ -932,7 +932,7 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
make_restrictinfo(root, qual,
true, false, false,
security_level,
- NULL, NULL, NULL));
+ NULL, NULL));
cq_min_security = Min(cq_min_security, security_level);
}
security_level++;
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index abc994dbf2..5c4a485d9f 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -275,7 +275,6 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
false,
join_or_rinfo->security_level,
NULL,
- NULL,
NULL);
/*
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index e3b43a7cf3..87ad2d0ab5 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -29,8 +29,7 @@ static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids);
+ Relids outer_relids);
static Expr *make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
bool is_pushed_down,
@@ -38,8 +37,7 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids);
+ Relids outer_relids);
/*
@@ -49,7 +47,7 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
*
* The is_pushed_down, outerjoin_delayed, and pseudoconstant flags for the
* RestrictInfo must be supplied by the caller, as well as the correct values
- * for security_level, outer_relids, and nullable_relids.
+ * for security_level and outer_relids.
* required_relids can be NULL, in which case it defaults to the actual clause
* contents (i.e., clause_relids).
*
@@ -69,8 +67,7 @@ make_restrictinfo(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids)
+ Relids outer_relids)
{
/*
* If it's an OR clause, build a modified copy with RestrictInfos inserted
@@ -84,8 +81,7 @@ make_restrictinfo(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids);
+ outer_relids);
/* Shouldn't be an AND clause, else AND/OR flattening messed up */
Assert(!is_andclause(clause));
@@ -98,8 +94,7 @@ make_restrictinfo(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids);
+ outer_relids);
}
/*
@@ -116,8 +111,7 @@ make_restrictinfo_internal(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids)
+ Relids outer_relids)
{
RestrictInfo *restrictinfo = makeNode(RestrictInfo);
Relids baserels;
@@ -132,7 +126,6 @@ make_restrictinfo_internal(PlannerInfo *root,
restrictinfo->can_join = false; /* may get set below */
restrictinfo->security_level = security_level;
restrictinfo->outer_relids = outer_relids;
- restrictinfo->nullable_relids = nullable_relids;
/*
* If it's potentially delayable by lower-level security quals, figure out
@@ -260,7 +253,7 @@ make_restrictinfo_internal(PlannerInfo *root,
*
* The same is_pushed_down, outerjoin_delayed, and pseudoconstant flag
* values can be applied to all RestrictInfo nodes in the result. Likewise
- * for security_level, outer_relids, and nullable_relids.
+ * for security_level and outer_relids.
*
* The given required_relids are attached to our top-level output,
* but any OR-clause constituents are allowed to default to just the
@@ -274,8 +267,7 @@ make_sub_restrictinfos(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids)
+ Relids outer_relids)
{
if (is_orclause(clause))
{
@@ -291,8 +283,7 @@ make_sub_restrictinfos(PlannerInfo *root,
pseudoconstant,
security_level,
NULL,
- outer_relids,
- nullable_relids));
+ outer_relids));
return (Expr *) make_restrictinfo_internal(root,
clause,
make_orclause(orlist),
@@ -301,8 +292,7 @@ make_sub_restrictinfos(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids);
+ outer_relids);
}
else if (is_andclause(clause))
{
@@ -318,8 +308,7 @@ make_sub_restrictinfos(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids));
+ outer_relids));
return make_andclause(andlist);
}
else
@@ -331,8 +320,7 @@ make_sub_restrictinfos(PlannerInfo *root,
pseudoconstant,
security_level,
required_relids,
- outer_relids,
- nullable_relids);
+ outer_relids);
}
/*
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index cbe99ca397..522b2c08cc 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -268,14 +268,6 @@ struct PlannerInfo
*/
Relids all_query_rels;
- /*
- * nullable_baserels is a Relids set of base relids that are nullable by
- * some outer join in the jointree; these are rels that are potentially
- * nullable below the WHERE clause, SELECT targetlist, etc. This is
- * computed in deconstruct_jointree.
- */
- Relids nullable_baserels;
-
/*
* join_rel_list is a list of all join-relation RelOptInfos we have
* considered in this planning run. For small problems we just scan the
@@ -1392,7 +1384,6 @@ typedef struct EquivalenceMember
Expr *em_expr; /* the expression represented */
Relids em_relids; /* all relids appearing in em_expr */
- Relids em_nullable_relids; /* nullable by lower outer joins */
bool em_is_const; /* expression is pseudoconstant? */
bool em_is_child; /* derived version for a child relation? */
Oid em_datatype; /* the "nominal type" used by the opfamily */
@@ -2418,9 +2409,7 @@ typedef struct LimitPath
* in parameterized scans, since pushing it into the join's outer side would
* lead to wrong answers.)
*
- * There is also a nullable_relids field, which is the set of rels the clause
- * references that can be forced null by some outer join below the clause.
- *
+ * XXX this comment needs work, if we don't remove it completely:
* outerjoin_delayed = true is subtly different from nullable_relids != NULL:
* a clause might reference some nullable rels and yet not be
* outerjoin_delayed because it also references all the other rels of the
@@ -2534,9 +2523,6 @@ typedef struct RestrictInfo
/* If an outer-join clause, the outer-side relations, else NULL: */
Relids outer_relids;
- /* The relids used in the clause that are nullable by lower outer joins: */
- Relids nullable_relids;
-
/*
* Relids in the left/right side of the clause. These fields are set for
* any binary opclause.
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 9b38627efd..1b02a1dc08 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -128,7 +128,6 @@ extern Expr *canonicalize_ec_expression(Expr *expr,
extern void reconsider_outer_join_clauses(PlannerInfo *root);
extern EquivalenceClass *get_eclass_for_sort_expr(PlannerInfo *root,
Expr *expr,
- Relids nullable_relids,
List *opfamilies,
Oid opcintype,
Oid collation,
@@ -216,7 +215,7 @@ extern List *build_index_pathkeys(PlannerInfo *root, IndexOptInfo *index,
extern List *build_partition_pathkeys(PlannerInfo *root, RelOptInfo *partrel,
ScanDirection scandir, bool *partialkeys);
extern List *build_expression_pathkey(PlannerInfo *root, Expr *expr,
- Relids nullable_relids, Oid opno,
+ Oid opno,
Relids rel, bool create_it);
extern List *convert_subquery_pathkeys(PlannerInfo *root, RelOptInfo *rel,
List *subquery_pathkeys,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index 95ecefdade..3e6e60f549 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -83,7 +83,6 @@ extern RestrictInfo *process_implied_equality(PlannerInfo *root,
Expr *item1,
Expr *item2,
Relids qualscope,
- Relids nullable_relids,
Index security_level,
bool below_outer_join,
bool both_const);
@@ -93,7 +92,6 @@ extern RestrictInfo *build_implied_join_equality(PlannerInfo *root,
Expr *item1,
Expr *item2,
Relids qualscope,
- Relids nullable_relids,
Index security_level);
extern void match_foreign_keys_to_quals(PlannerInfo *root);
diff --git a/src/include/optimizer/restrictinfo.h b/src/include/optimizer/restrictinfo.h
index c79bb420e4..f17c341c4b 100644
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -19,7 +19,7 @@
/* Convenience macro for the common case of a valid-everywhere qual */
#define make_simple_restrictinfo(root, clause) \
- make_restrictinfo(root, clause, true, false, false, 0, NULL, NULL, NULL)
+ make_restrictinfo(root, clause, true, false, false, 0, NULL, NULL)
extern RestrictInfo *make_restrictinfo(PlannerInfo *root,
Expr *clause,
@@ -28,8 +28,7 @@ extern RestrictInfo *make_restrictinfo(PlannerInfo *root,
bool pseudoconstant,
Index security_level,
Relids required_relids,
- Relids outer_relids,
- Relids nullable_relids);
+ Relids outer_relids);
extern RestrictInfo *commute_restrictinfo(RestrictInfo *rinfo, Oid comm_op);
extern bool restriction_is_or_clause(RestrictInfo *restrictinfo);
extern bool restriction_is_securely_promotable(RestrictInfo *restrictinfo,
commit 50343c8060dc19b5dc7dc24bca5dcbefc31c0f9a
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Jan 23 15:10:58 2023 -0500
Use constant TRUE for "dummy" clauses when throwing back outer joins.
This improves on a hack I introduced in commit 6a6522529. If we
have a left-join clause l.x = r.y, and a WHERE clause l.x = constant,
we generate r.y = constant and then don't really have a need for the
join clause. Currently we throw the join clause back anyway after
marking it redundant, so that the join search heuristics won't think
this is a clauseless join and avoid it. That was a kluge introduced
under time pressure, and after looking at it I thought of a better
way: let's just introduce constant-TRUE "join clauses" instead,
and get rid of them at the end.
This improves the generated plans for such cases by not having to
test a redundant join clause. We can also get rid of the ugly hack
used to mark such clauses as redundant for selectivity estimation.
The code added here should go away someday, if we can ever handle
outer-join clauses as normal eclasses. But the selectivity
simplifications are good cleanup in any case.
diff --git a/src/backend/optimizer/path/clausesel.c b/src/backend/optimizer/path/clausesel.c
index 61db6ad951..435438a173 100644
--- a/src/backend/optimizer/path/clausesel.c
+++ b/src/backend/optimizer/path/clausesel.c
@@ -715,12 +715,6 @@ clause_selectivity_ext(PlannerInfo *root,
return (Selectivity) 1.0;
}
- /*
- * If the clause is marked redundant, always return 1.0.
- */
- if (rinfo->norm_selec > 1)
- return (Selectivity) 1.0;
-
/*
* If possible, cache the result of the selectivity calculation for
* the clause. We can cache if varRelid is zero or the clause
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 6f81cc2b08..f0eadbcfb9 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -1954,14 +1954,11 @@ create_join_clause(PlannerInfo *root,
* If we don't find any match for a set-aside outer join clause, we must
* throw it back into the regular joinclause processing by passing it to
* distribute_restrictinfo_to_rels(). If we do generate a derived clause,
- * however, the outer-join clause is redundant. We still throw it back,
- * because otherwise the join will be seen as a clauseless join and avoided
- * during join order searching; but we mark it as redundant to keep from
- * messing up the joinrel's size estimate. (This behavior means that the
- * API for this routine is uselessly complex: we could have just put all
- * the clauses into the regular processing initially. We keep it because
- * someday we might want to do something else, such as inserting "dummy"
- * joinclauses instead of real ones.)
+ * however, the outer-join clause is redundant. We must still put some
+ * clause into the regular processing, because otherwise the join will be
+ * seen as a clauseless join and avoided during join order searching.
+ * We handle this by generating a constant-TRUE clause that is marked with
+ * required_relids that make it a join between the correct relations.
*
* Outer join clauses that are marked outerjoin_delayed are special: this
* condition means that one or both VARs might go to null due to a lower
@@ -1996,10 +1993,15 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* remove it from the list */
root->left_join_clauses =
foreach_delete_current(root->left_join_clauses, cell);
- /* we throw it back anyway (see notes above) */
- /* but the thrown-back clause has no extra selectivity */
- rinfo->norm_selec = 2.0;
- rinfo->outer_selec = 1.0;
+ /* throw back a dummy replacement clause (see notes above) */
+ rinfo = make_restrictinfo(root,
+ (Expr *) makeBoolConst(true, false),
+ true, /* is_pushed_down */
+ false, /* outerjoin_delayed */
+ false, /* pseudoconstant */
+ 0, /* security_level */
+ rinfo->required_relids,
+ rinfo->outer_relids);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
@@ -2017,10 +2019,15 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* remove it from the list */
root->right_join_clauses =
foreach_delete_current(root->right_join_clauses, cell);
- /* we throw it back anyway (see notes above) */
- /* but the thrown-back clause has no extra selectivity */
- rinfo->norm_selec = 2.0;
- rinfo->outer_selec = 1.0;
+ /* throw back a dummy replacement clause (see notes above) */
+ rinfo = make_restrictinfo(root,
+ (Expr *) makeBoolConst(true, false),
+ true, /* is_pushed_down */
+ false, /* outerjoin_delayed */
+ false, /* pseudoconstant */
+ 0, /* security_level */
+ rinfo->required_relids,
+ rinfo->outer_relids);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
@@ -2038,10 +2045,15 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* remove it from the list */
root->full_join_clauses =
foreach_delete_current(root->full_join_clauses, cell);
- /* we throw it back anyway (see notes above) */
- /* but the thrown-back clause has no extra selectivity */
- rinfo->norm_selec = 2.0;
- rinfo->outer_selec = 1.0;
+ /* throw back a dummy replacement clause (see notes above) */
+ rinfo = make_restrictinfo(root,
+ (Expr *) makeBoolConst(true, false),
+ true, /* is_pushed_down */
+ false, /* outerjoin_delayed */
+ false, /* pseudoconstant */
+ 0, /* security_level */
+ rinfo->required_relids,
+ rinfo->outer_relids);
distribute_restrictinfo_to_rels(root, rinfo);
}
}
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index 5c4a485d9f..e8972cd759 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -98,18 +98,13 @@ extract_restriction_or_clauses(PlannerInfo *root)
* joinclause that is considered safe to move to this rel by the
* parameterized-path machinery, even though what we are going to do
* with it is not exactly a parameterized path.
- *
- * However, it seems best to ignore clauses that have been marked
- * redundant (by setting norm_selec > 1). That likely can't happen
- * for OR clauses, but let's be safe.
*/
foreach(lc, rel->joininfo)
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
if (restriction_is_or_clause(rinfo) &&
- join_clause_is_movable_to(rinfo, rel) &&
- rinfo->norm_selec <= 1)
+ join_clause_is_movable_to(rinfo, rel))
{
/* Try to extract a qual for this rel only */
Expr *orclause = extract_or_clause(rinfo, rel);
@@ -355,7 +350,7 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
/* And hack cached selectivity so join size remains the same */
join_or_rinfo->norm_selec = orig_selec / or_selec;
- /* ensure result stays in sane range, in particular not "redundant" */
+ /* ensure result stays in sane range */
if (join_or_rinfo->norm_selec > 1)
join_or_rinfo->norm_selec = 1;
/* as explained above, we don't touch outer_selec */
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index 87ad2d0ab5..aafd74c7d2 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -424,6 +424,21 @@ restriction_is_securely_promotable(RestrictInfo *restrictinfo,
return false;
}
+/*
+ * Detect whether a RestrictInfo's clause is constant TRUE (note that it's
+ * surely of type boolean). No such WHERE clause could survive qual
+ * canonicalization, but equivclass.c may generate such RestrictInfos for
+ * reasons discussed therein. We should drop them again when creating
+ * the finished plan, which is handled by the next few functions.
+ */
+static inline bool
+rinfo_is_constant_true(RestrictInfo *rinfo)
+{
+ return IsA(rinfo->clause, Const) &&
+ !((Const *) rinfo->clause)->constisnull &&
+ DatumGetBool(((Const *) rinfo->clause)->constvalue);
+}
+
/*
* get_actual_clauses
*
@@ -443,6 +458,7 @@ get_actual_clauses(List *restrictinfo_list)
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
Assert(!rinfo->pseudoconstant);
+ Assert(!rinfo_is_constant_true(rinfo));
result = lappend(result, rinfo->clause);
}
@@ -454,6 +470,7 @@ get_actual_clauses(List *restrictinfo_list)
*
* Extract bare clauses from 'restrictinfo_list', returning either the
* regular ones or the pseudoconstant ones per 'pseudoconstant'.
+ * Constant-TRUE clauses are dropped in any case.
*/
List *
extract_actual_clauses(List *restrictinfo_list,
@@ -466,7 +483,8 @@ extract_actual_clauses(List *restrictinfo_list,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (rinfo->pseudoconstant == pseudoconstant)
+ if (rinfo->pseudoconstant == pseudoconstant &&
+ !rinfo_is_constant_true(rinfo))
result = lappend(result, rinfo->clause);
}
return result;
@@ -477,7 +495,7 @@ extract_actual_clauses(List *restrictinfo_list,
*
* Extract bare clauses from 'restrictinfo_list', separating those that
* semantically match the join level from those that were pushed down.
- * Pseudoconstant clauses are excluded from the results.
+ * Pseudoconstant and constant-TRUE clauses are excluded from the results.
*
* This is only used at outer joins, since for plain joins we don't care
* about pushed-down-ness.
@@ -499,13 +517,15 @@ extract_actual_join_clauses(List *restrictinfo_list,
if (RINFO_IS_PUSHED_DOWN(rinfo, joinrelids))
{
- if (!rinfo->pseudoconstant)
+ if (!rinfo->pseudoconstant &&
+ !rinfo_is_constant_true(rinfo))
*otherquals = lappend(*otherquals, rinfo->clause);
}
else
{
/* joinquals shouldn't have been marked pseudoconstant */
Assert(!rinfo->pseudoconstant);
+ Assert(!rinfo_is_constant_true(rinfo));
*joinquals = lappend(*joinquals, rinfo->clause);
}
}
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 522b2c08cc..38a6f1adc8 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2568,10 +2568,7 @@ typedef struct RestrictInfo
/* eval cost of clause; -1 if not yet set */
QualCost eval_cost pg_node_attr(equal_ignore);
- /*
- * selectivity for "normal" (JOIN_INNER) semantics; -1 if not yet set; >1
- * means a redundant clause
- */
+ /* selectivity for "normal" (JOIN_INNER) semantics; -1 if not yet set */
Selectivity norm_selec pg_node_attr(equal_ignore);
/* selectivity for outer join semantics; -1 if not yet set */
Selectivity outer_selec pg_node_attr(equal_ignore);
diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out
index 360d87e1ed..6338d080d9 100644
--- a/src/test/regress/expected/join.out
+++ b/src/test/regress/expected/join.out
@@ -4064,8 +4064,8 @@ explain (costs off)
select a.unique1, b.unique1, c.unique1, coalesce(b.twothousand, a.twothousand)
from tenk1 a left join tenk1 b on b.thousand = a.unique1 left join tenk1 c on c.unique2 =
coalesce(b.twothousand,a.twothousand)
where a.unique2 < 10 and coalesce(b.twothousand, a.twothousand) = 44;
- QUERY PLAN
----------------------------------------------------------------------------------------------
+ QUERY PLAN
+---------------------------------------------------------------
Nested Loop Left Join
-> Nested Loop Left Join
Filter: (COALESCE(b.twothousand, a.twothousand) = 44)
@@ -4076,7 +4076,7 @@ select a.unique1, b.unique1, c.unique1, coalesce(b.twothousand, a.twothousand)
-> Bitmap Index Scan on tenk1_thous_tenthous
Index Cond: (thousand = a.unique1)
-> Index Scan using tenk1_unique2 on tenk1 c
- Index Cond: ((unique2 = COALESCE(b.twothousand, a.twothousand)) AND (unique2 = 44))
+ Index Cond: (unique2 = 44)
(11 rows)
select a.unique1, b.unique1, c.unique1, coalesce(b.twothousand, a.twothousand)
@@ -4507,7 +4507,6 @@ where tt1.f1 = ss1.c0;
Output: tt4.f1
-> Nested Loop Left Join
Output: tt4.f1
- Join Filter: (tt3.f1 = tt4.f1)
-> Seq Scan on public.text_tbl tt3
Output: tt3.f1
Filter: (tt3.f1 = 'foo'::text)
@@ -4525,7 +4524,7 @@ where tt1.f1 = ss1.c0;
Output: (tt4.f1)
-> Seq Scan on public.text_tbl tt5
Output: tt4.f1
-(33 rows)
+(32 rows)
select 1 from
text_tbl as tt1
@@ -4632,24 +4631,22 @@ explain (costs off)
QUERY PLAN
-------------------------------------------------
Nested Loop Left Join
- Join Filter: (a.f1 = b.unique2)
-> Seq Scan on int4_tbl a
Filter: (f1 = 0)
-> Index Scan using tenk1_unique2 on tenk1 b
Index Cond: (unique2 = 0)
-(6 rows)
+(5 rows)
explain (costs off)
select * from tenk1 a full join tenk1 b using(unique2) where unique2 = 42;
QUERY PLAN
-------------------------------------------------
Merge Full Join
- Merge Cond: (a.unique2 = b.unique2)
-> Index Scan using tenk1_unique2 on tenk1 a
Index Cond: (unique2 = 42)
-> Index Scan using tenk1_unique2 on tenk1 b
Index Cond: (unique2 = 42)
-(6 rows)
+(5 rows)
--
-- test that quals attached to an outer join have correct semantics,
commit 4b851dd9acb73943d3d07220d7bf34f152b44dc1
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Jan 23 15:17:10 2023 -0500
Teach remove_useless_result_rtes to also remove useless FromExprs.
If a FromExpr has but one child, we can get rid of it even if it
has some quals, if we can move those quals up to the parent jointree
node. In particular, this works for a FromExpr just below the RHS
of a LEFT JOIN, because the quals of such a FromExpr are equivalent
to degenerate quals of the left join. By recursively applying this
rule, we can guarantee that the RHS child of a left join is either
not a FromExpr or has more than one child.
The point of this transformation is not so much to simplify the join
tree as it is to eliminate the problem for which commit 11086f2f2
invented the delay_upper_joins mechanism. The idea of that was
to ensure that quals in such a FromExpr would block outer join
rearrangement as if they were degenerate quals of the upper join.
But after this transformation, they *are* degenerate quals of the
upper join, and we don't need an indirect mechanism any more.
(Note that a multi-child FromExpr isn't a problem, because that must
represent an inner join, which can't commute with the outer join
anyway.)
This results in one visible change in regression outputs: in one test
case, there is now an explicit constant-false join filter condition,
which got there by being pushed up from a removed FromExpr. We still
detect that the child plan is dummy, but now that happens because
populate_joinrel_with_paths pushes the knowledge back down after
noticing that the parent left join has a constant-false join
condition. Since it's such a hokey case, I'm not concerned about
the extra qual condition, and see no point in making an effort to
eliminate it.
Having done that, the delay_upper_joins flag serves no purpose any
more and we can remove it, largely reverting 11086f2f2. (The end
game here is to get rid of check_outerjoin_delay altogether, but
first we must get rid of its side effects.) Although this patch
doesn't make any net code savings, I think it's still an improvement
because it replaces a fuzzily-defined action-at-a-distance flag
with a simple and provably correct jointree transformation.
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 7d957a47a4..7918bb6f0d 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4787,7 +4787,6 @@ compute_semi_anti_join_factors(PlannerInfo *root,
norm_sjinfo.commute_below = NULL;
/* we don't bother trying to make the remaining fields valid */
norm_sjinfo.lhs_strict = false;
- norm_sjinfo.delay_upper_joins = false;
norm_sjinfo.semi_can_btree = false;
norm_sjinfo.semi_can_hash = false;
norm_sjinfo.semi_operators = NIL;
@@ -4956,7 +4955,6 @@ approx_tuple_count(PlannerInfo *root, JoinPath *path, List *quals)
sjinfo.commute_below = NULL;
/* we don't bother trying to make the remaining fields valid */
sjinfo.lhs_strict = false;
- sjinfo.delay_upper_joins = false;
sjinfo.semi_can_btree = false;
sjinfo.semi_can_hash = false;
sjinfo.semi_operators = NIL;
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 56dd1073c5..d7cb11c851 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -743,7 +743,6 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
sjinfo->commute_below = NULL;
/* we don't bother trying to make the remaining fields valid */
sjinfo->lhs_strict = false;
- sjinfo->delay_upper_joins = false;
sjinfo->semi_can_btree = false;
sjinfo->semi_can_hash = false;
sjinfo->semi_operators = NIL;
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index fbb652e7b0..42da62c1c6 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -170,11 +170,10 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
int attroff;
/*
- * Must be a non-delaying left join to a single baserel, else we aren't
- * going to be able to do anything with it.
+ * Must be a left join to a single baserel, else we aren't going to be
+ * able to do anything with it.
*/
- if (sjinfo->jointype != JOIN_LEFT ||
- sjinfo->delay_upper_joins)
+ if (sjinfo->jointype != JOIN_LEFT)
return false;
if (!bms_get_singleton_member(sjinfo->min_righthand, &innerrelid))
@@ -570,13 +569,10 @@ reduce_unique_semijoins(PlannerInfo *root)
List *restrictlist;
/*
- * Must be a non-delaying semijoin to a single baserel, else we aren't
- * going to be able to do anything with it. (It's probably not
- * possible for delay_upper_joins to be set on a semijoin, but we
- * might as well check.)
+ * Must be a semijoin to a single baserel, else we aren't going to be
+ * able to do anything with it.
*/
- if (sjinfo->jointype != JOIN_SEMI ||
- sjinfo->delay_upper_joins)
+ if (sjinfo->jointype != JOIN_SEMI)
continue;
if (!bms_get_singleton_member(sjinfo->min_righthand, &innerrelid))
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 934432b9c0..c6134bc94e 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -130,8 +130,7 @@ static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool is_clone,
List **postponed_qual_list,
List **postponed_oj_qual_list);
-static bool check_outerjoin_delay(PlannerInfo *root, Relids *relids_p,
- bool is_pushed_down);
+static bool check_outerjoin_delay(PlannerInfo *root, Relids *relids_p);
static bool check_equivalence_delay(PlannerInfo *root,
RestrictInfo *restrictinfo);
static bool check_redundant_nullability_qual(PlannerInfo *root, Node *clause);
@@ -1439,8 +1438,6 @@ make_outerjoininfo(PlannerInfo *root,
sjinfo->commute_above_l = NULL;
sjinfo->commute_above_r = NULL;
sjinfo->commute_below = NULL;
- /* this always starts out false */
- sjinfo->delay_upper_joins = false;
compute_semijoin_info(root, sjinfo, clause);
@@ -1595,17 +1592,6 @@ make_outerjoininfo(PlannerInfo *root,
* Also, we must preserve ordering anyway if we have unsafe PHVs, or
* if either this join or the lower OJ is a semijoin or antijoin.
*
- * Here, we have to consider that "our join condition" includes any
- * clauses that syntactically appeared above the lower OJ and below
- * ours; those are equivalent to degenerate clauses in our OJ and must
- * be treated as such. Such clauses obviously can't reference our
- * LHS, and they must be non-strict for the lower OJ's RHS (else
- * reduce_outer_joins would have reduced the lower OJ to a plain
- * join). Hence the other ways in which we handle clauses within our
- * join condition are not affected by them. The net effect is
- * therefore sufficiently represented by the delay_upper_joins flag
- * saved for us by check_outerjoin_delay.
- *
* When we don't need to preserve ordering, check to see if outer join
* identity 3 applies, and if so, remove the lower OJ's ojrelid from
* our min_righthand so that commutation is allowed.
@@ -1619,7 +1605,7 @@ make_outerjoininfo(PlannerInfo *root,
jointype == JOIN_ANTI ||
otherinfo->jointype == JOIN_SEMI ||
otherinfo->jointype == JOIN_ANTI ||
- !otherinfo->lhs_strict || otherinfo->delay_upper_joins)
+ !otherinfo->lhs_strict)
{
/* Preserve ordering */
min_righthand = bms_add_members(min_righthand,
@@ -2344,8 +2330,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
/* Check to see if must be delayed by lower outer join */
outerjoin_delayed = check_outerjoin_delay(root,
- &relids,
- false);
+ &relids);
/*
* Now force the qual to be evaluated exactly at the level of joining
@@ -2371,8 +2356,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
/* Check to see if must be delayed by lower outer join */
outerjoin_delayed = check_outerjoin_delay(root,
- &relids,
- true);
+ &relids);
if (outerjoin_delayed)
{
@@ -2583,13 +2567,10 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
/*
* check_outerjoin_delay
* Detect whether a qual referencing the given relids must be delayed
- * in application due to the presence of a lower outer join, and/or
- * may force extra delay of higher-level outer joins.
+ * in application due to the presence of a lower outer join.
*
* If the qual must be delayed, add relids to *relids_p to reflect the lowest
- * safe level for evaluating the qual, and return true. Any extra delay for
- * higher-level joins is reflected by setting delay_upper_joins to true in
- * SpecialJoinInfo structs.
+ * safe level for evaluating the qual, and return true.
*
* For an is_pushed_down qual, we can evaluate the qual as soon as (1) we have
* all the rels it mentions, and (2) we are at or above any outer joins that
@@ -2614,24 +2595,10 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* For a non-pushed-down qual, this isn't going to determine where we place the
* qual, but we need to determine outerjoin_delayed anyway for use later in
* the planning process.
- *
- * Lastly, a pushed-down qual that references the nullable side of any current
- * join_info_list member and has to be evaluated above that OJ (because its
- * required relids overlap the LHS too) causes that OJ's delay_upper_joins
- * flag to be set true. This will prevent any higher-level OJs from
- * being interchanged with that OJ, which would result in not having any
- * correct place to evaluate the qual. (The case we care about here is a
- * sub-select WHERE clause within the RHS of some outer join. The WHERE
- * clause must effectively be treated as a degenerate clause of that outer
- * join's condition. Rather than trying to match such clauses with joins
- * directly, we set delay_upper_joins here, and when the upper outer join
- * is processed by make_outerjoininfo, it will refrain from allowing the
- * two OJs to commute.)
*/
static bool
check_outerjoin_delay(PlannerInfo *root,
- Relids *relids_p, /* in/out parameter */
- bool is_pushed_down)
+ Relids *relids_p) /* in/out parameter */
{
Relids relids;
bool outerjoin_delayed;
@@ -2669,10 +2636,6 @@ check_outerjoin_delay(PlannerInfo *root,
/* we'll need another iteration */
found_some = true;
}
- /* set delay_upper_joins if needed */
- if (is_pushed_down && sjinfo->jointype != JOIN_FULL &&
- bms_overlap(relids, sjinfo->min_lefthand))
- sjinfo->delay_upper_joins = true;
}
}
} while (found_some);
@@ -2709,12 +2672,12 @@ check_equivalence_delay(PlannerInfo *root,
/* must copy restrictinfo's relids to avoid changing it */
relids = bms_copy(restrictinfo->left_relids);
/* check left side does not need delay */
- if (check_outerjoin_delay(root, &relids, true))
+ if (check_outerjoin_delay(root, &relids))
return false;
/* and similarly for the right side */
relids = bms_copy(restrictinfo->right_relids);
- if (check_outerjoin_delay(root, &relids, true))
+ if (check_outerjoin_delay(root, &relids))
return false;
return true;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 320caebd87..8674ad674d 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -1044,10 +1044,11 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
/*
* If we have any RTE_RESULT relations, see if they can be deleted from
- * the jointree. This step is most effectively done after we've done
- * expression preprocessing and outer join reduction.
+ * the jointree. We also rely on this processing to flatten single-child
+ * FromExprs underneath outer joins. This step is most effectively done
+ * after we've done expression preprocessing and outer join reduction.
*/
- if (hasResultRTEs)
+ if (hasResultRTEs || hasOuterJoins)
remove_useless_result_rtes(root);
/*
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 9c96a558fc..eacfb66b31 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -130,6 +130,7 @@ static void reduce_outer_joins_pass2(Node *jtnode,
static void report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
int rtindex, Relids relids);
static Node *remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Node **parent_quals,
Relids *dropped_outer_joins);
static int get_result_relid(PlannerInfo *root, Node *jtnode);
static void remove_result_refs(PlannerInfo *root, int varno, Node *newjtloc);
@@ -3085,12 +3086,31 @@ report_reduced_full_join(reduce_outer_joins_pass2_state *state2,
/*
* remove_useless_result_rtes
* Attempt to remove RTE_RESULT RTEs from the join tree.
+ * Also, elide single-child FromExprs where possible.
*
* We can remove RTE_RESULT entries from the join tree using the knowledge
* that RTE_RESULT returns exactly one row and has no output columns. Hence,
* if one is inner-joined to anything else, we can delete it. Optimizations
* are also possible for some outer-join cases, as detailed below.
*
+ * This pass also replaces single-child FromExprs with their child node
+ * where possible. It's appropriate to do that here and not earlier because
+ * RTE_RESULT removal might reduce a multiple-child FromExpr to have only one
+ * child. We can remove such a FromExpr if its quals are empty, or if it's
+ * semantically valid to merge the quals into those of the parent node.
+ * While removing unnecessary join tree nodes has some micro-efficiency value,
+ * the real reason to do this is to eliminate cases where the nullable side of
+ * an outer join node is a FromExpr whose single child is another outer join.
+ * To correctly determine whether the two outer joins can commute,
+ * deconstruct_jointree() must treat any quals of such a FromExpr as being
+ * degenerate quals of the upper outer join. The best way to do that is to
+ * make them actually *be* quals of the upper join, by dropping the FromExpr
+ * and hoisting the quals up into the upper join's quals. (Note that there is
+ * no hazard when the intermediate FromExpr has multiple children, since then
+ * it represents an inner join that cannot commute with the upper outer join.)
+ * As long as we have to do that, we might as well elide such FromExprs
+ * everywhere.
+ *
* Some of these optimizations depend on recognizing empty (constant-true)
* quals for FromExprs and JoinExprs. That makes it useful to apply this
* optimization pass after expression preprocessing, since that will have
@@ -3131,6 +3151,7 @@ remove_useless_result_rtes(PlannerInfo *root)
root->parse->jointree = (FromExpr *)
remove_useless_results_recurse(root,
(Node *) root->parse->jointree,
+ NULL,
&dropped_outer_joins);
/* We should still have a FromExpr */
Assert(IsA(root->parse->jointree, FromExpr));
@@ -3184,9 +3205,14 @@ remove_useless_result_rtes(PlannerInfo *root)
* This recursively processes the jointree and returns a modified jointree.
* In addition, the RT indexes of any removed outer-join nodes are added to
* *dropped_outer_joins.
+ *
+ * jtnode is the current jointree node. If it could be valid to merge
+ * its quals into those of the parent node, parent_quals should point to
+ * the parent's quals list; otherwise, pass NULL for parent_quals.
*/
static Node *
remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
+ Node **parent_quals,
Relids *dropped_outer_joins)
{
Assert(jtnode != NULL);
@@ -3214,8 +3240,9 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
Node *child = (Node *) lfirst(cell);
int varno;
- /* Recursively transform child ... */
+ /* Recursively transform child, allowing it to push up quals ... */
child = remove_useless_results_recurse(root, child,
+ &f->quals,
dropped_outer_joins);
/* ... and stick it back into the tree */
lfirst(cell) = child;
@@ -3249,25 +3276,54 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
}
/*
- * If we're not at the top of the jointree, it's valid to simplify a
- * degenerate FromExpr into its single child. (At the top, we must
- * keep the FromExpr since Query.jointree is required to point to a
- * FromExpr.)
+ * If the FromExpr now has only one child, see if we can elide it.
+ * This is always valid if there are no quals, except at the top of
+ * the jointree (since Query.jointree is required to point to a
+ * FromExpr). Otherwise, we can do it if we can push the quals up to
+ * the parent node.
+ *
+ * Note: while it would not be terribly hard to generalize this
+ * transformation to merge multi-child FromExprs into their parent
+ * FromExpr, that risks making the parent join too expensive to plan.
+ * We leave it to later processing to decide heuristically whether
+ * that's a good idea. Pulling up a single child is always OK,
+ * however.
*/
- if (f != root->parse->jointree &&
- f->quals == NULL &&
- list_length(f->fromlist) == 1)
+ if (list_length(f->fromlist) == 1 &&
+ f != root->parse->jointree &&
+ (f->quals == NULL || parent_quals != NULL))
+ {
+ /*
+ * Merge any quals up to parent. They should be in implicit-AND
+ * format by now, so we just need to concatenate lists. Put the
+ * child quals at the front, on the grounds that they should
+ * nominally be evaluated earlier.
+ */
+ if (f->quals != NULL)
+ *parent_quals = (Node *)
+ list_concat(castNode(List, f->quals),
+ castNode(List, *parent_quals));
return (Node *) linitial(f->fromlist);
+ }
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
int varno;
- /* First, recurse */
+ /*
+ * First, recurse. We can accept pushed-up FromExpr quals from either
+ * child if the jointype is INNER, and we can accept them from the RHS
+ * child if the jointype is LEFT.
+ */
j->larg = remove_useless_results_recurse(root, j->larg,
+ (j->jointype == JOIN_INNER) ?
+ &j->quals : NULL,
dropped_outer_joins);
j->rarg = remove_useless_results_recurse(root, j->rarg,
+ (j->jointype == JOIN_INNER ||
+ j->jointype == JOIN_LEFT) ?
+ &j->quals : NULL,
dropped_outer_joins);
/* Apply join-type-specific optimization rules */
@@ -3278,9 +3334,9 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
/*
* An inner join is equivalent to a FromExpr, so if either
* side was simplified to an RTE_RESULT rel, we can replace
- * the join with a FromExpr with just the other side; and if
- * the qual is empty (JOIN ON TRUE) then we can omit the
- * FromExpr as well.
+ * the join with a FromExpr with just the other side.
+ * Furthermore, we can elide that FromExpr according to the
+ * same rules as above.
*
* Just as in the FromExpr case, we can't simplify if the
* other input rel references any PHVs that are marked as to
@@ -3295,20 +3351,34 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
!find_dependent_phvs_in_jointree(root, j->rarg, varno))
{
remove_result_refs(root, varno, j->rarg);
- if (j->quals)
+ if (j->quals != NULL && parent_quals == NULL)
jtnode = (Node *)
makeFromExpr(list_make1(j->rarg), j->quals);
else
+ {
+ /* Merge any quals up to parent */
+ if (j->quals != NULL)
+ *parent_quals = (Node *)
+ list_concat(castNode(List, j->quals),
+ castNode(List, *parent_quals));
jtnode = j->rarg;
+ }
}
else if ((varno = get_result_relid(root, j->rarg)) != 0)
{
remove_result_refs(root, varno, j->larg);
- if (j->quals)
+ if (j->quals != NULL && parent_quals == NULL)
jtnode = (Node *)
makeFromExpr(list_make1(j->larg), j->quals);
else
+ {
+ /* Merge any quals up to parent */
+ if (j->quals != NULL)
+ *parent_quals = (Node *)
+ list_concat(castNode(List, j->quals),
+ castNode(List, *parent_quals));
jtnode = j->larg;
+ }
}
break;
case JOIN_LEFT:
@@ -3346,8 +3416,9 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
/*
* We may simplify this case if the RHS is an RTE_RESULT; the
* join qual becomes effectively just a filter qual for the
- * LHS, since we should either return the LHS row or not. For
- * simplicity we inject the filter qual into a new FromExpr.
+ * LHS, since we should either return the LHS row or not. The
+ * filter clause must go into a new FromExpr if we can't push
+ * it up to the parent.
*
* There is a fine point about PHVs that are supposed to be
* evaluated at the RHS. Such PHVs could only appear in the
@@ -3365,11 +3436,18 @@ remove_useless_results_recurse(PlannerInfo *root, Node *jtnode,
{
Assert(j->rtindex == 0);
remove_result_refs(root, varno, j->larg);
- if (j->quals)
+ if (j->quals != NULL && parent_quals == NULL)
jtnode = (Node *)
makeFromExpr(list_make1(j->larg), j->quals);
else
+ {
+ /* Merge any quals up to parent */
+ if (j->quals != NULL)
+ *parent_quals = (Node *)
+ list_concat(castNode(List, j->quals),
+ castNode(List, *parent_quals));
jtnode = j->larg;
+ }
}
break;
case JOIN_FULL:
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index e8972cd759..0c10646f40 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -338,7 +338,6 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
sjinfo.commute_below = NULL;
/* we don't bother trying to make the remaining fields valid */
sjinfo.lhs_strict = false;
- sjinfo.delay_upper_joins = false;
sjinfo.semi_can_btree = false;
sjinfo.semi_can_hash = false;
sjinfo.semi_operators = NIL;
diff --git a/src/backend/optimizer/util/placeholder.c b/src/backend/optimizer/util/placeholder.c
index af10dbd124..8492a583dd 100644
--- a/src/backend/optimizer/util/placeholder.c
+++ b/src/backend/optimizer/util/placeholder.c
@@ -331,10 +331,7 @@ update_placeholder_eval_levels(PlannerInfo *root, SpecialJoinInfo *new_sjinfo)
/*
* Check for delays due to lower outer joins. This is the same logic
- * as in check_outerjoin_delay in initsplan.c, except that we don't
- * have anything to do with the delay_upper_joins flags; delay of
- * upper outer joins will be handled later, based on the eval_at
- * values we compute now.
+ * as in check_outerjoin_delay in initsplan.c.
*/
eval_at = phinfo->ph_eval_at;
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 38a6f1adc8..56df0e92eb 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2774,12 +2774,6 @@ typedef struct PlaceHolderVar
* upper-level outer joins even if it appears in their RHS). We don't bother
* to set lhs_strict for FULL JOINs, however.
*
- * delay_upper_joins is set true if we detect a pushed-down clause that has
- * to be evaluated after this join is formed (because it references the RHS).
- * Any outer joins that have such a clause and this join in their RHS cannot
- * commute with this join, because that would leave noplace to check the
- * pushed-down clause. (We don't track this for FULL JOINs, either.)
- *
* For a semijoin, we also extract the join operators and their RHS arguments
* and set semi_operators, semi_rhs_exprs, semi_can_btree, and semi_can_hash.
* This is done in support of possibly unique-ifying the RHS, so we don't
@@ -2794,8 +2788,8 @@ typedef struct PlaceHolderVar
* not allowed within join_info_list. We also create transient
* SpecialJoinInfos with jointype == JOIN_INNER for outer joins, since for
* cost estimation purposes it is sometimes useful to know the join size under
- * plain innerjoin semantics. Note that lhs_strict, delay_upper_joins, and
- * of course the semi_xxx fields are not set meaningfully within such structs.
+ * plain innerjoin semantics. Note that lhs_strict and the semi_xxx fields
+ * are not set meaningfully within such structs.
*/
#ifndef HAVE_SPECIALJOININFO_TYPEDEF
typedef struct SpecialJoinInfo SpecialJoinInfo;
@@ -2817,7 +2811,6 @@ struct SpecialJoinInfo
Relids commute_above_r; /* commuting OJs above this one, if RHS */
Relids commute_below; /* commuting OJs below this one */
bool lhs_strict; /* joinclause is strict for some LHS rel */
- bool delay_upper_joins; /* can't commute with upper RHS */
/* Remaining fields are set only for JOIN_SEMI jointype: */
bool semi_can_btree; /* true if semi_operators are all btree */
bool semi_can_hash; /* true if semi_operators are all hash */
diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out
index 6338d080d9..d2874c1cad 100644
--- a/src/test/regress/expected/join.out
+++ b/src/test/regress/expected/join.out
@@ -6048,12 +6048,13 @@ select * from int8_tbl i8 left join lateral
--------------------------------------
Nested Loop Left Join
Output: i8.q1, i8.q2, f1, (i8.q2)
+ Join Filter: false
-> Seq Scan on public.int8_tbl i8
Output: i8.q1, i8.q2
-> Result
Output: f1, i8.q2
One-Time Filter: false
-(7 rows)
+(8 rows)
explain (verbose, costs off)
select * from int8_tbl i8 left join lateral
commit 1ea6ac291281a805b5ae70e5835883161179ae1e
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Jan 23 15:22:26 2023 -0500
Remove the outerjoin_delayed mechanism.
We needed this before to prevent quals from getting evaluated below
outer joins that should null some of their vars. Now that we consider
varnullingrels while placing quals, that's taken care of
automatically, so throw the whole thing away.
This results in one cosmetic change in the regression test outputs,
where clauses that were not previously considered to be
EquivalenceClass candidates are now treated as ECs, and the EC
machinery chooses to emit an equivalent but not identical set of
clauses.
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index d95e70dfce..f5926ab89d 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -6522,7 +6522,6 @@ foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
expr,
true,
false,
- false,
root->qual_security_level,
grouped_rel->relids,
NULL);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index f0eadbcfb9..fe82b55445 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -200,7 +200,6 @@ process_equivalence(PlannerInfo *root,
make_restrictinfo(root,
(Expr *) ntest,
restrictinfo->is_pushed_down,
- restrictinfo->outerjoin_delayed,
restrictinfo->pseudoconstant,
restrictinfo->security_level,
NULL,
@@ -1144,11 +1143,8 @@ generate_base_implied_equalities_const(PlannerInfo *root,
{
RestrictInfo *restrictinfo = (RestrictInfo *) linitial(ec->ec_sources);
- if (bms_membership(restrictinfo->required_relids) != BMS_MULTIPLE)
- {
- distribute_restrictinfo_to_rels(root, restrictinfo);
- return;
- }
+ distribute_restrictinfo_to_rels(root, restrictinfo);
+ return;
}
/*
@@ -1959,15 +1955,6 @@ create_join_clause(PlannerInfo *root,
* seen as a clauseless join and avoided during join order searching.
* We handle this by generating a constant-TRUE clause that is marked with
* required_relids that make it a join between the correct relations.
- *
- * Outer join clauses that are marked outerjoin_delayed are special: this
- * condition means that one or both VARs might go to null due to a lower
- * outer join. We can still push a constant through the clause, but only
- * if its operator is strict; and we *have to* throw the clause back into
- * regular joinclause processing. By keeping the strict join clause,
- * we ensure that any null-extended rows that are mistakenly generated due
- * to suppressing rows not matching the constant will be rejected at the
- * upper outer join. (This doesn't work for full-join clauses.)
*/
void
reconsider_outer_join_clauses(PlannerInfo *root)
@@ -1997,7 +1984,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
true, /* is_pushed_down */
- false, /* outerjoin_delayed */
false, /* pseudoconstant */
0, /* security_level */
rinfo->required_relids,
@@ -2023,7 +2009,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
true, /* is_pushed_down */
- false, /* outerjoin_delayed */
false, /* pseudoconstant */
0, /* security_level */
rinfo->required_relids,
@@ -2049,7 +2034,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
true, /* is_pushed_down */
- false, /* outerjoin_delayed */
false, /* pseudoconstant */
0, /* security_level */
rinfo->required_relids,
@@ -2104,10 +2088,6 @@ reconsider_outer_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo,
opno = ((OpExpr *) rinfo->clause)->opno;
collation = ((OpExpr *) rinfo->clause)->inputcollid;
- /* If clause is outerjoin_delayed, operator must be strict */
- if (rinfo->outerjoin_delayed && !op_strict(opno))
- return false;
-
/* Extract needed info from the clause */
op_input_types(opno, &left_type, &right_type);
if (outer_on_left)
@@ -2224,10 +2204,6 @@ reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
right_relids;
ListCell *lc1;
- /* Can't use an outerjoin_delayed clause here */
- if (rinfo->outerjoin_delayed)
- return false;
-
/* Extract needed info from the clause */
Assert(is_opclause(rinfo->clause));
opno = ((OpExpr *) rinfo->clause)->opno;
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index c6134bc94e..0f4163bffd 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -130,9 +130,6 @@ static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
bool is_clone,
List **postponed_qual_list,
List **postponed_oj_qual_list);
-static bool check_outerjoin_delay(PlannerInfo *root, Relids *relids_p);
-static bool check_equivalence_delay(PlannerInfo *root,
- RestrictInfo *restrictinfo);
static bool check_redundant_nullability_qual(PlannerInfo *root, Node *clause);
static void check_mergejoinable(RestrictInfo *restrictinfo);
static void check_hashjoinable(RestrictInfo *restrictinfo);
@@ -738,15 +735,6 @@ create_lateral_join_info(PlannerInfo *root)
* A sub-joinlist represents a subproblem to be planned separately. Currently
* sub-joinlists arise only from FULL OUTER JOIN or when collapsing of
* subproblems is stopped by join_collapse_limit or from_collapse_limit.
- *
- * NOTE: when dealing with inner joins, it is appropriate to let a qual clause
- * be evaluated at the lowest level where all the variables it mentions are
- * available. However, we cannot push a qual down into the nullable side(s)
- * of an outer join since the qual might eliminate matching rows and cause a
- * NULL row to be incorrectly emitted by the join. Therefore, we artificially
- * OR the minimum-relids of such an outer join into the required_relids of
- * clauses appearing above it. This forces those clauses to be delayed until
- * application of the outer join (or maybe even higher in the join tree).
*/
List *
deconstruct_jointree(PlannerInfo *root)
@@ -758,9 +746,8 @@ deconstruct_jointree(PlannerInfo *root)
/*
* After this point, no more PlaceHolderInfos may be made, because
- * make_outerjoininfo and update_placeholder_eval_levels require all
- * active placeholders to be present in root->placeholder_list while we
- * crawl up the join tree.
+ * make_outerjoininfo requires all active placeholders to be present in
+ * root->placeholder_list while we crawl up the join tree.
*/
root->placeholdersFrozen = true;
@@ -798,31 +785,12 @@ deconstruct_jointree(PlannerInfo *root)
*/
if (root->join_info_list)
{
- /*
- * XXX hack: when we call distribute_qual_to_rels to process one of
- * these clauses, neither the owning SpecialJoinInfo nor any later
- * ones can appear in root->join_info_list, else the wrong things will
- * happen. Fake it out by emptying join_info_list and rebuilding it
- * as we go. This works because join_info_list is only appended to
- * during deconstruct_distribute, so we know we are examining
- * SpecialJoinInfos bottom-up, just like the first time. We can get
- * rid of this hack later, after fixing things so that
- * distribute_qual_to_rels doesn't have that requirement about
- * join_info_list.
- */
- root->join_info_list = NIL;
-
foreach(lc, item_list)
{
JoinTreeItem *jtitem = (JoinTreeItem *) lfirst(lc);
if (jtitem->oj_joinclauses != NIL)
deconstruct_distribute_oj_quals(root, item_list, jtitem);
-
- /* XXX Rest of hack: rebuild join_info_list as we go */
- if (jtitem->sjinfo)
- root->join_info_list = lappend(root->join_info_list,
- jtitem->sjinfo);
}
}
@@ -1265,11 +1233,7 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
/* And add the SpecialJoinInfo to join_info_list */
if (sjinfo)
- {
root->join_info_list = lappend(root->join_info_list, sjinfo);
- /* Each time we do that, recheck placeholder eval levels */
- update_placeholder_eval_levels(root, sjinfo);
- }
}
else
{
@@ -2155,8 +2119,8 @@ distribute_quals_to_rels(PlannerInfo *root, List *clauses,
* level, which will be ojscope not necessarily qualscope.
*
* At the time this is called, root->join_info_list must contain entries for
- * all and only those special joins that are syntactically below this qual;
- * in particular, the passed-in SpecialJoinInfo isn't yet in that list.
+ * at least those special joins that are syntactically below this qual.
+ * (We now need that only for detection of redundant IS NULL quals.)
*/
static void
distribute_qual_to_rels(PlannerInfo *root, Node *clause,
@@ -2174,7 +2138,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
{
Relids relids;
bool is_pushed_down;
- bool outerjoin_delayed;
bool pseudoconstant = false;
bool maybe_equivalence;
bool maybe_outer_join;
@@ -2328,19 +2291,12 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
maybe_equivalence = false;
maybe_outer_join = true;
- /* Check to see if must be delayed by lower outer join */
- outerjoin_delayed = check_outerjoin_delay(root,
- &relids);
-
/*
* Now force the qual to be evaluated exactly at the level of joining
* corresponding to the outer join. We cannot let it get pushed down
* into the nonnullable side, since then we'd produce no output rows,
* rather than the intended single null-extended row, for any
* nonnullable-side rows failing the qual.
- *
- * (Do this step after calling check_outerjoin_delay, because that
- * trashes relids.)
*/
Assert(ojscope);
relids = ojscope;
@@ -2354,32 +2310,16 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
*/
is_pushed_down = true;
- /* Check to see if must be delayed by lower outer join */
- outerjoin_delayed = check_outerjoin_delay(root,
- &relids);
-
- if (outerjoin_delayed)
- {
- /* Should still be a subset of current scope ... */
- Assert(root->hasLateralRTEs || bms_is_subset(relids, qualscope));
- Assert(ojscope == NULL || bms_is_subset(relids, ojscope));
-
- /*
- * Because application of the qual will be delayed by outer join,
- * we mustn't assume its vars are equal everywhere.
- */
- maybe_equivalence = false;
+ /*
+ * It's possible that this is an IS NULL clause that's redundant with
+ * a lower antijoin; if so we can just discard it. We need not test
+ * in any of the other cases, because this will only be possible for
+ * pushed-down clauses.
+ */
+ if (check_redundant_nullability_qual(root, clause))
+ return;
- /*
- * It's possible that this is an IS NULL clause that's redundant
- * with a lower antijoin; if so we can just discard it. We need
- * not test in any of the other cases, because this will only be
- * possible for pushed-down, delayed clauses.
- */
- if (check_redundant_nullability_qual(root, clause))
- return;
- }
- else if (!allow_equivalence)
+ if (!allow_equivalence)
{
/* Caller says it mustn't become an equivalence class */
maybe_equivalence = false;
@@ -2387,8 +2327,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
else
{
/*
- * Qual is not delayed by any lower outer-join restriction, so we
- * can consider feeding it to the equivalence machinery. However,
+ * Consider feeding qual to the equivalence machinery. However,
* if it's itself within an outer-join clause, treat it as though
* it appeared below that outer join (note that we can only get
* here when the clause references only nullable-side rels).
@@ -2411,7 +2350,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
restrictinfo = make_restrictinfo(root,
(Expr *) clause,
is_pushed_down,
- outerjoin_delayed,
pseudoconstant,
security_level,
relids,
@@ -2463,6 +2401,8 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
check_mergejoinable(restrictinfo);
/*
+ * XXX rewrite:
+ *
* If it is a true equivalence clause, send it to the EquivalenceClass
* machinery. We do *not* attach it directly to any restriction or join
* lists. The EC code will propagate it to the appropriate places later.
@@ -2498,8 +2438,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
{
if (maybe_equivalence)
{
- if (check_equivalence_delay(root, restrictinfo) &&
- process_equivalence(root, &restrictinfo, below_outer_join))
+ if (process_equivalence(root, &restrictinfo, below_outer_join))
return;
/* EC rejected it, so set left_ec/right_ec the hard way ... */
if (restrictinfo->mergeopfamilies) /* EC might have changed this */
@@ -2564,125 +2503,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
distribute_restrictinfo_to_rels(root, restrictinfo);
}
-/*
- * check_outerjoin_delay
- * Detect whether a qual referencing the given relids must be delayed
- * in application due to the presence of a lower outer join.
- *
- * If the qual must be delayed, add relids to *relids_p to reflect the lowest
- * safe level for evaluating the qual, and return true.
- *
- * For an is_pushed_down qual, we can evaluate the qual as soon as (1) we have
- * all the rels it mentions, and (2) we are at or above any outer joins that
- * can null any of these rels and are below the syntactic location of the
- * given qual. We must enforce (2) because pushing down such a clause below
- * the OJ might cause the OJ to emit null-extended rows that should not have
- * been formed, or that should have been rejected by the clause. (This is
- * only an issue for non-strict quals, since if we can prove a qual mentioning
- * only nullable rels is strict, we'd have reduced the outer join to an inner
- * join in reduce_outer_joins().)
- *
- * To enforce (2), scan the join_info_list and merge the required-relid sets of
- * any such OJs into the clause's own reference list. At the time we are
- * called, the join_info_list contains only outer joins below this qual. We
- * have to repeat the scan until no new relids get added; this ensures that
- * the qual is suitably delayed regardless of the order in which OJs get
- * executed. As an example, if we have one OJ with LHS=A, RHS=B, and one with
- * LHS=B, RHS=C, it is implied that these can be done in either order; if the
- * B/C join is done first then the join to A can null C, so a qual actually
- * mentioning only C cannot be applied below the join to A.
- *
- * For a non-pushed-down qual, this isn't going to determine where we place the
- * qual, but we need to determine outerjoin_delayed anyway for use later in
- * the planning process.
- */
-static bool
-check_outerjoin_delay(PlannerInfo *root,
- Relids *relids_p) /* in/out parameter */
-{
- Relids relids;
- bool outerjoin_delayed;
- bool found_some;
-
- /* fast path if no special joins */
- if (root->join_info_list == NIL)
- return false;
-
- /* must copy relids because we need the original value at the end */
- relids = bms_copy(*relids_p);
- outerjoin_delayed = false;
- do
- {
- ListCell *l;
-
- found_some = false;
- foreach(l, root->join_info_list)
- {
- SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(l);
-
- /* do we reference any nullable rels of this OJ? */
- if (bms_overlap(relids, sjinfo->min_righthand) ||
- (sjinfo->jointype == JOIN_FULL &&
- bms_overlap(relids, sjinfo->min_lefthand)))
- {
- /* yes; have we included all its rels in relids? */
- if (!bms_is_subset(sjinfo->min_lefthand, relids) ||
- !bms_is_subset(sjinfo->min_righthand, relids))
- {
- /* no, so add them in */
- relids = bms_add_members(relids, sjinfo->min_lefthand);
- relids = bms_add_members(relids, sjinfo->min_righthand);
- outerjoin_delayed = true;
- /* we'll need another iteration */
- found_some = true;
- }
- }
- }
- } while (found_some);
-
- /* replace *relids_p */
- bms_free(*relids_p);
- *relids_p = relids;
- return outerjoin_delayed;
-}
-
-/*
- * check_equivalence_delay
- * Detect whether a potential equivalence clause is rendered unsafe
- * by outer-join-delay considerations. Return true if it's safe.
- *
- * The initial tests in distribute_qual_to_rels will consider a mergejoinable
- * clause to be a potential equivalence clause if it is not outerjoin_delayed.
- * But since the point of equivalence processing is that we will recombine the
- * two sides of the clause with others, we have to check that each side
- * satisfies the not-outerjoin_delayed condition on its own; otherwise it might
- * not be safe to evaluate everywhere we could place a derived equivalence
- * condition.
- */
-static bool
-check_equivalence_delay(PlannerInfo *root,
- RestrictInfo *restrictinfo)
-{
- Relids relids;
-
- /* fast path if no special joins */
- if (root->join_info_list == NIL)
- return true;
-
- /* must copy restrictinfo's relids to avoid changing it */
- relids = bms_copy(restrictinfo->left_relids);
- /* check left side does not need delay */
- if (check_outerjoin_delay(root, &relids))
- return false;
-
- /* and similarly for the right side */
- relids = bms_copy(restrictinfo->right_relids);
- if (check_outerjoin_delay(root, &relids))
- return false;
-
- return true;
-}
-
/*
* check_redundant_nullability_qual
* Check to see if the qual is an IS NULL qual that is redundant with
@@ -2697,25 +2517,33 @@ static bool
check_redundant_nullability_qual(PlannerInfo *root, Node *clause)
{
Var *forced_null_var;
- Index forced_null_rel;
ListCell *lc;
/* Check for IS NULL, and identify the Var forced to NULL */
forced_null_var = find_forced_null_var(clause);
if (forced_null_var == NULL)
return false;
- forced_null_rel = forced_null_var->varno;
/*
* If the Var comes from the nullable side of a lower antijoin, the IS
- * NULL condition is necessarily true.
+ * NULL condition is necessarily true. If it's not nulled by anything,
+ * there is no point in searching the join_info_list. Otherwise, we need
+ * to find out whether the nulling rel is an antijoin.
*/
+ if (forced_null_var->varnullingrels == NULL)
+ return false;
+
foreach(lc, root->join_info_list)
{
SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
- if (sjinfo->jointype == JOIN_ANTI &&
- bms_is_member(forced_null_rel, sjinfo->syn_righthand))
+ /*
+ * This test will not succeed if sjinfo->ojrelid is zero, which is
+ * possible for an antijoin that was converted from a semijoin; but in
+ * such a case the Var couldn't have come from its nullable side.
+ */
+ if (sjinfo->jointype == JOIN_ANTI && sjinfo->ojrelid != 0 &&
+ bms_is_member(sjinfo->ojrelid, forced_null_var->varnullingrels))
return true;
}
@@ -2907,7 +2735,6 @@ process_implied_equality(PlannerInfo *root,
restrictinfo = make_restrictinfo(root,
(Expr *) clause,
true, /* is_pushed_down */
- false, /* outerjoin_delayed */
pseudoconstant,
security_level,
relids,
@@ -2999,7 +2826,6 @@ build_implied_join_equality(PlannerInfo *root,
restrictinfo = make_restrictinfo(root,
clause,
true, /* is_pushed_down */
- false, /* outerjoin_delayed */
false, /* pseudoconstant */
security_level, /* security_level */
qualscope, /* required_relids */
@@ -3071,8 +2897,7 @@ match_foreign_keys_to_quals(PlannerInfo *root)
* Note: for simple inner joins, any match should be in an eclass.
* "Loose" quals that syntactically match an FK equality must have
* been rejected for EC status because they are outer-join quals or
- * similar. We can still consider them to match the FK if they are
- * not outerjoin_delayed.
+ * similar. We can still consider them to match the FK.
*/
for (colno = 0; colno < fkinfo->nkeys; colno++)
{
@@ -3107,10 +2932,6 @@ match_foreign_keys_to_quals(PlannerInfo *root)
Var *leftvar;
Var *rightvar;
- /* Ignore outerjoin-delayed clauses */
- if (rinfo->outerjoin_delayed)
- continue;
-
/* Only binary OpExprs are useful for consideration */
if (!IsA(clause, OpExpr) ||
list_length(clause->args) != 2)
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 7842baea49..bae9688e46 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -894,7 +894,6 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
make_restrictinfo(root,
(Expr *) onecq,
rinfo->is_pushed_down,
- rinfo->outerjoin_delayed,
pseudoconstant,
rinfo->security_level,
NULL, NULL));
@@ -930,7 +929,7 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
/* not likely that we'd see constants here, so no check */
childquals = lappend(childquals,
make_restrictinfo(root, qual,
- true, false, false,
+ true, false,
security_level,
NULL, NULL));
cq_min_security = Min(cq_min_security, security_level);
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index 0c10646f40..85ecdfc14f 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -267,7 +267,6 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
orclause,
true,
false,
- false,
join_or_rinfo->security_level,
NULL,
NULL);
diff --git a/src/backend/optimizer/util/placeholder.c b/src/backend/optimizer/util/placeholder.c
index 8492a583dd..9c6cb5eba7 100644
--- a/src/backend/optimizer/util/placeholder.c
+++ b/src/backend/optimizer/util/placeholder.c
@@ -134,7 +134,6 @@ find_placeholder_info(PlannerInfo *root, PlaceHolderVar *phv)
phinfo->ph_eval_at = bms_copy(phv->phrels);
Assert(!bms_is_empty(phinfo->ph_eval_at));
}
- /* ph_eval_at may change later, see update_placeholder_eval_levels */
phinfo->ph_needed = NULL; /* initially it's unused */
/* for the moment, estimate width using just the datatype info */
phinfo->ph_width = get_typavgwidth(exprType((Node *) phv->phexpr),
@@ -284,99 +283,6 @@ find_placeholders_in_expr(PlannerInfo *root, Node *expr)
list_free(vars);
}
-/*
- * update_placeholder_eval_levels
- * Adjust the target evaluation levels for placeholders
- *
- * The initial eval_at level set by find_placeholder_info was the set of
- * rels used in the placeholder's expression (or the whole subselect below
- * the placeholder's syntactic location, if the expr is variable-free).
- * If the query contains any outer joins that can null any of those rels,
- * we must delay evaluation to above those joins.
- *
- * We repeat this operation each time we add another outer join to
- * root->join_info_list. It's somewhat annoying to have to do that, but
- * since we don't have very much information on the placeholders' locations,
- * it's hard to avoid. Each placeholder's eval_at level must be correct
- * by the time it starts to figure in outer-join delay decisions for higher
- * outer joins.
- *
- * In future we might want to put additional policy/heuristics here to
- * try to determine an optimal evaluation level. The current rules will
- * result in evaluation at the lowest possible level. However, pushing a
- * placeholder eval up the tree is likely to further constrain evaluation
- * order for outer joins, so it could easily be counterproductive; and we
- * don't have enough information at this point to make an intelligent choice.
- */
-void
-update_placeholder_eval_levels(PlannerInfo *root, SpecialJoinInfo *new_sjinfo)
-{
- ListCell *lc1;
-
- foreach(lc1, root->placeholder_list)
- {
- PlaceHolderInfo *phinfo = (PlaceHolderInfo *) lfirst(lc1);
- Relids syn_level = phinfo->ph_var->phrels;
- Relids eval_at;
- bool found_some;
- ListCell *lc2;
-
- /*
- * We don't need to do any work on this placeholder unless the
- * newly-added outer join is syntactically beneath its location.
- */
- if (!bms_is_subset(new_sjinfo->syn_lefthand, syn_level) ||
- !bms_is_subset(new_sjinfo->syn_righthand, syn_level))
- continue;
-
- /*
- * Check for delays due to lower outer joins. This is the same logic
- * as in check_outerjoin_delay in initsplan.c.
- */
- eval_at = phinfo->ph_eval_at;
-
- do
- {
- found_some = false;
- foreach(lc2, root->join_info_list)
- {
- SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc2);
-
- /* disregard joins not within the PHV's sub-select */
- if (!bms_is_subset(sjinfo->syn_lefthand, syn_level) ||
- !bms_is_subset(sjinfo->syn_righthand, syn_level))
- continue;
-
- /* do we reference any nullable rels of this OJ? */
- if (bms_overlap(eval_at, sjinfo->min_righthand) ||
- (sjinfo->jointype == JOIN_FULL &&
- bms_overlap(eval_at, sjinfo->min_lefthand)))
- {
- /* yes; have we included all its rels in eval_at? */
- if (!bms_is_subset(sjinfo->min_lefthand, eval_at) ||
- !bms_is_subset(sjinfo->min_righthand, eval_at))
- {
- /* no, so add them in */
- eval_at = bms_add_members(eval_at,
- sjinfo->min_lefthand);
- eval_at = bms_add_members(eval_at,
- sjinfo->min_righthand);
- if (sjinfo->ojrelid)
- eval_at = bms_add_member(eval_at, sjinfo->ojrelid);
- /* we'll need another iteration */
- found_some = true;
- }
- }
- }
- } while (found_some);
-
- /* Can't move the PHV's eval_at level to above its syntactic level */
- Assert(bms_is_subset(eval_at, syn_level));
-
- phinfo->ph_eval_at = eval_at;
- }
-}
-
/*
* fix_placeholder_input_needed_levels
* Adjust the "needed at" levels for placeholder inputs
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index aafd74c7d2..63d41486bc 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -25,7 +25,6 @@ static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
Expr *clause,
Expr *orclause,
bool is_pushed_down,
- bool outerjoin_delayed,
bool pseudoconstant,
Index security_level,
Relids required_relids,
@@ -33,7 +32,6 @@ static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
static Expr *make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
bool is_pushed_down,
- bool outerjoin_delayed,
bool pseudoconstant,
Index security_level,
Relids required_relids,
@@ -45,7 +43,7 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
*
* Build a RestrictInfo node containing the given subexpression.
*
- * The is_pushed_down, outerjoin_delayed, and pseudoconstant flags for the
+ * The is_pushed_down and pseudoconstant flags for the
* RestrictInfo must be supplied by the caller, as well as the correct values
* for security_level and outer_relids.
* required_relids can be NULL, in which case it defaults to the actual clause
@@ -63,7 +61,6 @@ RestrictInfo *
make_restrictinfo(PlannerInfo *root,
Expr *clause,
bool is_pushed_down,
- bool outerjoin_delayed,
bool pseudoconstant,
Index security_level,
Relids required_relids,
@@ -77,7 +74,6 @@ make_restrictinfo(PlannerInfo *root,
return (RestrictInfo *) make_sub_restrictinfos(root,
clause,
is_pushed_down,
- outerjoin_delayed,
pseudoconstant,
security_level,
required_relids,
@@ -90,7 +86,6 @@ make_restrictinfo(PlannerInfo *root,
clause,
NULL,
is_pushed_down,
- outerjoin_delayed,
pseudoconstant,
security_level,
required_relids,
@@ -107,7 +102,6 @@ make_restrictinfo_internal(PlannerInfo *root,
Expr *clause,
Expr *orclause,
bool is_pushed_down,
- bool outerjoin_delayed,
bool pseudoconstant,
Index security_level,
Relids required_relids,
@@ -119,7 +113,6 @@ make_restrictinfo_internal(PlannerInfo *root,
restrictinfo->clause = clause;
restrictinfo->orclause = orclause;
restrictinfo->is_pushed_down = is_pushed_down;
- restrictinfo->outerjoin_delayed = outerjoin_delayed;
restrictinfo->pseudoconstant = pseudoconstant;
restrictinfo->has_clone = false; /* may get set by caller */
restrictinfo->is_clone = false; /* may get set by caller */
@@ -251,7 +244,7 @@ make_restrictinfo_internal(PlannerInfo *root,
* implicit-AND lists at top level of RestrictInfo lists. Only ORs and
* simple clauses are valid RestrictInfos.
*
- * The same is_pushed_down, outerjoin_delayed, and pseudoconstant flag
+ * The same is_pushed_down and pseudoconstant flag
* values can be applied to all RestrictInfo nodes in the result. Likewise
* for security_level and outer_relids.
*
@@ -263,7 +256,6 @@ static Expr *
make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
bool is_pushed_down,
- bool outerjoin_delayed,
bool pseudoconstant,
Index security_level,
Relids required_relids,
@@ -279,7 +271,6 @@ make_sub_restrictinfos(PlannerInfo *root,
make_sub_restrictinfos(root,
lfirst(temp),
is_pushed_down,
- outerjoin_delayed,
pseudoconstant,
security_level,
NULL,
@@ -288,7 +279,6 @@ make_sub_restrictinfos(PlannerInfo *root,
clause,
make_orclause(orlist),
is_pushed_down,
- outerjoin_delayed,
pseudoconstant,
security_level,
required_relids,
@@ -304,7 +294,6 @@ make_sub_restrictinfos(PlannerInfo *root,
make_sub_restrictinfos(root,
lfirst(temp),
is_pushed_down,
- outerjoin_delayed,
pseudoconstant,
security_level,
required_relids,
@@ -316,7 +305,6 @@ make_sub_restrictinfos(PlannerInfo *root,
clause,
NULL,
is_pushed_down,
- outerjoin_delayed,
pseudoconstant,
security_level,
required_relids,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 56df0e92eb..f843659a18 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2398,24 +2398,12 @@ typedef struct LimitPath
* conditions. Possibly we should rename it to reflect that meaning? But
* see also the comments for RINFO_IS_PUSHED_DOWN, below.)
*
- * RestrictInfo nodes also contain an outerjoin_delayed flag, which is true
- * if the clause's applicability must be delayed due to any outer joins
- * appearing below it (ie, it has to be postponed to some join level higher
- * than the set of relations it actually references).
- *
* There is also an outer_relids field, which is NULL except for outer join
* clauses; for those, it is the set of relids on the outer side of the
* clause's outer join. (These are rels that the clause cannot be applied to
* in parameterized scans, since pushing it into the join's outer side would
* lead to wrong answers.)
*
- * XXX this comment needs work, if we don't remove it completely:
- * outerjoin_delayed = true is subtly different from nullable_relids != NULL:
- * a clause might reference some nullable rels and yet not be
- * outerjoin_delayed because it also references all the other rels of the
- * outer join(s). A clause that is not outerjoin_delayed can be enforced
- * anywhere it is computable.
- *
* To handle security-barrier conditions efficiently, we mark RestrictInfo
* nodes with a security_level field, in which higher values identify clauses
* coming from less-trusted sources. The exact semantics are that a clause
@@ -2489,9 +2477,6 @@ typedef struct RestrictInfo
/* true if clause was pushed down in level */
bool is_pushed_down;
- /* true if delayed by lower outer join */
- bool outerjoin_delayed;
-
/* see comment above */
bool can_join pg_node_attr(equal_ignore);
diff --git a/src/include/optimizer/placeholder.h b/src/include/optimizer/placeholder.h
index 31e1578e82..acb9cf9f05 100644
--- a/src/include/optimizer/placeholder.h
+++ b/src/include/optimizer/placeholder.h
@@ -22,8 +22,6 @@ extern PlaceHolderVar *make_placeholder_expr(PlannerInfo *root, Expr *expr,
extern PlaceHolderInfo *find_placeholder_info(PlannerInfo *root,
PlaceHolderVar *phv);
extern void find_placeholders_in_jointree(PlannerInfo *root);
-extern void update_placeholder_eval_levels(PlannerInfo *root,
- SpecialJoinInfo *new_sjinfo);
extern void fix_placeholder_input_needed_levels(PlannerInfo *root);
extern void add_placeholders_to_base_rels(PlannerInfo *root);
extern void add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
diff --git a/src/include/optimizer/restrictinfo.h b/src/include/optimizer/restrictinfo.h
index f17c341c4b..c9e30776c5 100644
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -19,12 +19,11 @@
/* Convenience macro for the common case of a valid-everywhere qual */
#define make_simple_restrictinfo(root, clause) \
- make_restrictinfo(root, clause, true, false, false, 0, NULL, NULL)
+ make_restrictinfo(root, clause, true, false, 0, NULL, NULL)
extern RestrictInfo *make_restrictinfo(PlannerInfo *root,
Expr *clause,
bool is_pushed_down,
- bool outerjoin_delayed,
bool pseudoconstant,
Index security_level,
Relids required_relids,
diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out
index d2874c1cad..806732cee9 100644
--- a/src/test/regress/expected/join.out
+++ b/src/test/regress/expected/join.out
@@ -4020,10 +4020,10 @@ explain (costs off)
select q1, unique2, thousand, hundred
from int8_tbl a left join tenk1 b on q1 = unique2
where coalesce(thousand,123) = q1 and q1 = coalesce(hundred,123);
- QUERY PLAN
---------------------------------------------------------------------------------------
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------
Nested Loop Left Join
- Filter: ((COALESCE(b.thousand, 123) = a.q1) AND (a.q1 = COALESCE(b.hundred, 123)))
+ Filter: ((COALESCE(b.thousand, 123) = COALESCE(b.hundred, 123)) AND (a.q1 = COALESCE(b.hundred, 123)))
-> Seq Scan on int8_tbl a
-> Index Scan using tenk1_unique2 on tenk1 b
Index Cond: (unique2 = a.q1)
commit 40210d3392166d1e277e1948db56829ce38ad1bd
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon Jan 23 15:28:34 2023 -0500
Invent "join domains" to replace the below_outer_join hack.
EquivalenceClasses are now understood as applying within a "join
domain", which is a set of inner-joined relations (possibly underneath
an outer join). We no longer need to treat an EC from below an outer
join as a second-class citizen.
I have hopes of eventually being able to treat outer-join clauses via
EquivalenceClasses, by means of only applying deductions within the
EC's join domain. There are still problems in the way of that, though,
so for now the reconsider_outer_join_clause logic is still here.
I haven't been able to get rid of is_pushed_down either, but I wonder
if that could be recast using JoinDomains.
I had to hack one test case in postgres_fdw.sql to make it still test
what it was meant to, because postgres_fdw is inconsistent about
how it deals with quals containing non-shippable expressions; see
https://postgr.es/m/1691374.1671659838@sss.pgh.pa.us. That should
be improved, but I don't think it's within the scope of this patch
series.
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 2350cfe148..d5fc61446a 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2513,7 +2513,7 @@ SELECT * FROM local_tbl LEFT JOIN (SELECT ft1.*, COALESCE(ft1.c3 || ft2.c3, 'foo
ALTER SERVER loopback OPTIONS (DROP extensions);
ALTER SERVER loopback OPTIONS (ADD fdw_startup_cost '10000.0');
EXPLAIN (VERBOSE, COSTS OFF)
-SELECT * FROM local_tbl LEFT JOIN (SELECT ft1.* FROM ft1 INNER JOIN ft2 ON (ft1.c1 = ft2.c1 AND ft1.c1 < 100 AND
ft1.c1= postgres_fdw_abs(ft2.c2))) ss ON (local_tbl.c3 = ss.c3) ORDER BY local_tbl.c1 FOR UPDATE OF local_tbl;
+SELECT * FROM local_tbl LEFT JOIN (SELECT ft1.* FROM ft1 INNER JOIN ft2 ON (ft1.c1 = ft2.c1 AND ft1.c1 < 100 AND
(ft1.c1- postgres_fdw_abs(ft2.c2)) = 0)) ss ON (local_tbl.c3 = ss.c3) ORDER BY local_tbl.c1 FOR UPDATE OF local_tbl;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
LockRows
@@ -2527,7 +2527,7 @@ SELECT * FROM local_tbl LEFT JOIN (SELECT ft1.* FROM ft1 INNER JOIN ft2 ON (ft1.
Output: ft1.c1, ft1.c2, ft1.c3, ft1.c4, ft1.c5, ft1.c6, ft1.c7, ft1.c8, ft1.*, ft2.*
-> Foreign Scan
Output: ft1.c1, ft1.c2, ft1.c3, ft1.c4, ft1.c5, ft1.c6, ft1.c7, ft1.c8, ft1.*, ft2.*
- Filter: (ft1.c1 = postgres_fdw_abs(ft2.c2))
+ Filter: ((ft1.c1 - postgres_fdw_abs(ft2.c2)) = 0)
Relations: (public.ft1) INNER JOIN (public.ft2)
Remote SQL: SELECT r4."C 1", r4.c2, r4.c3, r4.c4, r4.c5, r4.c6, r4.c7, r4.c8, CASE WHEN
(r4.*)::textIS NOT NULL THEN ROW(r4."C 1", r4.c2, r4.c3, r4.c4, r4.c5, r4.c6, r4.c7, r4.c8) END, CASE WHEN (r5.*)::text
ISNOT NULL THEN ROW(r5."C 1", r5.c2, r5.c3, r5.c4, r5.c5, r5.c6, r5.c7, r5.c8) END, r5.c2 FROM ("S 1"."T 1" r4 INNER
JOIN"S 1"."T 1" r5 ON (((r5."C 1" = r4."C 1")) AND ((r4."C 1" < 100)))) ORDER BY r4.c3 ASC NULLS LAST
-> Sort
@@ -2535,18 +2535,18 @@ SELECT * FROM local_tbl LEFT JOIN (SELECT ft1.* FROM ft1 INNER JOIN ft2 ON (ft1.
Sort Key: ft1.c3
-> Merge Join
Output: ft1.c1, ft1.c2, ft1.c3, ft1.c4, ft1.c5, ft1.c6, ft1.c7, ft1.c8, ft1.*, ft2.*,
ft2.c2
- Merge Cond: ((ft1.c1 = (postgres_fdw_abs(ft2.c2))) AND (ft1.c1 = ft2.c1))
+ Merge Cond: (ft1.c1 = ft2.c1)
+ Join Filter: ((ft1.c1 - postgres_fdw_abs(ft2.c2)) = 0)
-> Sort
Output: ft1.c1, ft1.c2, ft1.c3, ft1.c4, ft1.c5, ft1.c6, ft1.c7, ft1.c8, ft1.*
Sort Key: ft1.c1
-> Foreign Scan on public.ft1
Output: ft1.c1, ft1.c2, ft1.c3, ft1.c4, ft1.c5, ft1.c6, ft1.c7, ft1.c8,
ft1.*
Remote SQL: SELECT "C 1", c2, c3, c4, c5, c6, c7, c8 FROM "S 1"."T 1"
WHERE(("C 1" < 100))
- -> Sort
- Output: ft2.*, ft2.c1, ft2.c2, (postgres_fdw_abs(ft2.c2))
- Sort Key: (postgres_fdw_abs(ft2.c2)), ft2.c1
+ -> Materialize
+ Output: ft2.*, ft2.c1, ft2.c2
-> Foreign Scan on public.ft2
- Output: ft2.*, ft2.c1, ft2.c2, postgres_fdw_abs(ft2.c2)
+ Output: ft2.*, ft2.c1, ft2.c2
Remote SQL: SELECT "C 1", c2, c3, c4, c5, c6, c7, c8 FROM "S 1"."T 1"
ORDERBY "C 1" ASC NULLS LAST
(32 rows)
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index c37aa80383..1e50be137b 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -681,7 +681,7 @@ SELECT * FROM local_tbl LEFT JOIN (SELECT ft1.*, COALESCE(ft1.c3 || ft2.c3, 'foo
ALTER SERVER loopback OPTIONS (DROP extensions);
ALTER SERVER loopback OPTIONS (ADD fdw_startup_cost '10000.0');
EXPLAIN (VERBOSE, COSTS OFF)
-SELECT * FROM local_tbl LEFT JOIN (SELECT ft1.* FROM ft1 INNER JOIN ft2 ON (ft1.c1 = ft2.c1 AND ft1.c1 < 100 AND
ft1.c1= postgres_fdw_abs(ft2.c2))) ss ON (local_tbl.c3 = ss.c3) ORDER BY local_tbl.c1 FOR UPDATE OF local_tbl;
+SELECT * FROM local_tbl LEFT JOIN (SELECT ft1.* FROM ft1 INNER JOIN ft2 ON (ft1.c1 = ft2.c1 AND ft1.c1 < 100 AND
(ft1.c1- postgres_fdw_abs(ft2.c2)) = 0)) ss ON (local_tbl.c3 = ss.c3) ORDER BY local_tbl.c1 FOR UPDATE OF local_tbl;
ALTER SERVER loopback OPTIONS (DROP fdw_startup_cost);
ALTER SERVER loopback OPTIONS (ADD extensions 'postgres_fdw');
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 6b368b08b2..ba00b99249 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -468,7 +468,6 @@ _outEquivalenceClass(StringInfo str, const EquivalenceClass *node)
WRITE_BITMAPSET_FIELD(ec_relids);
WRITE_BOOL_FIELD(ec_has_const);
WRITE_BOOL_FIELD(ec_has_volatile);
- WRITE_BOOL_FIELD(ec_below_outer_join);
WRITE_BOOL_FIELD(ec_broken);
WRITE_UINT_FIELD(ec_sortref);
WRITE_UINT_FIELD(ec_min_security);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index fe82b55445..d4b4a4fc7c 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -35,6 +35,7 @@
static EquivalenceMember *add_eq_member(EquivalenceClass *ec,
Expr *expr, Relids relids,
+ JoinDomain *jdomain,
EquivalenceMember *parent,
Oid datatype);
static bool is_exprlist_member(Expr *node, List *exprs);
@@ -67,6 +68,7 @@ static bool reconsider_outer_join_clause(PlannerInfo *root,
bool outer_on_left);
static bool reconsider_full_join_clause(PlannerInfo *root,
OuterJoinClauseInfo *ojcinfo);
+static JoinDomain *find_join_domain(PlannerInfo *root, Relids relids);
static Bitmapset *get_eclass_indexes_for_relids(PlannerInfo *root,
Relids relids);
static Bitmapset *get_common_eclass_indexes(PlannerInfo *root, Relids relids1,
@@ -75,8 +77,8 @@ static Bitmapset *get_common_eclass_indexes(PlannerInfo *root, Relids relids1,
/*
* process_equivalence
- * The given clause has a mergejoinable operator and can be applied without
- * any delay by an outer join, so its two sides can be considered equal
+ * The given clause has a mergejoinable operator and is not an outer-join
+ * qualification, so its two sides can be considered equal
* anywhere they are both computable; moreover that equality can be
* extended transitively. Record this knowledge in the EquivalenceClass
* data structure, if applicable. Returns true if successful, false if not
@@ -88,16 +90,11 @@ static Bitmapset *get_common_eclass_indexes(PlannerInfo *root, Relids relids1,
* Then, *p_restrictinfo will be replaced by a new RestrictInfo, which is what
* the caller should use for further processing.
*
- * If below_outer_join is true, then the clause was found below the nullable
- * side of an outer join, so its sides might validly be both NULL rather than
- * strictly equal. We can still deduce equalities in such cases, but we take
- * care to mark an EquivalenceClass if it came from any such clauses. Also,
- * we have to check that both sides are either pseudo-constants or strict
- * functions of Vars, else they might not both go to NULL above the outer
- * join. (This is the main reason why we need a failure return. It's more
- * convenient to check this case here than at the call sites...)
+ * jdomain is the join domain within which the given clause was found.
+ * This limits the applicability of deductions from the EquivalenceClass,
+ * as described in optimizer/README.
*
- * We also reject proposed equivalence clauses if they contain leaky functions
+ * We reject proposed equivalence clauses if they contain leaky functions
* and have security_level above zero. The EC evaluation rules require us to
* apply certain tests at certain joining levels, and we can't tolerate
* delaying any test on security_level grounds. By rejecting candidate clauses
@@ -120,7 +117,7 @@ static Bitmapset *get_common_eclass_indexes(PlannerInfo *root, Relids relids1,
bool
process_equivalence(PlannerInfo *root,
RestrictInfo **p_restrictinfo,
- bool below_outer_join)
+ JoinDomain *jdomain)
{
RestrictInfo *restrictinfo = *p_restrictinfo;
Expr *clause = restrictinfo->clause;
@@ -208,19 +205,6 @@ process_equivalence(PlannerInfo *root,
return false;
}
- /*
- * If below outer join, check for strictness, else reject.
- */
- if (below_outer_join)
- {
- if (!bms_is_empty(item1_relids) &&
- contain_nonstrict_functions((Node *) item1))
- return false; /* LHS is non-strict but not constant */
- if (!bms_is_empty(item2_relids) &&
- contain_nonstrict_functions((Node *) item2))
- return false; /* RHS is non-strict but not constant */
- }
-
/*
* We use the declared input types of the operator, not exprType() of the
* inputs, as the nominal datatypes for opfamily lookup. This presumes
@@ -285,11 +269,10 @@ process_equivalence(PlannerInfo *root,
Assert(!cur_em->em_is_child); /* no children yet */
/*
- * If below an outer join, don't match constants: they're not as
- * constant as they look.
+ * Match constants only within the same JoinDomain (see
+ * optimizer/README).
*/
- if ((below_outer_join || cur_ec->ec_below_outer_join) &&
- cur_em->em_is_const)
+ if (cur_em->em_is_const && cur_em->em_jdomain != jdomain)
continue;
if (!ec1 &&
@@ -326,7 +309,6 @@ process_equivalence(PlannerInfo *root,
if (ec1 == ec2)
{
ec1->ec_sources = lappend(ec1->ec_sources, restrictinfo);
- ec1->ec_below_outer_join |= below_outer_join;
ec1->ec_min_security = Min(ec1->ec_min_security,
restrictinfo->security_level);
ec1->ec_max_security = Max(ec1->ec_max_security,
@@ -362,7 +344,6 @@ process_equivalence(PlannerInfo *root,
ec1->ec_relids = bms_join(ec1->ec_relids, ec2->ec_relids);
ec1->ec_has_const |= ec2->ec_has_const;
/* can't need to set has_volatile */
- ec1->ec_below_outer_join |= ec2->ec_below_outer_join;
ec1->ec_min_security = Min(ec1->ec_min_security,
ec2->ec_min_security);
ec1->ec_max_security = Max(ec1->ec_max_security,
@@ -375,7 +356,6 @@ process_equivalence(PlannerInfo *root,
ec2->ec_derives = NIL;
ec2->ec_relids = NULL;
ec1->ec_sources = lappend(ec1->ec_sources, restrictinfo);
- ec1->ec_below_outer_join |= below_outer_join;
ec1->ec_min_security = Min(ec1->ec_min_security,
restrictinfo->security_level);
ec1->ec_max_security = Max(ec1->ec_max_security,
@@ -391,9 +371,8 @@ process_equivalence(PlannerInfo *root,
{
/* Case 3: add item2 to ec1 */
em2 = add_eq_member(ec1, item2, item2_relids,
- NULL, item2_type);
+ jdomain, NULL, item2_type);
ec1->ec_sources = lappend(ec1->ec_sources, restrictinfo);
- ec1->ec_below_outer_join |= below_outer_join;
ec1->ec_min_security = Min(ec1->ec_min_security,
restrictinfo->security_level);
ec1->ec_max_security = Max(ec1->ec_max_security,
@@ -409,9 +388,8 @@ process_equivalence(PlannerInfo *root,
{
/* Case 3: add item1 to ec2 */
em1 = add_eq_member(ec2, item1, item1_relids,
- NULL, item1_type);
+ jdomain, NULL, item1_type);
ec2->ec_sources = lappend(ec2->ec_sources, restrictinfo);
- ec2->ec_below_outer_join |= below_outer_join;
ec2->ec_min_security = Min(ec2->ec_min_security,
restrictinfo->security_level);
ec2->ec_max_security = Max(ec2->ec_max_security,
@@ -436,16 +414,15 @@ process_equivalence(PlannerInfo *root,
ec->ec_relids = NULL;
ec->ec_has_const = false;
ec->ec_has_volatile = false;
- ec->ec_below_outer_join = below_outer_join;
ec->ec_broken = false;
ec->ec_sortref = 0;
ec->ec_min_security = restrictinfo->security_level;
ec->ec_max_security = restrictinfo->security_level;
ec->ec_merged = NULL;
em1 = add_eq_member(ec, item1, item1_relids,
- NULL, item1_type);
+ jdomain, NULL, item1_type);
em2 = add_eq_member(ec, item2, item2_relids,
- NULL, item2_type);
+ jdomain, NULL, item2_type);
root->eq_classes = lappend(root->eq_classes, ec);
@@ -535,7 +512,7 @@ canonicalize_ec_expression(Expr *expr, Oid req_type, Oid req_collation)
*/
static EquivalenceMember *
add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
- EquivalenceMember *parent, Oid datatype)
+ JoinDomain *jdomain, EquivalenceMember *parent, Oid datatype)
{
EquivalenceMember *em = makeNode(EquivalenceMember);
@@ -544,6 +521,7 @@ add_eq_member(EquivalenceClass *ec, Expr *expr, Relids relids,
em->em_is_const = false;
em->em_is_child = (parent != NULL);
em->em_datatype = datatype;
+ em->em_jdomain = jdomain;
em->em_parent = parent;
if (bms_is_empty(relids))
@@ -612,6 +590,7 @@ get_eclass_for_sort_expr(PlannerInfo *root,
Relids rel,
bool create_it)
{
+ JoinDomain *jdomain;
Relids expr_relids;
EquivalenceClass *newec;
EquivalenceMember *newem;
@@ -623,6 +602,12 @@ get_eclass_for_sort_expr(PlannerInfo *root,
*/
expr = canonicalize_ec_expression(expr, opcintype, collation);
+ /*
+ * Since SortGroupClause nodes are top-level expressions (GROUP BY, ORDER
+ * BY, etc), they can be presumed to belong to the top JoinDomain.
+ */
+ jdomain = linitial_node(JoinDomain, root->join_domains);
+
/*
* Scan through the existing EquivalenceClasses for a match
*/
@@ -656,11 +641,10 @@ get_eclass_for_sort_expr(PlannerInfo *root,
continue;
/*
- * If below an outer join, don't match constants: they're not as
- * constant as they look.
+ * Match constants only within the same JoinDomain (see
+ * optimizer/README).
*/
- if (cur_ec->ec_below_outer_join &&
- cur_em->em_is_const)
+ if (cur_em->em_is_const && cur_em->em_jdomain != jdomain)
continue;
if (opcintype == cur_em->em_datatype &&
@@ -689,7 +673,6 @@ get_eclass_for_sort_expr(PlannerInfo *root,
newec->ec_relids = NULL;
newec->ec_has_const = false;
newec->ec_has_volatile = contain_volatile_functions((Node *) expr);
- newec->ec_below_outer_join = false;
newec->ec_broken = false;
newec->ec_sortref = sortref;
newec->ec_min_security = UINT_MAX;
@@ -705,7 +688,7 @@ get_eclass_for_sort_expr(PlannerInfo *root,
expr_relids = pull_varnos(root, (Node *) expr);
newem = add_eq_member(newec, copyObject(expr), expr_relids,
- NULL, opcintype);
+ jdomain, NULL, opcintype);
/*
* add_eq_member doesn't check for volatile functions, set-returning
@@ -1185,11 +1168,16 @@ generate_base_implied_equalities_const(PlannerInfo *root,
ec->ec_broken = true;
break;
}
+
+ /*
+ * We use the constant's em_jdomain as qualscope, so that if the
+ * generated clause is variable-free (i.e, both EMs are consts) it
+ * will be enforced at the join domain level.
+ */
rinfo = process_implied_equality(root, eq_op, ec->ec_collation,
cur_em->em_expr, const_em->em_expr,
- bms_copy(ec->ec_relids),
+ const_em->em_jdomain->jd_relids,
ec->ec_min_security,
- ec->ec_below_outer_join,
cur_em->em_is_const);
/*
@@ -1257,11 +1245,16 @@ generate_base_implied_equalities_no_const(PlannerInfo *root,
ec->ec_broken = true;
break;
}
+
+ /*
+ * The expressions aren't constants, so the passed qualscope will
+ * never be used to place the generated clause. We just need to
+ * be sure it covers both expressions, so ec_relids will serve.
+ */
rinfo = process_implied_equality(root, eq_op, ec->ec_collation,
prev_em->em_expr, cur_em->em_expr,
- bms_copy(ec->ec_relids),
+ ec->ec_relids,
ec->ec_min_security,
- ec->ec_below_outer_join,
false);
/*
@@ -2074,6 +2067,7 @@ reconsider_outer_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo,
bool outer_on_left)
{
RestrictInfo *rinfo = ojcinfo->rinfo;
+ SpecialJoinInfo *sjinfo = ojcinfo->sjinfo;
Expr *outervar,
*innervar;
Oid opno,
@@ -2150,6 +2144,7 @@ reconsider_outer_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo,
EquivalenceMember *cur_em = (EquivalenceMember *) lfirst(lc2);
Oid eq_op;
RestrictInfo *newrinfo;
+ JoinDomain *jdomain;
if (!cur_em->em_is_const)
continue; /* ignore non-const members */
@@ -2165,7 +2160,9 @@ reconsider_outer_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo,
cur_em->em_expr,
bms_copy(inner_relids),
cur_ec->ec_min_security);
- if (process_equivalence(root, &newrinfo, true))
+ /* This equality holds within the OJ's child JoinDomain */
+ jdomain = find_join_domain(root, sjinfo->syn_righthand);
+ if (process_equivalence(root, &newrinfo, jdomain))
match = true;
}
@@ -2300,6 +2297,7 @@ reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
EquivalenceMember *cur_em = (EquivalenceMember *) lfirst(lc2);
Oid eq_op;
RestrictInfo *newrinfo;
+ JoinDomain *jdomain;
if (!cur_em->em_is_const)
continue; /* ignore non-const members */
@@ -2315,7 +2313,9 @@ reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
cur_em->em_expr,
bms_copy(left_relids),
cur_ec->ec_min_security);
- if (process_equivalence(root, &newrinfo, true))
+ /* This equality holds within the lefthand child JoinDomain */
+ jdomain = find_join_domain(root, sjinfo->syn_lefthand);
+ if (process_equivalence(root, &newrinfo, jdomain))
matchleft = true;
}
eq_op = select_equality_operator(cur_ec,
@@ -2330,7 +2330,9 @@ reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
cur_em->em_expr,
bms_copy(right_relids),
cur_ec->ec_min_security);
- if (process_equivalence(root, &newrinfo, true))
+ /* This equality holds within the righthand child JoinDomain */
+ jdomain = find_join_domain(root, sjinfo->syn_righthand);
+ if (process_equivalence(root, &newrinfo, jdomain))
matchright = true;
}
}
@@ -2359,6 +2361,29 @@ reconsider_full_join_clause(PlannerInfo *root, OuterJoinClauseInfo *ojcinfo)
return false; /* failed to make any deduction */
}
+/*
+ * find_join_domain
+ * Find the highest JoinDomain enclosed within the given relid set.
+ *
+ * (We could avoid this search at the cost of complicating APIs elsewhere,
+ * which doesn't seem worth it.)
+ */
+static JoinDomain *
+find_join_domain(PlannerInfo *root, Relids relids)
+{
+ ListCell *lc;
+
+ foreach(lc, root->join_domains)
+ {
+ JoinDomain *jdomain = (JoinDomain *) lfirst(lc);
+
+ if (bms_is_subset(jdomain->jd_relids, relids))
+ return jdomain;
+ }
+ elog(ERROR, "failed to find appropriate JoinDomain");
+ return NULL; /* keep compiler quiet */
+}
+
/*
* exprs_known_equal
@@ -2648,6 +2673,7 @@ add_child_rel_equivalences(PlannerInfo *root,
new_relids = bms_add_members(new_relids, child_relids);
(void) add_eq_member(cur_ec, child_expr, new_relids,
+ cur_em->em_jdomain,
cur_em, cur_em->em_datatype);
/* Record this EC index for the child rel */
@@ -2775,6 +2801,7 @@ add_child_join_rel_equivalences(PlannerInfo *root,
new_relids = bms_add_members(new_relids, child_relids);
(void) add_eq_member(cur_ec, child_expr, new_relids,
+ cur_em->em_jdomain,
cur_em, cur_em->em_datatype);
}
}
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 7591a7d81d..e69dc31237 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -2321,18 +2321,6 @@ select_mergejoin_clauses(PlannerInfo *root,
* canonical pathkey list, but redundant eclasses can't appear in
* canonical sort orderings. (XXX it might be worth relaxing this,
* but not enough time to address it for 8.3.)
- *
- * Note: it would be bad if this condition failed for an otherwise
- * mergejoinable FULL JOIN clause, since that would result in
- * undesirable planner failure. I believe that is not possible
- * however; a variable involved in a full join could only appear in
- * below_outer_join eclasses, which aren't considered redundant.
- *
- * This case *can* happen for left/right join clauses: the outer-side
- * variable could be equated to a constant. Because we will propagate
- * that constant across the join clause, the loss of ability to do a
- * mergejoin is not really all that big a deal, and so it's not clear
- * that improving this is important.
*/
update_mergeclause_eclasses(root, restrictinfo);
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 4c99b28d0a..1b11852814 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -6210,10 +6210,7 @@ prepare_sort_from_pathkeys(Plan *lefttree, List *pathkeys,
* the pathkey's EquivalenceClass. For now, we take the first
* tlist item found in the EC. If there's no match, we'll generate
* a resjunk entry using the first EC member that is an expression
- * in the input's vars. (The non-const restriction only matters
- * if the EC is below_outer_join; but if it isn't, it won't
- * contain consts anyway, else we'd have discarded the pathkey as
- * redundant.)
+ * in the input's vars.
*
* XXX if we have a choice, is there any way of figuring out which
* might be cheapest to execute? (For example, int4lt is likely
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 0f4163bffd..35b2dc1034 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -61,7 +61,7 @@ typedef struct JoinTreeItem
{
/* Fields filled during deconstruct_recurse: */
Node *jtnode; /* jointree node to examine */
- bool below_outer_join; /* is it below an outer join? */
+ JoinDomain *jdomain; /* join domain for its ON/WHERE clauses */
Relids qualscope; /* base+OJ Relids syntactically included in
* this jointree node */
Relids inner_join_rels; /* base+OJ Relids syntactically included
@@ -87,13 +87,13 @@ typedef struct PostponedQual
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
- bool below_outer_join,
+ JoinDomain *parent_domain,
List **item_list);
static void deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
List **postponed_qual_list);
static void process_security_barrier_quals(PlannerInfo *root,
int rti, Relids qualscope,
- bool below_outer_join);
+ JoinDomain *jdomain);
static void mark_rels_nulled_by_join(PlannerInfo *root, Index ojrelid,
Relids lower_rels);
static SpecialJoinInfo *make_outerjoininfo(PlannerInfo *root,
@@ -107,7 +107,7 @@ static void deconstruct_distribute_oj_quals(PlannerInfo *root,
List *jtitems,
JoinTreeItem *jtitem);
static void distribute_quals_to_rels(PlannerInfo *root, List *clauses,
- bool below_outer_join,
+ JoinDomain *jdomain,
SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
@@ -119,7 +119,7 @@ static void distribute_quals_to_rels(PlannerInfo *root, List *clauses,
List **postponed_qual_list,
List **postponed_oj_qual_list);
static void distribute_qual_to_rels(PlannerInfo *root, Node *clause,
- bool below_outer_join,
+ JoinDomain *jdomain,
SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
@@ -740,6 +740,7 @@ List *
deconstruct_jointree(PlannerInfo *root)
{
List *result;
+ JoinDomain *top_jdomain;
List *item_list = NIL;
List *postponed_qual_list = NIL;
ListCell *lc;
@@ -751,6 +752,10 @@ deconstruct_jointree(PlannerInfo *root)
*/
root->placeholdersFrozen = true;
+ /* Fetch the already-created top-level join domain for the query */
+ top_jdomain = linitial_node(JoinDomain, root->join_domains);
+ top_jdomain->jd_relids = NULL; /* filled during deconstruct_recurse */
+
/* Start recursion at top of jointree */
Assert(root->parse->jointree != NULL &&
IsA(root->parse->jointree, FromExpr));
@@ -761,12 +766,15 @@ deconstruct_jointree(PlannerInfo *root)
/* Perform the initial scan of the jointree */
result = deconstruct_recurse(root, (Node *) root->parse->jointree,
- false,
+ top_jdomain,
&item_list);
/* Now we can form the value of all_query_rels, too */
root->all_query_rels = bms_union(root->all_baserels, root->outer_join_rels);
+ /* ... which should match what we computed for the top join domain */
+ Assert(bms_equal(root->all_query_rels, top_jdomain->jd_relids));
+
/* Now scan all the jointree nodes again, and distribute quals */
foreach(lc, item_list)
{
@@ -804,10 +812,9 @@ deconstruct_jointree(PlannerInfo *root)
* deconstruct_recurse
* One recursion level of deconstruct_jointree's initial jointree scan.
*
- * Inputs:
- * jtnode is the jointree node to examine
- * below_outer_join is true if this node is within the nullable side of a
- * higher-level outer join
+ * jtnode is the jointree node to examine, and parent_domain is the
+ * enclosing join domain. (We must add all base+OJ relids appearing
+ * here or below to parent_domain.)
*
* item_list is an in/out parameter: we add a JoinTreeItem struct to
* that list for each jointree node, in depth-first traversal order.
@@ -817,7 +824,7 @@ deconstruct_jointree(PlannerInfo *root)
*/
static List *
deconstruct_recurse(PlannerInfo *root, Node *jtnode,
- bool below_outer_join,
+ JoinDomain *parent_domain,
List **item_list)
{
List *joinlist;
@@ -828,7 +835,6 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
/* Make the new JoinTreeItem, but don't add it to item_list yet */
jtitem = palloc0_object(JoinTreeItem);
jtitem->jtnode = jtnode;
- jtitem->below_outer_join = below_outer_join;
if (IsA(jtnode, RangeTblRef))
{
@@ -836,6 +842,10 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
/* Fill all_baserels as we encounter baserel jointree nodes */
root->all_baserels = bms_add_member(root->all_baserels, varno);
+ /* This node belongs to parent_domain */
+ jtitem->jdomain = parent_domain;
+ parent_domain->jd_relids = bms_add_member(parent_domain->jd_relids,
+ varno);
/* qualscope is just the one RTE */
jtitem->qualscope = bms_make_singleton(varno);
/* A single baserel does not create an inner join */
@@ -848,6 +858,9 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
int remaining;
ListCell *l;
+ /* This node belongs to parent_domain, as do its children */
+ jtitem->jdomain = parent_domain;
+
/*
* Recurse to handle child nodes, and compute output joinlist. We
* collapse subproblems into a single joinlist whenever the resulting
@@ -866,7 +879,7 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
int sub_members;
sub_joinlist = deconstruct_recurse(root, lfirst(l),
- below_outer_join,
+ parent_domain,
item_list);
sub_item = (JoinTreeItem *) llast(*item_list);
jtitem->qualscope = bms_add_members(jtitem->qualscope,
@@ -894,6 +907,8 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
+ JoinDomain *child_domain,
+ *fj_domain;
JoinTreeItem *left_item,
*right_item;
List *leftjoinlist,
@@ -902,13 +917,15 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
switch (j->jointype)
{
case JOIN_INNER:
+ /* This node belongs to parent_domain, as do its children */
+ jtitem->jdomain = parent_domain;
/* Recurse */
leftjoinlist = deconstruct_recurse(root, j->larg,
- below_outer_join,
+ parent_domain,
item_list);
left_item = (JoinTreeItem *) llast(*item_list);
rightjoinlist = deconstruct_recurse(root, j->rarg,
- below_outer_join,
+ parent_domain,
item_list);
right_item = (JoinTreeItem *) llast(*item_list);
/* Compute qualscope etc */
@@ -922,21 +939,32 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
break;
case JOIN_LEFT:
case JOIN_ANTI:
+ /* Make new join domain for my quals and the RHS */
+ child_domain = makeNode(JoinDomain);
+ child_domain->jd_relids = NULL; /* filled by recursion */
+ root->join_domains = lappend(root->join_domains, child_domain);
+ jtitem->jdomain = child_domain;
/* Recurse */
leftjoinlist = deconstruct_recurse(root, j->larg,
- below_outer_join,
+ parent_domain,
item_list);
left_item = (JoinTreeItem *) llast(*item_list);
rightjoinlist = deconstruct_recurse(root, j->rarg,
- true,
+ child_domain,
item_list);
right_item = (JoinTreeItem *) llast(*item_list);
- /* Compute qualscope etc */
+ /* Compute join domain contents, qualscope etc */
+ parent_domain->jd_relids =
+ bms_add_members(parent_domain->jd_relids,
+ child_domain->jd_relids);
jtitem->qualscope = bms_union(left_item->qualscope,
right_item->qualscope);
/* caution: ANTI join derived from SEMI will lack rtindex */
if (j->rtindex != 0)
{
+ parent_domain->jd_relids =
+ bms_add_member(parent_domain->jd_relids,
+ j->rtindex);
jtitem->qualscope = bms_add_member(jtitem->qualscope,
j->rtindex);
root->outer_join_rels = bms_add_member(root->outer_join_rels,
@@ -951,13 +979,15 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
jtitem->nonnullable_rels = left_item->qualscope;
break;
case JOIN_SEMI:
+ /* This node belongs to parent_domain, as do its children */
+ jtitem->jdomain = parent_domain;
/* Recurse */
leftjoinlist = deconstruct_recurse(root, j->larg,
- below_outer_join,
+ parent_domain,
item_list);
left_item = (JoinTreeItem *) llast(*item_list);
rightjoinlist = deconstruct_recurse(root, j->rarg,
- below_outer_join,
+ parent_domain,
item_list);
right_item = (JoinTreeItem *) llast(*item_list);
/* Compute qualscope etc */
@@ -973,19 +1003,36 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
jtitem->nonnullable_rels = NULL;
break;
case JOIN_FULL:
- /* Recurse */
+ /* The FULL JOIN's quals need their very own domain */
+ fj_domain = makeNode(JoinDomain);
+ root->join_domains = lappend(root->join_domains, fj_domain);
+ jtitem->jdomain = fj_domain;
+ /* Recurse, giving each side its own join domain */
+ child_domain = makeNode(JoinDomain);
+ child_domain->jd_relids = NULL; /* filled by recursion */
+ root->join_domains = lappend(root->join_domains, child_domain);
leftjoinlist = deconstruct_recurse(root, j->larg,
- true,
+ child_domain,
item_list);
left_item = (JoinTreeItem *) llast(*item_list);
+ fj_domain->jd_relids = bms_copy(child_domain->jd_relids);
+ child_domain = makeNode(JoinDomain);
+ child_domain->jd_relids = NULL; /* filled by recursion */
+ root->join_domains = lappend(root->join_domains, child_domain);
rightjoinlist = deconstruct_recurse(root, j->rarg,
- true,
+ child_domain,
item_list);
right_item = (JoinTreeItem *) llast(*item_list);
/* Compute qualscope etc */
+ fj_domain->jd_relids = bms_add_members(fj_domain->jd_relids,
+ child_domain->jd_relids);
+ parent_domain->jd_relids = bms_add_members(parent_domain->jd_relids,
+ fj_domain->jd_relids);
jtitem->qualscope = bms_union(left_item->qualscope,
right_item->qualscope);
Assert(j->rtindex != 0);
+ parent_domain->jd_relids = bms_add_member(parent_domain->jd_relids,
+ j->rtindex);
jtitem->qualscope = bms_add_member(jtitem->qualscope,
j->rtindex);
root->outer_join_rels = bms_add_member(root->outer_join_rels,
@@ -1087,7 +1134,7 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
process_security_barrier_quals(root,
varno,
jtitem->qualscope,
- jtitem->below_outer_join);
+ jtitem->jdomain);
}
else if (IsA(jtnode, FromExpr))
{
@@ -1105,7 +1152,7 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
if (bms_is_subset(pq->relids, jtitem->qualscope))
distribute_qual_to_rels(root, pq->qual,
- jtitem->below_outer_join,
+ jtitem->jdomain,
NULL,
root->qual_security_level,
jtitem->qualscope, NULL, NULL,
@@ -1120,7 +1167,7 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
* Now process the top-level quals.
*/
distribute_quals_to_rels(root, (List *) f->quals,
- jtitem->below_outer_join,
+ jtitem->jdomain,
NULL,
root->qual_security_level,
jtitem->qualscope, NULL, NULL,
@@ -1221,7 +1268,7 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
/* Process the JOIN's qual clauses */
distribute_quals_to_rels(root, my_quals,
- jtitem->below_outer_join,
+ jtitem->jdomain,
sjinfo,
root->qual_security_level,
jtitem->qualscope,
@@ -1258,7 +1305,7 @@ deconstruct_distribute(PlannerInfo *root, JoinTreeItem *jtitem,
static void
process_security_barrier_quals(PlannerInfo *root,
int rti, Relids qualscope,
- bool below_outer_join)
+ JoinDomain *jdomain)
{
RangeTblEntry *rte = root->simple_rte_array[rti];
Index security_level = 0;
@@ -1281,7 +1328,7 @@ process_security_barrier_quals(PlannerInfo *root,
* pushed up to top of tree, which we don't want.
*/
distribute_quals_to_rels(root, qualset,
- below_outer_join,
+ jdomain,
NULL,
security_level,
qualscope,
@@ -1991,7 +2038,7 @@ deconstruct_distribute_oj_quals(PlannerInfo *root,
is_clone = !has_clone;
distribute_quals_to_rels(root, quals,
- true,
+ otherjtitem->jdomain,
sjinfo,
root->qual_security_level,
this_qualscope,
@@ -2020,7 +2067,7 @@ deconstruct_distribute_oj_quals(PlannerInfo *root,
{
/* No commutation possible, just process the postponed clauses */
distribute_quals_to_rels(root, jtitem->oj_joinclauses,
- true,
+ jtitem->jdomain,
sjinfo,
root->qual_security_level,
qualscope,
@@ -2045,7 +2092,7 @@ deconstruct_distribute_oj_quals(PlannerInfo *root,
*/
static void
distribute_quals_to_rels(PlannerInfo *root, List *clauses,
- bool below_outer_join,
+ JoinDomain *jdomain,
SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
@@ -2064,7 +2111,7 @@ distribute_quals_to_rels(PlannerInfo *root, List *clauses,
Node *clause = (Node *) lfirst(lc);
distribute_qual_to_rels(root, clause,
- below_outer_join,
+ jdomain,
sjinfo,
security_level,
qualscope,
@@ -2092,8 +2139,7 @@ distribute_quals_to_rels(PlannerInfo *root, List *clauses,
* These will be dealt with in later steps of deconstruct_jointree.
*
* 'clause': the qual clause to be distributed
- * 'below_outer_join': true if the qual is from a JOIN/ON that is below the
- * nullable side of a higher-level outer join
+ * 'jdomain': the join domain containing the clause
* 'sjinfo': join's SpecialJoinInfo (NULL for an inner join or WHERE clause)
* 'security_level': security_level to assign to the qual
* 'qualscope': set of base+OJ rels the qual's syntactic scope covers
@@ -2124,7 +2170,7 @@ distribute_quals_to_rels(PlannerInfo *root, List *clauses,
*/
static void
distribute_qual_to_rels(PlannerInfo *root, Node *clause,
- bool below_outer_join,
+ JoinDomain *jdomain,
SpecialJoinInfo *sjinfo,
Index security_level,
Relids qualscope,
@@ -2196,12 +2242,8 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* RestrictInfo lists for the moment, but eventually createplan.c will
* pull it out and make a gating Result node immediately above whatever
* plan node the pseudoconstant clause is assigned to. It's usually best
- * to put a gating node as high in the plan tree as possible. If we are
- * not below an outer join, we can actually push the pseudoconstant qual
- * all the way to the top of the tree. If we are below an outer join, we
- * leave the qual at its original syntactic level (we could push it up to
- * just below the outer join, but that seems more complex than it's
- * worth).
+ * to put a gating node as high in the plan tree as possible, which we can
+ * do by assigning it the full relid set of the current JoinDomain.
*/
if (bms_is_empty(relids))
{
@@ -2211,25 +2253,20 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
relids = bms_copy(ojscope);
/* mustn't use as gating qual, so don't mark pseudoconstant */
}
- else
+ else if (contain_volatile_functions(clause))
{
/* eval at original syntactic level */
relids = bms_copy(qualscope);
- if (!contain_volatile_functions(clause))
- {
- /* mark as gating qual */
- pseudoconstant = true;
- /* tell createplan.c to check for gating quals */
- root->hasPseudoConstantQuals = true;
- /* if not below outer join, push it to top of tree */
- if (!below_outer_join)
- {
- relids =
- get_relids_in_jointree((Node *) root->parse->jointree,
- true, false);
- qualscope = bms_copy(relids);
- }
- }
+ /* again, can't mark pseudoconstant */
+ }
+ else
+ {
+ /* eval at join domain level */
+ relids = bms_copy(jdomain->jd_relids);
+ /* mark as gating qual */
+ pseudoconstant = true;
+ /* tell createplan.c to check for gating quals */
+ root->hasPseudoConstantQuals = true;
}
}
@@ -2319,23 +2356,8 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
if (check_redundant_nullability_qual(root, clause))
return;
- if (!allow_equivalence)
- {
- /* Caller says it mustn't become an equivalence class */
- maybe_equivalence = false;
- }
- else
- {
- /*
- * Consider feeding qual to the equivalence machinery. However,
- * if it's itself within an outer-join clause, treat it as though
- * it appeared below that outer join (note that we can only get
- * here when the clause references only nullable-side rels).
- */
- maybe_equivalence = true;
- if (outerjoin_nonnullable != NULL)
- below_outer_join = true;
- }
+ /* Feed qual to the equivalence machinery, if allowed by caller */
+ maybe_equivalence = allow_equivalence;
/*
* Since it doesn't mention the LHS, it's certainly not useful as a
@@ -2401,16 +2423,14 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
check_mergejoinable(restrictinfo);
/*
- * XXX rewrite:
- *
* If it is a true equivalence clause, send it to the EquivalenceClass
* machinery. We do *not* attach it directly to any restriction or join
* lists. The EC code will propagate it to the appropriate places later.
*
- * If the clause has a mergejoinable operator and is not
- * outerjoin-delayed, yet isn't an equivalence because it is an outer-join
- * clause, the EC code may yet be able to do something with it. We add it
- * to appropriate lists for further consideration later. Specifically:
+ * If the clause has a mergejoinable operator, yet isn't an equivalence
+ * because it is an outer-join clause, the EC code may still be able to do
+ * something with it. We add it to appropriate lists for further
+ * consideration later. Specifically:
*
* If it is a left or right outer-join qualification that relates the two
* sides of the outer join (no funny business like leftvar1 = leftvar2 +
@@ -2438,7 +2458,7 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
{
if (maybe_equivalence)
{
- if (process_equivalence(root, &restrictinfo, below_outer_join))
+ if (process_equivalence(root, &restrictinfo, jdomain))
return;
/* EC rejected it, so set left_ec/right_ec the hard way ... */
if (restrictinfo->mergeopfamilies) /* EC might have changed this */
@@ -2628,8 +2648,9 @@ distribute_restrictinfo_to_rels(PlannerInfo *root,
* "qualscope" is the nominal syntactic level to impute to the restrictinfo.
* This must contain at least all the rels used in the expressions, but it
* is used only to set the qual application level when both exprs are
- * variable-free. Otherwise the qual is applied at the lowest join level
- * that provides all its variables.
+ * variable-free. (Hence, it should usually match the join domain in which
+ * the clause applies.) Otherwise the qual is applied at the lowest join
+ * level that provides all its variables.
*
* "security_level" is the security level to assign to the new restrictinfo.
*
@@ -2657,7 +2678,6 @@ process_implied_equality(PlannerInfo *root,
Expr *item2,
Relids qualscope,
Index security_level,
- bool below_outer_join,
bool both_const)
{
RestrictInfo *restrictinfo;
@@ -2706,27 +2726,16 @@ process_implied_equality(PlannerInfo *root,
/*
* If the clause is variable-free, our normal heuristic for pushing it
* down to just the mentioned rels doesn't work, because there are none.
- * Apply at the given qualscope, or at the top of tree if it's nonvolatile
- * (which it very likely is, but we'll check, just to be sure).
+ * Apply it as a gating qual at the given qualscope.
*/
if (bms_is_empty(relids))
{
- /* eval at original syntactic level */
+ /* eval at join domain level */
relids = bms_copy(qualscope);
- if (!contain_volatile_functions(clause))
- {
- /* mark as gating qual */
- pseudoconstant = true;
- /* tell createplan.c to check for gating quals */
- root->hasPseudoConstantQuals = true;
- /* if not below outer join, push it to top of tree */
- if (!below_outer_join)
- {
- relids =
- get_relids_in_jointree((Node *) root->parse->jointree,
- true, false);
- }
- }
+ /* mark as gating qual */
+ pseudoconstant = true;
+ /* tell createplan.c to check for gating quals */
+ root->hasPseudoConstantQuals = true;
}
/*
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 8674ad674d..db5ff6fdca 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -625,6 +625,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
root->init_plans = NIL;
root->cte_plan_ids = NIL;
root->multiexpr_params = NIL;
+ root->join_domains = NIL;
root->eq_classes = NIL;
root->ec_merging_done = false;
root->last_rinfo_serial = 0;
@@ -654,6 +655,13 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
root->non_recursive_path = NULL;
root->partColsUpdated = false;
+ /*
+ * Create the top-level join domain. This won't have valid contents until
+ * deconstruct_jointree fills it in, but the node needs to exist before
+ * that so we can build EquivalenceClasses referencing it.
+ */
+ root->join_domains = list_make1(makeNode(JoinDomain));
+
/*
* If there is a WITH list, process each WITH query and either convert it
* to RTE_SUBQUERY RTE(s) or build an initplan SubPlan structure for it.
@@ -6534,6 +6542,7 @@ plan_cluster_use_sort(Oid tableOid, Oid indexOid)
root->query_level = 1;
root->planner_cxt = CurrentMemoryContext;
root->wt_param_id = -1;
+ root->join_domains = list_make1(makeNode(JoinDomain));
/* Build a minimal RTE for the rel */
rte = makeNode(RangeTblEntry);
@@ -6655,6 +6664,7 @@ plan_create_index_workers(Oid tableOid, Oid indexOid)
root->query_level = 1;
root->planner_cxt = CurrentMemoryContext;
root->wt_param_id = -1;
+ root->join_domains = list_make1(makeNode(JoinDomain));
/*
* Build a minimal RTE.
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index eacfb66b31..870d84b29d 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -991,6 +991,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
subroot->init_plans = NIL;
subroot->cte_plan_ids = NIL;
subroot->multiexpr_params = NIL;
+ subroot->join_domains = NIL;
subroot->eq_classes = NIL;
subroot->ec_merging_done = false;
subroot->last_rinfo_serial = 0;
@@ -1012,6 +1013,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
subroot->hasRecursion = false;
subroot->wt_param_id = -1;
subroot->non_recursive_path = NULL;
+ /* We don't currently need a top JoinDomain for the subroot */
/* No CTEs to worry about */
Assert(subquery->cteList == NIL);
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index f843659a18..62d9460258 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -307,6 +307,9 @@ struct PlannerInfo
/* List of Lists of Params for MULTIEXPR subquery outputs */
List *multiexpr_params;
+ /* list of JoinDomains used in the query (higher ones first) */
+ List *join_domains;
+
/* list of active EquivalenceClasses */
List *eq_classes;
@@ -1278,11 +1281,46 @@ typedef struct StatisticExtInfo
List *exprs;
} StatisticExtInfo;
+/*
+ * JoinDomains
+ *
+ * A "join domain" defines the scope of applicability of deductions made via
+ * the EquivalenceClass mechanism. Roughly speaking, a join domain is a set
+ * of base+OJ relations that are inner-joined together. More precisely, it is
+ * the set of relations at which equalities deduced from an EquivalenceClass
+ * can be enforced or should be expected to hold. The topmost JoinDomain
+ * covers the whole query (so its jd_relids should equal all_query_rels).
+ * An outer join creates a new JoinDomain that includes all base+OJ relids
+ * within its nullable side, but (by convention) not the OJ's own relid.
+ * A FULL join creates two new JoinDomains, one for each side.
+ *
+ * Notice that a rel that is below outer join(s) will thus appear to belong
+ * to multiple join domains. However, any of its Vars that appear in
+ * EquivalenceClasses belonging to higher join domains will have nullingrel
+ * bits preventing them from being evaluated at the rel's scan level, so that
+ * we will not be able to derive enforceable-at-the-rel-scan-level clauses
+ * from such ECs. We define the join domain relid sets this way so that
+ * domains can be said to be "higher" or "lower" when one domain relid set
+ * includes another.
+ *
+ * The JoinDomains for a query are computed in deconstruct_jointree.
+ * We do not copy JoinDomain structs once made, so they can be compared
+ * for equality by simple pointer equality.
+ */
+typedef struct JoinDomain
+{
+ pg_node_attr(no_copy_equal, no_read)
+
+ NodeTag type;
+
+ Relids jd_relids; /* all relids contained within the domain */
+} JoinDomain;
+
/*
* EquivalenceClasses
*
- * Whenever we can determine that a mergejoinable equality clause A = B is
- * not delayed by any outer join, we create an EquivalenceClass containing
+ * Whenever we identify a mergejoinable equality clause A = B that is
+ * not an outer-join clause, we create an EquivalenceClass containing
* the expressions A and B to record this knowledge. If we later find another
* equivalence B = C, we add C to the existing EquivalenceClass; this may
* require merging two existing EquivalenceClasses. At the end of the qual
@@ -1296,6 +1334,18 @@ typedef struct StatisticExtInfo
* that all or none of the input datatypes are collatable, so that a single
* collation value is sufficient.)
*
+ * Strictly speaking, deductions from an EquivalenceClass hold only within
+ * a "join domain", that is a set of relations that are innerjoined together
+ * (see JoinDomain above). For the most part we don't need to account for
+ * this explicitly, because equality clauses from different join domains
+ * will contain Vars that are not equal() because they have different
+ * nullingrel sets, and thus we will never falsely merge ECs from different
+ * join domains. But Var-free (pseudoconstant) expressions lack that safety
+ * feature. We handle that by marking "const" EC members with the JoinDomain
+ * of the clause they came from; two nominally-equal const members will be
+ * considered different if they came from different JoinDomains. This ensures
+ * no false EquivalenceClass merges will occur.
+ *
* We also use EquivalenceClasses as the base structure for PathKeys, letting
* us represent knowledge about different sort orderings being equivalent.
* Since every PathKey must reference an EquivalenceClass, we will end up
@@ -1310,11 +1360,6 @@ typedef struct StatisticExtInfo
* entry: consider SELECT random() AS a, random() AS b ... ORDER BY b,a.
* So we record the SortGroupRef of the originating sort clause.
*
- * We allow equality clauses appearing below the nullable side of an outer join
- * to form EquivalenceClasses, but these have a slightly different meaning:
- * the included values might be all NULL rather than all the same non-null
- * values. See src/backend/optimizer/README for more on that point.
- *
* NB: if ec_merged isn't NULL, this class has been merged into another, and
* should be ignored in favor of using the pointed-to class.
*
@@ -1339,7 +1384,6 @@ typedef struct EquivalenceClass
* for child members (see below) */
bool ec_has_const; /* any pseudoconstants in ec_members? */
bool ec_has_volatile; /* the (sole) member is a volatile expr */
- bool ec_below_outer_join; /* equivalence applies below an OJ */
bool ec_broken; /* failed to generate needed clauses? */
Index ec_sortref; /* originating sortclause label, or 0 */
Index ec_min_security; /* minimum security_level in ec_sources */
@@ -1348,11 +1392,11 @@ typedef struct EquivalenceClass
} EquivalenceClass;
/*
- * If an EC contains a const and isn't below-outer-join, any PathKey depending
- * on it must be redundant, since there's only one possible value of the key.
+ * If an EC contains a constant, any PathKey depending on it must be
+ * redundant, since there's only one possible value of the key.
*/
#define EC_MUST_BE_REDUNDANT(eclass) \
- ((eclass)->ec_has_const && !(eclass)->ec_below_outer_join)
+ ((eclass)->ec_has_const)
/*
* EquivalenceMember - one member expression of an EquivalenceClass
@@ -1387,6 +1431,7 @@ typedef struct EquivalenceMember
bool em_is_const; /* expression is pseudoconstant? */
bool em_is_child; /* derived version for a child relation? */
Oid em_datatype; /* the "nominal type" used by the opfamily */
+ JoinDomain *em_jdomain; /* join domain containing the source clause */
/* if em_is_child is true, this links to corresponding EM for top parent */
struct EquivalenceMember *em_parent pg_node_attr(read_write_ignore);
} EquivalenceMember;
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 1b02a1dc08..736d78ea4c 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -122,7 +122,7 @@ typedef bool (*ec_matches_callback_type) (PlannerInfo *root,
extern bool process_equivalence(PlannerInfo *root,
RestrictInfo **p_restrictinfo,
- bool below_outer_join);
+ JoinDomain *jdomain);
extern Expr *canonicalize_ec_expression(Expr *expr,
Oid req_type, Oid req_collation);
extern void reconsider_outer_join_clauses(PlannerInfo *root);
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index 3e6e60f549..5fc900737d 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -84,7 +84,6 @@ extern RestrictInfo *process_implied_equality(PlannerInfo *root,
Expr *item2,
Relids qualscope,
Index security_level,
- bool below_outer_join,
bool both_const);
extern RestrictInfo *build_implied_join_equality(PlannerInfo *root,
Oid opno,
Hello Tom
I just noticed your new efforts in this area.
I wanted to recurr to my old thread [1] considering constant propagation of quals.
You gave an elaborated explanation at that time, but my knowledge was/is not yet sufficient to reveil the technical details.
In our application the described method is widespread used with much success (now at pg15.1 Fedora), but for unexperienced SQL authors this is not really obviously to choose (i.e. using the explicit constant xx_season=3 as qual). This always requires a "Macro" processor to compose the queries (in my case php) and a lot of programmer effort in the source code.
I can't review/understand your patchset for the planner, but since it covers the same area, the beformentioned optimization could perhaps be addressed too.
With respect of the nullability of these quals I immediately changed all of them to NOT NULL, which seems the most natural way when these quals are also used for partioning.
[1] https://www.postgresql.org/message-id/1571413123735.26467@nidsa.net
Thanks for looking
Hans Buschmann
Hans Buschmann <buschmann@nidsa.net> writes: > I just noticed your new efforts in this area. > I wanted to recurr to my old thread [1] considering constant propagation of quals. > [1] https://www.postgresql.org/message-id/1571413123735.26467@nidsa.net Yeah, this patch series is not yet quite up to the point of improving that. That area is indeed the very next thing I want to work on, and I did spend some effort on it last month, but I ran out of time to get it working. Maybe we'll have something there for v17. regards, tom lane
I wrote: > Hans Buschmann <buschmann@nidsa.net> writes: >> I just noticed your new efforts in this area. >> I wanted to recurr to my old thread [1] considering constant propagation of quals. >> [1] https://www.postgresql.org/message-id/1571413123735.26467@nidsa.net > Yeah, this patch series is not yet quite up to the point of improving > that. That area is indeed the very next thing I want to work on, and > I did spend some effort on it last month, but I ran out of time to get > it working. Maybe we'll have something there for v17. BTW, to clarify what's going on there: what I want to do is allow the regular equivalence-class machinery to handle deductions from equality operators appearing in LEFT JOIN ON clauses (maybe full joins too, but I'd be satisfied if it works for one-sided outer joins). I'd originally hoped that distinguishing pre-nulled from post-nulled variables would be enough to make that safe, but it's not. Here's an example: select ... from t1 left join t2 on (t1.x = t2.y and t1.x = 1); If we turn the generic equivclass.c logic loose on these clauses, it will deduce t2.y = 1, which is good, and then apply t2.y = 1 at the scan of t2, which is even better (since we might be able to turn that into an indexscan qual). However, it will also try to apply t1.x = 1 at the scan of t1, and that's just wrong, because that will eliminate t1 rows that should come through with null extension. My current plan for making this work is to define EquivalenceClass-generated clauses as applying within "join domains", which are sets of inner-joined relations, and in the case of a one-sided outer join then the join itself belongs to the same join domain as its right-hand side --- but not to the join domain of its left-hand side. This would allow us to push EC clauses from an outer join's qual down into the RHS, but not into the LHS, and then anything leftover would still have to be applied at the join. In this example we'd have to apply t1.x = t2.y or t1.x = 1, but not both, at the join. I got as far as inventing join domains, in the 0012 patch of this series, but I haven't quite finished puzzling out the clause application rules that would be needed for this scenario. Ordinarily an EC containing a constant would be fully enforced at the scan level (i.e., apply t1.x = 1 and t2.y = 1 at scan level) and generate no additional clauses at join level; but that clearly doesn't work anymore when some of the scans are outside the join domain. I think that the no-constant case might need to be different too. I have some WIP code but nothing I can show. Also, this doesn't seem to help for full joins. We can treat the two sides as each being their own join domains, but then the join's own ON clause doesn't belong to either one, since we can't throw away rows from either side on the basis of a restriction from ON. So it seems like we'll still need ad-hoc logic comparable to reconsider_full_join_clause, if we want to preserve that optimization. I'm only mildly discontented with that, but still discontented. regards, tom lane
On Tue, Jan 24, 2023 at 12:31 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
I wrote:
> Hans Buschmann <buschmann@nidsa.net> writes:
>> I just noticed your new efforts in this area.
>> I wanted to recurr to my old thread [1] considering constant propagation of quals.
>> [1] https://www.postgresql.org/message-id/1571413123735.26467@nidsa.net
> Yeah, this patch series is not yet quite up to the point of improving
> that. That area is indeed the very next thing I want to work on, and
> I did spend some effort on it last month, but I ran out of time to get
> it working. Maybe we'll have something there for v17.
BTW, to clarify what's going on there: what I want to do is allow
the regular equivalence-class machinery to handle deductions from
equality operators appearing in LEFT JOIN ON clauses (maybe full
joins too, but I'd be satisfied if it works for one-sided outer
joins). I'd originally hoped that distinguishing pre-nulled from
post-nulled variables would be enough to make that safe, but it's
not. Here's an example:
select ... from t1 left join t2 on (t1.x = t2.y and t1.x = 1);
If we turn the generic equivclass.c logic loose on these clauses,
it will deduce t2.y = 1, which is good, and then apply t2.y = 1 at
the scan of t2, which is even better (since we might be able to turn
that into an indexscan qual). However, it will also try to apply
t1.x = 1 at the scan of t1, and that's just wrong, because that
will eliminate t1 rows that should come through with null extension.
Is there a particular comment or README where that last conclusion is explained so that it makes sense. Intuitively, I would expect t1.x = 1 to be applied during the scan of t1 - it isn't like the output of the join is allowed to include t1 rows not matching that condition anyway.
IOW, I thought the more verbose but equivalent syntax for that was:
select ... from (select * from t1 as insub where insub.x = 1) as t1 left join t2 on (t1.x = t2.y)
Thanks!
David J.
"David G. Johnston" <david.g.johnston@gmail.com> writes:
> On Tue, Jan 24, 2023 at 12:31 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> select ... from t1 left join t2 on (t1.x = t2.y and t1.x = 1);
>>
>> If we turn the generic equivclass.c logic loose on these clauses,
>> it will deduce t2.y = 1, which is good, and then apply t2.y = 1 at
>> the scan of t2, which is even better (since we might be able to turn
>> that into an indexscan qual). However, it will also try to apply
>> t1.x = 1 at the scan of t1, and that's just wrong, because that
>> will eliminate t1 rows that should come through with null extension.
> Is there a particular comment or README where that last conclusion is
> explained so that it makes sense.
Hm? It's a LEFT JOIN, so it must not eliminate any rows from t1.
A row that doesn't have t1.x = 1 will appear in the output with
null columns for t2 ... but it must still appear, so we cannot
filter on t1.x = 1 in the scan of t1.
regards, tom lane
On Tue, Jan 24, 2023 at 1:25 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
"David G. Johnston" <david.g.johnston@gmail.com> writes:
> On Tue, Jan 24, 2023 at 12:31 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> select ... from t1 left join t2 on (t1.x = t2.y and t1.x = 1);
>>
>> If we turn the generic equivclass.c logic loose on these clauses,
>> it will deduce t2.y = 1, which is good, and then apply t2.y = 1 at
>> the scan of t2, which is even better (since we might be able to turn
>> that into an indexscan qual). However, it will also try to apply
>> t1.x = 1 at the scan of t1, and that's just wrong, because that
>> will eliminate t1 rows that should come through with null extension.
> Is there a particular comment or README where that last conclusion is
> explained so that it makes sense.
Hm? It's a LEFT JOIN, so it must not eliminate any rows from t1.
A row that doesn't have t1.x = 1 will appear in the output with
null columns for t2 ... but it must still appear, so we cannot
filter on t1.x = 1 in the scan of t1.
Ran some queries, figured it out. Sorry for the noise. I had turned the behavior of the RHS side appearing in the ON clause into a personal general rule then tried to apply it to the LHS (left join mental model) without working through the rules from first principles.
David J.
On Tue, Jan 24, 2023 at 4:38 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Richard, are you planning to review this any more? I'm getting
a little antsy to get it committed. For such a large patch,
it's surprising it's had so few conflicts to date.
Sorry for the delayed reply. I don't have any more review comments at
the moment, except a nitpicking one.
In optimizer/README at line 729 there is a query as
SELECT * FROM a
LEFT JOIN (SELECT * FROM b WHERE b.z = 1) ss ON (a.x = b.y)
WHERE a.x = 1;
I think it should be
SELECT * FROM a
LEFT JOIN (SELECT * FROM b WHERE b.z = 1) ss ON (a.x = ss.y)
WHERE a.x = 1;
I have no objection to get it committed.
Thanks
Richard
the moment, except a nitpicking one.
In optimizer/README at line 729 there is a query as
SELECT * FROM a
LEFT JOIN (SELECT * FROM b WHERE b.z = 1) ss ON (a.x = b.y)
WHERE a.x = 1;
I think it should be
SELECT * FROM a
LEFT JOIN (SELECT * FROM b WHERE b.z = 1) ss ON (a.x = ss.y)
WHERE a.x = 1;
I have no objection to get it committed.
Thanks
Richard
Richard Guo <guofenglinux@gmail.com> writes:
> Sorry for the delayed reply. I don't have any more review comments at
> the moment, except a nitpicking one.
> In optimizer/README at line 729 there is a query as
> SELECT * FROM a
> LEFT JOIN (SELECT * FROM b WHERE b.z = 1) ss ON (a.x = b.y)
> WHERE a.x = 1;
> I think it should be
> SELECT * FROM a
> LEFT JOIN (SELECT * FROM b WHERE b.z = 1) ss ON (a.x = ss.y)
> WHERE a.x = 1;
Oh, good catch, thanks.
> I have no objection to get it committed.
I'll push forward then. Thanks for reviewing!
regards, tom lane
On Mon, Jan 23, 2023 at 03:38:06PM -0500, Tom Lane wrote: > Richard, are you planning to review this any more? I'm getting > a little antsy to get it committed. For such a large patch, > it's surprising it's had so few conflicts to date. The patch broke this query: select from pg_inherits inner join information_schema.element_types right join (select from pg_constraint as sample_2) on true on false, lateral (select scope_catalog, inhdetachpending from pg_publication_namespace limit 3); ERROR: could not devise a query plan for the given query
On Mon, Feb 13, 2023 at 7:58 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
The patch broke this query:
select from pg_inherits inner join information_schema.element_types
right join (select from pg_constraint as sample_2) on true
on false, lateral (select scope_catalog, inhdetachpending from pg_publication_namespace limit 3);
ERROR: could not devise a query plan for the given query
Thanks for the report! I've looked at it a little bit and traced down
to function have_unsafe_outer_join_ref(). The comment there says
* In practice, this test never finds a problem ...
* ...
* It still seems worth checking
* as a backstop, but we don't go to a lot of trouble: just reject if the
* unsatisfied part includes any outer-join relids at all.
This seems not correct as showed by the counterexample. ISTM that we
need to do the check honestly as what the other comment says
* If the parameterization is only partly satisfied by the outer rel,
* the unsatisfied part can't include any outer-join relids that could
* null rels of the satisfied part.
The NOT_USED part of code is doing this check. But I think we need a
little tweak. We should check the nullable side of related outer joins
against the satisfied part, rather than inner_paramrels. Maybe
something like attached.
However, this test seems to cost some cycles after the change. So I
wonder if it's worthwhile to perform it, considering that join order
restrictions should be able to guarantee there is no problem here.
BTW, here is a simplified query that can trigger this issue on HEAD.
select * from t1 inner join t2 left join (select null as c from t3 left join t4 on true) as sub on true on true, lateral (select c, t1.a from t5 offset 0 ) ss;
Thanks
Richard
to function have_unsafe_outer_join_ref(). The comment there says
* In practice, this test never finds a problem ...
* ...
* It still seems worth checking
* as a backstop, but we don't go to a lot of trouble: just reject if the
* unsatisfied part includes any outer-join relids at all.
This seems not correct as showed by the counterexample. ISTM that we
need to do the check honestly as what the other comment says
* If the parameterization is only partly satisfied by the outer rel,
* the unsatisfied part can't include any outer-join relids that could
* null rels of the satisfied part.
The NOT_USED part of code is doing this check. But I think we need a
little tweak. We should check the nullable side of related outer joins
against the satisfied part, rather than inner_paramrels. Maybe
something like attached.
However, this test seems to cost some cycles after the change. So I
wonder if it's worthwhile to perform it, considering that join order
restrictions should be able to guarantee there is no problem here.
BTW, here is a simplified query that can trigger this issue on HEAD.
select * from t1 inner join t2 left join (select null as c from t3 left join t4 on true) as sub on true on true, lateral (select c, t1.a from t5 offset 0 ) ss;
Thanks
Richard
Вложения
Richard Guo <guofenglinux@gmail.com> writes:
> Thanks for the report! I've looked at it a little bit and traced down
> to function have_unsafe_outer_join_ref(). The comment there says
> * In practice, this test never finds a problem ...
> This seems not correct as showed by the counterexample.
Right. I'd noticed that the inner loop of that function was not
reached in our regression tests, and incorrectly concluded that it
was not reachable --- but I failed to consider cases where the
inner rel's parameterization depends on Vars from multiple places.
> The NOT_USED part of code is doing this check. But I think we need a
> little tweak. We should check the nullable side of related outer joins
> against the satisfied part, rather than inner_paramrels. Maybe
> something like attached.
Agreed.
> However, this test seems to cost some cycles after the change. So I
> wonder if it's worthwhile to perform it, considering that join order
> restrictions should be able to guarantee there is no problem here.
Yeah, I think we should reduce it to an Assert check. It shouldn't be
worth the cycles to run in production, and that will also make it easier
to notice in sqlsmith testing if anyone happens across another
counterexample.
Pushed that way. Thanks for the report and the patch!
regards, tom lane
On Mon, Feb 13, 2023 at 03:33:15PM +0800, Richard Guo wrote: > On Mon, Feb 13, 2023 at 7:58 AM Justin Pryzby <pryzby@telsasoft.com> wrote: > > The patch broke this query: > > > > select from pg_inherits inner join information_schema.element_types > > right join (select from pg_constraint as sample_2) on true > > on false, lateral (select scope_catalog, inhdetachpending from > > pg_publication_namespace limit 3); > > ERROR: could not devise a query plan for the given query > BTW, here is a simplified query that can trigger this issue on HEAD. > > select * from t1 inner join t2 left join (select null as c from t3 left > join t4 on true) as sub on true on true, lateral (select c, t1.a from t5 > offset 0 ) ss; It probably doesn't need to be said that the original query was reduced from sqlsmith... But I mention that now to make it searchable. Thanks, -- Justin
Hello!
I'm having doubts about this fix but most likely i don't understand something.
Could you help me to figure it out, please.
The thing is that for custom scan nodes as readme says:
"INDEX_VAR is abused to signify references to columns of a custom scan tuple type"
But INDEX_VAR has a negative value, so it can not be used in varnullingrels bitmapset.
And therefore this improvement seems will not work with custom scan nodes and some
extensions that use such nodes.
If i'm wrong in my doubts and bitmapset for varnullingrels is ok, may be add a check before
adjust_relid_set() call like this:
@@ -569,9 +569,10 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context)
{
if (var->varno == context->rt_index)
var->varno = context->new_index;
- var->varnullingrels = adjust_relid_set(var->varnullingrels,
- context->rt_index,
- context->new_index);
+ if (context->rt_index >= 0 && context->new_index >= 0)
+ var->varnullingrels = adjust_relid_set(var->varnullingrels,
+ context->rt_index,
+
With the best wishes,
--
Anton A. Melnikov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
"Anton A. Melnikov" <aamelnikov@inbox.ru> writes:
> The thing is that for custom scan nodes as readme says:
> "INDEX_VAR is abused to signify references to columns of a custom scan tuple type"
> But INDEX_VAR has a negative value, so it can not be used in varnullingrels bitmapset.
> And therefore this improvement seems will not work with custom scan nodes and some
> extensions that use such nodes.
Under what circumstances would you be trying to inject INDEX_VAR
into a nullingrel set? Only outer-join relids should ever appear there.
AFAICS the change you propose would serve only to mask bugs.
regards, tom lane
Hello and sorry for the big delay in reply! On 04.05.2023 15:22, Tom Lane wrote: > AFAICS the change you propose would serve only to mask bugs. Yes, it's a bad decision. > Under what circumstances would you be trying to inject INDEX_VAR > into a nullingrel set? Only outer-join relids should ever appear there. The thing is that i don't try to push INDEX_VAR into a nullingrel set at all, i just try to replace the existing rt_index that equals to INDEX_VAR in Var nodes with the defined positive indexes by using ChangeVarNodes_walker() function call. It checks if the nullingrel contains the existing rt_index and does it need to be updated too. It calls bms_is_member() for that, but the last immediately throws an error if the value to be checked is negative like INDEX_VAR. But we are not trying to corrupt the existing nullingrel with this bad index, so it doesn't seem like an serious error. And this index certainly cannot be a member of the Bitmapset. Therefore it also seems better and more logical to me in the case of an index that cannot possibly be a member of the Bitmapset, immediately return false. Here is a patch like that. With the best regards, -- Anton A. Melnikov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
Вложения
[ back from PGCon ... ]
"Anton A. Melnikov" <aamelnikov@inbox.ru> writes:
> On 04.05.2023 15:22, Tom Lane wrote:
>> Under what circumstances would you be trying to inject INDEX_VAR
>> into a nullingrel set? Only outer-join relids should ever appear there.
> The thing is that i don't try to push INDEX_VAR into a nullingrel set at all,
> i just try to replace the existing rt_index that equals to INDEX_VAR in Var nodes with
> the defined positive indexes by using ChangeVarNodes_walker() function call.
Hmm. That implies that you're changing plan data structures around after
setrefs.c, which doesn't seem like a great design to me --- IMO that ought
to happen in PlanCustomPath, which will still see the original varnos.
However, it's probably not worth breaking existing code for this, so
now I agree that ChangeVarNodes ought to (continue to) allow negative
rt_index.
> Therefore it also seems better and more logical to me in the case of an index that
> cannot possibly be a member of the Bitmapset, immediately return false.
> Here is a patch like that.
I do not like the blast radius of this patch. Yes, I know about that
comment in bms_is_member --- I wrote it, if memory serves. But it's
stood like that for more than two decades, and I believe it's caught
its share of mistakes. This issue doesn't seem like a sufficient
reason to change a globally-visible behavior.
I think the right thing here is not either of your patches, but
to tweak adjust_relid_set() to not fail on negative oldrelid.
I'll go make it so.
regards, tom lane
On 08.06.2023 19:58, Tom Lane wrote: > I think the right thing here is not either of your patches, but > to tweak adjust_relid_set() to not fail on negative oldrelid. > I'll go make it so. Thanks! This fully solves the problem with ChangeVarNodes() that i wrote above. > Hmm. That implies that you're changing plan data structures around after > setrefs.c, which doesn't seem like a great design to me --- IMO that ought > to happen in PlanCustomPath, which will still see the original varnos. My further searchers led to the fact that it is possible to immediately set the necessary varnos during custom_scan->scan.plan.targetlist creation and leave the сustom_scan->custom_scan_tlist = NIL rather than changing them later using ChangeVarNodes(). This resulted in a noticeable code simplification. Thanks a lot for pointing on it! Sincerely yours, -- Anton A. Melnikov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company