WIP Fix issue #7674 about UPDATE SET(..), with indirection #7675
Conversation
Queries of the form: ``` UPDATE ... SET (a, b) = (SELECT '1', '2') UPDATE ... SET (b, a) = (SELECT '2', '1') ``` Should do the same thing, but currently the order of the attributes in `SET (...)` as rewriten for pushdown is only based on the physical ordering of the attributes in the relation. This leads to several subtle problems including situation where a DROPped then reADDed attributes will change its placement in the attribute list. There are maybe more tests to add in other situation where a SET (MULTIEXPR) is possible, though UPDATE form is pretty unique as alternatives are not supported by citus: `(INSERT .. ON CONFLICT SET (a,b).....`
ruleutils in Citus is based on PostgreSQL source code, but in PostgreSQL ruleutils is not used at the planner stage. For instance, it is assumed after parser that targetList are ordered as they were read, but it's not true after rewriter, the resulting rewrite tree is then provided to planner (and citus), but the ordering of the list is not granted anymore. It's similar to others previous issues reported and still open, as well as to other bugfixes/improvment over time, the most noticable being the ProcessIndirection() which is for domain and similar. However, the implications of this bug are huge for users of `UPDATE SET (...)`: 1. if you used to order by columns order, you're maybe safe: `SET (col1, col2, col3, ...)` 2. if you used not to order by column order: `SET (col2, col1, col3, ...)` then you probably found a problem, or you have one. Note about 1. that despite appearance and your QA, you are at risk: if physical columns ordering is changed (for example after DROPping/ADDing some), the same query which use to apparently works well will silently update other columns... As it is this code is not optimized for performance, not sure it'll be needed.
c2main
force-pushed
the
fix-update-multi-attr-select
branch
from
099eab0
to
73f4f67
Compare
|
@naisila just a message to help you find this PR when you'll get a chance to look at it... |
There was a problem hiding this comment.
#5692 is also trying to fix quite a similar problem. Reviewers had objected to that PR because we want to avoid modifying ruleutils as much as possible and try to fix such issues through the distributed planner. Have you tried to investigate this issue through the lens of the distributed planner hook?
Thank you for the feedback. No I didn't look at distributed planner hook, I will evaluate this approach. |
I just looked:
I understand the motivation to not touch too much ruleutils, but also my understanding is that ruleutils has been imported precisely to achieve a goal which is not achieved because it contains several limitation due to reasons well explained in Onur's PR (tree rewritten and optimized by postgres before being in the hands of citus). By touching only distributed planner I fear that the bug will just reappear in some corner case situation, but I am still exploring, sharing here as I'm sure those points have been discussed in the past but I didn't search history... |
|
If the patch is elsewhere then I suppose it will be required to do essentially what IMHO, this is a "codepath" expecting to diverge a lot from upstream because they deserve distinct purpose, for many cases the solution is just to not allow that in Citus and maybe it's an alternative to consider here. Like some other similar queries. Maybe I missed alternatives ? |
|
Thanks for your investigations. The team will look at this in more detail as soon as we get a chance. |
This reverts commit 73f4f67.
I've tried to patch in lower layer, it's possible, but less convenient and more error prone. If it works for you this way I can squash the commits and maybe improve names/comments ? |
It'll probably be interesting to add more test related to indirection here.
We may need to reorder parts of the planner tree we are receiving in dsitributed_planner hook because it has been optimized by PostgreSQL. However we only need to rewrite tree which will be used to produce a query string, otherwise it's pointless. Proceed very early in the planner hook, which is probably a good place to pull up some similar fixes applied here and there. See the exported function from citus_ruleutils.c: void RebuildParserTreeFromPlannerTree(Query *query) Added a new include in citus_ruleutils.c: miscadmin.h from PostgreSQL, because it contains the check_stack_depth() function which is a good idea to use as we don't know the size of the query..
c2main
force-pushed
the
fix-update-multi-attr-select
branch
from
6d69c20
to
1cf93c1
Compare
Issue reported to Data Bene support. See #7674
The fix looks good, at least it breaks nothing and the bug not visible.
I am unsure to add all regressions tests where I did, maybe their own file?
Also the PR is not yet ready regarding citus_indent, but if you can already provide some feedback.