I have been working on adding support for columnar format export to Postgres to speed up analytics queries.
I've created an extension that achieves this functionality here.
I"m looking to improve the performance of this extension to enable drop-in analytics support for Postgres. Some immediate improvements I have in mind are:
- Reduce memory consumption when exporting table data to columnar format
- Create a native planner / execution hook that can read columnar data with vectorised operations.
It would be very helpful if you could take a look and suggest improvements to the extension.
Hopefully, this extension can be shipped by default with postgres at some point in the future.