Обсуждение: Re: Fastest memmove in C
Just came across this excellent research work done on memmove by T Herselman.
Bearing in mind memory is constantly moved in any database. This excellent set of library would be very handy for those who work
at the deep end of postgresql.
In a nut shell, the library checks the hardware available ,32/64 Intel (i core with SSE features loadu/storeu)) AMD then uses different function depending on the size of memory block
that has to be moved. The article also claims that the library also checks for signed and unsigned memory block, I haven’t checked this.
Excellent research and could be well worth checking out. As it could improve the performance of postgresql engine.
Please check the article for all your questions.
http://www.codeproject.com/Articles/1110153/Apex-memmove-the-fastest-memcpy-memmove-on-x-x-EVE
FarjadFarid(ChkNet) wrote: > Excellent research and could be well worth checking out. As it could > improve the performance of postgresql engine. 0) We certainly do a lot of memory copying. 1) this work is under the "Code Project Open License" which doesn't look compatible with our Postgres license on first blush. Maybe T Herselman would agree to share under the Postgres licence, which would make things easier. 2) from the description, the code is probably tailored to specific compilers. Even if it's faster now in some mainstream compilers, it won't be in yesterday or tomorrow's ones. Also, there are probably going to be variations depending on CPU features. 3) How much faster does Postgres get when you replace memcpy/memmove with these Apex versions? If the percentage to be gained is small, then there's probably no point in spending a lot of effort into providing platform-specific implementations of these things anyway. If you have enough interest in this topic, you could try using these Apex versions in Postgres and measuring an improvement, to get a discussion going. -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Hi Álvaro, Based on the article. This research has been tested with various hardware. The code automatically detects the hardware and based on the size of memory to be copied it will choose one of 3 routines. Regarding license I don't much problem with "Code Project Open License". Why not contact the author. He sounds like a good techie person. Álvaro wrote >2) from the description, the code is probably tailored to specific compilers. Even if it's faster now in some mainstream compilers, it won't be in yesterday or tomorrow's ones. Also, there >are probably going to be variations depending on CPU features. Based on the article he has checked this libraries from Microsoft and GCC/LLVM/Clang/stdlib and their compilers. As this is written in C. It should be easy to transfer it to whatever you like. In the past, on a pretty badly structure programmed, I have used two compilers on a single project simply to catch different issues. I don't see this as a major issue and the advantage could be considerable. I have not tested this with postgresql. The article suggests the following benchmark >These are only ESTIMATES taken from the original article, which did not include my fastest implementations which were yet to come; so these estimates are from older slower variations. >large copy (>= 128 bytes) >32-bit = 40% faster >64-bit = 30% faster >small copy (< 128-bytes) >15%~40% faster >These are very old numbers! The functions included here are faster! Depending on hardware of course! These should provide a great overall performance boost. Hope this might help. Best Regards -----Original Message----- From: pgsql-general-owner@postgresql.org [mailto:pgsql-general-owner@postgresql.org] On Behalf Of Alvaro Herrera Sent: 06 July 2016 22:15 To: FarjadFarid(ChkNet) <farjad.farid@checknetworks.com> Cc: pgsql-general@postgresql.org Subject: Re: [GENERAL] Fastest memmove in C FarjadFarid(ChkNet) wrote: > Excellent research and could be well worth checking out. As it could > improve the performance of postgresql engine. 0) We certainly do a lot of memory copying. 1) this work is under the "Code Project Open License" which doesn't look compatible with our Postgres license on first blush. Maybe T Herselman would agree to share under the Postgres licence, which would make things easier. 2) from the description, the code is probably tailored to specific compilers. Even if it's faster now in some mainstream compilers, it won't be in yesterday or tomorrow's ones. Also, there are probably going to be variations depending on CPU features. 3) How much faster does Postgres get when you replace memcpy/memmove with these Apex versions? If the percentage to be gained is small, then there's probably no point in spending a lot of effort into providing platform-specific implementations of these things anyway. If you have enough interest in this topic, you could try using these Apex versions in Postgres and measuring an improvement, to get a discussion going. -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
On Thu, Jul 7, 2016 at 6:12 AM, FarjadFarid(ChkNet) <farjad.farid@checknetworks.com> wrote: > > > Hi Álvaro, > > Based on the article. This research has been tested with various hardware. > The code automatically detects the hardware and based on the size of memory > to be copied it will choose one of 3 routines. > > Regarding license I don't much problem with "Code Project Open License". Why > not contact the author. He sounds like a good techie person. > > Álvaro wrote >>2) from the description, the code is probably tailored to specific > compilers. Even if it's faster now in some mainstream compilers, it won't > be in yesterday or tomorrow's ones. Also, there >are probably going to be > variations depending on CPU features. > > Based on the article he has checked this libraries from Microsoft and > GCC/LLVM/Clang/stdlib and their compilers. As this is written in C. It > should be easy to transfer it to whatever you like. > In the past, on a pretty badly structure programmed, I have used two > compilers on a single project simply to catch different issues. I don't see > this as a major issue and the advantage could be considerable. > > I have not tested this with postgresql. > > The article suggests the following benchmark Well, testing is the key here. Microbechmarks demonstrating the value are not enough; proven statistically relevant benchmarks generated from postgres are the data points needed to make an assessment. My recommendation would be to dynamically link in these routines to postgres and start running a battery of memory heavy tests (start with pgbench -S on a small database). Ideally you could tease out some measurable benefit; license discussions and formal integration strategies are secondary to that IMO. merlin
* Merlin Moncure (mmoncure@gmail.com) wrote: > Well, testing is the key here. Microbechmarks demonstrating the value > are not enough; proven statistically relevant benchmarks generated > from postgres are the data points needed to make an assessment. My > recommendation would be to dynamically link in these routines to > postgres and start running a battery of memory heavy tests (start with > pgbench -S on a small database). Ideally you could tease out some > measurable benefit; license discussions and formal integration > strategies are secondary to that IMO. While I agree with this, I'm trying to figure out why this isn't being incorporated into glibc instead..? Perhaps I missed it, but I didn't see a discussion of that in the article. I'd certainly rather rely on glibc if we can, though I know that we've ended up implementing our own routines at times too. On the other hand, if there's a reason the glibc folks don't want this, we should consider that.. Thanks! Stephen
Вложения
Stephen Frost <sfrost@snowman.net> writes: > While I agree with this, I'm trying to figure out why this isn't being > incorporated into glibc instead..? Indeed --- it seems far more useful to integrate this sort of work at the libc level than to ask applications to support their own replacements for libc routines. regards, tom lane
As the article demonstrated, performance of memove libraries is dependent on the size of memory copied and the hardware. Therefore it is both a software and hardware function. This post was simply put out here as something that might be of use. It is Ok if it is of no interest. -----Original Message----- From: pgsql-general-owner@postgresql.org [mailto:pgsql-general-owner@postgresql.org] On Behalf Of Tom Lane Sent: 07 July 2016 18:02 To: Stephen Frost <sfrost@snowman.net> Cc: Merlin Moncure <mmoncure@gmail.com>; FarjadFarid(ChkNet) <farjad.farid@checknetworks.com>; Alvaro Herrera <alvherre@2ndquadrant.com>; PostgreSQL General <pgsql-general@postgresql.org> Subject: Re: [GENERAL] Fastest memmove in C Stephen Frost <sfrost@snowman.net> writes: > While I agree with this, I'm trying to figure out why this isn't being > incorporated into glibc instead..? Indeed --- it seems far more useful to integrate this sort of work at the libc level than to ask applications to support their own replacements for libc routines. regards, tom lane -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general