Would it work to use ALTER SYSTEM (postgresql.auto.conf)? Or is that copied by pg_rewind also?
Even if so, using the ALTER SYSTEM command after pg_rewind might be an easy way of correcting the port back to what you want without doing more complicated processing or re-writing the entire postgresql.conf.