The server was AWS's "serverless" Aurora service, I'm not sure where logs for that would be found. I did not see any error logs generated.
I found a solution, but I still think the docs do not illustrate the regression, at the very least. To resolve this issue I had to paste my existing password into the master password of the server. I was able to connect to the database after that "change".
No configuration changes, just re-inputting my existing password into the server, then I was able to remotely connect with psql 14.