Thank you so much for your explanation. I realized that the gist index was used many many times. So, I couldn't drop the gist index permanently.
The point of your question is that it is being used even though it is slower. So the fact that it is used many times doesn't mean anything, we already know it is used. The better question is, are all of those uses ones which could be done just as well or better with the btree? I don't see any good ways to answer that definitively, other than by just trying the drop. But I suspect the answer is 'yes'. As far as I know, the only reason to use btree_gist is so you can get access to multi-column gist indexes in which one of the *other* columns needs/benefits from gist. Using it for a single-column index just doesn't make sense to me.