After some investigation and benchmarking, it looks like the best PIR protocol for this use case is YPIR+SP (from February). On a single compute- and network-constrained server, with users on constrained (and possibly metered) networks, this would amount to providing service to up to 1000 users while keeping latencies reasonable; by (quadratically) scaling the server(s) enough, that could become up to 100,000. That means this method of message routing could definitely work, although I look every day in case new protocols are published.
There’s this post of mine, also this article gives some background on the application of PIR to anonymous messaging. Basically, I’m trying to do a basic version of that, but using a state-of-the-art PIR protocol introduced in this article. It’s still not great performance-wise, but it’s enough to be practical (as stated, many thousands of users given enough resources).