Speed up fetching latest stream positions via cache (#17606)

The idea is to engineer it so that the vast majority of the rooms can
stay in the cache, so we can just ignore them.