Currently we copy `users_who_share_room` needlessly about three times, which is expensive when the set is large (which it can easily be).