Turns out deferred.cancel sometimes throws, so we do that last to ensure
that we always do resolve the new deferred.