That is just bad logic. If all three refs are looking at the ball - who is watching the rest of the court? Only one ref needs to see something to make a call.
Clearly all three officials saw the play, otherwise all three would not have huddled together to discuss the call. Yes, you only need one official to make a call, but that does not mean the other two didn't see anything.
What do you think went on in the discussion?
Ref 1 - "You see it hit the rim?"
Ref 2 - "I was watching Rafer on the wing."
Ref 3 - "I was looking at the release."
Ref 1 - "Well, I guess I'll make the call by myself. Good huddle, guys."
Believe it or not, as blind as they may be sometimes, officials do have the ability to focus their vision on a number of places in a few seconds. You can easily divert your focus from one area to another, while still maintaining peripheral vision of the court.