Can I ask what makes you say this? have you watched a great deal of film including what happens when Gooden try's to defend other teams centers? For instance when watching the milwaukee Bucks this year have you scene Gooden try to defend a center and seen him overpowered or not long enough to contest a shot? Box score watching can only go so far.
Yes, I have watched him in Milwaukee + the Clippers + the Bulls + Mavericks + Spurs + Cavs + Magic going all the way back to his rookie season in Memphis.
The problem has consistent throughout his career. His main problem in individual defense is his lack of power making him vulnerable against players with bigger size. They push him around, get deep position in the paint and higher percentage shots as a result.
However, the man-to-man defense is not the biggest problem with Gooden. It's his team defense. He is frequently out of position, late on defensive rotations and provides no shot-blocking presence at the basket.
When Gooden is in the game, perimeter players on dribble penetration can frequently get high percentage shots at the rim because he was too late or in an poor position to properly contest the shot. His poor defensive positioning also leads to bountiful opportunities for perimeter slashers to dump the ball off to big men rolling to the basket for easy garbage baskets.
Drew Gooden is wholly inadequate to be a team's last line of defense because of his atrocious team defense. They could live with his weak man-to-man D, especially against lesser offensive threats, if he played solid team defense ... but that is simply not the case.
In order to allow Drew Gooden to be (somewhat) effective, Gooden needs to be kept at the power forward position alongside someone who is an above average defender and is preferably a shot-blocking presence.