No one, especially not Brown, and especially not for the reasons you listed. He is our best perimeter defender. He was 10th place in DPOY last year, which means the rest of the league recognizes how good he is. He is versatile enough to block Durant's jumpshot, steal the ball from Curry, and block Love in the post.
Combine that with Brown's 40% from 3 last year, and he is probably one of the best 3/D players in the league. Who are you gonna replace him with that would be an upgrade to either defense or shooting?