Let me preface with I think Ainge may take Brown at three. I wouldn't be happy, but I could see it.
However, you're confusing being able to dunk with power as being able to finish at the rim. Jaylen Brown is a subpar, or at least was a subpar finisher for Cal. He shot 45% at the rim because he's a bit wreckless and will throw up wild shots in traffic.
I also don't view him as a plus defender at this point. He may be able to hold his own, but this guy is clearly living on a reputation vs. anything quantifiable he did up until this point. Absolutely he does have the tools to be great defender but he's getting overrated right now. He doesn't have the defensive intangibles like a Justise Winslow or a Marcus Smart.
I don't hate Brown if we were at 8-12 but I wouldn't take him at 3. He's got good but not freakish size/length, is erratic and has wobbly mechanics on his shot with no in between game. Overrated feel for the game presently, as well. Solid prospect but not someone 'I'd risk our pick on.