I'm just taking a wild guess, but maybe when he is guarding bigger 2 guards verses opposing point guards his defense takes a hit. I would be curious to know if the advanced stats can break it down by what position he is guarding, or even height of opposing player.
JBCat,
I think you are onto something regarding AB having to guard a lot of taller SG's!!! I think if Avery was a little stronger, he could get away with it better like Smart does in guarding SF's, but then he'd likely lose some of his quickness.
Smitty77