While I don't think that entirely explains it (our rebounding certainly hurts, too), I must admit that I have felt for awhile now that he's struggling to defend the stretch 4's we see. He's been especially bad, at least to the eye test, on PnR coverage. He constantly overhelps on the ball-handler, and he seems slow on coverages and rotations a lot of times, too.
KO or JJ would be an interesting starter at the 4.
This also makes you think about adding a Cousins/Noel type, and if they'd actually be a good pairing or not. Noel can at least for sure guard out on the perimeter, but I question a Horford/Cousins pairing defensively.
Would love a Blake Griffin at the 4 next to him, though that seems a pipe dream at the moment. Perhaps a Milsap possibility wouldn't be too bad after all.