My personal opinion: for Stevens all mistakes aren't equal and defensive mistakes are the worst mistakes you can make.
With all mistakes not being equal, the biggest complaints for guys like Morris and Rozier from the fans seems to be their shot selection. I think this bothers Stevens the least. Also getting scored on or getting caught in a pick and roll or getting beat on a rotation, etc. doesn't seem to be as big of a sin as just flat out making the wrong defensive play. Just my personal opinion of how Stevens thinks. Also maybe he expects more from Brown so he's harder on him?
While Brown's a good 1-on-1 defender, I think he makes a lot of mistakes from a help/team defensive perspective. Started noticing this when I asked the same question a few games ago. And now I specifically see things like this from Brown now that I'm watching closely for them (but to be fair, not watching others with the same microscope, also there's a lot of plays where Brown plays stellar D as well):
....deleted stuff ...
I think if we were to go back and look at games where Brown doesn't play a lot of minutes, you'll see a lot of plays like these. These are the plays that get him yanked by Stevens. Do others make as many mistakes or as grievous of mistakes? I don't know. But this is what I think keeps Brown from getting more minutes.
You can anecdotally find defensive mess-ups by any player you want if you go back and search for them. Even guys like Al Horford make the occasional goof.
Since January 1, the team's defensive rating with Jaylen on the floor is 107.9 points per 100 possessions. With Jaylen on the bench it has been a horrific 120.7. That's a gigantic 12.8 point difference!
For comparison, the team defense has been 2.5 points
worse with Morris on the floor as opposed to off.
That suggests that, in the aggregate, maybe Jaylen isn't anywhere close to the defensive screw-up that an anecdotal selection of plays would portray?
On an individual level, Jaylen is currently second on the team in shot contention, causing opponent's to shoot at a -3.5 differential FG%. That's on 9.3 DFGA per game, which is a lot considering his minutes are significantly less than a full time starter. That's reflective of the fact that he's often tasked with guarding one of the other team's better scorers. He's third on the team in Defensive Win Shares per 48.
If Brad really is dinging Jaylen's minutes based on the occasional defensive mistake, without really understanding the overall aggregate impact on the team defense, then he's ... perhaps not as smart as many would like to think?