It's pretty simple. The teams that have stout fronts that can get to Brady without sending extra rushers tend to do well against him. This is because 1, Brady has not had a strong running game to support him since earlier in his career. 2, Brady is not the most mobile quarterback and while he can break a run once in a blue moon he cannot do it consistently like scrambling QBs can. This also means he can't create as much extra time for himself when eluding the initial rush. If the D also has good corners like the Broncos do, they can allocate double teams to the top targets and force Brady to go to inconsistent third or fourth options.
Honestly though, Brady simply didn't have a good game besides the pass rush getting to him. He tried to force the ball in the first half and those ended up being the difference. He had to know that the Broncos offense was not going to be able to score points and it would be a defensive slugfest, so turnovers were going to be even more critical to avoid.
He also missed some throws, though it's hard to expect him to be perfect when he's constantly afraid the rush is going to strip sack him, or when a blown block forces him to throw the ball earlier than the optimal time. He seemed to zero in on certain guys, either because he felt he didn't have the time to go through his reads or he just panicked.
It was probably a C/C+ game for him. Harsh due to the rush he faced, but fair considering Brady has set the bar so high for his own performances.
The missed read on Gronk being open on the 2 pt. conversion was pretty bad. That was a case where it seemed he just stuck to his initial read and didn't feel confident looking for another option, then stuck it into a real high traffic area.
I would like to see them invest in an OT, a WR to replace LaFell, and a RB this offseason. Dion Lewis is an injury risk and they need insurance for Solder/Vollmer. Vollmer will be 32 next season, his future replacement can't be Marcus Cannon. They need a good WR who is more durable than Edelman, Gronk, and Amendola, all of whom have spotty injury histories. Expecting them to all be healthy like they were in 2014-15 is a big risk.