A couple of thoughts on the defense:
1) Bradley is a legitimate loss. He could go toe to toe with some of the best guards in the league and had the lateral quickness to provide resistance against elite ball-handlers.
2) Crowder's defense has been overstated for the past year plus. He stopped playing as hard on that end of the floor once he became established. He does have some toughness/physicality, but he is certainly not irreplaceable.
3) Smart is our best defender, and I would place him comfortably in the top 10 defenders in the NBA, so I view us as having one top option to place on another team's best scorer as long as they are 6'6" or smaller (ideally).
4) Brown/Hayward/Morris/Horford/Smart/Tatum/Semi/Yabu/etc. give us rare size, versatility, and athleticism to switch most screens when we want to. That ability is very valuable, especially in the playoffs and in late game situations.
5) Brown would seem to have high defensive upside with his athleticism, size, and effort. His defensive development is key if we are going to be a good, rather than just okay, defensive team.
6) Our biggest issue is with defensive rebounding. As others have noted, that can turn a good defensive team into a mediocre one based on extra possessions.
If I had to make a prediction, I would say that our FG% allowed will be high (in the 3-10 range) because of our versatility, athleticism, and ability to switch (and only having Irving as a real black hole defensively in our rotation). However, I am worried about our defensive rebounding dragging down our overall defensive efficiency and points allowed, so I think in terms of holistic stats we will end up in the bottom of that range around 10th overall.
I do think we have a legitimate chance to have one of, if not the, best offense in the league. If we can field a top 3 offense and a top 10 defense, that makes us a legitimate contender. Only Golden State could likely be significantly better on both sides of the ball if we reach our potential.