People forget that we had a maxed out and completely untradeable Vin Baker on the books when Ainge took over. Couple Vin with an openly hostile Antoine, who had just come off probably the worst all-star season ever (20 points on 20 shots per game, a rebounding rate just under 10%), and a hobbled Tony Battie (who couldn't even practice) and you can see why he was looking to call it quits on that core.
From there he looked to buy low on talent with our weak trade offerings (injured Raef, injured Wally, red flag Ricky, injured Googs, disgruntled Payton, disappointing Mihm, returning Toine), take cheap flyers (Mike James, Chucky Atkins, Jumaine Jones, Olowokandi) and stockpile draft picks.
And his most maligned acquisition, Raef, while a big injury risk, wasn't all that bad at the time: still relatively young and a unique talent. How many big men can place top 5 in the league in shot-blocking and hit 3s at 36-40%? He would have been an improvement on Toine if he stayed healthy.
Even Blount looked like a decent signing at the time. The guy was probably the best mid-range shooting big in the league after KG, played decent defense, ran the floor like a gazelle and had just busted his butt to become one of the few bright spots on that super depressing 2003-2004 team.