I think that a lot of people readjusted their expectations based on the no. 1 seed, which is understandable. Even if most people knew that Cleveland was better, they probably expected the Celtics to at least be at the top of that second tier which includes Toronto and Washington.
I wouldn't mind the Celtics losing to a team that only finished four games behind them record-wise and has a starting lineup of one overall #1 pick, two overall #3 picks, another lottery pick and an accomplished center in his prime. However, it would still be a little disappointing in that it makes it easier to sell free agents and trade targets on coming here when you can definitively say this is the team that is closest to challenging LeBron in a couple years or so.
Also, it's frustrating when it doesn't appear that certain players are playing up to their potential. It wasn't really widely publicized until recently that Bradley was dealing with injuries, for example. Crowder has taken a step back defensively and Smart hasn't really upped his game for the playoffs.
Nobody is really accusing them of not playing hard, just that they should play up to their ability or better. What we should acknowledge though is that Washington is probably a little better than we thought, especially defensively.