I don't think that IT is as much of a defensive liability as people say. He used to be, but like Evan Turner, he has improved. He gets back in transition, stays in front of his man, doesn't foul much, and forces a surprising number of turnovers. He isn't a threat to block jumpshots, and there are some matchups that are clearly bad for him. Relative to AB and MS, he's a bad defender. Relative to a typical starter, I think he's below-average but not a liability.
I love watching Smart and I hope he gets 30 minutes every night, but IT should continue to start.