To be honest with you, I don't think the point guard position is especially impactful on modern defense, in general. At that position I mostly care about ability to create turnovers, focus, and effort. IT rates well for me in those categories.
Yes, he's an issue in terms of individual matchups. However, defense should not, in theory, be the main problem for the Celts, and I think defensive weakness at the point guard position is the easiest sort of defensive issue to cover up.
Damian Lillard and Kyrie Irving are also pretty poor defenders, they just don't get criticized for it as much because of their size. When IT is taken advantage of on that end, it stands out more because he's smaller.
I agree with your concerns about his longevity, but I think the Celts may just have to swallow hard and accept that risk. IT is as close as they have to a franchise player right now, and I think if they let him walk they'll have a darned hard time replacing his production. I don't think you can have a really good team in the modern NBA without a high level offensive talent at point guard.