Kelly's not a great defender right now, like most rookies. But, I don't think his problem right now stem from not being athletic enough. Again, like most rookies, it seems to me that his problem is that he falls asleep and/or gets overwhelmed at times.
He's plenty athletic for a seven footer. He may not have the highest vertical or run the fastest 40, but he's very mobile. That counts as athleticism in my book.