Unless you’re getting back a star and Timelord was the only way to get said star without including Brown, then you don’t trade him, certainly not for role players. You’d just be trading one hole in the roster for another.
Timelord is probably one of the more valuable starting-level but not star-level players at this point in time given his mix of productivity, age, and super cost-controlled contract moving forward. For Toronto, I’m not sure if I see something there I’m super excited to trade him for knowing that it’d also take away our starting 5.