People who want Irving gone should absolutely want Rozier to stay and be the starting PG. After all, his main beef was seemingly with Kyrie. If Rozier is starting, the expectation should be that he will be more free to be himself and more team oriented.
I should clarify that I absolutely want Irving back, but wanting both of them gone doesn't make much sense. It's really Team Kyrie vs Team Rozier and Team Rozier seems to be winning out - even if people don't realize that's what they really want.