In terms of Boston sports, my wife moved to Boston with no sports allegiances of any kind. We went to a few games at Fenway, but she found the fans obnoxious at best, and passively racist at worst, and we stopped going. We went to minor league games instead. Pawtucket was great, but New Hampshire was as bad as Fenway, and this was AA baseball.
We didn't go to any Celtics games while she was there, but my one Celtics game in 2002 left me with a similar bad taste (except the fans were blatantly homophobic instead of passively racist). Despite them being my favorite team, and living in Boston for another decade, I never returned to the Garden (or Fleet Center/whatever name it went by.)
We moved to DC in 2012. Nats fans are much nicer than Sox fans. I can show up to a Wizards game in a Celtics shirt, and no one says anything, except maybe some playful banter. Minor-league baseball is as family-friendly as it should be, unlike we found it to be at the games we went to in New Hampshire.
Maybe for some people there is a history that goes back decades, but our exposure to and knowledge of Boston fandom is largely from the last decade or so, and even though I'm a fan of the Celtics, my opinion of Boston fans isn't that far off from those who root for rival teams.
Mind you, I don't think it's all fans, and it's probably not a majority of fans, but there are enough to create a very off-putting atmosphere in a way that we haven't experienced elsewhere. (We've also been to MLB games together in Pittsburgh, Baltimore, and the Mets. We haven't seen the Yankees because we imagine it would be similar to Boston.)
And that's not to insult Boston. We'd move back tomorrow if we could. But it is real.
Was the NH team the Fisher Cats, the Jays' AA club in Manchester?
I find that interesting because I've been to more sporting events than I can count and I've literally never heard a racist comment from anyone.
I've never been to a Fisher Cats game, but I've been to dozens of Sea Dogs games, and the fans are classy and great. Same with the Red Claws and Pirates. I even went to a WWE event recently (not a fan, but it was surprisingly awesome). Maybe it's because Portland is such a nice city, but the fans come from all over, so you'd think there would be some racists.
The only game I've been to in Manchester was a Celtics preseason game, there were no problems there. But Manchester overall is a bit economically depressed, and there are some unpleasant characters around.
Never seen any racism at the UNH football and hockey games in Durham.
Only been to a couple pro games in Boston, but I didn't hear any racism or homophobia. Maybe it was a small sample size, and I've been lucky.
On my travels I did see both the Toronto Blue Jays and Anaheim Angels play in their respective towns. The fans definitely seemed less intoxicated and more relaxed than in Boston. There were a remarkable number of Jays jerseys around Toronto (and some Expos ones too!), but they are certainly a mellower breed.
I think this local intensity - mostly for the pro teams - makes the bad apples that much worse, but it doesn't necessarily mean racism is more prevalent, it just might be more open due to the level of "passion" and intoxication.