Espn rankings

[fairweatherfan]

Yet another statistics strawman thread, eh?  Rather than defend the formula for the umpteenth time (it's publicly available, there's more than enough info to figure out why teams are where they are), here are some things I'd like tweaked in it.

- some sort of diminishing return for blowouts.  I don't know the data on this, but I'd say the difference between winning by 1 and by 6 is more meaningful than the difference between winning by 21 and 26.  The model treats them the same but there's a lot more noise that goes into settling the final score in a blowout. 

- a similar weighted return for beating bad teams.  I'm guessing that teams that blow out bad teams but constantly lose close ones to good teams (ie Miami) don't tend to do well in the playoffs.  I'm guessing the predictive validity of defeating teams you might face in the playoffs is higher than stomping on the Wizards a bunch of times. 

I'm not sure if this should be symmetrical for losing to bad teams, though, as it seems pretty common for top veteran teams (like us last year) to have problems with that.


It seems like both of these would help rein in surprising results a bit and strengthen the model.

[BballTim]

ESPN has two sets of rankings (Hollinger's math-based model with daily updates and Stein's subjective one), and the Celtics are on top of the Heat in Stein's (2nd overall to the Lakers (last week - they haven't updated the page yet but will today):

http://espn.go.com/nba/powerrankings

It just proves Hollinger's model is seriously flawed (SOS is just raw W-L, with no factors such as a 2nd level W-L).

Yes very flawed. I think the first couple of years he had this, it did a pretty good job of predicting the best team in the NBA. However, there might have been some luck involved there. He needs to relook at his method.

The first couple of years it also had a decent sample size. Things will level out once SoS and the sample gets sufficient. Even now you can see its slowly getting the top teams in the league ranked in some order.

Well last year he had alot of teams over the Lakers and Celtics if i remember correctly.

It would be nice for his model to go a layer deeper to analyze teams performances against good competition. In other words factor in some kind of relation to margin of victory over teams over .500.

Since when is the "model" supposed to predict the finals participants everytime?

It already factors in strength of schedule and margin of victory, to add yet another factor in attempt to "conform" to people's expectations is poor model making. Similar to what the BCS has tried to do.

  Maybe, instead of adding factors to "conform" to people's expectations, he could add another factor to "improve the accuracy" of his prediction.

[pearljammer10]

Its just my opinion, but I think Hollinger is the worst that ESPN has to offer. His rankins on both teams and players are just terrible.

[Fafnir]

  Maybe, instead of adding factors to "conform" to people's expectations, he could add another factor to "improve the accuracy" of his prediction.

Has it been all that inaccurate? I mean I know you have talkedabout how it underestimated the C's because of injuries, but the goal is a fully automated model. Its very hard to deal with injuries in any manner other than subjectively.

[KCattheStripe]

You should not get extra points for whooping on the Nets and the Timberwolves.

[Fafnir]

It seems like both of these would help rein in surprising results a bit and strengthen the model.

Both ideas sound good, but without actually running the adjustments its really hard to say how such changes would alter the rankings. (not to mention their predictive power)

I mean the skewed sample of beating up on bad teams by 10+ points most games and losing to good teams in close ones doesn't occur all that often. Unless it held up as a season wide trend I don't think it'd come into play much.

[BballTim]

  Maybe, instead of adding factors to "conform" to people's expectations, he could add another factor to "improve the accuracy" of his prediction.

Has it been all that inaccurate? I mean I know you have talkedabout how it underestimated the C's because of injuries, but the goal is a fully automated model. Its very hard to deal with injuries in any manner other than subjectively.

  He measures strength of schedule and he measures margin of victory. But I would say that record (and possibly margin of victory) against teams with winning records is a pretty useful determinant. Weeding out teams that kill weak teams and lose most of their games vs better teams seems to be doable, and I really don't think teams like that play all that great in the playoffs.

[nba is the worst]

ESPN has two sets of rankings (Hollinger's math-based model with daily updates and Stein's subjective one), and the Celtics are on top of the Heat in Stein's (2nd overall to the Lakers (last week - they haven't updated the page yet but will today):

http://espn.go.com/nba/powerrankings

It just proves Hollinger's model is seriously flawed (SOS is just raw W-L, with no factors such as a 2nd level W-L).

Yes very flawed. I think the first couple of years he had this, it did a pretty good job of predicting the best team in the NBA. However, there might have been some luck involved there. He needs to relook at his method.

The first couple of years it also had a decent sample size. Things will level out once SoS and the sample gets sufficient. Even now you can see its slowly getting the top teams in the league ranked in some order.

Well last year he had alot of teams over the Lakers and Celtics if i remember correctly.

It would be nice for his model to go a layer deeper to analyze teams performances against good competition. In other words factor in some kind of relation to margin of victory over teams over .500.

Since when is the "model" supposed to predict the finals participants everytime?

It already factors in strength of schedule and margin of victory, to add yet another factor in attempt to "conform" to people's expectations is poor model making. Similar to what the BCS has tried to do.

I disagree that SOS shouldn't have another layer beyond simple W-L record. How about including the W-L record of a team's opponents (same process but a 2nd layer) for just one example? How could that make the model worse?

[Fafnir]

I disagree that SOS shouldn't have another layer beyond simple W-L record. How about including the W-L record of a team's opponents (same process but a 2nd layer) for just one example? How could that make the model worse?

More data doesn't always make model better. Frequently the increased predictive power does not offset the increased noise another variable creates.

I don't know if thats the case here, but you don't know if adding a second layer would improve the model signifigantly.

[papa shuttlesworth]

I think people give Hollinger too much credit for the poor quality of the his PER and power ranking systems.  He did have some input into its design, but it is mostly just a regression.  All he did was pick the variables that most often correlate to success.

There are a lot of problems I can find in it (such as the Celtics always seem to have more back to backs and more overall games early in the season than the Lakers), but they are often just seen from a Celtics fan's perspective.  Fans of other teams could poke similar holes based on their view of their team.

I do think that the Heat's current ranking is interesting.  But all it means is that it is possible to blow out bad teams and lose to quality teams.  Over the course of the season, these strange outliers should sort themselves out.

This is why it is not good to pay too much attention to these sorts of rankings early in the season and to never follow them blindly without some context.

[MBunge]

The only problem with Hollinger's rankings is when he and others try to pretend it's better at determining the best team than win-loss or head-to-head record.  I'm not sure if it's the rise of fantasy leagues or what, but the idea that stats are a better predictor of greatness than winning has become way too common.

Mike

[LooseCannon]

I think people are missing the point of Hollinger's power rankings.  He's not trying to come up with the best possible model for predicting which teams are the best.  He doesn't want to use a method that does something like multiple regressions or other advanced calculations.  He doesn't want to incorporate 20 variables.  He's trying to come up with a quick and easy to calculate method that will identify which teams might be better or worse than their win-loss record indicates.

[MBunge]

He's trying to come up with a quick and easy to calculate method that will identify which teams might be better or worse than their win-loss record indicates.

You are what your record says you are.  Has Hollinger's rankings ever done a good job of predicting when seeming bad teams would start winning more or seemingly good teams would fall on hard times?

Mike

[BballTim]

I think people are missing the point of Hollinger's power rankings.  He's not trying to come up with the best possible model for predicting which teams are the best.  He doesn't want to use a method that does something like multiple regressions or other advanced calculations.  He doesn't want to incorporate 20 variables.  He's trying to come up with a quick and easy to calculate method that will identify which teams might be better or worse than their win-loss record indicates.

  What's the value of quick and easy? Just being able to explain his formula?

[greenpride32]

Stein has the C's at #1 this week, ahead of the Hornets.  The Lakers dropped 2 spots to #3.  But hey this isn't college football.  The rankings are meaningless.  If you win all your games by 1 point but win the ones that count, that's better than winning a bunch by 15 and losing the important ones by 1-2 points.