[Paint Touches] How accurate are KenPom's preseason rankings?

[Uncle Rico]

How long until the games start so we can argue about actual things and not abstracts?

Hahaha, thinking this argument will stop.  I think Bob Dukiet did a better job than Wojo and an argument can be made Mike Deane was underrated. 

[tower912]

What?!?!?   You are saying roqqet level silly things to provoke a reaction?!?!?!?

[withoutbias]

Oh god.  Now the number nerds are claiming the only accurate rankings are the teams that are on every single ballot?  So week to week the only teams that are really "ranked" will change because the number of teams that are on every single ballot might change, and those are the only ones that can have an accurate ranking?  So if a team is ranked 25th on every single ballot but another team is ranked in the top 10 of every ballot but some dumb SID or DOBO forgot that they exist and we can't say they're ranked?

Call it whatever you want to.  Whether that's "this team is ranked third" or "this team has the third most votes this week," the team with the 26th most votes is "ranked" 26th.

[The Sultan]

Oh god.  Now the number nerds are claiming the only accurate rankings are the teams that are on every single ballot?  So week to week the only teams that are really "ranked" will change because the number of teams that are on every single ballot might change, and those are the only ones that can have an accurate ranking?  So if a team is ranked 25th on every single ballot but another team is ranked in the top 10 of every ballot but some dumb SID or DOBO forgot that they exist and we can't say they're ranked?

Congratulations. I don't think anyone will manage to misrepresent more what TAMU said more than you just did.

[brewcity77]

Call it whatever you want to.  Whether that's "this team is ranked third" or "this team has the third most votes this week," the team with the 26th most votes is "ranked" 26th.

No, it's not and never has been, which is why the poll goes to 25 and there's no "26" only others receiving votes.

[Pakuni]

No, it's not and never has been, which is why the poll goes to 25 and there's no "26" only others receiving votes.

Are the point totals for the "others receiving votes" calculated differently than the top 25?

[The Equalizer]

Let's just call this out for what it is . . . an attempt to "own" the Providence writer using a deeply flawed yet valid-sounding statistical analysis. 

Flaws:

1. A sample size of 11 is woefully insufficient to draw any conclusions about the accuracy of KenPom in a universe of ~360 D1 teams. 

2. Similarly, a sample consisting only of Big East teams is not random enough to draw any conclusions.

3. There was no basis provided for comparison provided (i.e.. similar research into other preseason ranking performances over the same time period). 

4. Separating the 19 top-twenty predictions (13/19 or nearly 70% predicted within 15 places) from the entire sample (48/99 or 48% predicted within 15 places) doesn't appear to have any purpose other than to allow the reported performance to be improved from 48% accurate within 15 spots to 70% accurate within 15 spots.

While this article makes for good fodder for bar room or message board debates, nobody could rightfully conclude that KenPom's predictions are any more or less accurate than anyone else's from what was presented.

[Pakuni]

Again, no one is saying it is gospel. Just one tool that has a history of being decently accurate.

Define "decently accurate."
Because KenPom is no more decently accurate at predicting the top 25 than a bunch of sportswriters. And yeah, that's a small sample size, but so is Paint Touches' sample size of just one 11-team conference.
I just don't think getting within 30 spots 2/3 of the time is all that amazing. I mean, how often does a AP preseason top 25 team not finish as one of the best 30-55 teams in the country? I bet it's a lot less than one in three.

Again, no one is saying KenPom is useless.

[avid1010]

Ken Pom crushed Vegas...still making money off of his stuff...his 1H totals for mid-major conferences are golden. 

[PGsHeroes32]

Oh god.  Now the number nerds are claiming the only accurate rankings are the teams that are on every single ballot?  So week to week the only teams that are really "ranked" will change because the number of teams that are on every single ballot might change, and those are the only ones that can have an accurate ranking?  So if a team is ranked 25th on every single ballot but another team is ranked in the top 10 of every ballot but some dumb SID or DOBO forgot that they exist and we can't say they're ranked?

Call it whatever you want to.  Whether that's "this team is ranked third" or "this team has the third most votes this week," the team with the 26th most votes is "ranked" 26th.

You really gotta just stop chiming in on things.

[LAZER]

Ken Pom crushed Vegas...still making money off of his stuff...his 1H totals for mid-major conferences are golden.

Where are 1H totals on his site?

[withoutbias]

You really gotta just stop chiming in on things.

Go cry some more about managers not leaving their starters in for 7+ innings anymore or batters swinging at first pitch strikes or act like a team falling behind 2-0 in the third inning of game one means the series is over and the manager should be beheaded on the pitching mound.

[TAMU, Knower of Ball]

Again, no one is saying KenPom is useless.

I'm pretty sure you have, at least the preseason numbers. But if I misunderstood, mea culpa.

Define "decently accurate."
Because KenPom is no more decently accurate at predicting the top 25 than a bunch of sportswriters. And yeah, that's a small sample size, but so is Paint Touches' sample size of just one 11-team conference.
I just don't think getting within 30 spots 2/3 of the time is all that amazing. I mean, how often does a AP preseason top 25 team not finish as one of the best 30-55 teams in the country? I bet it's a lot less than one in three.

The problem with the comparison you are trying to make here is sample size. The AP poll is not based on algorithm, it's people's opinions. When they are making their rankings, they aren't looking at all 363 teams (missed that some teams were added this season). In a given season, there's maybe what, 40-60 teams that you could reasonably argue would in the opening top 25? So picking the top 25 out of 40-60 teams is not that hard and it's unlikely that you're going to miss by more than 20 or 30 spots.

KenPom (and others like it) are based on algorithms. There is no opinion. There is also no bonus for being a high major team or a perceived better program. It's all based on the numbers the players produced a season ago. The #1 team (Kentucky this year) was evaluated in the exact same manor as the #363 team (IUPUI). IUPUI has just as much of a chance as Kentucky to be highly rated, but they don't have the numbers to back it up. So in that context, missing by 30 spots out for 363 (~8.3%) isn't amazing, but it's also not bad. In the case of the Big East, KP has done a little better than missing by 30 spots in most cases.

[PGsHeroes32]

Go cry some more about managers not leaving their starters in for 7+ innings anymore or batters swinging at first pitch strikes or act like a team falling behind 2-0 in the third inning of game one means the series is over and the manager should be beheaded on the pitching mound.

Well this is when you know youve lost.

Let everyone know when your ability to read and comprehend is up to par.

[Pakuni]

The problem with the comparison you are trying to make here is sample size.

You say this in defense of an article that examined only one conference, and a small one at that.

KenPom (and others like it) are based on algorithms. There is no opinion. There is also no bonus for being a high major team or a perceived better program. It's all based on the numbers the players produced a season ago. The #1 team (Kentucky this year) was evaluated in the exact same manor as the #363 team (IUPUI). IUPUI has just as much of a chance as Kentucky to be highly rated, but they don't have the numbers to back it up.

These two thoughts are contradictory.
In reality, IUPUI does not have as much a chance as Kentucky to be highly rated, because their ratings (for now) is based on what happened last year. Teams are largely bound by the previous season's results.
Which is fun to ponder, when you consider how different Kentucky's roster looks from one year to the next.

So in that context, missing by 30 spots out for 363 (~8.3%) isn't amazing, but it's also not bad.

It's also not something we know. Saying KenPom missing by 30 spots out of 363 is just extrapolating from the small and not representative sample size of a single 11-team conference. Until someone puts in the work to look at all those teams - or at least a larger, more representative group of teams - we have no idea how accurate he is.
And I'll take a pass on doing that work for now.

[avid1010]

Where are 1H totals on his site?

Under fan match i believe.  You won't find any discrepancies with vegas on the high major games...but if you hunt the mid/low majors there is value...especially when playing the under.  It's drying up more and more each year.

[brewcity77]

Are the point totals for the "others receiving votes" calculated differently than the top 25?

In regards to calling a team #26? Yes, absolutely.

[Pakuni]

In regards to calling a team #26? Yes, absolutely.

So, no?

[WhiteTrash]

So, no?

I think you are both right. I'd guess 98/100 times the top 'other receiving votes' would be #26 if they voted for 26 teams. But the flaw in actually proclaiming them #26 n a vote for the top 25 becomes easy to see if you look at an example of having a top 15 poll but you can only vote for the top 10. You may not even get to 15 teams ranked even if all the voters felt one or multiple teams were between 11-15.

[wadesworld]

I think if you didn't even give a person a team's name but just gave them the conference that team plays for and told them to predict the final KenPom ranking for that team, and did that for every D1 college basketball team, and then averaged out how many spots off you were by conference, most semi-knowledgeable college basketball fans are probably going to be somewhat close to "within 30 spots" on average for conference.

The realistic range for a Big East basketball team really isn't one through 358.  It's really like one to 150, with a large majority really being like 30-70, and that's the same with most power conferences.

Just like the realistic range for SWAC isn't 1 to 358.  It's really like 200-358.

[Newsdreams]

Welcome to the MOPES analytics thread.

[brewcity77]

So, no?

As I said, yes. Yes does not mean no.

[panda]

Welcome to the MOPES analytics thread.

Lol

[Pakuni]

As I said, yes. Yes does not mean no.

You wrote "in regards to calling a team #26," whatever that means.
What I'm asking is whether the way they tabulate points for the top 25 is different than the way they tabulate points for the others receiving votes.

[rocky_warrior]

What I'm asking is whether the way they tabulate points for the top 25 is different than the way they tabulate points for the others receiving votes.

I don't believe so - which is why I think it's just as valid to call a team "#26" as it is "#3" (or whatever).  Whether that ranking is accurate or not (in either case) is the original point of this conversation though.  Or at least I think it is.