[Cracked Sidewalks] BPI Should be Removed from Team Sheets

[brewcity77]

After noticing some teams were ranked notably lower or higher by the BPI quality metric, which is an ESPN property that appears on the official NCAA Team Sheets, I decided to dig in deeper to see if there was any correlation between those disparities and the media contracts for the teams that were helped or hindered by ESPN's metric. The results do not look like a coincidence.

https://www.crackedsidewalks.com/2022/02/bpi-should-be-removed-from-team-sheets.html

[TAMU, Knower of Ball]

Interesting stuff Brew. It would be interesting to see if that pattern has been true for the past few years.

[NCMUFan]

What is the FOX developed metric? The BEI - Big East Index
Just kiddin.

[CountryRoads]

Great article.

[MU Fan in Connecticut]

After reading your article, in all seriousness, one of the data points in the secret sauce of the BPI formula must be "does the team have a ESPN contract".

[Spotcheck Billy]

This needs much wider distribution from the looks of it.

[Oldgym]

This needs much wider distribution from the looks of it.

Absolutely does.

[IL Warrior]

I think we've all suspected the possibility of something fishy, but seeing the numbers is eye-opening. Major props to brew for digging into this!

A few suggestions to further the investigation:
1. Look at past years, as mentioned by TAMU. ESPN/ACC/B1G/Big12/SEC defenders will cry "small sample size" if they see data limited to 21 teams.
2. Find a way to adjust for non-uniform distribution of team quality. Each system provides a rating for each team, and then orders them by that rating to give a ranking. For example, looking at Kenpom adjusted efficiency margin, Marquette (#24) is closer to #39 Indiana than #21 Ohio State. Additionally, teams closer to the "middle" are likely to have a wider disparity in rank than teams at the top or bottom, assuming the ratings follow something similar to a bell curve. Some disparities between Kenpom/Sagarin and BPI rankings are much easier (or harder) to explain than others.

[brewcity77]

Interesting stuff Brew. It would be interesting to see if that pattern has been true for the past few years.

If I have time, I'll dig back further. Definitely more work, but could be worthwhile. Though honestly, even if it's a one year thing, the perception of impropriety would be enough that I think the NCAA would be better served doing away with BPI.

[panda]

Kenpom significantly overvalued Wisconsin for years and the reason was clear. Any specifics with BPI and the teams you mentioned?

[MuggsyB]

After noticing some teams were ranked notably lower or higher by the BPI quality metric, which is an ESPN property that appears on the official NCAA Team Sheets, I decided to dig in deeper to see if there was any correlation between those disparities and the media contracts for the teams that were helped or hindered by ESPN's metric. The results do not look like a coincidence.

https://www.crackedsidewalks.com/2022/02/bpi-should-be-removed-from-team-sheets.html

Tremendous job here Brew. 

[The Sultan]

If I have time, I'll dig back further. Definitely more work, but could be worthwhile. Though honestly, even if it's a one year thing, the perception of impropriety would be enough that I think the NCAA would be better served doing away with BPI.

Whether or not this is statistically significant deserves some scrutiny no doubt.

But if it's a one year thing, I disagree that they should remove the metric. What if the "one year thing" showed the opposite results?  Favoring the Fox properties? Should they remove it then?

Just because something looks bad, it doesn't mean that it is bad.

[mugrad_89]

Really interesting stuff - as others have mentioned, it would be interesting to see how this looked in previous years.  What I do know is that there are not 42 teams better than Marquette.

[panda]

Pac12 still has a TV deal with espn. How does that factor in?

[MUpugnacity]

Whether or not this is statistically significant deserves some scrutiny no doubt.

But if it's a one year thing, I disagree that they should remove the metric. What if the "one year thing" showed the opposite results?  Favoring the Fox properties? Should they remove it then?

Just because something looks bad, it doesn't mean that it is bad.

Yeah. They should remove it regardless. There is a conflict of interest and it doesn't matter if it might be on the up and up.

When there is a conflict of interest the burden should be on the conflicted party to prove they are being ethical. So if espn wants to keep the BPI on the team sheets they need to be transparent and show the math behind the system so neutral parties can verify its legitimacy.

[The Sultan]

Yeah. They should remove it regardless. There is a conflict of interest and it doesn't matter if it might be on the up and up.

When there is a conflict of interest the burden should be on the conflicted party to prove they are being ethical. So if espn wants to keep the BPI on the team sheets they need to be transparent and show the math behind the system so neutral parties can verify its legitimacy.

It's only a potential conflict if this is more than a one year thing. Otherwise it's a one year anomaly. 

And I'm not even sure it's a conflict anyway. How does ESPN benefit from this?

[Galway Eagle]

It's only a potential conflict if this is more than a one year thing. Otherwise it's a one year anomaly. 

And I'm not even sure it's a conflict anyway. How does ESPN benefit from this?

Are you asking how ESPN benefits from boosting the profiles of teams that are signed to ESPN and not Fox?

[brewcity77]

Whether or not this is statistically significant deserves some scrutiny no doubt.

But if it's a one year thing, I disagree that they should remove the metric. What if the "one year thing" showed the opposite results?  Favoring the Fox properties? Should they remove it then?

Just because something looks bad, it doesn't mean that it is bad.

To the question, yes, because the perception of bias and potential conspiracy just stirs a pot that could be avoided by simply removing BPI (there are 2 resume metrics, why do they need 3 quality ones?) or replacing BPI with T-Rank, Haslametrics, or EvanMiya, all of which provide a comparable data point without contractual complications.

The choice is between perception of bias and no perception of bias. For the sake of transparency and the satisfaction of both fans and NCAA member institutions, why not take the contractual issues out when there's no need to have them there?

[GB Warrior]

It's only a potential conflict if this is more than a one year thing. Otherwise it's a one year anomaly. 

And I'm not even sure it's a conflict anyway. How does ESPN benefit from this?

Fact vs appearance. Obviously, a factual conflict of interests would be immensely damaging. But the appearance of a conflict can be damaging as well if there's any doubt cast on outcomes.

[brewcity77]

It's only a potential conflict if this is more than a one year thing. Otherwise it's a one year anomaly. 

And I'm not even sure it's a conflict anyway. How does ESPN benefit from this?

If their teams are in the NCAA tournament? Marketing. People remember March, so the next year, if a viewer's choice is between two ACC teams on ESPN that made the field or two MWC teams on FS1 that didn't but might have been better teams, which are they watching? If you're looking for a game to watch on your tablet, do you pick the game with the Belmont team you watched win a First Four game or the Colorado State team that would've been better metrically if not for BPI but was playing in the NIT the same night?

We know that ESPN has tampered with leagues. We know ESPN worked with Pitt and Syracuse to undermine the Big East and benefit the ACC. Why wouldn't they be willing to do the same to get their contractual partners tourney credits that are worth millions of dollars to those leagues and more exposure and marketing for the teams they put on the air from November to March?

Whether it's the reality or not, the reality we know is bad enough the the perception of impropriety should be avoided.

[MUCam]

We are talking about two separate things here and I think the problem Fluffy is having is the tendency of the initial article - and some of the responses to it - to conflate those two issues.

Issue One: Appearance of impropriety and the possibility of a conflict of interest. That issue is wholly independent of the data provided. The premise is simple: ESPN has a financial stake in which teams make the tournament and which teams do not. For the sake of avoiding that conflict of interest, any metric supplied by ESPN should be removed.

Issue Two: The factual analysis of one year of data leading to the insinuation (conclusion) that ESPN is skewing their data to their financial benefit. That data is, in my humble opinion, statistically irrelevant. It would be like flipping a coin three times, it landing on heads three times, and then deducing that heads is more likely than tails.

I think the data is interesting. At the very least it warrants further investigation. But I also think it's a bit unfair to use one year of data to try to prove ESPN is acting in bad faith.

Now, let me be clear. I would not be surprised that a farther dive would begin to corroborate that initial assumption, but right now we don't have it.

On the other hand, regardless of the data, the mere fact that ESPN has a financial stake in which teams do well and which teams do not, creates in my opinion a perception and appearance of impropriety that should be eliminated. I just won't go so far as to say that this year's data standing alone supports my opinion.

[The Sultan]

Issue One: Appearance of impropriety and the possibility of a conflict of interest. That issue is wholly independent of the data provided. The premise is simple: ESPN has a financial stake in which teams make the tournament and which teams do not. For the sake of avoiding that conflict of interest, any metric supplied by ESPN should be removed.

I have two issues with this.  Just because there is a *perceived* conflict of interest, that doesn't mean a conflict of interest actually exists.  Potential conflicts can be disclosed, but don't necessarily have to require action, if the data shows no statistical significance over time. 

I also question how much ESPN would actually benefit by this supposed conflict of interest.  Let's say that an ESPN team and a Fox team are on the bubble together and the ESPN team gets in.  Or a couple Fox teams look underseeded and ESPN teams look overseeded.  How does that materially benefit ESPN?  Are they going to get more eyeballs next year, and thus charge more for advertisements, if Notre Dame makes the tournament and Creighton does not?  Ditto if Nova ends up a 4-seed while Illinois is a 3-seed?

I just don't see the material benefit from ESPN's POV.

Issue Two: The factual analysis of one year of data leading to the insinuation (conclusion) that ESPN is skewing their data to their financial benefit. That data is, in my humble opinion, statistically irrelevant. It would be like flipping a coin three times, it landing on heads three times, and then deducing that heads is more likely than tails.

I think the data is interesting. At the very least it warrants further investigation. But I also think it's a bit unfair to use one year of data to try to prove ESPN is acting in bad faith.

I agree with this.  I asked this yesterday, but what if the data showed the opposite and "overranked" the Fox schools?  Would we still be suggesting its removal?  I think that's doubtful.

[Uncle Rico]

I have two issues with this.  Just because there is a *perceived* conflict of interest, that doesn't mean a conflict of interest actually exists.  Potential conflicts can be disclosed, but don't necessarily have to require action, if the data shows no statistical significance over time. 

I also question how much ESPN would actually benefit by this supposed conflict of interest.  Let's say that an ESPN team and a Fox team are on the bubble together and the ESPN team gets in.  Or a couple Fox teams look underseeded and ESPN teams look overseeded.  How does that materially benefit ESPN?  Are they going to get more eyeballs next year, and thus charge more for advertisements, if Notre Dame makes the tournament and Creighton does not?  Ditto if Nova ends up a 4-seed while Illinois is a 3-seed?

I just don't see the material benefit from ESPN's POV.


I agree with this.  I asked this yesterday, but what if the data showed the opposite and "overranked" the Fox schools?  Would we still be suggesting its removal?  I think that's doubtful.

This is the way I lean.

I simply think BPI isn't very good and should be replaced for that reason.

However, this is still an interesting piece by Brew.  It's worth exploring

[The Sultan]

If their teams are in the NCAA tournament? Marketing. People remember March, so the next year, if a viewer's choice is between two ACC teams on ESPN that made the field or two MWC teams on FS1 that didn't but might have been better teams, which are they watching? If you're looking for a game to watch on your tablet, do you pick the game with the Belmont team you watched win a First Four game or the Colorado State team that would've been better metrically if not for BPI but was playing in the NIT the same night?

I think this is extremely tenuous.  Do you really think people are going to watch one game versus another because it involves one of the last teams picked in the NCAA versus the NIT?  I would think that is WAY down on the list for why people chose what game to watch.

We know that ESPN has tampered with leagues. We know ESPN worked with Pitt and Syracuse to undermine the Big East and benefit the ACC. Why wouldn't they be willing to do the same to get their contractual partners tourney credits that are worth millions of dollars to those leagues and more exposure and marketing for the teams they put on the air from November to March?

Whether it's the reality or not, the reality we know is bad enough the the perception of impropriety should be avoided.

Tampering with leagues has a clear material benefit for ESPN.  There is absolutely no doubt about that.

[brewcity77]

If Fox or CBS had a metric on the team sheet, it would be just as problematic in my opinion. All it takes is this lining up once to create a bad look.

Further, T-Rank is more respected and referenced by non-ESPN national media and doesn't have this type of conflict. Why not just use that instead of one with contractual conflicts of interest? Or Haslametrics or EvanMiya?