Athletics Veritas is a weekly series aimed at helping higher education executives, faculty, and other stakeholders stay tuned in on trending national issues impacting college athletics, especially NCAA Division I. Athletics Veritas is created by senior DI athletic administrators around the nation.
The Annual Argument Against The NCAA Tournament Selection Committee
Executive Summary
With the 68-team fields in the NCAA Division I basketball tournaments revealed on Sunday night, the organization’s annual cash cow is ready to return to some normalcy after the 2021 COVID-19 bubble event.
The critiques of the human-led selection committees returned in force on Sunday, with renewed calls for a more streamlined, data-driven approach to choosing the teams and seeds for the field.
This year’s men’s and women’s basketball selection committees included two more members, bringing the group to 12.
That group spends the entire college basketball season observing as many games as possible either in person or on television, using their own notes in addition to several respected basketball analytics sites to inform their opinions of the field.
The NCAA maintains extensive parameters for the selection process and uses a computer program to assist the committee in the bracketing process.
The NCAA introduced the NCAA Evaluation Tool, better known as the NET, to men’s and women’s basketball in 2018 with the goal of producing more thorough, comprehensive analytical evaluations of teams.
The organization previously used the Ratings Percentage Index, a source of significant frustration in the college hoops community because critics said it overvalued strength of schedule and undervalued the importance of winning games.
Critics of the current selection committee setup say the human factor creates the certainty of inherent biases influencing the selection and, perhaps more importantly, the seeding process.
The NCAA’s response to criticism over the years has always included the argument that the selection committees have extensive analytics tools at their disposal, from Ken Pomeroy’s industry-standard efficiency rankings to Bart Torvik’s popular tempo-free data.
To so many in the sports world, Selection Sunday is basically a national holiday. The day marks the beginning of one of the most popular sporting events in the world, the NCAA Tournament, and the drama of two selection shows unveiling the 68-team men’s and women’s basketball fields is exactly the kind of sports theater Americans love.
But the process to get to that point is under significant fire in the college basketball community, primarily because of the inclusion of humans in the selection and seeding of teams. The primary criticism: Why let human bias potentially (almost certainly) influence that process? The primary solution for the critics: Let well-constructed algorithms do the job.
To understand the parameters of the debate, you have to fully grasp the way the NCAA picks and ranks the teams in its hallmark events, which bring in more than $1 billion in revenue between the two.
The 68-team fields are put together by selection committees, which are made up of athletics directors and conference commissioners. Committee appointments run over several years. This year’s men’s committee, for example, was chaired by Southland Conference commissioner Tom Burnett, along with three other conference commissioners and eight athletics directors, including six representing the so-called power leagues.
The members spend the entire basketball season evaluating teams across the country. It’s not uncommon to spot them at games through the season, and they are afforded unfettered access to film of every game played through the campaign.
While committee members are entitled to use any metrics they want to evaluate teams, the NCAA’s in-house computer program used for the actual selection process focuses on the ranking of teams using the NCAA Evaluation Tool, better known as the NET. The computer program also references predictive/efficiency ratings from Ken Pomeroy, Jeff Sagarin and ESPN as well as result-based metrics from ESPN and the Kevin Pauga Index.
Both the men’s and women’s committees expanded from 10 to 12 members for the 2021-22 campaign, and each of those members is assigned at least two conferences to specifically evaluate during the season. Building familiarity with those specific leagues and teams is intended to help committee members discuss the ins and outs of NCAA Tournament contenders with other members, detailing potentially impactful injuries and other issues with each squad.
This is the human element of the process that is, in the NCAA’s eyes, immensely helpful.
Take, for example, a team that started hot, winning 15 of its 17 games and beating some high-level opponents. But two of that team’s stars suffered injuries that kept them out of a big portion of league play. Their team went from a 15-2 record to 18-9, with a few bad losses along the way, before the injured players returned and helped them finish 23-10 with a loss in the conference tournament semifinals.
In this scenario, provided the team has quality wins and good metrics, the additional intel that the 3-7 stretch came during a time when numerous key players were hurt would mostly just impact seeding. But if it was a so-called “bubble team,” with metrics that compete with other teams right on the cutline of the 68-team field as an at-large bid, the human elements become a key part of the consideration process.
The critique of the selection committee, manifested most often by Bart Torvik, who runs a popular college hoops analytics site, is that humans cannot offer completely unbiased, data-driven evaluations of teams in the process. To the exact contrary of the NCAA’s belief, Torvik has specifically mentioned the potential pratfalls with humans trying to evaluate teams based on their injuries.
“It's always wrongheaded for the selection committee to try to weigh games differently because a player was injured, etc.,” Torvik tweeted in December. “But trying to (do) it this year would be just utterly insane. Just reward teams for victories and penalize teams for losses. You know: sports.”
There has also long been criticism of the nebulous “eye test” aspect of the selection committee. In other words, the eye test means committee members are empowered to determine if they think a team is worth NCAA Tournament selection based on what they’ve seen when watching them. Do they look good? Play an attractive style? That kind of stuff.
That’s a dangerous allowance for committee members, critics say, because it allows far too much room for biases.
“I wish we could retire this phrase,” Seth Davis, a CBS Sports college basketball analyst and managing editor at The Athletic, said on CBS in 2021, “because it suggests that what someone sees is more important than actual results.”
The bottom line is that no one will ever be fully satisfied with the selection process for the NCAA Tournament for a variety of reasons. But it will be a hotly-debated subject as long as there is a Big Dance in men’s and women’s basketball.
Just think of the many avenues created for conversation after each Selection Sunday: Who unfairly got left out? Who was unfairly included? Which teams were overseeded or underseeded? Which top seeds received unfair pairings in their regions?
Then there is the cottage industry that propped up around the process itself, called “bracketology,” that has spawned numerous media personalities over the years and is currently led by ESPN’s Joe Lunardi. “Is Team X in or out if it wins tonight? Well, Joe says …” That corner of the college basketball media world spends the entire year evaluating the committee’s procedures to try to project how the 12 human beings on it might produce a tournament field in March.
Would that part of this process, which plays a huge role in building up the anticipation of Selection Sunday, go away with the committee?
These are all aspects of the conversation surrounding the NCAA”s billion-dollar events, but none of them particularly matter to the primary goal: Put the best 36 at-large teams and 32 conference champions into a field of 68 and let them compete for a national championship. Where it gets complicated, and may always be complicated, is defining “best.”
Athletics Veritas is presented for information purposes only and should not be considered advice or counsel on NCAA compliance matters. For guidance on NCAA rules and processes, always consult your university’s athletics compliance office, conference office, and/or the NCAA.
