What Data Can, and Should, Mean to Travel Advisors

It is, we are often told, the Age of Data. Everything seems to be “data-driven.” Data is much like the fog – you can see a lot at the same time, but you can’t really see anything until you cut through what it obscures.

A recent article in silive.com reported on the “10 airlines most likely to bump you from your flight.” The data was derived from the Department of Transportation's monthly Air Travel Consumer Report for April 2022. The silive.com article relied on another article appearing on stacker.com that ranked seventeen airlines based on involuntary bumpings. The DOT rankings refer to situations in which “Passengers are bumped when the airline oversells a flight and shifts or compensates passengers afterward.” Lists like these are popular ways of looking at problematic situations but they often exaggerate the risks.

The bumping data in the April 2022 report is actually from the last quarter of 2021, so here we are in the middle of May talking about information that is almost five months old. And, as stacker.com notes,

Airlines in this report also include their branded codeshare partners. This section furnishes data on the number of passengers who hold confirmed reservations and are denied boarding (“bumped”) from a flight because it is oversold. These figures include only passengers whose oversold flight departs without them; they do not include passengers affected by canceled, delayed, or diverted flights.

Finally, stacker.com also notes that “While not ranked on this metric, SkyWest Airlines had the highest rate of voluntary denied boardings, and Southwest Airlines had the highest absolute number of involuntary denied boardings.”

Denied boarding (bumping) data might seem like information consumers would want to know when booking travel. Maybe. The articles raise the question of under what circumstances should a travel advisor alert a client to such information prior to final booking?

First, we need to scope the problem by viewing all the available data that might be relevant. In addition to Oversales/Bumping for the 4^th quarter of 2021, the DOT report includes:

- Flight Delays (Overall and by Carrier, Airport, Time of Day)

- Cancellations

- Causes of Delays

- Tarmac Delays (Domestic and International)

- Mishandled Baggage

- Mishandled Wheelchairs and Scooters and

- Consumer Complaints in twelve categories (with multiple sub-categories).

All in varying degrees of detail for 56 pages in total length. The report is updated monthly.

The researchers at stacker.com ranked the seventeen airlines reported by DOT for the 4^th quarter of 2021 in order of worst to best, based on denied boarding per 10,000 passengers boarded. That ranking is the opposite of how DOT displays them. Silive.com then reported only on the ten worst:

1. Frontier Airlines

• Involuntarily denied boarding: 887 (1.55 per 10K)
• Total passengers boarded: 5,734,906

2. PSA Airlines

• Involuntarily denied boarding: 162 (0.46 per 10K)
• Total passengers boarded: 3,522,061

3. Envoy Air

• Involuntarily denied boarding: 165 (0.45 per 10K)
• Total passengers boarded: 3,647,596

4. Southwest Airlines

• Involuntarily denied boarding: 1,304 (0.36 per 10K)
• Total passengers boarded: 35,778,696

5. American Airlines

• Involuntarily denied boarding: 1,171 (0.36 per 10K)
• Total passengers boarded: 32,951,348

6) Horizon Air

• Involuntarily denied boarding: 38 (0.27 per 10K)
• Total passengers boarded: 1,430,372

7. Republic Airways

• Involuntarily denied boarding: 101 (0.22 per 10K)
• Total passengers boarded: 4,664,483

8. Mesa Airlines

• Involuntarily denied boarding: 47 (0.18 per 10K)
• Total passengers boarded: 2,600,969

9. SkyWest Airlines

• Involuntarily denied boarding: 160 (0.16 per 10K)
• Total passengers boarded: 10,216,924

10. Spirit Airlines

• Involuntarily denied boarding: 94 (0.12 per 10K)
• Total passengers boarded: 8,160,830

Based on those numbers alone, Frontier looks bad. Its bumping rate per 10,000 passengers is more than three times that of the next carrier. The second worst carrier (PSA) in turn has a rate about three times worse than the 10th ranked carrier (Spirit).

What does this really mean? From a bumping standpoint, is booking on Frontier really 12.9 times riskier than booking on Spirit? And if that’s true, since Delta, Endeavor, and Allegiant had no involuntary denied boardings in this period, is traveling on Frontier infinitely riskier than any of those airlines? Not likely.

To understand the true implications, we must broaden the scope of the data to include more information. Using DOT’s 10,000 boardings standard, Frontier’s bumping ratio is .000155 or 1.55 percent. Since 10,000 is an arbitrary number, we can also consider 877 (Frontier’s total actual bumpings) as a share of its total passenger boardings of 5,734,906. Now Frontier’s ratio drops to only .015 percent. That is a small chance, and it is the more realistic way to consider the risk.

Why? Because an accurate assessment requires that the risk be measured against the total population of people who could experience the problem. If we’re considering Frontier alone, it makes no sense to include passengers who flew on American. But it is essential to have all who were exposed to the risk on Frontier; that is the total passengers boarded during the period in question.

You can learn about all of this and more from the book, Know Your Chances, Understanding Health Statistics (Steven Woloshin M.D and others, 2008). Or, for a longer treatment of rational thinking, see Steven Pinker’s Rationality, What It is, Why It Seems Scarce, Why It Matters (2021).

The key point here is to avoid isolating a small subset of relevant information, as this tends to exaggerate actual risk. A more realistic assessment must take into account the general prevalence of a problem (the base rate).

Viewing all the relevant information in this case also requires looking at the bumping difference between Frontier and the other “worst” carriers. American is fifth-worst on this list of ten, with 1,171 bumpings, equating to 0.36 per 10,000 boarded. But American’s total boardings were 32,951,348 (5.75 times larger than Frontier). American’s bumpings thus were only .0036 percent of total boardings. That is considerably smaller than Frontier but, at these levels, is this type of risk a matter of real concern?

Consider this. The data can change a lot over time due to circumstances that may not occur in the future. Looking backward at a small set of data can therefore be inherently misleading, creating risks of overstating a very small problem. For example, Spirit is the tenth “worst” airline for the 4^th quarter of 2021, but in 2020 it was the worst of the worst ten. And it had only 99 bumped travelers in the 4^th quarter of 2020, representing .002 percent of its boarded passengers. 

In my view, it is neither practical nor necessary for a travel advisor to present such information to every potential or actual air traveler. The first reason is that most of the information will have nothing specific and material to do with the individual’s plans and objectives. Overloading the client with irrelevant information is simply bad practice.

On the other hand, the advisor who has properly qualified the client will know whether there are specific aspects of the planned trip for which concerns implicated by some of the data may be important. Take the case of the client who is the afternoon keynote speaker at a conference and is traveling to the conference city the same day. An airline’s history of involuntary denied boardings, flight delays, or cancellations may be of interest even if the risks are very small.

Why? Because the consequence of being bumped is large. The advisor probably should disclose them and provide context. The client thus informed can then decide for herself how much risk to take and whether, for example, to travel the day before the conference. Similar concerns would arise with elderly travelers for whom sitting around an airport for long periods could be problematic.

At the same time, the travel advisor should not overstate the risks. Doing so can create unnecessary stress, among other things. It can lead to decisions that have little basis in actual risk of an adverse event. Again, these are assessments best left to the traveler armed with critical information in context. This can be a difficult decision for the advisor because, if the passenger is bumped from the same-day flight and misses his speaking engagement, he may well blame the advisor who “failed to warn me.”

The ultimate rule of thumb is to assess the situation for the impact of a problem – how serious would it be to miss the conference? – and if there is a potentially serious impact, give the client the relevant risk information in context and let the client decide. Present alternatives – “you can travel the day before and I can get you a room in the conference hotel or one nearby.” Then the client who makes the wrong choice cannot really blame you for the outcome.