BLS Labor Market Forecasts:  A Weak Foundation for Academic Program Evaluation

April 11, 2022

When evaluating current and new academic programs, community colleges usually focus on the needs of employers, in particular local employers.  This emphasis enables graduates to find jobs and fuels local economic growth.  

Fast-Growth Can Lead to Misconceptions

However, industries, occupations, and the economy move at a startling pace in our society.  Data science, AI, and machine learning barely existed ten years ago; now, they are large and fast-growing sectors.  Truck driving is a huge occupation, but it may be wiped out by automation within the next decade or two.  The economy itself can change rapidly, from boom to bust in 2008 and again in 2020.  Even prognostications about broad labor market trends are often wrong.  Remember, a few years ago, all the folks talking about the grim economic outlook, automation, and the future shortage of jobs?  Yet, now we are facing a shortage of workers, not jobs.

The Problem with BLS Forecasting

Given the rapid and unpredictable changes in the economy,  labor market forecasts are very inaccurate.  In particular, the forecasts by Standard Occupation Code (SOC) from the Bureau of Labor Statistics seldom come to pass.  Let’s start with a simple analysis we ran to determine if BLS correctly predicted whether an occupation would grow or decline.  As illustrated below, about 62% of BLS forecasts correctly estimate whether growth in an occupation will be positive or negative (“Same Sign” in the forecast as the actual).  This seems pretty good until you realize that a coin toss would be right 50% of the time (see endnote).

Then it gets worse.  Many institutions and some states use the BLS forecast to decide which current and new programs to invest in, so correctly predicting positive growth is very important.  However, 27% of positive forecasts turn out to be for SOCs that decline, which is slightly worse than a coin toss.  The forecast was better at predicting declines, only missing 10% of the time.  Overall, the BLS forecast is about the same as a coin toss, even for a relatively simple prediction of occupational growth or decline.

We expect the forecast to do more than predict whether a field will grow or decline – we want to know how fast it will grow or shrink.  My next post, will show how close BLS gets to predicting actual growth rates.

Throughout this series, please keep in mind that the folks at BLS are neither ignorant nor lazy.  The problem is that they are trying to predict the results of a chaotic system where slight fluctuations in a distant country, a nascent technology, or the DNA of a virus could cause major changes in the US economy.

Endnote on The Coin Toss Analogy

Growth predictions can either be positive, negative, or, in a very few cases, zero.  For simplicity, let’s ignore the zeros.  There can only be two outcomes in this case:  positive growth or decline;  actual growth would also have two outcomes.  A coin toss has two outcomes, as well.  We will flip two coins for this exercise, one representing the forecast and one representing the actuals.  Let’s assume “heads” stands for growth and tails stands for decline.  Over a few hundred rounds, the results should look like this:

 

 

Outcome 1

Outcome 2

Outcome 3

Outcome 4

Coin 1:  “Forecast”

Heads:
Growth

Tails:

Decline

Heads:

Growth

Tails:

Decline

Coin 2:
“Actual”

Heads:
Growth

Tails:

Decline

Tails:

Decline

Heads:

Growth

Prediction equals actual

25% Yes

25% Yes

25% No

25% No

% of Tosses

50% of “Forecasts = Actuals”

25% Forecast Growth, but Actual Declined

25% Forecast a Decline, but Actual Grew

Robert Atkins

CEO AND FOUNDER OF GRAY ASSOCIATES

Bob led Gray’s entry into the education industry and the development of Gray’s proprietary industry databases and service offerings. He has worked directly with many of Gray’s education clients, consulting with CEOs and CMOs on business strategy, pricing, location selection, curricular efficiency, and program strategy.

Related Posts

International Students Are Returning to the US

Good news! The total number of international students enrolled at US institutions has rebounded after falling sharply in 2020-21. We took a deeper dive into the data to see what is driving this growth, what programs international students are interested in, and where those students are located.

How to Find Academic Opportunity with Emerging Programs

Every year, Gray features five new and emerging academic programs with high potential in our 5 Emerging Programs webinar. Get ready for our upcoming webinar with the following blog and find out why your institution should start the new year by thinking outside of the box and considering all of the possibilities!

Top 10 Blogs on Academic Program Evaluation

The most wonderful time of the year is here – a time for friends, family, and being thankful. This is the perfect opportunity to relax, unwind, and catch up on some good reading. We picked out ten articles on academic program evaluation that you don’t want to miss.

About Us
closeup photo of

Gray Associates, Inc. is a software and services firm focused on higher education. We help education clients develop fact-based institutional and marketing strategies that maximize outcomes for students, the school, and its constituencies.

Let’s Socialize

Popular Post