BLS Labor Market Forecasts:  A Weak Foundation for Academic Program Evaluation

When evaluating current and new academic programs, community colleges usually focus on the needs of employers, in particular local employers.  This emphasis enables graduates to find jobs and fuels local economic growth.  

Fast-Growth Can Lead to Misconceptions

However, industries, occupations, and the economy move at a startling pace in our society.  Data science, AI, and machine learning barely existed ten years ago; now, they are large and fast-growing sectors.  Truck driving is a huge occupation, but it may be wiped out by automation within the next decade or two.  The economy itself can change rapidly, from boom to bust in 2008 and again in 2020.  Even prognostications about broad labor market trends are often wrong.  Remember, a few years ago, all the folks talking about the grim economic outlook, automation, and the future shortage of jobs?  Yet, now we are facing a shortage of workers, not jobs.

The Problem with BLS Forecasting

Given the rapid and unpredictable changes in the economy,  labor market forecasts are very inaccurate.  In particular, the forecasts by Standard Occupation Code (SOC) from the Bureau of Labor Statistics seldom come to pass.  Let’s start with a simple analysis we ran to determine if BLS correctly predicted whether an occupation would grow or decline.  As illustrated below, about 62% of BLS forecasts correctly estimate whether growth in an occupation will be positive or negative (“Same Sign” in the forecast as the actual).  This seems pretty good until you realize that a coin toss would be right 50% of the time (see endnote).

Then it gets worse.  Many institutions and some states use the BLS forecast to decide which current and new programs to invest in, so correctly predicting positive growth is very important.  However, 27% of positive forecasts turn out to be for SOCs that decline, which is slightly worse than a coin toss.  The forecast was better at predicting declines, only missing 10% of the time.  Overall, the BLS forecast is about the same as a coin toss, even for a relatively simple prediction of occupational growth or decline.

We expect the forecast to do more than predict whether a field will grow or decline – we want to know how fast it will grow or shrink.  My next post, will show how close BLS gets to predicting actual growth rates.

Throughout this series, please keep in mind that the folks at BLS are neither ignorant nor lazy.  The problem is that they are trying to predict the results of a chaotic system where slight fluctuations in a distant country, a nascent technology, or the DNA of a virus could cause major changes in the US economy.

###

Endnote on The Coin Toss Analogy

Growth predictions can either be positive, negative, or, in a very few cases, zero.  For simplicity, let’s ignore the zeros.  There can only be two outcomes in this case:  positive growth or decline;  actual growth would also have two outcomes.  A coin toss has two outcomes, as well.  We will flip two coins for this exercise, one representing the forecast and one representing the actuals.  Let’s assume “heads” stands for growth and tails stands for decline.  Over a few hundred rounds, the results should look like this:

 

 

Outcome 1

Outcome 2

Outcome 3

Outcome 4

Coin 1:  “Forecast”

Heads:
Growth

Tails:

Decline

Heads:

Growth

Tails:

Decline

Coin 2:
“Actual”

Heads:
Growth

Tails:

Decline

Tails:

Decline

Heads:

Growth

Prediction equals actual

25% Yes

25% Yes

25% No

25% No

% of Tosses

50% of “Forecasts = Actuals”

25% Forecast Growth, but Actual Declined

25% Forecast a Decline, but Actual Grew

Share on facebook
Share on twitter
Share on linkedin
Share on email

Related Posts

Using Employment Data: The Direct Prep Fallacy

Some assume clear links between academic programs and occupations for graduates, which are neatly embodied in the NCES crosswalk between programs and occupations. In theory, they are right. In practice, it is bunk for graduates with a bachelor’s degree.

Highest Paying Jobs for College Majors

Gray did a deep dive into lucrative jobs for graduates and while the results are mixed, there is one program overall that wins the day. It is not in engineering.

About Us
closeup photo of

Gray Associates, Inc. is a strategy consulting firm focused on higher education. We help education clients develop fact-based institutional and marketing strategies that maximize outcomes for students, the school, and its constituencies.

Let’s Socialize

Share on facebook
Share on twitter
Share on linkedin
Share on email
Popular Post