REGISTER   |   LOGIN   |   HELP
Home  |  Blogs  |  Message Boards  |  Webinars  |  Resources |  By Channel
John Barnes

Don't You Regress With Your Regressions!

NO RATINGS
1 saves
View Comments: Newest First | Oldest First | Threaded View
Page 1 / 2   >   >>
John Barnes
User Rank
Blogger
Re: Regression to the mean: Average versus Outlier
John Barnes   12/9/2012 5:16:14 PM
NO RATINGS
Louis, I'd phrase it just a little differently, that it's a matter of what field you're in.  If you're a sandwich maker, you can be right there on that regression line, or even a bit below it, and still work.  But if you're going to be a pro tennis player, you need to be an outlier -- way over to the right and consistently above that regression line.  And if you want to be a pop star ... all that, plus being an extreme outlier (i.e. VERY lucky).


Something I just noticed thanks to this excellent blog post (which I found in David Brin's even-more-excellenter blog post) is that although the black swan metaphor is catchy, the book itself suffers from the problem that the author thinks that because you can't predict the specifics, you'll always be surprised by rare events.  But often just knowing that the surprise is possible carries a lot of information; we know a massive earthquake, bigger than the California Big One, will happen someday with an epicenter somewhere around New Madrid, Missouri; we know that the use of nuclear EMP to disable machinery around a wide area, which is an extremely likely thing for a government or terrorist organization to do in the next fifty years,  also means that an enormous number of financial records (such as bank accounts) could be wiped; we know that if a late-season hurricane, a high plains blizzard, and a large arctic air mass all converge on the East Coast megalopolis .... oh, wait.

Before the Panic of 1907 it was well-known that just a few insurance companies covered the city of San Francisco, and their stock was mostly owned by a few large banks in New York, which in turn were linchpins of that city's (and therefore the country's) banking system (back before the Federal Reserve).  How likely was it that suddenly all those insurance companies would have to pay up on all their policies all at once?  (Hint: what happened in San Francisco in 1906?)


The extreme outliers are important and worthy of our attention, even if it's just to confirm they are flukes.

Louis Watson
User Rank
Blogger
Regression to the mean: Average versus Outlier
Louis Watson   12/9/2012 4:58:51 PM
NO RATINGS
Thanks John, for the refresher on Regression, I do think that because it is not a sexy tool - it's information often get's ignored.  But I like how you have shown how regression to the mean explains a whole host of occurrences ( I really like the sophomore slump example). 

It seems like everyone is destine to be average except for the occasional outlier, which is I guess how it has always been - so the question is how to become and stay an outlier against the odds ?

Very interesting insight and food for thought - thanks again John.

John Barnes
User Rank
Blogger
Re: Investors too
John Barnes   12/4/2012 12:59:20 AM
NO RATINGS
Seth -- that's very true, though not itself an example of regression to the mean.  But an analyst would still use regression to the mean as an alternate hypothesis, to demonstrate the reality of fatigue.  Here's how the analyst would reason it out:

In either a genuine regression-to-the-mean case, or in the case of fatigue,  nearly all top performers in the first period evaluated would fall down the ladder in some later period, but in a classic regression to the mean, the fall would be overwhelmingly likely to happen between period 1 and period 2.  If there were any surviving top performers who had top-performed in both period 1 and period 2, then the fall would be overwhelmingly likely (and with the same probability) between 2 and 3, and so on; a high probability of a top performer taking a fall between any two periods, and that probability would be fairly constant.

But in fatigue, you'd see the probability of a fall start out fairly low and rise with time, probably nonlinearly.  You could quickly confirm this by plotting and/or regressing your residuals (statistical errors) against time.  Comparing the errors of the two models would quickly reveal that fatigue explained things much better.

So a reasonably sharp analyst, put on the problem, could tell the manager that this was not regression to the mean (which fundamentally can't be fixed) but a situation where the right resources applied correctly (sabbaticals, incentives) could make everyone better off. 

That's the beauty of always considering and testing for regression to the mean; whether you find it or not, it always puts you a long step closer to understanding the situation.

SethBreedlove
User Rank
Data Doctor
Re: Investors too
SethBreedlove   12/3/2012 8:54:35 PM
NO RATINGS
When it comes to employee performance, fatigue must be countered in. Very few people can be top performers for years.   I had problems with one boss when I became less productive, yet still the top producer.  You can drive a Ferrari at top speed only for so long before it breaks down.

John Barnes
User Rank
Blogger
Re: Investors too
John Barnes   12/3/2012 4:48:14 PM
NO RATINGS
PredictableChaos:

Absolutely right, and very wise to boot!  In so many ways the most important thing we learn from regression to the mean is not to give ourselves too much credit.

PredictableChaos
User Rank
Data Doctor
Investors too
PredictableChaos   12/3/2012 4:41:39 PM
NO RATINGS
 

This happens with investors and financial advisors too - one hot streak early in a career and you're a legend in your own mind.  Which can lead to overconfidence and all kinds of nutty behavior.

So I guess I should be happy that my first stock investments were more like touching a hot stove.  I learned humility much faster than if I'd picked a big winner.

John Barnes
User Rank
Blogger
Re: I'm regressing...but I digress
John Barnes   12/3/2012 2:50:29 PM
NO RATINGS
The strange this is that some of the hotshots created by regression to the mean believe their own hype; they think there must be something they're doing that's causing the streaks.  Addicted gamblers are notoriously that way; they think they had mojo that accounted for that one wonderful time early in their career that everything went so well, and they can spend (and destroy) the rest of their lives trying to get that illusory mojo back.  But some businesses, occupations, and situations are just streaky by nature (Claude Shannon, back in the 1940s, worked out why in a purely random process, streakiness is more likely than steadiness) and quiet, persistent, do-it-right-every-time effort only pays out on the average over the very long run, so more "streak addicts" are born every year -- and end up wasting their lives chasing after the "streak magic" that doesn't exist. 

Callmebob
User Rank
Master Analyst
I'm regressing...but I digress
Callmebob   12/3/2012 2:44:41 PM
NO RATINGS
John, on the money with your sales and sports stars slumping or falling off a cliff comparisons. I've been involved in sales for years, many of them as a sales manager. I've always looked for performance consistency over hotshots and spikes. Why? The mercurial hotshots or rainmakers are prone to be hot and cold, not steady, even dipping below the mean. And as a sales manager I would never completely tie my wagon to a star sales person or customer for that matter.

There was one guy I know (used to work with him) who was one of these slam-bam type of sales people who would outshine everyone...but only for brief periods and then he'd sink and his sales would dip. That caused management to start asking questions about commitment andeffort. This guy was always ready and would typically respond by not pumping up his effort but quickly bailing out and going to a new company where they were impressed with his hot sales record. He'd typically last a year at a company and leave before the annual performance review.

BethSchultz
User Rank
Blogger
Re: Good reminder
BethSchultz   12/3/2012 2:24:36 PM
NO RATINGS
@John, the freelancer's bane! (Which I know well from a previous life, although I didn't know it had anything to do with regression to the mean.) 

 

John Barnes
User Rank
Blogger
Re: Good reminder
John Barnes   12/3/2012 2:17:30 PM
NO RATINGS
Beth, I learned it the hard way, from someone who was not a better analyst than me but who paid attention to the analytics at a time when I didn't.  She pointed to the variability in payment times and amounts from various clients, and said, "John, right here is where you'll go broke.  You can afford to have all this wobble in the behavior of the minor clients, and let it even out, but the first time a major client does that, you'll be broke."


Unfortunately she was absolutely right; one day the single client that was 40% of my income turned into an outlier for payment time and for commissioning new work.  It's not the average wave, but the biggest one, that can sink you.


And it was all perfectly predictable from the fact that clients in that business showed a distinct pattern of regressing to the mean.  The rest of you don't need to put your hand on the hot stove to find out it's a bad idea; just sniff my charred fingers!

Page 1 / 2   >   >>
More Blogs from John Barnes
Analysts would do well to get out of the rut of using linear regressions by default.
Sometimes your results require accuracy and sometimes precision. Knowing the difference matters.
Rule-based behavior models offer a good alternative to guesswork and folk wisdom.
Keeping these three words, often jumbled in business discourse, separate and precise can help you be a better decision maker.
Quick Poll
Digital Audio
Latest Archived Broadcast
In this A2 Radio episode, analytics thought leader Tom Davenport will take you into the worlds of business and sports, and talk about what one can learn from the other.
August 20th 1pm EDT Wednesday
AllAnalytics House Ad
Readerboards
Have a question or topic but don't want to write a blog? Post it on our readerboards and get feedback from the community!
MORE READERBOARDS
AllAnalytics Video Blogs
Canada Post on Data Delivery
James Smith, lead of enterprise data governance for ...

02:53

2 comments
Big Data Checks In
Roger Ares, vice president of analytics at Hyatt ...

03:45

3 comments
Good Data, Smarter Travel
iJET International's Rich Murnane, Director of ...

2:50

4 comments
Healthcare Data Needs a ...
Tackling healthcare data management challenges ...

2:39

0 comments
In the Talent Sweet Spot
Gen Y shapes the new analytics workforce.

3:25

0 comments
A2 on the Road
What we found at last month's SAS Global Forum 2014.

2:28

0 comments
T-Mobile Hears Data's Call
Internal customer data provides an analytics ...

2:50

0 comments
A Is for Analytics in Academia
Professors and students agree that access to ...

4:11

0 comments
Advocating for Analytics ...
Analytics and business experts explore what it ...

5:51

0 comments
Analytic Myths: True or False?
Analytics experts give us their perspective on four ...

4:29

0 comments
We'll Be Your Eyes & Ears
We'll be on the scene at SAS Global Forum events in ...

2:15

0 comments
7 Tips for Deploying ...
We chat with Analise Polsky, a data visualization ...

33:15

0 comments
Top Big Data Platforms
All Analytics editors Beth Schultz and Michael ...

31:53

3 comments
Attention on Retail Shoppers
The retail store of the future will track customers ...

02:14

16 comments
Demand-Driven Forecasting
Charles Chase, chief industry consultant for the ...

02:22

1 comment
Live Video
On-demand Video with Chat
As retailers evolve toward an omnichannel environment, much of their success will depend on how effectively they use big-data and analytics.
Upcoming Events
for the Business and IT Communities
Executive forums with additional hands-on learning opportunities offered around the world
Each ideal for practitioners, Business leaders & senior executives
2014 VA Interactive Roadshow -- Detroit
The 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Aug. 7, 2014
Detroit, Michigan
2014 VA Interactive Roadshow -- Chicago
The 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Sept. 16, 2014
Chicago, Illinois
2014 VA Interactive Roadshow -- Cary, NC
The 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Oct. 10, 2014
Cary, North Carolina
2014 VA Interactive Roadshow -- Boston
The 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Nov. 4, 2014
Boston, Massachusetts
2014 VA Interactive Roadshow -- Atlanta
The 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Dec. 11, 2014
Atlanta, Georgia
AllAnalytics on Twitter
AllAnalytics Twitter Feed
AllAnalytics Videos
Intro to Visual Analytics
Find a way to visualize your data and watch it come ...

1:58

0 comments
Visual Analytics, Instant ...
Analytics results delivered in visual form are ...

2:06

5 comments
Big Data, Fast Infrastructure
Big data calls for a high-performance analytics ...

3:35

6 comments
Red Hot Analytics
Jayson Tipp, Redbox VP of Analytics and CRM, ...

3:51

7 comments
Hotelier Checks Out Analytics
InterContinental Hotels Group has woven analytics ...

06:55

11 comments
Like Us on Facebook
Point/CounterpointBlog
LEADERS FROM THE BUSINESS AND IT COMMUNITIES DUEL OVER CRITICAL TECHNOLOGY ISSUES

The Current Discussion

Visual Analytics: Who Carries the Onus?
The Issue: Data visualization is an up-and-coming technology for businesses that want to deliver analytical results in a visual way, enabling analysts the ability to spot patterns more easily and business users to absorb the insight at a glance and better understand what questions to ask of the data. But does it make more sense to train everybody to handle the visualization mandate or bring on visualization expertise? Our experts are divided on the question.
The Speakers: Hyoun Park, Principal Analyst, Nucleus Research; Jonathan Schwabish, US Economist & Data Visualizer
MORE POINT/COUNTERPOINT BLOGS
About Us  |  Contact Us  |  Help  |  Register  |  Twitter  |  Facebook  |  RSS


Beth Schultz
Canada Post on Data Delivery

5|12|14   |   02:53   |   (2) comments


James Smith, lead of enterprise data governance for the Canadian postal service, explains how and why the organization has embraced an enterprise data governance program.
Michael Steinhart
Big Data Checks In

4|30|14   |   03:45   |   (3) comments


The hospitality industry gathers massive amounts of customer data, and mining that data effectively can yield tremendous results in terms of improved CRM, better-targeted marketing spend, and more efficient back-end processes. Roger Ares, vice president of analytics at Hyatt Corp., discusses the ways he and his staff use big data.
Beth Schultz
Good Data, Smarter Travel

4|24|14   |   2:50   |   (4) comments


Charged with keeping track of travel assets, including employees, iJET International relies on data management best-practices and advanced analytics to keep its clients in the know on current and potential world events affecting travel, Rich Murnane, Director of Enterprise Data Operations & Data Architect, told All Analytics in an interview from the 2014 SAS Global Forum Executive Conference.
Beth Schultz
Healthcare Data Needs a Booster

4|23|14   |   2:39   |   (0) comments


W. Ed Hammond, Director of the Duke Center for Health Informatics, spoke from the recent 2014 SAS Global Forum Executive Conference about the data management challenges involved in healthcare today.
Beth Schultz
In the Talent Sweet Spot

4|23|14   |   3:25   |   (0) comments


Jason Dorsey, chief strategy officer for the Center for Generational Kinetics and keynote speaker at last month's SAS Global Forum 2014, describes how Gen Y professionals are enhancing the makeup of multigenerational analytics organizations.
Beth Schultz
A2 on the Road

4|22|14   |   2:28   |   (0) comments


From analytics talent development to the power of visual analytics, All Analytics found a variety of common themes circulating throughout the exhibition floor and session discussions at the 2014 SAS Global Forum and SAS Global Forum Executive Conference events held last month in Washington, DC.
Beth Schultz
T-Mobile Hears Data's Call

4|22|14   |   2:50   |   (0) comments


Talking with All Analytics live from the 2014 SAS Global Forum Executive Conference, Eric Helmer, senior manager of campaign design and execution for T-Mobile, discussed the importance of customer data -- starting internally -- in devising the mobile operator's marketing plans.
Beth Schultz
A Is for Analytics in Academia

4|21|14   |   4:11   |   (0) comments


Interviewed live at SAS Global Forum 2014, professors and students agree that access to enterprise analytics software in academic programs better prepares graduates for their careers.
Beth Schultz
Advocating for Analytics Culture

4|21|14   |   5:51   |   (0) comments


Speaking at the recent SAS Global Forum Executive Conference, analytics executives, business experts, and SAS insiders explore what it means to build an analytics culture.
Beth Schultz
Analytic Myths: True or False?

4|21|14   |   4:29   |   (0) comments


At the recent 2014 SAS Global Forum Executive Conference, five analytics experts give us their perspective on whether four common myths about IT and analytics are true or false.
Beth Schultz
We'll Be Your Eyes & Ears

3|20|14   |   2:15   |   (0) comments


We'll be on the scene at SAS Global Forum events in Washington, D.C., March 23 to March 25, glad to share what we learn with our community members.
Beth Schultz
7 Tips for Deploying Visualization

3|7|14   |   33:15   |   (0) comments


We chat with Analise Polsky, a data visualization thought leader on the SAS Best Practices team, about what you need to know before you deploy data visualization.
Michael Steinhart
Choosing a Big-Data Analytics Platform

2|19|14   |   31:53   |   (3) comments


The big-data analytics market can be a confusing place. Among the vendors vying for your dollars are traditional database management providers, Hadoop startup services, and IT giants. In this video, All Analytics editors Beth Schultz and Michael Steinhart sit down in a Google+ Hangout on Air with Doug Henschen, executive editor of InformationWeek. Henschen discusses use cases for big-data analytics, purchase considerations, and his recent roundup of the top 16 big-data analytics platforms.

Related posts:

— Michael Steinhart, Circle me on Google+ Follow me on TwitterVisit my LinkedIn pageFriend me on Facebook, Executive Editor, AllAnalytics.com

Michael Steinhart
Keeping a Close Eye on Shoppers

2|13|14   |   02:14   |   (16) comments


At the National Retail Federation BIG Show last month, All Analytics executive editor Michael Steinhart noted a host of solutions for tracking and analyzing customer activity in retail stores. From Bluetooth beacons to RFID tags to NFC connections to video analytics, retailers must find the right combination of tools to help optimize the shopper experience, streamline operations, and boost revenues.

Related posts:

— Michael Steinhart, Circle me on Google+ Follow me on TwitterVisit my LinkedIn pageFriend me on Facebook, Executive Editor, AllAnalytics.com

Michael Steinhart
Real-Time Demand Drives Forecasting

2|11|14   |   02:22   |   (1) comment


The days when historical shipment trends and gut feelings were enough to forecast retail demand accurately are long over. SAS chief industry consultant Charles Chase outlines the benefits of pulling real-time sales information from point-of-sale and product scanner systems, then flowing that data into dynamic forecasting tools from SAS.

— Michael Steinhart, Circle me on Google+ Follow me on TwitterVisit my LinkedIn pageFriend me on Facebook, Executive Editor, AllAnalytics.com

Videos
Intro to Visual Analytics

6|5|13   |   1:58   |   (0) comments


With today's advanced visual analytics tools, you can stream data into memory for real-time processing, provide users the ability to explore and manipulate the data, and bring your data to life for the business.
Videos
Visual Analytics, Instant Insight

5|16|13   |   2:06   |   (5) comments


Dynamic data visualizations let analysts and business users interact with the data, changing variables or drilling down into data points, and see results in a flash. Advance your use of data visualization with tools that support features like auto-charting, explanatory pop-ups, and mobile sharing.
Videos
Big Data, Fast Infrastructure

2|14|12   |   3:35   |   (6) comments


No doubt your enterprise is amassing loads of data for fact-based decision-making. Hand in hand with all that data comes big computational requirements. Can traditional IT infrastructure handle the increasing number and complexity of your analytical work? Probably not, which is why you need a backend rethink. Big data calls for a high-performance analytics infrastructure, as Fern Halper, a partner at the IT consulting and research firm, Hurwitz & Associates, discusses here.
Videos
Red Hot Analytics

1|10|12   |   3:51   |   (7) comments


Redbox's bright-red DVD kiosks are all but ubiquitous these days, located in more than 28,000 spots across the country. Jayson Tipp, Redbox VP of Analytics and CRM, provides an insider's look at how the company has accomplished its phenomenal nine-year growth.
Videos
Hotelier Checks In With Analytics

12|14|11   |   06:55   |   (11) comments


InterContinental Hotels Group (IHG), a seven-brand global hotelier, has woven analytics into the fabric of its operations. David Schmitt, director of performance strategy and planning, shares IHG's analytics story and his lessons learned.