Ariella Brown

Algorithms' Dark Side: Embedding Bias into Code

NO RATINGS
View Comments: Newest First | Oldest First | Threaded View
tomsg
User Rank
Data Doctor
Re: Biased assumptions in algorithms
tomsg   2/22/2017 9:11:56 PM
NO RATINGS
I would be interested in knowing this as well. 

PredictableChaos
User Rank
Prospector
Re: Biased assumptions in algorithms
PredictableChaos   2/22/2017 5:36:46 PM
NO RATINGS
@Lyndon -

Is there any detail you can share with us about how the algorithms were slanted against public transportation?

Sounds like an interesting topic if it's not too technical.

Lyndon_Henry
User Rank
Blogger
Biased assumptions in algorithms
Lyndon_Henry   2/22/2017 4:45:50 PM
NO RATINGS
..

Ariella writes


"The models being used today are opaque, unregulated, and incontestable, even when they're wrong. The math destruction posed by algorithms is the result of models that reinforce barriers, keeping particular demographic populations disadvantaged by identifying them as less worthy of credit, education, job opportunities, parole, etc.

Now the organizations and businesses that make those decisions can point to the authority of the algorithm and so shut down any possible discussion that question the decision. In that way, big data can be misused to increase inequality. As algorithms are not created in a vacuum but are born of minds operating in a human context that already has some set assumptions, they actually can extend the reach of human biases rather than counteract them.


 

These observations jiggled my memory neurons, and I recalled my first-hand experience with the phenomenon of "math destruction" in a somewhat different context – urban transportation planning. That was in the late 1970s, when transport planners could still sort of cast a cloak over their computer processing innards, and the public was expected to treat it like mysteries of the occult. But, assisted by influential politicans like Lloyd Doggett and Ann Richards, I managed to get access to the inner sanctum (e.g. critical segments of code) and raise questions about the assumptions being incorporated into the algorithms that were determining that public transportation was a loser and churning out the need for more highways instead to handle all the suburuban sprawl and generated traffic the models predicted was inevitable. 

Fortunately, I was able to challenge the modeling  process and somewhat alter the course of local planning, and this has also been done elsewhere in the intervening years. Unfortunately, that success has not been sustained, and recent planning methodology has reverted to the supremacy of the occult, with the cloak now being maintained under the rationale of securing "proprietary information".

 

PredictableChaos
User Rank
Prospector
Re: Just heard about the book
PredictableChaos   2/22/2017 4:40:42 PM
NO RATINGS
Weapons of Math Destruction is a great title.

I see two types of problems here. In one case, some algorithms can discriminate inappropriately based on inputs. FB had one recently where you could buy ads that were targeted by demographics. This caused some legal issues if the ads were for, as one example, housing.

In the second case, the algorithm is appropriate and objective, at least as far as inputs. But the outputs from the algorithm are a reflection of society. For example an auto insurer includes previous claim records as one of their metrics and finds that certain demographics end up with higher rates as a direct result.

Ariella
User Rank
Prospector
Re: Adding it, too
Ariella   2/22/2017 4:38:33 PM
NO RATINGS
@SethBreedlove Oh, yes, I've been there. Even when you point out that said "policy" seems to apply to some people but not to others, they will not admit that people are the ones making the decision about application.

Ariella
User Rank
Prospector
Re: Adding it, too
Ariella   2/22/2017 4:37:14 PM
NO RATINGS
@Jessica @Zimana I hope you enjoy reading it. She also has a few articles avilable online.

SethBreedlove
User Rank
Data Doctor
Re: Adding it, too
SethBreedlove   2/22/2017 4:13:05 PM
NO RATINGS
Re " "Even algorithms have parents, and those parents are computer programmers, with their values and assumptions"  Very important point:  Algorithems appear to faceless but they are products of other people's motivations.  

It kind of reminds of when I speak to a company and I hear someone say "That's our policey." to which my response is "Your policey is what ever you say it is."  The same thing is true for algorithms.  The results are what ever it was programmed to be. 

RobertS453
User Rank
Prospector
Pending Review
RobertS453   2/22/2017 3:12:02 PM
NO RATINGS
This comment is waiting for review by our moderators.

Jessica Davis
User Rank
Blogger
Adding it, too
Jessica Davis   2/22/2017 2:46:43 PM
NO RATINGS
I remember reading about this book when it came out. I just downloaded it to my Kindle. Excited to read it, too.

Zimana
User Rank
Blogger
Just heard about the book
Zimana   2/22/2017 2:19:22 PM
NO RATINGS
I just heard about the book Weapons of Math Destruction through a friend who used it for a reference in his book. I am looking forward to reading it for my writings as well.

Information Resources
More Blogs from Ariella Brown
Social media and mobile devices may seem like a vast and scary unknown to parents of young children with phones and other devices. Now machine learning is being applied to the problem of protecting the kids.
Supply is not meeting the demand for analytics and data science professionals. Here's how one boot camp is looking to fill that gap.
Organizations around the world still have plenty of work ahead of them in terms of building trust in their data and analytics strategies, according to a report by KPMG.
Led by actor Gary Sinise, a pair of organizations are leveraging technology to provide smart homes to disabled veterans.
Radio Show
A2 Conversations
UPCOMING
Jessica Davis
Data Analysts in Training: Meeting Tomorrow's Demand


3/8/2017   REGISTER   0
ARCHIVE
Jessica Davis
Our Bodies, Our Data: Medical Records For Sale


2/21/2017  LISTEN   62
ARCHIVE
Jessica Davis
Energy Analytics: Using Data to Find Savings


2/14/2017  LISTEN   44
ARCHIVE
Jessica Davis
Sharpen Your Analytics & Data Management Strategy


2/8/2017  LISTEN   74
ARCHIVE
Jessica Davis
Analytics: Make the Most of Data's Potential in 2017


1/19/2017  LISTEN   19
ARCHIVE
Jessica Davis
A2 Radio: Can You Trust Your Data?


12/20/2016  LISTEN   70
ARCHIVE
James M. Connolly
Retail Analytics: See Where Style Meets Statistics


12/6/2016  LISTEN   53
ARCHIVE
James M. Connolly
Why the IoT Matters to Your Business


11/29/2016  LISTEN   45
ARCHIVE
James M. Connolly
Will Data and Humans Become Friends in 2017?


11/22/2016  LISTEN   40
ARCHIVE
James M. Connolly
We Can Build Smarter Cities


10/20/2016  LISTEN   31
ARCHIVE
James M. Connolly
Visualization: Let Your Data Speak


10/13/2016  LISTEN   70
Information Resources
Quick Poll
Quick Poll
About Us  |  Contact Us  |  Help  |  Register  |  Twitter  |  Facebook  |  RSS