Count me among the most enthused about all the attention that everything data, from big data to analytics to quality to culture, is getting these days. Though the attention has grown steadily for some time, the subject is enjoying a distinct and welcome uptick. Data is all the rage -- and not just in the technology community, as evidenced in recent New York Times features on the promise and the perils of big data.
But Im worried. Everything data seems to demand top-down leadership. As editor in chief Beth Schultz opined not so long ago, creating a data-driven culture must be driven from the top, for example. In my day job, I consult on data quality, and I espouse the same thing.
But really, must everything data come from the top down? I started my career at AT&T Bell Labs as what we would now call a data miner. My job -- my whole job -- was to work with people throughout AT&T to understand their problems and with others in the labs to think differently about those problems and propose new solutions. I was fortunate enough to contribute two or three important ideas, but AT&T's president never heard about them.
Since setting out on my own, Ive been blessed with dozens of aggressive mid-level to upper-level managers as clients. They grew sick and tired of dealing with bad data and concluded that there had to be a different way. And they found it, often improving quality by an order of magnitude or more. Take that, Redman and Schultz. Maybe senior leadership isnt essential, after all!
Alas, I dont think that conclusion stands up to even the gentlest scrutiny.
Bell Labs is but a shadow of its former self. And what happened to the other great national labs? Or the smaller, more focused labs that once decorated the American business landscape? Theyre gone and, sadly, unlikely to return. Writing in the NYT, Jon Gertner, author of the forthcoming The Idea Factory: Bell Labs and the Great Age of American Innovation, explains the demise of Labs-like, data-fueled innovation.
And my view of the diffusion of innovation at AT&T is far too optimistic. The teams I worked with laid the foundation for data quality improvement (even earning two patents) and completed projects that saved the company more than $100 million annually, but we did not even get a chance to work on other important data quality issues.
Finally, practically any manager can start a data quality effort within his or her span of control, but Ive yet to observe such an undertaking expand without the leadership of a more senior manager. It seems to me that data quality programs go just as far as the highest-ranking leader can credibly demand.
Anything data, never mind everything data, is truly transformative. The data revolution can start anywhere. But Schultz is right. For it to penetrate everywhere, top-down leadership of change is essential. Would you agree?