Prodded by the Obama administration, the Census Bureau has announced the scrapping of the annual statistical publication (close to the hearts of many, it would seem) as a cost-cutting measure. Effective Oct. 1, the bureau is no longer collecting data for the statistical compendium, an official site notice reads.
The Statistical Abstract, which the Census Bureau describes as “the authoritative and comprehensive summary of statistics on the social, political, and economic organization of the United States," includes about 1,000 pages of data. It can tell you things like the number of fire departments active in firefighting in 2008 and how many fires they fought that year.
Robert J. Samuelson, an economics columnist for the Washington Post, delineates the abstract's virtues in an August op-ed. It presents a huge amount of information from government and private sources all in one place, and it provides footnotes to the information sources that allow readers to seek more details if they wish.
As unintentional as it may have been, Samuelson makes the greatest case against the Statistical Abstract's continued publication. Many other technologies, organizations, and resources bring together considerable data in one place. Compare the amount of data contained in the listings of a single Google search, for example, to the amount of data contained in the 1,000 printed pages of the Statistical Abstract. And Google presents its data at a cost that's much less (to the public, at least) than the $2.9 million in taxpayer dollars reportedly required to assemble the abstract each year.
Also, as Samuelson observes, this data is all from somewhere else, just like the data brought to you in a Google search or as links in a social network stream. The abstract is, in official words, a “summary” and nothing more. Today, researchers have many more comprehensive summaries of vast amounts of data at their disposal.
What’s behind all this fear that the retirement of a summary of statistical data with a 19th century origin will somehow make us less informed? In an age of government bloat, shouldn't we be embracing this as the cost-saving measure it's intended to be? Well, like the disappearance of news printed in ink on dead trees, I suspect our fear has something to do with our difficulty in separating the tools from the functions they perform.
Possibly as early as the third century BC, a Jewish sect known as the Essenes is believed to have used a group of caves on the shore of the Dead Sea to store a library of 972 texts, including biblical and other documents, on parchment and papyrus scrolls. Today the information would probably fit on your hard drive. Would paper stored in caves be better?
Sure, the abstract contains a lot of neat data all conveniently collected in one place, but it’s all available elsewhere, too -- or could soon be, so long as Washington continues to work toward a goal of transparency. It’s much more important for the analytics community to hold our leaders’ feet to the fire so that vital data is always available in easily accessible form than for us to preserve an archaic tool used to present that data. What do you think?
Thanks for your comments and welcome to the community! We do quite a bit of sparring around here on these issues but all friendly, of course. I respect the opinions expressed here but would like to point out that, in many cases, the same, or similar data can be gathered from the original sources, in many cases also government agencies like the U.S. Department of Agriculture simply by clicking through the links already provided on the abstract's own online spreadsheets. This is not data collected by big ruthless corporations, agenda driven political action groups etc. but data collected by one government agency which is then taken by another government agency and reduced into tables. If our argument is that the extraction of this data and translation into comparable form from its original source, which is also the government, is costly on a par with the expense of creating the abstract, this is something that is debatable in my mind. I believe a detailed cost analysis would be needed to prove the point.
Beth, thanks for the welcome. I am not sure what it means to "implicitly trust the data." I trust it enough to utilize it in testing early hypotheses in order to determine whether to delve further into a particular issue. Resources found on Google may not come with such a high degree of initial credibility.
Of course, such reliance on other sources could come with time and experience. Apparently, there are many who have not had good experiences with other sources, yet. Hence, the opposition to the demise of the Statistical Abstract.
Some might even wonder where to start looking for sources of data that represent the entire nation, rather than whatever segment of society is funding the database. It may be more than urban myth that private sector data is often gathered or disseminated in support of a cause. Federal data, on the other hand, provides an a priori source of rebuttable information.
Someone studying societal trends could get a "snapshot" of the nation.
Likewise, a company considering entering a new market may study the governmental data for clues on how to approach the new territory. They would not have to rely solely on data from competitors, industry organizations, or other sources. And, they would have a basis for judging a source's credibility.
If they make the decision to proceed, they will eventually gather their own data, from their own experience with their own customers which will prompt them to pursue their initial activities, refine their approach, or reconsider the decision to participate in that market.
While the Federal data does not need to be presented as a printed book, it does need to be (A) readily accessible so that it is available to anyone - from a concerned citizen or the smallest business to the most lucrative venture - and (B) in a form that will not change over time so historical studies will remain feasible.
The Statistical Abstract is a useful place to start - whether in print, HTML, Excel, or a SAS database. In print, scattered across the nation in private and public libraries, it meets the criteria of both accessibility and permanence.
Steven, welcome to the AllAnalytics.com community and thanks for sharing your first-hand experience using the Abstract. I would imagine, too, that you implicitly trust the data presented in the Abstract. I'm guessing you'd not likely be able to say the same of some of the types of data served up via Google or other sources?
The Statistical Abstract is something that has been the base for a lot of work, a place to begin. Its presence has been something to depend upon when starting a new project. I don't need to refer to it everyday, but I am glad it is there when I need it. I suppose I could use Google, but it seems that they present me with a lot of websites that are not relevant - along with a few that may be useful.
Not having the Statistical Abstract is likely to result in much greater ineffeciencies which, in turn, will result in costs to taxpayers who would have used it. I haven't sat down to compute the relative costs, but would not be surprised if the privatized costs will exceed the cost-savings to the government. In other words, it is a cost to the taxpayers; a hidden tax.
I'm new here, but not new to the use of satistics. Government statistics tend to be rather broadbased, but fairly neutral - with less of an "agenda" to slant them. Could the Statistical Abstract be replaced by private data sources or by a compilation like Google? Sure. Not everything has to be done by government, just because government has always done them. But, there is something a bit "overboard" in saying "the abstract contains a lot of neat data all conveniently collected in one place, but it’s all available elsewhere, too -- or could soon be." Apparently, not. If so, then there would not be the reaction. If users could just go on using data from their new sources, they wouldn't care. Everyone is too busy to fight over things that do not matter. This matters. Keeping the Statistical Abstract matters.
Shawn, You present a strong argument in favor of moving on from tradition, the resources we have now at our disposal are mind boggling, however, I am going to go against reason here and argue the publishing of this value resource continue.
In a government and society filled with bloat, what is 2.9M with a deficit in the trillions ? This falls under the category of things we should in my opinion be spending on, in addition to basic infrastructure improvements, education, youth programs...etc.
I signed the petition to keep the Abstract alive ( and I am guessing you did too ) - because in my opinion it is the right thing to do.
A reply to Shawn's comment: "I think that a private non-profit or for-profit entity could probably carry on in the government's place producing the abstract annually at potentially considerable cost savings and increased efficiency."
I think this really is the heart of the debate, and it gets at a larger debate about what kind of society we want to live in. (No really, it does.) The printed version of Stat Abstract sells for 40 bucks. The online version is free. I think we all understand that it costs a heck of a lot more than that to produce it, or else its elimination wouldn't be at issue. Any private entity which wanted to take up the monumental task of replacing the Statistical Abstract would need to charge substantially more to make the effort sustainable, at a price that would place the data outside the reach of most of the people who use the current, taxpayer-supported version.
So the question boils down to whether we think it's worth $3 million a year to place this information in the hands of everyone, or only the select few who could afford the much higher price tag of a privately published replacement? In other words, does society have a role in paying for a basic set of statistical data to make sure it's available to everyone, whether it be for your high school kid's homework or a solo practitioner or somebody who is launching the next startup out of his or her garage?
I'll leave it to others to decide whether a "no" answer to that question is more of a libertarian perspective or an elitist one, but I think we would be a sadder, more backward-looking society if we decided that public investments like this didn't pay us back many times over.
I think Mathematiker's point about bringing the data together in one place is right on target, but I also want to address the assumption that all this data will still be findable out there in some other format from some other provider if the Stat Abstract is no longer here to bring it all together.
The Preface to Stat Abstract says that both government and private sources contribute to its mix of data. I checked on what that would mean if we lost the publication, and found out that about 100 of those private sources, which contribute to 179 tables in the book, require copyright permission, meaning that almost 13 percent of the tables in the book are copyrighted. All but a few of those tables are approved for the online and CD-ROM versions.
This large chunk of copyrighted data will no longer be publicly accessible by any means, short of separately negotiated agreements with those approximately 100 private sources (have fun). So Statistical Abstract is literally irreplaceable, because no one outside the Census Bureau's Statistical Compendia Branch is capable of collecting it all without re-establishing all those copyright arrangements -- that is, without doing what the Statistical Compendia Branch already does, and does so much better, more efficiently, and more economically than anyone else could.
Like anyone else who has looked at the numbers, I am horrified by the state of our public finances and the threats they pose to our future, but I don't think that the places where government provides business and the taxpaying public with the most fantastic returns on investment are the places you look to first for cuts. I think this case serves as a useful reminder that the opposite of good government isn't less government; it's bad government.
Thanks so much for taking the time to contribute to the conversation and, as Beth suggested, don't ever be afraid to voice your opinions. That's what the boards are for and we all really appreciate the feedback. I guess my point isn't so much that the abstract can't serve a purpose. I suspect it probably could, especially in its current digital form. As I suggested to John, in a comment on his Point post, I think that a private non-profit or for-profit entity could probably carry on in the government's place producing the abstract annually at potentially considerable cost savings and increased efficiency. The question is whether something like the Abstract must exist thus justifying public expenditures to keep it up and running. I would argue that the answer to this question is that the Abstract is expendable for all the reasons I have mentioned. I'm glad you mentioned newspapers. As a veteran of the newspaper industry for almost two decades and at several different papers, I suppose I hold a somewhat different view than those who have faithfully read (and even some who have worked at) newspapers and tend to romanticize their inherent value. While it is true that freedom of the press is a cornerstone of our democracy, it must be remembered that this does not necessarily imply only newspapers as they exist today. In fact, the current "objective journalism" model granting newspapers their authoritative role has been a very successful sort of mass marketing campaign which has thrived in the industry for over a century increasing readership and advertising revenue by broadening customer base. Arguably first employed at the liberal New York Times under pioneering publisher Adolph Ochs and then his descendents, it eventually spread to the conservative Los Angeles Times and Chicago Tribune and to the liberal Washington Post. But as anyone who has ever worked at a newspaper can tell you, it is just that: a marketing campaign. Coverage of issues still varies from paper to paper based on the political affiliation, business and social connections of its publisher, editors etc. and the approach has come at a terrible cost, fewer operating newspapers thanks, not simply to radio, television and the Internet, but to lack of differentiation and overall blandness of content. Again, in my view, a single source of data is not necessarily a virtue. It is simply easier to digest. The explosion of data on the Internet and from other sources makes both the Abstract and newspapers obsolete in their current form and both will need to reinvent themselves in order to be relevant to future generations.
Diego Klabjan, chair of the INFORMS University Analytics Program Committee and program director for Northwestern University's Master of Science in Analytics program, gives his advice for figuring out where to get an advanced analytics degree.
2014 VA Interactive Roadshow -- Cary, NCThe 2014 VA Interactive Roadshow will feature SAS® Data Management and SAS® Visual Analytics experts covering topics like prepping data for VA and VA integration with SAS® Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Essential Practice Skills for Analytics Professionals Drawing on best practices from the field, this INFORMS course helps analytics professionals add value from beginning to end: listening to clients, framing the central problem, scoping a project, defining metrics for success, creating a work plan, assembling data and expert sources, selecting modeling approaches, validating and verifying analytical results, communicating and presenting results to clients, driving organizational change, and assessing impact.
Analytics 2014 The Analytics 2014 Conference is a two-day, educational event for anyone who is serious about analytics. This annual event brings together hundreds of professionals, industry experts and leading researchers in the field of analytics. All Analytics members save $500 on conference fees by using promo code ACAA.
Premier Business Leadership Series 2014 The Premier Business Leadership Series is an exclusive event for senior executives and decision makers that focuses on solving the current issues that affect governments and businesses globally. The Series is a unique learning and networking experience focused on the most innovative leadership strategies and analytic solutions for competing in todayâ€™s global economy.
2014 VA Interactive Roadshow -- BostonThe 2014 VA Interactive Roadshow will feature SAS® Data Management and SAS® Visual Analytics experts covering topics like prepping data for VA and VA integration with SAS® Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Data Exploration & Visualization Get hands-on training that focuses on the critical steps in the process of analyzing data: accessing and extracting data, cleaning and preparing data, exploring and visualizing data. This INFORMS course will use several of the most popular software tools intensively, and provide an overview of the range of software options.
Foundations of Modern Predictive Analytics In this INFORMS course, learn about modern predictive analytics, the science of discovering and exploiting complex data relationships. This course will give participants hands-on practice in handling real data types, real business problems and practical methods for delivering business-useful results.
2014 VA Interactive Roadshow -- AtlantaThe 2014 VA Interactive Roadshow will feature SAS® Data Management and SAS® Visual Analytics experts covering topics like prepping data for VA and VA integration with SAS® Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
LEADERS FROM THE BUSINESS AND IT COMMUNITIES DUEL OVER CRITICAL TECHNOLOGY ISSUES
The Current Discussion
Visual Analytics: Who Carries the Onus? The Issue: Data visualization is an up-and-coming technology for businesses that want to deliver analytical results in a visual way, enabling analysts the ability to spot patterns more easily and business users to absorb the insight at a glance and better understand what questions to ask of the data. But does it make more sense to train everybody to handle the visualization mandate or bring on visualization expertise? Our experts are divided on the question. The Speakers: Hyoun Park, Principal Analyst, Nucleus Research; Jonathan Schwabish, US Economist & Data Visualizer
The hospitality industry gathers massive amounts of customer data, and mining that data effectively can yield tremendous results in terms of improved CRM, better-targeted marketing spend, and more efficient back-end processes. Roger Ares, vice president of analytics at Hyatt Corp., discusses the ways he and his staff use big data.
Charged with keeping track of travel assets, including employees, iJET International relies on data management best-practices and advanced analytics to keep its clients in the know on current and potential world events affecting travel, Rich Murnane, Director of Enterprise Data Operations & Data Architect, told All Analytics in an interview from the 2014 SAS Global Forum Executive Conference.
Jason Dorsey, chief strategy officer for the Center for Generational Kinetics and keynote speaker at last month's SAS Global Forum 2014, describes how Gen Y professionals are enhancing the makeup of multigenerational analytics organizations.
From analytics talent development to the power of visual analytics, All Analytics found a variety of common themes circulating throughout the exhibition floor and session discussions at the 2014 SAS Global Forum and SAS Global Forum Executive Conference events held last month in Washington, DC.
Talking with All Analytics live from the 2014 SAS Global Forum Executive Conference, Eric Helmer, senior manager of campaign design and execution for T-Mobile, discussed the importance of customer data -- starting internally -- in devising the mobile operator's marketing plans.
The big-data analytics market can be a confusing place. Among the vendors vying for your dollars are traditional database management providers, Hadoop startup services, and IT giants. In this video, All Analytics editors Beth Schultz and Michael Steinhart sit down in a Google+ Hangout on Air with Doug Henschen, executive editor of InformationWeek. Henschen discusses use cases for big-data analytics, purchase considerations, and his recent roundup of the top 16 big-data analytics platforms.
At the National Retail Federation BIG Show last month, All Analytics executive editor Michael Steinhart noted a host of solutions for tracking and analyzing customer activity in retail stores. From Bluetooth beacons to RFID tags to NFC connections to video analytics, retailers must find the right combination of tools to help optimize the shopper experience, streamline operations, and boost revenues.
The days when historical shipment trends and gut feelings were enough to forecast retail demand accurately are long over. SAS chief industry consultant Charles Chase outlines the benefits of pulling real-time sales information from point-of-sale and product scanner systems, then flowing that data into dynamic forecasting tools from SAS.
With today's advanced visual analytics tools, you can stream data into memory for real-time processing, provide users the ability to explore and manipulate the data, and bring your data to life for the business.
Dynamic data visualizations let analysts and business users interact with the data, changing variables or drilling down into data points, and see results in a flash. Advance your use of data visualization with tools that support features like auto-charting, explanatory pop-ups, and mobile sharing.
No doubt your enterprise is amassing loads of data for fact-based decision-making. Hand in hand with all that data comes big computational requirements. Can traditional IT infrastructure handle the increasing number and complexity of your analytical work? Probably not, which is why you need a backend rethink. Big data calls for a high-performance analytics infrastructure, as Fern Halper, a partner at the IT consulting and research firm, Hurwitz & Associates, discusses here.
Redbox's bright-red DVD kiosks are all but ubiquitous these days, located in more than 28,000 spots across the country. Jayson Tipp, Redbox VP of Analytics and CRM, provides an insider's look at how the company has accomplished its phenomenal nine-year growth.
InterContinental Hotels Group (IHG), a seven-brand global hotelier, has woven analytics into the fabric of its operations. David Schmitt, director of performance strategy and planning, shares IHG's analytics story and his lessons learned.