Open data isn't exactly the sexy stuff that gets politicos salivating, but data transparency advocates are hoping it's appealing enough in all the right places.
A lot of people are closely watching how the proposed Digital Accountability and Transparency (DATA) Act fares in Congress. This legislation, re-introduced May 21 after failing to earn approval in 2012, aims to bring out data transparency in federal spending information. It has bipartisan support in both congressional chambers, but it is proceeding more quickly through the House than in the Senate, where it stalled last year, too. In the House, the Committee on Oversight and Government Reform has jurisdiction over the legislation and has approved it unanimously. However, the Senate's jurisdictional entity, the Homeland Security and Governmental Affairs Committee (HSGAC), hasn't acted on the DATA Act.
The Data Transparency Coalition (DTC), a group of technology companies, nonprofit organizations, and individuals, supports the publication of federal data online in consistent, machine-readable formats. Hudson Hollister, the DTC's executive director, told listeners who tuned into our A2 Radio show last week that the group is doing all it can to persuade the HSGAC not to let the legislation go unmarked this year, too.
Hudson Hollister, DTC
The DATA Act would expand the openness that the government tried establishing through what Hollister called the "quite limited" portal USASpending.gov, which provides summaries of federal grants and contracts. The bill would expand the openness initiative by requiring the website to post data on internal spending (for purposes like salaries, supplies, and maintenance) and to provide transaction-by-transaction disclosure, instead of just a summary for each grant and contract.
Importantly, the DATA Act also would move USASpending.gov from under the auspices of the White House's Office of Management and Budget to the Treasury Department's purview. "Theoretically, we'll have the same people who are working on standardizing government financial information also in charge of getting it published, subject to those standards," Hollister said. That would be a drastic improvement, because "right now, standardization and publication are nobody's job."
Should the DATA Act pass, the Treasury would be required to establish standard data formats (to make reports searchable) and to figure out consistent identifiers for basic concepts like agency, program, grantee, grant, contract, and contractee. "Right now, these are all different. There's not even a common identifier across the government for agency -- and we've only got about 100 different federal agencies. It's that bad. There isn't any data architecture."
Of course, the bill also would require agencies to implement the identifiers across all their many systems. "The process will be a long one." But the payoff would be well worth it. "What we'll wind up with is standardized datasets covering federal spending that allow you to relate everything to everything." For example:
You might be able to pull up a particular federal program and see all grants, contracts, and internal spending made under it. "That's impossible today."
You might pull up a particular federal contractor or a grant recipient and see each of the payments it got, on what dates, and from which agencies. "That's impossible today."
You might trace money from start to finish. "Right now that's a complicated process." Congress appropriates the money, and then the Treasury signs a warrant saying which agency gets it. That agency allocates the funds to a particular purpose, obligates the funds to a particular recipient, and instructs the Treasury to pay the recipient. The Treasury then processes the payment. That's complicated enough, but it gets even worse if a subgrantee or subcontractor is involved.
The possible implications for analytics are pretty cool. "For the first time, we'll be able to search through all these stages, all these entitities, and all these transactions without having to do any manual labor."
Think about how such openness could help avert the type of government waste and fraud we've seen in cases like the Solyndra bankruptcy, which cost taxpayers more than $500 million. A DTC member tested this out. In a manually intensive process (because there are no common identifiers), the member came up with a list of stimulus-related loan guarantee recipients that also were registrants with the Securities and Exchange Commission. "After it finished that manual labor, it was able to crunch the numbers, also by manually inputting a lot of those filings, and found that, when you apply investment analyst standards, far and away the riskiest recipient of a federal stimulus loan guarantee based on its SEC filings was Solyndra."
Had such analysis been possible when the government was handing out those loan guarantees, chances are pretty good a different decision would have been reached about one particular company, if not many others.
Hollister said the Executive Branch has given no official statement on the DATA Act, but if such legislation could help the government avoid such scandal and waste, it's hard to imagine what reasonable officials would opt against it. But maybe I'm being too naive. What do you think?
Hear our full A2 Radio broadcast with Hollister on demand by clicking here.
So that brings the question, assuming this passes how much is it going to cost us to standardize data across all of these departments? I'm sure it will be a painful process to begin but the long term gains should more than make up for the work required to bring everyone together.
Well then I hope this or at least parts of it make it through. I can imagine how the CBO must deal with this type of issue. My guess is that they spend almost as much time trying to decipher the data as they do making sure everything adds up.
This can only be a good thing. But, being an extrememely complicated puzzle to be put together by bureaucrats and politicians, it may not work as promised or come anytime within a reasonable time. Ever tried to navigate throught a government website? Not an easy thing to do, and this open data project can hopefully work many times more usefully than the typical government project.
@SaneIT -- good question, and the truth is that some of this has been happening, but piecemeal. One of Hollister's big points is that no single entity within the government has been charged with establishing a standard data framework. That's why the DATA Act would put Treasury in charge.
A lot of this sounds like something that should have happened long ago. Why would you not want a common framework for all the data that our government is using to make decisions every day? I don't think it will stop every instance of abuse or that we'll avoid every mistake but at least having that commonality will let those making decisions know what they are asking for and what they are looking at.
Diego Klabjan, chair of the INFORMS University Analytics Program Committee and program director for Northwestern University's Master of Science in Analytics program, gives his advice for figuring out where to get an advanced analytics degree.
What Works: Open Source Analytics Software International Institute for Analytics WebinarOn Wednesday, Sept. 24, join IIA CEO and Co-Founder Jack Phillips, along with featured guest Gary Spakes, as we explore the five modernization stages that analytics hardware/software have experienced. We will discuss the considerations when calculating total cost of ownership of the analytics ecosystem.
2014 VA Interactive Roadshow -- Cary, NCThe 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Essential Practice Skills for Analytics Professionals Drawing on best practices from the field, this INFORMS course helps analytics professionals add value from beginning to end: listening to clients, framing the central problem, scoping a project, defining metrics for success, creating a work plan, assembling data and expert sources, selecting modeling approaches, validating and verifying analytical results, communicating and presenting results to clients, driving organizational change, and assessing impact.
Analytics 2014 The Analytics 2014 Conference is a two-day, educational event for anyone who is serious about analytics. This annual event brings together hundreds of professionals, industry experts and leading researchers in the field of analytics. All Analytics members save $500 on conference fees by using promo code ACAA.
Premier Business Leadership Series 2014 The Premier Business Leadership Series is an exclusive event for senior executives and decision makers that focuses on solving the current issues that affect governments and businesses globally. The Series is a unique learning and networking experience focused on the most innovative leadership strategies and analytic solutions for competing in todayâs global economy.
2014 VA Interactive Roadshow -- BostonThe 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Data Exploration & Visualization Get hands-on training that focuses on the critical steps in the process of analyzing data: accessing and extracting data, cleaning and preparing data, exploring and visualizing data. This INFORMS course will use several of the most popular software tools intensively, and provide an overview of the range of software options.
Foundations of Modern Predictive Analytics In this INFORMS course, learn about modern predictive analytics, the science of discovering and exploiting complex data relationships. This course will give participants hands-on practice in handling real data types, real business problems and practical methods for delivering business-useful results.
2014 VA Interactive Roadshow -- AtlantaThe 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
LEADERS FROM THE BUSINESS AND IT COMMUNITIES DUEL OVER CRITICAL TECHNOLOGY ISSUES
The Current Discussion
Visual Analytics: Who Carries the Onus? The Issue: Data visualization is an up-and-coming technology for businesses that want to deliver analytical results in a visual way, enabling analysts the ability to spot patterns more easily and business users to absorb the insight at a glance and better understand what questions to ask of the data. But does it make more sense to train everybody to handle the visualization mandate or bring on visualization expertise? Our experts are divided on the question. The Speakers: Hyoun Park, Principal Analyst, Nucleus Research; Jonathan Schwabish, US Economist & Data Visualizer
The hospitality industry gathers massive amounts of customer data, and mining that data effectively can yield tremendous results in terms of improved CRM, better-targeted marketing spend, and more efficient back-end processes. Roger Ares, vice president of analytics at Hyatt Corp., discusses the ways he and his staff use big data.
Charged with keeping track of travel assets, including employees, iJET International relies on data management best-practices and advanced analytics to keep its clients in the know on current and potential world events affecting travel, Rich Murnane, Director of Enterprise Data Operations & Data Architect, told All Analytics in an interview from the 2014 SAS Global Forum Executive Conference.
Jason Dorsey, chief strategy officer for the Center for Generational Kinetics and keynote speaker at last month's SAS Global Forum 2014, describes how Gen Y professionals are enhancing the makeup of multigenerational analytics organizations.
From analytics talent development to the power of visual analytics, All Analytics found a variety of common themes circulating throughout the exhibition floor and session discussions at the 2014 SAS Global Forum and SAS Global Forum Executive Conference events held last month in Washington, DC.
Talking with All Analytics live from the 2014 SAS Global Forum Executive Conference, Eric Helmer, senior manager of campaign design and execution for T-Mobile, discussed the importance of customer data -- starting internally -- in devising the mobile operator's marketing plans.
The big-data analytics market can be a confusing place. Among the vendors vying for your dollars are traditional database management providers, Hadoop startup services, and IT giants. In this video, All Analytics editors Beth Schultz and Michael Steinhart sit down in a Google+ Hangout on Air with Doug Henschen, executive editor of InformationWeek. Henschen discusses use cases for big-data analytics, purchase considerations, and his recent roundup of the top 16 big-data analytics platforms.
At the National Retail Federation BIG Show last month, All Analytics executive editor Michael Steinhart noted a host of solutions for tracking and analyzing customer activity in retail stores. From Bluetooth beacons to RFID tags to NFC connections to video analytics, retailers must find the right combination of tools to help optimize the shopper experience, streamline operations, and boost revenues.
The days when historical shipment trends and gut feelings were enough to forecast retail demand accurately are long over. SAS chief industry consultant Charles Chase outlines the benefits of pulling real-time sales information from point-of-sale and product scanner systems, then flowing that data into dynamic forecasting tools from SAS.
With today's advanced visual analytics tools, you can stream data into memory for real-time processing, provide users the ability to explore and manipulate the data, and bring your data to life for the business.
Dynamic data visualizations let analysts and business users interact with the data, changing variables or drilling down into data points, and see results in a flash. Advance your use of data visualization with tools that support features like auto-charting, explanatory pop-ups, and mobile sharing.
No doubt your enterprise is amassing loads of data for fact-based decision-making. Hand in hand with all that data comes big computational requirements. Can traditional IT infrastructure handle the increasing number and complexity of your analytical work? Probably not, which is why you need a backend rethink. Big data calls for a high-performance analytics infrastructure, as Fern Halper, a partner at the IT consulting and research firm, Hurwitz & Associates, discusses here.
Redbox's bright-red DVD kiosks are all but ubiquitous these days, located in more than 28,000 spots across the country. Jayson Tipp, Redbox VP of Analytics and CRM, provides an insider's look at how the company has accomplished its phenomenal nine-year growth.
InterContinental Hotels Group (IHG), a seven-brand global hotelier, has woven analytics into the fabric of its operations. David Schmitt, director of performance strategy and planning, shares IHG's analytics story and his lessons learned.