Open Data: Averting Another Solyndra

Open data isn't exactly the sexy stuff that gets politicos salivating, but data transparency advocates are hoping it's appealing enough in all the right places.

A lot of people are closely watching how the proposed Digital Accountability and Transparency (DATA) Act fares in Congress. This legislation, re-introduced May 21 after failing to earn approval in 2012, aims to bring out data transparency in federal spending information. It has bipartisan support in both congressional chambers, but it is proceeding more quickly through the House than in the Senate, where it stalled last year, too. In the House, the Committee on Oversight and Government Reform has jurisdiction over the legislation and has approved it unanimously. However, the Senate's jurisdictional entity, the Homeland Security and Governmental Affairs Committee (HSGAC), hasn't acted on the DATA Act.

The Data Transparency Coalition (DTC), a group of technology companies, nonprofit organizations, and individuals, supports the publication of federal data online in consistent, machine-readable formats. Hudson Hollister, the DTC's executive director, told listeners who tuned into our A2 Radio show last week that the group is doing all it can to persuade the HSGAC not to let the legislation go unmarked this year, too.

Hudson Hollister, DTC
Hudson Hollister, DTC

The DATA Act would expand the openness that the government tried establishing through what Hollister called the "quite limited" portal, which provides summaries of federal grants and contracts. The bill would expand the openness initiative by requiring the website to post data on internal spending (for purposes like salaries, supplies, and maintenance) and to provide transaction-by-transaction disclosure, instead of just a summary for each grant and contract.

Importantly, the DATA Act also would move from under the auspices of the White House's Office of Management and Budget to the Treasury Department's purview. "Theoretically, we'll have the same people who are working on standardizing government financial information also in charge of getting it published, subject to those standards," Hollister said. That would be a drastic improvement, because "right now, standardization and publication are nobody's job."

Should the DATA Act pass, the Treasury would be required to establish standard data formats (to make reports searchable) and to figure out consistent identifiers for basic concepts like agency, program, grantee, grant, contract, and contractee. "Right now, these are all different. There's not even a common identifier across the government for agency -- and we've only got about 100 different federal agencies. It's that bad. There isn't any data architecture."

Of course, the bill also would require agencies to implement the identifiers across all their many systems. "The process will be a long one." But the payoff would be well worth it. "What we'll wind up with is standardized datasets covering federal spending that allow you to relate everything to everything." For example:

  • You might be able to pull up a particular federal program and see all grants, contracts, and internal spending made under it. "That's impossible today."
  • You might pull up a particular federal contractor or a grant recipient and see each of the payments it got, on what dates, and from which agencies. "That's impossible today."
  • You might trace money from start to finish. "Right now that's a complicated process." Congress appropriates the money, and then the Treasury signs a warrant saying which agency gets it. That agency allocates the funds to a particular purpose, obligates the funds to a particular recipient, and instructs the Treasury to pay the recipient. The Treasury then processes the payment. That's complicated enough, but it gets even worse if a subgrantee or subcontractor is involved.

The possible implications for analytics are pretty cool. "For the first time, we'll be able to search through all these stages, all these entitities, and all these transactions without having to do any manual labor."

Think about how such openness could help avert the type of government waste and fraud we've seen in cases like the Solyndra bankruptcy, which cost taxpayers more than $500 million. A DTC member tested this out. In a manually intensive process (because there are no common identifiers), the member came up with a list of stimulus-related loan guarantee recipients that also were registrants with the Securities and Exchange Commission. "After it finished that manual labor, it was able to crunch the numbers, also by manually inputting a lot of those filings, and found that, when you apply investment analyst standards, far and away the riskiest recipient of a federal stimulus loan guarantee based on its SEC filings was Solyndra."

Had such analysis been possible when the government was handing out those loan guarantees, chances are pretty good a different decision would have been reached about one particular company, if not many others.

Hollister said the Executive Branch has given no official statement on the DATA Act, but if such legislation could help the government avoid such scandal and waste, it's hard to imagine what reasonable officials would opt against it. But maybe I'm being too naive. What do you think?

Hear our full A2 Radio broadcast with Hollister on demand by clicking here.

Beth Schultz, Editor in Chief

Beth Schultz has more than two decades of experience as an IT writer and editor.  Most recently, she brought her expertise to bear writing thought-provoking editorial and marketing materials on a variety of technology topics for leading IT publications and industry players.  Previously, she oversaw multimedia content development, writing and editing for special feature packages at Network World. In particular, she focused on advanced IT technology and its impact on business users and in so doing became a thought leader on the revolutionary changes remaking the corporate datacenter and enterprise IT architecture. Beth has a keen ability to identify business and technology trends, developing expertise through in-depth analysis and early adopter case studies. Over the years, she has earned more than a dozen national and regional editorial excellence awards for special issues from American Business Media, American Society of Business Press Editors,, and others.

Midmarket Companies: Bring on the Big Data

The "big" in big data is no reflection of the size of the organization embracing its potential.

Push Yourself to New Analytical Discoveries

Take inspiration from Christopher Columbus as you pursue your analytical journeys.

Re: A common framework
  • 7/31/2013 11:56:33 AM

I haven't seen any budget figures, but I think we can definitely expect some short-term pain but long-term gain!

Re: A common framework
  • 7/31/2013 11:41:20 AM

So that brings the question, assuming this passes how much is it going to cost us to standardize data across all of these departments?  I'm sure it will be a painful process to begin but the long term gains should more than make up for the work required to bring everyone together.

Re: A common framework
  • 7/26/2013 8:09:41 AM

SaneIT, me too. The Obama administration has paid a lot of attention to open data, but this legislation, at least in regards to federal spending, really takes the mandate to a whole other level. 


Re: A common framework
  • 7/26/2013 7:16:37 AM

Well then I hope this or at least parts of it make it through.  I can imagine how the CBO must deal with this type of issue.  My guess is that they spend almost as much time trying to decipher the data as they do making sure everything adds up.

Re: A common framework
  • 7/25/2013 10:11:14 AM

This can only be a good thing. But, being an extrememely complicated puzzle to be put together by bureaucrats and politicians, it may not work as promised or come anytime within a reasonable time. Ever tried to navigate throught a government website? Not an easy thing to do, and this open data project can hopefully work many times more usefully than the typical government project.

Re: A common framework
  • 7/25/2013 7:54:43 AM

@SaneIT -- good question, and the truth is that some of this has been happening, but piecemeal. One of Hollister's big points is that no single entity within the government has been charged with establishing a standard data framework. That's why the DATA Act would put Treasury in charge. 

A common framework
  • 7/25/2013 7:51:36 AM

A lot of this sounds like something that should have happened long ago.  Why would you not want a common framework for all the data that our government is using to make decisions every day?  I don't think it will stop every instance of abuse or that we'll avoid every mistake but at least having that commonality will let those making decisions know what they are asking for and what they are looking at.