Big Data Knows Which Way the Wind Blows


Weather analytics is a relatively new discipline, but big data is driving greater accuracy and wider applications.

Weather is often given the chaos theory treatment, where it's considered so sensitive to the initial conditions that sparked its movement -- and may be influenced in drastic ways by small changes -- that an approximation of what may come can be completely and utterly wrong.

Of course, that doesn't mean we stop trying to predict it. And we're getting better, but we're still far from certain and unable to give anything remotely accurate over a long period of time. But big data analytics may be about to change that.

Spearheading the trend is a company called Weather Analytics, which plans to use traditional metrics like barometric pressure, fluid dynamics, and numerical weather prediction together with big data to create a much more accurate model for long-term forecasting.

Traditionally, forecasts grow less accurate as the timeframe moves beyond a month or two. Weather Analytics believes that it can make accurate-enough reports on the scale of years, and that probability indexes can be created to help clients weigh the possible costs of weather-related interference in their long-term plans.

The data Weather Analytics uses to generate its predictions comes from over 33 years of weather reports from across the globe, as well as ongoing data from many countries today. The company site says it uses "more than 50 times the global historical data reported by any other source or provider."

Of course, all models affected by chaos theory are destined to be unpredictable, and it seems unlikely that Weather Analytics will be able to create a system that is wholly accurate, but company founder Bill Pardue told Forbes recently that he is convinced they can do better than what's currently available.

Is it raining where you are?
In looking for global weather databases before the venture even began, Pardue and his partners spoke to many organizations that claimed to have extensive records, but none did. Most repositories were limited to weather patterns in North America and Western Europe, he told Forbes. Meanwhile, MIT research meteorologist John Keller had been archiving and scrubbing global weather data in his spare time at home. His database formed the foundation of the new company.

The combination of historical records and real-time feeds should give Weather Analytics an edge, and if you consider its client base, it's already off and running. In less than a year of operation, Forbes reports, Weather Analytics is inundated with client requests, many of which involve weather pattern data in places where information has been scarce. One client was interested in weather effects on African cotton crops, for example, and another wanted to find out how weather in the British Virgin Islands might affect a 24-month construction project.

No doubt, Weather Analytics will also do what every other big data firm does and rent out portions of its data for other analytics efforts.

What do you think, members? Will big data finally enable accurate long-term weather prediction? Would this kind of service be useful to you? Share your thoughts below.

Related posts:

Jon Martindale, Technology Journalist

Jon Martindale is a technology journalist and hardware reviewer, having been covering new developments in the field for most of his professional career. In that time he's tested the latest and greatest releases from the big hardware companies of the world, as well as writing about new software releases, industry movements,and Internet activism.

How EU's Data Privacy Law Will Impact You

The EU's new General Data Protection Regulation or GDPR offers data protection for consumers and harsh penalties for violators. Here's what you need to know.

Analytics Drive Virtual Reality Game Development

Developing virtual reality games presents more and different challenges than developing conventional video games. Here's how data analytics is helping the process.


Re: Long term Forecast
  • 3/10/2014 3:03:50 PM
NO RATINGS

Looks like this year's Polar Vortex weather has sparked another big data project, as reported in InformationWeek earlier this week.

Re: Long term Forecast
  • 3/6/2014 11:15:51 AM
NO RATINGS

True, Jon. But this seems even more difficult given that the weather intelligence might not prove itself of value for five or more years. I'm not suggesting there's not value here, because I do believe some companies and organizations can and will be able to make better decisions with weather intelligence. It just seems a tough business model to me!

Re: Long term Forecast
  • 3/6/2014 10:43:19 AM
NO RATINGS

I suppose though, that's the same with any new company. Until you've proven your service worthwhile, your customer base will be limited. 

Re: Long term Forecast
  • 3/4/2014 4:12:23 PM
NO RATINGS

It must be a tough sell to a lot of companies, though, as @kq4ym points out. How can a company today be assured that the weather intelligence on which it's basing its long-range decisions on is of any value? 

Re: Long term Forecast
  • 3/4/2014 1:34:34 PM
NO RATINGS

Thanks for all the comments everyone. Glad this is a topic that's piqued your interest. 

It's funny you mention data translation, as it's something I wrote about last year:

http://www.theenterprisecloudsite.com/author.asp?section_id=2844&doc_id=264800

It can go horrible wrong.

On the subject of weather analytics and its accuracy, I don't think this company is necessarily trying to say it can offer you a forecast. Think on the scale of mega-corporations and building firms looking to tackle projects that will take multiple years to finish. In costing up the next few years, it has to take into consideration weather. While analytics can't tell if it A or B is going to happen on what day, it can give an overall idea of the likelyness of certain factors, which can then become part of the budget. 

However, there is the potential with this sort of data to bring about more accurate predictions, as it factors in quite remote weather data, which those producing traditional forecasts don't necessarily consider. Weather's global and long term nature (it has technically been ongoing for billions of years) it's always going to be difficult to be 100 per cent accurate, but by considering decades worth of weather data and global information, we might be getting closer to something that resembles accuracy. 
 

Re: Long term Forecast
  • 3/4/2014 9:09:30 AM
NO RATINGS

@Lyndon, sounds like we were probably watching the same news report! 

As for willing victims -- urr, customers -- you raise a good point, one I think that spans beyond a desire for weather intelligence. Do organizations today rush to data because they're being told all over the place that they're supposed to be more data-driven in their decision making? There is real danger in that.

Re: Long term Forecast
  • 3/3/2014 9:23:27 PM
NO RATINGS

..

Beth writes


This whole winter has been way off target for national forecasters, who predicted a rather mild winter. Apparently, from a news report I watched the other night, only the folks at the Farmer's Almanac got it right from the start.


 

NBC News the other night also reported the astounding accuracy of Farmers Almanac compared with the National Weather Service.

If Weather Analytics is claiming the ability to predict weather years ahead, we've got a bit of a wait before we can assess their accuracy.

Interesting that, short of such a proven record, they're able to market their service. I'd expect that some wealthy moguls or well-placed government bureaucrats would be willing to spend money on Tarot card readings after the right marketing pitch.

 

Re: Long term Forecast
  • 3/3/2014 8:32:20 AM
NO RATINGS

It appears the new company is relying on using it's data from more remote areas to sell it's services. While the additional data may improve forecasting a bit for those areas, I'm not convinced big data is anytime soon going to improve long term forecasting anywhere. There's just way too many variables that are still unknown that still makes it exceedingly difficult to come up with an accurate model.

Re: Long term Forecast
  • 3/3/2014 8:27:06 AM
NO RATINGS

This whole winter has been way off target for national forecasters, who predicted a rather mild winter. Apparently, from a news report I watched the other night, only the folks at the Farmer's Almanac got it right from the start.

Re: Long term Forecast
  • 3/2/2014 8:49:39 PM
NO RATINGS

I think the current back-and-forth over the storm that's supposed to dump 8 inches of snow on me... no, wait, 3-6... okay, make it 2... okay, a coating to an inch, is what's making people doubt forecasts. Thursday, they were telling us to brace for the wrath of winter, and now it's "meh." 

Page 1 / 2   >   >>
INFORMATION RESOURCES
ANALYTICS IN ACTION
CARTERTOONS
VIEW ALL +
QUICK POLL
VIEW ALL +