- by Jamescon, Editor
- 1/5/2015 1:43:50 PM
@kq4ym. So what we need is predictive analytics to tell us what data we will need in the future. Of course, that gets into the eternal loop of needing to know what data we need to build out the predictive analytics.
- by kq4ym, Data Doctor
- 1/5/2015 10:35:39 AM
There does seem to be a temptation to store everything because the "costs" are getting so low. But, it's not always easy to figure just what data that's not used now might well be very useful in the future. If we had a good way to predict the future it wouldn't be so hard to choose. But that's not going to happen.
- 1/5/2015 9:34:48 AM
@Broadway- Well, I think that's the perception, but I wonder if we take into account the greater complexity, the increased staff, the capabilities of the staff to handle larger data, and the fact that the money could be going elsewhere, I wonder if it is true for some mid-sized and smaller companies.
- by magneticnorth0, Data Doctor
- 12/31/2014 8:07:03 PM
@Jamescon I think academic researchers will serve us well by analyzing organizations' database/transaction logs. Nothing like a disinterested third party to look into this. Just thinking about the process makes me feel geekily giddy.
I wonder how much of the logs are kept, though? If they stay for long, then that's big data.
- by Jamescon, Editor
- 12/31/2014 12:12:06 PM
@Seth. Thanks for sharing the item about the professor at UMass Dartmouth (not to be confused with the college in New Hampshire that was the inspiration for Animal House) and his Playstation supercomputer. Outside the box thinking -- innovation -- is a strange beast. We all say we want it, but we are afraid of giving up on the status quo. It's not just a people thing, either. We've all worked at companies that say they want a culture of innovation, and they would love to emulate those companies that do it. Oh, but change is tough. Those all-day management offsites that are aimed at crafting new ideas and strategies typically result in great ideas, but the only things that get implemented are a couple of incremental changes in ops.
Innovation isn't just about how you think but how you and your organization adapt to change, how they embrace those changes. Sometimes that even means rewarding failure, when a seemingly good idea just doesn't work out.
It's tough but it can be done. That's why I think that 20% of data that holds promise but not immediate reward can be a key element in innovation. The organization still has to embrace that change, but the data can provide insight, ideas, and some degree of validation for innovation.
- by SethBreedlove, Data Doctor
- 12/30/2014 10:11:16 PM
I'm wondering if we will start to see companies start to be more inventive about their big data needs. For example, Gaurav Khanna a Dr. at the University of Massachusetts Dartmouth physics department built a super computer out of 200 Playstation 3s. An ingenious low-cost solution to running out and buying a supercomputer. While it can't do everything a real supercomputer can do, it shows me that there will be a lot of out of the box thinking in the future.
- 12/30/2014 9:52:08 PM
The problem with the supply chain analogy is that everything in the supply chain is somehting you know you will use. If you make cars from steel you don't stockpile aluminum just in case you decide to make cars from aluminum. Why stockpile data you aren't using?
On the other hand, if you buy enough random stuff, you might learn to make a new product with it. You might be able to do that with data.
- by Broadway0474, Blogger
- 12/30/2014 9:45:28 PM
@David, are you gambling or hedging your bets? Not sure if the comparison works, but for years the idea of a super lean supply chain was all the rage, until one disaster after another struck those supply chains. I remember delving into the impact of the big Japanese earthquake a few years back, and because so many carmakers were reliant on one supplier for a particular part, of that OEM got shut down in the quake, that carmaker couldn't make cars ... at all. I think it's similar here, but instead of risks, a lean data operation risks missing unexpected opportunities.
- 12/30/2014 8:12:33 PM
@Broadway- See, I'm actually in favor of keeping everything or running super lean. If you're keeping some middle ground unless you are really good at predicting the future, you're just gambling.