- by SethBreedlove, Data Doctor
- 7/12/2013 12:16:31 AM
It can be okay to make assumptions, but they need to be documented that they are educated guesses. Sometimes we have to make guesses. Where we get into trouble is when we present those assumptions as facts.
- 7/10/2013 6:05:15 PM
This article was focused specifically on assumptions as they relate to statistical hypothesis testing, a different case, I believe, than what you are describing.
- 7/10/2013 5:32:46 PM
Meta, I was a continuity coordinator for a company before. All I did was assume, matter a fact it was my job to do so. I have to write all my plans according to assumption. I had to think about contingency plans from hostile customers to natural disasters. I know that being a continuity coordinator we over assume things, but I think in our case it is like an insurance plan. It is better to have it, and not have it when something goes wrong. I think it is when someone assume and not following up on there assumption, that is when things get sketchy.
- by BethSchultz, Blogger
- 7/10/2013 3:18:38 PM
OK, fair enough -- bad choice of wording on my end. I was really thinking along the lines that folks can't affort to get lazy or go into automaton mode and not THINK as they undertake these sorts of projects.
- 7/10/2013 2:56:15 PM
Well, I would not say that it is about keeping on your toes. Rather, I would say that it is about learning the proper use of tools. You don't (or shouldn't) use power tools without reading the manual, wearing safety glasses and so on. Statistical tests call for similar preparation. The necessary information may be found in textbooks, software documentation and so on. The reason people get in trouble is not that the process is too complex, but rather that they do not bother to use the proper process.
- by BethSchultz, Blogger
- 7/10/2013 2:31:17 PM
So it seems to me that we can substitute "data" for "pipes" and "intuition" for "expertise" and we'd have the same balancing act in just about any company we look into today!
As for knowing what assumptions are reasonable to make -- sounds like everybody has to keep on their toes!
- 7/10/2013 1:30:33 PM
The fellow evaluating pipes had the thought that he was an expert and could pick a better sample than a random sample. So, what's the problem with that? His next step was going to be a statistical analysis that assumes the sample is random. If he was right in his belief that he knew which pipes were most at risk, he might get away with that. But, if he was wrong, he would run the risk of failing to detect important weaknesses. Also, even if his expertise did give him some ability to identify vulnerable pipes (very likely true to some degree), it's stil not a good idea to trust that entirely. If he did that and a failure occurred later in one of the pipes he had determined to be low risk, it would not look good for him at litigation time.
In his case, I would say the ideal was to do two things 1) use a random sample as the basis for a statistical analysis, and 2) also use his engineering expertise to identify those pipes he believed to be at high risk, and inspect those as well. Kind of like wearing suspenders and a belt.
As for how to know what assumptions it is reasonable to make, that's not a simple thing. You need to learn the assumptions associated with each test you use, how well the test stands up if the assumptions are not quite true (a test that can tolerate a little error here is called "robust") and what alternatives to use when you can't assume much. This is the heart of statistical analysis.