It has never been easier to collect raw data on anything. Thanks to the internet and advances in computing technology, you can easily gather as much data as you need on any topic you could imagine.
Trouble is, what do you do with it when you’ve got it? Data is, after all, just raw numbers, or occasionally comments. On its own, it means very little, and makes even less sense.
Which is why successful companies employ people or entire departments whose job is to analyse this incoming data and make it into a readable, easily accessed format.
If you do not have anyone to do this essential job, it won’t matter how much data you collect, you are never going to get the information you actually need. A good data analysis will tell you what questions were asked, of how many people, and, most importantly of all, how reliable the raw data was.
A survey of one or two questions asked to a small number of people is never going to be as reliable as a 20 question survey asked of thousands.
The term “significance” has a very specific meaning when you’re discussing statistics. The level of significance of a statistic is the level of confidence you can have in the answer you get.
Generally, those involved with data analysis don’t consider a result significant unless there is at least a 95% certainty that it’s correct.
It should be pointed out that a 95% level of certainty does not mean that the program works on 95% of people, or that it will work 95% of the time.
It simply means that there’s only a 5% possibility that it isn’t what’s influencing the variables involved and so it is causing the changes that it seems to be associated with.