Here's an interesting trend that keeps showing up: A growing number of surveys are being based on informal groups of respondents rather than statistically significant population samples. This means that the results of these surveys may or may not reflect the broader population being studied. There appears to be three drivers of this trend:
1. Cost and speed - it is much cheaper, easier and quicker to do surveys informally, rather than investing in the rigor of statistically significant research methodology. The appeal of doing 100 quick-and-dirty surveys instead of investing in one statistically rigorous survey is obvious.
2. The availability of easy-to-use Internet survey tools - Tools like Survey Monkey make it easy to do quick and simple online surveys. This has resulted in a substantial increase in the number of surveys conducted (and results published) which may or may not measure the population that the survey is trying to understand. Think of it as a litmus test without really knowing whose chemistry you're testing.
3. There is value in the information produced by non-statistical surveys - While informal survey results are not projectable to a broader population, they can be useful. They provide something to react to quickly and cheaply which can help decision-makers and researchers think out-of-the-box. Also, time or resource constraints often eliminate the use of formal surveys and some information is often better than none.
We see the value of non-statistical surveys and occasionally use them in our work. We find them useful for surfacing issues and research topics and in the early stages of our research to help us scope our work. We also like that they are quick, easy and cheap to do.
Link to original post