Is traditional market research under threat?

With the advent of online interview panels gathering information has never been easier. Forums such as Facebook and Twitter are cheap and fast information gathering platforms. A new study from Carnegie Mellon University shows that analyzing data from Twitter yields the same results as conducting a public opinion poll.

A CMU team from the computer science department looked at sentiments expressed in a billion Twitter messages between 2008 and 2009. The researchers then use simple text analysis methods to filter out updates about the economy and politics and determine if the overall sentiment of the update was positive or negative. The CMU team found that people’s attitudes on consumer confidence and presidential job approval were similar to the results generated by well-reputed, telephone-conducted public opinion polls, such as those conducted by Reuters, Gallup and pollster.com.

For at least some topics, CMU Assistant Professor Noah Smith thinks this kind of passive information gathering could work. “With seven million or more messages being tweeted each day, this data stream potentially allows us to take the temperature of the population very quickly,” Smith said. “The results are noisy, as are the results of polls. Opinion pollsters have learned to compensate for these distortions, while we’re still trying to identify and understand the noise in our data. Given that, I’m excited that we get any signal at all from social media that correlates with the polls.”

The CMU researchers did notice that Twitter sentiments had much more day-to-day variation compared to data gathered from traditional polling data. To compensate, the team averaged the Twitter results over a number of days; at that point, the results were generally quite similar to polling data.

For example, on Twitter as in life, consumer confidence slumped in 2008 and started to revive last spring. And Twitter updates showed the same general decrease in presidential job performance approval through 2009 as was seen in traditional polls. There were a few discrepancies — enough that the CMU folks don’t recommend using Twitter to poll for election results just yet. Still, researchers hope that better natural language processing (NLP) techniques will make Twitter and other social media a valuable source of public opinion information in the future.

The question is, is traditional information gathering under threat by the advent of social media? How reliable is the output of this information? What are the pit falls of this kind of information gathering? Food for thought .

Source. mashable.com