Is your analysis representative of the overall population?

Is your analysis representative of the overall population?

Who uses the internet?

Here's a quick summary of the breakdown of social activity by age in the United States.

Percentage of adults in the United States who use social networks as of January 2018, by age group
Percentage of adults in the United States who use social networks as of January 2018, by age group

It shows that with age comes lower activity, but even among the 50-64-year-old consumers it's still at a whopping 64%.

As a comparison, the percentage of Americans between the ages of 50-64 who regularly take surveys is less than 5%.

Next, take a look at the graphic below, which outlines where consumers spend their time when on the internet.

Most popular daily online activities of adult internet users in the United States as of February 2017, by age group.
Most popular daily online activities of adult internet users in the United States as of February 2017, by age group.

Again, this is divided by age: light blue is 18-29. Dark blue is 30-59. And Grey is anything over 59.

If we focus on the 60 or younger crowd, we can see that the distribution of internet activities is relatively even. The younger group tends to use instant messaging, social networking, and video content more, while the older crowd over-indexes on news sites and web searches (which often lead to blogs, news portals, and forums). Even for platforms that are popular among younger audiences, such as social media, the divide between the 18-29 and 30-59 age groups is not as significant as it used to be. (Note that this data is from 2017 and the divide has likely lessened further, partly due to increased smartphone use.)

We use many various data sources to capture users of all ages

  1. This is why we collect data from all kinds of platforms - forums, news sites, blogs, YouTube, product review sites and e-commerce sites. 
  2. We know of the skews inherent in certain data sources. It is for this reason that we apply various corrections at the level of data input each month. This allows us to ensure that the input data closely resembles an even distribution of the US population from 18-64. 
  3. Our mathematical models are tested and validated through hundreds of experiments, both internal and external. Our clients have many a times pitted our results against those gathered through quantitative means and have found a difference of less than 5%. This, given the expediency and level of detail inherent in ethnography, is a massive gain for any client who'd otherwise get mathematical results with no context or understanding of intrinsic human motivation.
Culture is culture. There's no difference between online and offline.

We have a terrible habit as human beings to dismiss things that seem unfamiliar or uncomfortable to us. So, if we're not comfortable with the study of culture on the internet, we tend to brand it as something only relevant to the internet. That argument might have held up in the early 90s when a significantly smaller and highly educated and tech-savvy audience was online. But in today's world, that argument does not stand.

We are now hyper-connected through our cellphones. There isn't a single cohort of consumers that one cannot find or study on the internet. Why? Because through pseudonyms and text-based engagement, consumers feel they can open up on the internet in ways that they cannot in person. They can truly express their beliefs, values, fears, without fear of backlash from their loved ones. They may experience backlash from strangers, but there's a level of personal separation that creates honesty in online discourse and makes it by far THE RICHEST source of information on human beings and their beliefs about pretty much anything and everything.

When we conduct a survey, we don't say ”oh that represents survey culture.” We are more than happy extrapolating the data from 1024 respondents to the entire population, but we struggle with a study that observes tens of thousands of online consumers. Why? Because it's new or different and that makes us uncomfortable. The more we recognize that, the easier it will become to accept the hard facts about the value of the internet to research in the modern world.

[PS: We're not even going into the merits of observational data vs. respondent data here because that is a whole other set of advantages that the internet serves up.]

No research is devoid of skews

In the end, no research is completely free from bias. Surveys are typically answered only by certain types of people, and therefore don't fully represent the range of beliefs in the marketplace. Interviews rely on a small sample size, making large extrapolations difficult, and often involve people who are being paid to participate. Mixed-method approaches require judgement to link qualitative and quantitative results, and are therefore subject to bias as well. The point is, nothing is free from bias. However, in the modern world, we now know there are ways to minimize it. Which is exactly what we've done by:

  1. Relying on extremely large N samples (big data from millions of consumers). This is a proven way to minimize bias as shown through the Nobel prize winning framework of Prof. Daniel Kahneman (creator of Behavioral Economics).
  2. Building an industry-leading methodology of data gathering and machine-led contextual analysis that allows us to ensure that inherent biases at the platform level are minimized.
  3. Collecting and analyzing the meta-data behind our own data gathering to constantly look out for ways to get our sample set as close as imaginable to the national distribution. We believe in reflexive ethnography. Which is why we created Predictive Anthropology.
  4. Building a mathematical model (that runs the demographic calculations) that is best-in-class, designed by experts in statistical modelling in the computer and social sciences.

The proof is ALWAYS in the pudding

Ultimately, none of this matters. 

What really gets the ball rolling is our performance. 

Our clients have used our data to launch brands from the ground up, create and launch new products, improve existing products with formula/packaging/labeling changes, launch new innovation platforms, even invest in and/or acquire companies. And they've done all of this at a fraction of the time and cost that they'd otherwise waste in research.

This is the power of Lux for Predictive Anthropology.

We added billions in new revenue to our clients' business, and our work has just begun.