Overview of Google Surveys. Why it is not suitable for serious research

Before starting to develop or improve a product, the team must collect the necessary inputs: the demand for the product on the market, the characteristics expected by the target audience, the price range, the portrait of the audience (real or potential), competitive decisions, and so on. Panel polls are excellent for working on a new product – online polls conducted on a specialized platform.

The advantages of online panels include:

Having a database of registered respondents,
Ability to customize the logic of the sequence of questions.

This allows you to interview a wide audience within a single survey, distributing respondent segments across different branches of the questionnaire.

In addition, most panels have functionality that increases the convenience of analyzing the received data:

Quantitative and percentage data for each answer,
View answers for each respondent,
Graphs and other clear visualization of the results,
The possibility of using cross-tabs,
Additional data, such as the time taken for each respondent.

It is also worth noting that panel polls are a relatively inexpensive form of research. The cost of the survey depends on the number of questions in the questionnaire, the specifics and size of the sample of respondents. So, the complete passage of the questionnaire for 15-20 questions by one respondent costs, on average, 100-150 rubles (depending on the online panel).

Read more about panel polls in the article by Nina Kulikova “Survey-response: how panel surveys affect the life of a product.”

The number of online platforms for audience research is constantly increasing. In this article, we will tell you about a service that not all researchers have heard about, and which is rarely shown in the info field.

Google surveys

The service appeared in 2012 under the name Google Consumer Surveys, in 2016 the name was reduced to the current Google surveys. Someone might think that this service is something similar to, well-known, Google Forms, but this is not so.

Both services, Surveys and Forms, are survey tools. Their main difference is that you can not only integrate the Surveys profile on the site for data collection, but also use it in the online panel format.

In this article, we will consider in detail the capabilities of the system and go through all the steps from registering in the service to receiving the survey results.

registration

The first problem that can be encountered already at the registration stage is the lack of a list of Russian countries. As it turns out, even if the country is not on the list, this does not prevent a poll from being launched for its citizens.

Create Survey

After registration, you can begin to create a survey. The researcher is given a wide selection of question formats (13 types), and the difference between some is difficult to determine:

The only answer
Two answers with an image (choice between one of two answers + image),
Multiple answer options (multiple choice),
Several answer options with an image (multiple selection + image),
Rating scale (the answer to the question is marked on the scale – 1/5, 3/7, etc.),
Rating scale with text (text, question in the text + answer is marked on the scale),
Rating scale with image,
Open (the answer fits into the text box),
Open with image,
A series of images (selection of two images with a click),
Menu with an image (the only answer with an image),
The choice between large images (a series of images, but up to 5 options),
Video (questions become available after watching the video).

It should be borne in mind that Google Surveys has a significant limitation on the number of questions: no more than 10 in one profile. This is directly related to the technology used to search for respondents – replacing the paywall with the survey (we will return to the question of the quality and reliability of the answers of such respondents). There is another type of respondent – users of the Google Opinion Rewards Android application, but in Russia when creating a survey it is impossible to select such respondents.

After compiling a list of questions, you need to choose one of two types of respondents: a combination of visitors to regular and mobile websites or users of Google Opinion Rewards. For a number of countries, you can specify sample settings: gender, age, and geography of users within the country.

For Russia, audience targeting is not provided, the polls use only random sampling (no settings), a combination of regular and mobile traffic visitors. Thus, screening questions remain the only filter of the audience for the Russian Federation (in one questionnaire there can be up to 4 questions for selecting respondents).

When adding screening questions, remember that they have their own limitations. in connection with the service policy:

You can’t specify the city of residence,
Place of work (company),
Financial information (availability of debts, loans, etc.),
Etc.

After compiling the questionnaire and setting audience parameters, you need to specify the target number of responses and the frequency of the survey (from once to regularly once a quarter). The cost of one completed questionnaire, by default, varies from 0.1 € to 1.50 €, and may be higher when using a specific screening. After you have set up the questionnaire, it must be sent for verification to the service department, as well as get the final cost. This step may be the most difficult moment to start the survey.

The procedure itself takes no more than a day, but passing the test according to all the rules of the service (especially if there are screening issues) can be obtained far from the first time. It all depends on the tasks in the questions and the restrictions that the service imposes. Moreover, the rules are scattered across several pages of the service’s help, and unexpected problems may appear at the stage of checking the questionnaire. So, for example, for some reason, you cannot leave an open answer for numerical values. Even more problems arise due to the human factor when checking the questionnaire. It often turns out that by sending the same questionnaire for verification, you can get completely different edits.

It should be noted that in most panels the conditions for compiling questionnaires are explicitly stated, as well as information prohibited for distribution, for example:

Using profanity,
Questions about illegal services and products,
Offensive language
Information on violence and unlawful acts,
Any information of a pornographic nature,
Questions about narcotic, psychotropic substances,
Any other points related to violation of applicable law.

Typically, the validation of a questionnaire is related to the setting of its logic, i.e. the rules for the transition of respondents between questions, depending on the answers. At the same time, creating a survey remains responsible for the content of the questionnaire, wording of questions, answer options, and more.

When using Google Surveys to collect data in a widget on your site, you need to be prepared to pay for each response to the questionnaire. All functional services to run polls (even on your site) must be paid. But with the active use of such an instrument, paying for a “license” for a certain period is more often more profitable than paying for each answer.

To run surveys, you should consider a wider range of services, for example, Questionnaire, Survey Monkey, Survey Gizmo. Assess their capabilities and cost as applied to their tasks, as well as visualization of data collection widgets on the site. And then make a decision.

Results Analysis Functionality

After the support service has validated the questionnaire, it is sent to the respondents, and we begin to receive answers. The functional analysis of the results is quite standard.

For each answer there is a graph with the results.

It is possible to create cross-diagrams.

You can apply filters by gender, age, and geography (where applicable).

Unloading results is available only in Excel format.

At the same time, there is no way to exclude answers within the framework of the service; this can only be done manually in Excel-upload. Unlike other online panels, in which it is possible to exclude already collected answers (for example, invalid due to the short time taken to complete the questionnaire) and not pay for them, Google Surveys must pay for all collected answers.

Surveys on Google Surveys are shown as an alternative to paywall on sites of various publications. This raises doubts about the quality of respondents’ answers. In fact, the respondents’ motivation is aimed at overcoming the paywall, i.e. rapid completion of the survey, which naturally affects the answers. Moreover, in the questionnaire itself there is no any check for obviously incorrect / noise answers.

As a result, as expected, the quality of the answers is greatly affected: in our test survey, meaningless answers to open questions reached 30%, and how many of them were put randomly in closed questions remains a mystery. Any agreement on the results after the completion of the survey is not possible, so that all excluded respondents only reduce the final paid sample.

This factor has a very negative impact on the survey results. So, in most panels there is always the opportunity to challenge the quality of the respondent’s answers.

Summary

With a comparable cost of the questionnaire passed, as well as taking into account the labor required to start the survey, Google Survey loses to analogues like the Questionnaire or Oprosso in key parameters:

Organization of the survey at the creation stage: limitation on the number of questions and difficulties with launching due to non-obvious validation.

Lack of customization of the logic for passing the questionnaire: there is no way to divide the audience into segments within the questionnaire, only filtering.

Selection of respondents. Unfortunately, there are very few proposed filters, and targeting in Russia is impossible in principle.

The quality of the answers provided.

The inability to exclude and appeal obviously wrong answers.

Whether you should try this service is up to you to decide, but based on the results of our acquaintance with it, the conclusion was unambiguous: Google Survey is not applicable for serious product research.

This article was prepared jointly with UX-analyst AGIMA Maksimenko Kirill.