How to find out how much traffic is coming from hidden requests in Google Ads [скрипт]

How to find out how much traffic is coming from hidden requests in Google Ads [скрипт]

In early September, Google cut its search terms report. It used to be a source of live semantics and negative phrases. And now … Now what? Do you know how much traffic goes for undefined requests?

You can compare “before” and “after” using a script. It collects data on clicks / impressions / costs for hidden requests for a specified period, displays the data in a google table and builds a graph.

We have already assessed the “scale of the disaster” in our accounts and shared our observations. Try it yourself.

Background: what happened to the Google Ads search report

In early September, a notification appeared in Google Ads accounts:

Google said there won’t be as many phrases in the search terms report as it used to be. AT SearchEngineLand comments a Google spokesman explained the changes:

To comply with our standards in the field of personal data protection and to ensure the safety of information about users, we have changed the “Search queries” report – now it will display only those queries that were asked by a significant part of users.

The logic is clear. Requests with confidential data such as “lost passport surname number” should not be publicly available.

But we are talking about all requests that are entered in isolated cases. And there are many such requests. And this is already creating problems for advertisers. For instance:

  • In a narrow topic, especially in conditions of tight targeting and budget, it is difficult to collect semantics and get inexpensive traffic. As we did before: we launched advertising by keywords in broad accordance with the modifier, after which we replenished the list of keywords based on search queries. As a result, in a few weeks we reached inexpensive and stable traffic (here’s a case). Now the list of available requests has been reduced.
  • The search terms report is also useful for finding negative keywords. We see for which irrelevant queries there were transitions, and add them to negative keywords. In a couple of weeks, advertising by keywords in broad / phrasal match we achieve the fact that the percentage of non-targeted clicks falls, and after it the cost per lead.

The question is how much Google cut the search terms report and what it left us in the end. We found a script that estimates how much traffic comes from unknown requests and tested it on several accounts. But first things first.

Script for calculating the percentage of unrecognized requests

The script was shared by Frederick Vallaeys in article at SearchEngineLand. The script generates a report that shows by day the percentage of impressions, clicks or costs for hidden requests.

It works like this: it extracts data from two reports (with the Ad Network Type = Search filter):

The script compares the data and displays the result in Google Sheets.

Setting up and getting the result

Copy the script code to the account you want to analyze (how to work with scripts – described here).

Make changes to the code:

  1. In the line var EMAILS, be sure to specify an email – it will have access to the report.
  2. In the var PRIMARY_METRIC line, specify which metric you want to get a slice by. The default is Cost (costs), you can replace it with Clicks (clicks) or Impressions (impressions).
  3. Optionally, in the line var LAST_N_DAYS, you can set the period for retrieving statistics. The default is 30 days. We recommend setting 60 days for clarity.

That’s all. Run the script, and you receive a notification in your mail. The link will be a report. On the first sheet – data by day, on the second – a graph.

Here is the code:

/***************************************************
* Undefined Search Terms Report
* @version 1.1
* @author: Naman Jindal (Optmyzr)
****************************************************/

var LAST_N_DAYS = 30; // Number of previous days to include in report
var EMAILS = ['example@example.com']; // Array of Emails to be notified and given access to the results in a Google Sheet
var PRIMARY_METRIC = 'Cost'; // E.g. Impressions, Cost, Clicks

function main() {
  var map = {};
  var DATE_RANGE = getAdWordsFormattedDate(LAST_N_DAYS, 'yyyyMMdd') + ',' + getAdWordsFormattedDate(1, 'yyyyMMdd');
  var query = [
    'SELECT Date, Impressions, Cost, Clicks FROM ACCOUNT_PERFORMANCE_REPORT',
    'WHERE AdNetworkType1 = SEARCH', 'DURING', DATE_RANGE
  ].join(' ');
  
  var rows = AdsApp.report(query).rows();
  while(rows.hasNext()) {
    var row = rows.next();
    map[row.Date] = {
      'ACTUAL': 0,
      'QUERIES': 0
    };
    map[row.Date].ACTUAL = parseInt(row[PRIMARY_METRIC], 10);
  }
  
  var query = [
    'SELECT Date, Query, Impressions, Cost, Clicks FROM SEARCH_QUERY_PERFORMANCE_REPORT',
    'WHERE AdNetworkType1 = SEARCH',
    'DURING', DATE_RANGE
  ].join(' ');
  
  var rows = AdsApp.report(query).rows();
  while(rows.hasNext()) {
    var row = rows.next();
    map[row.Date].QUERIES += parseInt(row[PRIMARY_METRIC], 10);
  }
  
  var output = [];
  for(var date in map) {
    output.push([date, map[date].ACTUAL, map[date].QUERIES, (map[date].ACTUAL - map[date].QUERIES) / map[date].ACTUAL]);
  }
  
  if(!output.length) {
    Logger.log('No data in the account'); 
  }
  
  var TEMPLATE_URL = 'https://docs.google.com/spreadsheets/d/1G1-zPqm0kqQjZSPEwS8cfndYVzkswbyl80SiLlqiPF8/edit#gid=0';
  var template = SpreadsheetApp.openByUrl(TEMPLATE_URL);
  var ss = template.copy(AdsApp.currentAccount().getName() + ' - Undefined Search Terms Report by ' + PRIMARY_METRIC);
  ss.addEditors(EMAILS);
  
  var tab = ss.getSheets()[0];
  tab.getRange(2,1,tab.getLastRow(),tab.getLastColumn()).clearContent();
  tab.getRange(2,1,output.length,output[0].length).setValues(output).sort([{'column': 1, 'ascending': true}]); 
  
  var msg = 'Hi,nPlease find below the undefined search terms report for your Google Ads account:n'+ss.getUrl();
  MailApp.sendEmail(EMAILS.join(','), AdsApp.currentAccount().getName() + ' - Undefined Search Terms Report by ' + PRIMARY_METRIC, msg); 
  Logger.log("Your report is ready at " + ss.getUrl());
}


function round_(num,n) {    
  return +(Math.round(num + "e+"+n)  + "e-"+n);
}

function getAdWordsFormattedDate(d, format){
  var date = new Date();
  date.setDate(date.getDate() - d);
  return Utilities.formatDate(date,AdsApp.currentAccount().getTimeZone(),format);
}

Results: traffic on hidden requests soared to 50-60%

We analyzed traffic over the past 60 days across three accounts. For the purity of the experiment, we took accounts with only one active campaign. We share observations.

1. Online store of catering equipment (B2B)

Traffic – about 100 clicks per day. The campaign was launched on “lamp” keys like “buy a refrigeration unit”.

We loaded the script, executed and got a table with data:

On the second tab of the report, the graph:

On September 1, the percentage of traffic for hidden requests soared from almost 0 to 55%. And then it remained at about the same level. That is, we no longer understand which requests are driving more than half of the traffic.

Previously, the report had many requests related to hardware modifications. For example, “tables for catering nso1m”, “tables nso 1m87B”, etc. Now Google has hidden most of them, leaving only the obvious.

2. Online store of children’s goods

Traffic – about 150 clicks per day.

The situation is similar. But there was a bit more unidentified traffic here initially. The increase in its share began on 31.08 (up to 21%) and continued on 1.09 (up to 65.7%).

3. Online service for small businesses

Traffic – 200-250 clicks per day. In contrast to the previous examples, here ads are launched by keywords in broad, phrase and exact match (and not broadly with a modifier).

There is also a jump in “unidentified” traffic here – but its share is less (about 40%).

It is difficult to look for patterns in numbers – it all depends on the specific account and the settings used. Nevertheless, it is clear that about half of the traffic now goes for hidden requests.

Where to get semantics and negative keywords for Google ads now

In theory, you can overcome hidden queries if you create separate ad groups for each key in an exact match. Then key data is the same as query data. But, objectively, hardly anyone will do this.

What is worth focusing on:

  1. Carefully collect semantics before launching a campaign, especially when it comes to narrow niches. For example, we described non-trivial ways of choosing semantics for SEO – the first 3 are quite suitable for context.
  2. Work out negative keywords as much as possible before launching the campaign. What and how to do – described in the guide by negative keywords. They also talked about the possibilities of automating the selection of negative keywords using the free media planner Click.ru
  3. Look towards auto strategies (current list with analysis – link). Here you need to test – it may or may not fire. So, at the beginning of September, we created a test automatic campaign for Click.ru and received interesting results (we will write about this in one of the following articles).

Let’s summarize

  • Hidden searches on Google Ads are not new. They were there even before the search terms were cut. But their share was no more than a few percent, in some cases it could reach 20%. Now their share reaches 50-60% and more.
  • Most queries are hidden if the campaign was launched on broad match keywords. For an exact match, the percentage of hidden requests is below.
  • Despite the cut in the report, it still works. Yes, this is not the same abundance of search phrases. But still they are. Therefore, after launching an advertisement, go here and add (as much as possible) the semantics with new keys and negative keywords.

Analyze your accounts to understand how much traffic is coming from unidentified requests. We will be glad if you share your observations in the comments.

Similar Posts

Leave a Reply Cancel reply