How to find out how much traffic is coming from hidden requests in Google Ads [скрипт]

Here is the code:

/***************************************************
* Undefined Search Terms Report
* @version 1.1
* @author: Naman Jindal (Optmyzr)
****************************************************/

var LAST_N_DAYS = 30; // Number of previous days to include in report
var EMAILS = ['example@example.com']; // Array of Emails to be notified and given access to the results in a Google Sheet
var PRIMARY_METRIC = 'Cost'; // E.g. Impressions, Cost, Clicks

function main() {
  var map = {};
  var DATE_RANGE = getAdWordsFormattedDate(LAST_N_DAYS, 'yyyyMMdd') + ',' + getAdWordsFormattedDate(1, 'yyyyMMdd');
  var query = [
    'SELECT Date, Impressions, Cost, Clicks FROM ACCOUNT_PERFORMANCE_REPORT',
    'WHERE AdNetworkType1 = SEARCH', 'DURING', DATE_RANGE
  ].join(' ');
  
  var rows = AdsApp.report(query).rows();
  while(rows.hasNext()) {
    var row = rows.next();
    map[row.Date] = {
      'ACTUAL': 0,
      'QUERIES': 0
    };
    map[row.Date].ACTUAL = parseInt(row[PRIMARY_METRIC], 10);
  }
  
  var query = [
    'SELECT Date, Query, Impressions, Cost, Clicks FROM SEARCH_QUERY_PERFORMANCE_REPORT',
    'WHERE AdNetworkType1 = SEARCH',
    'DURING', DATE_RANGE
  ].join(' ');
  
  var rows = AdsApp.report(query).rows();
  while(rows.hasNext()) {
    var row = rows.next();
    map[row.Date].QUERIES += parseInt(row[PRIMARY_METRIC], 10);
  }
  
  var output = [];
  for(var date in map) {
    output.push(Wed, 10 May 2023 18:59:16 +0000.ACTUAL, mapWed, 10 May 2023 18:59:16 +0000.QUERIES, (mapWed, 10 May 2023 18:59:16 +0000.ACTUAL - mapWed, 10 May 2023 18:59:16 +0000.QUERIES) / mapWed, 10 May 2023 18:59:16 +0000.ACTUAL]);
  }
  
  if(!output.length) {
    Logger.log('No data in the account'); 
  }
  
  var TEMPLATE_URL = 'https://docs.google.com/spreadsheets/d/1G1-zPqm0kqQjZSPEwS8cfndYVzkswbyl80SiLlqiPF8/edit#gid=0';
  var template = SpreadsheetApp.openByUrl(TEMPLATE_URL);
  var ss = template.copy(AdsApp.currentAccount().getName() + ' - Undefined Search Terms Report by ' + PRIMARY_METRIC);
  ss.addEditors(EMAILS);
  
  var tab = ss.getSheets()[0];
  tab.getRange(2,1,tab.getLastRow(),tab.getLastColumn()).clearContent();
  tab.getRange(2,1,output.length,output[0].length).setValues(output).sort([{'column': 1, 'ascending': true}]); 
  
  var msg = 'Hi,nPlease find below the undefined search terms report for your Google Ads account:n'+ss.getUrl();
  MailApp.sendEmail(EMAILS.join(','), AdsApp.currentAccount().getName() + ' - Undefined Search Terms Report by ' + PRIMARY_METRIC, msg); 
  Logger.log("Your report is ready at " + ss.getUrl());
}


function round_(num,n) {    
  return +(Math.round(num + "e+"+n)  + "e-"+n);
}

function getAdWordsFormattedDate(d, format){
  var date = new Date();
  date.setDate(date.getDate() - d);
  return Utilities.formatDate(date,AdsApp.currentAccount().getTimeZone(),format);
}

Results: traffic on hidden requests soared to 50-60%

We analyzed traffic over the past 60 days across three accounts. For the purity of the experiment, we took accounts with only one active campaign. We share observations.

1. Online store of catering equipment (B2B)

Traffic – about 100 clicks per day. The campaign was launched on “lamp” keys like “buy a refrigeration unit”.

We loaded the script, executed and got a table with data:

On the second tab of the report, the graph:

On September 1, the percentage of traffic for hidden requests soared from almost 0 to 55%. And then it remained at about the same level. That is, we no longer understand which requests are driving more than half of the traffic.

Previously, the report had many requests related to hardware modifications. For example, “tables for catering nso1m”, “tables nso 1m87B”, etc. Now Google has hidden most of them, leaving only the obvious.

2. Online store of children’s goods

Traffic – about 150 clicks per day.

The situation is similar. But there was a bit more unidentified traffic here initially. The increase in its share began on 31.08 (up to 21%) and continued on 1.09 (up to 65.7%).

3. Online service for small businesses

Traffic – 200-250 clicks per day. In contrast to the previous examples, here ads are launched by keywords in broad, phrase and exact match (and not broadly with a modifier).

There is also a jump in “unidentified” traffic here – but its share is less (about 40%).

It is difficult to look for patterns in numbers – it all depends on the specific account and the settings used. Nevertheless, it is clear that about half of the traffic now goes for hidden requests.

Where to get semantics and negative keywords for Google ads now

In theory, you can overcome hidden queries if you create separate ad groups for each key in an exact match. Then key data is the same as query data. But, objectively, hardly anyone will do this.

What is worth focusing on:

1. Carefully collect semantics before launching a campaign, especially when it comes to narrow niches. For example, we described non-trivial ways of choosing semantics for SEO – the first 3 are quite suitable for context.
2. Work out negative keywords as much as possible before launching the campaign. What and how to do – described in the guide by negative keywords. They also talked about the possibilities of automating the selection of negative keywords using the free media planner Click.ru…
3. Look towards auto strategies (current list with analysis – link). Here you need to test – it may or may not fire. So, at the beginning of September, we created a test automatic campaign for Click.ru and received interesting results (we will write about this in one of the following articles).

Let’s summarize

• Hidden searches on Google Ads are not new. They were there even before the search terms were cut. But their share was no more than a few percent, in some cases it could reach 20%. Now their share reaches 50-60% and more.
• Most queries are hidden if the campaign was launched on broad match keywords. For an exact match, the percentage of hidden requests is below.
• Despite the cut in the report, it still works. Yes, this is not the same abundance of search phrases. But still they are. Therefore, after launching an advertisement, go here and add (as much as possible) the semantics with new keys and negative keywords.

Analyze your accounts to understand how much traffic is coming from unidentified requests. We will be glad if you share your observations in the comments.