Real-time URL protection while preserving privacy
For over 15 years, Google Safe Browsing has protected users from phishing, malware, unwanted software, and more by identifying and alerting users to potentially dangerous sites on more than 5 billion devices worldwide. As attackers become more sophisticated, we see the need for defenses that can adapt as quickly as the threats they protect against. That's why we're excited to announce a new version of Safe Browsing that will provide real-time, privacy-preserving URL protection for users using standard mode Safe Browsing protection in Chrome.
Current Scenario
Chrome automatically protects you by flagging potentially dangerous sites and files with Safe Browsing, which detects thousands of unsafe sites every day and adds them to its lists of dangerous sites and files.
Until now, for privacy and performance reasons, Chrome first checked visited sites against a locally stored list of known unsafe sites, which is updated every 30-60 minutes – this is done using hash-based checks.
But dangerous sites have adapted – today, most of them exist for less than 10 minutes, which means that by the time the locally stored list of known dangerous sites is updated, many of them have already managed to slip through and cause harm if the user visited them during this period. In addition, the list of dangerous sites in Safe Browsing continues to grow rapidly. Not all devices have the resources necessary to maintain this growing list, and may not always receive and apply list updates at the frequency required for full protection.
Enhanced Protection Mode Safe Browsing helps you stay ahead of these threats with technologies such as real-time list checking and AI-based classification of malicious URLs and web pages. We created this mode as an option to give users the ability to choose whether to share more security-related data to get stronger protection. This mode has proven to be very beneficial from checking listings in real time, so we decided to move it to the standard default security mode with a new API that doesn't pass the URLs of the sites you visit to Google.
Implement secure, privacy-preserving real-time browsing
How it works
To ensure real-time protection, you must check against a list maintained on the Safe Browsing server. The list on the server can add unsafe sites immediately after detecting them, so it is able to intercept sites quickly. It can also grow as large as needed since the Safe Browsing server is not limited in the same way as user devices.
What happens behind the scenes in Chrome:
When you visit a site, Chrome first checks its cache to make sure the site's address (URL) is already known to be safe (see “Fast and reliable experience” for more information).
If the URL you are visiting is not in the cache, it may not be secure, so real-time verification is necessary.
Chrome obfuscates URLs following rules URL hashing rules to convert it to a 32 byte full hash.
Chrome truncates full hashes to 4-byte hash prefixes.
Chrome encrypts the hash prefixes and sends them to a confidential server (for more information, see the Keeping your data private section).
The Privacy Server strips all possible user IDs and sends encrypted hash prefixes to the Safe Browsing server over a TLS connection, where the requests are mixed with many requests from other Chrome users.
The Safe Browsing server decrypts hash prefixes and matches them against a server-side database, returning full hashes of all unsafe URLs that match one of the hash prefixes sent by Chrome.
Once Chrome receives unsafe full hashes, it checks them against the full hashes of the visited URLs.
If at least one match is found, Chrome will issue a warning.
Keeping your data private
To maintain user privacy, we partner with Fastlya cloud platform providing content delivery, computing, security and surveillance services, and uses a privacy server Oblivious HTTP (OHTTP) you can learn more about Fastly's commitment to user privacy at Customer Trust page. With OHTTP, Safe Browsing doesn't see your IP address, and your Safe Browsing checks are mixed with checks sent by other Chrome users. This means that Safe Browsing cannot match the URLs you submit while browsing the web.
Before the hash prefixes leave your device, Chrome encrypts them using the Safe Browsing service's public key. These encrypted hash prefixes are then sent to the privacy server. Since the privacy server does not know the private key, it cannot decrypt the hash prefixes, which ensures privacy from the privacy server itself.
The privacy server then removes potential user identifiers, such as your IP address, and sends encrypted hash prefixes to the Safe Browsing server. The privacy server operates independently of Fastly, which means Google does not have access to potential user identifiers (including IP address and User Agent) from the original request. Once the Safe Browsing server receives the encrypted hash prefixes from the privacy server, it decrypts them using its private key and then proceeds to check the list on the server side.
Ultimately, Safe Browsing sees your URL hash prefixes but not your IP address, and the privacy server sees your IP address but not your hash prefixes. Neither party has access to your identity or hash prefixes. This way, your online activities remain private.
Speed and reliability
Compared to hash-based verification, real-time verification requires sending a request to the server, which increases latency. We have applied several techniques to ensure a smooth and responsive browser experience.
Before performing a real-time check, Chrome checks the global and local cache on your device to avoid unnecessary delays.
The global cache is a list of hashes of known safe URLs that is maintained by Safe Browsing. Chrome receives it in the background. If the full hash of the URL is found in the global cache, Chrome will consider it safe and perform a check based on the hash.
The local cache, in turn, is a list of complete hashes saved from previous Safe Browsing checks. If there is a match in the local cache and the cache has not yet expired, Chrome will not send a request to the Safe Browsing server in real time.
Both caches are stored in memory, so checking them is much faster than sending a request over the network in real time.
Additionally, Chrome uses a wait mechanism in case of failed or slow requests. If real-time requests fail, Chrome goes into sleep mode and switches to hash-based checks for a while.
We are also in the process of implementing asynchronous mechanism, which will allow the site to load while the real-time scan is in progress. This will improve the user experience since real-time verification will not block the page from loading.
What real-time, privacy-preserving URL protection means to you
For Chrome users
In the latest update to Chrome for desktop, Android, and iOS, we've enhanced the default Safe Browsing protection mode to scan sites using the Safe Browsing protocol in real time, without sharing your browsing history with Google. You don't need to take any action to take advantage of this enhanced feature.
If you need more reliable protection, we recommend that you enable enhanced protection Safe Browsing. You may be wondering why you need enhanced protection if in standard protection mode you get real-time URL protection – the fact is that in standard protection mode the real-time protection feature can only protect you from sites that Safe Browsing has already been confirmed as unsafe. On the other hand, Enhanced Protection mode is able to use additional information along with advanced machine learning models to protect against sites that Safe Browsing has not yet confirmed as unsafe, for example because the site was created very recently or is hiding its true behavior from Safe Browsing's detection systems .
Enhanced protection goes beyond real-time URL inspection, providing deep scanning for suspicious files and additional protection against suspicious Chrome extensions.
For enterprises
Real-time scanning in the standard Safe Browsing protection mode is enabled by default in Chrome. If necessary, it can be configured using a policy SafeBrowsingProxiedRealTimeChecksAllowed. It's also worth noting that for this feature to work in Chrome, businesses may need to explicitly allow traffic to Fastly's privacy server. If the server is unavailable, Chrome will downgrade the check to hash-based checks.
For developers
While Chrome is the first platform to make these protections available, we plan to make them available to developers who want to use them for non-commercial purposes through the Safe Browsing API. Using the API, developers and privacy server operators can collaborate to better protect users of their products from rapidly evolving threats from adversaries while preserving their privacy. To learn more, stay tuned for our developer documentation, which will be published on the site Google for developers.
