Increasing Attack Surface on perimeter pentest

Below the cut we will figure out what to do with this.

We will look at the example of our company, Jet Infosystems. Our main domain is jet.su. The task is to find as many IP addresses, domains and subdomains that belong to the company as possible. We'll start by looking up IP addresses, continue by looking for other 2nd level domains, and end with subdomains.

The approach we describe below can also be used for bug hunting if the scope includes all subdomains.

Searching for IP Addresses

If the organization is not small, then most likely it has an autonomous system (AS).

An Autonomous System (AS) is a group of IP addresses that is managed by one or more network operators and routes traffic on the Internet using the BGP protocol.

Data about all AS is stored in the database of regional Internet registries (Regional Internet Registry – RIR), in our case – RIPE.

RIPE is one of five regional Internet registries. RIPE's area of ​​responsibility is Europe, Central Asia and the Middle East.

Through Whois we can contact RIPE for details and, for example, find out in which AS the IP address is located. First, let's look at the A-record (IP address) of the jet.su domain:

❯ nslookup jet.su 
Server:     172.20.10.1 
Address:    172.20.10.1#53  

Non-authoritative answer: 
Name:   jet.su 
Address: 193.203.101.194

Then let's check who owns the IP address 193.203.101.194:

❯ whois 193.203.101.194 
... 
org-name:       CJSC Jet Infosystems 
country:        RU 
org-type:       OTHER 
address:        Russia 127015 Moscow 
address:        Bolshaya Novodmitrovskaya 14, Bldg. 1 
... 
route:          193.203.101.0/24 
origin:         AS29258 
descr:          JETINFO-NET 
mnt-by:         AS2578-MNT 
created:        2022-01-18T13:34:17Z 
last-modified:  2022-01-18T13:34:17Z 
source:         RIPE # Filtered 
...

The legal address and name of the organization coincide with the real data, which means that the IP really belongs to the company and not to the Internet/CDN/cloud provider or hoster. This means we can use the ASN information from the field originwhich owns the IP address, to find out the range of subnets that the company owns.

To do this, you can use one of the resources that provide ASN information, for example, https://bgp.he.net/and look for our ASN there – AS29258.

Great! We received a list of subnets that belong to the company. But pentesters are very lazy people who rarely want to go out from home from the terminal, so colleagues from ProjectDiscovery created the tool asnmap for the following purposes:

❯ asnmap -asn AS29258 -silent 
193.203.100.0/23

So, we have a subnet that belongs to the organization – 193.203.100.0/23. Now let's move on to searching for domains and subdomains.

Domain Search

The search process is divided into two types – horizontal and vertical:

• Horizontal search – we look for domains of the 2nd level and/or higher that are not subdomains of the company’s main domain. In our case – jet.su, jetinfosystems.com, etc.

• Vertical search – search for subdomains of domains found at the horizontal search stage.

Let's focus on horizontal search first.

After receiving a range of IP addresses, the first thing you can and should do is request PTR records for this range.

A PTR (Pointer) record is a type of DNS record that is used to associate an IP address with its domain name. Not to be confused with an A record, where a domain name is associated with an IP address.

Reverse DNS

The PTR record can be requested using standard Linux utilities such as nslookup,digbut for mass scanning there are more convenient tools – dnsrecon, dnsx and others. I'm using dnsx.

Let's set dnsx to the found subnet:

echo "193.203.100.0/23" | dnsx -ptr -resp -o reverse_dns.txt 

Among the subdomains of the main domain jet.su we see other domains, for example, jet.msk.su. Let's filter the output to get all results except jet.su subdomains:

cat reverse_dns.txt | grep -v jet.su

Next, we’ll make a filter to remove subdomains and get only the main domains:

cat reverse_dns.txt | grep -v jet.su | awk -F ' ' '{print $3}' | sed 's/\[//' | sed 's/\]//' | cut -d '.' -f2- | sort | uniq

A good result is four new domains that are related to the company’s infrastructure.

Next, we will use TLS to search for information about company domains and subdomains in TLS certificates.

TLS certificates – SAN and CN fields

A TLS certificate is usually issued for a specific domain name or a wildcard for all subdomains. Of course, the certificate stores information about which domain it is issued to. Here is an example for https://jet.su:

In the CN field we see the domain for which the certificate is issued.

CN (subject Common Names) is the initial field used to indicate the domain name for which the certificate was issued.

There is another field where domains are indicated – SAN.

SAN (Subject Alternative Names) is an extension that allows you to specify additional domain names for which the certificate is also valid.

Knowing the range of IP addresses, you can find out which of them have port 443 open (and others that could potentially have TLS – 8443, 10443…), then go to each of them and look at which domains have certificates issued for them. But don’t do it manually – here again the guys from ProjectDiscovery help us out with a tool tlsx.

Let's run tlsx through our subnet and collect information from the SAN and CN fields:

echo "193.203.100.0/23" | tlsx -p 443,3443,8443,9443,10443 -cn -san -o tlsx_result.txt

Let's filter the result to display only unique domains:

cat tlsx_result.txt | grep -v jet.su | awk -F ' ' '{print $2}' | sed 's/\[//' | sed 's/\]//' | cut -d '.' -f2- | sort | uniq

We can manually remove jet.msk.su subdomains – thus, TLS Probing gave us two company domains.

Not bad, but I would like more.

Reverse Whois

One of the most useful sources for horizontal domain searches is Reverse Whois. It's like Whois reverse only, but only for searching by fields from Whois. Initially, Whois allows us to obtain information about the org field – by IP address or domain name. The ability to reverse search – by org (or other field) – is not available by default.

Let's look at an example. Whois contains information about the organization to which the domain is registered – Joint Stock Company “Jet Infosystems”:

❯ whois jet.su 
% TCI Whois Service. Terms of use: 
% https://tcinet.ru/documents/whois_ru_rf.pdf (in Russian) 
% https://tcinet.ru/documents/whois_su.pdf (in Russian)  

domain:        JET.SU 
nserver:       relay2.jet.su. 193.203.100.130 
nserver:       relay3.jet.su. 2.58.70.121 
nserver:       relay5.jet.su. 79.171.174.21 
nserver:       relay7.jet.su. 51.250.17.123 
nserver:       relay.jet.su. 193.203.100.131 
state:         REGISTERED, DELEGATED 
org:           Joint Stock Company "Jet Infosystems" 
phone:         +74954117601 fax:           +74954117602 
e-mail:        postmaster@jet.su registrar:     RUCENTER-SU 
created:       2003-06-26T20:00:00Z 
paid-till:     2024-06-26T21:00:00Z 
free-date:     2024-07-30 
source:        TCI

We are interested in all domains in the Whois database whose org field matches Joint Stock Company “Jet Infosystems”.

If we implement such a service ourselves, then we need to somehow get all domains in all zones .com, .ru, etc. (some domain registrars provide the ability to download all registered domains in certain zones, for example, Here domains in the .ru zone), then get information from Whois for each… Well, you already understand that we won’t do this manually.

There are various services that provide the ability to search by Whois. Let's take advantage https://www.whoisxmlapi.com/ (they have a free API for 500 requests per month, and it produces more results than completely free analogues) and a simple 200-line script from GitHub https://github.com/Sybil-Scan/revwhoix:

revwhoix -k "Joint Stock Company \"Jet Infosystems\""

Excellent results – 17 2nd level domains that have the field org corresponds Joint Stock Company “Jet Infosystems”. Let's combine the results with the previous stages:

cat tlsx_uniq.txt reverse_dns_uniq.txt reverse_whois_results.txt | sort | uniq

Total 21 domains. Let's move on!

Search subdomains

We have already talked about Reverse DNS and TLS certificates for horizontal search. But these same sources can also be used for vertical.

What will happen next:

• Search the database of issued TLS certificates (crt.sh)

• Various subdomain search services

• Brute force subdomains

Certificate Search (crt.sh)

The principle is similar to Reverse Whois, only the search is based on fields in certificates. This is what crt.sh produces when requested by jet.su:

A lot of tools under the hood use data from crt.sh. I use one of these – subfinder (yes, ProjectDiscovery again and again).

Services for searching subdomains

In addition to crt.sh, there are many services that collect information about subdomains on their own (Security Trails, BinaryEdge, Chaos…) or which may contain information about subdomains (GitHub, Web Archive, VirusTotal). Some of them are completely free, some provide limited access to the API.

Subfinder, in addition to crt.sh, contains the following free services:

• alienvault

• anubis

• commoncrawl

• digitorus

• dnsdumpster

• hackertarget

• rapiddns

• sitedossier

• waybackarchive

In addition to the default services, I use the following set of APIs in my subfinder:

• binaryedge

• chaos

• fullhunt

• github

• intelx

• securitytrails

• shodan

• whoisxmlapi

The above services are partially free – each has demo access for a limited number of API requests.

Here are the launch results subfinder to jet.su request:

subfinder -d jet.su -all -stats

Semi-free securitytrails consistently produces the most data on subdomains, so Highly recommended!

Bruteforce and Permutations

The next step in subdomain reconnaissance is to search for third-level domains this dictionary (4th bruce if the scope is very small). The dictionary is quite large – about 10 million lines, but the brute force goes through relatively quickly (~1 hour). For brute I use the puredns tool:

puredns bruteforce /usr/share/wordlists/best-dns-wordlist.txt jet.su -w subdomains_jet.su.txt

Puredns requires that you explicitly specify the DNS servers to which queries will be sent in the resolvers.txt file in the puredns startup folder (or through the -r flag). I use this list. You also need to keep in mind that puredns is a fairly powerful tool, which in brute force mode with a 10 million dictionary will most likely crash your home Wi-Fi router, so I recommend running it from a VPS.

After this stage, everything is ready for the finale – compiling a dictionary using permutations of already found subdomains.

Permutations from the found subdomains are a way to create a dictionary for smarter brute force. The principle is as follows: based on a given list of subdomains, other potential candidates are generated using various mutations. I use alterx from ProjectDiscovery to create permutations.

The example below shows the operating principle of alterx:

❯ cat test.txt
example.com
test.example.com
❯ cat test.txt | alterx

___ ____ _ __
/_| / / /____ ____| |/_/
/ __ |/ / __/ -_) __/> <
/_/ |_/_/\__/\__/_/ /_/|_|

projectdiscovery.io

[INF] Current alterx version v0.0.3 (outdated)
testwww2.example.com
plugins-test.example.com
storage-test.example.com
test.local.example.com
test.access.example.com
testengineering.example.com
builds.example.com
test-private.example.com
test2018.example.com
login.example.com
mssql.example.com
…
…
test-promotion.example.com
test.www1.example.com
testlab.example.com
test.dev.example.com
[INF] Generated 691 permutations in 0.0039s

Alterx generated 691 variants from two examples, example.com and test.example.com, which could well be 3rd and 4th level domains in an organization.

For our goal, we need to collect subdomains using subfinder and brute force for each domain found at the horizontal search stage, then create permutations from them. Let's demonstrate this on the main domain jet.su:

cat subdomains_jet.su.txt | alterx | dnsx -o alterx_resolved_jet.su.txt

As a result, alterx found 151 subdomains. But not all of them will be useful to us – for example, absolutely any iva.jet.su subdomains will be resolved, because the DNS is configured that way.

By the way, puredns can detect wildcard subdomains, which greatly simplifies the subsequent filter. It is enough to run puredns with the resolve parameter and feed it the wordlist generated by alterx:

puredns resolve alterx_jet.su.txt

We are primarily interested in what new alterx discovered. To do this, I will compare the result obtained with previously discovered subdomains. For such comparisons of results I use anew:

cat alterx_resolved_jet.su.txt | grep -v iva.jet.su | anew -d subdomains_resolved_jet.su.txt

In the output of alterx we see domains that have not been seen before, for example, jas-test.jet.su. That is, alterx generated jas-test.jet.su from the previously found domain jas.jet.su.

Thus, creating a dictionary by rearranging already found subdomains gives us a more “personalized” dictionary, which can then allow us to find interesting services during pentesting or bug hunting.

Conclusion

So we walked through the work flow of searching for assets on the perimeter. Of course, there are nuances at each point, for example, an organization can rent some resources (hosting, clouds), and they will not be part of its AS, but there will be network connectivity with the company’s main infrastructure. Then you need to look for the presence of traces of the company in a particular hosting/cloud, but this is a topic for a separate article.

There are other sources of subdomains, for example: mobile applications, favicon hash search, HTTP headers (CSP), crawling of discovered web resources. In this article, we looked at the most basic ones and those that give the greatest results.

In total, the methodology looks like this:

1. Collecting the customer's IP address:

2. Horizontal domain search:

   • Reverse Whois (whoisxmlapi, revwhoix);

   • Reverse DNS (dnsx);

   • Scraping TLS Certificates (tlsx).

3. Vertical domain search:

   • Reverse DNS (dnsx);

   • Scraping TLS Certificates (tlsx);

   • crt.sh and other services (subfinder);

   • bruteforce subdomains (puredns);

   • permutations from found domains (alterx);

   • resolve found domains (dnsx).

The stakeholder should then look at the list to add or remove resources if necessary.

Perhaps someone approaches the task of reconning assets on the perimeter in a different way. Share your cases in the comments, it will be interesting to read 🙂

Alexander Starikov

Information security consultant, Jet Infosystems