8 Tools You Need to Gather Information Effectively
Content
The Importance of Reconnaissance
Automated tools
Conclusion
Everyone knows that reconnaissance is the most important step in bug bounty and penetration testing of web applications. Vulnerability hunters who conduct quality reconnaissance are well-deserved rewards because they are more likely to discover hidden features and resources. This gives them an advantage and significantly increases the likelihood of finding vulnerabilities.
However, not every vulnerability hunter can afford to spend the time and effort to conduct a thorough reconnaissance to map all the elements within a program.
This article is designed to help you better understand what it takes to effectively scout and find more vulnerabilities in your favorite bug bounty program.
The Importance of Reconnaissance
Reconnaissance plays a key role in bug bounties, being the first step in any security or penetration test. Skipping this step may result in some applications or functions within the program remaining untested, reducing the likelihood of finding vulnerabilities.
However, reconnaissance is a repetitive and tedious task that must be performed every time a new bug bounty program is launched. Fortunately, there are many automated and open source tools that can make this process easier. Let's look at the 8 most commonly used tools that vulnerability hunters use to automate reconnaissance.
Automated tools
In this article, we will discuss 8 essential tools that will help you perform a comprehensive scan to gather information about a target. Each of these tools has its own unique benefits, but you can also use alternatives if you prefer.
Let's start with the first step – transferring assets using Amass.
Amass
Amass is an open source tool for advanced attack surface management developed by OWASP. It is designed to collect information about assets using both passive and active transfer methods. Amass combines different data sources and enumeration methods to help identify all the hosts on your target list.
To quickly check subdomains of the root domain (using only passive sources), you can use the following command:
amass enum -d example.com -passive
If you'd like to dive deeper into Amass's capabilities, we've created a detailed guide for you, available on the blog: Link to article: Hunting for Subdomains.
This tool is also easy to install and configure; it is available on GitHub: Repository link: OWASP Amass.
Advice! Set up API keys in the configuration file to allow Amass to use more sources and provide more accurate results. Some external sources provide free API keys with limited usage.
Google/Bing/GitHub Dorking
Another way to find related assets or domains of your target is to use search engines.
Search engines allow you to find more information about a target, from indexed files, login panels and administrative portals to new subdomains. As a vulnerability hunter, you should always use Google, Bing, and Github to find more information.
Most commonly used search filters
Below is a list of the most popular search filters that you can use to find interesting indexed files that often contain sensitive data:
GitHub analysis
Another widely used service is GitHub. Companies often use GitHub to deploy, store code, and collaborate.
GitHub intelligence is extremely valuable because some companies accidentally upload sensitive data to public repositories. This may allow unauthorized users to access administrative portals with elevated privileges. In other cases, you may find links to hidden assets, files, or settings.
However, analyzing GitHub can be challenging, especially when dealing with a large target where it is not clear where to start or what to look for. Luckily, there are tools that make this process easier, such as Trufflehog And Gitleaks.
Trufflehog — a tool for finding secrets and confidential data in code
Gitleaks — a tool for finding information leaks and confidential data in Git repositories
Once you have compiled a list of target systems, hosts, and subdomains, you need to filter out the non-functioning hosts. Eyewitness is a powerful tool that helps not only with checking active hosts and their screenshots, but also with performing basic technology analysis. This allows you to quickly view a list of targets and evaluate each host individually. This approach makes it easy to identify login pages (and admin panels) as well as other interesting or suspicious HTTP responses.
Wappalyzer
Technology analysis is a great way to find out what technologies or services your target runs on. However, Eyewitness is only capable of performing basic analysis of active hosts.
To go further, you can use Wappalyzer (browser extension) or HTTPX (a tool powered by Wappalyzer) to analyze the target list. Understanding the technologies and services that the target system runs on is the next step in reconnaissance and will significantly help in the future when searching for vulnerabilities.
GAU (GetAllUrls)
Content discovery plays an important role in intelligence, as we discussed in one of our recent articles. Vulnerability hunters who engage in content discovery are typically well rewarded as they often find unused and untested features, functionality, and endpoints. This can significantly increase your chances of finding new vulnerabilities.
GAU (short for GetAllUrls) is an open source tool that helps you quickly extract URLs, links, and other indexed files from the Wayback Machine and other archiving and indexing systems!
This tool is also easy to use. To return links from a list of target addresses, you can use the following command:
cat targets.txt | gau
Ffuf
Brute force is another method for identifying hidden API endpoints, admin panels, or other files that have not been mentioned anywhere before.
Ffuf is a powerful tool that can brute force almost anything over HTTP, from basic content discovery (like links and files) to virtual hosts and parameters at very high speed!
Here is the command to perform brute force on your target:
ffuf -u https://example.com -w /path/to/wordlist.txt
An article that discusses its capabilities in detail.
Arjun
Request parameters and request bodies are where most user input is received and where most vulnerabilities occur. Parameter brute force is a reliable way to detect received input parameters and trigger unintended actions that often lead to security vulnerabilities.
Arjun is a powerful tool that supports brute force parameters across several different HTTP request bodies and content types:
arjun -u https://example.com/example.php
Advice! If you want to brute force parameters using Burp Suite, check out the extension ParamMiner! It is capable of finding hidden parameters and is often used to identify web cache poisoning vulnerabilities!
LinkFinder
JavaScript files are a treasure trove of information for vulnerability hunters because they often contain many links and endpoints that could not be found using other content detection methods.
LinkFinder is a simple tool that does a great job of finding new links, URLs, and other referenced files and endpoints in JavaScript code.
LinkFinder is also easy to use, here's how to quickly parse a JavaScript file:
python3 linkfinder.py -i https://example.com/app.jsConclusion
We all know that reconnaissance is an important phase in vulnerability hunting. This article looked at 8 of the most essential tools that we believe can help you conduct better intelligence and increase your chances of finding security vulnerabilities.
Now it's time to put your skills (and new tools) to the test!
