Web tools, or where to start pentester?

Web tools, or where to start pentester?


Continuing to talk about useful tools for pentester. In the new article we will look at tools for analyzing the security of web applications.

Our colleague BeLove already did a similar compilation about seven years ago. It is interesting to see which tools have retained and strengthened their positions, and which have faded into the background and are now rarely used.


Note that this also applies to Burp Suite, but there will be a separate publication about it and its useful plugins.

Content:



Amass


Amass is a Go tool for searching and searching DNS subdomains and mapping an external network. Amass is an OWASP project created to show how organizations on the Internet look to an outsider. Amass gets the names of subdomains in various ways, the tool uses both recursive enumeration of subdomains and search in open sources.

To find connected network segments and autonomous system numbers, Amass uses the IP addresses obtained during operation. All found information is used to build a network map.

Pros:


  • Information gathering techniques include:
      * DNS - brute-force subdomains in a dictionary, bruteforce sub-domains, “smart” enumeration using mutations based on the found sub-domains, reverse DNS queries and search for DNS servers on which it is possible to make a request for a zone transfer (AXFR);

    * Search for open sources - Ask, Baidu, Bing, CommonCrawl, DNSDB, DNSDumpster, DNSTable, Dogpile, Exalead, FindSubdomains, Google, IPv4Info, Netcraft, PTRArchive, Riddler, SiteDossier, ThreatCrowd, VirusTotal, Yahoo;

    * Search TLS certificate databases - Censys, CertDB, CertSpotter, Crtsh, Entrust;

    * Using the search engine APIs - BinaryEdge, BufferOver, CIRCL, HackerTarget, PassiveTotal, Robtex, SecurityTrails, Shodan, Twitter, Umbrella, URLScan;

    * Search the Internet for web archives: ArchiveIt, ArchiveToday, Arquivo, LoCArchive, OpenUKArchive, UKGovArchive, Wayback;
  • Integration with Maltego;
  • Provides the most complete coverage for the task of finding DNS subdomains.

Cons:


  • Be careful with amass.netdomains - he will try to access each IP address in the identified infrastructure and obtain domain names from reverse DNS queries and TLS certificates. This is a "loud" technique, it can reveal your intelligence actions in the organization under study.
  • High memory consumption can consume up to 2 GB of RAM in different settings, which will not allow running this tool in the cloud on a cheap VDS.



Altdns


Altdns is a Python tool for compiling dictionaries for iterating over DNS subdomains. Allows you to generate many options for subdomains using mutations and permutations. To do this, use words that are often found in subdomains (for example: test, dev, staging), all mutations and permutations are applied to already known subdomains, which can be submitted to the input of Altdns. The output is a list of variations of subdomains that may exist, and this list can later be used for DNS brute force.

Pros:


  • Works well with large data sets.

aquatone


aquatone - was previously better known as another tool for searching subdomains, but the author himself abandoned this in favor of the above-mentioned Amass. Now aquatone is rewritten to Go and more geared for pre-exploration of websites. To do this, aquatone passes through the specified domains and searches for websites on different ports, after which it collects all the information about the site and makes a screenshot. Convenient for quick preliminary reconnaissance of websites, after which you can select priority targets for attacks.

Pros:


  • At the output, it creates a group of files and folders that are convenient to use when continuing to work with other tools:
      * HTML report with collected screenshots and response headings grouped by similarity;

    * File with all the URLs on which websites were found;

      * File with statistics and data page;

    * Folder with files containing the response headers from the found targets;

    * Folder with files containing the response body from the found targets;

    * Screenshots of found websites;
  • Supports working with XML reports from Nmap and Masscan;
  • Uses headless Chrome/Chromium to render screenshots.

Cons:


  • It may attract the attention of intrusion detection systems, and therefore requires adjustment.

The screenshot was made for one of the old versions of aquatone (v0.5.0), in which the search for DNS subdomains was implemented. Older versions can be found on the release page .

Aquatone v0.5.0 Screenshot

MassDNS


MassDNS is another tool for finding DNS subdomains. Its main difference is that it makes DNS queries directly to many different DNS resolvers and does so with considerable speed.

Pros:


  • Fast - able to resolve more than 350 thousand names per second.

Cons:


  • MassDNS can cause a significant load on the DNS resolvers used, which can lead to a ban on these servers or complaints to your provider. In addition, it will cause a large load on the company's DNS servers, if they have them and if they are responsible for the domains you are trying to resolve.
  • The list of resolvers is currently outdated, but if you select broken DNS resolvers and add new known ones, everything will be fine.



nsec3map


nsec3map is a Python tool for getting a complete list of DNSSEC protected domains.

Pros:


  • Quickly detect hosts in DNS zones with a minimal number of queries if DNSSEC support is enabled in the zone;
  • As part of a plugin for John the Ripper, which can be used to crack the resulting NSEC3 hashes.

Cons:


  • Many DNS errors are not handled correctly;
  • There is no automatic parallelization of processing NSEC records - you have to split the namespace manually;
  • High memory consumption.

Acunetix


Acunetix is a web vulnerability scanner that automates the process of checking web application security. Tests the application for SQL injection, XSS, XXE, SSRF, and many other web vulnerabilities. However, just like any other scanner of multiple web vulnerabilities does not replace the pentester, since complex chains of vulnerabilities or vulnerabilities in logic cannot be found. But it covers a lot of different vulnerabilities, including different CVEs, which the pentester could have forgotten, therefore, it is very convenient to get rid of routine checks.

Pros:


  • Low level of false positives;
  • Results can be exported as reports;
  • Performs a large number of checks for various vulnerabilities;
  • Parallel scanning of multiple hosts.

Cons:


  • There is no deduplication algorithm (Acunetix pages that are of the same functionality will be considered different, because different URLs lead to them), but the developers are working on it;
  • Requires installation on a separate web server, which makes it difficult to test client systems with a VPN connection and use the scanner in an isolated segment of the local client network;
  • It may “rustle” the service under study, for example, send too many attacking vectors to the communication form on the site, thereby greatly complicating business processes;
  • Is a proprietary and, accordingly, non-free solution.



Dirsearch


Dirsearch is a Python tool for brute-force directories and files on websites.

Pros:


  • Can distinguish real “200 OK” pages from “200 OK” pages, but with the text “page not found”;
  • Comes with a convenient dictionary that has a good balance between size and search efficiency. Contains standard paths typical of many CMS and technology stacks;
  • Its dictionary format, which allows you to achieve good efficiency and flexibility in searching files and directories;
  • Easy output - plain text, JSON;
  • Able to do throttling - a pause between requests, which is vital for any weak service.

Cons:


  • Extensions must be passed as a string, which is inconvenient if you need to transfer many extensions at once;
  • In order to use your dictionary, it will need to be slightly modified to the format of the Dirsearch dictionaries for maximum efficiency.



wfuzz


wfuzz - Python-fazzer web application. Probably one of the most famous web phasers. The principle is simple: wfuzz allows phasing any place in an HTTP request, which allows phasing of GET/POST parameters, HTTP headers, including Cookies and other authentication headers. At the same time, it is convenient for simple brute force directories and files, for which you need a good dictionary. It also has a flexible filter system, with which you can filter the responses from the website by different parameters, which allows you to achieve effective results.

Pros:


  • Multifunctional - modular, assembly takes a few minutes;
  • Convenient filtering and fuzzing mechanism
  • You can phase out any HTTP method, as well as any place in the HTTP request.

Cons:


  • Under development.



ffuf


ffuf - the web fuzzer on Go, created in a similar fashion to wfuzz, allows you to browse files, directories, URL paths, names and GET/POST parameter values, HTTP headers, including Host header for brute force virtual hosts. Wfuzz differs from its colleague by higher speed and some new features, for example, Dirsearch format dictionaries are supported.

Pros:


  • Filters are similar to wfuzz filters, allow you to flexibly configure brute force;
  • Allows fuzzing HTTP header values, POST request data and various parts of the URL, including the names and values ​​of GET parameters;
  • You can specify any HTTP method.

Cons:


  • Under development.



gobuster


gobuster is an intelligence tool for Go that has two modes of operation. The first one is used for brute-force files and directories on the website, the second one is used to iterate over the DNS subdomains. The tool initially does not support recursive enumeration of files and directories, which, of course, saves time, but on the other hand, the brute force of each new endpoint on the website needs to be launched separately.

Pros:


  • High speed for both brute-force DNS subdomains and brute-force files and directories.

Cons:


  • The current version does not support the installation of HTTP headers;
  • By default, it considers valid only some of the HTTP status codes (200,204,301,302,307).



Arjun


Arjun is a tool for brute-force hidden HTTP parameters in GET/POST parameters, as well as in JSON. The built-in dictionary has 25,980 words that Ajrun checks in almost 30 seconds. The trick is that Ajrun does not check each parameter separately, but checks immediately ~ 1000 parameters at a time and looks to see if the answer has changed. If the answer has changed, then divides this 1000 parameters into two parts and checks which of these parts affects the answer. Thus, using a simple binary search, a parameter or several hidden parameters are found that influenced the answer and, therefore, can exist.

Pros:


  • High speed due to binary search;
  • Support for GET/POST parameters, as well as parameters in the form of JSON;

Similarly, the Burp Suite plugin works with param-miner , which is also very good at searching for hidden HTTP parameters. We will tell you more about it in the upcoming article about Burp and its plugins.


LinkFinder


LinkFinder is a Python script for searching links in JavaScript files. Useful for finding hidden or forgotten endpoints/URLs in a web application.

Pros:


  • Quick;
  • There is a special plugin for Chrome based on LinkFinder.
.

Cons:


  • Inconvenient summary;
  • Does not analyze JavaScript over dynamics;
  • A fairly simple link search logic - if JavaScript is obfuscated in some way, or the links are initially missing and dynamically generated, then they will not be able to find anything.



JSParser


JSParser is a Python script that uses Tornado and JSBeautifier to analyze relative URLs from JavaScript files. Very useful for detecting AJAX requests and compiling a list of API methods with which the application interacts. Effectively paired with LinkFinder.

Pros:


  • Quick parsing javascript files.



sqlmap


sqlmap is probably one of the most famous tools for analyzing web applications. Sqlmap automates the search and operation of SQL injections, works with several SQL dialects, has in its arsenal a huge number of different techniques, ranging from quotes head-on and ending with complex vectors for time-based SQL injections. In addition, it has many techniques for further exploitation for various DBMS, therefore it is useful not only as a scanner for SQL injections, but also as a powerful tool for exploiting already found SQL injections.

Pros:


  • A large number of different techniques and vectors;
  • Low number of false positives;
  • Many options for fine tuning, various techniques, target database, tamper scripts for bypassing WAF;
  • Ability to create an output dump;
  • Many different operating options, for example, for some databases - automatic file upload/download, command execution ability (RCE) and others;
  • Support for direct connection to the database using data obtained during an attack;
  • You can submit a text file with the Burp results to the input - no need to manually compile all the command line attributes.

Cons:


  • It’s hard to customize, for example, to write some of your checks due to poor documentation for this;
  • Without the appropriate settings, conducts an incomplete set of checks, which can be misleading.



NoSQLMap


NoSQLMap is a Python tool for automating the search and operation of NoSQL injection. It is convenient to use not only in NoSQL databases, but also directly when auditing web applications using NoSQL.

Pros:


  • Like sqlmap, it not only allows you to find a potential vulnerability, but also checks whether it can be used by MongoDB and CouchDB.

Cons:


  • Does not support NoSQL for Redis, Cassandra, is being developed in this direction.


oxml_xxe


oxml_xxe is a tool for embedding XXE XML exploits into various file types that use an XML format in some form.

Pros:


  • Supports many common formats, such as DOCX, ODT, SVG, XML.

Cons:


  • PDF, JPEG, GIF support is not fully implemented;
  • Creates only one file. To solve this problem, you can use the docem , which can create a large number of files with paylodes in different places.

The aforementioned utilities do an excellent job with XXE testing when loading documents containing XML. But also do not forget that XML format handlers can occur in many other cases, for example, XML can be used as a data format instead of JSON.

Therefore, we recommend that you pay attention to the following repository, which contains a large variety of payloads: PayloadsAllTheThings .

tplmap


tplmap is a Python tool to automatically detect and exploit Server-Side Template Injection vulnerabilities, has settings similar to sqlmap and flags. It uses several different techniques and vectors, including blind-injections, and also has techniques for executing code and loading/unloading arbitrary files. In addition, it has in its arsenal techniques for a dozen different engines for templates and some techniques for searching eval () - like code injections in Python, Ruby, PHP, JavaScript. In case of successful operation opens an interactive console.

Pros:


  • A large number of different techniques and vectors;
  • Supports many engines for rendering templates;
  • A lot of maintenance techniques.

CeWL


CeWL is a Ruby dictionary generator designed to extract unique words from a specified web site, following links on a site to a specified depth. Compiled dictionary of unique words can be used later for brute-force passwords on services or brute-force files and directories on the same web site, or to attack hashes obtained using hashcat or John the Ripper. Useful in compiling a “target” list of potential passwords.

Pros:


  • Easy to use.

Cons:


  • You need to be careful with the depth of the search so as not to capture an extra domain.

Weakpass


Weakpass is a service containing many dictionaries with unique passwords. It is extremely useful for various tasks related to password cracking, ranging from simple online brute-force accounts to targeted services, ending off-line brute-force hashes obtained using hashcat or John The Ripper . There are about 8 billion passwords in length from 4 to 25 characters.

Pros:


  • Contains both specific dictionaries and dictionaries with the most frequently encountered passwords - you can choose a specific dictionary for your own needs;
  • Dictionaries updated and updated with new passwords;
  • Dictionaries are sorted by efficiency. You can choose the option for quick online brute, as well as for a detailed selection of passwords from the extensive dictionary with the latest leaks;
  • There is a calculator showing the password brutal time on your hardware.



In a separate group, we would like to bring the tools for CMS checks: WPScan, JoomScan and AEM hacker.

AEM_hacker


AEM hacker is a tool for detecting vulnerabilities in Adobe Experience Manager (AEM) applications.

Pros:


  • Can detect AEM applications from the list of URLs entered into it;
  • Contains scripts for obtaining RCE by loading a JSP shell or using SSRF.

JoomScan


JoomScan is a Perl tool to automate the detection of vulnerabilities when deploying a Joomla CMS.

Pros:


  • Able to find configuration flaws and admin settings problems;
  • Lists Joomla versions and related vulnerabilities, similarly for individual components;
  • Contains more than 1000 exploits for Joomla components;
  • Output summary reports in text and HTML formats.



WPScan


WPScan is a tool for scanning websites on WordPress, has vulnerabilities in its arsenal both for the WordPress engine and for some plug-ins.

Pros:


  • Able to list not only unsafe WordPress plugins and themes, but also get a list of users and TimThumb files;
  • Can brute force attack WordPress sites.

Cons:


  • Without the appropriate settings, conducts an incomplete set of checks, which can be misleading.



In general, different people prefer different tools for work: they are all good in their own way, and what one person liked, may not suit another. If you think that we have undeservedly bypassed some kind of good utility - write about it in the comments!

Source text: Web tools, or where to start pentester?