Open-Source Tools for Pen Testing: Putting Your Technique to Work

About a year ago, I had the opportunity to lead a roundtable of pen testers at RSA San Francisco to discuss the future of pen testing. I spoke with pen testers from Motorola, MasterCard and various branches of the United States Department of Defense (DoD). We discussed a few insights that I gained from working with pen testers in Japan – some of whom worked for the Japan Self-Defense Forces.

During the discussion, I asked them about the tools they use as they go about pen testing. Answers varied, but the consensus was that when they use tools during a professional pen test, the vast majority of these tools are Linux-based, open-source tools. Let me share with you a few of the tools they found to be particularly powerful.

Open-Source Tools and the Hacker Lifecycle

There are so many tools available, so the best way to frame the discussion is by referencing the reliable Lockheed-Martin Cyber Kill Chain, shown below. The Lockheed-Martin Cyber Kill Chain outlines the seven steps of a cyberattack:

1. Reconnaissance
2. Weaponization
3. Delivery
4. Exploitation
5. Installation
6. Command and control
7. Actions on objectives

Figure 1: The Lockheed-Martin Cyber Kill Chain

Let’s take a look at how open-source tools can help with different steps in the Lockheed-Martin Cyber Kill Chain.

Reconnaissance with Open-Source Intelligence (OSINT) Tools

Open-source tools often used to discover systems include Nmap, Shodan, Metagoofil and Maltego.

Let’s take a look at each of these:

• Nmap is the primary active network scanning tool. Alternatives include Zmap.
• Shodan is effectively “Google for hackers.” This website contains a massive, searchable database of hosts on the internet. It is an example of a passive OSINT tool, because you aren’t directly interacting with a networked resource, as you usually do with Nmap. You can also download command-line tools that use Shodan’s website, and use it in conjunction with other tools, including Shodan and Nmap.
• Metagoofil collects documents and data from websites.
• Maltego is a passive reconnaissance tool.

Nmap is usually the most-often discussed application. You use it to discover network nodes on the network. You can even use it to “fingerprint” that host to see exactly which particular operating system that host is using.

Figure 2: Operating system fingerprinting with Nmap

Figure 3: Using Maltego to gain open-source intelligence

Initial Compromise, Establish Foothold and Escalate Privileges: Command and Control

Arguably, the most effective way to pen test is to engage in social engineering. Primary tools often used in social engineering include the Social Engineering Toolkit, though I find that tool to be fairly outdated at this point. Applications such as Wifiphisher and ZMail are popular for social engineering. Additional tools used to conduct actual attacks include those listed below.

Figure 4: Using THC Hydra to defeat a simple FTP password

Figure 5: Capturing keystrokes from a user’s web browser using BeEF browser

Where Do I Get These Open-Source Tools?

You can download these tools directly from their websites. However, I find it easier to use a good Linux distribution (e.g., Ubuntu, Red Hat) and then install them from there. In fact, there are entire Linux distributions devoted to pen testing.

Pen testing poster child distros include Kali Linux and Parrot Linux. I conducted a webinar profiling Kali: Kali Linux Overview: Tips, Tricks and Traps for Cybersecurity Professionals. In fact, you should consider learning more about Linux itself, because it is a major skill that pen testers have. Quite a few others exist, including BackBox and Pentoo. I launch several of these from my VirtualBox application that I have on my Windows 10 system. There’s plenty of debate about which pen testing distro is the best.

Here are a few insights I’ve gained from talking with experienced pen testers:

1. Your pen testing and Linux skills are more important than any one distro or tool.
2. Hands-on experience teaches more and does more than any application.
3. Quite a few pen testers start with a clean version of Linux, and then install fresh instances of their favorite applications (e.g., Nmap, Metasploit and BurpSuite).
4. Some of the favorite open-source tools pen testers use aren’t even developed by a particular group. The pen testers I know often use Python BASH and other languages to create their own tools. This enables them to “live off of the land,” and allows them to be stealthier and more efficient.

Tools Aren’t Everything

When I discuss pen testing, red teaming or even blue teaming with working IT professionals, they invariably discuss how tools don’t make for good pen testers. Good pen testers use good tools. So, I want to emphasize that even though I’ve spent a lot of time here profiling particular applications, it’s much more important for you to understand how technologies work. In other words, a good pen tester does more than just learn how a specific application such as Nmap or Metasploit works.

Instead, they learn about things like:

1. How protocols work. For example, do you understand the 3-way TCP handshake, shown below?
   

   Figure 6: The TCP handshake

   And how much do you really understand how SSL/TLS works? Notice during the TLS session that steps 6 and 8 are circled. Those are the points where many times hackers try to exploit the TLS standard to negotiate a lower, more hackable, TLS cipher.

   

   Figure 7: SSL / TLS handshake

2. How web-based applications talk to each other. This means understanding how the Hypertext Transfer Protocol (HTTP) works, both in encrypted and un-encrypted forms.
   

   Figure 8: The Hypertext Transfer Protocol (HTTP) handshake

3. How Application Programming Interface (API) implementations work. Do you also understand their weaknesses?

Figure 9: API weaknesses

In short, you need to learn your protocols. Without that knowledge, you won’t do very well as a pen tester, no matter how cool these open-source applications may be.

What’s Next?

The next thing for you to do is practice your techniques on these half-baked applications and then move beyond mere open-source applications to real, professional tools. Yes, I’m kidding. Pen testing professionals use these tools every day – in many ways these are already next-level tools.

As you hone your skills using open source and proprietary applications, consider a few next steps:

1. Review the exam objectives for CompTIA certifications: Of course, CompTIA PenTest+ would be the most relevant certification, because it lists so many of these tools. CompTIA PenTest+ is recommended for people with 3 to 4 years of hands-on information security experience. CompTIA Linux+ also covers skills related to the open-source tools discussed in this article.
2. Get training in how today’s networks and cloud-based resources operate: After reviewing the exam objectives, you may find that you have some learning to do before getting certified. Both CompTIA PenTest+ and CompTIA Linux+ have a full suite of training products available, including eLearning, labs, exam prep and study guides. Read more about CompTIA training offerings.
3. Get legal, responsible hands-on knowledge: As you go about getting that hands-on experience, make sure that you do it in a proper, legal way. There’s an old joke in the pen testing industry: The fine line between hacking and pen testing is jail time. Don’t conduct tests on anyone else’s system unless you have a written contract, including a statement of work (SOW).

As you practice your skills over time, you’ll have quite a few professional-level, hands-on skills that you will find quite valuable in today’s job market.

CompTIA PenTest+ covers the tools needed for penetration testing. The new exam (PT0-002) comes out in October, but you can download the exam objectives now.