sam_aydlette

Introduction: The Foundation of Cybersecurity

Effective inventory management is the foundation upon which robust cybersecurity and risk management programs are built. The old truism remains valid - you can't secure what you don't know exists. Organizations frequently struggle with maintaining accurate inventories of their digital assets, creating significant security blind spots that leave them vulnerable to attacks.

This tutorial addresses a critical need in the cybersecurity ecosystem: the ability to objectively validate that vendors and organizations are performing adequate inventory management. Using free, unintrusive tools available to anyone, you can ethically assess the thoroughness of public asset inventory practices (with permission, of course). This capability is particularly valuable for security professionals evaluating vendor risk.

By following this tutorial, you'll learn how to validate that the foundation of an organization's security program is solid before building additional controls on top.

Before You Dive In: Legal and Ethical Considerations

A great way to practice this material is to target a public asset that you personally own and control. For example, to create this tutorial I targeted my own website which is hosted on AWS. But before trying these methods yourself, there are important boundaries to respect, even for assets you own:

• Hosting Provider Terms: Your web hosting agreement may have specific clauses about security scanning. Review these before proceeding.
• Shared Hosting Caution: If your website shares server resources with others, aggressive scanning could impact innocent bystanders.
• AWS-Specific Policies: For CloudFront/S3-hosted websites, AWS has particular rules regarding security testing:
  • Some basic scanning is permitted without approval
  • More intensive tests may require explicit permission
  • Aggressive scanning might trigger AWS's protection mechanisms, potentially limiting your own access

The golden rule is to document everything you do, scan during low-traffic periods, and respect rate limits to avoid triggering defensive measures.

Prerequisites: Your Security Testing Toolkit

Here's what you'll need:

• Kali Linux
• Python 3.x
• Shodan API key
• At least one public domain to target (legally and ethically)

How to install Kali Linux is outside the scope of this tutorial, but documentation for that can be found at www.kali.org.

Nmap is a powerful open-source network scanner that comes pre-installed on Kali Linux that is used for discovering hosts and services on computer networks, creating a "map" of the network. Masscan, also pre-installed on Kali Linux, is an Internet-scale port scanner, designed for high-speed scanning of large networks with a specialized focus on raw packet performance. Amass, developed by OWASP (Open Web Application Security Project), performs network mapping of attack surfaces and external asset discovery using open source information gathering and active reconnaissance techniques. Shodan is a web-accessible search engine for Internet-connected devices that allows users to find specific types of devices, vulnerable systems, and gather intelligence about network infrastructure. Shodan is not included in Kali Linux. It is a web application with a CLI. The free tier includes basic search functionality with limited results, while the paid tier provides more comprehensive scanning capabilities and higher query limits.

Accessing Shodan

Sign Up: Visit https://shodan.io and create a free account API Key: After registration, find your API key in your account settings

CLI Installation:

                        
                        # Install pip if needed
                        sudo apt update 
                        sudo apt install python3-pip

                        # Install the virtual environment package if needed
                        sudo apt install python3-venv

                        # Create and activate the virtual environment
                        python3 -m venv ~/shodan-env
                        source ~/shodan-env/bin/activate

                        # Now install shodan within the virtual environment
                        pip install shodan

                        # Initialize your API key
                        shodan init YOUR_API_KEY_HERE
                        

It should return Successfully Initialized

The Tutorial: Mapping Your Digital Territory

Step 1: Gather Basic Website Information

First, let's identify your website's basic footprint:

                        
                        # Get your website's IP address
                        host yourdomain.com

                        # Get DNS information
                        dig yourdomain.com

                        # Store the IP for later use (work smarter, not harder!)
                        export TARGET_IP=$(dig +short yourdomain.com)
                        echo $TARGET_IP
                        

Think of this as creating a basic map before the expedition. You're identifying the coordinates and street address of your digital property.

Step 2: Initial Reconnaissance with Nmap

Time to see what services are visible from the outside:

                        
                        # Format the IPs for Nmap by saving the IPs to a file and scan using the file 
                        echo $TARGET_IP > target_ips.txt 

                        # Basic port scan
                        sudo nmap -sV -iL target_ips.txt | tee initial_scan.txt
                        

The -sV flag identifies service versions. Other arguments, like -A can enable OS detection, version detection, script scanning, and traceroute.

These scans reveal what services are running and potentially vulnerable. To take the property analogy one step further, this is checking which doors and windows are visible from the street.

Step 3: Shodan Intelligence Gathering

Shodan shows what information is already publicly indexed about your website. Many of Shodan's rich functionality are disabled at the free tier, however some are available:

                        
                        # Search for your domain
                        shodan domain yourdomain.com

                        # Search for your IP directly
                        shodan host $TARGET_IP

                        # Look for historical data (what changed over time?)
                        shodan host $TARGET_IP --history
                        

This is akin to checking what information a curious stranger could easily find about your property without even visiting it (blueprint, home inspection report, etc.)

Step 4: Advanced Port Scanning with Masscan

Let's use Masscan's speed to discover ports that might be missed by other tools:

                        
                        # Full port scan of your website (at a reasonable rate)
                        sudo masscan -p1-65535 $TARGET_IP --rate=1000 -oJ masscan_results.json

                        # Compare with Nmap results (spot any differences!)
                        cat masscan_results.json | jq '.[]| .ports[].port' | sort -n > masscan_ports.txt
                        grep "open" detailed_scan.txt | grep -v "filtered" | awk '{print $1}' | sort -n > nmap_ports.txt
                        diff masscan_ports.txt nmap_ports.txt
                        

Why use both Nmap and Masscan? They're like different detectives with unique investigation styles:

• Different technologies: Nmap uses TCP/IP fingerprinting; Masscan uses asynchronous transmission
• Speed vs. depth: Masscan is fast but less detailed; Nmap provides richer context
• Cross-verification: Discrepancies between tools often reveal interesting security insights
• Edge case detection: Some services respond differently to different scanning techniques

This combination gives you both breadth and depth in your reconnaissance.

Step 5: Domain Intelligence with Amass

Now let's discover subdomains and related information:

                        
                        # Passive reconnaissance
                        amass enum -passive -d yourdomain.com -o amass_passive.txt

                        # More active enumeration
                        amass enum -active -d yourdomain.com -o amass_active.txt
                        

This reveals your domain's broader attack surface—like discovering not just your main property but all the connected outbuildings that someone might target.

Step 6: Creating Your Assessment Report

Time to compile everything into a comprehensive report:

                        
                        # Create report directory
                        mkdir website_assessment
                        cd website_assessment

                        # Move all results to report directory
                        mv ../initial_scan.txt ../detailed_scan.txt ../masscan_results.json ../amass_*.txt ./

                        # Generate a basic report
                        echo "Website Security Assessment Report" > assessment_report.txt
                        echo "Target: yourdomain.com ($TARGET_IP)" >> assessment_report.txt
                        echo "Date: $(date)" >> assessment_report.txt
                        echo "----------------------------------------" >> assessment_report.txt

                        # Add open ports and services
                        echo "OPEN PORTS AND SERVICES:" >> assessment_report.txt
                        grep "open" detailed_scan.txt | grep -v "filtered" >> assessment_report.txt
                        echo "----------------------------------------" >> assessment_report.txt

                        # Add discovered subdomains
                        echo "DISCOVERED SUBDOMAINS:" >> assessment_report.txt
                        cat amass_*.txt | sort | uniq >> assessment_report.txt
                        echo "----------------------------------------" >> assessment_report.txt

                        # Add Shodan intelligence
                        echo "SHODAN INTELLIGENCE:" >> assessment_report.txt
                        shodan host $TARGET_IP | grep -v "Last update" >> assessment_report.txt
                        

Conclusion: Putting It All Together

Congratulations! By completing these steps, you've either validated that your assets are secure, or you've found weaknesses that need to be addressed. Either way, it's better than not knowing. This assessment gives you valuable insights into:

• Visible services: What ports and services are exposed to the internet
• Domain footprint: The broader attack surface including subdomains
• Public intelligence: What information is already being collected by adversaries about the assets
• Potential vulnerabilities: Areas that might require additional hardening

This knowledge allows you to make informed security decisions before potential attackers discover weaknesses.