Category Archives: infosec

infosec python technical

Parsing OpenVAS reports in Python

I was using OpenVAS to do some network auditing and accessing report results via the (Greenbone Security assistant) web interface quite often seemed somewhat slow and clunky. The report is downloadable as an XML file though, and I’ve recently been getting familiar with parsing nmap XML files in python, so a bit of scripting later and voila! GOXParse (Glens OpenVAS XML Parser) – a command line tool to quickly search / filter through the openvas scan results.

As an added bonus, you can output a .csv file from an nmap scan using gnxparse.py and feed it to goxparse.py to provide an inline comparison of open ports.

$ ./goxparse.py --help
usage: goxparse.py filename.xml [OPTIONS]

Glens OpenVas XML Parser (goxparse)

positional arguments:
  file  File containing OpenVAS XML report

optional arguments:
  -h, --help                show this help message and exit
  -i, -ips                  Output unfiltered list of scanned ipv4 addresses
  -host [HOSTIP]            Host to generate a report for
  -cvssmin [CVSSMIN]        Minimum CVSS level to report
  -cvssmax [CVSSMAX]        Maximum CVSS level to report
  -threatlevel [THREAT]     Threat Level to match, LOG/LOW/MEDIUM/HIGH/CRITICAL
  -matchfile [MATCHFILE]        .csv file from which to match open ports, in format HOSTIP,port1,port2,port3
  -v, --version         show program's version number and exit

usage examples:
        goxparse.py ./scan.xml -ips
        goxparse.py ./scan.xml -host <HOSTIP>
        goxparse.py ./scan.xml -cvssmin 5 -cvssmax 8
        goxparse.py ./scan.xml -threatlevel HIGH

 

You can get goxparse from the bitbucket repo here.

infosec python technical

Parsing and Merging Nmap XML Report Files in Python

Here are a couple of tools I wrote in python to parse and merge/ join Nmap .xml report files.

TL;DR:

  • gnxparse.py outputs discovered host, port info from nmap .xml, optionally in the form of nmap command(s) to re-scan hosts.
  • gnxmerge.py glues multiple (<host> sections from) nmap XML reports together.
  • You can download them from the git repo here.

Problem:

Nmap is great for network auditing. Scanning from an internal, privileged, and/or fast network location (eg inside your firewall) is straightforward and fast, but doesn’t give you the whole picture – which discovered hosts and services are exposed from a different – eg external/public network.

To get this info, you could do a firewall config audit, but if you don’t have this access or just want to do a functional test of the firewall, you need to run another scan. For the same accuracy, you’ll want the full range (1-65535) portscan, and this takes time. Also, this kind of scan is noisy and may generate a lot of firewall/ips logs. Lastly, if you traverse an IPS/IDS with such a noisy scan, it may drop you as malicious, and the rest of the results are lost.

Solution:

An alternative approach is to do a full scan internally, and use the results to make a much quieter external scan targeting only known live hosts and services. gnxparse.py can generate nmap ‘rescan’ commands to run from outside the firewall, and gnxmerge.py helps tidy up the results by merging the multiple output files back into a single report.

The workflow goes something like this:

  1. Perform a thorough scan of your publically routable subnets from a location inside the firewall. (Using some fairly well tuned host discovery options in nmap, it takes me about 3 hours to scan the full port range on ~1000 hosts on a fast internal network.)
  2. Run gnxparse.py with the ‘rescan’ option on the .xml file generated from the internal nmap scan. This will output a bash script with individual nmap commands to probe only those hosts and services found to be up.
  3. Copy the script to an external host with nmap and run it. (For all discovered services on the ~1000 hosts scaned earlier, it takes me only about five minutes to do a re-scan).
  4. Run gnxmerge.py on the folder of individual .xml reports generated (one per host) if you need to produce a single Nmap XML report file for any reason, for example to load up in Zenmap[1] and review which of your services are exposed externally.

Hopefully someone else finds these tools useful. You can download  gnxparse.py and gnxmerge.py from my GNXTools repo on bitbucket.

 

[1] I am aware Zenmap can also load multiple nmap report files for viewing; though it does not merge/save. 

infosec linux sysadmin technical

Scanning and Reporting on SSL Cert Expiry Dates – an SSL Certificate Scanner using bash, php and jQuery

A while ago I cooked up a bash script to scan relevant internal subnets for ssl certs, save/parse a copy of the x509 data and list all the discovered info in a delimited text file for analysis in a spreadsheet.

This works well by itself, but for the convenience of quick lookups without involving excel or libreoffice, a web page can be useful. PHP provides a simple method for converting a delimited file into a table (fgetcsv() ), and jQuery has a great plugin called tablesorter which allows you to do some quick sorting and filtering right there in the browser. It didnt take long to mash these together into a one script web page to display the sortable certificate data at a glance.

Sample screenshot:

Screenshot sample of scancerts

The sample only shows the three dummy values I’ve included in the demo, but I’ve used this in production with 600+ scanned certs and it works well.

Scancerts has two main components:

  1. Bash script which eats a text file containing a list of networks to scan, uses openssl, sed, awk, grep, cut, etc to generate another text file containing a delimited list of discovered certs.
  2. PHP script which turns the delimited text file into a HTML table, and augments it with some jQuery so your browser can sort and filter the HTML table on the fly.

Installation Steps

  1. Create a web-accessible folder on your linux box
  2. Unpack the files in the provided archive to the web folder
  3. Make sure file/folder permissions are set correctly (and you can run PHP!)
  4. Add the subnets you want to scan into ‘subnets.txt’
  5. Make ‘scancerts’ executable
  6. Run scancerts and optionally add it to cron
  7. View a nice sortable html list of discovered certs

Download: scancerts_v0.1.tar.gz