EyeWitness

EyeWitness and Active Scanning

Christopher Truncer IT Security , Leave a comment

You can clone/download EyeWitness to test active scans here: https://github.com/ChrisTruncer/EyeWitness

Since its release, EyeWitness has purely been a passive scanner. Its focus has been to take screenshots of web applications, and attempt to identify any default credentials that might be associated with that web application. In my experience, it works well to save me the time from having to look up account information. While there are other features, this has been EyeWitness’s primary MO.

For a while, it’s been requested that EyeWitness integrate active scanning features to actually test and see what user accounts are genuinely valid, but it’s just never been in EyeWitness’s functionality.. until now.

Evan Pena (@Evan_Pena2003) just submitted a pull request to EyeWitness which has just been merged into the master branch. While it is currently limited in scope, active user account scanning functionality has just been merged into EyeWitness!

scan

raikia

When you use the –active-scan command line flag, EyeWitness will attempt to find common locations of login forms, and if one is found, it will attempt to validate commonly used credentials and see if they work for the specific web application being tested. Additionally, we are now able to create more targeted “signatures” that can be used to specify paths for each unique web application and the credentials that should be tested against it.

Datafile Creds

For example, the –active-scan feature now can test tomcat manager web applications with various username and password combinations to see if any are valid. If a valid user/pass combo is found, EyeWitness will not only alert within the console, but the report itself has a new category called “Successful Logins” to highlight the applications where EyeWitness could successfully log into it.

Raikiareport

As of now, the “database” of web applications isn’t large. However, this is the same situation we were in when EyeWitness began passively scanning for default credentials. As the database is built out, I’ll have the ability to add more web applications in and identify more credentials for each web app.

I’m really happy to have had this feature added in from the community, and I look forward to adding to the web application database and identify more credentials with each scan.

EyeWitness XML Parsing Updated

Christopher Truncer Development , , Leave a comment

Since the first release of EyeWitness, the tool has supported input formats of both text files, and XML output from Nessus and Nmap. There’s two main ways to parse XML, the Document Object Model (DOM) and Simple API for XML Parsing (SAX). The main difference is when using DOM to parse XML, the whole XML document is loaded into memory and analyzed. SAX is essentially a stream based parser and loads small pieces of the XML in memory at a time. DOM is faster at parsing XML since the entire file is loaded into memory, but this comes as a memory cost. EyeWitness has always used DOM based parsing since it is faster and I had yet to encounter a failure in parsing due to this design decision.

However, I can no longer make that claim. Viss has become my edge case (and I’m now certain that others who perform the same work/scans are as well. Also, I make this claim that he is my edge case jokingly :)). A couple months ago Viss had contacted me to let me know EyeWitness was crashing after trying to parse an XML document. This XML file was the output from (I believe) scanning the whole internet. It was a funny conversation, but showed that I needed to change how EyeWitness parses XML in order to handle these larger scans (I’m talking nearly a gig, I believe, for a single XML file).

EyeWitness has now been modified to use SAX-based XML parsing. The tradeoff is that there might be slightly longer XML parsing times, but this will ensure that EyeWitness is able to parse anything you can throw at it (until someone comes with my next edge case :)). The slightly longer parsing times for XML will likely not even be noticeable for nearly all users.

With this change, some small modifications to EyeWitness command line options have been added. Previously when you passed a file into EyeWitness, it would auto-detect if it is an XML file or text file, and parse accordingly. I’ve now separated the commands out, and you will either specify a file (with the -f flag) or an xml file (with the -x flag). This is now present within the help menu:

Xmlparse

This change has now been pushed into the master EyeWitness branch. If you have any questions, don’t hesitate to reach out and ask (@ChrisTruncer).

Thanks!

EyeWitness 2.0 Release and User Guide!

Christopher Truncer Development, IT Security , , 20 Comments

Github Repo: https://github.com/ChrisTruncer/EyeWitness

For the past few months, the EyeWitness codebase has gone through multiple refactors, nearly all by Rohan Vazarkar (@CptJesus), but also myself.  One thing that I try to preach, and any developer should agree with, is that there are three primary features a script/tool needs to have.  It needs to be usable, perform a service of value (otherwise why even write it), but it also needs to be stable.  With the recent updates to the back-end libraries EyeWitness uses, we failed on that last feature.  At times, EyeWitness would seem to just crash on us, but then run perfectly seconds later.  After many assessments of encountering these shared frustrations, we decided to start from scratch.

As of now, I’m happy to release EyeWitness 2.0, a complete re-write from the ground up.  We took stability as the driving goal for our design decisions, and which libraries to use, and made the 2.0 version something that we are really happy to finally push out.  This post will serve as a user’s guide, to show how to use the primary features of the tool.

First, our new menu:

EyeWitness Menu 2 0

Note: There’s more options available than what is displayed above.

First, let’s talk about the original/primary goal of EyeWitness, taking screenshots of websites!  We removed Ghost from EyeWitness, due to stability reasons, and added in two different methods of taking screenshots.  When using –web, EyeWitness will call selenium, which uses the actual browser (IceWeasel or Firefox) installed on your system, to take screenshots.  You won’t see s browser pop-up, but it’s running in the background, and taking screenshots of the URLs that you provided.

Due to the fact that we encountered a large number of issues when relying on a single library to take screenshots, we decided that we didn’t want to have another single point of failure, so we added in another option when it comes for web screenshots.  You can use “–headless” to use phantomjs to take screenshots of websites.  The nice thing about phantomjs is it’s very fast, and can run in a headless environment.

Finally, probably the most requested feature we’ve had for some time is for EyeWitness to thread its requests.  I’m really happy to say that threading has been added into EyeWitness!  By default, EyeWitness will use 10 threads, but this number can be adjusted by the user with the “–threads #NUMTHREADS” option.  The ability to run threaded scans has significantly cut back on the amount of time required to scan all URLs provided to EyeWitness.

Next, we’ve added in the ability to take screenshots and generate a report for RDP and VNC all because of the awesome work by Sylvain Peyrefitte (@citronneur) and the RDPY project.  Now, all you need to do is use the –rdp or –vnc switch, and EyeWitness will attempt to screenshot either service.  RDP screenshots will take a picture of the login screen, which will potentially also capture any other user accounts currently logged into the system.  VNC screenshots will attempt to connect to a VNC service which doesn’t require authentication, and capture a screenshot of the VNC environment.

RDP Scan

RDP Report

Due to the stability issues EyeWitness had in the past, we ran into instances where EyeWitness may get 90% of the way through a scan, crash on a single host, and would have to start all over.  This was another major point of frustration for Rohan and myself.  This was initially solved by Robin Wood (@digininja) when he provided us with a script that would generate a report based off of the currently available information that EyeWitness had gathered.  We also wanted to look into this, and have now added in proper resume support!  As of this 2.0 update, EyeWitness will use a sqlite database to track the systems that are queued up for screenshots, and then mark them complete in the database once done.  In the event of a crash, or of the user CTRL + C to quit out of the scan, the latest information will be saved to the sqlite DB, which is in the report output folder.  Once the user is ready to resume their scan, you just need to call EyeWitness with the –resume option, and pass the location of the database file.

EW Cancel

2nd updated resume

The above pictures are what you should see when canceling a scan, and then resuming it at a later time.

Finally, the last new feature I want to go over is the new report generated by EyeWitness.  EyeWitness will still attempt to identify default credentials for each web application that it scans.  However, a new report layout was created for the 2.0 release.  Not only does the EyeWitness report perform default credential checks, it also now will attempt to sort out and group different types of web applications together.  We’ve created a group called “High Value Targets” which may have admin login links available, or any other target that might be of immediate interest to the user, such as Tomcat servers.  The report will have a table of contents at the top, which can be used to jump right to any section of the report.

Report TOC

Report Section

Report Errors

Oh, and just one more thing.

One request that we had from Steve Borosh (@424f424f) was to make our report searchable.  Initially, we turned this down because we didn’t want to require a complete back-end database, we wanted EyeWitness Reports to be portable.  However, Rohan created a search script which will parse the sqlite database that’s created with each scan and generate a report based on what you’re searching for.  Want to test it out? It’s easy, just call the search script, pass in the path to the sqlite database, and provide the search strings.  A new report will be created containing the results!

Report Search

Search Results

We hope that this helps to explain the major updates to EyeWitness!  If there’s any questions you still have, feel free to contact @CptJesus or @ChrisTruncer on twitter, in #Veil on Freenode, or leave a comment below.  Thanks, and hope this helps!

EyeWitness Now in Ruby!

Christopher Truncer Development, Featured Category , Leave a comment

The best way that I learn languages, is to give myself a task, and force myself to write a script/program that carries out that task.  Well, in this case, I’ve been wanting to change some aspects of EyeWitness, and decided that porting EyeWitness to Ruby would be a great way to make those changes, and learn a new language.  After @harmjoy suggested that I look into learning Ruby, I decided to dive in and get it done.  Now, about two months of working on it, I’m happy to say the EyeWitness Ruby port is ready for its initial release.

To view and download EyeWitness, head to my Github account, or click here!

I will continue to keep the Python version of EyeWitness available, as there are a few differences between the two versions of EyeWitness.  These differences are:

  • Screenshot Library – By adding an additional library for capturing screenshots, the user can switch between the two in the event that one library encounters an issue when capturing select websites.
    • The Python version of EyeWitness uses Ghost to take screenshots of websites.  One of the benefits of this is that it can run headlessly.  However, I have seen an issue when given 3000+ websites, Ghost can freeze.  Also, Robin Wood (@digininja) pointed out that EyeWitness can completely crash while screenshotting websites.  This looks to be due to a potential file descriptor leak within Ghost.  Even though I rarely hit these issues, I still wanted a solution/alternative.  An alternative has been implemented in the Ruby version.
    • The Ruby version uses Selenium for capturing screenshots.  Specifically, Selenium-Webdriver will start up an instance of your web browser (default firefox),  and use the web browser to navigate and screenshot web pages.  For this first release, the ruby version is ONLY supporting Firefox (or a fork like Ice Weasel).  Future releases will support other browsers, such as Chrome.
  • User Agent Switching
    • The Ruby version of EyeWitness does not currently have the ability to dynamically switch user agents for every URL and perform the same comparison checks that the Python version can carry out.  This is because EyeWitness would have to instantiate a new selenium-webdriver object for every user agent, and that takes place in the form of a new web browser opening up.  I believe it would be more of a hassle/distraction to have a large number of web browsers open, so I have not implemented it in the Ruby version.  However, if this is needed, you can still use the user agent switching functionality within the Python version.
  • File Input
    • The Ruby version of Eyewitness requires you to specify the file type you are using as input.  If using Nmap XML output for EyeWitness, you will have to use the –nmap flag, for nessus, the –nessus flag.
  • Skip Sorting
    • The Ruby version has a –skip-sort flag.  This will tell EyeWitness not to group similar pages together, and just write out the report as it goes, vs. writing the report at the end after sorting all pages.

These seem to be the major differences within the two versions for the time being.  I personally believe the Ruby implementation of EyeWitness will be better to use for assessments.  If you encounter any issues, please be sure to report them to me!

Thanks!

EyeWitness Usage Guide

Christopher Truncer Development, Featured Category , , 9 Comments

NOTE: This post is now out of date – check this for the latest info – https://www.christophertruncer.com/eyewitness-2-0-release-and-user-guide/

I originally released EyeWitness in February in what I thought was pretty functional state.  When released, EyeWitness came in at about 400 lines of code.  Since February, it has had multiple new features added to it (which I will go over in this post), and its code base has expanded to about 1600 lines of code. I’d like this post to act as a usage guide of all normal usage scenarios that I can think of.

I’ll start off by describing how I normally use EyeWitness.  I typically call EyeWitness, provide it a text file (with each URL on a new line), and let it run.  If I have a .nessus file or nmap.xml output, and it has more than 350 URLs, I’ll run EyeWitness with the –createtargets flag (explained below), and output all the targets to a single text file.  I typically then split that file up into roughly 300 URLs per text file, and then either script up EyeWitness to run one after another, or run scans simultaneously.  However, different situations might cause EyeWitness to need to be used in a different manner, so hopefully this EyeWitness usage guide can help explain all of its features.

Python:

The bare bones, and likely most common, use of EyeWitness is to provide a single URL, or multiple URLs within a file for EyeWitness to screenshot and generate a report.  To provide a single URL, just use the –single flag as follows:
Single URL Scan

EyeWitness also accepts files for providing the URLs.  The file can be provided in the following formats:

  • Single text file with a URL on each line
  • Nmap XML output
  • .Nessus file
  • amap file output
To perform a scan, using any of these filetypes, just provide the -f flag as follows:
File Input Scan

By default, EyeWitness will attempt to screenshot the website, and have a max timeout of 7 seconds.  If it takes longer than 7 seconds to render the website, EyeWitness will skip to the next URL.  If you wish to change the timeout of EyeWitness, use the -t flag and set it to the max number of seconds you want it to wait to render a website. Set Timeout

Once EyeWitness has finished navigating to all URLs, and has generated a report, EyeWitness outputs the report to the same directory EyeWitness is in, and names it based off of the date and time the scan ran.  If you want to change the directory name that EyeWitness outputs its report to, use the -d flag and provide the name.  When using the -d flag, you can provide just a name, and EyeWitness will create the report using the provided name within the same directory as EyeWitness.  You can also provide the full path to a directory, and EyeWitness will create the report folder at that location (just make sure you have the proper write permissions).

Directory name change Full Path report

Sorted reporting was a feature brought up to me by Jason Frank (@jasonjfrank) as something that would be helpful when reviewing the EyeWitness report.  If we had a way to make EyeWitness analyze the different web applications, and group similar web apps together, then it would be easy to quickly sort through/review the groups you want to target.  We envisioned similar printers, mirrored web pages, etc. all grouped together within the report.  Lucky for us, Rohan Vazarkar (@cptjesus) worked on adding this feature in.  His pull request was merged in on April 22nd, and EyeWitness will now attempt to sort all results based off of their title within each report generated.

The –localscan option was added based on a request from David McGuire (@davidpmcguire).  We wanted a way to perform some basic port scanning for web servers once a machine has been compromised. Currently, one way to do it is to drop Nmap on the compromised machine, but if we did that, we’d have to install winpcap on the machine, which requires admin rights.  Instead of this, you can drop the windows Eyewitness binary, and provide the –localscan option with a CIDR range to scan.  EyeWitness will then try to find any ip listening on 80, 443, 8080, and 8443 within the provided range.  All live hosts listening on any of those ports will be added to a file that can be fed back into EyeWitness.

Localscan Portscanning

The –createtargets option came about when I wanted to have EyeWitness just provide me a list of all web servers from the XML output of Nmap or Nessus.  All web servers that EyeWitness finds within Nmap’s xml output, or the nessus file will be added to a file containing the target servers.  Just provide the filename you want the your targets file to be called.

createtargets

The user agent definition and cycling came about from working with Micah Hoffman (@webbreacher), Robin Wood (@digininja), and Chris John Riley (@ChrisJohnRiley).  After a lot of discussion on how best to carry out user agent switching and comparison, the feature was added in.  First, you can simply provide the –useragent option, and it will use any string you provide as the user agent.

Single User Agent

You can also use the –cycle option along with either browser, mobile, crawler, scanner, misc, or all.  When using this option, EyeWitness makes a baseline request.  It will then make subsequent requests with user agents of the “type” you specified.  If the subsequent requests deviate “too much” from the baseline request, the subsequent request will be added in to the report letting you know it was different from the baseline.  The deviation is currently based on the length of the source code the web server provides to EyeWitness.  By default, the deviation that’s used to measure if the requests are different is set to 50.  To change this value, use the –difference flag and provide the new value to use.

Uacycle Cycling Set Difference Value

Finally, the –jitter option was one that was discussed about at a NovaHackers meeting, and also requested by @ruddawg26.  To use this option, provide all the scan parameters you would normally provide, but add on the –jitter parameter at the end, and provide the base number of seconds that it deviates from.  Now, EyeWitness will randomize the order of the URLs provided (via text or XML), and will also have a random delay between each request.

Jitter command Jitter scan

Finally, EyeWitness has a –open flag.  If you provide the –open flag, each URL passed into EyeWitness will also be opened up in a web browser.  Your command string might look similar to the following:

Open option

Ruby:

EyeWitnessRubyHelp

To generate a report for a single website, you need to use the -s or –single flag and provide the URL.

For file based input, you will need to specify the filetype that you are providing.  If giving just a normal text file with each URL on a new line, use the -f or –filename switch.  If using providing Nmap XML output, you’ll need to use the –nmap flag, and .nessus based input requires the –nessus flag.

The –skip-sort flag is used to tell EyeWitness to not auto-group similar web pages together in the report.  This can be helpful if you want to see report pages as they are available, instead of waiting until the very end.  However, if this flag is used, similar pages will not be grouped together.

The –no-dns flag is used when you want EyeWitness to find web servers via their IP address, not their DNS name, while parsing Nmap XML output.

This pretty much covers the features of EyeWitness.  If anyone has any questions, don’t hesitate to get in touch with me.  Also, please be sure to send any signatures you might have made!

EyeWitness – A Rapid Web Application Triage Tool

Christopher Truncer Featured Category, IT Security , , , 17 Comments

More than half of the assessments that myself, and our team, go on include web applications.  Even on network level assessments, as we identify live machines within a target network, it’s fairly common for us to find a large number of web applications.  These web apps can be their own application for the customer’s purpose, or web front ends for various appliances (switches, VOIP phones, etc.).  I needed a way to be able to quickly get a quick look of all the devices serving up a web page, which would allow me to try to figure out the websites to prioritize.  Tim Tomes developed an awesome tool called PeepingTom which does what I needed.  It works great, and I recommend everyone check it out.

However, PeepingTom requires PhantomJS, and needs to be downloaded separately.  I’ve had a couple issues where it fails to grab a screenshot of the web application, and it intrigued me.  I started researching different ways to take screenshots with a python script, and stumbled upon Ghost.py.  Ghost is a self described “webkit based scriptable web browser for python”, and is able to very easily screenshot web pages.  At this point, I thought it would be a fun task to try to create my own tool which captures screenshots and generates a report as a thought exercise, and the end result is EyeWitness.

EyeWitnessUI

EyeWitness is designed to take a file, parse out the URLs, take a screenshot of the web pages, and generate a report of the screenshot along with some server header information.  EyeWitness is able to parse three different types of files, a general text file with each url on a new line, the xml output from a NMap scan, or a .nessus file.  Jason Hill (@jasonhillva) worked on creating the XML parsing code for EyeWitness, and provided a lot of feedback throughout writing it.  We also compared the results of both the XML and nessus parser to Tim Tomes’s in PeepingTom, and they are near identical, so we’re happy with the parsing capabilities.

In addition to providing the file name, you can also optionally provide a maximum timeout value.  The timeout value is the maximum amount of time EyeWitness waits for a web page to render, before moving on to the next URL in the list.

EyeWitnessCLI

EyeWitness will generate a report based on the screenshots it was able to grab, and will provide the header information alongside it.  The report is extremely similar to PeepingTom’s output because I honestly thought it contained a lot of useful information.

EyeWitnessReport

There is a couple things EyeWitness does to differentiate itself.  EyeWitness is able to identify web application default credentials for the web page that it is looking at.  When EyeWitness recognizes a web application, it will provide the default credentials along with the server header info.  Currently, EyeWitness has a small number of devices/webpages it can recognize in its signature file, however, that’s simply because I don’t have direct access to other machines at the moment.

Also, screenshots captured by EyeWitness are near full-size of the web application itself, and contains the entire page of the URL specified.  You’re able to easily look at the full screenshot by moving the slider around within the table, or simply click on the picture and access it in its own tab.

Another option EyeWitness provides is the ability to open all URLs within a web browser (on Kali) automatically, as it goes through the list of URLs.  So, as the tool runs, an iceweasel web browser will open tabs of all the URLs you provided within the input file.

I’d like to introduce a call to action.  As you find web pages that use default credentials for a web app, or networked devices, I’d love if you could send me the source code of the index page, along with the default credentials, to EyeWitness [at] christophertruncer [dot] com, or simply send a pull request to me with the signature you created in the signature file.  As I encounter applications with default credentials, or I am sent them, I will update EyeWitness to be able to identify and provide those default creds.

To add signatures to the signatures.txt file, simply add the “signature” which is used to uniquely identify the web app/device on a new line, use the “|” (pipe) as the delimiter, and then add the default credentials on the same line.

Thanks again for checking EyeWitness out, and hope that it can help you out on assessments!

EyeWitness can be cloned from – https://github.com/ChrisTruncer/EyeWitness

A slide deck I made for a NOVAHackers presentation is available here.