* Cleaned up and function-ized the code. Planning to move the functions to http.lua or similar to let other scripts (like http-userdir-enum.nse) use them.
* Better support for different HTTP error codes
* Significantly improved detection of 404 pages to prevent false positives. Some of the techniques used:
- Request a non-existent page and check the status code
- If it's 200, try to remove anything that may change (times, URI, filenames, etc), md5 it if ssl is available, and store it. Request a second 404 page and do the same. If they don't match, abort.
- If it's any other error code, store it, and look for it in addition to 404 Not Found
- Request the root folder, /
- If it returns a 301 Move Permanently or 401 Authentication Required, abort
* Abstracted the fingerprints into files in the nselib/data directory:
- http-fingerprints: common files/folders
- yokoso-fingerprints: common applications (from the Yokoso project, by InGuardians -- used under the Nmap license with pemission from Kevin Johnson -- http://seclists.org/nmap-dev/2009/q3/0685.html)
* Added a script-arg that can display all output (except known 404s), instead of just 200 OK and 401 Authentication Required
* Added a lot of debugging messages
1) Re-add 443 to the common ssl ports (it was accidentally removed in an earlier patch)
2) If the header doesn't return the number of pipelined requests to perform, default to 40 instead of leaving it nil
was incorrect; the hopDistance member can be much higher than the actual
number of hops recorded. It was 33 when the real distance was 17.
Instead, enumerate and count all the probes that got a response.
mapping from source port number to probes. Upon output, the map was
transformed in place to a map of TTL values to probes. Operations that
worked before output wouldn't work after output and vice versa. Now the
TTL-to-probe map is kept separate in a local variable.
the list of consolidated hops (the ones that match the reference trace)
was printed out one hop further than it should have been. So if a trace
diverged from the reference trace at the sixth hop, it would print out
the first six hops of the reference trace when it should have done only
five. This extra row, as well as being incorrect, could cause an
assertion failure by making the output table one row bigger than its
preallocated capacity.
I can find the bug that causes too many rows to be printed in debugging
mode in some cases. I get an assertion failure when running
nmap scanme.nmap.org/26 --top-ports 10 --traceroute -n -d
With this change, the bug manifests itself as some peculiar output:
4 -- 66.54.149.185
5 -- 63.211.250.17
5 39.17 ms 63.211.250.17
6 47.12 ms 4.68.107.190
7 39.72 ms 4.69.132.37
(Note the doubled 5 with the same IP address.)
10 -- 207.88.13.122
11 -- 207.88.12.46
12 -- 207.88.12.61
13 -- 65.106.1.57
13 100.77 ms 65.106.1.65
14 91.75 ms 65.106.5.162
(Note the doubled 13 with a different IP address.)
4 -- 66.54.149.185
5 -- 63.211.250.17
5 ...
6 39.32 ms 4.68.107.190
(Note the doubled 5 with a timeout.)
at debugging level 2 or higher and when a key is pressed. This is especially
useful when we have a rare deadlock and we want to know which scripts are
involved and where (a traceback is also printed).
nse_nsock_get_ssl. In older versions of OpenSSL (I don't know how old)
the argument to SSL_get_peer_certificate is not const and it caused a
compiler error reported by Ron Bowes.
