1
0
mirror of https://github.com/nmap/nmap.git synced 2026-01-31 18:49:08 +00:00
Commit Graph

266 Commits

Author SHA1 Message Date
ron
46711fa004 Fixed a big in the http library that cropped up while scanning multiple hosts at the same ip address. The reason this occurred is because the result would be cached based on the ip address, not the target name, so different vhosts on the same ip address would be considered the same by the cache. 2009-08-26 16:01:00 +00:00
ron
d63fba1dd6 Add some more fingerprints that a commercial tool just found in an application I'm assessing.
I realize we're getting too many fingerprints to handle, but we can always trim it down later.
2009-08-25 00:05:02 +00:00
ron
986ba5ab8c Realized the my check_get() function was elminating perfectly good hosts. Some of our printers use a '301 Moved Permanently' to redirect you to their login page from the root page, but my script was mistaking those for an off-site redirect. I changed check_get() to display a warning rather than causing the scan to fail. 2009-08-24 19:56:52 +00:00
ron
a02756a5e4 Adding a fix for false positives discovered in http-enum.nse and http-userdir-enum.nse. It seems that some hosts (or, at least, one host) returns different results for:
a) Lowercase path (a login page)
b) Paths containing uppercase (becomes a '400 Unknown Error' that indicates a corrupt harddrive)
c) Paths containing a path and no uppercase (the same login page as (a) is displayed, except the font changes colour
The server itself returns that it's 'ACOS HTTPD/1.1.4', and the telnet port says it's 'Lingo VoIP config telnetd'. OS guesses seem to indicate a Telephony device of some sort. In any case, this patch detects these conditions and doesn't bother checking the device if it cannot tell these errors apart
2009-08-24 03:42:01 +00:00
ron
a46298dcc2 Added a script-arg to http.lua: pipeline. Setting pipeline lets you artificially increase/lower pipelining. The primary reason I added it is because sorting through packet captures with a hundred requests pipelined is difficult, so now I can lower it for debugging. I suggest using '--script-args=pipeline=1,limit=5' if troubleshooting issues with http-enum.nse. 2009-08-24 03:26:01 +00:00
batrick
600ec30c9f [NSE] The HTTP Library now caches responses generated from http.get or
http.head (in revision . Because many scripts tend to request the same object,
this helps to prevent sending duplicate requests that consume network
resources needlessly.

The cache is transparent to the script writer in that it will return a cached
result if present when http.get/http.head is called. How the cache is used may
be controlled by options table (described in [1]) passed to the http functions.

Three new boolean values are present:
  o bypass_cache -- The contents of the cache is ignored for the request
                    (method == "GET" or "HEAD")
  o no_cache -- The result of the request is not saved in the cache
                (method == "GET" or "HEAD")
  o no_cache_body -- The body of the request is not saved in the cache
                     (method == "GET" or "HEAD")

The size of the cache is by default 1 MB. The size can be changed using the
script arg (--script-args) http-max-cache-size (e.g. --script-args
http-max-cache-size=1e8).

[1] http://nmap.org/nsedoc/lib/http.html#request
2009-08-24 01:15:28 +00:00
ron
85e7b4dd89 Some fixes to fingerprints suggested by David Fifield 2009-08-23 22:58:12 +00:00
ron
fbf2c3d6a7 Print the http status along with the result, if it isn't 200 OK. This was the original behaviour, but it got lost along the way. 2009-08-23 16:58:53 +00:00
ron
f8e2fe6429 Added a check for a blank body, due to some false positives I found during a random scan 2009-08-23 04:11:04 +00:00
ron
b2a3cd1114 Added a check for default Ruby on Rails image 2009-08-23 01:01:13 +00:00
ron
fbc8e180ca Converted http-userdir-enum.nse to use my http-enum.nse logic (from http.lua). Also moved the 'displayall' script-arg back into the script, not http.lua, because http-userdir-enum.nse actually wants to find non-200 pages (403 Access Denied, for instance). 2009-08-23 00:43:46 +00:00
ron
cf33de55f3 Moved the definition for the 'displayall' argument into http.lua. 2009-08-23 00:24:23 +00:00
ron
77e7738545 Moved the meat of the http-enum.nse code (the 404-detection, HEAD-detection, etc) into http.lua. It may be a better idea to use http-helper.lua or something similar, but for now http.lua works. I'm open to suggestiosn. 2009-08-23 00:22:20 +00:00
ron
30fe087fd3 Added a '/' before 'robots.txt' 2009-08-23 00:20:18 +00:00
ron
cf91bcf799 Added robots.txt (there's another script for it, but it fits with these fingerprints 2009-08-23 00:08:58 +00:00
ron
532f44d00f added rss/atom to http-fingerprints.nse 2009-08-23 00:00:20 +00:00
ron
c9a62acf31 A major overhaul of the http-enum.nse script:
* Cleaned up and function-ized the code. Planning to move the functions to http.lua or similar to let other scripts (like http-userdir-enum.nse) use them. 
* Better support for different HTTP error codes
* Significantly improved detection of 404 pages to prevent false positives. Some of the techniques used:
  - Request a non-existent page and check the status code
    - If it's 200, try to remove anything that may change (times, URI, filenames, etc), md5 it if ssl is available, and store it. Request a second 404 page and do the same. If they don't match, abort. 
    - If it's any other error code, store it, and look for it in addition to 404 Not Found
  - Request the root folder, /
    - If it returns a 301 Move Permanently or 401 Authentication Required, abort
* Abstracted the fingerprints into files in the nselib/data directory:
  - http-fingerprints: common files/folders
  - yokoso-fingerprints: common applications (from the Yokoso project, by InGuardians -- used under the Nmap license with pemission from Kevin Johnson -- http://seclists.org/nmap-dev/2009/q3/0685.html)
* Added a script-arg that can display all output (except known 404s), instead of just 200 OK and 401 Authentication Required
* Added a lot of debugging messages
2009-08-22 23:19:16 +00:00
ron
393110baf1 Fixed a bug where http.pipeline() would print a cryptic error if no requests were passed for it to perform. Changed it to print a warning if debugging is turned on, and return an empty set. 2009-08-22 22:23:03 +00:00
ron
08073b43b8 Fix two bugs in the http libraries:
1) Re-add 443 to the common ssl ports (it was accidentally removed in an earlier patch)
2) If the header doesn't return the number of pipelined requests to perform, default to 40 instead of leaving it nil
2009-08-21 23:42:07 +00:00
joao
2264838cf8 Fixed bug where socket error messages (like EOF) were appended to the HTTP pipeline response 2009-08-20 03:12:54 +00:00
ron
f2ed53ebfc Removed an undeclared variable in http.lua. 2009-08-18 02:20:33 +00:00
ron
6aee1ce91c Fixed a spelling mistake in http.lua. 2009-08-18 01:40:33 +00:00
batrick
d5748b1d83 If the response is already a table (from an error in http.request) then do not
try to parse a result, instead return nil.  This fixes an error where we try to
index match as though response is a string.
2009-08-15 14:53:28 +00:00
joao
a2dcbb7cd5 Fixing a bug where getPipelineMax was not returning any value, when it sould, at least, return 1. 2009-08-15 14:32:17 +00:00
david
6c766c58cd Add the nmap.get_ssl_certificate NSE function, which returns the peer
certificate of an SSL socket.
2009-08-13 23:32:33 +00:00
joao
0a3a1b091c Applying a lot of new features to http.lua:
- Cookie Support
- Pipelining Support
- New lib modularization (previously, single functions were taking care of too many tasks, we have splitted these tasks for better code reuse)
2009-08-12 01:40:06 +00:00
batrick
990cb8965d Removed unused bopt variable. 2009-08-11 05:56:41 +00:00
batrick
ac55b3ca78 Fixed a regexp escape sequence and shortened code. 2009-08-10 12:51:14 +00:00
joao
11be55c93c Included new ports to common ssl ports,
Fix for resource leak, explicitly closing sockets when connection was not considered succesful
2009-08-10 04:39:11 +00:00
david
fdd82d1576 Call tostring on table keys before concatentating them in
nsedebug.tostr, so it at least doesn't crash when a table has another
table as a key.
2009-08-03 23:48:06 +00:00
joao
24b5c338b6 New lib to deal with proxies from scripts.
This lib is being used on the proxy detection scripts, but can also be used to deal with proxy handshaking.
2009-07-30 02:11:21 +00:00
joao
6218ce14a5 Added POST and HEAD support to http.lua 2009-07-24 08:00:32 +00:00
jah
fe186c5500 Normalised the return values from dns.query() (and consequently
dns.findNiceAnswer() and associated answerFetcher functions).
(see http://seclists.org/nmap-dev/2009/q3/0253.html )

Improved the handling of MX records.
2009-07-22 10:04:41 +00:00
jah
dcfa102a1b Fix the indexing of Globals - found with the help of strict.lua and Patrick's
check_globals script.

Commented-out two functions which are currently unused:
svcctl_Type_tostr() and svcctl_State_tostr()
and which call currently unavailable functions:
svcctl_Type_str() and svcctl_State_str()

(see http://seclists.org/nmap-dev/2009/q3/0194.html )
2009-07-17 22:38:26 +00:00
ron
1b2392f73d Fixed an undeclared variable bug in a currently-unused codepath. 2009-07-17 21:56:11 +00:00
batrick
a8891ec0ca If the old _index metamethod does not exist then do not use it. This would
cause "attempt to index a nil value" errors when whatever is being made
"strict" did not have a metatable (the global environment for example).
2009-07-16 03:43:23 +00:00
batrick
ba4e924634 Corrected the NSEDoc formatting and wording. 2009-07-15 15:19:33 +00:00
batrick
9d5de2a9d1 Added strict library for libraries (and scripts). The module function is
replaced so that all NSE modules will have strict declared global checking at
runtime. This should catch most developer errors early. Please see [1] for
further reference into the problem.

[1] http://seclists.org/nmap-dev/2009/q3/0070.html
2009-07-15 15:14:00 +00:00
batrick
67f31fd170 Fixed a global access for 'opts' using check_globals script. 2009-07-15 14:16:14 +00:00
david
426ed74a8b Add the http-date.nse script. Also the function http.parse_date that
parses an HTTP date in any of three possible formats.

http-date.nse gets the Date: header field value from and HTTP server,
then displays it with how much it differs from local time.
2009-07-13 22:03:02 +00:00
joao
c455e12b7d http.lua using a different approach to deal with ssl connections. Now it is using the comm.tryssl method. 2009-07-13 15:24:14 +00:00
david
276a2f195c [NSE] Scripts that are listed by name with the --script option now
have their verbosity level automatically increased by one. Many
will print negative results ("no infection found") at a higher
verbosity level. The idea is that if you ask for a script
specifically, you are more interested in such results.
2009-07-12 19:35:56 +00:00
david
36b12c9a21 Make decoding of the "chunked" transfer encoding more robust in
http.lua. Previously the code was not careful about checking its input,
and would loop forever if it got something non-hex where it expected a
hex chunk-size. See http://seclists.org/nmap-dev/2009/q3/0013.html.

This code raises an error whenever decoding fails for whatever reason.
It also handles chunk extensions. The reference is section 3.6.1 of
RFC 2616.
2009-07-09 03:07:31 +00:00
batrick
90a712ae2b Patch to libraries that were inappropriately using globals.
Often two (or more) scripts using the same library would
overwrite the globals each was using. This would result
in (at best) an error or (at worst) a deadlock.

The patch changes the global accesses to local.
2009-07-07 00:20:52 +00:00
joao
53c5e64c43 Fixed local/global issues with some vars 2009-07-05 09:59:02 +00:00
batrick
1963fe7b96 Removed some code that did nothing. 2009-07-03 04:05:10 +00:00
batrick
6595577d6e Close the socket when finished getting the banner. 2009-07-01 21:43:49 +00:00
fyodor
9ca4041ec1 Merged -r13793:HEAD from nmap-exp/dev/nmap branch now that we're opening up trunk development
again.  Here are the items which were merged:

------------------------------------------------------------------------
r13971 | jah | 2009-06-29 14:30:27 -0700 (Mon, 29 Jun 2009) | 2 lines

Improved a pattern for matching HTTP status-line, tidy away some variables and
fix a typo.
------------------------------------------------------------------------
r13967 | daniel | 2009-06-29 13:47:04 -0700 (Mon, 29 Jun 2009) | 5 lines

o Added a convenience top-level BSD makefile redirecting BSD make
  to GNU make on BSD systems.  This should help prevent bogus
  error reports when users run "make" instead of "gmake" on BSD
  systems. [Daniel Roethlisberger]

------------------------------------------------------------------------
r13965 | batrick | 2009-06-29 06:50:11 -0700 (Mon, 29 Jun 2009) | 14 lines

[NSE] The NSE Nsock Library binding no longer relies on garbage collection to
monitor the use of socket "slots". A thread (script) attempting to connect must
first obtain one of a limited number of available socket locks (usually 10 or
--max-parallelism). The binding would use garbage collection of sockets to
determine when a thread has finished using its allocated sockets. This is
unfortunately slow and requires us to constantly run the garbage collector to
cause timely reallocation. I have changed the binding to now regularly inspect
allocated sockets in the nsock_loop function. Available sockets slots are now
immediately reallocated and done with far less execution time.

See [1] for benchmarks and further explanation.

[1] http://seclists.org/nmap-dev/2009/q2/0624.html

------------------------------------------------------------------------
r13964 | batrick | 2009-06-29 06:37:49 -0700 (Mon, 29 Jun 2009) | 10 lines

[NSE] Fixed a rare (and usually undetectable) bug that can cause a SEGFAULT.
The NSE nsock library binding may attempt to push values on the stack of
a thread that ended due to an error. It is possible that the internal
Lua stack was completely full and any further pushed values would result
in a segmentation memory violation.

This bug is very hard to reproduce with a SEGFAULT but is usually visible
when Lua assertion checks are turned on. A socket handler routine must be
called AFTER a thread has ended in error.

------------------------------------------------------------------------
r13963 | batrick | 2009-06-29 05:51:20 -0700 (Mon, 29 Jun 2009) | 3 lines

Fixed some global scoped variables to be local. This caused a many scripts to
overwrite each others' sockets, options, etc.

------------------------------------------------------------------------
r13939 | joao | 2009-06-27 16:07:35 -0700 (Sat, 27 Jun 2009) | 2 lines

Fixed port rule to include ssl pop3 port, now that pop3.lua supports SSL connections in function capabilities

------------------------------------------------------------------------
r13938 | joao | 2009-06-27 16:06:28 -0700 (Sat, 27 Jun 2009) | 2 lines

Added transparent SSL support using comm.tryssl

------------------------------------------------------------------------
r13937 | joao | 2009-06-27 16:05:19 -0700 (Sat, 27 Jun 2009) | 2 lines

Added transparent SSL support using comm.tryssl

------------------------------------------------------------------------
r13936 | joao | 2009-06-27 16:03:50 -0700 (Sat, 27 Jun 2009) | 2 lines

Added SSL transparent support using comm.tryssl

------------------------------------------------------------------------
r13935 | joao | 2009-06-27 16:02:39 -0700 (Sat, 27 Jun 2009) | 2 lines

Added SSL transparent support using comm.tryssl

------------------------------------------------------------------------
r13934 | joao | 2009-06-27 16:01:38 -0700 (Sat, 27 Jun 2009) | 2 lines

Added SSL transparent support using comm.tryssl

------------------------------------------------------------------------
r13933 | joao | 2009-06-27 16:00:27 -0700 (Sat, 27 Jun 2009) | 2 lines

SSL transparent support using comm.tryssl

------------------------------------------------------------------------
r13932 | joao | 2009-06-27 15:19:58 -0700 (Sat, 27 Jun 2009) | 2 lines

Included transparent ssl support to function pop3.capabilities using comm.tryssl

------------------------------------------------------------------------
r13931 | joao | 2009-06-27 15:19:06 -0700 (Sat, 27 Jun 2009) | 3 lines

New version of comm.lua with function tryssl, that transparently adds support to ssl connections


------------------------------------------------------------------------
r13930 | joao | 2009-06-27 14:50:38 -0700 (Sat, 27 Jun 2009) | 6 lines

Fixed buffering problem exposed by david on nmap-dev list.
The problem was solved using a buffer to receive the data, making the script work fine in cases where the ssh packets are fragmented.

A very similar solution was applied to ssh1.lua.


------------------------------------------------------------------------
r13928 | batrick | 2009-06-27 04:43:12 -0700 (Sat, 27 Jun 2009) | 18 lines

[NSE] We now propogate a NSE initiated yield on a script through all user
coroutines so that NSE may resume control. Previously, scripts that would yield
in a child coroutine (e.g. a script's child coroutine generated by Lua's
coroutine.create function) would give control back to the script. A script
would yield in this way by making a blocking socket operation. NSE would be
unable to correctly resume child coroutine when the socket operation is
finished processing.

By yielding the chain of coroutines a script has operating, we allow to NSE to
handle the socket operation properly. NSE would then resume the entire chain so
execution may correctly resume at the coroutine which initiated the socket
operation. This restores the "illusion" that a script executes without
interruption.

See [1] for more information, further explanation, and some use cases.

[1] http://seclists.org/nmap-dev/2009/q2/0586.html

------------------------------------------------------------------------
r13817 | david | 2009-06-18 15:57:29 -0700 (Thu, 18 Jun 2009) | 3 lines

Improve an OS fingerprint with a model number and broader matching.
Based on a follow-up report from a submitter.

------------------------------------------------------------------------
r13814 | josh | 2009-06-17 21:34:15 -0700 (Wed, 17 Jun 2009) | 3 lines

[zenmap] Added support to zenmap for the new SCTP options: -PY, -sY and -sZ


------------------------------------------------------------------------
r13797 | ron | 2009-06-17 11:02:18 -0700 (Wed, 17 Jun 2009) | 1 line

Applied a patch from Mak Kolibabi that enhances the output of smb-enum-processes. The output is now modeled after the output of the 'ps' tool for higher verbosity levels.
------------------------------------------------------------------------
r13795 | david | 2009-06-17 09:05:21 -0700 (Wed, 17 Jun 2009) | 6 lines

The configure script now allows cross-compiling by assuming that
libpcap is recent enough. Previously it would quit because a test
program could not be run. libpcap will always be recent enough when
the included copy is used. The patch was contributed by Mike
Frysinger.
2009-06-29 23:48:19 +00:00
david
aef834cd12 Do a print_debug in ssh2.lua when an incomplete packet is processed. 2009-06-14 17:52:47 +00:00
ron
ab59b09c03 Miscellaneous code cleanups in my smb-* libraries/scripts (removed unused includes, old debug code, etc.) 2009-06-10 23:13:15 +00:00