buildGet/buildRequest with a one-step build_request. Provide a new
function generic_request that can do a request for any given method
(get, head, and post are now defined in terms of this function).
options table, in http functions. It was unreasonable that this
yet-unused feature was given a more prominent place than even the header
and request body, both of which are in the options table.
This change doesn't affect any other scripts or libraries because none
of them use cookies. In the cases, like http.get, where cookies was an
optional final parameter, I just removed it. Where it was not the final
parameter, as in http.post and http.pGet, I left the parameter in place
but documented that it is ignored for backwards compatibility.
there is one. Even though section 4.4 of RFC 2616 says that sending a
body in response to a HEAD request is a MUST NOT, pyllyukko sent me a
sample from a server that does--"LuCI - Lua Configuration Interface".
initial request to get a value stored in the Keep-Alive header, which is
the size of the pipeline. It then iterates, doignt hat many requests at
once until the list of requests is exhausted. The prbolem was that in
the first round, it didn't count its initial Keep-Alive probe. So if the
server said it was good for 40 requests, we would send 41 before closing
the connection. Even worse was when the initial probe returned a
"Connection: close"; the pipeline would try another request before
closing the connection for the first time.
failure. It happened when there was an error getting the a response
at the beginning of a batch in http.pipeline. The symptoms of the
bug were:
NSE: Received only 0 of 1 expected reponses.
Decreasing max pipelined requests to 0.
NSOCK (0.1870s) Write request for 0 bytes...
nmap: nsock_core.c:516: handle_write_result: Assertion `bytesleft > 0' failed.
The error was reported by Brandon Enright and pyllyukko.
parsing mostly. Response parsing is centralized, and fewer operations
are done on raw HTTP data.
The biggest user-visible change is that http.request goes back to
returning a parsed result table, not raw HTTP data. I believe this is
how the function worked in the past; it's what the NSEDoc for the
function says. The only thing that used http.request was citrixxml.lua,
and this commit alters it to match the new expectations.
The other change is that the http.pipeline function no longer accepts
the "raw" option. The only script that used that was sql-injection.nse,
and this commit modifies that script as well.
HEAD request. This makes it look like the response to the HEAD request
has a body. As a result, http-enum can wrongly surmise that HEAD
requests don't work when it hits a cache entry while testing HEAD.
# nmap --script=http-date,http-enum -PN -p 80 -d2
NSE: NSE Script Threads (2) running:
NSE: Final http cache size (373 bytes) of max size of 1000000
NSE: HTTP: Host returns proper 404 result.
NSE: Final http cache size (1905 bytes) of max size of 1000000
NSE: Finished 'http-date' (thread: 0x8535a40).
NSE: HTTP: Warning: Host returned data when performing HEAD.
NSE: Checking if a GET request is going to work out
After this change:
NSE: NSE Script Threads (2) running:
NSE: Final http cache size (373 bytes) of max size of 1000000
NSE: HTTP: Host returns proper 404 result.
NSE: Final http cache size (1905 bytes) of max size of 1000000
NSE: Finished 'http-date' (thread: 0x8e75b00).
NSE: Final http cache size (1905 bytes) of max size of 1000000
NSE: HTTP: Host supports HEAD.
isChunked now checks for transfer-encoding: identity instead of transfer-encoding: chunked. If transfer encoding is present and it is not identity, chunked encoding is considered. Also rfc2616, section 4.4 (item 2)
a) Lowercase path (a login page)
b) Paths containing uppercase (becomes a '400 Unknown Error' that indicates a corrupt harddrive)
c) Paths containing a path and no uppercase (the same login page as (a) is displayed, except the font changes colour
The server itself returns that it's 'ACOS HTTPD/1.1.4', and the telnet port says it's 'Lingo VoIP config telnetd'. OS guesses seem to indicate a Telephony device of some sort. In any case, this patch detects these conditions and doesn't bother checking the device if it cannot tell these errors apart
http.head (in revision . Because many scripts tend to request the same object,
this helps to prevent sending duplicate requests that consume network
resources needlessly.
The cache is transparent to the script writer in that it will return a cached
result if present when http.get/http.head is called. How the cache is used may
be controlled by options table (described in [1]) passed to the http functions.
Three new boolean values are present:
o bypass_cache -- The contents of the cache is ignored for the request
(method == "GET" or "HEAD")
o no_cache -- The result of the request is not saved in the cache
(method == "GET" or "HEAD")
o no_cache_body -- The body of the request is not saved in the cache
(method == "GET" or "HEAD")
The size of the cache is by default 1 MB. The size can be changed using the
script arg (--script-args) http-max-cache-size (e.g. --script-args
http-max-cache-size=1e8).
[1] http://nmap.org/nsedoc/lib/http.html#request
1) Re-add 443 to the common ssl ports (it was accidentally removed in an earlier patch)
2) If the header doesn't return the number of pipelined requests to perform, default to 40 instead of leaving it nil
