From da4d624572200f5c8d783e28bd72152fa37eaeca Mon Sep 17 00:00:00 2001 From: nnposter Date: Sat, 29 Jul 2017 01:31:33 +0000 Subject: [PATCH] Refreshes the main documentation section for http.lua. Fixes #933 --- nselib/http.lua | 114 ++++++++++++++++++++++++------------------------ 1 file changed, 56 insertions(+), 58 deletions(-) diff --git a/nselib/http.lua b/nselib/http.lua index 39029b969..2644a8010 100644 --- a/nselib/http.lua +++ b/nselib/http.lua @@ -1,75 +1,38 @@ ---Implements the HTTP client protocol in a standard form that Nmap scripts can -- take advantage of. -- --- Because HTTP has so many uses, there are a number of interfaces to this library. --- The most obvious and common ones are simply get, post, --- and head; or, if more control is required, generic_request --- can be used. These functions do what one would expect. The get_url --- helper function can be used to parse and retrieve a full URL. +-- Because HTTP has so many uses, there are a number of interfaces to this +-- library. +-- +-- The most obvious and common ones are simply get, +-- post, and head; or, if more control is required, +-- generic_request can be used. These functions take host and port +-- as their main parameters and they do what one would expect. The +-- get_url helper function can be used to parse and retrieve a full +-- URL. -- -- HTTPS support is transparent. The library uses comm.tryssl to -- determine whether SSL is required for a request. -- -- These functions return a table of values, including: --- * status-line - A string representing the status, such as "HTTP/1.1 200 OK". In case of an error, a description will be provided in this line. +-- * status-line - A string representing the status, such as "HTTP/1.1 200 OK", followed by a newline. In case of an error, a description will be provided in this line. -- * status - The HTTP status value; for example, "200". If an error occurs during a request, then this value is going to be nil. -- * version - HTTP protocol version string, as stated in the status line. Example: "1.1" -- * header - An associative array representing the header. Keys are all lowercase, and standard headers, such as 'date', 'content-length', etc. will typically be present. -- * rawheader - A numbered array of the headers, exactly as the server sent them. While header['content-type'] might be 'text/html', rawheader[3] might be 'Content-type: text/html'. --- * cookies - A numbered array of the cookies the server sent. Each cookie is a table with the following keys: name, value, path, domain, and expires. --- * body - The full body, as returned by the server. +-- * cookies - A numbered array of the cookies the server sent. Each cookie is a table with the expected keys, such as name, value, path, domain, and expires. This table can be sent to the server in subsequent responses in the options table to any function (see below). +-- * body - The full body, as returned by the server. Chunked encoding is handled transparently. -- * fragment - Partially received body (if any), in case of an error. +-- * location - A numbered array of the locations of redirects that were followed. -- --- If a script is planning on making a lot of requests, the pipelining functions can --- be helpful. pipeline_add queues requests in a table, and --- pipeline_go performs the requests, returning the results as an array, --- with the responses in the same order as the queries were added. As a simple example: --- --- -- Start by defining the 'all' variable as nil --- local all = nil --- --- -- Add two 'GET' requests and one 'HEAD' to the queue. These requests are not performed --- -- yet. The second parameter represents the 'options' table, which we don't need. --- all = http.pipeline_add('/book', nil, all) --- all = http.pipeline_add('/test', nil, all) --- all = http.pipeline_add('/monkeys', nil, all, 'HEAD') --- --- -- Perform all three requests as parallel as Nmap is able to --- local results = http.pipeline_go('nmap.org', 80, all) --- --- --- At this point, results is an array with three elements. Each element --- is a table containing the HTTP result, as discussed above. --- --- One more interface provided by the HTTP library helps scripts determine whether or not --- a page exists. The identify_404 function will try several URLs on the --- server to determine what the server's 404 pages look like. It will attempt to identify --- customized 404 pages that may not return the actual status code 404. If successful, --- the function page_exists can then be used to determine whether or not --- a page existed. --- --- Some other miscellaneous functions that can come in handy are response_contains, --- can_use_head, and save_path. See the appropriate documentation --- for them. --- --- The response to each function is typically a table with the following keys: --- * status-line: The HTTP status line; for example, "HTTP/1.1 200 OK" (note: this is followed by a newline). In case of an error, a description will be provided in this line. --- * status: The HTTP status value; for example, "200". If an error occurs during a request, then this value is going to be nil. --- * version: HTTP protocol version string, as stated in the status line. Example: "1.1" --- * header: A table of header values, where the keys are lowercase and the values are exactly what the server sent --- * rawheader: A list of header values as "name: value" strings, in the exact format and order that the server sent them --- * cookies: A list of cookies that the server is sending. Each cookie is a table containing the keys name, value, and path. This table can be sent to the server in subsequent responses in the options table to any function (see below). --- * body: The body of the response --- * fragment: Partially received body (if any), in case of an error. --- * location: a list of the locations of redirects that were followed. --- --- Many of the functions optionally allow an 'options' table. This table can alter the HTTP headers --- or other values like the timeout. The following are valid values in 'options' (note: not all --- options will necessarily affect every function): +-- Many of the functions optionally allow an "options" input table, which can +-- modify the HTTP request or its processing in many ways like adding headers or +-- setting the timeout. The following are valid keys in "options" +-- (note: not all options will necessarily affect every function): -- * timeout: A timeout used for socket operations. -- * header: A table containing additional headers to be used for the request. For example, options['header']['Content-Type'] = 'text/xml' --- * content: The content of the message (content-length will be added -- set header['Content-Length'] to override). This can be either a string, which will be directly added as the body of the message, or a table, which will have each key=value pair added (like a normal POST request). --- * cookies: A list of cookies as either a string, which will be directly sent, or a table. If it's a table, the following fields are recognized: name, value, path, expires. Only name and value fields are required. +-- * content: The content of the message. This can be either a string, which will be directly added as the body of the message, or a table, which will have each key=value pair added (like a normal POST request). (A corresponding Content-Length header will be added automatically. Set header['Content-Length'] to override it). +-- * cookies: A list of cookies as either a string, which will be directly sent, or a table. If it's a table, the following fields are recognized: name, value and path. Only name and value fields are required. -- * auth: A table containing the keys username and password, which will be used for HTTP Basic authentication. -- If a server requires HTTP Digest authentication, then there must also be a key digest, with value true. -- If a server requires NTLM authentication, then there must also be a key ntlm, with value true. @@ -89,6 +52,41 @@ -- end -- end -- + +-- If a script is planning on making a lot of requests, the pipelining functions +-- can be helpful. pipeline_add queues requests in a table, and +-- pipeline_go performs the requests, returning the results as an +-- array, with the responses in the same order as the requests were added. +-- As a simple example: +-- +-- -- Start by defining the 'all' variable as nil +-- local all = nil +-- +-- -- Add two GET requests and one HEAD to the queue but these requests are +-- -- not performed yet. The second parameter represents the "options" table +-- -- (which we don't need in this example). +-- all = http.pipeline_add('/book', nil, all) +-- all = http.pipeline_add('/test', nil, all) +-- all = http.pipeline_add('/monkeys', nil, all, 'HEAD') +-- +-- -- Perform all three requests as parallel as Nmap is able to +-- local results = http.pipeline_go('nmap.org', 80, all) +-- +-- +-- At this point, results is an array with three elements. +-- Each element is a table containing the HTTP result, as discussed above. +-- +-- One more interface provided by the HTTP library helps scripts determine +-- whether or not a page exists. The identify_404 function will +-- try several URLs on the server to determine what the server's 404 pages look +-- like. It will attempt to identify customized 404 pages that may not return +-- the actual status code 404. If successful, the function +-- page_exists can then be used to determine whether or not a page +-- exists. +-- +-- Some other miscellaneous functions that can come in handy are +-- response_contains, can_use_head, and +-- save_path. See the appropriate documentation for details. -- -- @args http.max-cache-size The maximum memory size (in bytes) of the cache. -- @@ -895,8 +893,8 @@ end --- Builds a string to be added to the request mod_options table -- --- @param cookies A cookie jar just like the table returned parse_set_cookie. --- @param path If the argument exists, only cookies with this path are included to the request +-- @param cookies A cookie jar just like the table returned by parse_set_cookie. +-- @param path If the argument exists, only cookies with this path are included in the request -- @return A string to be added to the mod_options table local function buildCookies(cookies, path) if type(cookies) == 'string' then return cookies end