From ca77bf952a60deb29c2e2a18fa50770d91707564 Mon Sep 17 00:00:00 2001 From: david Date: Fri, 4 Jul 2008 01:05:04 +0000 Subject: [PATCH] Index scripting.xml. --- docs/scripting.xml | 582 +++++++++++++++++++++++++++++---------------- 1 file changed, 381 insertions(+), 201 deletions(-) diff --git a/docs/scripting.xml b/docs/scripting.xml index a5ad77c51..12ab8773c 100644 --- a/docs/scripting.xml +++ b/docs/scripting.xml @@ -1,8 +1,10 @@ +Nmap Scripting Engine (NSE) +scriptingNmap Scripting Engine +NSENmap Scripting Engine + Introduction - Nmap Scripting Engine - scriptingNmap Scripting Engine The Nmap Scripting Engine (NSE) is one of Nmap's most powerful and flexible features. It allows users to write (and share) simple scripts to automate a wide variety of networking @@ -11,10 +13,12 @@ growing and diverse set of scripts distributed with Nmap, or write their own to meet custom needs. - The Nmap project would like to thank Diman Todorov for - his excellent work building the initial NSE implementation and - writing much of this documentation. Stoiko Ivanov also - contributed greatly. The tasks we had in mind when + The Nmap project would like to thank Diman Todorov + Todorov, Diman + for his excellent work building the initial NSE implementation and + writing much of this documentation. Stoiko Ivanov + Ivanov, Stoiko + also contributed greatly. The tasks we had in mind when creating the system are: @@ -30,7 +34,9 @@ - More sophisticated version detection + + version detectionusing NSE + More sophisticated version detection The Nmap version detection system () is able to recognize thousands of different services through @@ -47,7 +53,9 @@ - Vulnerability detection + + vulnerability detection + Vulnerability detection When a new vulnerability is discovered, you often want @@ -65,7 +73,9 @@ backdoors to enable later reentry. Some of these can be detected by Nmap's regular expression based version detection. For example, within hours of the MyDoom worm hitting the - Internet, Jay Moran posted an Nmap version detection probe and + Internet, Jay Moran + Moran, Jay + posted an Nmap version detection probe and signature so that others could quickly scan their networks. For more complex worms and backdoors, NSE is needed instead. @@ -79,9 +89,12 @@ As a general scripting language, NSE could even be used to exploit vulnerabilities rather than just find them. The capability to add custom exploit scripts may be valuable - for some people (particularly penetration testers), though we aren't + for some people (particularly penetration testers), + penetration testing + though we aren't planning to turn Nmap into an exploitation framework like Metasploit. + Metasploit @@ -96,6 +109,8 @@ Scripts are written in the embedded Lua programming language. + Lua programming language + Lua programming languageNmap Scripting Engine The language itself is well documented in the books Programming @@ -111,17 +126,23 @@ The reference manual is also freely available online, as is the - first edition of Programming in - Lua. Given the availability of these excellent general + first edition of Programming in + Lua. Given the availability of these excellent general Lua programming references, this document only covers aspects and extensions specific to Nmap's scripting engine. - NSE is activated with the option (or - if you wish to specify a custom set of - scripts) and results are integrated into Nmap normal and XML - output. Two types of scripts are supported: service and host + NSE is activated with the + + option (or + + if you wish to specify a custom set of + scripts) and results are integrated into Nmap normal + normal output + and XML output. + XML output + Two types of scripts are supported: service and host scripts. Service scripts relate to a certain open port (service) on the target host, and any results they produce are included next to that port in the Nmap output port table. Host scripts, @@ -137,10 +158,12 @@ The reference manual is also which simply grabs the title of the root path of any web servers found. A sample host script is RIPE Query, which looks up and reports target IP ownership information. + script names, examples of Typical NSE Output +example of $ ./nmap -sC localhost -p 22,23,80,113 @@ -167,31 +190,40 @@ Nmap finished: 1 IP address (1 host up) scanned in 0.907 seconds Usage and Examples While NSE has a complex implementation for efficiency, it is - strikingly easy to use. Simply specify to - enable the most common scripts. Or specify the - option to choose your own scripts to + strikingly easy to use. Simply specify + + to enable the most common scripts. Or specify the + + + option to choose your own scripts to execute by providing categories, script file names, or the name of directories full of scripts you wish to execute. You can customize some scripts by providing arguments to them via the - option. The two - remaining options, and - , are generally only used for - script debugging and development. + + + option. The two remaining options, + + and , + + are generally only used for script debugging and development. Script Categories + script categories NSE scripts define a list of categories they belong to. Currently defined categories are safe, intrusive, malware, version, discovery, vuln, auth and - default. Categories are not case + default. + Categories are not case sensitive. The following list describes each category. + safe script category @@ -210,6 +242,7 @@ Nmap finished: 1 IP address (1 host up) scanned in 0.907 seconds + intrusive script category @@ -224,6 +257,7 @@ Nmap finished: 1 IP address (1 host up) scanned in 0.907 seconds + malware script category @@ -234,6 +268,8 @@ Nmap finished: 1 IP address (1 host up) scanned in 0.907 seconds + version script category + version detectionversion script caetgory @@ -249,6 +285,7 @@ Nmap finished: 1 IP address (1 host up) scanned in 0.907 seconds + discovery script category @@ -260,6 +297,7 @@ Nmap finished: 1 IP address (1 host up) scanned in 0.907 seconds + vuln script category @@ -270,6 +308,7 @@ Nmap finished: 1 IP address (1 host up) scanned in 0.907 seconds + auth script category @@ -280,6 +319,7 @@ Nmap finished: 1 IP address (1 host up) scanned in 0.907 seconds + default script category @@ -298,6 +338,7 @@ Nmap finished: 1 IP address (1 host up) scanned in 0.907 seconds Arguments to Scripts + script arguments You can pass arguments to NSE scripts via the option. The script-arguments generally are @@ -311,6 +352,8 @@ Nmap finished: 1 IP address (1 host up) scanned in 0.907 seconds look like: +example of +example of $ nmap -sC --script-args user=foo,pass=bar,anonFTP={pass=ftp@foobar.com} @@ -339,10 +382,8 @@ $ nmap -sC --script-args user=foo,pass=bar,anonFTP={pass=ftp@foobar.com} + - - -sC - Performs a script scan using the default set of scripts. It is @@ -353,7 +394,9 @@ $ nmap -sC --script-args user=foo,pass=bar,anonFTP={pass=ftp@foobar.com} - --script + + + @@ -367,13 +410,15 @@ will try to interpret the arguments at first as categories and afterwards as files or directories. Absolute paths are used as is, relative paths are searched in the following places until found: data filesdirectory search order -scriptsdirectory locations -NMAPDIR environment variable +scriptslocation of --datadir/; +NMAPDIR environment variable $(NMAPDIR)/; ~user/nmap/ (not searched on Windows); +NMAPDATADIR NMAPDATADIR/ or -./. A scripts/ subdirectory is also tried in each of these. Give the argument all to execute all scripts in the Nmap script database. +./. A scripts/ subdirectory +is also tried in each of these. Give the argument all to execute all scripts in the Nmap script database. If a directory is specified and found, Nmap loads all NSE @@ -387,7 +432,9 @@ extension does not have to be nse. Nmap scripts are stored in a scripts subdirectory of the Nmap data directory (see ) by default. Scripts are indexed in a database stored in -scripts/script.db. The database lists all of the +scripts/script.db. +script.db +The database lists all of the scripts in each category. A single script may be in several categories. @@ -396,10 +443,8 @@ categories. + - - --script-args - provides arguments to the scripts. See + - - --script-trace - @@ -431,10 +474,8 @@ categories. + - - --script-updatedb - This option is only useful if you have added or @@ -446,7 +487,9 @@ categories. which the script belongs to. Categories may be specified with the option. For efficiency reasons, NSE generates a - script.db file which maps + script.db + script.db + file which maps categories to the scripts they contain. If you changed tag directives or added/removed scripts, run nmap --script-updatedb. @@ -457,15 +500,21 @@ categories. Some of the Nmap options have effects on script scans. The most - prominent of these is . A version scan executes - the scripts in the version category. The scripts + prominent of these is . + + A version scan executes + the scripts in the version category. + version script category + The scripts in this category are slightly different than other scripts. Their output blends in with the version scan and they do not produce any script scan output. Another option which has effect on the scripting engine is - . The advance/aggressive mode of Nmap implies + . + + The advance/aggressive mode of Nmap implies the option . @@ -486,6 +535,8 @@ categories. Tracing a specific script. + example of + example of $ nmap --script=./showSSHVersion.nse --script-trace hostname @@ -499,6 +550,7 @@ categories. <literal>id</literal> Field + id script variable The script's id field is displayed in the Nmap output table if the script produces any output. It should be unique so users @@ -507,10 +559,12 @@ categories. still being meaningful enough for users to recognize. Some good examples are RIPE query, HTML title, and Kibuv worm. + script names, examples of <literal>description</literal> Field + description script variable The description describes what the script is testing for and any critical notes the user must be aware of. A good @@ -524,6 +578,7 @@ categories. <literal>author</literal> Field + author script variable The author field contains the script authors name and contact information. If you are worried about spam, you might want to omit or obscure your email address, or give your home page URL instead. This optional field is not used by NSE, but is important for giving script authors due credit or blame. @@ -531,6 +586,8 @@ categories. <literal>license</literal> Field + license script variable + license of scripts Nmap is a community project and we welcome all sorts of code contributions, including NSE scripts. So if you write a @@ -553,6 +610,8 @@ that. <literal>runlevel</literal> Field + license script variable + run level of scripts This optional field determines script execution order. When this section is absent the run level defaults to 1.0. A script @@ -573,6 +632,9 @@ that. Port and Host Rules + portrule script variable + hostrule script variable + rules in NSEportrule and hostrule There are two types of rules: host rules @@ -585,6 +647,9 @@ that. only matched against TCP or UDP ports in the open, open|filtered or unfiltered + open port state + open|filtered port state + unfiltered port state states. Host rules are matched exactly once against every scanned host. The action, like the rule, is a Lua function, which takes a host and port table as arguments. If the script is @@ -593,6 +658,7 @@ that. . Action + action script variable The action is the heart of an NSE script. It contains all of @@ -611,10 +677,12 @@ that. Script Language + Nmap Scripting Engine (NSE)parts of Nmap's scripting engine is essentially three distinct - parts. The largest part is the embeddable Lua interpreter. This + parts. The largest part is the embeddable Lua + interpreter. This is a lightweight language designed for extensibility. It offers a powerful and well documented API for interfacing with other software (such as Nmap). @@ -629,6 +697,7 @@ that. exception handling mechanism. + Nmap Scripting Engine (NSE)library The third part of NSE is the NSE library. This library was written to add special functionality and to augment the small @@ -638,6 +707,7 @@ that. Lua Base Language + Lua programming language The Nmap scripting language is an embedded API is in the Lua namespace nmap. This means that all calls to resources provided by Nmap have an nmap prefix. + nmap NSE module nmap.new_socket(), for example, returns a new socket wrapper object. The Nmap library layer also takes care of initializing the Lua context, scheduling parallel @@ -683,6 +754,7 @@ that. + Nmap Scripting Engine (NSE)modules Lua Extensions In addition to the significant built-in capabilities of Lua, we have written or integrated several extensions to make @@ -698,10 +770,14 @@ that. Bitwise Logical Operations + bit NSE module - Lua does not provide bitwise logical operations. Since they + Lua does not provide bitwise logical operations. + bitwise operations in NSE + Since they are often useful for low-level network communication, Reuben Thomas' + Thomas, Reuben bitwise operation library for Lua has been integrated into NSE. The arguments to the bitwise operation @@ -724,7 +800,7 @@ that. - bit.bnot(a) + Returns the one's complement of a. @@ -734,7 +810,7 @@ that. - bit.band(w1,...) + Returns the bitwise and of the @@ -745,7 +821,7 @@ that. - bit.bor(w1,...) + Returns the bitwise or of the w's. @@ -755,7 +831,7 @@ that. - bit.bxor(w1,...) + @@ -767,7 +843,7 @@ that. - bit.lshift(a,b) + Returns a shifted left b places—padded with zeros. @@ -777,7 +853,7 @@ that. - bit.rshift(a,b) + Returns a shifted logically right b places. @@ -787,7 +863,7 @@ that. - bit.arshift(a,b) + Returns a shifted arithmetically right b places. @@ -797,7 +873,7 @@ that. - bit.mod(a,b) + Returns the integer remainder of a divided by b. @@ -810,6 +886,7 @@ that. Binary Data Handling + bin NSE module A problem script authors often face is the necessity of encoding values into binary data. For example after analyzing a protocol the starting @@ -819,6 +896,7 @@ that. BinLib has been added to NSE, based on lpack by Luiz Henrique de Figueiredo. + Henrique de Figueiredo, Luiz The BinLib functions take a format string to encode and decode binary data. The operators of the format string are shown in . @@ -862,7 +940,7 @@ that. - bin.pack(fmt, p1, ...) + Returns a binary packed string. The format string describes how @@ -876,7 +954,7 @@ that. - bin.unpack(fmt, data, [init]) + Returns values read from the binary data string. @@ -897,6 +975,9 @@ that. + pcre NSE module + Perl Compatible Regular Expressions (PCRE)in NSE + regular expressionsin NSE Perl Compatible Regular Expressions @@ -906,7 +987,11 @@ that. powerful as standard regular expressions. So we have integrated Perl compatible regular expressions into Lua using libPCRE and a modified version of the Lua PCRE library - written by Reuben Thomas and Shmuel Zeigerman. These are + written by Reuben Thomas + Thomas, Reuben + and Shmuel Zeigerman. + Zeigerman, Shmuel + These are the same sort of regular expressions used by Nmap version detection. The main modification to their library is that the NSE version only supports PCRExpressions instead of both @@ -922,6 +1007,7 @@ that. namespace. + Perl Compatible Regular Expressions (PCRE)security vulnerabilities in LibPCRE has a history of security vulnerabilities allowing attackers who are able to compile arbitrary regular expressions to execute arbitrary code. More such @@ -940,7 +1026,7 @@ that. - pcre.new + Returns a compiled regular expression. The first @@ -976,7 +1062,7 @@ that. - pcre.flags + Returns a table of the available PCRE option flags @@ -993,7 +1079,7 @@ that. - pcre.version + Returns the version of the PCRE library in use as a @@ -1004,7 +1090,7 @@ that. - pcre.match + Returns the start point and the end point of @@ -1033,7 +1119,7 @@ if(s) code_to_be_done_on_match end - pcre.exec + This function is like match() except that a table returned as @@ -1049,7 +1135,7 @@ if(s) code_to_be_done_on_match end - pcre.gmatch + Tries to match the regular expression pcre_obj against string @@ -1071,10 +1157,12 @@ if(s) code_to_be_done_on_match end + IP Operations + ipOps NSE module The ipOps module provides some functions for manipulating IPv4 addresses. The functions reside inside the @@ -1082,8 +1170,10 @@ if(s) code_to_be_done_on_match end - - isPrivate + + private addressesin NSE + + checks whether an IP address, provided as a string in @@ -1096,7 +1186,7 @@ if(s) code_to_be_done_on_match end - todword + returns the IP address as DWORD value (i.e. the IP a.b.c.d becomes @@ -1106,7 +1196,7 @@ if(s) code_to_be_done_on_match end - get_parts_as_number + returns 4 numbers corresponding to the fields in dotted-quad notation. @@ -1119,6 +1209,7 @@ if(s) code_to_be_done_on_match end Short Portrules + shortport NSE module Since portrules are mostly the same for many scripts, the shortport module provides functions for the most common tests. @@ -1129,7 +1220,7 @@ if(s) code_to_be_done_on_match end - portnumber + The port argument is either a number or a table of numbers which are @@ -1139,7 +1230,7 @@ if(s) code_to_be_done_on_match end - service + The service argument is either a string or a table @@ -1155,7 +1246,7 @@ if(s) code_to_be_done_on_match end - port_or_service + This is a combination of the above functions, since many scripts @@ -1170,6 +1261,7 @@ if(s) code_to_be_done_on_match end Functional Programming Style List Operations + listop NSE module People used to programming in functional languages, such as Lisp or Haskell, appreciate their handling of lists very much. The listop module tries to bring much of the functionality from @@ -1180,7 +1272,7 @@ if(s) code_to_be_done_on_match end - is_empty + Returns true if the given list is empty. @@ -1189,7 +1281,7 @@ if(s) code_to_be_done_on_match end - is_list + Returns true if the given value is a list (or rather a table). @@ -1198,7 +1290,7 @@ if(s) code_to_be_done_on_match end - map + The provided function is applied to each element of the list @@ -1210,7 +1302,7 @@ if(s) code_to_be_done_on_match end - apply + All of the elements in the list are passed to a call of @@ -1222,7 +1314,7 @@ if(s) code_to_be_done_on_match end - filter + Returns a list containing only those elements for which the predicate @@ -1238,7 +1330,7 @@ if(s) code_to_be_done_on_match end - flatten + Since a list can itself contain lists as elements, @@ -1252,7 +1344,7 @@ if(s) code_to_be_done_on_match end - append + Returns a list containing all elements of list1 appended by all @@ -1262,7 +1354,7 @@ if(s) code_to_be_done_on_match end - cons + Returns a list containing value1 appended by value2, which may be @@ -1272,7 +1364,7 @@ if(s) code_to_be_done_on_match end - reverse + Returns a list containing all elements of the given list in inverted @@ -1282,7 +1374,7 @@ if(s) code_to_be_done_on_match end - car + Returns the first element of the given list. @@ -1291,7 +1383,7 @@ if(s) code_to_be_done_on_match end - ncar + Returns the nth (or first if n is omitted) element of the given list. @@ -1300,7 +1392,7 @@ if(s) code_to_be_done_on_match end - cdr + Returns a list containing all elements but the first of the @@ -1310,7 +1402,7 @@ if(s) code_to_be_done_on_match end - ncdr + Returns a list containing all elements but the first n of the @@ -1322,6 +1414,7 @@ if(s) code_to_be_done_on_match end String Buffer Operations + strbuf NSE module Lua's string operations are very flexible and offer an easy-to-use way to manipulate strings. Concatenation using the .. @@ -1348,7 +1441,7 @@ if(s) code_to_be_done_on_match end - new + Creates a new string buffer. The optional arguments are added @@ -1359,7 +1452,7 @@ if(s) code_to_be_done_on_match end - concat + Concatenates the value (which has to be either @@ -1372,7 +1465,7 @@ if(s) code_to_be_done_on_match end - eqbuf + Compares strbuf1 and strbuf2 @@ -1383,7 +1476,7 @@ if(s) code_to_be_done_on_match end - clear + Deletes all strings in strbuf. @@ -1392,7 +1485,7 @@ if(s) code_to_be_done_on_match end - dump + Dumps strbuf's contents as string. The second @@ -1406,6 +1499,7 @@ if(s) code_to_be_done_on_match end URL Manipulation Functions + url NSE module URL manipulation functions have obvious uses. Fortunately there is already an implementation of URL generation functions @@ -1418,7 +1512,7 @@ if(s) code_to_be_done_on_match end - parse_query + This function takes a query-string of the form name1=value1&name2=value2... and returns a table @@ -1431,7 +1525,7 @@ if(s) code_to_be_done_on_match end - build_query + This is the inverse function to parse_query(). @@ -1442,6 +1536,7 @@ if(s) code_to_be_done_on_match end Buffered Network I/O Helper Functions + match NSE module The match module was written to provide functions which can be used for delimiting data received by the @@ -1450,7 +1545,7 @@ if(s) code_to_be_done_on_match end - regex + This is actually a wrapper around NSE's PCRE library exec function (see ), thus @@ -1464,7 +1559,7 @@ if(s) code_to_be_done_on_match end - numbytes + Takes a number as argument and returns that @@ -1483,6 +1578,7 @@ if(s) code_to_be_done_on_match end HTTP Functions + http NSE module The http module provides functions for dealing with the client side of the http protocol. The functions reside inside the http namespace. @@ -1497,7 +1593,7 @@ if(s) code_to_be_done_on_match end - get + Fetches a resource with a GET request. @@ -1516,7 +1612,7 @@ if(s) code_to_be_done_on_match end - request + Sends request to host:port @@ -1534,7 +1630,7 @@ if(s) code_to_be_done_on_match end - get_url + Parses url and calls http.get @@ -1552,10 +1648,12 @@ if(s) code_to_be_done_on_match end Common Communication Functions + comm NSE module The comm module provides functions for common network discovery tasks such as banner-grabbing and making a quick exchange of data. These functions' return values are setup for use with exception handling via nmap.new_try(). + exceptions in NSE @@ -1572,7 +1670,7 @@ if(s) code_to_be_done_on_match end - get_banner + This function simply connects to the specified port number on @@ -1588,7 +1686,7 @@ if(s) code_to_be_done_on_match end - exchange + This function connects to the specified port number on the @@ -1606,6 +1704,8 @@ if(s) code_to_be_done_on_match end Data File Parsing Functions + datafiles NSE module + data files access to from NSE The datafiles module provides functions for reading and parsing Nmap's data files (e.g. nmap-protocol, nmap-rpc, @@ -1616,7 +1716,7 @@ if(s) code_to_be_done_on_match end - parse_protocols + This function reads and parses Nmap's nmap-protocols @@ -1631,7 +1731,7 @@ if(s) code_to_be_done_on_match end - parse_rpc + This function reads and parses Nmap's nmap-rpc @@ -1646,7 +1746,7 @@ if(s) code_to_be_done_on_match end - parse_services + This function reads and parses Nmap's nmap-services @@ -1668,6 +1768,7 @@ if(s) code_to_be_done_on_match end Various Utility Functions + stdnse NSE module The stdnse library contains various handy functions which are too small to justify modules of their own: @@ -1675,7 +1776,7 @@ if(s) code_to_be_done_on_match end - print_debug + Wrapper function around print_debug_unformatted() @@ -1690,7 +1791,7 @@ if(s) code_to_be_done_on_match end - strsplit + This function will certainly be appreciated by Perl programmers. @@ -1702,7 +1803,7 @@ if(s) code_to_be_done_on_match end - strjoin + Inverse function to strsplit(). Basically this is @@ -1713,7 +1814,7 @@ if(s) code_to_be_done_on_match end - tobinary + Converts the given number, n, to a string @@ -1723,7 +1824,7 @@ if(s) code_to_be_done_on_match end - tooctal + Converts the given number, n, to a string @@ -1733,7 +1834,7 @@ if(s) code_to_be_done_on_match end - tohex + Converts the given number, n, to a string @@ -1743,7 +1844,7 @@ if(s) code_to_be_done_on_match end - make_buffer + This function operates on a socket attempting to read data. @@ -1761,16 +1862,20 @@ if(s) code_to_be_done_on_match end + Nmap API + nmap NSE module + Nmap Scripting Engine (NSE)API NSE scripts have access to several Nmap facilities for writing flexible and elegant scripts. The API provides target host details such as port states and version detection results. It - also offers an interface to the Nsock library for efficient - network I/O. + also offers an interface to the Nsocklibrary + Nsock + for efficient network I/O. @@ -1779,8 +1884,9 @@ if(s) code_to_be_done_on_match end An effective Nmap scripting engine requires more than just a Lua interpreter. Users need easy access to the information Nmap has learned about the target hosts. This data is passed - as arguments to the NSE action method. The - arguments, host and + as arguments to the NSE action method. + action script variable + The arguments, host and port, are Lua tables which contain information on the target against which the script is executed. The following list describes each variable in the @@ -1791,7 +1897,7 @@ if(s) code_to_be_done_on_match end - host + This table is passed as a parameter to the rule and action @@ -1804,7 +1910,7 @@ if(s) code_to_be_done_on_match end - host.os + The os entry in the host table is @@ -1821,7 +1927,7 @@ if(s) code_to_be_done_on_match end - host.ip + Contains a string representation of the IP address of the target host. If the scan was run against a host name and the @@ -1833,7 +1939,7 @@ if(s) code_to_be_done_on_match end - host.name + Contains the reverse DNS entry of the scanned target host represented as a string. If the host has no reverse DNS entry, @@ -1843,7 +1949,7 @@ if(s) code_to_be_done_on_match end - host.targetname + Contains the name of the host as specified on the command line. If the target given on the command line contains a netmask or is an IP @@ -1853,7 +1959,7 @@ if(s) code_to_be_done_on_match end - host.directly_connected + A boolean value indicating whether or not the target host is directly connected (i.e. on the same network segment). @@ -1862,7 +1968,7 @@ if(s) code_to_be_done_on_match end - host.mac_addr + MAC address of the destination host (6-byte long binary string) or nil, if the host is not directly connected. @@ -1871,16 +1977,18 @@ if(s) code_to_be_done_on_match end - host.mac_addr_src + Our own MAC address, which was used to connect to the - host (either our network card's, or (with ) the spoofed address). + host (either our network card's, or (with ) + + the spoofed address). - host.interface + A string containing the interface name (dnet-style) through which packets to the host are sent. @@ -1889,7 +1997,7 @@ if(s) code_to_be_done_on_match end - host.bin_ip + The hosts IP as 4 byte long binary value. @@ -1897,7 +2005,7 @@ if(s) code_to_be_done_on_match end - host.bin_ip_src + Our hosts IP as 4 byte long binary value. @@ -1905,7 +2013,7 @@ if(s) code_to_be_done_on_match end - port + The port table is passed to the Lua script in the same @@ -1920,7 +2028,7 @@ if(s) code_to_be_done_on_match end - port.number + Contains the number of the currently scanned port. @@ -1931,7 +2039,7 @@ if(s) code_to_be_done_on_match end - port.protocol + Defines the protocol of the port. Valid values are @@ -1943,7 +2051,7 @@ if(s) code_to_be_done_on_match end - port.service + Contains a string representation of the service running on @@ -1958,7 +2066,7 @@ if(s) code_to_be_done_on_match end - port.version + This entry is a table which contains information @@ -2037,7 +2145,7 @@ if(s) code_to_be_done_on_match end - port.state + Contains information on the state of the port. @@ -2067,32 +2175,40 @@ if(s) code_to_be_done_on_match end - debuggingnmap.debugging + - Returns the debugging level as a non-negative integer. The + Returns the debugging level + debuggingin NSE + as a non-negative integer. The debugging level can be set with the + option (see ). - have_ssl + - Returns true if Nmap was compiled with SSL support, false + Returns true if Nmap was compiled with SSL support, + SSLin NSE + false otherwise. This can be used to avoid sending SSL probes when SSL is not available. - verbositynmap.verbosity + - Returns the verbosity level as a non-negative integer. The + Returns the verbosity level + verbosityin NSE + as a non-negative integer. The verbosity level can be set with the + option (see ). @@ -2100,9 +2216,9 @@ if(s) code_to_be_done_on_match end - fetchfile + data filesaccess to from NSE Allows access to Nmap's data files. fetchfile() searches for the specified file and returns a string containing @@ -2120,8 +2236,9 @@ nmap.fetchfile("nmap-rpc") - timing_levelnmap.timing_level + + timing templatesaccess to from NSE Returns the timing level as a non-negative integer. Possible return values vary from 0 to 5, corresponding to the six built-in Nmap @@ -2146,7 +2263,7 @@ nmap.fetchfile("nmap-rpc") - get_port_state + The get_port_state() call takes a @@ -2179,7 +2296,7 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) - set_port_state + The set_port_state() call takes a host table, @@ -2197,7 +2314,7 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) - set_port_version + NSE scripts are sometimes able to determine the @@ -2248,6 +2365,7 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) Various Utility Functions for Raw Packet Support + raw packetsin NSE NSE has support for sending raw ethernet frames and capturing packets. The following two functions may be handy in this context: @@ -2255,7 +2373,7 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) - nmap.clock_ms() + Returns a number representing the current time as milliseconds @@ -2265,7 +2383,7 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) - nmap.get_interface_link(interface_name) + For the provided dnet-style @@ -2301,6 +2419,7 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) Connect-style network I/O + sockets in NSE This part of the network API should be suitable for most classical network uses: Users create a socket, connect it to a remote address, send and receive data and close the socket again. @@ -2313,7 +2432,7 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) - nmap.new_socket() + The new_socket() Nmap call returns an @@ -2326,7 +2445,7 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) - connect + @@ -2384,6 +2503,7 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) In addition to these standard system error based messages are the following two NSE-specific errors: + SSLin NSE Sorry, you don't have OpenSSL. occurs if ssl is passed as third argument, but Nmap was compiled without OpenSSL support. @@ -2398,7 +2518,7 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) - send + The send method sends the data contained in the @@ -2439,7 +2559,7 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) - receive + The receive method does a non-blocking receive operation on @@ -2459,7 +2579,7 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) - receive_lines + Tries to receive at least n @@ -2485,7 +2605,7 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) - receive_bytes + Tries to receive at least n @@ -2508,7 +2628,7 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) - receive_buf + @@ -2583,7 +2703,7 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) - close + Closes an open connection. On success the returned value of @@ -2602,7 +2722,7 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) - get_info + This function returns information about the socket @@ -2621,7 +2741,7 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) - set_timeout + Sets the time, in milliseconds, after which input and @@ -2638,12 +2758,16 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) Raw packet network I/O + raw packetsin NSE For those cases where the connection oriented approach is too inflexible, NSE provides script developers with a more powerful option: raw packet network I/O. The greater flexibility comes, however, at the cost of a slightly more complex API. Receiving raw packets is - accomplished via a wrapper around Libpcap inside - the Nsock library. In order to keep the + accomplished via a wrapper around Libpcap + Libpcap + inside the Nsock library. + Nsock + In order to keep the capturing efficient it works in a three tiered approach: Opening a device for capturing, registering listeners to it and receiving packets. With each call to pcap_open() you have @@ -2669,7 +2793,7 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) - pcap_open + The pcap_open() call opens the socket for @@ -2685,7 +2809,7 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) - pcap_register + Starts the listening for incoming packages. The provided @@ -2701,7 +2825,7 @@ nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"}) - pcap_receive + Receives a captured packet. If successful, the return values are: @@ -2720,7 +2844,7 @@ error_message describes the occurred error. - pcap_close() + Closes the pcap device. @@ -2732,6 +2856,7 @@ error_message describes the occurred error. Receiving raw packets is a great feature, but it is also only the half job. Now for sending raw packets: To accomplish this NSE has access to a wrapper around the dnet library. + libdnet Currently NSE has the ability to send raw ethernet frames via the following API: @@ -2739,7 +2864,7 @@ error_message describes the occurred error. - new_dnet() + Creates and returns a new dnet_object, which can be used to @@ -2749,7 +2874,7 @@ error_message describes the occurred error. - ethernet_open + Opens the interface defined by the provided interface_name for sending ethernet frames @@ -2761,7 +2886,7 @@ error_message describes the occurred error. - ethernet_send + Sends the provided data as ethernet frame across the previously @@ -2775,7 +2900,7 @@ error_message describes the occurred error. - ethernet_close + Closes the interface. The only error which may be thrown is the same as for the ethernet_send() @@ -2790,11 +2915,14 @@ error_message describes the occurred error. Thread Mutexes + threads in NSE + mutexes in NSE Each thread made for a script (e.g. anonFTP.nse) will yield to other scripts whenever it makes a call on network objects (sending/receiving data). Some scripts need finer control over threads' execution. An example is the whois.nse script which queries whois + whois servers for each target. Because many concurrent queries often result in getting one's IP banned for abuse and a query may return additional information for targets other threads are running against, it is useful @@ -2817,7 +2945,7 @@ error_message describes the occurred error. - mutex + @@ -2887,6 +3015,7 @@ end Exception Handling + exceptions in NSE NSE provides an exception handling mechanism not present in the plain Lua language. The exception handling is tailored @@ -2956,7 +3085,8 @@ try(socket:send(result)) - The Registry<indexterm><primary>registry</primary></indexterm> + The Registry + registry (NSE) The registry is a normal Lua table. What is special about it is that it is visible by all scripts and it retains its state @@ -2972,7 +3102,9 @@ try(socket:send(result)) the registry which was left by the first script. This feature is particularly powerful in combination with the run level concept. A script with a higher run level can rely on entries - left behind for it by scripts with lower run levels. Remember + left behind for it by scripts with lower run levels. + run level of scripts + Remember however that the registry can be written by all scripts equally, so choose the keys for your entries wisely. The registry is stored in nmap.registry. The @@ -2985,16 +3117,20 @@ try(socket:send(result)) + Script Writing Tutorial + Nmap Scripting Engine (NSE)tutorial Suppose that you are convinced of the power of NSE. How do you go about writing your own script? Let's say that you want to extract information from an identification - server. Nmap used to have this functionality but it was removed + server. + auth service + Nmap used to have this functionality but it was removed because of inconsistencies in the code base. Fortunately, the protocol identd uses is pretty simple. Unfortunately, it is too complicated to be expressible in Nmap's version detection @@ -3034,6 +3170,7 @@ try(socket:send(result)) the ID is printed before the script's results in Nmap output. +id script variable id = "Service Owner" @@ -3042,6 +3179,7 @@ id = "Service Owner" The description field should contain a sentence or two describing what the script does. If anything about the script results might confuse or mislead users, and you can't eliminate the issue by improving the script or results text, it should be documented in the description string. +description script variable description = "Opens a connection to the scanned port, opens a connection to \ port 113, queries the owner of the service on the scanned port and prints it." @@ -3054,14 +3192,19 @@ port 113, queries the owner of the service on the scanned port and prints it." backslash (‘\’). They must also decide what categories the script belongs to. This script is a good example of a script which cannot be categorized clearly. It is - safe because we are not using the service + safe + safe script category + because we are not using the service for anything it was not intended for. On the other hand, it - is intrusive because we connect to a + is intrusive + intrusive script category + because we connect to a service on the target and therefore potentially give out information about us. To solve this dilemma we will place our script in two categories: +categories script variable categories = {"safe", "intrusive"} @@ -3093,6 +3236,7 @@ categories = {"safe", "intrusive"} the action. +portrule script variable portrule = function(host, port) local ident_port = { number=113, protocol="tcp" } @@ -3144,7 +3288,9 @@ end send() or receive() we can operate on the network socket. To avoid excessive error checking code we use NSE's - exception handling mechanism. We create a function which will + exception handling mechanism. + exceptions in NSE + We create a function which will be executed if an error occurs and call this function catch. Using this function we generate a try function. The try @@ -3169,6 +3315,7 @@ local localip, localport = client_service:get_info() + action NSE variable action = function(host, port) local owner = "" @@ -3210,10 +3357,13 @@ end + Version Detection Using NSE + Nmap Scripting Engine (NSE)sample scripts + version detectionusing NSE The version detection system built into Nmap was designed to @@ -3225,7 +3375,9 @@ end true nature. NSE has been integrated into Nmap's version detection framework to handle these cases. The scripts which extend the version scanner belong to the reserved category - version. This category cannot be run from + version. + version script category + This category cannot be run from the command line. It is only executed if the user has required a version scan. The following listing shows a simple script which demonstrates the use of the NSE version detection API. If either @@ -3242,17 +3394,17 @@ end description = "Demonstration of a version detection NSE script. It checks \ and reports the version of a remote web server. For real life purposes it is \ -better to use Nmap version detection (-sV)." -author = "Diman Todorov <diman.todorov@gmail.at>" -license = "Same as Nmap--See http://nmap.org/book/man-legal.html" +better to use Nmap version detection (-sV)."description script variable +author = "Diman Todorov <diman.todorov@gmail.at>"Todorov, Dimanauthor script variable +license = "Same as Nmap--See http://nmap.org/book/man-legal.html"license script variable -id = "HTTP version" +id = "HTTP version"id script variable -categories = {"version"} +categories = {"version"}categories script variableversion script category -runlevel = 1.0 +runlevel = 1.0runlevel script variable -portrule = function(host, port) +portrule = function(host, port)portrule script variable if (port.number == 80 or port.service == "http" ) and port.protocol == "tcp" @@ -3263,7 +3415,7 @@ portrule = function(host, port) end end -action = function(host, port) +action = function(host, port)action script variable local query = "GET / HTTP/2.1\r\n" query = query .. "Accept: */*\r\n" query = query .. "Accept-Language: en\r\n" @@ -3317,6 +3469,7 @@ end This is what the output of this script looks like: + example of $ ./nmap -sV localhost -p 80 @@ -3343,6 +3496,7 @@ Nmap finished: 1 IP address (1 host up) scanned in 9.317 seconds Finger-Test Script + Finger Results script The finger script (finger.nse) is a perfect example of how short typical NSE scripts are. @@ -3352,13 +3506,13 @@ Nmap finished: 1 IP address (1 host up) scanned in 9.317 seconds printed in Nmap's output. A detailed description of what the script actually does should go in the description field. -id="Finger Results" +id="Finger Results"id script variable -description="attempts to get a list of usernames via the finger service" +description="attempts to get a list of usernames via the finger service"description script variable -author = "Eddie Bell <ejlbell@gmail.com>" +author = "Eddie Bell <ejlbell@gmail.com>"Bell, Eddieauthor script variable -license = "Same as Nmap--See http://nmap.org/book/man-legal.html" +license = "Same as Nmap--See http://nmap.org/book/man-legal.html"license script variable The categories field is a table @@ -3384,9 +3538,9 @@ on a non-standard port) or still run against at least the port we expect it, should the version detection information not be available. -portrule = shortport.port_or_service(79, "finger") +portrule = shortport.port_or_service(79, "finger")portrule script variable -action = function(host, port) +action = function(host, port)action script variable local socket = nmap.new_socket() local results = "" local status = true @@ -3453,22 +3607,24 @@ end Service Owner Lookup via Identd + Service owner script + auth service showOwner.nse demonstrates the flexibility of the NSE, which is unmatched by other parts of Nmap. If the target is running an identd daemon it connects to it for each running service and tries to identify its owner. -id = "Service owner" +id = "Service owner"id script variable description = "Opens a connection to the scanned port, opens a connection to \ -port 113, queries the owner of the service on the scanned port and prints it." +port 113, queries the owner of the service on the scanned port and prints it."description script variable -author = "Diman Todorov <diman.todorov@gmail.com>" +author = "Diman Todorov <diman.todorov@gmail.com>"Todorov, Dimanauthor script variable -license = "Same as Nmap--See http://nmap.org/book/man-legal.html" +license = "Same as Nmap--See http://nmap.org/book/man-legal.html"license script variable -categories = {"default", "safe"} +categories = {"default", "safe"}categories script variabledefault script categorysafe script category @@ -3479,7 +3635,7 @@ returning a boolean. -portrule = function(host, port) +portrule = function(host, port) portrule script variable local auth_port = { number=113, protocol="tcp" } @@ -3544,9 +3700,11 @@ action = function(host, port) end + Implementation + Nmap Scripting Engine (NSE)implementation Now how does all this work? The following section describes some interesting aspects of the NSE. While the focus primarily lies on @@ -3623,18 +3781,26 @@ in the nmap namespace are also loaded. The search path for modules is set to the default one prepended by the nselib directory (which is searched in the locations Nmap searches for its data files and scripts). In this step the provided script arguments -also get stored inside the registry. +also get stored inside the registry. +registry (NSE) + The next phase of NSE initialization is loading the chosen scripts, which are the arguments provided to the - option or default, in + + + option or default, in case of a default script scan. The string version + version script category is appended, if version detection was enabled. The arguments afterwards are tried to be interpreted as script categories. This is done via a Lua C function in nse_init.cc called entry. - Inside script.db, for each category of a script, + Inside script.db, + script.db + script.db + for each category of a script, there is a call to Entry. If the category was chosen then the script is loaded. Every argument of that could not be interpreted as a category @@ -3656,12 +3822,19 @@ also get stored inside the registry. Matching of Scripts to Targets - After the initialization is finished the hostrules and - portrules are evaluated for each host in the current + After the initialization is finished the hostrules + hostrule script variable + and portrules + portrule script variable + are evaluated for each host in the current target group. At this check a list is built which contains the combinations of scripts and the hosts they will run against. It should be noted that the rules of all chosen scripts are -checked against all hosts and their open and open|filtered ports. +checked against all hosts and their open +open port state +and open|filtered +open|filtered port state +ports. Therefore it is advisable to leave the rules as simple as possible and to do all the computation inside the action, as a script will only be executed if it is run against a specific target. After the check those script-target combinations @@ -3680,8 +3853,9 @@ The mainloop function will work on each runlevel grouping of threads in order. Running Scripts - Nmap is able to perform NSE script scanning in parallel by - making use of Lua language features. In particular, + Nmap is able to perform NSE script scanning in parallel + parallelismin NSE + by making use of Lua language features. In particular, coroutines offer collaborative multi-threading so scripts can suspend themselves at defined points, and allow other coroutines to execute. Since network I/O, especially waiting for responses from @@ -3707,6 +3881,7 @@ The mainloop function will work on each runlevel grouping of threads in order. Adding C Modules to Nselib + Nmap Scripting Engine (NSE)C modules This section tries to give a short walk-through to adding nselib modules written in C (or C++) to Nmap's build system, since @@ -3718,7 +3893,9 @@ The mainloop function will work on each runlevel grouping of threads in order. functions they provide to Lua, which have to be of type lua_CFunction. Additionally they have to contain a function which is used to actually open the module. By convention these function names are luaopen_modulename. A good starting point for writing such modules is provided with - bit.c and pcre.c inside + bit.c + bite NSE module + and pcre.c inside the nselib/ subdirectory of Nmap's source tree, which are two C modules already provided by the nselib. C modules basically are shared libraries which get loaded at runtime by Lua. @@ -3747,7 +3924,9 @@ The mainloop function will work on each runlevel grouping of threads in order. itself. Linking with static libraries (e.g. libnbase) sometimes leads to problems with exporting symbols on some platforms (in our - case the x86_64-linux platform). To our knowledge no such + case the x86_64-linux platform). + x86_64 architecture + To our knowledge no such problems occur when linking against already existing shared libraries. @@ -3770,3 +3949,4 @@ The mainloop function will work on each runlevel grouping of threads in order. +