# Introduction

The Nmap Scripting Engine (NSE) is one of Nmap's most powerful and flexible features. It allows users to write (and share) simple scripts to automate a wide variety of networking tasks. Those scripts are then executed in parallel with the speed and efficiency you expect from Nmap. Users can rely on the growing and diverse set of scripts distributed with Nmap, or write their own to meet custom needs.

We designed NSE to be versatile, with the following tasks in mind:

Network discovery

  This is Nmap's bread and butter. Examples include looking up whois data based on the target domain, querying ARIN, RIPE, or APNIC for the target IP to determine ownership, performing identd lookups on open ports, SNMP queries, and listing available NFS/SMB/RPC shares and services.

More sophisticated version detection

  The Nmap version detection system () is able to recognize thousands of different services through its probe and regular expression based matching system, but it cannot recognize everything. For example, identifying the Skype v2 service requires two independent probes, which version detection isn't flexible enough to handle. Nmap could also recognize more SNMP services if it tried a few hundred different community names by brute force. Neither of these tasks are well suited to traditional Nmap version detection, but both are easily accomplished with NSE. For these reasons, version detection now calls NSE by default to handle some tricky services. This is described in .

Vulnerability detection

  When a new vulnerability is discovered, you often want to scan your networks quickly to identify vulnerable systems before the bad guys do. While Nmap isn't a comprehensive vulnerability scanner, NSE is powerful enough to handle even demanding vulnerability checks. Many vulnerability detection scripts have already been written and we plan to distribute more as they are written.

Backdoor detection

  Many attackers and some automated worms leave backdoors to enable later reentry. Some of these can be detected by Nmap's regular expression based version detection. For example, within hours of the MyDoom worm hitting the Internet, Jay Moran
  posted an Nmap version detection probe and signature so that others could quickly scan their networks. For more complex worms and backdoors, NSE is needed for reliable detection.

Vulnerability exploitation

  As a general scripting language, NSE can even be used to exploit vulnerabilities rather than just find them. The capability to add custom exploit scripts may be valuable for some people (particularly penetration testers),
  though we aren't planning to turn Nmap into an exploitation framework like Metasploit.

The listed items were our initial goals, but we expect that Nmap users will come up with inventive uses for NSE.

The Nmap Scripting Engine is a new Nmap feature which already works well, but is under active development. To provide the latest NSE news and updates, this chapter has been updated and posted for free online at .

Scripts are written in the embedded Lua programming language.
The language itself is well documented in the books Programming in Lua, Second Edition and Lua 5.1 Reference Manual. Programming in Lua, Second Edition and Lua 5.1 Reference Manual.

The reference manual is also freely available online, as is the first edition of Programming in Lua. Given the availability of these excellent general Lua programming references, this document only covers aspects and extensions specific to Nmap's scripting engine.

NSE is activated with the `-sC` option (or `--script` if you wish to specify a custom set of scripts) and results are integrated into Nmap normal
and XML output.
Two types of scripts are supported: service and host scripts. Service scripts relate to a certain open port (service) on the target host, and any results they produce are included next to that port in the Nmap output port table. Host scripts, on the other hand, run no more than once against each target IP and produce results below the port table. shows a typical script scan. Examples of service scripts producing output are: `Stealth SSH version`, which tricks some SSH servers into divulging version information without logging the attempt as they normally would; `Service Owner`, which connects to open ports, then performs a reverse-identd query to determine what username each is running under; and `HTML Title`, which simply grabs the title of the root path of any web servers found. A sample host script is `RIPE Query`, which looks up and reports target IP ownership information.

## Typical NSE output

    $ nmap -sC localhost -p 22,23,80,113

    Starting Nmap ( http://nmap.org )
    Interesting ports on localhost (127.0.0.1):
    PORT    STATE  SERVICE
    22/tcp  open   ssh
    |_ Stealth SSH version: SSH-1.99-OpenSSH_4.2
    |_ SSH protocol version 1: Server supports SSHv1
    23/tcp  closed telnet
    80/tcp  open   http
    |_ HTML title:Test Page for Apache Installation
    113/tcp closed auth

    Host script results:
    |_ RIPE Query: IP belongs to:          Internet Assigned Numbers Authority

    Nmap done: 1 IP address (1 host up) scanned in 0.91 seconds

# Usage and Examples

While NSE has a complex implementation for efficiency, it is strikingly easy to use. Simply specify `-sC`
to enable the most common scripts. Or specify the `--script`
option to choose your own scripts to execute by providing categories, script file names, or the name of directories full of scripts you wish to execute. You can customize some scripts by providing arguments to them via the `--script-args`
option. The two remaining options, `--script-trace`
and `--script-updatedb`,
are generally only used for script debugging and development.

## Script Categories

NSE scripts define a list of categories they belong to. Currently defined categories are `safe`, `intrusive`, `malware`, `version`, `discovery`, `vuln`, `auth`, `external` and `default`. Category names are not case sensitive. The following list describes each category.

`safe`

  Scripts which weren't designed to crash services, use large amounts of network bandwidth or other resources, or exploit security holes. These are less likely to offend remote sysadmins. Of course (as with all other Nmap features) we cannot guarantee that they won't ever cause adverse reactions. Most of these perform general network discovery. Examples are echoTest (sends a string to the UDP echo service) and showHTMLTitle (grabs the title from a web page).

`intrusive`

  These are scripts that cannot be classified in the `safe` category because the risks are too high that they will crash the target system, use up significant resources on the target host (such as bandwidth or CPU time), or otherwise be perceived as malicious by the target's system administrators.

`malware`

  These scripts test whether the target platform is infected by malware or backdoors.

`version`

  The scripts in this category are an extension to the version detection option and cannot be selected explicitly. They are selected to run only if version detection (`-sV`) was requested. Their output cannot be distinguished from version detection output and they do not produce service or host script results.

`discovery`

  These scripts try to actively learn more about the network by querying public registries, SNMP-enabled devices, directory services, and the like.

`vuln`

  These scripts check for specific known vulnerabilities and generally only report results if they are found.

`auth`

  These scripts try to determine authentication credentials on the target system, often through a brute-force attack.

`external`

  Scripts in this category may send data to a third-party database or other network resource. An example of this is `whois.nse`, which makes a connection to a whois
  server to learn about the address of the target. There is always the possibility that the operators of the third-party database will record anything you send to them, which in many cases will include your IP address and the address of the target. Most scripts involve traffic strictly between the scanning computer and the client; any that do not are placed in this category.

`default`

  These scripts are the default set and are run when using `-sC`, `-A` or `--script` without any arguments. This category can also be specified explicitly like any other using `--script=default`.

## Command-line Arguments

These are the five command line arguments specific to script-scanning:

`-sC`

  Performs a script scan using the default set of scripts. It is equivalent to `--script=default`. Some of the scripts in this category are considered intrusive and should not be run against a target network without permission.

`--script script-categories|directory|filename|all`

  Runs a script scan (like `-sC`) using the comma-separated list of script categories, individual scripts, or directories containing scripts, rather than the default set. Nmap first tries to interpret the arguments as categories, then (if that fails) as files or directories. A script or directory of scripts may be specified as an absolute or relative path. Absolute paths are used as supplied. Relative paths are searched for in the following places until found:
  `--datadir/`; `$NMAPDIR/`;
  `~/.nmap/` (not searched on Windows);
  NMAPDATADIR/ or
  `./`. A `scripts/` subdirectory is also tried in each of these.

  If a directory is specified and found, Nmap loads all NSE scripts (any filenames ending with `.nse`) from that directory. Filenames without the `nse` extension are ignored. Nmap does not search recursively into subdirectories to find scripts. If individual file names are specified, the file extension does not have to be `nse`.

  Nmap scripts are stored in a `scripts` subdirectory of the Nmap data directory by default (see ). For efficiency, scripts are indexed in a database stored in `scripts/script.db`.
  which lists the category or categories in which each script belongs. Give the argument `all` to execute all scripts in the Nmap script database.

  Malicious scripts are not run in a sandbox and thus could damage your system or invade your privacy. Never run scripts from third parties unless you trust the authors or have carefully audited the scripts yourself.

`--script-args`

  provides arguments to the scripts. See for a detailed explanation.

`--script-trace`

  This option is similar to `--packet-trace`, but works at the application level rather than packet by packet. If this option is specified, all incoming and outgoing communication performed by scripts is printed. The displayed information includes the communication protocol, source and target addresses, and the transmitted data. If more than 5% of transmitted data is unprintable, hex dumps are given instead.

`--script-updatedb`

  This option updates the script database found in `scripts/script.db` which is used by Nmap to determine the available default scripts and categories. It is only necessary to update the database if you have added or removed NSE scripts from the default `scripts` directory or if you have changed the categories of any script. This option is generally used by itself: `nmap --script-updatedb`.

Some of the Nmap options have effects on script scans. The most prominent of these is `-sV`.
A version scan executes the scripts in the `version` category.
The scripts in this category are slightly different than other scripts. Their output blends in with the version scan and they do not produce any script scan output.

Another option which has effect on the scripting engine is `-A`.
The advanced/aggressive mode of Nmap implies the option `-sC`.

## Arguments to Scripts

You can pass arguments to NSE scripts via the `--script-args` option. The script-arguments generally are name-value pairs, which are provided to the script as a Lua table called `args` inside the `nmap.registry` with the names as keys for the corresponding values. The values can either be strings or tables. Subtables can be used to pass arguments to scripts with a finer granularity (e.g. pass different usernames for different scripts). A typical nmap invocation with script arguments may look like:

`$ nmap -sC --script-args user=foo,pass=bar,anonFTP={pass=ftp@foobar.com}`

which would result in the Lua table:

    {user="foo",pass="bar",anonFTP={pass="nobody@foobar.com"}}

You could therefore access the username (`"foo"`) inside your script as `local username= nmap.registry.args.user`. As a general rule the subtables used to override options for scripts should be named as the script's `id`, otherwise scripts won't know where to retrieve their arguments.

## Usage Examples

A simple script scan using the default set of scripts

`$ nmap -sC example.com`

Tracing a specific script.

`$ nmap --script=./showSSHVersion.nse --script-trace example.com`

All scripts in a subdirectory named `mycustomscripts` in addition to all of Nmap's included scripts which are in the `safe` category.

`$ nmap --script=mycustomscripts,safe example.com`

# Script Format

NSE scripts consist of six descriptive fields along with either a port or host rule defining when the script should be executed and an action block containing the actual script instructions. Values can be assigned to these fields just as you would assign any other Lua variables. Their names must be lowercase as shown here.

## `id` Field

The script's `id` field is displayed in the Nmap output table if the script produces any output. It should be unique so users can identify exactly which script file produced a message. IDs should be kept short to conserve space in Nmap output, while still being meaningful enough for users to recognize. Some good examples are `RIPE query`, `HTML title`, and `Kibuv worm`.

## `description` Field

The `description` field describes what the script is testing for and any critical notes the user must be aware of. A good example is this description from a user-contributed recursive DNS script: Checks whether a nameserver on UDP port 53 allows queries for third party names. It is expected that recursion will be enabled on your own internal nameserver.

## `categories` Field

The `categories` field defines one or more categories to which a script belongs (see ). The categories are case-insensitive and may be specified in any order. They are listed in an array-like Lua table as in this example:

    categories = {"default", "discovery", "safe"}

## `author` Field

The `author` field contains the script authors name and contact information. If you are worried about spam, you might want to omit or obscure your email address, or give your home page URL instead. This optional field is not used by NSE, but is important for giving script authors due credit or blame.

## `license` Field

Nmap is a community project and we welcome all sorts of code contributions, including NSE scripts. So if you write a valuable script, don't keep it to yourself! The `license` field helps ensure that we have legal permission to distribute all the scripts which come with Nmap. All of those scripts currently use the standard Nmap license (described in ). They include the following line:

    license = "Same as Nmap--See http://nmap.org/book/man-legal.html"

The Nmap license is similar to the GNU GPL. Script authors may use a BSD-style license (no advertising clause) instead if they prefer that.

## `runlevel` Field

This optional field determines script execution order. When this section is absent the run level defaults to 1.0. A script with the run level 1.0 is run before any scripts with `runlevel` set to `2.5`, which in turn runs before any scripts with `runlevel 2.55`. Scripts with the same run level are run concurrently. One application of run levels is allowing scripts to depend on each other. If `script A` relies on some information gathered by `script B`, give `B` a lower run level than `A`. `Script B` can store information in the NSE registry for `A` to retrieve later. For information on the NSE registry see to .

## Port and Host Rules

Nmap uses the script rules to determine whether a script should be run against a target. A script contains either a port rule, which governs which ports of a target the scripts may run against, or a host rule, which specifies that the script should be run only once against a target IP and only if certain conditions are met. A rule is a Lua function that returns either `true` or `false`. The script action is only performed if the rule evaluates to `true`. The host rule accepts a host table as an argument and may test, for example, the IP address or hostname of the target. A port rule accepts both host and port tables as arguments for any TCP or UDP port in either the `open`
, `open|filtered`
, or `unfiltered`
port states. Port rules generally test factors such as the port number, port state, or listening service name in deciding whether to run against a port. Example rules are shown in .

## Action

The action is the heart of an NSE script. It contains all of the instructions to be executed when the script's port or host rule triggers. It is a Lua function which accepts the same arguments as the rule and can return either `nil` or a string. If a string is returned by a service script, the string and script ID are printed in the Nmap port table output. A string returned by a host script is printed below the port table. No output is produced if the script returns `nil`. For an example of an NSE action refer to .

# Script Language

The core of the Nmap Scripting Engine is an embeddable Lua interpreter. Lua is a lightweight language designed for extensibility. It offers a powerful and well documented API for interfacing with other software such as Nmap.

The second part of the Nmap Scripting Engine is the NSE Library, which connects Lua and Nmap. This layer handles issues such as initialization of the Lua interpreter, scheduling of parallel script execution, script retrieval and more. It is also the heart of the NSE network I/O framework and the exception handling mechanism. It also includes utility libraries to make scripts more powerful and convenient. The utility library modules and extensions are described in .

## Lua Base Language

The Nmap scripting language is an embedded Lua interpreter which was extended with libraries for interfacing with Nmap. The Nmap API is in the Lua namespace `nmap`. This means that all calls to resources provided by Nmap have an `nmap` prefix.
`nmap.new_socket()`, for example, returns a new socket wrapper object. The Nmap library layer also takes care of initializing the Lua context, scheduling parallel scripts and collecting the output produced by completed scripts.

During the planning stages, we considered several programming languages as the bases for Nmap scripting. One option was to implement a completely new programming language. The criteria imposed on the options were strict, NSE needed to be easy to use, small in size, compatible with the Nmap license, scalable, fast and parallelizable. There have been several efforts to design a security auditing language from scratch which have resulted in well-known awkward solutions. It was clear from the beginning that we would not go down this road. For a while the Guile Scheme interpreter was considered but the preference drifted towards Elk in favor of its more liberal license. But parallelizing Elk scripts would have been difficult. In addition, the subset of Nmap users familiar with functional programming is regarded too small to consider Scheme as an option. Larger interpreters like Perl, Python or Ruby are well-known and loved, but are difficult to embed efficiently. In the end, Lua exceeded in all criteria for NSE. It is small, distributed under the MIT license, has coroutines for efficient parallel script execution, was designed with embeddability in mind, has excellent documentation, and is actively developed by a large and committed community. Lua is even embedded in popular applications including Wireshark and Second Life.

# Lua Extensions

In addition to the significant built-in capabilities of Lua, we have written or integrated several extensions to make NSE scripts more powerful and convenient to write. These modules are compiled and installed along with Nmap. They have their own directory, `nselib`, which is installed in the configured datadir. Scripts need only `require` the default modules in order to use them.

&nse-modules;

# NSE Scripts

&nse-scripts;

# Nmap API

NSE scripts have access to several Nmap facilities for writing flexible and elegant scripts. The API provides target host details such as port states and version detection results. It also offers an interface to the Nsock
library for efficient network I/O.

## Information Passed to a Script

An effective Nmap scripting engine requires more than just a Lua interpreter. Users need easy access to the information Nmap has learned about the target hosts. This data is passed as arguments to the NSE `action` method.
The arguments, `host` and `port`, are Lua tables which contain information on the target against which the script is executed. If a script matched a hostrule, it gets only the `host` table, and if it matched a portrule it gets both `host` and `port`. The following list describes each variable in these two tables.

`host`

  This table is passed as a parameter to the rule and action functions. It contains information on the operating system run by the host (if the `-O` switch was supplied), the IP address and the host name of the scanned target.

`host.os`

  The `os` entry in the host table is an array of strings. The strings (as many as eight) are the names of the operating systems the target is possibly running. Strings are only entered in this array if the target machine is a perfect match for one or more OS database entries. If Nmap was run without the `-O` option, then `host.os` is `nil`.

`host.ip`

  Contains a string representation of the IP address of the target host. If the scan was run against a host name and the reverse DNS query returned more than one IP addresses then the same IP address is used as the one chosen for the scan.

`host.name`

  Contains the reverse DNS entry of the scanned target host represented as a string. If the host has no reverse DNS entry, the value of the field is an empty string.

`host.targetname`

  Contains the name of the host as specified on the command line. If the target given on the command line contains a netmask or is an IP address the value of the field is `nil`.

`host.directly_connected`

  A Boolean value indicating whether or not the target host is directly connected (i.e. on the same network segment).

`host.mac_addr`

  MAC address
  of the destination host (6-byte long binary string) or `nil`, if the host is not directly connected.

`host.mac_addr_src`

  Our own MAC address, which was used to connect to the host (either our network card's, or (with `--spoof-mac`)
  the spoofed address).

`host.interface`

  A string containing the interface name (dnet-style)
  through which packets to the host are sent.

`host.bin_ip`

  The target host's IPv4 address as 4 byte long binary value.

`host.bin_ip_src`

  Our host's (running Nmap) source IPv4 address as 4 byte long binary value.

`port`

  The port table is passed to the Lua script in the same fashion as the host table. It contains information about the port against which the script is running. If the script is run according to a host rule, then no port table is passed to the script. Port states on the target can still be requested from Nmap using the `nmap.get_port_state()` call.

`port.number`

  Contains the number of the currently scanned port.

`port.protocol`

  Defines the protocol of the port. Valid values are `"tcp"` and `"udp"`.

`port.service`

  Contains a string representation of the service running on `port.number` as detected by the Nmap service detection. If the `port.version` field is `nil` then Nmap has guessed the service based only on the port number. Otherwise this field is equal to `port.version.name`.

`port.version`

  This entry is a table which contains information retrieved by the Nmap version scanning engine. Some of the values (like service name, service type confidence, RPC related values) may be retrieved by Nmap even if a version scan was not required. Values which were not retrieved default to `nil`. The meaning of each value is given in the following table:

  ### `port.version` values

  Name | Description
  --- | ---
  `name` | Contains the service name Nmap will use for the port.
  `name_confidence` | Evaluates how confident the version detection is about the accuracy of `name`, from 1 (least confident) to 10.
  `product`, `version`, `extrainfo`, `hostname`, `ostype`, `devicetype` | These five variables are described in .
  `service_tunnel` | Contains the string `"none"` or `"ssl"` based on whether or not Nmap used SSL tunneling to detect the service.
  `service_fp` | The service fingerprint, if any, is provided in this value. This is described in .
  `rpc_status` | Contains a string value of `good_prog` if we were able to determine the program number of an RPC service listening on the port, `unknown` if the port appears to be RPC but we couldn't determine the program number, `not_rpc` if the port doesn't appear be RPC, or `untested` if we haven't checked for RPC status.
  `rpc_program`, `rpc_lowver`, `rpc_highver` | The detected RPC program number and the range of version numbers supported by that program. These will be `nil` if `rpc_status` is anything other than `good_prog`.

`port.state`

  Contains information on the state of the port. Service scripts are only run against ports in the `open` or `open|filtered` states, so `port.state` generally contains one of those values. Other values might appear if the port table is a result of the `get_port_state` function. You can adjust the port state using the `nmap.set_port_state()` call. This is normally done when an `open|filtered` port is determined to be `open`.

## Network I/O API

To allow for efficient and parallelizable network I/O, NSE provides an interface to Nsock, the Nmap socket library. The smart callback mechanism Nsock uses is fully transparent to NSE scripts. The main benefit of NSE's sockets is that they never block on I/O operations, allowing many scripts to be run in parallel. The I/O parallelism is fully transparent to authors of NSE scripts. In NSE you can either program as if you were using a single non-blocking socket or you can program as if your connection is blocking. Seemingly blocking I/O calls still return once a specified timeout has been exceeded. Two flavors of Network I/O are supported: connect-style and raw packet.

### Connect-style network I/O

This part of the network API should be suitable for most classical network uses: Users create a socket, connect it to a remote address, send and receive data and close the socket again. Everything up to the Transport layer (which is either TCP, UDP or SSL) is handled by the library.

An NSE socket is created by calling `nmap.new_socket()`, which returns a socket object. The socket object supports the usual `connect`, `send`, `receive`, and `close` methods. Additionally the functions `receive_bytes`, `receive_lines`, and `receive_buf` allow greater control of the receiving of data. shows the use of connect-style network operations. The `try` function is for error handling; see .

#### Connect-style I/O

    require("nmap")

    local socket = nmap.new_socket()
    socket:set_timeout(1000)
    try = nmap.new_try(function() socket:close() end)
    try(socket:connect(host.ip, port.number))
    try(socket:send("login"))
    response = try(socket:receive())
    socket:close()

### Raw packet network I/O

For those cases where the connection-oriented approach is too inflexible, NSE provides script developers with a more powerful option: raw packet network I/O. The greater flexibility comes, however, at the cost of a slightly more complex API. Receiving raw packets is accomplished via a wrapper around Libpcap
inside the Nsock library.

For efficiency, the interface for raw packet capturing works in three steps. First, a capture device is opened. Second, listeners are registered with the interface. Third, packets are received.

A handle for raw socket reads is created from an ordinary socket object using the `pcap_open()` method. This method takes a callback function, which computes a so-called packet hash from a packet along with its headers. This hash can return any binary string, which is later compared to the strings registered with the `pcap_register()` function. Normally the packet hash callback will extract some portion of the packet, such as its source address.

The pcap reader is instructed to listen for certain packets using the `pcap_register()` function. The function takes a binary string which is compared against the hash value of every packet received. Those packets whose hashes match any registered strings will be returned by the `pcap_receive()` method. Register the empty string to receive all packets.

A script then receives packets for which a listener has been registered by calling the `pcap_receive()` method. The method blocks until a packet is received or a timeout occurs.

The more general the packet hash computing function is kept, the more scripts may receive the packet and proceed with their execution. To use the packet capturing inside your script you have to create a socket with `nmap.newsocket()` and later close the socket with `socket_object:close()`—just like with the connection-based network I/O.

Receiving raw packets is a great feature, but it is also only half the job. Now for sending raw packets: To accomplish this NSE has access to a wrapper around the `dnet` library.

Unlike raw socket reads, raw packet writes are not through a standard socket object. Instead, the function `nmap.new_dnet()` creates a dnet object with ethernet sending methods. Open an interface with the `ethernet_open()` method. Send raw ethernet frames with `ethernet_send()`. Close the ethernet handle with `ethernet_close` when you're done.

## Thread Mutexes

Each thread made for a script (e.g. anonFTP.nse) will yield to other scripts whenever it makes a call on network objects (sending/receiving data). Some scripts need finer control over threads' execution. An example is the `whois.nse` script which queries whois
servers for each target. Because many concurrent queries often result in getting one's IP banned for abuse and a query may return additional information for targets other threads are running against, it is useful to have other threads pause while one thread is conducting a query.

To solve this problem, there is an nmap function, `mutex`, that provides a mutex usable by scripts. The mutex allows for only one thread to be working on an object. Threads waiting to work on this object are put in the waiting queue until they can get a "lock" on the mutex. A solution for the `whois.nse` problem above is to have each thread block on a mutex for script's ID field , thus ensuring only one thread is working so its results can be shared with other scripts which may not need to run and so queries to the whois servers are staggered.

`mutex = nmap.mutex(object)`

  Returns a function that works on a mutex for the object passed. This object can be any Lua data type except `nil`, `booleans`, and `numbers`. The returned function allows you to lock, try to lock, and release the mutex. Its first and only parameter must be one of the following:

  - `"lock"`—Make a blocking lock on the mutex. If the mutex is busy (another thread has a lock on it), then the thread will yield and wait. The function returns with the mutex locked.

  - `"trylock"`—Makes a non-blocking lock on the mutex. If the mutex is busy then it immediately returns with a return value of `false`. Otherwise the mutex locks the mutex and returns `true`.

  - `"done"`—Releases the mutex and allows another thread to lock it. If the thread does not have a lock on the mutex, an error will be raised.

  - `"running"`—Returns the thread locked on the mutex or `nil` if the mutex is not locked. This should only be used for debugging as it interferes with finished threads from being collected.

### Mutex manipulation

    id = "My Script's Unique ID";

    local mutex = nmap.mutex(id);
    function action(host, port)
      mutex "lock";
      -- do stuff
      mutex "done";
      return script_output;
    end

## Exception Handling

NSE provides an exception handling mechanism not present in the plain Lua language. The exception handling is tailored specifically for network I/O operations. The mechanism follows a functional programming paradigm rather than an object oriented programming paradigm. To create an exception handler the `nmap.new_try()` API method is used. This method returns a function, which takes a variable number of arguments, assumed to be the return values of another function. If an exception is detected in the return values (the first return value is false), then the script execution is aborted and no output is produced. Optionally you can pass a function to the `new_try()` method which will be called if an exception is caught. In this function you can perform required clean up operations.

shows cleanup exception handling at work. A new function named `catch` is defined to simply close the newly created socket in case of an error. It is then used to protect connection and communication attempts on that socket. If no catch function is specified, execution of the script aborts without further ado—open sockets will remain open. If the verbosity level is at least one or if the scan is performed in debugging mode a description of the uncaught error condition is printed on standard output. Note that it is currently not easily possible to group several statements in one try block. It is also important to remember that if the socket is not closed it will occupy memory until the next run of Lua's garbage collector.

### Exception handling example

    local result, socket, try, catch

    result = ""
    socket = nmap.new_socket()
    catch = function()
    socket:close()
    end
    try = nmap.new_try(catch)

    try(socket:connect(host.ip, port.number))
    result = try(socket:receive_lines(1))
    try(socket:send(result))

Writing a function which is treated properly by the try/catch mechanism is straightforward. The function should return multiple values. The first value should be a Boolean which is `true` upon successful completion of the function and `false` otherwise. If the function completed successfully the try construct consumes the indicator value and returns the remaining values. If the function failed then the second returned value must be a string describing the error condition. Note that if the value is not `nil` or `false` it is treated as `true` so you can return your value in the normal case and return `nil, error description` if an error occurs.

## The Registry

The registry is a normal Lua table. What is special about it is that it is visible by all scripts and it retains its state between script executions. Nmap does not scan every host specified on the command line at the same time, it puts them in smaller groups and these groups are scanned in parallel. The registry is rebuilt for every group, so information stored there is only deleted after NSE finishes processing the current target group. This implies of course that the registry is transient—it is not stored between Nmap executions. Every script can read the registry and write to it. If a script is running after another script, it can read some information in the registry which was left by the first script. This feature is particularly powerful in combination with the run level concept. A script with a higher run level can rely on entries left behind for it by scripts with lower run levels.
Remember however that the registry can be written by all scripts equally, so choose the keys for your entries wisely. The registry is stored in `nmap.registry`. The behavior of the registry allows caching of already calculated data. The cache can be seen by all scripts until the registry is rebuilt with the next target group.

# Script Writing Tutorial

Suppose that you are convinced of the power of NSE. How do you go about writing your own script? Let's say that you want to extract information from an identification server.
Nmap used to have this functionality but it was removed because of inconsistencies in the code base. Fortunately, the protocol identd uses is pretty simple. Unfortunately, it is too complicated to be expressible in Nmap's version detection language. Let's look at how the identification protocol works. First you connect to the identification server. Next you send a query of the form `port-on-server, port-on-client` terminated with a new line character. The server should then respond with a string of the form `port-on-server, port-on-client:response-type:address-information`. In case of an error the address information is omitted. This description is sufficient for our purposes, for more details refer to RFC 1413. The protocol cannot be modeled in Nmap's version detection language for two reasons. The first is that you need to know both the local and the remote port of a connection. Version detection does not provide this data. The second, more severe obstacle, is that you need two open connections to the target—one to the identification server and one to the port you want to query. Both obstacles are easily overcome with NSE.

The anatomy of a script is described in . In this section we will show how the described structure is utilized.

## The Head

The head of the script is essentially its meta information. This includes the fields: `id`, `description`, `categories`, `runlevel`, `author` and `license`. We are not going to change the run level, or worry about the author and license fields for now. The `id` of a script should uniquely identify it. If it is absent, the path to the script will be used as an id. We recommend to choose an id which concisely identifies the purpose of the script, since the ID is printed before the script's results in Nmap output.

    id = "Service Owner"

The description field should contain a sentence or two describing what the script does. If anything about the script results might confuse or mislead users, and you can't eliminate the issue by improving the script or results text, it should be documented in the `description` string.

    description = "Opens a connection to the scanned port, opens a connection to \
    port 113, queries the owner of the service on the scanned port and prints it."

Users must tell the Lua interpreter that the string continues on the following line by ending the line with a backslash (‘`\`’). They must also decide what categories the script belongs to. This script is a good example of a script which cannot be categorized clearly. It is `safe`
because we are not using the service for anything it was not intended for. On the other hand, it is `intrusive`
because we connect to a service on the target and therefore potentially give out information about us. To solve this dilemma we will place our script in two categories:

    categories = {"safe", "intrusive"}

## The Rule

The rule section is a Lua method which decides when the script's action should be performed and when it should be skipped. Usually this decision is based on the host and port information passed to the rule function. In the case of the identification script it is slightly more complicated than that. To decide whether to run the identification script on a given port we need to know if there is an identification server running on the target machine. Or more formally: the script should be run if (and only if) the currently scanned TCP port is open and TCP port 113 is also open. For now we will rely on the fact that identification servers listen on TCP port 113. Unfortunately NSE only gives us information about the currently scanned port.

To find out if port 113 is open we are going to use the `nmap.get_port_state()` method. If the identd port was not scanned, the `get_port_state` function returns `nil`. So we need to make sure that the table is not `nil`. We also check if both ports are in the `open` state. If this is the case, the action is executed, otherwise we skip the action.

    portrule = function(host, port)
      local ident_port = { number=113, protocol="tcp" }
      local identd = nmap.get_port_state(host, ident_port)

      if identd ~= nil and identd.state == "open" and port.state == "open" then
        return true
      else
        return false
      end
    end

This rule is almost correct, but still slightly buggy. Can you find the bug? It is a pretty subtle one. The problem is that this script fires on any kind of open port, TCP or UDP. The `connect()` method on the other hand assumes a TCP protocol unless it is explicitly told to use another protocol. Since the identification service is only defined for TCP connections, we need to narrow down the range of ports which fire our script. Our new rule only runs the script if the port is open, we are looking at a TCP port, and TCP port 113 is open. Writing the new and improved port rule is left as an exercise to the reader (or peek at the script in the latest Nmap distribution).

## The Mechanism

At last we implement the actual functionality. The script will first connect to the port on which we expect to find the identification server, then it will connect to the port we want information about. Afterward we construct a query string and parse the response. If we received a satisfactory response, we return the retrieved information.

First we need to create two socket objects. These objects represent the sockets we are going to use. By using object methods like `open()`, `close()`, `send()` or `receive()` we can operate on the network socket. To avoid excessive error checking code we use NSE's exception handling mechanism.
We create a function which will be executed if an error occurs and call this function `catch`. Using this function we generate a `try` function. The `try` function will call the `catch` function whenever there is an error condition in the tried block.

    action = function(host, port)
    local owner = ""

    local client_ident = nmap.new_socket()
    local client_service = nmap.new_socket()

    local catch = function()
      client_ident:close()
      client_service:close()
    end

    local try = nmap.new_try(catch)

    try(client_ident:connect(host.ip, 113))
    try(client_service:connect(host.ip, port.number))

    local localip, localport, remoteip,
    remoteport = client_service:get_info()

    local request = port.number .. ", " .. localport .. "\n"

    try(client_ident:send(request))

    owner = try(client_ident:receive_lines(1))

    if string.match(owner, "ERROR") then
      owner = nil
      --  owner = "Service owner could not be determined: " .. owner
    else
     owner = string.match(owner, "USERID : .+ : (.+)\n", 1)
    end

    try(client_ident:close())
    try(client_service:close())

    return owner
    end

Note that because we know that the remote port is stored in `port.number`, we could have ignored the last two return values of `client_service:get_info()` like this:

    local localip, localport  = client_service:get_info()

In this example we avoided telling the user if the service responded with an error. Instead we commented that line out and assigned `nil` to the owner variable. NSE scripts generally only return messages when they succeed.

# Script Documentation Writing

Scripts are used by more than just the author, so it is useful to have documentation for users and other developers to browse through when looking for scripts to run against a host. For this reason, NSE provides a system that will produce complete documentation for its library of scripts.

The documentation is put in comments around your code. The trigger for a documentation comment is `---`. Only the first line in the block of comments should use the prefix trigger; subsequent lines should use Lua's standard single line comment prefix: `--`.

The first paragraphs are used as a description for the code following the comments. The first sentence of this description is used as a summary (for a brief description in the index). Optional tags can follow the description using the format `@tag ...`. See for more information.

The first comment block not followed by a function or table definition will be used for a file's comment. This comment will serve for documenting general information concerning the script. The description of this tag should explain the script's purpose and what it does. The file comment should also contain an output tag showing expected output for the script and an args tag for any arguments the script expects.

## Documentation Format

Documentation for a script consists of three basic parts: a file comment, a function or table comment, and standard NSE script fields (e.g. `description` or `author`). A file comment is the first documentation comment in a script and is not followed by a function definition or table. The following example demonstrates a file comment:

    --- Checks if an FTP server allows anonymous logins.
    -- @output
    -- |_ Anonymous FTP: Anonymous login allowed

The standard NSE fields are used to gather other information about a script such as the author or its categories:

    author = "Eddie Bell"
    license = "Same as Nmap--See http://nmap.org/book/man-legal.html"
    id = "Anonymous FTP"
    categories = {"default", "intrusive"}

These fields will be present in the final documentation. The description field is used for the User Summary if a description in the file comment or the file comment itself is absent. It is best if the description field is succinct while the file comment has an extensive description of the script.

Finally, a documentation comment may precede a function or table definition:

    --- Convert two bytes into a 16bit number.
    --@param data String of data.
    --@param idx Index in the string (first of two consecutive bytes).
    --@return 16 bit number represented by the two bytes.
    function bto16(data, idx)
      local b1 = string.byte(data, idx)
      local b2 = string.byte(data, idx+1)
      -- (b2 & 0xff) | ((b1 & 0xff) << 8)
      return bit.bor(bit.band(b2, 255), bit.lshift(bit.band(b1, 255), 8))
    end

You may also choose to document the action, portrule, or hostrule functions. They are separated from other functions in the final documentation for improved visibility.

## Tags

The following tags can be used:

`@param`

  Describe function parameters. The first alphanumeric word must be the parameter being documented.

`@see`

  Refer to other functions or tables.

`@return`

  Describe the return values of a function.

`@usage`

  Describe the usage of a function or variable.

`@description`

  The description of a function or table; this is normally inferred from the beginning of the documentation comment.

`@name`

  The name of the function or table definition. This should be used only if the documentation system cannot infer the name by code analysis.

`@class`

  Like `@name`, if the name of the variable is not being correctly inferred through code analysis, you may specify the class (function or table) explicitly.

`@field`

  Describe a table field definition.

`@release`

  Free format string to describe the module or file release.

`@args`

  This tag is special to the file comment. It allows you specify arguments to the script (via --script-args) that your script uses. The first alphanumeric word (literally matching in Lua `([%w%p]+)`) is used as the name for the argument, the rest its description.

`@summary`

  This allows you to specify the summary explicitly different from the first sentence of the description.

`@output`

  This tag is special to thie file comment. It allows you to show typical output of a script. You may use "\n" to force a line break where you please.

# Version Detection Using NSE

The version detection system built into Nmap was designed to efficiently recognize the vast majority of protocols with a simple pattern matching syntax. Some protocols require a more complex approach, and a generalized scripting language is perfect for this. Skype v2 is one such protocol. It pretends to be an HTTP server, requiring multiple queries to determine its true nature. NSE has been integrated into Nmap's version detection framework to handle these cases. The scripts which extend the version scanner belong to the reserved category `version`.
This category cannot be run from the command line. It is only executed if the user has required a version scan. The following listing shows a simple script which demonstrates the use of the NSE version detection API. If either the TCP port 80 is open or the service has been determined to be HTTP, the script is triggered. Although it could be extended to recognize different HTTP servers, its only purpose is to show off the version detection API. It is not advisable to use NSE for version detection in the simple case of HTTP servers. The version detection variables have been filled with dummy entries to illustrate their effect on the Nmap output.

    description = "Demonstration of a version detection NSE script. It checks \
    and reports the version of a remote web server. For real life purposes it is \
    better to use Nmap version detection (-sV)."
    author = "Diman Todorov <diman.todorov@gmail.at>"
    license = "Same as Nmap--See http://nmap.org/book/man-legal.html"

    id = "HTTP version"

    categories = {"version"}

    runlevel = 1.0

    portrule = function(host, port)
      if (port.number == 80
          or port.service == "http" )
          and port.protocol == "tcp"
      then
        return true
      else
        return false
      end
    end

    action = function(host, port)
      local query = "GET / HTTP/2.1\r\n"
      query = query .. "Accept: */*\r\n"
      query = query .. "Accept-Language: en\r\n"
      query = query .. "User-Agent: Nmap NSE\r\n"
      query = query .. "Host: " .. host.ip .. ":" .. port.number .. "\r\n\r\n"

      local socket = nmap.new_socket()
      local catch = function()
        socket:close()
      end

      local try = nmap.new_try(catch)

      try(socket:connect(host.ip, port.number))
      try(socket:send(query))

      local response = ""
      local lines
      local status
      local value

      while true do
        status, lines = socket:receive_lines(1)

        if not status or value then
          break
        end

        response = response .. lines
        value = string.match(response, "Server: (.-)\n")
      end

      try(socket:close())

      if value then
        port.version.name = "[Name]"
        port.version.name_confidence = 10
        port.version.product = "[Product]"
        port.version.version = "[Version]"
        port.version.extrainfo = "[ExtraInfo]"
        port.version.hostname = "[HostName]"
        port.version.ostype = "[OSType]"
        port.version.devicetype = "[DeviceType]"

        port.version.service_tunnel = "none"
        port.version.fingerprint = nil
        nmap.set_port_version(host, port, "hardmatched")
      end
    end

This is what the output of this script looks like:

    $ nmap -sV localhost -p 80

    Starting Nmap ( http://nmap.org )
    Interesting ports on localhost (127.0.0.1):
    PORT   STATE SERVICE VERSION
    80/tcp open  [Name]  [Product] [Version] ([ExtraInfo])
    Service Info: Host: [HostName]; OS: [OSType]; Device: [DeviceType]

    Nmap done: 1 IP address (1 host up) scanned in 9.32 seconds

The name variable denotes the detected protocol name. The product, version and extrainfo variables are used to produce a human readable description of the server version. The remaining variables provide information deduced from the output of the server concerning the target host.

# Example Script

## Finger-Test Script

The finger script (`finger.nse`) is a perfect example of how short typical NSE scripts are.

first the information fields are filled out, note that the `id` field is kept short, this is important since it is printed in Nmap's output. A detailed description of what the script actually does should go in the `description` field.

    id="Finger Results"

    description="attempts to get a list of usernames via the finger service"

    author = "Eddie Bell <ejlbell@gmail.com>"

    license = "Same as Nmap--See http://nmap.org/book/man-legal.html"

The `categories` field is a table containing all the categories the script belongs to—These are used for script selection through the `--script` option.

    categories = {"discovery"}

You can use the facilities provided by the nselib () with `require`. Here we want to use shorter port rules.

    require "shortport"

We want to run the script against the finger service. So we test whether it is using the well-known finger port (`79/tcp`), or whether the service is named finger based on version detection results or in the port number's listing in `nmap-services`.

We want to check whether the service behind the port is finger, or whether it runs on finger's well-known port 79. Through this we can use the information gathered during the version scan (if finger runs on a non-standard port) or still run against at least the port we expect it, should the version detection information not be available.

    portrule = shortport.port_or_service(79, "finger")

    action = function(host, port)
      local socket = nmap.new_socket()
      local results = ""
      local status = true

The function `err_catch()` will be called for clean up, through NSE's exception handling mechanism. Here it only closes the previously opened socket (which should be enough in most cases).

    local err_catch = function()
      socket:close()
    end

The clean up function gets registered for exception handling via a call to `nmap.new_try()`

      local try = nmap.new_try(err_catch())

The script sets a timeout of 5000, which is equivalent to 5 seconds. Should any operation require more time we'll receive a `TIMEOUT` error message.

      socket:set_timeout(5000)

To make use of the exception handling we need to wrap calls to those functions which might return an error, inside `try()`

      try(socket:connect(host.ip, port.number, port.protocol))
      try(socket:send("\n\r"))

The call to `receive_lines()` is not wrapped in `try()`, because we don't want to abort the script just because we didn't receive the data we expected. Note that if there is less data than requested (100 lines), we will still receive it and the status will be `true`—subsequent calls would yield a `false` status.

      status, results = socket:receive_lines(100)
      socket:close()

The script returns a string if the call to `receive_lines()` was successful, otherwise it returns `nil`.

      return results
      end

# Implementation

Now how does all this work? The following section describes some interesting aspects of NSE. While the focus primarily lies on giving script writers a better feeling of what happens with scripts, it should also provide a starting point for understanding (and extending) the NSE sources.

## Initialization Phase

During its initialization stage, Nmap loads the Lua interpreter, including its provided libraries. These libraries are documented in the Lua Reference Manual. Here is a summary:

- The package library (namespace: `package`)—Lua's package-lib provides (among others) the `require` function, used to load modules from the nselib.

- The table library (namespace: `table`)—The table manipulation library contains many functions used to operate on `tables`—Lua's central data structure.

- The I/O library (namespace: `io`)—The Input/Output library offers functions such as reading files and reading the output from programs you execute.

- The OS library (namespace: `os`)—The Operating System library provides facilities of the operating system, including filesystem operations (renaming/removing files, temporary file creation) and access to the environment.

- The string library (namespace: `string`)—The string library helps you with functions used to manipulate strings inside Lua. Functions include: printf-style string formatting, pattern matching using Lua-style patterns, substring extraction, etc.

- The math library (namespace: `math`)—Numbers in Lua usually correspond to the `double` C type, so the math library provides access to rounding functions, trigonometric functions, random number generation, and more.

- The debug library (namespace: `debug`)—The debug library provides you with a somewhat lower level API to the Lua interpreter. Through it you can access functions along the execution stack, get function closures and object metatables, etc.

In addition to loading the libraries provided by Lua, the functions in the `nmap` namespace are loaded. The search paths are the same directories that Nmap searches for its data files and scripts, except that the `nselib` directory is appended to each. In this step the provided script arguments are stored inside the registry.

The next phase of NSE initialization is loading the chosen scripts, which are the arguments provided to the `--script`
option or `default`, in case of a default script scan. The string `version`
is appended, if version detection was enabled. The arguments afterwards are tried to be interpreted as script categories. This is done via a Lua C function in `nse_init.cc` called `entry`. Inside `script.db`,
for each category of a script, there is a call to `Entry`. If the category was chosen then the script is loaded. Every argument of `--script` that could not be interpreted as a category is loaded as a file or directory. If the file or directory could not be located, then an error is raised and the Script Engine aborts.

All of the `.nse` files inside a loaded directory are loaded as files. Each file loaded is executed by Lua. If a portrule is present, then it is saved in the porttests table with a portrule key and file closure value. Otherwise, if the script has a hostrule, then it is saved in the hosttests table in the same manner.

## Matching of Scripts to Targets

After the initialization is finished the `hostrules`
and `portrules`
are evaluated for each host in the current target group. At this check a list is built which contains the combinations of scripts and the hosts they will run against.

It should be noted that the rules of all chosen scripts are checked against all hosts and their `open`
and `open|filtered`
ports. Therefore it is advisable to leave the rules as simple as possible and to do all the computation inside the `action`, as a script will only be executed if it is run against a specific target. After the check those script-target combinations get their own Lua thread. A thread running against a host will have only a hostrule passed to the action closure whereas a thread running against a port will have both a hostrule and portrule passed. Each thread is stored with information relevant to the thread. This information includes the runlevel, target, target port (if applicable), host and port tables (passed to action), its type (running against a host or port), and its id. The mainloop function will work on each runlevel grouping of threads in order.

## Running Scripts

Nmap is able to perform NSE script scanning in parallel
by making use of Lua language features. In particular, coroutines offer collaborative multi-threading so scripts can suspend themselves at defined points, and allow other coroutines to execute. Since network I/O, especially waiting for responses from remote host, is the part of scripts which would consume most time with waiting, this is the point where scripts suspend themselves and let others execute. Each call to some of the functions of the Nsock wrapper causes the calling script to yield (pause). Once the request is processed by the Nsock library, the callback causes the script to be pushed from the waiting queue to the running queue, which will eventually let it resume its operation.

The mainloop function will maintain two sets of threads, running and waiting. Threads will be moved back and forth between the sets; when a thread yields, it is moved to the waiting group. Threads run in the running set will either yield, complete, or error. After all scripts are resumed in the running set, mainloop will place all yielded threads ready to be run in the running set. Threads are made "ready" by calling `process_waiting2running`. This process of running threads and moving paused threads to the waiting and running sets is repeated until no threads exist in either waiting or running.

## Adding C Modules to Nselib

Some of the modules included in nselib are not written in Lua but in C or C++. `bit` and `pcre` are two examples. This section describes how to write your own compiled extensions to nselib.

The C API of Lua is described at length in Programming in Lua, Second Edition, Programming in Lua, Second Edition, so this is a short summary. C modules consist of functions that follow the protocol of the lua_CFunction type. The functions are registered with Lua and assembled into a library by calling the `luaL_register`
function. A special initialization function provides the interface between the module and the rest of the NSE code. By convention the initialization function has a name of the form `luaopen_module`.

The smallest compiled modules that comes with NSE is `bit`,
and one of the most straightforward is `openssl`.
These modules serve as good examples for a beginning module writer. The source code for `bit` is in the files `nse_bit.cc` and `nse_bit.h`, and likewise the source for `openssl` is in `nse_openssl.cc` and `nse_openssl.h`. The other compiled modules usually follow this naming convention.

Let us look at the `openssl` module. One of the functions in `nse_openssl.cc` is `l_md5`, which calculates an MD5 digest. Its function prototype is

    static int l_md5(lua_State *L);

The prototype shows that `l_md5` matches the lua_CFunction type. The function is static because it does not have to be visible to other compiled code, it just needs an address so it can be registered with Lua. Later in the file we see `l_md5` entered into an array of type luaL_reg and associated with the name `md5`, the name it will be known by to NSE:

    static const struct luaL_reg openssllib[] = {
      { "md5", l_md5 },
      { NULL, NULL }
    };

Then the library is registered with a call to `luaL_register` inside the initialization function `luaopen_openssl`. Some lines relating to the registration of OpenSSL BIGNUM types have been omitted.

    LUALIB_API int luaopen_openssl(lua_State *L) {
      luaL_register(L, OPENSSLLIBNAME, openssllib);
      return 1;
    }

(`NSE_OPENSSLLIBNAME` is just the string `"openssl"`, the name of the module.) `luaopen_openssl` is the only function in the file that is exposed in `nse_openssl.h`.

Once a compiled module is written, it is added to NSE by including it in the list of standard libraries in `nse_init.cc`. Just follow the example of the modules that are already there. Then the names of the module's source files of the must be added to `Makefile.in` in the appropriate places. Again it is easiest to follow the example of the other modules. For the Windows build the new source files must be added to the `mswin32/nmap.vcproj` project file.