Lua 5.3 adds several awesome features of particular interest to nmap including
bitwise operators and integers, a utf8 library, and standard binary pack/unpack
functions.
In addition to adding Lua 5.3, this branch changes:
o Complete removal of the NSE bit library (in C), It has been replaced with
a new Lua library wrapping Lua 5.3's bit-wise operators.
o Complete removal of the NSE bin library (in C). It has been replaced with a
new Lua library wrapping Lua 5.3's string.pack|unpack functions.
o The bin.pack "B" format specifier (which has never worked correctly) is
unimplemented. All scripts/libraries which use it have been updated. Most
usage of this option was to allow string based bit-wise operations which are no
longer necessary now that Lua 5.3 provides integers and bit-wise operators.
o The base32/base64 libraries have been reimplemented using Lua 5.3's new
bitwise operators. (This library was the main user of the bin.pack "B" format
specifier.)
o A new "bits" library has been added for common bit hacks. Currently only has
a reverse function.
Thanks to David Fifield, Daniel Miller, Jacek Wielemborek, and Paulino
Calderon for testing this branch.
Unnecessary, probably, but this eliminates 2 of 7 string
creations/collections due to concatenation and reassignment. Also
eliminates 2 unneeded table lookups and tests (since host always has
host.ip and port always has port.number), and eliminates 1 extra test
for prerule and postrule scripts, since we can't have a port if we don't
have a host.
The charset argument was intended to be a table, since Lua doesn't allow
indexing strings with []. Unfortunately, irc-brute and tftp-enum passed
a string instead, which meant that the output was always the empty
string. This change allows both formats, and fixes those scripts.
irc-brute and tftp-enum had been broken since r33632 (no released
version of Nmap)
This allows stdnse.keys to be used in a __pairs metamethod to, for
instance, yield keys in sorted order. Using next() bypasses the __pairs
metamethod that would be called when pairs() was used. Otherwise,
infinite recursion was possible.
Mostly splitting function summaries (the first paragraph of NSEdoc) from
the body of the description to make the summary indexes shorter and
easier to scan.
Also fixed some unbalanced code tags like <code>foo</table>
This allows usage of verbose/debug in portrule/hostrule and access to all
functionality of NSE, including sockets. So for example, we can now do:
function portrule (host, port)
local response = http.get(host, port, "/");
stdnse.debug1(response.body)
...
end
The verbose/debug function did not work in rule functions because the
introspection API (getid, gettid, etc.) only work when NSE is in the main loop.
The main loop sets the required internal variable current needed by the API.
List of changes:
stdnse.lua:
o debug/verbose check the debugging/verbosity level much earlier to allow
returning if nothing will be printed.
o Simplified debug/verbose logic to handle the optional first argument
better.
o made debug/verbose local functions to avoid using globals and allow self
tail calls
nse_main.lua:
o The logic for adding threads via a rule function is simplified. So long as
the script has the desired rule function, a thread is always returned.
Evaluation of the rule function is done while NSE is in the main loop (i.e.
not in script:new_thread()). The rule function only determines if the action
function is run.
o [Not a change:] If the action function will be run or was run then we see
the usual "Starting X" and "Finished X" messages from NSE.
o Use Lua 5.2's pack function instead of the slightly more expensive
{n = select("#", ...), ...} idiom.
o New stdnse.getinfo introspection function which is used by stdnse.debug.
1. The first paragraph of a function's NSEdoc is used as a short
summary. Some of these were very long, so I split off a shorter summary.
2. Use asterisks (*) to denote bulletted lists, not 'o'
3. Wrap lines at 80 columns
4. a couple other spelling and formatting fixes
These implementations were all functionally identical. The replacement
has an extra feature of returning the index where the value was found,
currently unused.
This function will format a MAC address as colon-separated hex bytes.
It's really very simple: stdnse.tohex(mac, {separator=":"})
This commit updates all the instances I could find of the varying
convoluted attempts at performing this conversion.
If you ran the (fortunately non-default) http-domino-enum-passwords
script with the (fortunately also non-default)
domino-enum-passwords.idpath parameter against a malicious server,
it could cause an arbitrarily named file to to be written to the
client system. Thanks to Trustwave researcher Piotr Duszynski for
discovering and reporting the problem. We've fixed that script, and
also updated several other scripts to use a new
stdnse.filename_escape function for extra safety. This breaks our
record of never having a vulnerability in the 16 years that Nmap has
existed, but that's still a fairly good run. [David, Fyodor]
final script-args table. The rationale is, unfortunately shells interpret
quotes differently and so it can be hard to tell exactly what NSE ends up
seeing/producing. [Some discussion in #nmap on Freenode resulted in this
addition.]
Two changes here, both minor. First, explicitly assigning a new key to
nil does not add the key to the ordered set of keys. This better
emulates the behavior of regular tables.
> o = stdnse.output_table()
> o["test"] = nil
This previously resulted in output like this:
|_ test: nil
Now it simply omits the "test:" key.
Second, I needed a way to tell whether an output table was empty or not.
Since Lua's next() function doesn't call the __pairs metamethod, it was
always returning nil. Instead, I used the __call metamethod, since it
had the least preexisting semantic meaning:
> o = stdnse.output_table()
> =o()
false
> o["test"] = 1
> =o()
true
