The keyword patterns like "categories" or K "true" were consuming the
first part of patterns like --script 'broadcast-*' resulting in the
error "'broadcast-*' did not match a category, filename, or directory"
Changed to add a lookahead match for space, parentheses, or end-of-line
before considering a keyword to have matched.
Update the patterns used to extract hex chars from the string representation of a coroutine. It seems the string has changed in Lua 5.3 from "0xAB..." to "thread: AB..."; this was before:-
NSE: Starting http-feed M:nil against ...
NSE: http-feed M:nil spawning new thread (thread: 02C63A78).
NSE: Finished http-feed W:nil against ...
this is now:-
NSE: Starting http-feed M:02D6EAF0 against ...
NSE: http-feed M:02D6EAF0 spawning new thread (thread: 02D6E640).
NSE: Finished http-feed W:02D6E640 against ...
This allows usage of verbose/debug in portrule/hostrule and access to all
functionality of NSE, including sockets. So for example, we can now do:
function portrule (host, port)
local response = http.get(host, port, "/");
stdnse.debug1(response.body)
...
end
The verbose/debug function did not work in rule functions because the
introspection API (getid, gettid, etc.) only work when NSE is in the main loop.
The main loop sets the required internal variable current needed by the API.
List of changes:
stdnse.lua:
o debug/verbose check the debugging/verbosity level much earlier to allow
returning if nothing will be printed.
o Simplified debug/verbose logic to handle the optional first argument
better.
o made debug/verbose local functions to avoid using globals and allow self
tail calls
nse_main.lua:
o The logic for adding threads via a rule function is simplified. So long as
the script has the desired rule function, a thread is always returned.
Evaluation of the rule function is done while NSE is in the main loop (i.e.
not in script:new_thread()). The rule function only determines if the action
function is run.
o [Not a change:] If the action function will be run or was run then we see
the usual "Starting X" and "Finished X" messages from NSE.
o Use Lua 5.2's pack function instead of the slightly more expensive
{n = select("#", ...), ...} idiom.
o New stdnse.getinfo introspection function which is used by stdnse.debug.
string.gsub returns 2 values, the new string and the number of
replacements made. It also has a 4th argument, the number of
replacements to make. So when you use the return value of gsub as the
3rd argument, and no replacements were made, it instructs the next call
to not make any replacements. Thanks to Ron Bowes for reporting this
issue.
A user reported this crash when scanning a target whose name contained
the '%' character:
NSE: Script Engine Scan Aborted.
An error was thrown by the engine: nse_main.lua:322: invalid capture index
stack traceback:
[C]: in function 'gsub'
nse_main.lua:322: in function 'd'
nse_main.lua:377: in function 'start'
nse_main.lua:912: in function 'run'
nse_main.lua:1390: in function <nse_main.lua:1293>
[C]: in ?
I'm not sure how a name with '%' got resolved, but I was able to
reproduce the crash by adding this line to /etc/hosts:
127.0.0.1 a%40b
and then running
./nmap --script=banner a%40b -d --top-ports 5
The gsub function recognizes "%d", where d is a digit, as a capture
index. The constructed string is then passed to print_debug, which is
like printf. Therefore we escape every occurrence of "%" twice, to get
"%%%%".
final script-args table. The rationale is, unfortunately shells interpret
quotes differently and so it can be hard to tell exactly what NSE ends up
seeing/producing. [Some discussion in #nmap on Freenode resulted in this
addition.]
o Moved some specific behavior for resuming a thread to Thread:resume().
o Cleaned up the inheritance mechanism to use a static metatable (the Class
table itself).
o Worker main functions are no longer wrapped with a function to truncate to 0
results. Instead, we just check if a thread is a worker in Thread:set_output()
to prevent adding output by worker threads.
NSE: Script Engine Scan Aborted.
An error was thrown by the engine: attempt to get length of a number
value
stack traceback:
[C]: in function 'port_set_output'
/home/rt/builds/nmap/nse_main.lua:314: in function 'set_output'
/home/rt/builds/nmap/nse_main.lua:947: in function 'run'
/home/rt/builds/nmap/nse_main.lua:1330: in function
</home/rt/builds/nmap/nse_main.lua:1237>
[C]: in ?
This now allows return of anything stringifiable as second return value
without crashing (though this may not be what the script author intends,
better to not crash)
Scripts may now return a key–value table, or such a table in addition to
a string. The table will be automatically formatted for normal output
and will appear as a hierarchy of elements in XML output.
Some history and discussion of this development can be found at
https://secwiki.org/w/Nmap/Structured_Script_Output.
This is a merge of r29484:29569 from /nmap-exp/david/xml-output.
* replace require function calls with stndse.silent_require
* fixed a bug in nse_main that would fail creating scripts.db when a script
fails to load
* reworked some code to provide limited functionality even though SSL is not
present
The loaders list determines the order in which places are checked for
Lua modules. I had a problem on Windows when the nmap.exe directory
contained WinPcap's packet.dll. There was a runtime error when liblua
tried to open packet.dll instead of our packet.lua. This way we check
for our own files first.
the name of a file containing all of your desired NSE script
arguments. The arguments may be separated with commas or newlines
and may be overridden by arguments specified on the command-line
with --script-args. [Daniel Miller]
directories that nmap will search, e.g. ~/.nmap/nselib $NMAPDIR/nselib
/usr/share/nmap/nselib etc.
Resolves Daniel Miller's request [1].
[1] http://seclists.org/nmap-dev/2011/q3/817
