A few bugs crept in during the conversion to LPEG parsing:
1. Unicode parsing of UTF-16 surrogate pairs returned an ending position
that was off by 2, so that the last 2 characters in the escape code were
parsed as literal characters.
2. Arrays and objects were not being tagged as such with make_array and
make_object during parsing.
This commit fixes these bugs, and returns to using unicode.utf8_enc for
UTF-8 encoding instead of a separately-defined function. Also, the tests
at the end (which were helpful in finding both of the bugs above) were
converted to unittest.lua tests.
Building a string with var = var .. "something" has miserable time
complexities. This commit cleans up a lot of that in scripts, focusing
on packing of data with bin.pack and concatenations within loops.
Additionally, a few instances were replaced with string.rep
psl_tree now takes an accumulator argument. It adds lines to the result
table instead of concatenating them together in a string. Then psl_print
concats all the lines together.
The charset argument was intended to be a table, since Lua doesn't allow
indexing strings with []. Unfortunately, irc-brute and tftp-enum passed
a string instead, which meant that the output was always the empty
string. This change allows both formats, and fixes those scripts.
irc-brute and tftp-enum had been broken since r33632 (no released
version of Nmap)
The message stating that an unknown -P* option was used contained
references to old, deprecated forms:
-PT (now -PA)
-PB (now -PE -PA)
-PI (now -PE)
Additionally, -PR was missing.
Entering nsock_loop() with the poll engine activated and no registered FD
(timers only, for instance) should not directly return, but sleep until next
timeout.
stdnse.debug writes the IP to messages so scripts don't have to.
A couple of functions no longer needed the target IP passed to them: their
signature and nsedoc were updated accordingly, along with any calls to them.
Also fix some white space.
Update the patterns used to extract hex chars from the string representation of a coroutine. It seems the string has changed in Lua 5.3 from "0xAB..." to "thread: AB..."; this was before:-
NSE: Starting http-feed M:nil against ...
NSE: http-feed M:nil spawning new thread (thread: 02C63A78).
NSE: Finished http-feed W:nil against ...
this is now:-
NSE: Starting http-feed M:02D6EAF0 against ...
NSE: http-feed M:02D6EAF0 spawning new thread (thread: 02D6E640).
NSE: Finished http-feed W:02D6E640 against ...
In most cases (e.g. any of the nmap.socket operations), functions can
take full host and port tables instead of just host.ip and port.number.
This makes for cleaner-looking code and easier extensibility if we
decide to check for a protocol on both TCP and UDP, for instance.
Had a report via service fingerprint correction of netbus-version
reporting an unrelated service as NetBuster. This would happen for any
service on port 12345 that responds with a banner containing less than 2
carriage returns, or which closes the connection after the first NetBus
protocol message. Now, all netbus-* scripts require the banner to begin
with "NetBus" before they will continue.
