Merged -r13793:HEAD from nmap-exp/dev/nmap branch now that we're opening up trunk development

again.  Here are the items which were merged:

------------------------------------------------------------------------
r13971 | jah | 2009-06-29 14:30:27 -0700 (Mon, 29 Jun 2009) | 2 lines

Improved a pattern for matching HTTP status-line, tidy away some variables and
fix a typo.
------------------------------------------------------------------------
r13967 | daniel | 2009-06-29 13:47:04 -0700 (Mon, 29 Jun 2009) | 5 lines

o Added a convenience top-level BSD makefile redirecting BSD make
  to GNU make on BSD systems.  This should help prevent bogus
  error reports when users run "make" instead of "gmake" on BSD
  systems. [Daniel Roethlisberger]

------------------------------------------------------------------------
r13965 | batrick | 2009-06-29 06:50:11 -0700 (Mon, 29 Jun 2009) | 14 lines

[NSE] The NSE Nsock Library binding no longer relies on garbage collection to
monitor the use of socket "slots". A thread (script) attempting to connect must
first obtain one of a limited number of available socket locks (usually 10 or
--max-parallelism). The binding would use garbage collection of sockets to
determine when a thread has finished using its allocated sockets. This is
unfortunately slow and requires us to constantly run the garbage collector to
cause timely reallocation. I have changed the binding to now regularly inspect
allocated sockets in the nsock_loop function. Available sockets slots are now
immediately reallocated and done with far less execution time.

See [1] for benchmarks and further explanation.

[1] http://seclists.org/nmap-dev/2009/q2/0624.html

------------------------------------------------------------------------
r13964 | batrick | 2009-06-29 06:37:49 -0700 (Mon, 29 Jun 2009) | 10 lines

[NSE] Fixed a rare (and usually undetectable) bug that can cause a SEGFAULT.
The NSE nsock library binding may attempt to push values on the stack of
a thread that ended due to an error. It is possible that the internal
Lua stack was completely full and any further pushed values would result
in a segmentation memory violation.

This bug is very hard to reproduce with a SEGFAULT but is usually visible
when Lua assertion checks are turned on. A socket handler routine must be
called AFTER a thread has ended in error.

------------------------------------------------------------------------
r13963 | batrick | 2009-06-29 05:51:20 -0700 (Mon, 29 Jun 2009) | 3 lines

Fixed some global scoped variables to be local. This caused a many scripts to
overwrite each others' sockets, options, etc.

------------------------------------------------------------------------
r13939 | joao | 2009-06-27 16:07:35 -0700 (Sat, 27 Jun 2009) | 2 lines

Fixed port rule to include ssl pop3 port, now that pop3.lua supports SSL connections in function capabilities

------------------------------------------------------------------------
r13938 | joao | 2009-06-27 16:06:28 -0700 (Sat, 27 Jun 2009) | 2 lines

Added transparent SSL support using comm.tryssl

------------------------------------------------------------------------
r13937 | joao | 2009-06-27 16:05:19 -0700 (Sat, 27 Jun 2009) | 2 lines

Added transparent SSL support using comm.tryssl

------------------------------------------------------------------------
r13936 | joao | 2009-06-27 16:03:50 -0700 (Sat, 27 Jun 2009) | 2 lines

Added SSL transparent support using comm.tryssl

------------------------------------------------------------------------
r13935 | joao | 2009-06-27 16:02:39 -0700 (Sat, 27 Jun 2009) | 2 lines

Added SSL transparent support using comm.tryssl

------------------------------------------------------------------------
r13934 | joao | 2009-06-27 16:01:38 -0700 (Sat, 27 Jun 2009) | 2 lines

Added SSL transparent support using comm.tryssl

------------------------------------------------------------------------
r13933 | joao | 2009-06-27 16:00:27 -0700 (Sat, 27 Jun 2009) | 2 lines

SSL transparent support using comm.tryssl

------------------------------------------------------------------------
r13932 | joao | 2009-06-27 15:19:58 -0700 (Sat, 27 Jun 2009) | 2 lines

Included transparent ssl support to function pop3.capabilities using comm.tryssl

------------------------------------------------------------------------
r13931 | joao | 2009-06-27 15:19:06 -0700 (Sat, 27 Jun 2009) | 3 lines

New version of comm.lua with function tryssl, that transparently adds support to ssl connections


------------------------------------------------------------------------
r13930 | joao | 2009-06-27 14:50:38 -0700 (Sat, 27 Jun 2009) | 6 lines

Fixed buffering problem exposed by david on nmap-dev list.
The problem was solved using a buffer to receive the data, making the script work fine in cases where the ssh packets are fragmented.

A very similar solution was applied to ssh1.lua.


------------------------------------------------------------------------
r13928 | batrick | 2009-06-27 04:43:12 -0700 (Sat, 27 Jun 2009) | 18 lines

[NSE] We now propogate a NSE initiated yield on a script through all user
coroutines so that NSE may resume control. Previously, scripts that would yield
in a child coroutine (e.g. a script's child coroutine generated by Lua's
coroutine.create function) would give control back to the script. A script
would yield in this way by making a blocking socket operation. NSE would be
unable to correctly resume child coroutine when the socket operation is
finished processing.

By yielding the chain of coroutines a script has operating, we allow to NSE to
handle the socket operation properly. NSE would then resume the entire chain so
execution may correctly resume at the coroutine which initiated the socket
operation. This restores the "illusion" that a script executes without
interruption.

See [1] for more information, further explanation, and some use cases.

[1] http://seclists.org/nmap-dev/2009/q2/0586.html

------------------------------------------------------------------------
r13817 | david | 2009-06-18 15:57:29 -0700 (Thu, 18 Jun 2009) | 3 lines

Improve an OS fingerprint with a model number and broader matching.
Based on a follow-up report from a submitter.

------------------------------------------------------------------------
r13814 | josh | 2009-06-17 21:34:15 -0700 (Wed, 17 Jun 2009) | 3 lines

[zenmap] Added support to zenmap for the new SCTP options: -PY, -sY and -sZ


------------------------------------------------------------------------
r13797 | ron | 2009-06-17 11:02:18 -0700 (Wed, 17 Jun 2009) | 1 line

Applied a patch from Mak Kolibabi that enhances the output of smb-enum-processes. The output is now modeled after the output of the 'ps' tool for higher verbosity levels.
------------------------------------------------------------------------
r13795 | david | 2009-06-17 09:05:21 -0700 (Wed, 17 Jun 2009) | 6 lines

The configure script now allows cross-compiling by assuming that
libpcap is recent enough. Previously it would quit because a test
program could not be run. libpcap will always be recent enough when
the included copy is used. The patch was contributed by Mike
Frysinger.

BSDmakefile

@@ -0,0 +1,9 @@
+# $Id$
+# Redirect BSD make to GNU gmake for convenience
+
+USE_GNU:
+	@gmake $(.TARGETS)
+
+$(.TARGETS): USE_GNU
+
+.PHONY: USE_GNU

CHANGELOG

@@ -1,5 +1,20 @@
 # Nmap Changelog ($Id$); -*-text-*-
 
+o Added a convenience top-level BSD makefile redirecting BSD make
+  to GNU make on BSD systems.  This should help prevent bogus
+  error reports when users run "make" instead of "gmake" on BSD
+  systems. [Daniel Roethlisberger]
+
+o [Zenmap] Added support to zenmap for the SCTP options: -PY, -sY and -sZ, as
+  well as making a comment in zenmapCore/NmapOptions.py on how to add new
+  options. [Josh Marlow]
+
+o The configure script now allows cross-compiling by assuming that
+  libpcap is recent enough. Previously it would quit because a test
+  program could not be run. libpcap will always be recent enough when
+  the included copy is used. The patch was contributed by Mike
+  Frysinger.
+
 Nmap 4.90RC1 [2009-06-25]
 
 o [Zenmap] Fixed a display hanging problem on Mac OS X reported by

configure

@@ -7089,13 +7089,8 @@ if test $have_libpcap = yes; then
   { $as_echo "$as_me:$LINENO: checking if libpcap version is recent enough" >&5
 $as_echo_n "checking if libpcap version is recent enough... " >&6; }
   if test "$cross_compiling" = yes; then
-  { { $as_echo "$as_me:$LINENO: error: in \`$ac_pwd':" >&5
+  { $as_echo "$as_me:$LINENO: result: cross-compiling -- assuming yes" >&5
-$as_echo "$as_me: error: in \`$ac_pwd':" >&2;}
+$as_echo "cross-compiling -- assuming yes" >&6; }; have_libpcap=yes
-{ { $as_echo "$as_me:$LINENO: error: cannot run test program while cross compiling
-See \`config.log' for more details." >&5
-$as_echo "$as_me: error: cannot run test program while cross compiling
-See \`config.log' for more details." >&2;}
-   { (exit 1); exit 1; }; }; }
 else
   cat >conftest.$ac_ext <<_ACEOF
 /* confdefs.h.  */

configure.ac

@@ -386,7 +386,8 @@ int main() {
   exit(0);
 }],
 [AC_MSG_RESULT(yes); have_libpcap=yes],
-[AC_MSG_RESULT(no); have_libpcap=no])
+[AC_MSG_RESULT(no); have_libpcap=no],
+[AC_MSG_RESULT(cross-compiling -- assuming yes); have_libpcap=yes])
 LIBS="$LIBS_OLD"
 fi
 

nmap-os-db

@@ -3496,10 +3496,10 @@ T7(R=Y%DF=Y%T=FA-104%TG=FF%W=0%S=Z%A=S+%F=AR%O=%RD=0%Q=)
 U1(DF=N%T=FA-104%TG=FF%IPL=164%UN=0%RIPL=G%RID=G%RIPCK=G%RUCK=G%RUD=G)
 IE(DFI=N%T=FA-104%TG=FF%CD=S)
 
-# AV-TECH Digital Video Recorder (device connected to a surveillance camera)
+# AV-TECH Digital Video Recorder (device connected to a surveillance camera), AVT AVC 773W 4CH Network DVR
-Fingerprint AVtech digital video recorder
+Fingerprint AVtech AVC 773W digital video recorder
 Class AVtech | embedded || webcam
-SEQ(SP=C-1E%GCD=1-6%ISR=55-5F%TI=I%II=RI%SS=O|S%TS=U)
+SEQ(SP=C-1E%GCD=1-6%ISR=1B-5F%TI=I%II=RI%SS=O|S%TS=U)
 OPS(O1=M59E%O2=M578%O3=M280%O4=M59E%O5=M218%O6=M109)
 WIN(W1=FAF0%W2=FAF0%W3=FAF0%W4=FAF0%W5=FAF0%W6=FAF0)
 ECN(R=Y%DF=Y%T=FA-104%TG=FF%W=FAF0%O=M59E%CC=N%Q=)
@@ -3510,8 +3510,8 @@ T4(R=Y%DF=Y%T=FA-104%TG=FF%W=FAF0%S=A+%A=S%F=AR%O=%RD=0%Q=)
 T5(R=Y%DF=Y%T=FA-104%TG=FF%W=FAF0%S=A%A=S+%F=AR%O=%RD=0%Q=)
 T6(R=Y%DF=Y%T=FA-104%TG=FF%W=FAF0%S=A%A=S%F=AR%O=%RD=0%Q=)
 T7(R=Y%DF=Y%T=FA-104%TG=FF%W=FAF0%S=A%A=S+%F=AR%O=%RD=0%Q=)
-U1(DF=Y%T=FA-104%TG=FF%IPL=38%UN=27B6|306C|3949|425C|4B9A%RIPL=G%RID=G%RIPCK=G%RUCK=G%RUD=G)
+U1(DF=Y%T=FA-104%TG=FF%IPL=38%RIPL=G%RID=G%RIPCK=G%RUCK=G%RUD=G)
-IE(DFI=S%T=24-2E|30-40%TG=40%CD=S)
+IE(DFI=S%T=24-40%TG=40%CD=S)
 
 # Axis 70U Network document server
 Fingerprint AXIS 70U Network Document Server

nse_main.cc

@@ -23,6 +23,8 @@
 #define NSE_TRACEBACK "NSE_TRACEBACK"
 
 /* string keys used in interface with nse_main.lua */
+#define NSE_YIELD "NSE_YIELD"
+#define NSE_BASE "NSE_BASE"
 #define NSE_WAITING_TO_RUNNING "NSE_WAITING_TO_RUNNING"
 #define NSE_DESTRUCTOR "NSE_DESTRUCTOR"
 
@@ -151,10 +153,16 @@ static int dump_dir (lua_State *L)
   return 1;
 }
 
+/* This must call the l_nsock_loop function defined in nse_nsock.cc.
+ * That closure is created in luaopen_nsock in order to allow
+ * l_nsock_loop to have access to the nsock library environment.
+ */
 static int nsock_loop (lua_State *L)
 {
-  if (l_nsock_loop(luaL_checkint(L, 1)) == NSOCK_LOOP_ERROR)
+  lua_settop(L, 1);
-    luaL_error(L, "an error occurred in nsock_loop");
+  lua_getfield(L, LUA_REGISTRYINDEX, NSE_NSOCK_LOOP);
+  lua_pushvalue(L, 1);
+  lua_call(L, 1, 0);
   return 0;
 }
 
@@ -464,6 +472,14 @@ static int run_main (lua_State *L)
   return 0;
 }
 
+int nse_yield (lua_State *L)
+{
+  lua_getfield(L, LUA_REGISTRYINDEX, NSE_YIELD);
+  lua_pushthread(L);
+  lua_call(L, 1, 1); /* returns NSE_YIELD_VALUE */
+  return lua_yield(L, 1); /* yield with NSE_YIELD_VALUE */
+}
+
 void nse_restore (lua_State *L, int number)
 {
   luaL_checkstack(L, 5, "nse_restore: stack overflow");
@@ -506,6 +522,12 @@ void nse_destructor (lua_State *L, char what)
   lua_pop(L, what == 'a' ? 2 : 1);
 }
 
+void nse_base (lua_State *L)
+{
+  lua_getfield(L, LUA_REGISTRYINDEX, NSE_BASE);
+  lua_call(L, 0, 1); /* returns base thread */
+}
+
 static lua_State *L_NSE = NULL;
 
 void open_nse (void)

nse_main.h

@@ -31,8 +31,10 @@ class Target;
 
 
 /* API */
+int nse_yield (lua_State *);
 void nse_restore (lua_State *, int);
 void nse_destructor (lua_State *, char);
+void nse_base (lua_State *);
 
 void open_nse (void);
 void script_scan (std::vector<Target *> &targets);

nse_main.lua

@@ -32,6 +32,8 @@
 
 local NAME = "NSE";
 
+local YIELD = "NSE_YIELD";
+local BASE = "NSE_BASE";
 local WAITING_TO_RUNNING = "NSE_WAITING_TO_RUNNING";
 local DESTRUCTOR = "NSE_DESTRUCTOR";
 
@@ -84,6 +86,45 @@ do -- Append the nselib directory to the Lua search path
   package.path = package.path..";"..path.."?.lua";
 end
 
+-- NSE_YIELD_VALUE
+-- This is the table C uses to yield a thread with a unique value to
+-- differentiate between yields initiated by NSE or regular coroutine yields.
+local NSE_YIELD_VALUE = {};
+
+do
+  -- This is the method by which we allow a script to have nested
+  -- coroutines. If a sub-thread yields in an NSE function such as
+  -- nsock.connect, then we propogate the yield up. These replacements
+  -- to the coroutine library are used only by Script Threads, not the engine.
+
+  local function handle (co, status, ...)
+    if status and NSE_YIELD_VALUE == ... then -- NSE has yielded the thread
+      return handle(co, resume(co, yield(NSE_YIELD_VALUE)));
+    else
+      return status, ...;
+    end
+  end
+
+  function coroutine.resume (co, ...)
+    return handle(co, resume(co, ...));
+  end
+
+  local resume = coroutine.resume; -- local reference to new coroutine.resume
+  local function aux_wrap (status, ...)
+    if not status then
+      return error(..., 2);
+    else
+      return ...;
+    end
+  end
+  function coroutine.wrap (f)
+    local co = create(f);
+    return function (...)
+      return aux_wrap(resume(co, ...));
+    end
+  end
+end
+
 -- Some local helper functions --
 
 local log_write, verbosity, debugging =
@@ -430,16 +471,30 @@ local function run (threads)
     hosts[thread.host][thread.co] = true;
   end
 
+  -- Map of yielded threads to the base Thread
+  local yielded_base = setmetatable({}, {__mode = "kv"});
+  -- _R[YIELD] is called by nse_yield in nse_main.cc
+  _R[YIELD] = function (co)
+    yielded_base[co] = current; -- set base
+    return NSE_YIELD_VALUE; -- return NSE_YIELD_VALUE
+  end
+  _R[BASE] = function ()
+    return current.co;
+  end
   -- _R[WAITING_TO_RUNNING] is called by nse_restore in nse_main.cc
   _R[WAITING_TO_RUNNING] = function (co, ...)
-    if waiting[co] then -- ignore a thread not waiting
+    local base = yielded_base[co] or all[co]; -- translate to base thread
-      pending[co], waiting[co] = waiting[co], nil;
+    if base then
-      pending[co].args = {n = select("#", ...), ...};
+      co = base.co;
+      if waiting[co] then -- ignore a thread not waiting
+        pending[co], waiting[co] = waiting[co], nil;
+        pending[co].args = {n = select("#", ...), ...};
+      end
     end
   end
   -- _R[DESTRUCTOR] is called by nse_destructor in nse_main.cc
   _R[DESTRUCTOR] = function (what, co, key, destructor)
-    local thread = all[co] or current;
+    local thread = yielded_base[co] or all[co] or current;
     if thread then
       local ch = thread.close_handlers;
       if what == "add" then
@@ -456,7 +511,6 @@ local function run (threads)
   -- Loop while any thread is running or waiting.
   while next(running) or next(waiting) do
     local nr, nw = table_size(running), table_size(waiting);
-    cnse.nsock_loop(50); -- Allow nsock to perform any pending callbacks
     if cnse.key_was_pressed() then
       print_verbose(1, "Active NSE Script Threads: %d (%d waiting)\n",
           nr+nw, nw);
@@ -489,7 +543,12 @@ local function run (threads)
             traceback(co, tostring(result)));
         thread:close();
       elseif status(co) == "suspended" then
-        waiting[co] = thread;
+        if result == NSE_YIELD_VALUE then
+          waiting[co] = thread;
+        else
+          thread:d("%THREAD yielded unexpectedly and cannot be rerun.");
+          thread:close();
+        end
       elseif status(co) == "dead" then
         hosts[thread.host][co] = nil;
         if type(result) == "string" then
@@ -516,12 +575,13 @@ local function run (threads)
       end
     end
 
+    cnse.nsock_loop(50); -- Allow nsock to perform any pending callbacks
     -- Move pending threads back to running.
     for co, thread in pairs(pending) do
       pending[co], running[co] = nil, thread;
     end
 
-    collectgarbage "collect"; -- important for collecting used sockets & proxies
+    collectgarbage "step";
   end
 
   progress "endTask";

nse_nmaplib.cc

@@ -241,7 +241,7 @@ static int aux_mutex (lua_State *L)
       }
       lua_pushthread(L);
       lua_rawseti(L, lua_upvalueindex(1), lua_objlen(L, lua_upvalueindex(1))+1);
-      return lua_yield(L, 0);
+      return nse_yield(L);
     case DONE:
       lua_pushthread(L);
       if (!lua_equal(L, -1, lua_upvalueindex(2)))

nse_nsock.cc

@@ -40,6 +40,8 @@ extern "C"
 
 extern NmapOps o;
 
+static int l_nsock_loop(lua_State * L);
+
 static int l_nsock_connect(lua_State * L);
 
 static int l_nsock_send(lua_State * L);
@@ -182,6 +184,15 @@ static size_t table_length(lua_State * L, int index)
   return len;
 }
 
+static void weak_table(lua_State * L, int narr, int nrec, const char *mode)
+{
+  lua_createtable(L, narr, nrec);
+  lua_createtable(L, 0, 1);
+  lua_pushstring(L, mode);
+  lua_setfield(L, -2, "__mode");
+  lua_setmetatable(L, -2);
+}
+
 static std::string hexify(const unsigned char *str, size_t len)
 {
   size_t num = 0;
@@ -237,49 +248,20 @@ static void set_thread (lua_State *L, int index, struct l_nsock_udata *n)
 /* Some constants used for enforcing a limit on the number of open sockets
  * in use by threads. The maximum value between MAX_PARALLELISM and
  * o.maxparallelism is the max # of threads that can have connected sockets
- * (open). THREAD_PROXY, SOCKET_PROXY, and CONNECT_WAITING are tables in the
+ * (open).
- * nsock C functions' environment, at LUA_ENVIRONINDEX, that hold sockets and
- * threads used to enforce this. THREAD_PROXY has <Thread, Userdata> pairs
- * that associate a thread to a proxy userdata. This table has weak keys and
- * values so threads and the proxy itself can be collected. SOCKET_PROXY
- * has <Socket, Userdata> pairs that associate a socket to a proxy userdata.
- * SOCKET_PROXY has weak keys (to allow the collection of sockets) and strong
- * values, so the proxies are not collected when an associated socket is open.
  *
- * All the sockets used by a thread have the same Proxy Userdata. When all
+ * THREAD_SOCKETS is a weak keyed table of <Thread, Socket Table> pairs.
- * sockets in use by a thread are closed or collected, the entry in the
+ * A socket table is a weak keyed table (socket keys with garbage values) of
- * THREAD_PROXY table is cleared, freeing up a slot for another thread
+ * sockets the Thread has allocated but not necessarily open). You may 
- * to make connections. When a slot is freed, proxy_gc is called, via the
+ * test for an open socket by checking whether its nsiod field in the
- * userdata's __gc metamethod, which will add a thread in WAITING to running.
+ * socket userdata structure is not NULL.
+ *
+ * CONNECT_WAITING is a weak keyed table of <Thread, Garbage Value> pairs.
+ * The table contains threads waiting to make a socket connection.
  */
 #define MAX_PARALLELISM   10
-#define THREAD_PROXY       1           /* <Thread, Userdata> */
+#define THREAD_SOCKETS     1           /* <Thread, Table of Sockets (keys)> */
-#define SOCKET_PROXY       2           /* <Socket, Userdata> */
+#define CONNECT_WAITING    2           /* Threads waiting to lock */
-#define CONNECT_WAITING    3           /* Threads waiting to lock */
-#define PROXY_META         4           /* Proxy userdata's metatable */
-
-static int proxy_gc(lua_State * L)
-{
-  lua_rawgeti(L, LUA_ENVIRONINDEX, CONNECT_WAITING);
-  lua_pushnil(L);
-  if (lua_next(L, -2) != 0)
-  {
-    lua_State *thread = lua_tothread(L, -2);
-
-    nse_restore(thread, 0);
-    lua_pushnil(L);
-    lua_replace(L, -2);                // replace boolean
-    lua_settable(L, -3);               // remove thread from waiting
-  }
-  return 0;
-}
-
-static void new_proxy(lua_State * L)
-{
-  lua_newuserdata(L, 0);
-  lua_rawgeti(L, LUA_ENVIRONINDEX, PROXY_META);
-  lua_setmetatable(L, -2);
-}
 
 /* int socket_lock (lua_State *L)
  *
@@ -292,64 +274,85 @@ static void new_proxy(lua_State * L)
 static int socket_lock(lua_State * L)
 {
   lua_settop(L, 1);
-  lua_rawgeti(L, LUA_ENVIRONINDEX, THREAD_PROXY);
+  lua_rawgeti(L, LUA_ENVIRONINDEX, THREAD_SOCKETS);
-  lua_pushthread(L);
+  nse_base(L);
-  lua_gettable(L, -2);
+  lua_rawget(L, -2);
-  if (!lua_isnil(L, -1))
+  if (lua_istable(L, -1))
   {
-    // Thread already has open sockets. Add the new socket to SOCKET_PROXY
+    /* Thread already has a "lock" with open sockets. Place the new socket
-    lua_rawgeti(L, LUA_ENVIRONINDEX, SOCKET_PROXY);
+     * in its sockets table */
-    lua_pushvalue(L, 1);               // socket
+    lua_pushvalue(L, 1);
-    lua_pushvalue(L, -3);              // proxy userdata
-    lua_settable(L, -3);
-    lua_pop(L, 1);                     // SOCKET_PROXY
     lua_pushboolean(L, true);
-  } else if (table_length(L, 2) >= MAX(MAX_PARALLELISM, o.max_parallelism))
+    lua_rawset(L, -3);
+  } else if (table_length(L, 2) <= MAX(MAX_PARALLELISM, o.max_parallelism))
   {
-    // Too many threads have sockets open. Add thread to waiting. The caller
+    /* There is room for this thread to open sockets */
-    // is expected to yield. (see the connect function in luaopen_nsock)
+    nse_base(L);
-    lua_rawgeti(L, LUA_ENVIRONINDEX, CONNECT_WAITING);
+    weak_table(L, 0, 0, "k"); /* weak socket references */
-    lua_pushthread(L);
+    lua_pushvalue(L, 1); /* socket */
     lua_pushboolean(L, true);
-    lua_settable(L, -3);
+    lua_rawset(L, -3); /* add to sockets table */
-    lua_pop(L, 1);                     // CONNECT_WAITING
+    lua_rawset(L, 2); /* add new <Thread, Sockets Table> Pair
-    return lua_yield(L, 0);
+                       * to THREAD_SOCKETS */
   } else
   {
-    // There is room for this thread to open sockets. Make a new proxy userdata
+    /* Too many threads have sockets open. Add thread to waiting. The caller
-    // and add it to the THREAD_PROXY and SOCKET_PROXY tables.
+     * is expected to yield. (see the connect function in luaopen_nsock) */
-    new_proxy(L);
+    lua_rawgeti(L, LUA_ENVIRONINDEX, CONNECT_WAITING);
-    lua_rawgeti(L, LUA_ENVIRONINDEX, THREAD_PROXY);
+    nse_base(L);
-    lua_pushthread(L);
-    lua_pushvalue(L, -3);              // proxy
-    lua_settable(L, -3);
-    lua_pop(L, 1);                     // THREAD_PROXY)
-    lua_rawgeti(L, LUA_ENVIRONINDEX, SOCKET_PROXY);
-    lua_pushvalue(L, 1);               // Socket
-    lua_pushvalue(L, -3);              // proxy
-    lua_settable(L, -3);
-    lua_pop(L, 2);                     // proxy, SOCKET_PROXY
     lua_pushboolean(L, true);
+    lua_settable(L, -3);
+    return nse_yield(L);
   }
+  lua_pushboolean(L, true);
   return 1;
 }
 
-/* void socket_unlock (lua_State *L, int index)
+static void socket_unlock(lua_State * L)
- *
- * index is the location of the userdata on the stack.
- * A socket has been closed or collected, remove it from the SOCKET_PROXY
- * table.
- */
-static void socket_unlock(lua_State * L, int index)
 {
-  lua_pushvalue(L, index);             // socket
+  int top = lua_gettop(L);
-  lua_rawgeti(L, LUA_ENVIRONINDEX, SOCKET_PROXY);
-  lua_pushvalue(L, -2);                // socket
-  lua_pushnil(L);
-  lua_settable(L, -3);
-  lua_pop(L, 2);                       // socket, SOCKET_PROXY
-}
 
+  lua_gc(L, LUA_GCSTOP, 0); /* don't collect threads during iteration */
+
+  lua_rawgeti(L, LUA_ENVIRONINDEX, THREAD_SOCKETS);
+  lua_pushnil(L);
+  while (lua_next(L, -2) != 0)
+  {
+    unsigned open = 0;
+
+    lua_pushnil(L);
+    while (lua_next(L, -2) != 0) /* for each socket */
+    {
+      lua_pop(L, 1); /* pop garbage boolean */
+      if (((struct l_nsock_udata *) lua_touserdata(L, -1))->nsiod != NULL)
+        open++;
+    }
+
+    if (open == 0) /* thread has no open sockets? */
+    {
+      lua_pushvalue(L, -2); /* thread key */
+      lua_pushnil(L);
+      lua_rawset(L, top+1); /* THREADS_SOCKETS */
+
+      lua_rawgeti(L, LUA_ENVIRONINDEX, CONNECT_WAITING);
+      lua_pushnil(L);
+      if (lua_next(L, -2) != 0)
+      {
+        lua_pop(L, 1); /* pop garbage boolean */
+        nse_restore(lua_tothread(L, -1), 0);
+        lua_pushnil(L);
+        lua_rawset(L, -3); /* remove thread from waiting */
+      }
+      lua_pop(L, 1); /* CONNECT_WAITING */
+    }
+
+    lua_pop(L, 1); /* pop sockets table */
+  }
+
+  lua_gc(L, LUA_GCRESTART, 0);
+
+  lua_settop(L, top);
+}
 
 void l_nsock_clear_buf(lua_State * L, l_nsock_udata * udata);
 
@@ -382,36 +385,22 @@ int luaopen_nsock(lua_State * L)
     {NULL, NULL}
   };
 
-  /* Set up an environment for all nsock C functions to share. This is
+  /* Set up an environment for all nsock C functions to share.
-   * especially important to make the THREAD_PROXY, SOCKET_PROXY, and
+   * This table particularly contains the THREAD_SOCKETS and
-   * CONNECT_WAITING tables available. These values can be accessed at the
+   * CONNECT_WAITING tables.
-   * pseudo-index LUA_ENVIRONINDEX. These tables are documented where the
+   * These values are accessed at the Lua pseudo-index LUA_ENVIRONINDEX.
-   * #defines are above. */
+   */
-  lua_createtable(L, 5, 0);
+  lua_createtable(L, 2, 0);
   lua_replace(L, LUA_ENVIRONINDEX);
 
-  lua_createtable(L, 0, 10);           // THREAD_PROXY
+  weak_table(L, 0, MAX_PARALLELISM, "k");
-  lua_createtable(L, 0, 1);            // metatable
+  lua_rawseti(L, LUA_ENVIRONINDEX, THREAD_SOCKETS);
-  lua_pushliteral(L, "kv");            // weak keys and values
-  lua_setfield(L, -2, "__mode");
-  lua_setmetatable(L, -2);
-  lua_rawseti(L, LUA_ENVIRONINDEX, THREAD_PROXY);
 
-  lua_createtable(L, 0, 193);          // SOCKET_PROXY (large amount of room)
+  weak_table(L, 0, 1000, "k");
-  lua_createtable(L, 0, 1);            // metatable
-  lua_pushliteral(L, "k");             // weak keys
-  lua_setfield(L, -2, "__mode");
-  lua_setmetatable(L, -2);
-  lua_rawseti(L, LUA_ENVIRONINDEX, SOCKET_PROXY);
-
-  lua_createtable(L, 0, 499);          // CONNECT_WAITING (large amount of
-                                       // room)
   lua_rawseti(L, LUA_ENVIRONINDEX, CONNECT_WAITING);
 
-  lua_createtable(L, 0, 1);            // PROXY_META = metatable for proxies
+  lua_pushcfunction(L, l_nsock_loop);
-  lua_pushcclosure(L, proxy_gc, 0);
+  lua_setfield(L, LUA_REGISTRYINDEX, NSE_NSOCK_LOOP);
-  lua_setfield(L, -2, "__gc");
-  lua_rawseti(L, LUA_ENVIRONINDEX, PROXY_META);
 
   /* Load the connect function */
   if (luaL_loadstring(L, connect) != 0)
@@ -476,9 +465,16 @@ int l_nsock_new(lua_State * L)
   return 1;
 }
 
-int l_nsock_loop(int tout)
+static int l_nsock_loop(lua_State * L)
 {
-  return nsock_loop(nsp, tout);
+  int tout = luaL_checkint(L, 1);
+
+  /* clean up old socket locks */
+  socket_unlock(L);
+
+  if (nsock_loop(nsp, tout) == NSOCK_LOOP_ERROR)
+    luaL_error(L, "a fatal error occurred in nsock_loop");
+  return 0;
 }
 
 int l_nsock_checkstatus(lua_State * L, nsock_event nse)
@@ -513,26 +509,32 @@ int l_nsock_checkstatus(lua_State * L, nsock_event nse)
 
 static int l_nsock_connect(lua_State * L)
 {
+  enum type {TCP, UDP, SSL};
+  static const char * const op[] = {"tcp", "udp", "ssl", NULL};
+
   l_nsock_udata *udata = (l_nsock_udata *) luaL_checkudata(L, 1, "nsock");
-
   const char *addr = luaL_checkstring(L, 2);
-
   unsigned short port = (unsigned short) luaL_checkint(L, 3);
-
+  int what = luaL_checkoption(L, 4, "tcp", op);
-  const char *how = luaL_optstring(L, 4, "tcp");
 
   const char *error;
-
   struct addrinfo *dest;
-
   int error_id;
 
   l_nsock_clear_buf(L, udata);
 
+#ifndef HAVE_OPENSSL
+  if (what == SSL)
+  {
+    lua_pushboolean(L, false);
+    lua_pushstring(L, "Sorry, you don't have OpenSSL\n");
+    return 2;
+  }
+#endif
+
   error_id = getaddrinfo(addr, NULL, NULL, &dest);
   if (error_id)
   {
-    socket_unlock(L, 1);
     error = gai_strerror(error_id);
     lua_pushboolean(L, false);
     lua_pushstring(L, error);
@@ -540,7 +542,6 @@ static int l_nsock_connect(lua_State * L)
   }
   if (dest == NULL)
   {
-    socket_unlock(L, 1);
     lua_pushboolean(L, false);
     lua_pushstring(L, "getaddrinfo returned success but no addresses");
     return 2;
@@ -559,48 +560,26 @@ static int l_nsock_connect(lua_State * L)
   if (o.ipoptionslen)
     nsi_set_ipoptions(udata->nsiod, o.ipoptions, o.ipoptionslen);
 
-  switch (how[0])
+  switch (what)
   {
-    case 't':
+    case TCP:
-      if (strcmp(how, "tcp"))
-        goto error;
       nsock_connect_tcp(nsp, udata->nsiod, l_nsock_connect_handler,
           udata->timeout, &udata->yield, dest->ai_addr, dest->ai_addrlen, port);
       break;
-    case 'u':
+    case UDP:
-      if (strcmp(how, "udp"))
-        goto error;
       nsock_connect_udp(nsp, udata->nsiod, l_nsock_connect_handler,
           &udata->yield, dest->ai_addr, dest->ai_addrlen, port);
       break;
-    case 's':
+    case SSL:
-      if (strcmp(how, "ssl"))
-        goto error;
-#ifdef HAVE_OPENSSL
       nsock_connect_ssl(nsp, udata->nsiod, l_nsock_connect_handler,
           udata->timeout, &udata->yield, dest->ai_addr, dest->ai_addrlen, port,
           udata->ssl_session);
       break;
-#else
-      socket_unlock(L, 1);
-      lua_pushboolean(L, false);
-      lua_pushstring(L, "Sorry, you don't have OpenSSL\n");
-      return 2;
-#endif
-    default:
-      goto error;
-      break;
   }
 
   freeaddrinfo(dest);
   set_thread(L, 1, udata);
-  return lua_yield(L, 0);
+  return nse_yield(L);
-
-error:
-  socket_unlock(L, 1);
-  freeaddrinfo(dest);
-  luaL_argerror(L, 4, "invalid connection method");
-  return 0;
 }
 
 void l_nsock_connect_handler(nsock_pool nsp, nsock_event nse, void *yield)
@@ -609,6 +588,8 @@ void l_nsock_connect_handler(nsock_pool nsp, nsock_event nse, void *yield)
 
   lua_State *L = y->thread;
 
+  if (lua_status(L) != LUA_YIELD) return;
+
   if (o.scriptTrace())
   {
     l_nsock_trace(nse_iod(nse), "CONNECT", TO);
@@ -647,7 +628,7 @@ static int l_nsock_send(lua_State * L)
   nsock_write(nsp, udata->nsiod, l_nsock_send_handler, udata->timeout,
       &udata->yield, string, string_len);
   set_thread(L, 1, udata);
-  return lua_yield(L, 0);
+  return nse_yield(L);
 }
 
 void l_nsock_send_handler(nsock_pool nsp, nsock_event nse, void *yield)
@@ -656,6 +637,8 @@ void l_nsock_send_handler(nsock_pool nsp, nsock_event nse, void *yield)
 
   lua_State *L = y->thread;
 
+  if (lua_status(L) != LUA_YIELD) return;
+
   if (l_nsock_checkstatus(L, nse) == NSOCK_WRAPPER_SUCCESS)
   {
     nse_restore(y->thread, 1);
@@ -682,7 +665,7 @@ static int l_nsock_receive(lua_State * L)
       &udata->yield);
 
   set_thread(L, 1, udata);
-  return lua_yield(L, 0);
+  return nse_yield(L);
 }
 
 static int l_nsock_receive_lines(lua_State * L)
@@ -704,7 +687,7 @@ static int l_nsock_receive_lines(lua_State * L)
       &udata->yield, nlines);
 
   set_thread(L, 1, udata);
-  return lua_yield(L, 0);
+  return nse_yield(L);
 }
 
 static int l_nsock_receive_bytes(lua_State * L)
@@ -726,7 +709,7 @@ static int l_nsock_receive_bytes(lua_State * L)
       &udata->yield, nbytes);
 
   set_thread(L, 1, udata);
-  return lua_yield(L, 0);
+  return nse_yield(L);
 }
 
 void l_nsock_receive_handler(nsock_pool nsp, nsock_event nse, void *yield)
@@ -735,6 +718,8 @@ void l_nsock_receive_handler(nsock_pool nsp, nsock_event nse, void *yield)
 
   lua_State *L = y->thread;
 
+  if (lua_status(L) != LUA_YIELD) return;
+
   char *rcvd_string;
 
   int rcvd_len = 0;
@@ -900,8 +885,6 @@ static int l_nsock_close(lua_State * L)
 {
   l_nsock_udata *udata = (l_nsock_udata *) luaL_checkudata(L, 1, "nsock");
 
-  socket_unlock(L, 1);                 // Unlock the socket.
-
   /* Never ever collect nse-pcap connections. */
   if (udata->ncap_socket)
   {
@@ -985,13 +968,13 @@ static int l_nsock_receive_buf(lua_State * L)
       /* if we didn't have enough data in the buffer another nsock_read() was
        * scheduled - its callback will put us in running state again */
       set_thread(L, 1, udata);
-      return lua_yield(L, 0);
+      return nse_yield(L);
     }
     return 2;
   }
   /* yielding with 3 arguments since we need them when the callback arrives */
   set_thread(L, 1, udata);
-  return lua_yield(L, 0);
+  return nse_yield(L);
 }
 
 void l_nsock_receive_buf_handler(nsock_pool nsp, nsock_event nse, void *yield)
@@ -1002,6 +985,8 @@ void l_nsock_receive_buf_handler(nsock_pool nsp, nsock_event nse, void *yield)
 
   lua_State *L = y->thread;
 
+  if (lua_status(L) != LUA_YIELD) return;
+
   char *rcvd_string;
 
   int rcvd_len = 0;
@@ -1185,6 +1170,8 @@ static void l_nsock_sleep_handler(nsock_pool nsp, nsock_event nse, void *udata)
 {
   lua_State *L = (lua_State *) udata;
 
+  if (lua_status(L) != LUA_YIELD) return;
+
   assert(nse_status(nse) == NSE_STATUS_SUCCESS);
   nse_restore(L, 0);
 }
@@ -1201,7 +1188,7 @@ int l_nsock_sleep(lua_State * L)
   msecs = (int) (secs * 1000 + 0.5);
   nsock_timer_create(nsp, l_nsock_sleep_handler, msecs, L);
 
-  return lua_yield(L, 0);
+  return nse_yield(L);
 }
 
 /****************** NCAP_SOCKET ***********************************************/
@@ -1527,7 +1514,7 @@ int l_nsock_pcap_receive(lua_State * L)
   /* no data yet? suspend thread */
   nr->suspended = 1;
 
-  return lua_yield(L, 0);
+  return nse_yield(L);
 }
 
 /* (free) excute callback function from lua script */

nse_nsock.h

@@ -3,11 +3,12 @@
 
 int luaopen_nsock(lua_State *);
 int l_nsock_new(lua_State *);
-int l_nsock_loop(int tout);
 int l_nsock_sleep(lua_State *L);
 
 int l_dnet_new(lua_State *);
 int l_dnet_get_interface_link(lua_State *);
 
+#define NSE_NSOCK_LOOP "NSOCK_LOOP"
+
 #endif
 

nselib/comm.lua

@@ -21,6 +21,9 @@
 
 module(... or "comm", package.seeall)
 
+require 'nsedebug'
+require 'datafiles'
+
 -- Makes sure that opts exists and the default proto is there
 local initopts = function(opts)
 	if not opts then
@@ -91,20 +94,10 @@ end
 -- @return Data (if status is true) or error string (if status is false).
 get_banner = function(host, port, opts)
 	opts = initopts(opts)
-
+	opts.recv_before = true
-	local status, sock = setup_connect(host, port, opts)
+	local socket, nothing, correct, banner = tryssl(host, port, "", opts)
-	local ret
+	if socket then return true, banner end
-
+	return false, banner
-	if not status then
-		-- sock is an error message in this case
-		return status, sock
-	end
-
-	status, ret = read(sock, opts)
-
-	sock:close()
-
-	return status, ret
 end
 
 --- This function connects to the specified port number on the specified
@@ -144,3 +137,131 @@ exchange = function(host, port, data, opts)
 	return status, ret
 end
 
+--- This function just checks if the provided port number is on a list
+-- of ports that usually provide services with ssl
+--
+-- @param port_number The number of the port to check
+-- @return bool True if port is usually ssl, otherwise false
+local function is_ssl(port_number)
+	local common_ssl_ports = {22, 443, 465, 995, 587, 6697, 6679, 8443}
+	local table_size = table.maxn(common_ssl_ports)	
+	local i = 0
+	while i < table_size do
+		if port_number == common_ssl_ports[i] then return true end
+		i = i + 1
+	end
+	return false
+end
+
+--- This function returns best protocol order for trying  to open a 
+-- conenction based on port and service information
+--
+-- The first value is the best option, the second is the worst
+-- @param port The port table
+-- @return Best option ("tcp" or "ssl")
+-- @return Worst option ("tcp" or "ssl")
+local function bestoption(port)
+	if type(port) == 'table' then
+		if port.version and port.version.service_tunnel and port.version.service_tunnel == "ssl" then return "ssl","tcp" end
+		if port.version and port.version.name_confidence and port.version.name_confidence > 6 then return "tcp","ssl" end
+		if is_ssl(port.number) then return "ssl","tcp" end
+	elseif type(port) == 'number' then
+		if is_ssl(port) then return "ssl","tcp" end
+	end
+	return "tcp","ssl"
+end
+
+--- This function opens a connection, sends the first data payload and
+--  check if a response is correctly received (what means that the 
+--  protocol used is fine)
+--
+-- Possible options:
+-- timeout: generic timeout value
+-- connect_timeout: especific timeout for connection
+-- request_timeout: especific timeout for requests
+-- recv_before: receive data before sending first payload
+--
+-- Default timeout is set to 8000.
+--
+-- @param ip The destination host IP
+-- @param port The destination host port
+-- @param protocol The protocol for the connection
+-- @param data The first data payload of the connection
+-- @return sd The socket descriptor, nil if no connection is estabilished
+-- @return response The response received for the payload
+-- @return early_resp If opt recv_before is true, returns the value
+-- of the first receive (before sending data)
+local function opencon(ip, port, protocol, data, opts)
+	local sd = nmap.new_socket()
+
+	-- check for connect_timeout or timeout option
+
+	if opts and opts.connect_timeout then 
+		sd:set_timeout(opts.connect_timeout)
+	elseif opts and opts.timeout then
+		sd:set_timeout(opts.timeout)
+	else
+		sd:set_timeout(8000)
+	end
+
+	local status = sd:connect(ip, port, protocol)
+	if not status then return nil, nil, nil end
+
+	-- check for request_timeout or timeout option
+
+	if opts and opts.request_timeout then
+		sd:set_timeout(opts.request_timeout)
+	elseif opts and opts.timeout then
+		sd:set_timeout(opts.timeout)
+	else
+		sd:set_timeout(8000)
+	end
+
+    local response, early_resp;
+	if opts and opts.recv_before then status, early_resp = sd:receive() end
+	if #data > 0 then
+		sd:send(data)
+		status, response = sd:receive()
+	else
+		if not opts and opts.recv_before then
+			nsedebug.print_debug("Using comm.tryssl without first data payload and recv_first." .. 
+					     "\nImpossible to test the connection for the correct protocol!")
+		end
+		response = early_resp
+	end
+	if not status then return nil, response, early_resp end
+	return sd, response, early_resp
+end
+
+--- This function tries to open a connection based on the best
+--  option about which is the correct protocol
+--
+--  If the best option fails, the function tries the other option
+--
+--  This function allows writing nse scripts in a way that the
+--  API will take care of ssl issues, making failure detection
+--  transparent to the programmer
+--
+-- @param host The host table
+-- @param port The port table
+-- @param sslservice The name of the ssl service version
+-- @param data The first data payload of the connection
+-- @param opts Options, such as timeout
+-- @return sd The socket descriptor
+-- @return response The response received for the payload
+-- @return correctOpt Correct option for connection guess
+-- @return earlyResp If opt recv_before is true, returns the value
+-- of the first receive (before sending data)
+function tryssl(host, port, data, opts)
+	local opt1, opt2 = bestoption(port)
+	if type(port) == 'table' then port = port.number end	
+	if type(host) == 'table' then host = host.ip end
+	local best = opt1
+	local sd, response, early_resp = opencon(host, port, opt1, data, opts)
+	if not sd then
+		sd, response, early_resp = opencon(host, port, opt2, data, opts)
+		best = opt2
+	end
+	if not sd then best = "none" end
+	return sd, response, best, early_resp
+end

nselib/pop3.lua

@@ -8,6 +8,7 @@ local HAVE_SSL = false
 require 'base64'
 require 'bit'
 require 'stdnse'
+require 'comm'
 
 if pcall(require,'openssl') then
   HAVE_SSL = true
@@ -148,21 +149,21 @@ end
 function capabilities(host, port)
    local socket = nmap.new_socket()
    local capas = {}
-   socket:set_timeout(10000)
+   local opts = {timeout=10000, recv_before=true}
-   local proto = (port.version and port.version.service_tunnel == "ssl" and "ssl") or "tcp"
+   local i = 1
-   if not socket:connect(host.ip, port.number, proto) then return nil, "Could Not Connect" end
 
-   status, line = socket:receive_lines(1)
+   socket, line, bopt, first_line = comm.tryssl(host, port, "CAPA\r\n" , opts)
-   if not stat(line) then return nil, "No Response" end
+   if not socket then return nil, "Could Not Connect" end
+   if not stat(first_line) then return nil, "No Response" end
+  
+   if string.find(first_line, "<[%p%w]+>") then capas.APOP = true end
    
-   if string.find(line, "<[%p%w]+>") then capas.APOP = true end
+   lines = stdnse.strsplit("\r\n",line)
-   
+   line = lines[1]
-   socket:send("CAPA\r\n")
+
-   status, line = socket:receive_buf("\r\n", false)
    if not stat(line) then 
       capas.capa = false
    else 
-      status, line = socket:receive_buf("\r\n", false)
       while line do
 	 if line ~= "." then
 	    local capability = string.sub(line, string.find(line, "[%w-]+"))
@@ -179,7 +180,8 @@ function capabilities(host, port)
 	 else
 	    break 
 	 end
-	 status, line = socket:receive_buf("\r\n", false)
+	 line = lines[i]
+	 i = i + 1
       end
    end
    socket:close()

nselib/ssh1.lua

@@ -6,11 +6,45 @@
 
 module(... or "ssh1",package.seeall)
 
-local bin = require "bin"
+require "bin"
-local bit = require "bit"
+require "bit"
-local math = require "math"
+require "math"
-local stdnse = require "stdnse"
+require "stdnse"
-local openssl = require "openssl"
+require "openssl"
+
+--- Retrieve the size of the packet that is being received
+--  and checks if it is fully received
+-- 
+--  This function is very similar to the function generated
+--  with match.numbytes(num) function, except that this one
+--  will check for the number of bytes on-the-fly, based on
+--  the written on the SSH packet.
+--
+--  @param buffer The receive buffer
+--  @return packet_length, packet_length or nil
+--  the return is similar to the lua function string:find()
+check_packet_length = function( buffer )
+  local payload_length, packet_length, offset
+  offset, payload_length = bin.unpack( ">I", buffer )
+  padding = 8 - payload_length % 8
+  assert(payload_length)
+  packet_length = buffer:len()
+  if payload_length + 4 + padding > packet_length then return nil end
+  return packet_length, packet_length
+end
+
+--- Receives a complete SSH packet, even if fragmented
+--  this function is an abstraction layer to deal with
+--  checking the packet size to know if there is any more
+--  data to receive.
+--
+--  @param socket The socket used to receive the data
+--  @return status True or false
+--  @return packet The packet received
+receive_ssh_packet = function( socket )
+  status, packet = socket:receive_buf(check_packet_length)
+  return status, packet
+end
 
 --- Fetch an SSH-1 host key.
 -- @param host Nmap host table.
@@ -33,7 +67,7 @@ fetch_host_key = function(host, port)
   if not status then socket:close(); return end
 
   local data, packet_length, padding, offset
-  status,data = socket:receive()
+  status,data = receive_ssh_packet( socket )
   socket:close()
   if not status then return end
 

nselib/ssh2.lua

@@ -15,6 +15,38 @@ transport = {}
 -- table of SSH-2 constants
 local SSH2
 
+--- Retrieve the size of the packet that is being received
+--  and checks if it is fully received
+-- 
+--  This function is very similar to the function generated
+--  with match.numbytes(num) function, except that this one
+--  will check for the number of bytes on-the-fly, based on
+--  the written on the SSH packet.
+--
+--  @param buffer The receive buffer
+--  @return packet_length, packet_length or nil
+--  the return is similar to the lua function string:find()
+check_packet_length = function( buffer )
+  local packet_length, offset
+  offset, packet_length = bin.unpack( ">I", buffer )
+  assert(packet_length)
+  if packet_length + 4 > buffer:len() then return nil end
+  return packet_length, packet_length
+end
+
+--- Receives a complete SSH packet, even if fragmented
+--  this function is an abstraction layer to deal with
+--  checking the packet size to know if there is any more
+--  data to receive.
+--
+--  @param socket The socket used to receive the data
+--  @return status True or false
+--  @return packet The packet received
+transport.receive_packet = function( socket )
+  status, packet = socket:receive_buf(check_packet_length)
+  return status, packet
+end
+
 --- Pack a multiprecision integer for sending.
 -- @param bn <code>openssl</code> bignum.
 -- @return Packed multiprecision integer.
@@ -47,9 +79,8 @@ transport.payload = function( packet )
   offset, packet_length, padding_length = bin.unpack( ">Ic", packet )
   assert(packet_length and padding_length)
   payload_length = packet_length - padding_length - 1
-  -- Add 4 for the packet_length field.
+  if packet_length ~= packet:len() then
-  if packet_length + 4 > packet:len() then
+    stdnse.print_debug("SSH-2 packet doesn't match length: payload_length is %d but total length is only %d.", packet_length, packet:len())
-    stdnse.print_debug("SSH-2 packet too short: payload_length is %d but total length is only %d.", packet_length, packet:len())
     return nil
   end
   offset, payload = bin.unpack( ">A" .. payload_length, packet, offset )
@@ -136,7 +167,7 @@ fetch_host_key = function( host, port, key_type )
   if not status then socket:close(); return end
 
   local kex_init
-  status, kex_init = socket:receive_bytes(1)
+  status, kex_init = transport.receive_packet( socket )
   if not status then socket:close(); return end
   kex_init = transport.parse_kex_init( transport.payload( kex_init ) )
 
@@ -158,7 +189,7 @@ fetch_host_key = function( host, port, key_type )
   if not status then socket:close(); return end
 
   local kexdh_reply
-  status, kexdh_reply = socket:receive_bytes(1)
+  status, kexdh_reply = transport.receive_packet( socket )
   kexdh_reply = transport.payload( kexdh_reply )
   -- check for proper msg code
   if kexdh_reply:byte(1) ~= SSH2.SSH_MSG_KEXDH_REPLY then

scripts/http-open-proxy.nse

@@ -51,10 +51,8 @@ require "url"
 --@param result connection result
 --@return true if any of the status is found, otherwise false
 function check_code(result)
-	local status = false
+	if string.match(result:lower(),"^http/%d\.%d%s*200") then return true end
-	if string.match(result:lower(),"^http.*200.*") then return true end
+	if string.match(result:lower(),"^http/%d\.%d%s*30[12]") then return true end
-	if string.match(result:lower(),"^http.*301.*") then return true end	
-	if string.match(result:lower(),"^http.*302.*") then return true end
 	return false
 end
 
@@ -63,9 +61,9 @@ end
 --@param pattern The pattern to be searched
 --@return true if pattern is found, otherwise false
 function check_pattern(result, pattern)
-	lines = stdnse.strsplit("\n", result)
+	local lines = stdnse.strsplit("\n", result)
-	i = 1
+	local i = 1
-	n = table.getn(lines)
+	local n = table.getn(lines)
 	while true do
 		if i > n then return false end
 		if string.match(lines[i]:lower(),pattern) then return true end
@@ -90,14 +88,9 @@ end
 portrule = shortport.port_or_service({8123,3128,8000,8080},{'polipo','squid-http','http-proxy'})
 
 action = function(host, port)
-	local response
-	local i
 	local retval
-	local supported_methods = "\nMethods succesfully tested: "
+	local supported_methods = "\nMethods successfully tested: "
 	local fstatus = false
-
-	-- Default url = nmap.org
-	-- Default host = nmap.org
 	local test_url = "http://www.google.com"
 	local hostname = "www.google.com"
 	local pattern = "^server: gws"

scripts/irc-info.nse

@@ -22,8 +22,10 @@ categories = {"default", "discovery"}
 
 require("stdnse")
 require "shortport"
+require("nsedebug")
+require("comm")
 
-portrule = shortport.port_or_service({6666,6667},"irc")
+portrule = shortport.port_or_service({6666,6667,6697,6679},{"irc","ircs"})
 
 init = function()
   -- Start of MOTD, we'll take the server name from here
@@ -71,10 +73,6 @@ init = function()
     or pcre.new("^ERROR :(.*)", 0, "C")
 end
 
-
-
-
-
 action = function(host, port)
   local sd = nmap.new_socket()
   local curr_nick = random_nick()
@@ -119,14 +117,12 @@ action = function(host, port)
 
   init()
 
-  sd:connect(host.ip, port.number)
+  sd, line = comm.tryssl(host, port, "USER nmap +iw nmap :Nmap Wuz Here\nNICK " .. curr_nick .. "\n")
-
+  if not sd then return "Unable to open connection" end
-  sd:send("USER nmap +iw nmap :Nmap Wuz Here\nNICK " .. curr_nick .. "\n")
 
   buf = stdnse.make_buffer(sd, "\r?\n")
 
   while true do
-    local line = buf()
     if (not line) then break end
 
     -- This one lets us know we've connected, pre-PONGed, and got a NICK
@@ -203,6 +199,7 @@ action = function(host, port)
       return make_output()
     end
 
+    line = buf()
   end
 
   return make_output()

scripts/pop3-brute.nse

@@ -16,7 +16,7 @@ require 'pop3'
 require 'shortport'
 require 'unpwdb'
 
-portrule = shortport.port_or_service({110}, "pop3")
+portrule = shortport.port_or_service({110, 995}, {"pop3","pop3s"})
 
 action = function(host, port)
    local pMeth = nmap.registry.args.pop3loginmethod
@@ -45,14 +45,15 @@ action = function(host, port)
    local status
    local line
    local socket = nmap.new_socket()
-
+   local opts = {timeout=10000, recv_before=true}
-   if not socket:connect(host.ip, port.number) then return end -- no connection
    
-   status, line = socket:receive_lines(1)
+   socket, nothing, bopt, line = comm.tryssl(host, port, "" , opts)
+
+   if not socket then return end -- no connection 
    if not stat(line) then return end -- no pop-connection
 
    local apopChallenge = string.match(line, "<[%p%w]+>") 
-   
+  
    if pMeth == "APOP" then 
       additional = apopChallenge 
    end
@@ -62,6 +63,7 @@ action = function(host, port)
    status, getUser = unpwdb.usernames()
    if (not status) then return end
 
+
    local currUser = getUser()
    while currUser do
       local getPW
@@ -83,8 +85,13 @@ action = function(host, port)
 	 elseif (perror == pop3.err.userError) then
 	    currPw = nil
 	 else
-	    return
+            socstatus = socket:connect(host.ip, port.number, bopt)
-	 end
+	    if not socstatus 
+	       then return
+               else recvstatus, line = socket:receive()
+                    if not stat(line) then return end -- no connection
+            end
+         end
       end
       currUser = getUser()
       getPW("reset")

scripts/pop3-capabilities.nse

@@ -15,13 +15,13 @@ server version may be available.
 author = "Philip Pickering <pgpickering@gmail.com>"
 license = "Same as Nmap--See http://nmap.org/book/man-legal.html"
 
-categories = {"default"}
+categories = {"default","discovery"}
 
 require 'pop3'
 require 'shortport'
 require 'stdnse'
 
-portrule = shortport.port_or_service({110}, "pop3")
+portrule = shortport.port_or_service({110,995},{"pop3","pop3s"})
 
 action = function(host, port)
   local capa, err = pop3.capabilities(host, port)

scripts/smb-enum-processes.nse

@@ -1,19 +1,19 @@
 description = [[
 Pulls a list of processes from the remote server over SMB. This will determine
 all running processes, their process IDs, and their parent processes. It is done
-by querying the remote registry service, which is disabled by default on Vista; on 
+by querying the remote registry service, which is disabled by default on Vista;
-all other Windows versions, it requires Administrator privilges. 
+on all other Windows versions, it requires Administrator privileges.
 
-Since this requires administrator privileges, it isn't especially useful for a 
+Since this requires administrator privileges, it isn't especially useful for a
 penetration tester, since they can effectively do the same thing with metasploit
-or other tools. It does, however, provide for a quick way to get process lists 
+or other tools. It does, however, provide for a quick way to get process lists
-for a bunch of systems at the same time. 
+for a bunch of systems at the same time.
 
-WARNING: I have experienced crashes in regsvc.exe while making registry calls 
+WARNING: I have experienced crashes in regsvc.exe while making registry calls
-against a fully patched Windows 2000 system; I've fixed the issue that caused it,
+against a fully patched Windows 2000 system; I've fixed the issue that caused
-but there's no guarantee that it (or a similar vuln in the same code) won't show 
+it, but there's no guarantee that it (or a similar vuln in the same code) won't
-up again. Since the process automatically restarts, it doesn't negatively impact 
+show up again. Since the process automatically restarts, it doesn't negatively
-the system, besides showing a message box to the user. 
+impact the system, besides showing a message box to the user.
 ]]
 
 ---
@@ -24,167 +24,174 @@ the system, besides showing a message box to the user.
 ---
 -- @output
 -- Host script results:
--- |  smb-enum-processes:  
+-- |  smb-enum-processes:
--- |  -+-Idle(0)---System(8)---SMSS(140)-+-WINLOGON(160)-+-SERVICES(212)-+-spoolsv(432)
+-- |_ Idle, System, smss, csrss, winlogon, services, logon.scr, lsass, spoolsv, msdtc, VMwareService, svchost, alg, explorer, VMwareTray, VMwareUser, wmiprvse
--- |   |                                 |               |               +-mstask(536)
+--
--- |   |                                 |               |               +-WinMgmt(592)
--- |   |                                 |               |               +-svchost(620)
--- |   |                                 |               |               `-regsvc(1136)
--- |   |                                 |               `-LSASS(224)
--- |   |                                 `-CSRSS(164)
--- |   +-Unknown(296)---explorer(344)-+-firefox(636)---WinRAR(736)
--- |   |                              +-keyfinder(848)
--- |   |                              `-CMD(956)
--- |   +-Unknown(400)---IEXPLORE(1036)
--- |_  `-Unknown(840)---DRWTSN32(1192)
--- 
 -- --
 -- Host script results:
--- |  smb-enum-processes:  
+-- |  smb-enum-processes:
--- |  Idle [0] (parent: 0, priority: 0, threads: 1, handles: 0)
+-- |  `+-Idle
--- |  System [8] (parent: 0, priority: 8, threads: 34, handles: 190)
+-- |   | `-System
--- |  smss [140] (parent: 8, priority: 11, threads: 6, handles: 33)
+-- |   |   `-smss
--- |  winlogon [160] (parent: 140, priority: 13, threads: 14, handles: 335)
+-- |   |     `+-csrss
--- |  csrss [164] (parent: 140, priority: 13, threads: 10, handles: 229)
+-- |   |      `-winlogon
--- |  services [212] (parent: 160, priority: 9, threads: 33, handles: 462)
+-- |   |        `+-services
--- |  lsass [224] (parent: 160, priority: 9, threads: 13, handles: 267)
+-- |   |         | `+-spoolsv
--- |  SPOOLSV [412] (parent: 212, priority: 8, threads: 10, handles: 95)
+-- |   |         |  +-msdtc
--- |  svchost [448] (parent: 212, priority: 8, threads: 24, handles: 369)
+-- |   |         |  +-VMwareService
--- |  mstask [516] (parent: 212, priority: 8, threads: 6, handles: 89)
+-- |   |         |  +-svchost
--- |  VMwareService.e [572] (parent: 212, priority: 13, threads: 4, handles: 95)
+-- |   |         |  `-alg
--- |  winmgmt [648] (parent: 212, priority: 8, threads: 3, handles: 89)
+-- |   |         +-logon.scr
--- |  cmd [700] (parent: 212, priority: 8, threads: 1, handles: 28)
+-- |   |         `-lsass
--- |  explorer [720] (parent: 620, priority: 8, threads: 10, handles: 239)
+-- |   +-explorer
--- |  VMwareUser [748] (parent: 720, priority: 8, threads: 1, handles: 30)
+-- |   | `+-VMwareTray
--- |  VMwareTray [764] (parent: 720, priority: 8, threads: 1, handles: 30)
+-- |   |  `-VMwareUser
--- |_ regsvc [868] (parent: 212, priority: 8, threads: 4, handles: 76)
+-- |_  `-wmiprvse
+--
+-- --
+-- Host script results:
+-- |  smb-enum-processes:
+-- |   PID  PPID  Priority Threads Handles
+-- |  ----- ----- -------- ------- -------
+-- |      0     0        0       1       0 `+-Idle
+-- |      4     0        8      49     395  | `-System
+-- |    252     4       11       3      19  |   `-smss
+-- |    300   252       13      10     338  |     `+-csrss
+-- |    324   252       13      18     513  |      `-winlogon
+-- |    372   324        9      16     272  |        `+-services
+-- |    872   372        8      12     121  |         | `+-spoolsv
+-- |    896   372        8      13     151  |         |  +-msdtc
+-- |   1172   372       13       3      53  |         |  +-VMwareService
+-- |   1336   372        8      20     158  |         |  +-svchost
+-- |   1476   372        8       6      90  |         |  `-alg
+-- |    376   324        4       1      22  |         +-logon.scr
+-- |    384   324        9      23     394  |         `-lsass
+-- |   1720  1684        8       9     259  +-explorer
+-- |   1796  1720        8       1      42  | `+-VMwareTray
+-- |   1808  1720        8       1      44  |  `-VMwareUser
+-- |_  1992   580        8       7     179  `-wmiprvse
 -----------------------------------------------------------------------
 
 author = "Ron Bowes"
 copyright = "Ron Bowes"
 license = "Same as Nmap--See http://nmap.org/book/man-legal.html"
-categories = {"discovery","intrusive"}
+categories = {"discovery", "intrusive"}
 
 require "bin"
-require 'msrpc'
+require "msrpc"
-require 'msrpcperformance'
+require "msrpcperformance"
-require 'smb'
+require "smb"
-require 'stdnse'
+require "stdnse"
-
--- Strings used to separate processes from one another.
-local separators = {
-	first	= "-+-";
-	last	= " `-";
-	middle	= " +-";
-	only 	= "---";
-}
-
-function psl_add (psl, ps)
-	-- Add process.
-	psl[ps.pid] = ps
-
-	-- Add dummy parent if no real one exists.
-	if psl[ps.ppid] == nil then
-		psl[ps.ppid] = {
-			name	= 'Unknown';
-			pid	= ps.ppid;
-			ppid	= ps.ppid;
-		}
-	end
-end
 
 function psl_mode (list, i)
 	local mode
 
 	-- Decide connector for process.
-	if table.maxn(list) == 1 then
+	if #list == 1 then
 		mode = "only"
 	elseif i == 1 then
 		mode = "first"
-	elseif i == table.maxn(list) then
+	elseif i < #list then
-		mode = "last"
-	else
 		mode = "middle"
+	else
+		mode = "last"
 	end
 
 	return mode
 end
 
-function psl_print (psl)
+function psl_print (psl, lvl)
+	-- Print out table header.
 	local result = ""
+	if lvl == 2 then
+		result = result .. " PID  PPID  Priority Threads Handles\n"
+		result = result .. "----- ----- -------- ------- -------\n"
+	end
 
 	-- Find how many root processes there are.
 	local roots = {}
-	for i,ps in pairs(psl) do
+	for i, ps in pairs(psl) do
 		if psl[ps.ppid] == nil or ps.ppid == ps.pid then
 			table.insert(roots, i)
 		end
 	end
 	table.sort(roots)
 
-	-- Create vertical sibling link.
+	-- Create vertical sibling bars.
 	local bars = {}
-	if table.maxn(roots) ~= 1 then
+	if #roots ~= 1 then
 		table.insert(bars, 2)
 	end
 
 	-- Print out each root of the tree.
-	for i,root in ipairs(roots) do
+	for i, root in ipairs(roots) do
 		local mode = psl_mode(roots, i)
-		result = result .. psl_tree(psl, root, 0, bars, mode)
+		result = result .. psl_tree(psl, root, 0, bars, mode, lvl)
 	end
 
 	return result
 end
 
-function psl_tree (psl, pid, column, bars, mode)
+function psl_tree (psl, pid, column, bars, mode, lvl)
 	local ps = psl[pid]
 
 	-- Delete vertical sibling link.
-	if mode == 'last' then
+	if mode == "last" then
 		table.remove(bars)
 	end
 
-	-- Print lead-in.
+	-- Print information table.
-	local prefix = ''
+	local info = ""
-	if mode == 'middle' or mode == 'last' then
+	if lvl == 2 then
-		prefix = '\n'
+		info = info .. string.format("% 5d ", ps.pid)
+		info = info .. string.format("% 5d ", ps.ppid)
+		info = info .. string.format("% 8d ", ps.prio)
+		info = info .. string.format("% 7d ", ps.thrd)
+		info = info .. string.format("% 7d ", ps.hndl)
+	end
 
-		local i = 1
+	-- Print vertical sibling bars.
-		for j = 1, column do
+	local prefix = ""
-			if table.maxn(bars) >= i and
+	local i = 1
-				bars[i] == j then
+	for j = 1, column do
-				prefix = prefix .. '|'
+		if bars[i] == j then
-				i = i + 1
+			prefix = prefix .. "|"
-			else
+			i = i + 1
-				prefix = prefix .. ' '
+		else
-			end
+			prefix = prefix .. " "
 		end
 	end
 
-	-- Format process itself.
+	-- Strings used to separate processes from one another.
-	output = separators[mode] .. ps.name .. '(' .. ps.pid .. ')'
+	local separators = {
-	column = column + #output
+		first	= "`+-";
-	local result = prefix .. output
+		last	= " `-";
+		middle	= " +-";
+		only	= "`-";
+	}
 
-	-- Find process' children.
+	-- Format process itself.
+	local result = "\n" .. info .. prefix .. separators[mode] .. ps.name
+
+	-- Find children of the process.
 	local children = {}
-	for child_pid,child in pairs(psl) do
+	for child_pid, child in pairs(psl) do
 		if child_pid ~= pid and child.ppid == pid then
 			table.insert(children, child_pid)
 		end
 	end
 	table.sort(children)
 
-	-- Create vertical sibling link between children.
+	-- Add vertical sibling link between children.
-	if table.maxn(children) > 1 then
+	column = column + #separators[mode]
+	if #children > 1 then
 		table.insert(bars, column + 2)
 	end
 
-	-- Format process' children.
+	-- Format process's children.
-	for i,pid in ipairs(children) do
+	for i, pid in ipairs(children) do
 		local mode = psl_mode(children, i)
-		result = result .. psl_tree(psl, pid, column, bars, mode)
+		result = result .. psl_tree(psl, pid, column, bars, mode, lvl)
 	end
 
 	return result
@@ -195,15 +202,12 @@ hostrule = function(host)
 end
 
 action = function(host)
-
+	local process, response, result, status
-	local status, result
-	local process
-	local response = " \n"
 
 	-- Get the process list
 	status, result = msrpcperformance.get_performance_data(host, "230")
-	if(status == false) then
+	if status == false then
-		if(nmap.debugging() > 0) then
+		if nmap.debugging() > 0 then
 			return "ERROR: " .. result
 		else
 			return nil
@@ -211,61 +215,76 @@ action = function(host)
 	end
 
 	-- Get the process table
-	process = result['Process']
+	process = result["Process"]
 
---	for i, v in pairs(result['Processor']['_Total']) do
+	-- Put the processes into an array, and sort them by pid.
---		io.write(string.format("i = %s\n", i))
---	end
-
-	-- Put the processes into an array, and sort them by process id
 	local names = {}
 	for i, v in pairs(process) do
-		if(i ~= '_Total') then
+		if i ~= "_Total" then
 			names[#names + 1] = i
 		end
 	end
-	table.sort(names, function (a, b) return process[a]['ID Process'] < process[b]['ID Process'] end)
+	table.sort(names, function (a, b) return process[a]["ID Process"] < process[b]["ID Process"] end)
 
-	-- Put the processes into an array indexed by process id and with a value equal to the name (so we can look it up 
+	-- Put the processes into an array indexed by pid and with a value equal
-	-- easily when we need to)
+        -- to the name (so we can look it up easily when we need to).
 	local process_id = {}
 	for i, v in pairs(process) do
-		process_id[v['ID Process']] = i
+		process_id[v["ID Process"]] = i
 	end
 
-
+	-- Fill the process list table.
-	if(nmap.verbosity() == 1) then
+	--
-		local psl = {}
+	-- Used fields:
-		for i,name in ipairs(names) do
+	--   Creating Process ID
-			if(name ~= '_Total') then
+	--   Handle Count
-				psl_add(psl, {
+	--   ID Process
-					name	= name;
+	--   Priority Base
-					pid	= process[name]['ID Process'];
+	--   Thread Count
-					ppid	= process[name]['Creating Process ID'];
+	--
-				})
+	-- Unused fields:
-			end
+	--   % Privileged Time
+	--   % Processor Time
+	--   % User Time
+	--   Elapsed Time
+	--   IO Data Bytes/sec
+	--   IO Data Operations/sec
+	--   IO Other Bytes/sec
+	--   IO Other Operations/sec
+	--   IO Read Bytes/sec
+	--   IO Read Operations/sec
+	--   IO Write Bytes/sec
+	--   IO Write Operations/sec
+	--   Page Faults/sec
+	--   Page File Bytes
+	--   Page File Bytes Peak
+	--   Pool Nonpaged Bytes
+	--   Pool Paged Bytes
+	--   Private Bytes
+	--   Virtual Bytes
+	--   Virtual Bytes Peak
+	--   Working Set
+	--   Working Set Peak
+	local psl = {}
+	for i, name in ipairs(names) do
+		if name ~= "_Total" then
+			psl[process[name]["ID Process"]] = {
+				name	= name;
+				pid	= process[name]["ID Process"];
+				ppid	= process[name]["Creating Process ID"];
+				prio	= process[name]["Priority Base"];
+				thrd	= process[name]["Thread Count"];
+				hndl	= process[name]["Handle Count"];
+			}
 		end
-		response = ' \n' .. psl_print(psl)
+	end
-	elseif(nmap.verbosity() > 1) then
-		for i = 1, #names, 1 do
-			local name = names[i]
-			if(name ~= '_Total') then
-				local parent = process_id[process[name]['Creating Process ID']]
-				if(parent == nil) then
-					parent = "n/a"
-				end
 
---				response = response .. string.format("%6d %24s (Parent: %24s, Priority: %4d, Threads: %4d, Handles: %4d)\n", process[name]['ID Process'], name, parent, process[name]['Priority Base'], process[name]['Thread Count'], process[name]['Handle Count'])
+	-- Produce final output.
-
+	if nmap.verbosity() == 0 then
-				response = response .. string.format("%s [%d] (parent: %s, priority: %s, threads: %s, handles: %s)\n", name, process[name]['ID Process'], process[name]['Creating Process ID'], process[name]['Priority Base'], process[name]['Thread Count'], process[name]['Handle Count'])
-			end
-			
-		end
-	else
 		response = stdnse.strjoin(", ", names)
+	else
+		response = " \n" .. psl_print(psl, nmap.verbosity())
 	end
 
 	return response
 end
-
-

scripts/smtp-commands.nse

@@ -52,6 +52,7 @@ categories = {"default", "discovery", "safe"}
 
 require "shortport"
 require "stdnse"
+require "comm"
 
 portrule = shortport.port_or_service({25, 587, 465}, "smtp")
 
@@ -70,29 +71,25 @@ action = function(host, port)
 	end
 	
 	local try = nmap.new_try(catch)
+
+	opt = {timeout=4000, recv_before=true}
 	
-	local proto = (port.version and port.version.service_tunnel == "ssl" and "ssl") or port.protocol
+	socket = comm.tryssl(host, port, "\n", opt)
-	local attempt = try(socket:connect(host.ip, port.number, proto))
+	if not socket then
-	if attempt then
+		stdnse.print_debug("Problem connecting to " .. host.ip .. " on port " .. port.number .. " using ssl and tcp protocols.")
-		if nmap.verbosity() >= 2 or nmap.debugging() >= 1 then -- only tell you it fails if you are debugging or verbose X 2
+		return
-			return ("Problem connecting to %s on port %d using protocol %s%s"):format(host.ip, port.number, port.protocol, (proto == "ssl" and " (ssl)") or "")
-		else
-			return -- if you aren't debugging, just return with nothing
-		end
 	end
-	
+
-	result = try(socket:receive_lines(1))
-	
 	local query = "EHLO example.org\r\n"
 	try(socket:send(query))
 	resultEHLO = try(socket:receive_lines(1))
-   
+
 	if not (string.match(resultEHLO, "^250")) then
 --		stdnse.print_debug("1","%s",resultEHLO)
 --		stdnse.print_debug("1","EHLO with errors or timeout.  Enable --script-trace to see what is happening.")
 		resultEHLO = ""
 	end
-	
+
 	if resultEHLO ~= "" then
 		
 		resultEHLO = string.gsub(resultEHLO, "250 OK[\r\n]", "") -- 250 OK (needed to have the \r\n in there)
@@ -111,7 +108,7 @@ action = function(host, port)
 	local query = "HELP\r\n"
 	try(socket:send(query))
 	resultHELP = try(socket:receive_lines(1))
-	
+
 	if not (string.match(resultHELP, "^214")) then
 --		stdnse.print_debug("1","%s",resultHELP)
 --		stdnse.print_debug("1","HELP with errors or timeout.  Enable --script-trace to see what is happening.")
@@ -126,7 +123,7 @@ action = function(host, port)
 		resultHELP = string.gsub(resultHELP, "%s+", " ") -- get rid of extra spaces
 		resultHELP = "\nHELP " .. resultHELP
 	end
-	
+
 	result = resultEHLO .. resultHELP
 	
 	socket:close()

scripts/smtp-open-relay.nse

@@ -17,10 +17,11 @@ Checks if an SMTP server is an open relay.
 categories = {"demo"}
 
 require "shortport"
+require "comm"
 
 ourdomain="scanme.org"
 
-portrule = shortport.port_or_service(25, "smtp")
+portrule = shortport.port_or_service({25, 465, 587}, {"smtp", "smtps"})
 
 action = function(host, port)
 	local socket = nmap.new_socket()
@@ -31,20 +32,17 @@ action = function(host, port)
 	local tor = {}
 	local i
 
-	socket:set_timeout(10000);
+	opt = {timeout=10000, recv_before=true}
-	socket:connect(host.ip, port.number, port.protocol)
+	socket, result = comm.tryssl(host, port, "EHLO " ..ourdomain.."\r\n", opt)
-	
+	if not socket then
-	status, result = socket:receive_lines(1)
+		return "Unable to estabilish connection"
+	end
 
 	if (result == "TIMEOUT") then
 		socket:close()
 		return "Timeout. Try incresing settimeout, or enhance this."
 	end
 
--- Introduce ourselves...
-	socket:send("EHLO "..ourdomain.."\r\n")
-	status, result = socket:receive_lines(1)
-
 -- close socket and return if there's an smtp status code != 250
 	if not string.match(result, "^250") then
 		socket:close()

scripts/sql-injection.nse

@@ -18,6 +18,7 @@ require('shortport')
 require('stdnse')
 require('strbuf')
 require('listop')
+require('comm')
 
 author = "Eddie Bell <ejlbell@gmail.com>"
 license = "Same as Nmap--See http://nmap.org/book/man-legal.html"
@@ -32,7 +33,7 @@ local soc
 local catch = function() soc:close() end
 local try = nmap.new_try(catch)
 
-portrule = shortport.service("http")
+portrule = shortport.port_or_service({80, 443}, {"http","https"})
 
 --[[
 Download a page from host:port http server. The url is passed 
@@ -43,11 +44,12 @@ local function get_page(host, port, httpurl)
 	local lines = ""
 	local status = true
 	local response = ""
+	local opts = {timeout=10000, recv_before=false}
 
 	-- connect to webserver 
-	soc = nmap.new_socket()
+	--soc = nmap.new_socket()
-	soc:set_timeout(4000)
+	--soc:set_timeout(4000)
-	try(soc:connect(host.ip, port.number))
+	--try(soc:connect(host.ip, port.number))
 
 	httpurl = string.gsub(httpurl, "&amp;", "&")
 	--print(filename .. ": " .. httpurl) 
@@ -59,8 +61,9 @@ local function get_page(host, port, httpurl)
 	query = query .. "Accept-Language: en"
 	query = query .. "User-Agent: Mozilla/5.0 (compatible; Nmap Scripting Engine; http://nmap.org/book/nse.html)"
 	query = query .. "Host: " .. host.ip .. ":" .. port.number 
-	try(soc:send(strbuf.dump(query, '\r\n') .. '\r\n\r\n'))
+	--try(soc:send(strbuf.dump(query, '\r\n') .. '\r\n\r\n'))
 
+	soc, response, bopt = comm.tryssl(host, port, strbuf.dump(query, '\r\n') .. '\r\n\r\n' , opts)
 	while true do
 		status, lines = soc:receive_lines(1)
 		if not status then break end

scripts/telnet-brute.nse

@@ -9,6 +9,7 @@ categories = {'auth', 'intrusive'}
 require('shortport')
 require('stdnse')
 require('strbuf')
+require('comm')
 
 local soc
 local catch = function() soc:close() end
@@ -52,7 +53,7 @@ local new_auth_iter = function()
 		{'visitor', ''}, {'netman', 'netman'}, {'Admin', 'Admin'},
 		{'manager', 'manager'}, {'security', 'security'},
 		{'username', 'password'}, {'user', 'pass'}, 
-
+		
 		-- sentinel 
 		{nil, nil}
 	}
@@ -184,12 +185,14 @@ action = function(host, port)
 	
 	pair = nil ; status = 3 ; count = 0
 	auth_iter = new_auth_iter(); 
+	
+	local opts = {timeout=4000}
 
-	soc = nmap.new_socket()
+	soc, line, best_opt = comm.tryssl(host, port, "\n",opts)
-	soc:set_timeout(4000)
+  	if not soc then return "Unable to open connection" end
 
 	-- continually try user/pass pairs (reconnecting, if we have to)
-    -- until we find a valid one or we run out of pairs
+        -- until we find a valid one or we run out of pairs
 	while not (status == 1) do
 
 		if status == 2 or status == 3 then
@@ -206,7 +209,7 @@ action = function(host, port)
 		if not user then break end
 
 		if status == 3 or status == 4 then
-			try(soc:connect(host.ip, port.number, port.protocol))
+			try(soc:connect(host.ip, port.number, best_opt))
 		end
 
 		status, pair = brute_cred(user, pass)