a layer 4 protocol used mostly for telephony related applications.
This brings the following new features:
o SCTP INIT chunk port scan (-sY): open ports return an INIT-ACK
chunk, closed ones an ABORT chunk. This is the SCTP equivalent
of a TCP SYN stealth scan.
o SCTP COOKIE-ECHO chunk port scan (-sZ): open ports are silent,
closed ports return an ABORT chunk.
o SCTP INIT chunk ping probes (-PY): host discovery using SCTP
INIT chunk packets.
o SCTP-specific IP protocol scan (-sO -p sctp).
o SCTP-specific traceroute support (--traceroute).
o The ability to use the deprecated Adler32 algorithm as specified
in RFC 2960 instead of CRC32C from RFC 4960 (--adler32).
o 42 well-known SCTP ports were added to the nmap-services file.
Part of the work on SCTP support was kindly sponsored by
Compass Security AG, Switzerland. [Daniel Roethlisberger]
right type of probe (0 with 8, 14 with 13, 18 with 17). With the new
default ping, I was scanning a network where an echo reply would
mistakenly be interpreted as a response to a timestamp request, even
though that host didn't respond to timestamp requests. That host would
become the global ping host, and all its probes would be dropped,
slowing the scan way down. A ping scan of a /24 took over 1,000 seconds
when it should have taken about 10.
exhaustive testing of 90 different probes, this one emerged as the
best four-probe combination, finding 14% more Internet hosts than
the previous default, -PE -PA80. The default for nonroot users is
-PS80,443, replacing the previous default of -PS80. In addition,
ping probes are now sent in order of effectiveness (-PE first) so
that less likely probes may not have to be sent.
Found whacked packet protocol 17 in get_ping_pcap_result
and rewrite it
Received packet with protocol 17; ignoring.
The message is printed when we receive a packet we can't use during a
ping scan, but it's not "whacked" to receive a UDP packet during a TCP
scan for example.
that it is set properly for ping scan. Previously status reports during
ping scan always looked like
Stats: 25:34:33 elapsed; 991232 hosts completed (72530 up), 0 undergoing Ping Scan
Note the "0 undergoing". Now that number will be set to 4096 or whatever
the current ping scan group size happens to be.
increases the scan dealy with an increase in max_successful_tryno. When I
reverted a bunch of changes in r11651, I removed the moved code, leaving the
scan delay increase nowhere. This puts it back in ultrascan_port_probe_update
where it was before.
Don't make a host the global ping host until it moves to the completed
hosts list, and only change the global ping probe if the new probe is no
worse than the old (according to pingprobe_is_better).
Restore the ping magnifier for host congestion window updates.
Ignore the timing of certain ICMP errors that are likely to be rate
limited and don't change the port or host state. Avoid making timing
pings out of probes that elicit such errors. This used to be done only
for port scans and only at -T4 and above (and didn't prohibit the
creation of timing pings). Now it is done for host discovery too, and at
all timing levels.
Gracefully handle updates from the recent past in RateMeter. Doesn't
affect performance, but avoids a rare assertion failure.
num_probes_active == 0 in HostScanStats::completed. The reason for this is
fairly subtle and I didn't realize it at first: We have to make sure there are
no active probes because once in the completed list, probes don't time out.
Probes that are active stay active in the count. If the congestion window ever
falls below the number of these active probes, the program will hang waiting
for them to time out.
We could get away with this in the case of up hosts, because we call
HostScanStats::destroyAllOutstandingProbes in that case. We could do that in
the down case too, but that would prohibit a down host from being found up
later on. That's currently a matter of some luck; we don't keep sending probes
after a host is down but will accept replies to any other probes that have
already been sent.
source address didn't match the target address. Fyodor correctly pointed out
that this is wrong for UDP scans, when we need to slow down for a firewall
sending unreachables to know which probes don't elicit one. I'm going to try
something a little different in nmap-perf.
is up or down, we can move it to the completed list, regardless of any active
probes. However I can imagine changing this so that we move it when it is found
up, or when it is found down and there are no probes left to send. That would
give a down host a chance to become up with a different probe later on.
checking if a pingprobe has been set. We always use PORT_UNKNOWN during host
discovery (rather than HOST_UP or HOST_DOWN) to avoid conflicts with other
PORT_* constants. See the log for r8784.
num_probes_outstanding() == 0. The active probes are probes that haven't timed
out; the outstanding probes includes those and some timed-out probes. When a
host makes it to the completed list with outstanding probes, it is unlikely
they will ever be removed, so we weren't sending any global pings in some
cases. A host only moves to the completed list with it has no active probes,
and any active probes it has after that are global pings.
Also remove only timed-out ping probes from the ping host. All other probes are
timed out but waiting for a response.
me) in r8784 when I made the change to retain ping probes between ping scanning
and port scanning.
It only gets set during host discovery scans, because that's how it was before,
but it might be advantageous to set it during port scans as well.
target. In my testing, TCP scanning certain hosts, other hosts send back ICMP
destination unreachables (admin prohibited) for some ports (the Microsoft
ports). If that's the only response we got from a host, we would make it the
timing ping probe. But the admin-prohibiteds come back at some very slow rate,
much slower than the 1.25/s ping interval. So most of them are dropped, keeping
the congestion window perpetually at 1 and slowly increasing the scan delay. In
a -F scan these hosts could take over twice as long as any other host.
I also put in some missing adjust_timing arguments to the host and ping update
functions whenever adjust_timing could be false (the default is true). I think
those were just oversights. I also made it so that we don't select a new ping
probe when adjust_timing is false; there's no point sending ping probes whose
responses we're going to ignore.
positive response that results in being marked filtered, the reply was an ICMP
error. Such probes are now ranked a 2 along with UDP, IP proto, and other
probes that get ICMP replies that are likely to be rate limited. Previously
they got the highest rank, 6, because they were TCP probes other than SYN to an
open port. SYN to an open port retains its rank of 3.
socket). It's possible to get an EHOSTUNREACH or EHOSTDOWN (and possibly other
errors) against a nonexistent LAN host and formerly the socket wasn't closed.
This allowed Nmap's internal count of sockets to be less that the true count of
open sockets. It would eventually try to open too many sockets and quit with
the message
Socket creation in sendConnectScanProbe: Too many open files (24)
Be aware that in this message 24 is the errno, not the number of open files.
i.e., the probe didn't just time out. This was the case before r11253 when I
mistakenly removed it. I meant only to allow ultrascan_adjust_timing to be
called when rcvdtime == NULL, but as a side effect I also allowed
max_successful_tryno to increase in that case too. I was able to get that bit
of code to run with rcvdtime == NULL in a normal scan, but I don't think it
happens often.
I also improved the comments in that block of code.
GroupScanStats::probeSent. This will allow updating timing variables for a
per-host rate limiter.
This fixes a bug, which was that decoy probes were not recorded by the scan
rate meter. Decoy scans would show a lower scan rate than the actual: with four
decoys the rate printed would be 1/5 of actual. This only affects printed
output, not the actual scan rate.
replied to or times out. Previously the scaling factor was increased as soon a
s a probe was sent. So if you sent 10 probes right away at the beginning of the
scan, the first reply that came back would be scaled by 10 (= 10/1, and the
next by 9/2, etc.). Now if every probe is replied to then the scaling factor
will be exactly 1 throughout the scan.
completed hosts from the lists after two minutes. These hosts are
kept around in case there is a late response, but this draws the
line on how long we wait and hence keep this information in memory.
See http://seclists.org/nmap-dev/2008/q3/0902.html for more.
whether a probe response counts as a drop for scan delay purposes. This avoids
an integer overflow in TIMEVAL_SUBTRACT that caused all responses to probes
retransmitted from the retry stack to be counted as drops. This would cause the
scan to grind to a near-halt, with the scan delay at 1000 ms, if even a few
hundred probes were retransmitted from the bench.
Increased max_successful_tryno for 192.168.0.190 to 1 (packet drop)
Increased max_successful_tryno for 192.168.0.190 to 2 (packet drop)
Increasing send delay for 192.168.0.190 from 0 to 5 due to 216 out of 718 dropped probes since last increase.
Increased max_successful_tryno for 192.168.0.190 to 3 (packet drop)
Increasing send delay for 192.168.0.190 from 5 to 10 due to 92 out of 305 dropped probes since last increase.
Increasing send delay for 192.168.0.190 from 10 to 20 due to 11 out of 11 dropped probes since last increase.
Increasing send delay for 192.168.0.190 from 20 to 40 due to 11 out of 11 dropped probes since last increase.
Increasing send delay for 192.168.0.190 from 40 to 80 due to 11 out of 11 dropped probes since last increase.
Increasing send delay for 192.168.0.190 from 80 to 160 due to 11 out of 11 dropped probes since last increase.
Increasing send delay for 192.168.0.190 from 160 to 320 due to 11 out of 11 dropped probes since last increase.
...
The problem was in this bit of code:
if ((!rcvdtime && TIMEVAL_SUBTRACT(probe->sent, hss->sdn.last_boost) > 0) ||
(probe->tryno > 0 && TIMEVAL_SUBTRACT(probe->prevSent, hss->sdn.last_boost) > 0)) {
the TIMEVAL_SUBTRACT(probe->prevSent, hss->sdn.last_boost) > 0) to be specific.
When a probe is retransmitted, the time it was sent is recorded in the prevSent
member of the retransmit probe. prevSent is properly set in retransmitProbe,
but it is not set in sendNextRetryStackProbe, which sends probes that have been
moved from the bench to the retry stack. The problem is that when probes are
moved to the bench they are compressed to probespecs and lose most of their
auxiliary information, like the send time. When they are retransmitted as real
UltraProbe objects, their prevSent message is left initialized to { 0, 0 }.
That led to the integer overflow, with TIMEVAL_SUBTRACT returning a nonsense
(but positive) value.
I fixed it by using TIMEVAL_AFTER(...), which works like
TIMEVAL_SUBTRACT(...) > 0 except that it is immune to integer overflows. Every
other timeval is after { 0, 0 }, so the condition is false for probes
retransmitted from the bench, as it should be. However this is not the most
correct solution. Better would be to somehow store each probe's send time with
it on the bench so it could be restored when it is retransmitted. The way the
bench and the retry stack work makes that cumbersome though, and this is the
only place prevSent is used, so I think this solution is acceptable.
formula algebraically reduced to
1.0 - gstats->numprobes / host->freshPortsLeft(), which doesn't regard the
number of sent probes, leading to long stalls in the completion when, for
example, maxtries is increased.
probes that will be sent and calculate from that. This makes the 100%
completion match up closely with the end of the scan, as the estimate gets
better as the scan gets closer to finishing. It also works against filtered
hosts. There is still the related problem that the completion time estimate
uses a global average to estimate completion rate, so it tends to
underestimate.
estimates, and don't consider the number of outstanding probes or number of
retries. The old code would overestimate completion, causing a period of 99.99%
completion at the end of a scan (which caused earlier estimates to be too
soon). For a long UDP scan this puts the completion estimate right on after a
little time to allow for convergence. See
http://www.bamsoftware.com/wiki/Nmap/CompletionTimeEstimates.
the present. This allows the sending rate to temporarily exceed the
maximum to keep from being slowed too much by other delays in the scan
engine. See the discussion at http://seclists.org/nmap-dev/2008/q3/0236.html.
