1
0
mirror of https://github.com/nmap/nmap.git synced 2025-12-09 06:01:28 +00:00
Files
nmap/docs/TODO

584 lines
28 KiB
Plaintext

TODO $Id: TODO 11866 2009-01-24 23:10:05Z fyodor $ -*-text-*-
o Ncat SSL issues. See http://seclists.org/nmap-dev/2009/q1/0319.html
o Defensive coding review of ncat_proxy.* [David]
o Look at Dario Ciccarone's email from 5/1/07 about IPID sequence
issues, and consider adding IPID sequence test for closed-port-tcp as
they apparently can be different. [David]
o Also fix bug which causes SEQ to not be printed if the TCP open
port tests fail to produce results, even though the II and
(upcoming) CI tests may have useful results. [David]
o Write Ncat users' guide, demonstrating all the neat stuff you can do
with it. This should probably be in DocBook XML so it can be an NNS
chapter. You might want to query nmap-dev for list of neat things
people do with ncat (or look around for what people do with nc).
Testing it out for examples might expose areas for improvement as
well. [David]
o Consider converting this file to emacs org-mode
(http://orgmode.org/) format. [Fyodor]
o That format is still plain text and can be read/edited by vi
users, etc.
o Determine what we should do about the IE.DLI OS detection test
o It appears that of the 1657 results for this test in nmap-os-db,
1656 are DLI=S and the remaining one is DLI=100
o Is the test not working right (producing the proper results
against targets), or is it just a generally useless test for
which virtually all targets respond the same way?
o Are there other "useless" tests in nmap-os-db? It is worth
checking, IMHO.
o [Ncat] Let people set up authenticated proxies using
--listen and --proxy-auth together (right now we don't support
that). [David]
o [Ncat] The sys_wrap.c/.h code contains a whole bunch of capitalized
versions of system calls (Fork(), Socket(), Sscanf(), etc.) which
are mostly the same as the standard version except that they cause
ncat to quit if they are triggered. They also may be used partially
for portability. The main issues are:
1) Because the function quits in the case of errors, it doesn't
always have the context to print a useful error message (and
even when it does, it often doesn't -- for example Fopen could
print the filename, but doesn't.) Also, sometimes these
functions are called when quitting really isn't the desired
outcome of an error.
2) Some could be replaced by code in nbase, for example, Malloc
basically does the same thing as our safe_malloc already used
throughout Nmap.
So we should probably consider simplifying/removing this code to the
extent possible. But we need to remember to add error detection to
the callers where necessary rather than blindly switching from
(e.g.) Connect() to connect(). [Kris or David]
o Look into whether we should loosen/change the global congestion
control system to address possible cases of one target host with many
dropped packets slowing down the whole group. See
http://seclists.org/nmap-dev/2008/q1/0096.html .
* Related possibility: Fix --nogcc to gracefully handle ping scans.
Right now it seems to go WAY TOO FAST (e.g. several thousand
packets per second on my DSL line).
o Device categorization improvements
o Examine Nmap's device categorization in nmap-os-deb and
nmap-service-probes. Decide if some small categories which have
never really took off should be consolidated, or whether others
should be split off. For example, maybe there are some groups in
'specialized' or other misc. categories which are now large enough
to split off. Personally, I wouldn't give anything its own
category unless there are at least half a dozen of them and no
other category really fits them well. We should use a combined
system for nmap-os-db and nmap-service-probes.
o Add a classification sect1 to os-detection.xml
(http://nmap.org/book/osdetect.html) to cover how Nmap handles OS
classification. It should include a list with descriptions of
each device type recognized by Nmap. Version-detection.xml should
reference (link to) it in the approprate place.
[Doug has done some initial work on this. For example, see
nmap/docs/device-types.txt]
o [NSE] Open proxy detection script?
o We have http-open-proxy.nse, but we should probably either extrand
that to handle other types of proxies (such as SOCKS and HTTP
CONNECT) or create more scripts to handle those other proxy types.
o Prepare for Summer of Code
o Brainstorm for ideas
o Create new ideas page
o Apply to participate in program again
o Advertise for applicants
o Evaluate applicants
o Decide which applicants we want, and who would be best for
mentoring them.
o Make Zenmap settings get upgraded when the Zenmap executable is
upgraded. The per-user configuration files such as scan_profile.usp
and zenmap.conf are never overwritten once installed by Zenmap, so
changes and fixes to those files don't reach anyone who has
installed Zenmap already. This is most noticeable with changes to
profiles and highlight definitions are notably affected. This fix
may involve hard-coding settings that are not normally configured by
users (like highlighting) or updating the per-user files at startup
(only those parts that haven't been changed by the user).
o Process the latest version detection submissions. We now have more
than 1,700 of them queued up. [Doug]
o [Ndiff] Rethink the output format. In particular, I would like to
always have the old state on the left and the new state on the
right: "was filtered, is open," not "is open, was filtered." I also
like the context diff output of MadHat's nmap-diff. [David]
o [Ncat] Consider supporting server certificate verification when used
in client SSL mode.
o For now we document in user's guide that it is not secure.
o If we're going to verify cert's etc., we need to also make sure we
are actually using secure ciphers. We may need to update nsock to
support cipher selection, because we want fast ones for version
detection, but usually want secure ones for NSE and/or ncat.
o Do we want to check all this by default, or offer an option for
it? Doing it by default is more secure, though it can be annoying
when a certificate has expired, is self-signed, you connect to
domain.com when the certificate is for www.domain.com, etc. If it
is done by deault, we might just print an error message. Whreas
if we have a special option, it may be OK to exit and refuse the
connection.
o What certs should we allow? Same as the browsers do? Maybe get
rid of Comodo? Maybe we should fail to recognize any certs with MD5
in the trust chain?
o What about people who are running their own SSL service and just
want to specify the cert file they use, because they generated it
themself and not from a trusted CA.
o Need to check expiration, domain, etc. if we're checking certs at
all.
o We can probably get away with not doing revocation checking, as
long as we document that we don't.
o Look into memory consumption of UDP scans with -p- and large
hostgroups. See if there is a way to prevent them from eating up gigs
of RAM.
o Fix the directory function(s) in nse_fs.cc to be usable by scripts and
improve flexibility. [this entry added by Patrick]
o Work on NSE Performance in general
o Ask Coverity if they'll scan latest version of Nmap.
o Start project to make Nmap a Featured Article on Wikipedia.
o Add Nmap web board.
o Create Nmap wiki
o Consider rethinking Nmap's -s* syntax for specifing scan types
o Current problems with this -s syntax:
o We already use like 20 of the 26 letters, so we end up with
things like SCTP scan using -sY
o Can make Nmap command lines hard to read, particularly given
that we often need to improvise to find a letter which isn't
taken.
o Problematic for scan types -sI and -b which require arguments
o Inconsistencies. For example, -sC and -sV do script scan and
version detection, respectively, and yet for OS detection we use
-sO.
o Possible solution:
o We might want to just give them normal option strings, so you
could do --maimon instead of -sM, for example. For extremely
common options such as SYN scan, UDP scan, version detection, we
could perhaps find good single letter options as an alias to the
longer one.
o Another idea is to use something like --scantype syn,udp,sctp,
which is a lot longer for single-type scans, but shorter when
you're combining mulitiple ones. Doesn't allow for individual
scan arguments easily. I (Fyodor) think I prefer the idea above
of just givem them top level arguments.
o Obviously this will take some discussion/brainstorming on nmap-dev.
o libnmap organization for UNIX and Windows
o Then change Nmap and Zenmap to simply call this library
o [NSE] We may want to consider a better exception handling method -- one
which doesn't require wrapping every I/O line in its own try function
call.
o Consider adding boolean expressions to --script arguments. For
example, see Patrick's implementation at
http://seclists.org/nmap-dev/2008/q3/0300.html .
o Consider whether we should include some sort of NSE debugger. Or we
could include something simpler. For example, some developers (such
as Ron) already make use of Patrick's traceback.nse in their
experimental trees.
o Figure out what to do about NSE mutexes:
http://seclists.org/nmap-dev/2008/q3/0276.html .
o Consider whether to let Zenmap Topology graph export the images to
svg/png/etc. Also think about printing.
o Perhaps --traceroute should set currenths->distance because right
now, I do an -O scan against scanme.nmap.org, and it does not figure
out the distance. So the fingerprint shows no distance element and
Nmap doesn't print "Network Distance" in the results line. That may
be OK (Nmap probably isn't receiving the probe response needed for
this, and maybe doesn't want to print the TG), but even when I do
--traceroute I get no distance printed. Yet Nmap clearly knows the
distance since the traceroute shows all the hops up to and including
the target (scanme.nmap.org).
o Look into building RPMs with SSL support. Statically linking to
OpenSSL on Linux for the RPMs didn't work for me last time I
tried. [Fyodor]
o Improve the "run Zenmap as root" menu item to work on distributions
without su-to-root. We might even want to improve Zenmap so that it
itself does not have to run as root, and just executes Nmap that
way. Rather than not showing Zenmap as root on the Menu of
non-working systems, it might be better to have it but let it give
an error message (and then, perhaps, run as nonroot) so that users
of those distributions are more likely to contribute a fix. We also
might want to look at how the distributions themselves package Zenmap.
o Consider enhancing the new OS Assist system to handle version
detection too. [SOC task?]
o Change Nmap signature files to use the .sig extension rather than
.gpg.txt, as that seems to be what gpg recommends. In fact, gpg
will automatically verify the right file if it exists after dropping
the .sig (or .asc) extension. I may need to configure .htaccess to
serve .sig files properly. Update nmap-install.xml
accordingly. Suggested by tic at eternalrealm.net by email on 7/13/08.
o Do -p- Internet UDP scans.
o Consider adding the rtt value for each host, at least in verbose
mode, to Nmap output.
o NSE-INF: Would be great if NSE scripts could be made to NOT run as
root.
o Deal with new Python 2.6 Zenmap build warnings:
C:\Python26\lib\site-packages\py2exe\build_exe.py:16: DeprecationWarning: the sets module is deprecated
import sets
http://sourceforge.net/tracker/index.php?func=detail&aid=2314799&group_id=15583&atid=115583
o Look a bit more at default version detection timing.
o Deal with UDP retransmission for version detection ( I think I
should just do a second run of all probes for UDP if it fails to
match anything). The advantage there is that no retransmissions are
neccessary if the service is found. Then again, per-probe
retransmission would let us redo the most likely probes (the one(s)
that match the port number) quickly. Lost packets should probably
affect ideal_parallelism.
o Figure out why I [Fyodor] get a bunch of "Operation not permitted" errors
when I launch a scan on SYN such as:
/home/fyodor/nmap-exp/fyodor-perf/nmap -nogcc -T4 -n -v -p- --portpingfreq 250 -oA /home/fyodor/nmap-misc/logs/WorldScan/portpingfreq/logs/portpingfreq-250-1%T-%D 67.15.236.34 67.15.236.36 81.174.236.66 81.174.236.119 170.140.20.160 170.140.20.174 202.138.180.9 202.138.180.17 202.138.180.132 209.20.64.112
The errors look like:
sendto in send_ip_packet: sendto(7, packet, 44, 0, 170.140.20.174, 16) => Operation not permitted
Offending packet: TCP 64.13.134.4:59820 > 170.140.20.174:59120 S ttl=39 id=19927 iplen=44 seq=2425535549 win=4096 <mss 1460>
sendto in send_ip_packet: sendto(7, packet, 44, 0, 67.15.236.36, 16) => Operation not permitted
Offending packet: TCP 64.13.134.4:59820 > 67.15.236.36:15030 S ttl=57 id=50640 iplen=44 seq=2425535549 win=2048 <mss 1460>
Discovered open port 49394/tcp on 170.140.20.174
sendto in send_ip_packet: sendto(7, packet, 44, 0, 170.140.20.174, 16) => Operation not permitted
Offending packet: TCP 64.13.134.4:59819 > 170.140.20.174:8256 S ttl=48 id=38510 iplen=44 seq=2425601084 win=1024 <mss 1460>
May be related to connection tracking and high scan rates. See
http://seclists.org/nmap-dev/2008/q4/0652.html
http://www.shorewall.net/FAQ.htm#faq26
Others have reported similar issues even without connection tracking. See
http://seclists.org/nmap-dev/2006/q3/0277.html
http://seclists.org/nmap-dev/2007/q2/0292.html
o Get better password data for unpw
o perhaps from Solar Designer.
o perhaps add phpbb hack data (there is at least a list of 28,635
passwords in phpbb_users.sql, and possibly more in other files.
o Consider making the ping scan default be more comprehensive. Note
that I got 23% more Internet boxes found out of a 50K sample (see host
enumeration chapter of my book for details). Maybe I should
experiment a bit more to ensure they are real boxes and not network
artifacts and figure out exactly which tests are helping the most.
If I do this change, I'll have to update the host enumeration chapter.
o Nmaprc-related - Create a system to store Nmap defaults/preferences
in an nmaprc file.
o nmaprc should be in ~/.nmap on UNIX
o On Windows, we may need a registry key to find the .nmaprc
o Obtain Nmap data directory information from nmaprc at runtime rather than
compiled in -- among other advantages this is needed to make
relocateable rpm.
o Make RPM relocatable (requires somehow avoiding storing paths in the
binary)
o Perhaps Lua could be used for the TODO?
o .nmaprc for keeping defaults, etc.
o Nmaprc infrastructure, hook to new timing variables
o Nmaprc man page
o Default timing mode
o Default NSE arguments, such as user agent
o Maybe Default source IP (-S) argument
o should be a way to specify your own .nmaprc
o Maybe lets you add a directory and template for saving all
scans.
o Search for nmap on google news, on google web, and add appropriate
links to press page and the like.
o Maybe nping -- like hping2 but uses Nmap infrastructure and to a
large degree the same command-line options as Nmap.
o Think about Nmap or NSE http framework. Scanning http paths to see
if they exist is in some ways similar to scanning to see which ports
are open.
o Website: Create shr (shared) directory in svn, which will contain
directories shared between the Insecure.org network of sites
(e.g. templates, error, css). Then sites such as sectools,
nmap.org, insecure.org can just check that out via externals
declaration (or, I suppose, symlink). CSS directives will then use
/shr/css/insecdb.css etc. ). [Fyodor]
o NSE Security Review
o Consider what, if any, vulnerabilities or security risks NSE has
with respect to buffer overflows, format string bugs, any other
maliciously formatted responses from target systems, etc. Maybe
address the known risk of malicious scripts too.
o Consider that NSE runs scripts as root
o Zenmap script selection interface for deciding which NSE scripts to
run.
o Get new Zenmap logo
o consider putting back on top-right of command constructor wizard
(there used to be umit logo there).
o Maybe that can be done after the release by soliciting ideas.
o Make Zenmap splash screen
o nmap.cgi web interface for Nmap
-- Should have "demo" mode that only allows users to scan their own addy
o Create or collect some great ./configure ascii art.
o Add randomizer to configure script so that a random ASCII art from
docs/leet-nmap-ascii-art*.txt is printed. I think I'll start naming
them leet-nmap-ascii-art-submittername.txt.
o [Note: This one is too big to do right now, but is a good one to
keep in mind for later ]
Write a general scanning engine for abusing applications for port
scanning purposes. This would handle scanning through SOCKS and HTTP
proxies, and the existing FTP bounce scan would also be ported to this
engine. Proxy chaining must be supported. According to
rembrandt@jpberlin.de, you can also do this with the "forwarding"
commands on imap servers.
o Before you start on this one, read the code for the main port
scanning engine code (ultra_scan()) and also the version detection
code (service_scan()). And the version detection paper at
http://www.insecure.org/nmap/vscan/ . If you understand all that,
you may be ready for this project :). This is important, because it
is easy to do poorly. The tough part is high performance and clean
code which is general enough that all these different applications
can be scanned through using the same basic engine.
o You may want to run your intended structure (the most important
Classes and such through nmap-dev before you begin serious coding).
o Add general regression unit testing system to Nmap
o Talk to Libpcap folks about incorporating (at least some of) my
changes from libpcap/NMAP_MODIFICATIONS.
o Add --evil to set the RFC3514 evil bit.
ftp://ftp.rfc-editor.org/in-notes/rfc3514.txt
o We're not going to add this right now.
o The Nmap web page is beginning to show its age. Ah, who am I
kidding, it was showing its age 5 years ago :). It could do with an
upgrade to XHTML+CSS. It could also do with a whole redesign, but I
think that can be done as a second step after converting to
XHTML+CSS with roughly the same look. Though adding a few more
modern touches (like hover interaction on the menu bar) wouldn't
hurt. This is a moderatly big project, which will involve: o
Designing the new XHTML+CSS to look similar to the current HTML
pages, but be extensible enough that it can be redesigned in the
(near) future by mostly just changing the CSS and graphics.
o Converting the existing Nmap pages to the new XHTML format.
This will likely include using open source programs and likely
modifying them or creating your own scripts to help with the
process. To apply for this task, you need to have some web
development experience and an example XHTML+CSS web page you
have created online.
o Provide an option to send a comment in scan packet data for target
network. Examples: --comment "Scan conducted by Marc Reis from
SecOps, extension 2147" or --comment "pH33r my l3eT
s|&lt;iLLz! I'll 0wN UR b0x!"
o Note, this shouldn't be implemented yet.
o I should add code to Nmap to bail if sizeof(char) isn't 1.
Otherwise there could be security risks if it is not one on any
platforms.
o consider changing status field from "up" and "down" to "online" and
"offline".
o I need an output-autoflush option of some sort. This could be
useful to ensure I get all the --packet_trace and debug data before
Nmap crashes. Actually, I'm not sure that is so critical.
o Consider implementing RPC scan with ultra_scan or something else.
Right now it is the only program using pos_scan. On the other hand,
I'm not sure TCP rpc scanning is appropriate for ultra_scan.
o Look at all the pcap functions, there are some like
pcap_findalldevs() which could be quite useful. There are mails to
the Nmap list relating to suggested improvements --
http://seclists.org/lists/nmap-dev/2004/Apr-Jun/0024.html .
Actually I do indirectly use that for Windows. I wonder if they work
for UNIX?
o Update Nmap entry on Linux Online -
http://www.linux.org/apps/AppId_1979.html
o Proxy scan through
o Note mail from mugz about proxy scanning:
> #1 I use nmap to find "open" socks ports (stealth/random mode)
> #2 I use Sockcheck5 (which uses an existing proxy to run its
> scan through) to determine which of the "open" ports are "unsecure"
> #3 I use sockbounce (or sockbounce4) which can be used to relay from
> socks proxy to socks proxy to target and estabish tcp connections,
> (telnet, ssh http, etc).
His later mail says:
> during that time, i found many 'open' - a fairly high percentage,
> perhaps one in 30 were insecure.
> you might want to take a look at: http://blitzed.org/bopm/
> I use code from this to check the IP's with 'open' socks ports for
> insecurity (I had to tinker with it a bit to make it work like i wanted,
> the command line "bopchecker" seems to work well.
o perhaps each 'match' line in nmap-service-probes should have a
maximum lines, bytes, and/or time by which a response should be
available. Once that much time (or many bytes or lines) have passed,
that match can be considered 'failed' and ignored in subsequent runs.
Once all matches are considered failed, that probe is done. This
could be a useful optimization and is arguably better than the less
granular 'totalwaitms'. Or I could just have a simple function that
looks at whether a given regex could possibly match something
starting with the received data (not too hard since almost all of
the current regexes are anchored). But before doing this, I should
look long and hard at how many of the probes have every match
capable of doing this. In particular, many of the softmatch lines
don't offer many chars anchored at the front.
o Add detection of duplicate machines via IP.ID uber-technique.
Maybe I should use uptime timestamps too. Oh, and MAC addresses too.
o Separate nbase into its own Windows library in the same way as Andy did
with iphlpapi .
o Look into iplog ( http://ojnk.sourceforge.net/ ) -z option which is
supposed to fool OS detection.
o security audit of Nmap code
o Nmap / Nmap-hackers FAQ
o random tip database
DONE:
o NSE should offer some way to sleep/yield for a given amount of
time. This would allow other scripts to run while a script has
nothing to do. Possible uses:
o Many services have rate limits (or you might just want to use them
for politeness). For example, a web site spidering application
might want to limit HTTP requests to some number per second to avoid
pissing off the target webmaster more than is necessary (or prevent
getting auto-blocked). Similarly, whois servers often will block
IPs which query them too often in a short period. Or maybe you
don't want to exceed the threshold limits of an IDS.
o Example current scripts which might benefit: sql-injection, whois
(possibly), pop3-brute, etc.
o If we don't currently have a way for a cpu-bound NSE script to
yield, then perhaps this could help us implement such a mechanism.
But maybe coroutine.yield already does the trick.
o The mechanism needs to be documented, and ideally should be
implemented in at least one of the scripts shipped with Nmap.
o Consider adding a way for requesting timing status updates at a
given interval (such as every 5 seconds) to XML and/or normal
output. This would be useful for people who run Nmap from scripts
or other higher level applications. [David]
o Ncat --allow/--deny bug: "--allow and --deny only support host
specification by IP address, and give no warning when you use
another form such as a host name." Should probably use same syntax
as --exclude. We also want to at least do verification at the
beginning to make sure all the entries are legitimately formed. We
probably want to do things like DNS resolution at the beginning
too. Otherwise we might have a DNS failure when we actually get a
connection and perhaps have to reject the connection wrongly, or
risk a false negative. [David]
o Fix this overflow:
Stats: 93:57:40 elapsed; 254868 hosts completed (2048 up), 2048 undergoing UDP Scan
UDP Scan Timing: About 11.34% done; ETC: 03:21 (-688:-41:-48 remaining)
[Done by David and Henri Doreau]
o Ncat -- perhaps connection brokering should support UDP as well as
(its existing support for) TCP? Actually this does raise issues
such as deciding what list of UDP systems to forward a packet too.
Its obviously not like TCP where you have a list of open
connections. Ncat could build such a list, but, for example, would
never know when to remove the host. For now, David is just going to
adjust the error message to encourage people to email nmap-dev
describing their usage scenario if they want this feature.
o Ncat documentation should note that no SSL certificate verification
is done (maybe we should offer an option to do so, if OpenSSL makes
that easy).
o Done in the new Ncat user's guide
o Fix dns-zone-transfer infinite recursion bug described at
http://seclists.org/nmap-dev/2009/q1/0317.html. It sounds like the
best approach is to use our dns.lua library rather than having
dns-zone-transfer do its own DNS packet parsing.
o Fix XML escaping issue so that improper chars from NSE scripts or
elsewhere can't cause corrupt XML files. See
http://seclists.org/nmap-dev/2009/q1/0316.html for an example. [David]
o Look into whether we should increase the frequency of port scan
pings. See http://seclists.org/nmap-dev/2008/q1/0096.html . Note
that Fyodor already increased them a bit in 2008. Might not need
more. [David did extensive testing of this one already]
o Find way to document NSE library script arguments and perhaps have
them bubble up to scripts themselves. For example, I had to read
the SNMP library source code to determine the script argument to
specify the SNMP community name for snmp-sysdescr
(http://nmap.org/nsedoc/scripts/snmp-sysdescr.html). Maybe we could
just standardize on something like we do with SMB library and the
scripts which call it (http://nmap.org/nsedoc/modules/smb.html,
http://nmap.org/nsedoc/scripts/smb-check-vulns.html). [David]
o If it wouldn't bloat things too much, it would be nice to include
ndiff in the Nmap win32 zip distribution files.
o Reported NSE crash:
"Assertion failed - file ..\nse_main.cc line 314
lua_gettop(L_script_scan) == 0"
o He says: "After looking at this closer, it appears the assertion
occurs if I include the IP where the scan is run from. For us, I'm
running this on IP 57, which is a VMware Windows Server image. If
I eliminate that IP from the range it successfully completed the
scan for all other devices."
o Seems to be fixed. He can no longer reproduce the problem with
4.85BETA3.
o Deal with GTK DLL problem with Nmap 4.85BETA1: [Fyodor]
o David's installer seems to work--he's using a different GTK
distribution. I'll try that. Works! Done!
o Details on problem: http://seclists.org/nmap-dev/2009/q1/0207.html
o Quick workaround done for 4.85BETA2, but better solution needed.
o "SCRIPT ENGINE (250.600s): ./scripts/rpcinfo.nse against
a.b.c.d:<port> ended with error: ./nselib/datafiles.lua:114: attempt
to index global 'arg' (a nil value)"
-- http://seclists.org/nmap-dev/2009/q1/0227.html [Patrick]
o Consider making the TODO list public
o Done: http://seclists.org/nmap-dev/2009/q1/0175.html
o Probably remove all of the "done" items since that is easier than
reviewing them.
o Might as well add to insecure.org/nmap/data/
o Maybe a bug tracker is a better approach.