Fixes #3020

* Adding sample schemas shipped with Oracle DB

Added sample schemas that are included with the Oracle Guides. They are unbelievably common in live systems from personal experience.

https://docs.oracle.com/cd/E11882_01/server.112/e10575/tdpsg_user_accounts.htm#TDPSG20303

* Adding  as an oracle system database for exclusion

`APPQOSSYS` schema is used by Oracle Database QoS Management

https://docs.oracle.com/cd/E11882_01/server.112/e24611/install_config.htm

.github/CODE_OF_CONDUCT.md

@@ -0,0 +1,46 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at dev@sqlmap.org. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

.github/CONTRIBUTING.md

@@ -24,7 +24,6 @@ Many [people](https://raw.github.com/sqlmapproject/sqlmap/master/doc/THANKS.md)
 In order to maintain consistency and readability throughout the code, we ask that you adhere to the following instructions:
 
 * Each patch should make one logical change.
-* Wrap code to 76 columns when possible.
 * Avoid tabbing, use four blank spaces instead.
 * Before you put time into a non-trivial patch, it is worth discussing it privately by [email](mailto:dev@sqlmap.org).
 * Do not change style on numerous files in one single pull request, we can [discuss](mailto:dev@sqlmap.org) about those before doing any major restyling, but be sure that personal preferences not having a strong support in [PEP 8](http://www.python.org/dev/peps/pep-0008/) will likely to be rejected.

.travis.yml

@@ -1,4 +1,7 @@
 language: python
+sudo: false
+git:
+    depth: 1
 python:
     - "2.6"
     - "2.7"

LICENSE

@@ -1,7 +1,7 @@
 COPYING -- Describes the terms under which sqlmap is distributed. A copy
 of the GNU General Public License (GPL) is appended to this file.
 
-sqlmap is (C) 2006-2017 Bernardo Damele Assumpcao Guimaraes, Miroslav Stampar.
+sqlmap is (C) 2006-2018 Bernardo Damele Assumpcao Guimaraes, Miroslav Stampar.
 
 This program is free software; you may redistribute and/or modify it under
 the terms of the GNU General Public License as published by the Free
@@ -31,6 +31,9 @@ interpretation of derived works with some common examples. Our
 interpretation applies only to sqlmap - we do not speak for other people's
 GPL works.
 
+This license does not apply to the third-party components. More details can
+be found inside the file 'doc/THIRD-PARTY.md'.
+
 If you have any questions about the GPL licensing restrictions on using
 sqlmap in non-GPL works, we would be happy to help. As mentioned above,
 we also offer alternative license to integrate sqlmap into proprietary
@@ -343,29 +346,3 @@ PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
 POSSIBILITY OF SUCH DAMAGES.
 
                      END OF TERMS AND CONDITIONS
-
-****************************************************************************
-
-This license does not apply to the following components:
-
-* The Ansistrm library located under thirdparty/ansistrm/.
-* The Beautiful Soup library located under thirdparty/beautifulsoup/.
-* The Bottle library located under thirdparty/bottle/.
-* The Chardet library located under thirdparty/chardet/.
-* The ClientForm library located under thirdparty/clientform/.
-* The Colorama library located under thirdparty/colorama/.
-* The Fcrypt library located under thirdparty/fcrypt/.
-* The Gprof2dot library located under thirdparty/gprof2dot/.
-* The KeepAlive library located under thirdparty/keepalive/.
-* The Magic library located under thirdparty/magic/.
-* The MultipartPost library located under thirdparty/multipartpost/.
-* The Odict library located under thirdparty/odict/.
-* The Oset library located under thirdparty/oset/.
-* The PrettyPrint library located under thirdparty/prettyprint/.
-* The PyDes library located under thirdparty/pydes/.
-* The SocksiPy library located under thirdparty/socks/.
-* The Termcolor library located under thirdparty/termcolor/.
-* The XDot library located under thirdparty/xdot/.
-* The icmpsh tool located under extra/icmpsh/.
-
-Details for the above packages can be found in the THIRD-PARTY.md file.

README.md

@@ -1,9 +1,11 @@
 # sqlmap
 
-[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
 
 sqlmap is an open source penetration testing tool that automates the process of detecting and exploiting SQL injection flaws and taking over of database servers. It comes with a powerful detection engine, many niche features for the ultimate penetration tester and a broad range of switches lasting from database fingerprinting, over data fetching from the database, to accessing the underlying file system and executing commands on the operating system via out-of-band connections.
 
+**The sqlmap project is sponsored by [Netsparker Web Application Security Scanner](https://www.netsparker.com/?utm_source=github.com&utm_medium=referral&utm_content=sqlmap+repo&utm_campaign=generic+advert).**
+
 Screenshots
 ----
 

doc/translations/README-bg-BG.md

@@ -1,6 +1,6 @@
 # sqlmap
 
-[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![Лиценз](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![Лиценз](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
 
 sqlmap e инструмент за тестване и проникване, с отворен код, който автоматизира процеса на откриване и използване на недостатъците на SQL база данните чрез SQL инжекция, която ги взима от сървъра. Снабден е с мощен детектор, множество специални функции за най-добрия тестер и широк спектър от функции, които могат да се използват за множество цели - извличане на данни от базата данни, достъп до основната файлова система и изпълняване на команди на операционната система.
 

doc/translations/README-es-MX.md

@@ -1,6 +1,6 @@
 # sqlmap
 
-[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
 
 sqlmap es una herramienta para pruebas de penetración "penetration testing" de software libre que automatiza el proceso de detección y explotación de fallos mediante inyección de SQL además de tomar el control de servidores de bases de datos. Contiene un poderoso motor de detección, así como muchas de las funcionalidades escenciales para el "pentester" y una amplia gama de opciones desde la recopilación de información para identificar el objetivo conocido como "fingerprinting" mediante la extracción de información de la base de datos, hasta el acceso al sistema de archivos subyacente para ejecutar comandos en el sistema operativo a través de conexiones alternativas conocidas como "Out-of-band".
 

doc/translations/README-fr-FR.md

@@ -1,6 +1,6 @@
 # sqlmap
 
-[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
 
 **sqlmap** est un outil Open Source de test d'intrusion. Cet outil permet d'automatiser le processus de détection et d'exploitation des failles d'injection SQL afin de prendre le contrôle des serveurs de base de données. __sqlmap__ dispose d'un puissant moteur de détection utilisant les techniques les plus récentes et les plus dévastatrices de tests d'intrusion comme L'Injection SQL, qui permet d'accéder à la base de données, au système de fichiers sous-jacent et permet aussi l'exécution des commandes sur le système d'exploitation.
 
@@ -13,7 +13,7 @@ Les captures d'écran disponible [ici](https://github.com/sqlmapproject/sqlmap/w
 Installation
 ----
 
-Vous pouvez télécharger le plus récent fichier tarball en cliquant [ici](https://github.com/sqlmapproject/sqlmap/tarball/master). Vous pouvez aussi télécharger le plus récent archive zip [ici](https://github.com/sqlmapproject/sqlmap/zipball/master).
+Vous pouvez télécharger le fichier "tarball" le plus récent en cliquant [ici](https://github.com/sqlmapproject/sqlmap/tarball/master). Vous pouvez aussi télécharger l'archive zip la plus récente [ici](https://github.com/sqlmapproject/sqlmap/zipball/master).
 
 De préférence, télécharger __sqlmap__ en le [clonant](https://github.com/sqlmapproject/sqlmap):
 
@@ -21,7 +21,7 @@ De préférence, télécharger __sqlmap__ en le [clonant](https://github.com/sql
 
 sqlmap fonctionne sur n'importe quel système d'exploitation avec la version **2.6.x** et **2.7.x** de [Python](http://www.python.org/download/)  
 
-Usage
+Utilisation
 ----
 
 Pour afficher une liste des fonctions de bases et des commutateurs (switches), tapez:
@@ -33,7 +33,7 @@ Pour afficher une liste complète des options et des commutateurs (switches), ta
     python sqlmap.py -hh
 
 Vous pouvez regarder un vidéo [ici](https://asciinema.org/a/46601) pour plus d'exemples.
-Pour obtenir un aperçu des ressources de __sqlmap__, une liste des fonctionnalités prises en charge et la description de toutes les options, ainsi que des exemples , nous vous recommandons de consulter [le wiki](https://github.com/sqlmapproject/sqlmap/wiki/Usage).
+Pour obtenir un aperçu des ressources de __sqlmap__, une liste des fonctionnalités prises en charge, la description de toutes les options, ainsi que des exemples, nous vous recommandons de consulter [le wiki](https://github.com/sqlmapproject/sqlmap/wiki/Usage).
 
 Liens
 ----
@@ -41,7 +41,7 @@ Liens
 * Page d'acceuil: http://sqlmap.org
 * Téléchargement: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) ou [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master)
 * Commits RSS feed: https://github.com/sqlmapproject/sqlmap/commits/master.atom
-* Issue tracker: https://github.com/sqlmapproject/sqlmap/issues
+* Suivi des issues: https://github.com/sqlmapproject/sqlmap/issues
 * Manuel de l'utilisateur: https://github.com/sqlmapproject/sqlmap/wiki
 * Foire aux questions (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ
 * Twitter: [@sqlmap](https://twitter.com/sqlmap)

doc/translations/README-gr-GR.md

@@ -1,6 +1,6 @@
 # sqlmap
 
-[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
 
 Το sqlmap είναι πρόγραμμα ανοιχτού κώδικα, που αυτοματοποιεί την εύρεση και εκμετάλλευση ευπαθειών τύπου SQL Injection σε βάσεις δεδομένων. Έρχεται με μια δυνατή μηχανή αναγνώρισης ευπαθειών, πολλά εξειδικευμένα χαρακτηριστικά για τον απόλυτο penetration tester όπως και με ένα μεγάλο εύρος επιλογών αρχίζοντας από την αναγνώριση της βάσης δεδομένων, κατέβασμα δεδομένων της βάσης, μέχρι και πρόσβαση στο βαθύτερο σύστημα αρχείων και εκτέλεση εντολών στο απευθείας στο λειτουργικό μέσω εκτός ζώνης συνδέσεων.
 

doc/translations/README-hr-HR.md

@@ -1,6 +1,6 @@
 # sqlmap
 
-[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
 
 sqlmap je alat namijenjen za penetracijsko testiranje koji automatizira proces detekcije i eksploatacije sigurnosnih propusta SQL injekcije te preuzimanje poslužitelja baze podataka. Dolazi s moćnim mehanizmom za detekciju, mnoštvom korisnih opcija za napredno penetracijsko testiranje te široki spektar opcija od onih za prepoznavanja baze podataka, preko dohvaćanja podataka iz baze, do pristupa zahvaćenom datotečnom sustavu i izvršavanja komandi na operacijskom sustavu korištenjem tzv. "out-of-band" veza.
 

doc/translations/README-id-ID.md

@@ -1,6 +1,6 @@
 # sqlmap
 
-[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
 
 sqlmap merupakan alat _(tool)_ bantu _open source_ dalam melakukan tes penetrasi yang mengotomasi proses deteksi dan eksploitasi kelemahan _SQL injection_ dan pengambil-alihan server basisdata. sqlmap dilengkapi dengan pendeteksi canggih, fitur-fitur hanal bagi _penetration tester_, beragam cara untuk mendeteksi basisdata, hingga mengakses _file system_ dan mengeksekusi perintah dalam sistem operasi melalui koneksi _out-of-band_. 
 

doc/translations/README-it-IT.md

@@ -1,6 +1,6 @@
 # sqlmap
 
-[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
 
 sqlmap è uno strumento open source per il penetration testing. Il suo scopo è quello di rendere automatico il processo di scoperta ed exploit di vulnerabilità di tipo SQL injection al fine di compromettere database online. Dispone di un potente motore per la ricerca di vulnerabilità, molti strumenti di nicchia anche per il più esperto penetration tester ed un'ampia gamma di controlli che vanno dal fingerprinting di database allo scaricamento di dati, fino all'accesso al file system sottostante e l'esecuzione di comandi nel sistema operativo attraverso connessioni out-of-band.
 

doc/translations/README-ja-JP.md

@@ -1,6 +1,6 @@
 # sqlmap
 
-[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
 
 sqlmapはオープンソースのペネトレーションテスティングツールです。SQLインジェクションの脆弱性の検出、活用、そしてデータベースサーバ奪取のプロセスを自動化します。
 強力な検出エンジン、ペネトレーションテスターのための多くのニッチ機能、持続的なデータベースのフィンガープリンティングから、データベースのデータ取得やアウトオブバンド接続を介したオペレーティング・システム上でのコマンド実行、ファイルシステムへのアクセスなどの広範囲に及ぶスイッチを提供します。

doc/translations/README-pl-PL.md

@@ -1,6 +1,6 @@
 # sqlmap
 
-[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
 
 sqlmap to open sourceowe narzędzie do testów penetracyjnych, które automatyzuje procesy detekcji, przejmowania i testowania odporności serwerów SQL na podatność na iniekcję niechcianego kodu. Zawiera potężny mechanizm detekcji, wiele niszowych funkcji dla zaawansowanych testów penetracyjnych oraz szeroki wachlarz opcji począwszy od identyfikacji bazy danych, poprzez wydobywanie z nich danych, a nawet pozwalającuch na dostęp do systemu plików o uruchamianie poleceń w systemie operacyjnym serwera poprzez niestandardowe połączenia.
 

doc/translations/README-pt-BR.md

@@ -1,6 +1,6 @@
 # sqlmap
 
-[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
 
 sqlmap é uma ferramenta de teste de penetração de código aberto que automatiza o processo de detecção e exploração de falhas de injeção SQL. Com essa ferramenta é possível assumir total controle de servidores de banco de dados em páginas web vulneráveis, inclusive de base de dados fora do sistema invadido. Ele possui um motor de detecção poderoso, empregando as últimas e mais devastadoras técnicas de teste de penetração por SQL Injection, que permite acessar a base de dados, o sistema de arquivos subjacente e executar comandos no sistema operacional.
 

doc/translations/README-tr-TR.md

@@ -1,6 +1,6 @@
 # sqlmap
 
-[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
 
 sqlmap sql injection açıklarını otomatik olarak tespit ve istismar etmeye yarayan açık kaynak bir penetrasyon aracıdır. sqlmap gelişmiş tespit özelliğinin yanı sıra penetrasyon testleri sırasında gerekli olabilecek bir çok aracı, -uzak veritabınınından, veri indirmek, dosya sistemine erişmek, dosya çalıştırmak gibi - işlevleri de barındırmaktadır.
 

doc/translations/README-zh-CN.md

@@ -1,6 +1,6 @@
 # sqlmap
 
-[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
 
 sqlmap 是一个开源的渗透测试工具，可以用来自动化的检测，利用SQL注入漏洞，获取数据库服务器的权限。它具有功能强大的检测引擎,针对各种不同类型数据库的渗透测试的功能选项，包括获取数据库中存储的数据，访问操作系统文件甚至可以通过外带数据连接的方式执行操作系统命令。
 

extra/__init__.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 pass

extra/beep/__init__.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 pass

extra/beep/beep.py

@@ -3,8 +3,8 @@
 """
 beep.py - Make a beep sound
 
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import os

extra/cloak/__init__.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 pass

extra/cloak/cloak.py

@@ -3,8 +3,8 @@
 """
 cloak.py - Simple file encryption/compression utility
 
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import os

extra/dbgtool/__init__.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 pass

extra/dbgtool/dbgtool.py

@@ -3,8 +3,8 @@
 """
 dbgtool.py - Portable executable to ASCII debug script converter
 
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import os

extra/mssqlsig/update.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import codecs

extra/safe2bin/__init__.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 pass

extra/safe2bin/safe2bin.py

@@ -3,8 +3,8 @@
 """
 safe2bin.py - Simple safe(hex) to binary format converter
 
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import binascii

extra/shutils/blanks.sh

@@ -1,7 +1,7 @@
 #!/bin/bash
 
 # Copyright (c) 2006-2013 sqlmap developers (http://sqlmap.org/)
-# See the file 'doc/COPYING' for copying permission
+# See the file 'LICENSE' for copying permission
 
 # Removes trailing spaces from blank lines inside project files
 find . -type f -iname '*.py' -exec sed -i 's/^[ \t]*$//' {} \;

extra/shutils/duplicates.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
-# Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-# See the file 'doc/COPYING' for copying permission
+# Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+# See the file 'LICENSE' for copying permission
 
 # Removes duplicate entries in wordlist like files
 

extra/shutils/pep8.sh

@@ -1,7 +1,7 @@
 #!/bin/bash
 
 # Copyright (c) 2006-2013 sqlmap developers (http://sqlmap.org/)
-# See the file 'doc/COPYING' for copying permission
+# See the file 'LICENSE' for copying permission
 
 # Runs pep8 on all python files (prerequisite: apt-get install pep8)
 find . -wholename "./thirdparty" -prune -o -type f -iname "*.py" -exec pep8 '{}' \;

extra/shutils/pydiatra.sh

@@ -1,7 +1,7 @@
 #!/bin/bash
 
 # Copyright (c) 2006-2013 sqlmap developers (http://sqlmap.org/)
-# See the file 'doc/COPYING' for copying permission
+# See the file 'LICENSE' for copying permission
 
 # Runs py2diatra on all python files (prerequisite: pip install pydiatra)
 find . -wholename "./thirdparty" -prune -o -type f -iname "*.py" -exec py2diatra '{}' \; | grep -v bare-except

extra/shutils/pyflakes.sh

@@ -1,7 +1,7 @@
 #!/bin/bash
 
 # Copyright (c) 2006-2013 sqlmap developers (http://sqlmap.org/)
-# See the file 'doc/COPYING' for copying permission
+# See the file 'LICENSE' for copying permission
 
 # Runs pyflakes on all python files (prerequisite: apt-get install pyflakes)
 find . -wholename "./thirdparty" -prune -o -type f -iname "*.py" -exec pyflakes '{}' \;

extra/shutils/pylint.py

@@ -20,11 +20,11 @@ def check(module):
         print "CHECKING ", module
         pout = os.popen("pylint --rcfile=/dev/null %s" % module, 'r')
         for line in pout:
-            if  re.match("\AE:", line):
+            if re.match(r"\AE:", line):
                 print line.strip()
             if __RATING__ and "Your code has been rated at" in line:
                 print line
-                score = re.findall("\d.\d\d", line)[0]
+                score = re.findall(r"\d.\d\d", line)[0]
                 total += float(score)
                 count += 1
 

extra/shutils/pypi.sh

@@ -16,8 +16,8 @@ cat > $TMP_DIR/setup.py << EOF
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 from setuptools import setup, find_packages
@@ -60,8 +60,8 @@ cat > sqlmap/__init__.py << EOF
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import os
@@ -161,7 +161,7 @@ Links
 .. |Python 2.6|2.7| image:: https://img.shields.io/badge/python-2.6|2.7-yellow.svg
    :target: https://www.python.org/
 .. |License| image:: https://img.shields.io/badge/license-GPLv2-red.svg
-   :target: https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING
+   :target: https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/LICENSE
 .. |Twitter| image:: https://img.shields.io/badge/twitter-@sqlmap-blue.svg
    :target: https://twitter.com/sqlmap
 

extra/shutils/regressiontest.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
-# Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-# See the file 'doc/COPYING' for copying permission
+# Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+# See the file 'LICENSE' for copying permission
 
 import codecs
 import inspect

extra/sqlharvest/__init__.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 pass

extra/sqlharvest/sqlharvest.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import cookielib

lib/__init__.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 pass

lib/controller/__init__.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 pass

lib/controller/action.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 from lib.controller.handler import setHandler

lib/controller/checks.py

@@ -1,12 +1,13 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import copy
 import httplib
+import logging
 import os
 import random
 import re
@@ -42,6 +43,7 @@ from lib.core.common import readInput
 from lib.core.common import showStaticWords
 from lib.core.common import singleTimeLogMessage
 from lib.core.common import singleTimeWarnMessage
+from lib.core.common import unArrayizeValue
 from lib.core.common import urlencode
 from lib.core.common import wasLastResponseDBMSError
 from lib.core.common import wasLastResponseHTTPError
@@ -73,6 +75,7 @@ from lib.core.settings import CANDIDATE_SENTENCE_MIN_LENGTH
 from lib.core.settings import CHECK_INTERNET_ADDRESS
 from lib.core.settings import CHECK_INTERNET_VALUE
 from lib.core.settings import DEFAULT_GET_POST_DELIMITER
+from lib.core.settings import DEV_EMAIL_ADDRESS
 from lib.core.settings import DUMMY_NON_SQLI_CHECK_APPENDIX
 from lib.core.settings import FI_ERROR_REGEX
 from lib.core.settings import FORMAT_EXCEPTION_STRINGS
@@ -107,6 +110,9 @@ def checkSqlInjection(place, parameter, value):
     if value.isdigit():
         kb.cache.intBoundaries = kb.cache.intBoundaries or sorted(copy.deepcopy(conf.boundaries), key=lambda boundary: any(_ in (boundary.prefix or "") or _ in (boundary.suffix or "") for _ in ('"', '\'')))
         boundaries = kb.cache.intBoundaries
+    elif value.isalpha():
+        kb.cache.alphaBoundaries = kb.cache.alphaBoundaries or sorted(copy.deepcopy(conf.boundaries), key=lambda boundary: not any(_ in (boundary.prefix or "") or _ in (boundary.suffix or "") for _ in ('"', '\'')))
+        boundaries = kb.cache.alphaBoundaries
     else:
         boundaries = conf.boundaries
 
@@ -140,8 +146,7 @@ def checkSqlInjection(place, parameter, value):
                 # error message, simple heuristic check or via DBMS-specific
                 # payload), ask the user to limit the tests to the fingerprinted
                 # DBMS
-                if kb.reduceTests is None and not conf.testFilter and (intersect(Backend.getErrorParsedDBMSes(), \
-                   SUPPORTED_DBMS, True) or kb.heuristicDbms or injection.dbms):
+                if kb.reduceTests is None and not conf.testFilter and (intersect(Backend.getErrorParsedDBMSes(), SUPPORTED_DBMS, True) or kb.heuristicDbms or injection.dbms):
                     msg = "it looks like the back-end DBMS is '%s'. " % (Format.getErrorParsedDBMSes() or kb.heuristicDbms or injection.dbms)
                     msg += "Do you want to skip test payloads specific for other DBMSes? [Y/n]"
                     kb.reduceTests = (Backend.getErrorParsedDBMSes() or [kb.heuristicDbms]) if readInput(msg, default='Y', boolean=True) else []
@@ -150,9 +155,7 @@ def checkSqlInjection(place, parameter, value):
             # message, via simple heuristic check or via DBMS-specific
             # payload), ask the user to extend the tests to all DBMS-specific,
             # regardless of --level and --risk values provided
-            if kb.extendTests is None and not conf.testFilter and (conf.level < 5 or conf.risk < 3) \
-               and (intersect(Backend.getErrorParsedDBMSes(), SUPPORTED_DBMS, True) or \
-               kb.heuristicDbms or injection.dbms):
+            if kb.extendTests is None and not conf.testFilter and (conf.level < 5 or conf.risk < 3) and (intersect(Backend.getErrorParsedDBMSes(), SUPPORTED_DBMS, True) or kb.heuristicDbms or injection.dbms):
                 msg = "for the remaining tests, do you want to include all tests "
                 msg += "for '%s' extending provided " % (Format.getErrorParsedDBMSes() or kb.heuristicDbms or injection.dbms)
                 msg += "level (%d)" % conf.level if conf.level < 5 else ""
@@ -200,7 +203,7 @@ def checkSqlInjection(place, parameter, value):
                     continue
 
                 match = re.search(r"(\d+)-(\d+)", test.request.columns)
-                if injection.data and match:
+                if match and injection.data:
                     lower, upper = int(match.group(1)), int(match.group(2))
                     for _ in (lower, upper):
                         if _ > 1:
@@ -236,9 +239,7 @@ def checkSqlInjection(place, parameter, value):
 
             # Skip tests if title, vector or DBMS is not included by the
             # given test filter
-            if conf.testFilter and not any(conf.testFilter in str(item) or \
-               re.search(conf.testFilter, str(item), re.I) for item in \
-               (test.title, test.vector, payloadDbms)):
+            if conf.testFilter and not any(conf.testFilter in str(item) or re.search(conf.testFilter, str(item), re.I) for item in (test.title, test.vector, payloadDbms)):
                     debugMsg = "skipping test '%s' because its " % title
                     debugMsg += "name/vector/DBMS is not included by the given filter"
                     logger.debug(debugMsg)
@@ -246,9 +247,7 @@ def checkSqlInjection(place, parameter, value):
 
             # Skip tests if title, vector or DBMS is included by the
             # given skip filter
-            if conf.testSkip and any(conf.testSkip in str(item) or \
-               re.search(conf.testSkip, str(item), re.I) for item in \
-               (test.title, test.vector, payloadDbms)):
+            if conf.testSkip and any(conf.testSkip in str(item) or re.search(conf.testSkip, str(item), re.I) for item in (test.title, test.vector, payloadDbms)):
                     debugMsg = "skipping test '%s' because its " % title
                     debugMsg += "name/vector/DBMS is included by the given skip filter"
                     logger.debug(debugMsg)
@@ -280,9 +279,9 @@ def checkSqlInjection(place, parameter, value):
                 # Skip DBMS-specific test if it does not match the
                 # previously identified DBMS (via DBMS-specific error message)
                 if kb.reduceTests and not intersect(payloadDbms, kb.reduceTests, True):
-                    debugMsg = "skipping test '%s' because the parsed " % title
-                    debugMsg += "error message(s) showed that the back-end DBMS "
-                    debugMsg += "could be %s" % Format.getErrorParsedDBMSes()
+                    debugMsg = "skipping test '%s' because the heuristic " % title
+                    debugMsg += "tests showed that the back-end DBMS "
+                    debugMsg += "could be '%s'" % unArrayizeValue(kb.reduceTests)
                     logger.debug(debugMsg)
                     continue
 
@@ -330,6 +329,23 @@ def checkSqlInjection(place, parameter, value):
                 logger.debug(debugMsg)
                 continue
 
+            if stype == PAYLOAD.TECHNIQUE.UNION:
+                match = re.search(r"(\d+)-(\d+)", test.request.columns)
+                if match and not injection.data:
+                    _ = test.request.columns.split('-')[-1]
+                    if conf.uCols is None and _.isdigit() and int(_) > 10:
+                        if kb.futileUnion is None:
+                            msg = "it is not recommended to perform "
+                            msg += "extended UNION tests if there is not "
+                            msg += "at least one other (potential) "
+                            msg += "technique found. Do you want to skip? [Y/n] "
+                            kb.futileUnion = not readInput(msg, default='Y', boolean=True)
+
+                        if kb.futileUnion is False:
+                            debugMsg = "skipping test '%s'" % title
+                            logger.debug(debugMsg)
+                            continue
+
             infoMsg = "testing '%s'" % title
             logger.info(infoMsg)
 
@@ -492,12 +508,16 @@ def checkSqlInjection(place, parameter, value):
                                         errorResult = Request.queryPage(errorPayload, place, raise404=False)
                                         if errorResult:
                                             continue
-                                    elif not any((conf.string, conf.notString, conf.regexp, conf.code, kb.nullConnection)):
+                                    elif kb.heuristicPage and not any((conf.string, conf.notString, conf.regexp, conf.code, kb.nullConnection)):
                                         _ = comparison(kb.heuristicPage, None, getRatioValue=True)
                                         if _ > kb.matchRatio:
                                             kb.matchRatio = _
                                             logger.debug("adjusting match ratio for current parameter to %.3f" % kb.matchRatio)
 
+                                    # Reducing false-positive "appears" messages in heavily dynamic environment
+                                    if kb.heavilyDynamic and not Request.queryPage(reqPayload, place, raise404=False):
+                                        continue
+
                                     injectable = True
 
                                 elif threadData.lastComparisonRatio > UPPER_RATIO_BOUND and not any((conf.string, conf.notString, conf.regexp, conf.code, kb.nullConnection)):
@@ -505,8 +525,13 @@ def checkSqlInjection(place, parameter, value):
                                     trueSet = set(getFilteredPageContent(truePage, True, "\n").split("\n"))
                                     falseSet = set(getFilteredPageContent(falsePage, True, "\n").split("\n"))
 
+                                    if threadData.lastErrorPage and threadData.lastErrorPage[1]:
+                                        errorSet = set(getFilteredPageContent(threadData.lastErrorPage[1], True, "\n").split("\n"))
+                                    else:
+                                        errorSet = set()
+
                                     if originalSet == trueSet != falseSet:
-                                        candidates = trueSet - falseSet
+                                        candidates = trueSet - falseSet - errorSet
 
                                         if candidates:
                                             candidates = sorted(candidates, key=lambda _: len(_))
@@ -534,7 +559,13 @@ def checkSqlInjection(place, parameter, value):
                                         falseSet = set(extractTextTagContent(falseRawResponse))
                                         falseSet = falseSet.union(__ for _ in falseSet for __ in _.split())
 
-                                        candidates = filter(None, (_.strip() if _.strip() in trueRawResponse and _.strip() not in falseRawResponse else None for _ in (trueSet - falseSet)))
+                                        if threadData.lastErrorPage and threadData.lastErrorPage[1]:
+                                            errorSet = set(extractTextTagContent(threadData.lastErrorPage[1]))
+                                            errorSet = errorSet.union(__ for _ in errorSet for __ in _.split())
+                                        else:
+                                            errorSet = set()
+
+                                        candidates = filter(None, (_.strip() if _.strip() in trueRawResponse and _.strip() not in falseRawResponse else None for _ in (trueSet - falseSet - errorSet)))
 
                                         if candidates:
                                             candidates = sorted(candidates, key=lambda _: len(_))
@@ -571,10 +602,10 @@ def checkSqlInjection(place, parameter, value):
                             # body for the test's <grep> regular expression
                             try:
                                 page, headers, _ = Request.queryPage(reqPayload, place, content=True, raise404=False)
-                                output = extractRegexResult(check, page, re.DOTALL | re.IGNORECASE) \
-                                        or extractRegexResult(check, threadData.lastHTTPError[2] if wasLastResponseHTTPError() else None, re.DOTALL | re.IGNORECASE) \
-                                        or extractRegexResult(check, listToStrValue([headers[key] for key in headers.keys() if key.lower() != URI_HTTP_HEADER.lower()] if headers else None), re.DOTALL | re.IGNORECASE) \
-                                        or extractRegexResult(check, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None, re.DOTALL | re.IGNORECASE)
+                                output = extractRegexResult(check, page, re.DOTALL | re.IGNORECASE)
+                                output = output or extractRegexResult(check, threadData.lastHTTPError[2] if wasLastResponseHTTPError() else None, re.DOTALL | re.IGNORECASE)
+                                output = output or extractRegexResult(check, listToStrValue((headers[key] for key in headers.keys() if key.lower() != URI_HTTP_HEADER.lower()) if headers else None), re.DOTALL | re.IGNORECASE)
+                                output = output or extractRegexResult(check, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None, re.DOTALL | re.IGNORECASE)
 
                                 if output:
                                     result = output == "1"
@@ -628,11 +659,12 @@ def checkSqlInjection(place, parameter, value):
                                 Backend.forceDbms(kb.dbmsFilter[0])
                             elif not Backend.getIdentifiedDbms():
                                 if kb.heuristicDbms is None:
-                                    warnMsg = "using unescaped version of the test "
-                                    warnMsg += "because of zero knowledge of the "
-                                    warnMsg += "back-end DBMS. You can try to "
-                                    warnMsg += "explicitly set it with option '--dbms'"
-                                    singleTimeWarnMessage(warnMsg)
+                                    if kb.heuristicTest == HEURISTIC_TEST.POSITIVE or injection.data:
+                                        warnMsg = "using unescaped version of the test "
+                                        warnMsg += "because of zero knowledge of the "
+                                        warnMsg += "back-end DBMS. You can try to "
+                                        warnMsg += "explicitly set it with option '--dbms'"
+                                        singleTimeWarnMessage(warnMsg)
                                 else:
                                     Backend.forceDbms(kb.heuristicDbms)
 
@@ -642,18 +674,6 @@ def checkSqlInjection(place, parameter, value):
                                 infoMsg += "there is at least one other (potential) "
                                 infoMsg += "technique found"
                                 singleTimeLogMessage(infoMsg)
-                            elif not injection.data:
-                                _ = test.request.columns.split('-')[-1]
-                                if _.isdigit() and int(_) > 10:
-                                    if kb.futileUnion is None:
-                                        msg = "it is not recommended to perform "
-                                        msg += "extended UNION tests if there is not "
-                                        msg += "at least one other (potential) "
-                                        msg += "technique found. Do you want to skip? [Y/n] "
-
-                                        kb.futileUnion = not readInput(msg, default='Y', boolean=True)
-                                    if kb.futileUnion is False:
-                                        continue
 
                             # Test for UNION query SQL injection
                             reqPayload, vector = unionTest(comment, place, parameter, value, prefix, suffix)
@@ -670,7 +690,7 @@ def checkSqlInjection(place, parameter, value):
 
                         kb.previousMethod = method
 
-                        if conf.dummy or conf.offline:
+                        if conf.offline:
                             injectable = False
 
                     # If the injection test was successful feed the injection
@@ -855,8 +875,7 @@ def checkFalsePositives(injection):
 
     retVal = True
 
-    if all(_ in (PAYLOAD.TECHNIQUE.BOOLEAN, PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED) for _ in injection.data) or\
-      (len(injection.data) == 1 and PAYLOAD.TECHNIQUE.UNION in injection.data and "Generic" in injection.data[PAYLOAD.TECHNIQUE.UNION].title):
+    if all(_ in (PAYLOAD.TECHNIQUE.BOOLEAN, PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED) for _ in injection.data) or (len(injection.data) == 1 and PAYLOAD.TECHNIQUE.UNION in injection.data and "Generic" in injection.data[PAYLOAD.TECHNIQUE.UNION].title):
         pushValue(kb.injection)
 
         infoMsg = "checking if the injection point on %s " % injection.place
@@ -917,7 +936,7 @@ def checkSuhosinPatch(injection):
 
     if injection.place == PLACE.GET:
         debugMsg = "checking for parameter length "
-        debugMsg += "constrainting mechanisms"
+        debugMsg += "constraining mechanisms"
         logger.debug(debugMsg)
 
         pushValue(kb.injection)
@@ -926,7 +945,7 @@ def checkSuhosinPatch(injection):
         randInt = randomInt()
 
         if not checkBooleanExpression("%d=%s%d" % (randInt, ' ' * SUHOSIN_MAX_VALUE_LENGTH, randInt)):
-            warnMsg = "parameter length constrainting "
+            warnMsg = "parameter length constraining "
             warnMsg += "mechanism detected (e.g. Suhosin patch). "
             warnMsg += "Potential problems in enumeration phase can be expected"
             logger.warn(warnMsg)
@@ -953,7 +972,7 @@ def checkFilteredChars(injection):
 
     # inference techniques depend on character '>'
     if not any(_ in injection.data for _ in (PAYLOAD.TECHNIQUE.ERROR, PAYLOAD.TECHNIQUE.UNION, PAYLOAD.TECHNIQUE.QUERY)):
-        if not checkBooleanExpression("%d>%d" % (randInt+1, randInt)):
+        if not checkBooleanExpression("%d>%d" % (randInt + 1, randInt)):
             warnMsg = "it appears that the character '>' is "
             warnMsg += "filtered by the back-end server. You are strongly "
             warnMsg += "advised to rerun with the '--tamper=between'"
@@ -967,6 +986,11 @@ def heuristicCheckSqlInjection(place, parameter):
         logger.debug(debugMsg)
         return None
 
+    if kb.heavilyDynamic:
+        debugMsg = "heuristic check skipped because of heavy dynamicity"
+        logger.debug(debugMsg)
+        return None
+
     origValue = conf.paramDict[place][parameter]
     paramType = conf.method if conf.method not in (None, HTTPMETHOD.GET, HTTPMETHOD.POST) else place
 
@@ -1050,13 +1074,13 @@ def heuristicCheckSqlInjection(place, parameter):
 
     if value.lower() in (page or "").lower():
         infoMsg = "heuristic (XSS) test shows that %s parameter " % paramType
-        infoMsg += "'%s' might be vulnerable to cross-site scripting attacks" % parameter
+        infoMsg += "'%s' might be vulnerable to cross-site scripting (XSS) attacks" % parameter
         logger.info(infoMsg)
 
     for match in re.finditer(FI_ERROR_REGEX, page or ""):
         if randStr1.lower() in match.group(0).lower():
             infoMsg = "heuristic (FI) test shows that %s parameter " % paramType
-            infoMsg += "'%s' might be vulnerable to file inclusion attacks" % parameter
+            infoMsg += "'%s' might be vulnerable to file inclusion (FI) attacks" % parameter
             logger.info(infoMsg)
             break
 
@@ -1142,16 +1166,18 @@ def checkDynamicContent(firstPage, secondPage):
             count += 1
 
             if count > conf.retries:
-                warnMsg = "target URL is too dynamic. "
+                warnMsg = "target URL content appears to be too dynamic. "
                 warnMsg += "Switching to '--text-only' "
                 logger.warn(warnMsg)
 
                 conf.textOnly = True
                 return
 
-            warnMsg = "target URL is heavily dynamic"
-            warnMsg += ". sqlmap is going to retry the request"
-            logger.critical(warnMsg)
+            warnMsg = "target URL content appears to be heavily dynamic. "
+            warnMsg += "sqlmap is going to retry the request(s)"
+            singleTimeLogMessage(warnMsg, logging.CRITICAL)
+
+            kb.heavilyDynamic = True
 
             secondPage, _, _ = Request.queryPage(content=True)
             findDynamicContent(firstPage, secondPage)
@@ -1167,7 +1193,7 @@ def checkStability():
     like for instance string matching (--string).
     """
 
-    infoMsg = "testing if the target URL is stable"
+    infoMsg = "testing if the target URL content is stable"
     logger.info(infoMsg)
 
     firstPage = kb.originalPage  # set inside checkConnection()
@@ -1185,7 +1211,7 @@ def checkStability():
 
     if kb.pageStable:
         if firstPage:
-            infoMsg = "target URL is stable"
+            infoMsg = "target URL content is stable"
             logger.info(infoMsg)
         else:
             errMsg = "there was an error checking the stability of page "
@@ -1195,12 +1221,11 @@ def checkStability():
             logger.error(errMsg)
 
     else:
-        warnMsg = "target URL is not stable. sqlmap will base the page "
+        warnMsg = "target URL content is not stable. sqlmap will base the page "
         warnMsg += "comparison on a sequence matcher. If no dynamic nor "
         warnMsg += "injectable parameters are detected, or in case of "
         warnMsg += "junk results, refer to user's manual paragraph "
-        warnMsg += "'Page comparison' and provide a string or regular "
-        warnMsg += "expression to match on"
+        warnMsg += "'Page comparison'"
         logger.warn(warnMsg)
 
         message = "how do you want to proceed? [(C)ontinue/(s)tring/(r)egex/(q)uit] "
@@ -1315,14 +1340,19 @@ def checkWaf():
     retVal = False
     payload = "%d %s" % (randomInt(), IDS_WAF_CHECK_PAYLOAD)
 
-    value = "" if not conf.parameters.get(PLACE.GET) else conf.parameters[PLACE.GET] + DEFAULT_GET_POST_DELIMITER
-    value += agent.addPayloadDelimiters("%s=%s" % (randomStr(), payload))
+    if PLACE.URI in conf.parameters:
+        place = PLACE.POST
+        value = "%s=%s" % (randomStr(), agent.addPayloadDelimiters(payload))
+    else:
+        place = PLACE.GET
+        value = "" if not conf.parameters.get(PLACE.GET) else conf.parameters[PLACE.GET] + DEFAULT_GET_POST_DELIMITER
+        value += "%s=%s" % (randomStr(), agent.addPayloadDelimiters(payload))
 
     pushValue(conf.timeout)
     conf.timeout = IDS_WAF_CHECK_TIMEOUT
 
     try:
-        retVal = Request.queryPage(place=PLACE.GET, value=value, getRatioValue=True, noteResponseTime=False, silent=True)[1] < IDS_WAF_CHECK_RATIO
+        retVal = Request.queryPage(place=place, value=value, getRatioValue=True, noteResponseTime=False, silent=True)[1] < IDS_WAF_CHECK_RATIO
     except SqlmapConnectionException:
         retVal = True
     finally:
@@ -1382,6 +1412,9 @@ def identifyWaf():
     retVal = []
 
     for function, product in kb.wafFunctions:
+        if retVal and "unknown" in product.lower():
+            continue
+
         try:
             logger.debug("checking for WAF/IPS/IDS product '%s'" % product)
             found = function(_)
@@ -1408,7 +1441,7 @@ def identifyWaf():
             message = "WAF/IPS/IDS specific response can be found in '%s'. " % filename
             message += "If you know the details on used protection please "
             message += "report it along with specific response "
-            message += "to 'dev@sqlmap.org'"
+            message += "to '%s'" % DEV_EMAIL_ADDRESS
             logger.warn(message)
 
         message = "are you sure that you want to "
@@ -1441,16 +1474,16 @@ def checkNullConnection():
     infoMsg = "testing NULL connection to the target URL"
     logger.info(infoMsg)
 
-    try:
-        pushValue(kb.pageCompress)
-        kb.pageCompress = False
+    pushValue(kb.pageCompress)
+    kb.pageCompress = False
 
-        page, headers, _ = Request.getPage(method=HTTPMETHOD.HEAD)
+    try:
+        page, headers, _ = Request.getPage(method=HTTPMETHOD.HEAD, raise404=False)
 
         if not page and HTTP_HEADER.CONTENT_LENGTH in (headers or {}):
             kb.nullConnection = NULLCONNECTION.HEAD
 
-            infoMsg = "NULL connection is supported with HEAD method (Content-Length)"
+            infoMsg = "NULL connection is supported with HEAD method ('Content-Length')"
             logger.info(infoMsg)
         else:
             page, headers, _ = Request.getPage(auxHeaders={HTTP_HEADER.RANGE: "bytes=-1"})
@@ -1458,11 +1491,10 @@ def checkNullConnection():
             if page and len(page) == 1 and HTTP_HEADER.CONTENT_RANGE in (headers or {}):
                 kb.nullConnection = NULLCONNECTION.RANGE
 
-                infoMsg = "NULL connection is supported with GET method (Range)"
-                infoMsg += "'%s'" % kb.nullConnection
+                infoMsg = "NULL connection is supported with GET method ('Range')"
                 logger.info(infoMsg)
             else:
-                _, headers, _ = Request.getPage(skipRead = True)
+                _, headers, _ = Request.getPage(skipRead=True)
 
                 if HTTP_HEADER.CONTENT_LENGTH in (headers or {}):
                     kb.nullConnection = NULLCONNECTION.SKIP_READ
@@ -1470,9 +1502,8 @@ def checkNullConnection():
                     infoMsg = "NULL connection is supported with 'skip-read' method"
                     logger.info(infoMsg)
 
-    except SqlmapConnectionException, ex:
-        errMsg = getSafeExString(ex)
-        raise SqlmapConnectionException(errMsg)
+    except SqlmapConnectionException:
+        pass
 
     finally:
         kb.pageCompress = popValue()
@@ -1519,6 +1550,16 @@ def checkConnection(suppressOutput=False):
         else:
             kb.errorIsNone = True
 
+
+        threadData = getCurrentThreadData()
+
+        if kb.redirectChoice == REDIRECTION.YES and threadData.lastRedirectURL and threadData.lastRedirectURL[0] == threadData.lastRequestUID:
+            if conf.hostname in (threadData.lastRedirectURL[1] or "") and threadData.lastRedirectURL[1].startswith("https://"):
+                conf.url = re.sub(r"https?://", "https://", conf.url)
+                match = re.search(r":(\d+)", threadData.lastRedirectURL[1])
+                port = match.group(1) if match else 443
+                conf.url = re.sub(r":\d+/", ":%s/" % port, conf.url)
+
     except SqlmapConnectionException, ex:
         if conf.ipv6:
             warnMsg = "check connection to a provided "
@@ -1549,8 +1590,8 @@ def checkInternet():
     content = Request.getPage(url=CHECK_INTERNET_ADDRESS, checking=True)[0]
     return CHECK_INTERNET_VALUE in (content or "")
 
-def setVerbosity():  # Cross-linked function
+def setVerbosity():  # Cross-referenced function
     raise NotImplementedError
 
-def setWafFunctions():  # Cross-linked function
+def setWafFunctions():  # Cross-referenced function
     raise NotImplementedError

lib/controller/controller.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import os
@@ -152,12 +152,15 @@ def _formatInjection(inj):
             vector = "%s%s" % (vector, comment)
         data += "    Type: %s\n" % PAYLOAD.SQLINJECTION[stype]
         data += "    Title: %s\n" % title
-        data += "    Payload: %s\n" % urldecode(payload, unsafe="&", plusspace=(inj.place != PLACE.GET and kb.postSpaceToPlus))
+        data += "    Payload: %s\n" % urldecode(payload, unsafe="&", spaceplus=(inj.place != PLACE.GET and kb.postSpaceToPlus))
         data += "    Vector: %s\n\n" % vector if conf.verbose > 1 else "\n"
 
     return data
 
 def _showInjections():
+    if conf.wizard and kb.wizardMode:
+        kb.wizardMode = False
+
     if kb.testQueryCount > 0:
         header = "sqlmap identified the following injection point(s) with "
         header += "a total of %d HTTP(s) requests" % kb.testQueryCount
@@ -242,11 +245,13 @@ def _saveToResultsFile():
     for key, value in results.items():
         place, parameter, notes = key
         line = "%s,%s,%s,%s,%s%s" % (safeCSValue(kb.originalUrls.get(conf.url) or conf.url), place, parameter, "".join(techniques[_][0].upper() for _ in sorted(value)), notes, os.linesep)
-        conf.resultsFP.writelines(line)
+        conf.resultsFP.write(line)
 
     if not results:
         line = "%s,,,,%s" % (conf.url, os.linesep)
-        conf.resultsFP.writelines(line)
+        conf.resultsFP.write(line)
+
+    conf.resultsFP.flush()
 
 def start():
     """
@@ -281,7 +286,7 @@ def start():
         try:
 
             if conf.checkInternet:
-                infoMsg = "[INFO] checking for Internet connection"
+                infoMsg = "checking for Internet connection"
                 logger.info(infoMsg)
 
                 if not checkInternet():
@@ -366,9 +371,8 @@ def start():
                             conf.data = urldecode(conf.data) if conf.data and urlencode(DEFAULT_GET_POST_DELIMITER, None) not in conf.data else conf.data
 
                         else:
-                            if targetUrl.find("?") > -1:
-                                firstPart = targetUrl[:targetUrl.find("?")]
-                                secondPart = targetUrl[targetUrl.find("?") + 1:]
+                            if '?' in targetUrl:
+                                firstPart, secondPart = targetUrl.split('?', 1)
                                 message = "Edit GET data [default: %s]: " % secondPart
                                 test = readInput(message, default=secondPart)
                                 test = _randomFillBlankFields(test)
@@ -402,8 +406,7 @@ def start():
             if conf.nullConnection:
                 checkNullConnection()
 
-            if (len(kb.injections) == 0 or (len(kb.injections) == 1 and kb.injections[0].place is None)) \
-                and (kb.injection.place is None or kb.injection.parameter is None):
+            if (len(kb.injections) == 0 or (len(kb.injections) == 1 and kb.injections[0].place is None)) and (kb.injection.place is None or kb.injection.parameter is None):
 
                 if not any((conf.string, conf.notString, conf.regexp)) and PAYLOAD.TECHNIQUE.BOOLEAN in conf.tech:
                     # NOTE: this is not needed anymore, leaving only to display
@@ -581,11 +584,11 @@ def start():
                     errMsg += "(e.g. GET parameter 'id' in 'www.site.com/index.php?id=1')"
                     raise SqlmapNoneDataException(errMsg)
                 else:
-                    errMsg = "all tested parameters appear to be not injectable."
+                    errMsg = "all tested parameters do not appear to be injectable."
 
                     if conf.level < 5 or conf.risk < 3:
-                        errMsg += " Try to increase '--level'/'--risk' values "
-                        errMsg += "to perform more tests."
+                        errMsg += " Try to increase values for '--level'/'--risk' options "
+                        errMsg += "if you wish to perform more tests."
 
                     if isinstance(conf.tech, list) and len(conf.tech) < 5:
                         errMsg += " Rerun without providing the option '--technique'."
@@ -608,15 +611,9 @@ def start():
 
                     if kb.heuristicTest == HEURISTIC_TEST.POSITIVE:
                         errMsg += " As heuristic test turned out positive you are "
-                        errMsg += "strongly advised to continue on with the tests. "
-                        errMsg += "Please, consider usage of tampering scripts as "
-                        errMsg += "your target might filter the queries."
+                        errMsg += "strongly advised to continue on with the tests."
 
-                    if not conf.string and not conf.notString and not conf.regexp:
-                        errMsg += " Also, you can try to rerun by providing "
-                        errMsg += "either a valid value for option '--string' "
-                        errMsg += "(or '--regexp')."
-                    elif conf.string:
+                    if conf.string:
                         errMsg += " Also, you can try to rerun by providing a "
                         errMsg += "valid value for option '--string' as perhaps the string you "
                         errMsg += "have chosen does not match "
@@ -629,8 +626,8 @@ def start():
 
                     if not conf.tamper:
                         errMsg += " If you suspect that there is some kind of protection mechanism "
-                        errMsg += "involved (e.g. WAF) maybe you could retry "
-                        errMsg += "with an option '--tamper' (e.g. '--tamper=space2comment')"
+                        errMsg += "involved (e.g. WAF) maybe you could try to use "
+                        errMsg += "option '--tamper' (e.g. '--tamper=space2comment')"
 
                     raise SqlmapNotVulnerableException(errMsg.rstrip('.'))
             else:

lib/controller/handler.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 from lib.core.common import Backend
@@ -56,19 +56,19 @@ def setHandler():
     """
 
     items = [
-                  (DBMS.MYSQL, MYSQL_ALIASES, MySQLMap, MySQLConn),
-                  (DBMS.ORACLE, ORACLE_ALIASES, OracleMap, OracleConn),
-                  (DBMS.PGSQL, PGSQL_ALIASES, PostgreSQLMap, PostgreSQLConn),
-                  (DBMS.MSSQL, MSSQL_ALIASES, MSSQLServerMap, MSSQLServerConn),
-                  (DBMS.SQLITE, SQLITE_ALIASES, SQLiteMap, SQLiteConn),
-                  (DBMS.ACCESS, ACCESS_ALIASES, AccessMap, AccessConn),
-                  (DBMS.FIREBIRD, FIREBIRD_ALIASES, FirebirdMap, FirebirdConn),
-                  (DBMS.MAXDB, MAXDB_ALIASES, MaxDBMap, MaxDBConn),
-                  (DBMS.SYBASE, SYBASE_ALIASES, SybaseMap, SybaseConn),
-                  (DBMS.DB2, DB2_ALIASES, DB2Map, DB2Conn),
-                  (DBMS.HSQLDB, HSQLDB_ALIASES, HSQLDBMap, HSQLDBConn),
-                  (DBMS.INFORMIX, INFORMIX_ALIASES, InformixMap, InformixConn),
-            ]
+        (DBMS.MYSQL, MYSQL_ALIASES, MySQLMap, MySQLConn),
+        (DBMS.ORACLE, ORACLE_ALIASES, OracleMap, OracleConn),
+        (DBMS.PGSQL, PGSQL_ALIASES, PostgreSQLMap, PostgreSQLConn),
+        (DBMS.MSSQL, MSSQL_ALIASES, MSSQLServerMap, MSSQLServerConn),
+        (DBMS.SQLITE, SQLITE_ALIASES, SQLiteMap, SQLiteConn),
+        (DBMS.ACCESS, ACCESS_ALIASES, AccessMap, AccessConn),
+        (DBMS.FIREBIRD, FIREBIRD_ALIASES, FirebirdMap, FirebirdConn),
+        (DBMS.MAXDB, MAXDB_ALIASES, MaxDBMap, MaxDBConn),
+        (DBMS.SYBASE, SYBASE_ALIASES, SybaseMap, SybaseConn),
+        (DBMS.DB2, DB2_ALIASES, DB2Map, DB2Conn),
+        (DBMS.HSQLDB, HSQLDB_ALIASES, HSQLDBMap, HSQLDBConn),
+        (DBMS.INFORMIX, INFORMIX_ALIASES, InformixMap, InformixConn),
+    ]
 
     _ = max(_ if (conf.get("dbms") or Backend.getIdentifiedDbms() or kb.heuristicExtendedDbms or "").lower() in _[1] else None for _ in items)
     if _:

lib/core/__init__.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 pass

lib/core/agent.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import re
@@ -39,6 +39,7 @@ from lib.core.settings import BOUNDED_INJECTION_MARKER
 from lib.core.settings import DEFAULT_COOKIE_DELIMITER
 from lib.core.settings import DEFAULT_GET_POST_DELIMITER
 from lib.core.settings import GENERIC_SQL_COMMENT
+from lib.core.settings import INFERENCE_MARKER
 from lib.core.settings import NULL
 from lib.core.settings import PAYLOAD_DELIMITER
 from lib.core.settings import REPLACEMENT_MARKER
@@ -96,6 +97,7 @@ class Agent(object):
         paramString = conf.parameters[place]
         paramDict = conf.paramDict[place]
         origValue = getUnicode(paramDict[parameter])
+        newValue = getUnicode(newValue) if newValue else newValue
 
         if place == PLACE.URI or BOUNDED_INJECTION_MARKER in origValue:
             paramString = origValue
@@ -104,7 +106,7 @@ class Agent(object):
             else:
                 origValue = filter(None, (re.search(_, origValue.split(BOUNDED_INJECTION_MARKER)[0]) for _ in (r"\w+\Z", r"[^\"'><]+\Z", r"[^ ]+\Z")))[0].group(0)
             origValue = origValue[origValue.rfind('/') + 1:]
-            for char in ('?', '=', ':'):
+            for char in ('?', '=', ':', ','):
                 if char in origValue:
                     origValue = origValue[origValue.rfind(char) + 1:]
         elif place == PLACE.CUSTOM_POST:
@@ -119,8 +121,8 @@ class Agent(object):
                 origValue = _.split('=', 1)[1] if '=' in _ else ""
         elif place == PLACE.CUSTOM_HEADER:
             paramString = origValue
-            origValue = origValue.split(kb.customInjectionMark)[0]
             origValue = origValue[origValue.find(',') + 1:]
+            origValue = origValue.split(kb.customInjectionMark)[0]
             match = re.search(r"([^;]+)=(?P<value>[^;]*);?\Z", origValue)
             if match:
                 origValue = match.group("value")
@@ -137,7 +139,7 @@ class Agent(object):
                 value = origValue
             elif where == PAYLOAD.WHERE.NEGATIVE:
                 if conf.invalidLogical:
-                    match = re.search(r'\A[^ ]+', newValue)
+                    match = re.search(r"\A[^ ]+", newValue)
                     newValue = newValue[len(match.group() if match else ""):]
                     _ = randomInt(2)
                     value = "%s%s AND %s=%s" % (origValue, match.group() if match else "", _, _ + 1)
@@ -206,16 +208,6 @@ class Agent(object):
 
         return retVal
 
-    def fullPayload(self, query):
-        if conf.direct:
-            return self.payloadDirect(query)
-
-        query = self.prefixQuery(query)
-        query = self.suffixQuery(query)
-        payload = self.payload(newValue=query)
-
-        return payload
-
     def prefixQuery(self, expression, prefix=None, where=None, clause=None):
         """
         This method defines how the input expression has to be escaped
@@ -302,24 +294,28 @@ class Agent(object):
         if payload is None:
             return
 
-        _ = (
-                ("[DELIMITER_START]", kb.chars.start), ("[DELIMITER_STOP]", kb.chars.stop),\
-                ("[AT_REPLACE]", kb.chars.at), ("[SPACE_REPLACE]", kb.chars.space), ("[DOLLAR_REPLACE]", kb.chars.dollar),\
-                ("[HASH_REPLACE]", kb.chars.hash_), ("[GENERIC_SQL_COMMENT]", GENERIC_SQL_COMMENT)
-            )
-        payload = reduce(lambda x, y: x.replace(y[0], y[1]), _, payload)
+        replacements = (
+            ("[DELIMITER_START]", kb.chars.start),
+            ("[DELIMITER_STOP]", kb.chars.stop),
+            ("[AT_REPLACE]", kb.chars.at),
+            ("[SPACE_REPLACE]", kb.chars.space),
+            ("[DOLLAR_REPLACE]", kb.chars.dollar),
+            ("[HASH_REPLACE]", kb.chars.hash_),
+            ("[GENERIC_SQL_COMMENT]", GENERIC_SQL_COMMENT)
+        )
+        payload = reduce(lambda x, y: x.replace(y[0], y[1]), replacements, payload)
 
-        for _ in set(re.findall(r"\[RANDNUM(?:\d+)?\]", payload, re.I)):
+        for _ in set(re.findall(r"(?i)\[RANDNUM(?:\d+)?\]", payload)):
             payload = payload.replace(_, str(randomInt()))
 
-        for _ in set(re.findall(r"\[RANDSTR(?:\d+)?\]", payload, re.I)):
+        for _ in set(re.findall(r"(?i)\[RANDSTR(?:\d+)?\]", payload)):
             payload = payload.replace(_, randomStr())
 
         if origValue is not None and "[ORIGVALUE]" in payload:
             origValue = getUnicode(origValue)
             payload = getUnicode(payload).replace("[ORIGVALUE]", origValue if origValue.isdigit() else unescaper.escape("'%s'" % origValue))
 
-        if "[INFERENCE]" in payload:
+        if INFERENCE_MARKER in payload:
             if Backend.getIdentifiedDbms() is not None:
                 inference = queries[Backend.getIdentifiedDbms()].inference
 
@@ -331,7 +327,7 @@ class Agent(object):
                 else:
                     inferenceQuery = inference.query
 
-                payload = payload.replace("[INFERENCE]", inferenceQuery)
+                payload = payload.replace(INFERENCE_MARKER, inferenceQuery)
             elif not kb.testMode:
                 errMsg = "invalid usage of inference payload without "
                 errMsg += "knowledge of underlying DBMS"
@@ -370,7 +366,7 @@ class Agent(object):
         rootQuery = queries[Backend.getIdentifiedDbms()]
         hexField = field
 
-        if 'hex' in rootQuery:
+        if "hex" in rootQuery:
             hexField = rootQuery.hex.query % field
         else:
             warnMsg = "switch '--hex' is currently not supported on DBMS %s" % Backend.getIdentifiedDbms()
@@ -755,13 +751,13 @@ class Agent(object):
             if fromTable and query.endswith(fromTable):
                 query = query[:-len(fromTable)]
 
-        topNumRegex = re.search("\ATOP\s+([\d]+)\s+", query, re.I)
+        topNumRegex = re.search(r"\ATOP\s+([\d]+)\s+", query, re.I)
         if topNumRegex:
             topNum = topNumRegex.group(1)
             query = query[len("TOP %s " % topNum):]
             unionQuery += "TOP %s " % topNum
 
-        intoRegExp = re.search("(\s+INTO (DUMP|OUT)FILE\s+\'(.+?)\')", query, re.I)
+        intoRegExp = re.search(r"(\s+INTO (DUMP|OUT)FILE\s+'(.+?)')", query, re.I)
 
         if intoRegExp:
             intoRegExp = intoRegExp.group(1)
@@ -809,7 +805,7 @@ class Agent(object):
         stopLimit = None
         limitCond = True
 
-        topLimit = re.search("TOP\s+([\d]+)\s+", expression, re.I)
+        topLimit = re.search(r"TOP\s+([\d]+)\s+", expression, re.I)
 
         limitRegExp = re.search(queries[Backend.getIdentifiedDbms()].limitregexp.query, expression, re.I)
 
@@ -936,7 +932,7 @@ class Agent(object):
             limitedQuery += " %s" % limitStr
 
         elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.DB2):
-            if not " ORDER BY " in limitedQuery:
+            if " ORDER BY " not in limitedQuery:
                 limitStr = limitStr.replace(") WHERE LIMIT", " ORDER BY 1 ASC) WHERE LIMIT")
             elif " ORDER BY " in limitedQuery and "SELECT " in limitedQuery:
                 limitedQuery = limitedQuery[:limitedQuery.index(" ORDER BY ")]
@@ -957,7 +953,7 @@ class Agent(object):
                 orderBy = limitedQuery[limitedQuery.index(" ORDER BY "):]
                 limitedQuery = limitedQuery[:limitedQuery.index(" ORDER BY ")]
 
-            notDistincts = re.findall("DISTINCT[\(\s+](.+?)\)*\s+", limitedQuery, re.I)
+            notDistincts = re.findall(r"DISTINCT[\(\s+](.+?)\)*\s+", limitedQuery, re.I)
 
             for notDistinct in notDistincts:
                 limitedQuery = limitedQuery.replace("DISTINCT(%s)" % notDistinct, notDistinct)
@@ -974,7 +970,7 @@ class Agent(object):
                     limitedQuery = limitedQuery.replace(" (SELECT TOP %s" % startTopNums, " (SELECT TOP %d" % num)
                     forgeNotIn = False
                 else:
-                    topNum = re.search("TOP\s+([\d]+)\s+", limitedQuery, re.I).group(1)
+                    topNum = re.search(r"TOP\s+([\d]+)\s+", limitedQuery, re.I).group(1)
                     limitedQuery = limitedQuery.replace("TOP %s " % topNum, "")
 
             if forgeNotIn:
@@ -990,7 +986,7 @@ class Agent(object):
                     limitedQuery += "NOT IN (%s" % (limitStr % num)
                     limitedQuery += "%s %s ORDER BY %s) ORDER BY %s" % (self.nullAndCastField(uniqueField or field), fromFrom, uniqueField or "1", uniqueField or "1")
                 else:
-                    match = re.search(" ORDER BY (\w+)\Z", query)
+                    match = re.search(r" ORDER BY (\w+)\Z", query)
                     field = match.group(1) if match else field
 
                     if " WHERE " in limitedQuery:
@@ -1070,7 +1066,7 @@ class Agent(object):
         """
 
         _ = re.escape(PAYLOAD_DELIMITER)
-        return extractRegexResult("(?s)%s(?P<result>.*?)%s" % (_, _), value)
+        return extractRegexResult(r"(?s)%s(?P<result>.*?)%s" % (_, _), value)
 
     def replacePayload(self, value, payload):
         """
@@ -1078,7 +1074,7 @@ class Agent(object):
         """
 
         _ = re.escape(PAYLOAD_DELIMITER)
-        return re.sub("(?s)(%s.*?%s)" % (_, _), ("%s%s%s" % (PAYLOAD_DELIMITER, getUnicode(payload), PAYLOAD_DELIMITER)).replace("\\", r"\\"), value) if value else value
+        return re.sub(r"(?s)(%s.*?%s)" % (_, _), ("%s%s%s" % (PAYLOAD_DELIMITER, getUnicode(payload), PAYLOAD_DELIMITER)).replace("\\", r"\\"), value) if value else value
 
     def runAsDBMSUser(self, query):
         if conf.dbmsCred and "Ad Hoc Distributed Queries" not in query:

lib/core/bigarray.py

@@ -1,15 +1,16 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 try:
-   import cPickle as pickle
+    import cPickle as pickle
 except:
-   import pickle
+    import pickle
 
+import bz2
 import itertools
 import os
 import sys
@@ -18,6 +19,7 @@ import tempfile
 from lib.core.enums import MKSTEMP_PREFIX
 from lib.core.exception import SqlmapSystemException
 from lib.core.settings import BIGARRAY_CHUNK_SIZE
+from lib.core.settings import BIGARRAY_COMPRESS_LEVEL
 
 DEFAULT_SIZE_OF = sys.getsizeof(object())
 
@@ -27,10 +29,12 @@ def _size_of(object_):
     """
 
     retval = sys.getsizeof(object_, DEFAULT_SIZE_OF)
+
     if isinstance(object_, dict):
         retval += sum(_size_of(_) for _ in itertools.chain.from_iterable(object_.items()))
     elif hasattr(object_, "__iter__"):
         retval += sum(_size_of(_) for _ in object_)
+
     return retval
 
 class Cache(object):
@@ -48,7 +52,7 @@ class BigArray(list):
     List-like class used for storing large amounts of data (disk cached)
     """
 
-    def __init__(self):
+    def __init__(self, items=[]):
         self.chunks = [[]]
         self.chunk_length = sys.maxint
         self.cache = None
@@ -56,13 +60,18 @@ class BigArray(list):
         self._os_remove = os.remove
         self._size_counter = 0
 
+        for item in items:
+            self.append(item)
+
     def append(self, value):
         self.chunks[-1].append(value)
+
         if self.chunk_length == sys.maxint:
             self._size_counter += _size_of(value)
             if self._size_counter >= BIGARRAY_CHUNK_SIZE:
                 self.chunk_length = len(self.chunks[-1])
                 self._size_counter = None
+
         if len(self.chunks[-1]) >= self.chunk_length:
             filename = self._dump(self.chunks[-1])
             self.chunks[-1] = filename
@@ -76,18 +85,20 @@ class BigArray(list):
         if len(self.chunks[-1]) < 1:
             self.chunks.pop()
             try:
-                with open(self.chunks[-1], "rb") as fp:
-                    self.chunks[-1] = pickle.load(fp)
+                with open(self.chunks[-1], "rb") as f:
+                    self.chunks[-1] = pickle.loads(bz2.decompress(f.read()))
             except IOError, ex:
                 errMsg = "exception occurred while retrieving data "
                 errMsg += "from a temporary file ('%s')" % ex.message
-                raise SqlmapSystemException, errMsg
+                raise SqlmapSystemException(errMsg)
+
         return self.chunks[-1].pop()
 
     def index(self, value):
         for index in xrange(len(self)):
             if self[index] == value:
                 return index
+
         return ValueError, "%s is not in list" % value
 
     def _dump(self, chunk):
@@ -95,8 +106,8 @@ class BigArray(list):
             handle, filename = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.BIG_ARRAY)
             self.filenames.add(filename)
             os.close(handle)
-            with open(filename, "w+b") as fp:
-                pickle.dump(chunk, fp, pickle.HIGHEST_PROTOCOL)
+            with open(filename, "w+b") as f:
+                f.write(bz2.compress(pickle.dumps(chunk, pickle.HIGHEST_PROTOCOL), BIGARRAY_COMPRESS_LEVEL))
             return filename
         except (OSError, IOError), ex:
             errMsg = "exception occurred while storing data "
@@ -104,20 +115,21 @@ class BigArray(list):
             errMsg += "make sure that there is enough disk space left. If problem persists, "
             errMsg += "try to set environment variable 'TEMP' to a location "
             errMsg += "writeable by the current user"
-            raise SqlmapSystemException, errMsg
+            raise SqlmapSystemException(errMsg)
 
     def _checkcache(self, index):
         if (self.cache and self.cache.index != index and self.cache.dirty):
             filename = self._dump(self.cache.data)
             self.chunks[self.cache.index] = filename
+
         if not (self.cache and self.cache.index == index):
             try:
-                with open(self.chunks[index], "rb") as fp:
-                    self.cache = Cache(index, pickle.load(fp), False)
+                with open(self.chunks[index], "rb") as f:
+                    self.cache = Cache(index, pickle.loads(bz2.decompress(f.read())), False)
             except IOError, ex:
                 errMsg = "exception occurred while retrieving data "
                 errMsg += "from a temporary file ('%s')" % ex.message
-                raise SqlmapSystemException, errMsg
+                raise SqlmapSystemException(errMsg)
 
     def __getstate__(self):
         return self.chunks, self.filenames
@@ -127,19 +139,19 @@ class BigArray(list):
         self.chunks, self.filenames = state
 
     def __getslice__(self, i, j):
-        retval = BigArray()
         i = max(0, len(self) + i if i < 0 else i)
         j = min(len(self), len(self) + j if j < 0 else j)
-        for _ in xrange(i, j):
-            retval.append(self[_])
-        return retval
+
+        return BigArray(self[_] for _ in xrange(i, j))
 
     def __getitem__(self, y):
         if y < 0:
             y += len(self)
+
         index = y / self.chunk_length
         offset = y % self.chunk_length
         chunk = self.chunks[index]
+
         if isinstance(chunk, list):
             return chunk[offset]
         else:
@@ -150,6 +162,7 @@ class BigArray(list):
         index = y / self.chunk_length
         offset = y % self.chunk_length
         chunk = self.chunks[index]
+
         if isinstance(chunk, list):
             chunk[offset] = value
         else:

lib/core/convert.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 try:
@@ -80,7 +80,7 @@ def base64unpickle(value, unsafe=False):
         if len(self.stack) > 1:
             func = self.stack[-2]
             if func not in PICKLE_REDUCE_WHITELIST:
-                raise Exception, "abusing reduce() is bad, Mkay!"
+                raise Exception("abusing reduce() is bad, Mkay!")
         self.load_reduce()
 
     def loads(str):
@@ -94,7 +94,7 @@ def base64unpickle(value, unsafe=False):
 
     try:
         retVal = loads(base64decode(value))
-    except TypeError: 
+    except TypeError:
         retVal = loads(base64decode(bytes(value)))
 
     return retVal
@@ -166,7 +166,7 @@ def htmlunescape(value):
 
     retVal = value
     if value and isinstance(value, basestring):
-        codes = (('&lt;', '<'), ('&gt;', '>'), ('&quot;', '"'), ('&nbsp;', ' '), ('&amp;', '&'))
+        codes = (("&lt;", '<'), ("&gt;", '>'), ("&quot;", '"'), ("&nbsp;", ' '), ("&amp;", '&'), ("&apos;", "'"))
         retVal = reduce(lambda x, y: x.replace(y[0], y[1]), codes, retVal)
         try:
             retVal = re.sub(r"&#x([^ ;]+);", lambda match: unichr(int(match.group(1), 16)), retVal)
@@ -174,7 +174,7 @@ def htmlunescape(value):
             pass
     return retVal
 
-def singleTimeWarnMessage(message):  # Cross-linked function
+def singleTimeWarnMessage(message):  # Cross-referenced function
     sys.stdout.write(message)
     sys.stdout.write("\n")
     sys.stdout.flush()
@@ -193,7 +193,7 @@ def stdoutencode(data):
                 warnMsg = "cannot properly display Unicode characters "
                 warnMsg += "inside Windows OS command prompt "
                 warnMsg += "(http://bugs.python.org/issue1602). All "
-                warnMsg += "unhandled occurances will result in "
+                warnMsg += "unhandled occurrences will result in "
                 warnMsg += "replacement with '?' character. Please, find "
                 warnMsg += "proper character representation inside "
                 warnMsg += "corresponding output files. "

lib/core/data.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 from lib.core.datatype import AttribDict

lib/core/datatype.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import copy

lib/core/decorators.py

@@ -1,10 +1,12 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
+import hashlib
+
 def cachedmethod(f, cache={}):
     """
     Method with a cached content
@@ -13,14 +15,9 @@ def cachedmethod(f, cache={}):
     """
 
     def _(*args, **kwargs):
-        try:
-            key = (f, tuple(args), frozenset(kwargs.items()))
-            if key not in cache:
-                cache[key] = f(*args, **kwargs)
-        except:
-            key = "".join(str(_) for _ in (f, args, kwargs))
-            if key not in cache:
-                cache[key] = f(*args, **kwargs)
+        key = int(hashlib.md5("".join(str(_) for _ in (f, args, kwargs))).hexdigest()[:8], 16)
+        if key not in cache:
+            cache[key] = f(*args, **kwargs)
 
         return cache[key]
 

lib/core/defaults.py

@@ -1,27 +1,27 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 from lib.core.datatype import AttribDict
 
 _defaults = {
-   "csvDel":       ',',
-   "timeSec":      5,
-   "googlePage":   1,
-   "verbose":      1,
-   "delay":        0,
-   "timeout":      30,
-   "retries":      3,
-   "saFreq":       0,
-   "threads":      1,
-   "level":        1,
-   "risk":         1,
-   "dumpFormat":   "CSV",
-   "tech":         "BEUSTQ",
-   "torType":      "SOCKS5",
+    "csvDel": ',',
+    "timeSec": 5,
+    "googlePage": 1,
+    "verbose": 1,
+    "delay": 0,
+    "timeout": 30,
+    "retries": 3,
+    "saFreq": 0,
+    "threads": 1,
+    "level": 1,
+    "risk": 1,
+    "dumpFormat": "CSV",
+    "tech": "BEUSTQ",
+    "torType": "SOCKS5",
 }
 
 defaults = AttribDict(_defaults)

lib/core/dicts.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 from lib.core.enums import DBMS
@@ -184,10 +184,10 @@ DUMP_REPLACEMENTS = {" ": NULL, "": BLANK}
 
 DBMS_DICT = {
     DBMS.MSSQL: (MSSQL_ALIASES, "python-pymssql", "https://github.com/pymssql/pymssql", "mssql+pymssql"),
-    DBMS.MYSQL: (MYSQL_ALIASES, "python-pymysql", "https://github.com/petehunt/PyMySQL/", "mysql"),
+    DBMS.MYSQL: (MYSQL_ALIASES, "python-pymysql", "https://github.com/PyMySQL/PyMySQL", "mysql"),
     DBMS.PGSQL: (PGSQL_ALIASES, "python-psycopg2", "http://initd.org/psycopg/", "postgresql"),
-    DBMS.ORACLE: (ORACLE_ALIASES, "python cx_Oracle", "http://cx-oracle.sourceforge.net/", "oracle"),
-    DBMS.SQLITE: (SQLITE_ALIASES, "python-sqlite", "http://packages.ubuntu.com/quantal/python-sqlite", "sqlite"),
+    DBMS.ORACLE: (ORACLE_ALIASES, "python cx_Oracle", "https://oracle.github.io/python-cx_Oracle/", "oracle"),
+    DBMS.SQLITE: (SQLITE_ALIASES, "python-sqlite", "https://docs.python.org/2/library/sqlite3.html", "sqlite"),
     DBMS.ACCESS: (ACCESS_ALIASES, "python-pyodbc", "https://github.com/mkleehammer/pyodbc", "access"),
     DBMS.FIREBIRD: (FIREBIRD_ALIASES, "python-kinterbasdb", "http://kinterbasdb.sourceforge.net/", "firebird"),
     DBMS.MAXDB: (MAXDB_ALIASES, None, None, "maxdb"),
@@ -208,54 +208,60 @@ FROM_DUMMY_TABLE = {
 }
 
 SQL_STATEMENTS = {
-    "SQL SELECT statement":  (
-            "select ",
-            "show ",
-            " top ",
-            " distinct ",
-            " from ",
-            " from dual",
-            " where ",
-            " group by ",
-            " order by ",
-            " having ",
-            " limit ",
-            " offset ",
-            " union all ",
-            " rownum as ",
-            "(case ",        ),
+    "SQL SELECT statement": (
+        "select ",
+        "show ",
+        " top ",
+        " distinct ",
+        " from ",
+        " from dual",
+        " where ",
+        " group by ",
+        " order by ",
+        " having ",
+        " limit ",
+        " offset ",
+        " union all ",
+        " rownum as ",
+        "(case ",
+    ),
 
-    "SQL data definition":   (
+    "SQL data definition": (
         "create ",
         "declare ",
         "drop ",
         "truncate ",
-        "alter ",            ),
+        "alter ",
+    ),
 
     "SQL data manipulation": (
-            "bulk ",
-            "insert ",
-            "update ",
-            "delete ",
-            "merge ",
-            "load ",         ),
+        "bulk ",
+        "insert ",
+        "update ",
+        "delete ",
+        "merge ",
+        "load ",
+    ),
 
-    "SQL data control":      (
-            "grant ",
-            "revoke ",       ),
+    "SQL data control": (
+        "grant ",
+        "revoke ",
+    ),
 
-    "SQL data execution":    (
-            "exec ",
-            "execute ",
-            "values ", 
-            "call ",         ),
+    "SQL data execution": (
+        "exec ",
+        "execute ",
+        "values ",
+        "call ",
+    ),
 
-    "SQL transaction":       (
-            "start transaction ",
-            "begin work ",
-            "begin transaction ",
-            "commit ",
-            "rollback ",     ),
+    "SQL transaction": (
+        "start transaction ",
+        "begin work ",
+        "begin transaction ",
+        "commit ",
+        "rollback ",
+    ),
 }
 
 POST_HINT_CONTENT_TYPES = {

lib/core/dump.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import cgi
@@ -46,6 +46,7 @@ from lib.core.settings import METADB_SUFFIX
 from lib.core.settings import MIN_BINARY_DISK_DUMP_SIZE
 from lib.core.settings import TRIM_STDOUT_DUMP_SIZE
 from lib.core.settings import UNICODE_ENCODING
+from lib.core.settings import UNSAFE_DUMP_FILEPATH_REPLACEMENT
 from lib.core.settings import WINDOWS_RESERVED_NAMES
 from thirdparty.magic import magic
 
@@ -140,7 +141,7 @@ class Dump(object):
             try:
                 elements = set(elements)
                 elements = list(elements)
-                elements.sort(key=lambda x: x.lower() if isinstance(x, basestring) else x)
+                elements.sort(key=lambda _: _.lower() if isinstance(_, basestring) else _)
             except:
                 pass
 
@@ -191,7 +192,7 @@ class Dump(object):
             userSettings = userSettings[0]
 
         users = userSettings.keys()
-        users.sort(key=lambda x: x.lower() if isinstance(x, basestring) else x)
+        users.sort(key=lambda _: _.lower() if isinstance(_, basestring) else _)
 
         if conf.api:
             self._write(userSettings, content_type=content_type)
@@ -285,7 +286,7 @@ class Dump(object):
                     colType = None
 
                     colList = columns.keys()
-                    colList.sort(key=lambda x: x.lower() if isinstance(x, basestring) else x)
+                    colList.sort(key=lambda _: _.lower() if isinstance(_, basestring) else _)
 
                     for column in colList:
                         colType = columns[column]
@@ -377,7 +378,7 @@ class Dump(object):
                     if count is None:
                         count = "Unknown"
 
-                    tables.sort(key=lambda x: x.lower() if isinstance(x, basestring) else x)
+                    tables.sort(key=lambda _: _.lower() if isinstance(_, basestring) else _)
 
                     for table in tables:
                         blank1 = " " * (maxlength1 - len(normalizeUnicode(table) or unicode(table)))
@@ -414,16 +415,16 @@ class Dump(object):
         elif conf.dumpFormat in (DUMP_FORMAT.CSV, DUMP_FORMAT.HTML):
             if not os.path.isdir(dumpDbPath):
                 try:
-                    os.makedirs(dumpDbPath, 0755)
+                    os.makedirs(dumpDbPath)
                 except:
                     warnFile = True
 
-                    _ = unicodeencode(re.sub(r"[^\w]", "_", unsafeSQLIdentificatorNaming(db)))
+                    _ = unicodeencode(re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, unsafeSQLIdentificatorNaming(db)))
                     dumpDbPath = os.path.join(conf.dumpPath, "%s-%s" % (_, hashlib.md5(unicodeencode(db)).hexdigest()[:8]))
 
                     if not os.path.isdir(dumpDbPath):
                         try:
-                            os.makedirs(dumpDbPath, 0755)
+                            os.makedirs(dumpDbPath)
                         except Exception, ex:
                             try:
                                 tempDir = tempfile.mkdtemp(prefix="sqlmapdb")
@@ -441,7 +442,7 @@ class Dump(object):
 
                             dumpDbPath = tempDir
 
-            dumpFileName = os.path.join(dumpDbPath, "%s.%s" % (unsafeSQLIdentificatorNaming(table), conf.dumpFormat.lower()))
+            dumpFileName = os.path.join(dumpDbPath, re.sub(r'[\\/]', UNSAFE_DUMP_FILEPATH_REPLACEMENT, "%s.%s" % (unsafeSQLIdentificatorNaming(table), conf.dumpFormat.lower())))
             if not checkFile(dumpFileName, False):
                 try:
                     openFile(dumpFileName, "w+b").close()
@@ -450,9 +451,9 @@ class Dump(object):
                 except:
                     warnFile = True
 
-                    _ = re.sub(r"[^\w]", "_", normalizeUnicode(unsafeSQLIdentificatorNaming(table)))
+                    _ = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, normalizeUnicode(unsafeSQLIdentificatorNaming(table)))
                     if len(_) < len(table) or IS_WIN and table.upper() in WINDOWS_RESERVED_NAMES:
-                        _ = unicodeencode(re.sub(r"[^\w]", "_", unsafeSQLIdentificatorNaming(table)))
+                        _ = unicodeencode(re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, unsafeSQLIdentificatorNaming(table)))
                         dumpFileName = os.path.join(dumpDbPath, "%s-%s.%s" % (_, hashlib.md5(unicodeencode(table)).hexdigest()[:8], conf.dumpFormat.lower()))
                     else:
                         dumpFileName = os.path.join(dumpDbPath, "%s.%s" % (_, conf.dumpFormat.lower()))
@@ -611,9 +612,9 @@ class Dump(object):
                             mimetype = magic.from_buffer(value, mime=True)
                             if any(mimetype.startswith(_) for _ in ("application", "image")):
                                 if not os.path.isdir(dumpDbPath):
-                                    os.makedirs(dumpDbPath, 0755)
+                                    os.makedirs(dumpDbPath)
 
-                                _ = re.sub(r"[^\w]", "_", normalizeUnicode(unsafeSQLIdentificatorNaming(column)))
+                                _ = re.sub(r"[^\w]", UNSAFE_DUMP_FILEPATH_REPLACEMENT, normalizeUnicode(unsafeSQLIdentificatorNaming(column)))
                                 filepath = os.path.join(dumpDbPath, "%s-%d.bin" % (_, randomInt(8)))
                                 warnMsg = "writing binary ('%s') content to file '%s' " % (mimetype, filepath)
                                 logger.warn(warnMsg)

lib/core/enums.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 class PRIORITY:
@@ -22,6 +22,15 @@ class SORT_ORDER:
     FIFTH = 4
     LAST = 100
 
+# Reference: https://docs.python.org/2/library/logging.html#logging-levels
+class LOGGING_LEVELS:
+    NOTSET = 0
+    DEBUG = 10
+    INFO = 20
+    WARNING = 30
+    ERROR = 40
+    CRITICAL = 50
+
 class DBMS:
     ACCESS = "Microsoft Access"
     DB2 = "IBM DB2"
@@ -118,14 +127,30 @@ class HASH:
     MSSQL_OLD = r'(?i)\A0x0100[0-9a-f]{8}[0-9a-f]{80}\Z'
     MSSQL_NEW = r'(?i)\A0x0200[0-9a-f]{8}[0-9a-f]{128}\Z'
     ORACLE = r'(?i)\As:[0-9a-f]{60}\Z'
-    ORACLE_OLD = r'(?i)\A[01-9a-f]{16}\Z'
+    ORACLE_OLD = r'(?i)\A[0-9a-f]{16}\Z'
     MD5_GENERIC = r'(?i)\A[0-9a-f]{32}\Z'
     SHA1_GENERIC = r'(?i)\A[0-9a-f]{40}\Z'
-    SHA224_GENERIC = r'(?i)\A[0-9a-f]{28}\Z'
-    SHA384_GENERIC = r'(?i)\A[0-9a-f]{48}\Z'
-    SHA512_GENERIC = r'(?i)\A[0-9a-f]{64}\Z'
-    CRYPT_GENERIC = r'(?i)\A(?!\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\Z)(?![0-9]+\Z)[./0-9A-Za-z]{13}\Z'
-    WORDPRESS = r'(?i)\A\$P\$[./0-9A-Za-z]{31}\Z'
+    SHA224_GENERIC = r'(?i)\A[0-9a-f]{56}\Z'
+    SHA256_GENERIC = r'(?i)\A[0-9a-f]{64}\Z'
+    SHA384_GENERIC = r'(?i)\A[0-9a-f]{96}\Z'
+    SHA512_GENERIC = r'(?i)\A[0-9a-f]{128}\Z'
+    CRYPT_GENERIC = r'\A(?!\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\Z)(?![0-9]+\Z)[./0-9A-Za-z]{13}\Z'
+    JOOMLA = r'\A[0-9a-f]{32}:\w{32}\Z'
+    WORDPRESS = r'\A\$P\$[./0-9a-zA-Z]{31}\Z'
+    APACHE_MD5_CRYPT = r'\A\$apr1\$.{1,8}\$[./a-zA-Z0-9]+\Z'
+    UNIX_MD5_CRYPT = r'\A\$1\$.{1,8}\$[./a-zA-Z0-9]+\Z'
+    APACHE_SHA1 = r'\A\{SHA\}[a-zA-Z0-9+/]+={0,2}\Z'
+    VBULLETIN = r'\A[0-9a-fA-F]{32}:.{30}\Z'
+    VBULLETIN_OLD = r'\A[0-9a-fA-F]{32}:.{3}\Z'
+    SSHA = r'\A\{SSHA\}[a-zA-Z0-9+/]+={0,2}\Z'
+    SSHA256 = r'\A\{SSHA256\}[a-zA-Z0-9+/]+={0,2}\Z'
+    SSHA512 = r'\A\{SSHA512\}[a-zA-Z0-9+/]+={0,2}\Z'
+    DJANGO_MD5 = r'\Amd5\$[^$]+\$[0-9a-f]{32}\Z'
+    DJANGO_SHA1 = r'\Asha1\$[^$]+\$[0-9a-f]{40}\Z'
+    MD5_BASE64 = r'\A[a-zA-Z0-9+/]{22}==\Z'
+    SHA1_BASE64 = r'\A[a-zA-Z0-9+/]{27}=\Z'
+    SHA256_BASE64 = r'\A[a-zA-Z0-9+/]{43}=\Z'
+    SHA512_BASE64 = r'\A[a-zA-Z0-9+/]{86}==\Z'
 
 # Reference: http://www.zytrax.com/tech/web/mobile_ids.html
 class MOBILES:
@@ -217,40 +242,40 @@ class REDIRECTION:
 
 class PAYLOAD:
     SQLINJECTION = {
-                        1: "boolean-based blind",
-                        2: "error-based",
-                        3: "inline query",
-                        4: "stacked queries",
-                        5: "AND/OR time-based blind",
-                        6: "UNION query",
-                   }
+        1: "boolean-based blind",
+        2: "error-based",
+        3: "inline query",
+        4: "stacked queries",
+        5: "AND/OR time-based blind",
+        6: "UNION query",
+    }
 
     PARAMETER = {
-                    1: "Unescaped numeric",
-                    2: "Single quoted string",
-                    3: "LIKE single quoted string",
-                    4: "Double quoted string",
-                    5: "LIKE double quoted string",
-                }
+        1: "Unescaped numeric",
+        2: "Single quoted string",
+        3: "LIKE single quoted string",
+        4: "Double quoted string",
+        5: "LIKE double quoted string",
+    }
 
     RISK = {
-                0: "No risk",
-                1: "Low risk",
-                2: "Medium risk",
-                3: "High risk",
-           }
+        0: "No risk",
+        1: "Low risk",
+        2: "Medium risk",
+        3: "High risk",
+    }
 
     CLAUSE = {
-                0: "Always",
-                1: "WHERE",
-                2: "GROUP BY",
-                3: "ORDER BY",
-                4: "LIMIT",
-                5: "OFFSET",
-                6: "TOP",
-                7: "Table name",
-                8: "Column name",
-             }
+        0: "Always",
+        1: "WHERE",
+        2: "GROUP BY",
+        3: "ORDER BY",
+        4: "LIMIT",
+        5: "OFFSET",
+        6: "TOP",
+        7: "Table name",
+        8: "Column name",
+    }
 
     class METHOD:
         COMPARISON = "comparison"

lib/core/exception.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 class SqlmapBaseException(Exception):

lib/core/log.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import logging

lib/core/option.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import binascii
@@ -337,7 +337,7 @@ def _feedTargetsDict(reqFile, addedTargetUrls):
 
                 if not host:
                     errMsg = "invalid format of a request file"
-                    raise SqlmapSyntaxException, errMsg
+                    raise SqlmapSyntaxException(errMsg)
 
                 if not url.startswith("http"):
                     url = "%s://%s:%s%s" % (scheme or "http", host, port or "80", url)
@@ -402,7 +402,7 @@ def _loadQueries():
         errMsg = "something appears to be wrong with "
         errMsg += "the file '%s' ('%s'). Please make " % (paths.QUERIES_XML, getSafeExString(ex))
         errMsg += "sure that you haven't made any changes to it"
-        raise SqlmapInstallationException, errMsg
+        raise SqlmapInstallationException(errMsg)
 
     for node in tree.findall("*"):
         queries[node.attrib['value']] = iterate(node)
@@ -434,7 +434,7 @@ def _setMultipleTargets():
         files.sort()
 
         for reqFile in files:
-            if not re.search("([\d]+)\-request", reqFile):
+            if not re.search(r"([\d]+)\-request", reqFile):
                 continue
 
             _feedTargetsDict(os.path.join(conf.logFile, reqFile), addedTargetUrls)
@@ -666,7 +666,7 @@ def _setDBMSAuthentication():
     debugMsg = "setting the DBMS authentication credentials"
     logger.debug(debugMsg)
 
-    match = re.search("^(.+?):(.*?)$", conf.dbmsCred)
+    match = re.search(r"^(.+?):(.*?)$", conf.dbmsCred)
 
     if not match:
         errMsg = "DBMS authentication credentials value must be in format "
@@ -687,12 +687,12 @@ def _setMetasploit():
 
     if IS_WIN:
         try:
-            import win32file
+            __import__("win32file")
         except ImportError:
             errMsg = "sqlmap requires third-party module 'pywin32' "
             errMsg += "in order to use Metasploit functionalities on "
             errMsg += "Windows. You can download it from "
-            errMsg += "'http://sourceforge.net/projects/pywin32/files/pywin32/'"
+            errMsg += "'https://sourceforge.net/projects/pywin32/files/pywin32/'"
             raise SqlmapMissingDependence(errMsg)
 
         if not conf.msfPath:
@@ -700,7 +700,7 @@ def _setMetasploit():
                 retVal = None
 
                 try:
-                    from  _winreg import ConnectRegistry, OpenKey, QueryValueEx, HKEY_LOCAL_MACHINE
+                    from _winreg import ConnectRegistry, OpenKey, QueryValueEx, HKEY_LOCAL_MACHINE
                     _ = ConnectRegistry(None, HKEY_LOCAL_MACHINE)
                     _ = OpenKey(_, key)
                     retVal = QueryValueEx(_, value)[0]
@@ -784,7 +784,7 @@ def _setMetasploit():
 
     if not msfEnvPathExists:
         errMsg = "unable to locate Metasploit Framework installation. "
-        errMsg += "You can get it at 'http://www.metasploit.com/download/'"
+        errMsg += "You can get it at 'https://www.metasploit.com/download/'"
         raise SqlmapFilePathException(errMsg)
 
 def _setWriteFile():
@@ -861,7 +861,7 @@ def _setDBMS():
     logger.debug(debugMsg)
 
     conf.dbms = conf.dbms.lower()
-    regex = re.search("%s ([\d\.]+)" % ("(%s)" % "|".join([alias for alias in SUPPORTED_DBMS])), conf.dbms, re.I)
+    regex = re.search(r"%s ([\d\.]+)" % ("(%s)" % "|".join([alias for alias in SUPPORTED_DBMS])), conf.dbms, re.I)
 
     if regex:
         conf.dbms = regex.group(1)
@@ -931,7 +931,7 @@ def _setTamperingFunctions():
 
             try:
                 module = __import__(filename[:-3].encode(sys.getfilesystemencoding() or UNICODE_ENCODING))
-            except (ImportError, SyntaxError), ex:
+            except Exception, ex:
                 raise SqlmapSyntaxException("cannot import tamper script '%s' (%s)" % (filename[:-3], getSafeExString(ex)))
 
             priority = PRIORITY.NORMAL if not hasattr(module, "__priority__") else module.__priority__
@@ -1046,7 +1046,7 @@ def _setSocketPreConnect():
     if conf.disablePrecon:
         return
 
-    def _():
+    def _thread():
         while kb.get("threadContinue") and not conf.get("disablePrecon"):
             try:
                 for key in socket._ready:
@@ -1078,6 +1078,7 @@ def _setSocketPreConnect():
                     break
                 else:
                     try:
+                        candidate.shutdown(socket.SHUT_RDWR)
                         candidate.close()
                     except socket.error:
                         pass
@@ -1090,7 +1091,7 @@ def _setSocketPreConnect():
         socket.socket._connect = socket.socket.connect
         socket.socket.connect = connect
 
-        thread = threading.Thread(target=_)
+        thread = threading.Thread(target=_thread)
         setDaemon(thread)
         thread.start()
 
@@ -1127,7 +1128,7 @@ def _setHTTPHandlers():
             _ = urlparse.urlsplit(conf.proxy)
         except Exception, ex:
             errMsg = "invalid proxy address '%s' ('%s')" % (conf.proxy, getSafeExString(ex))
-            raise SqlmapSyntaxException, errMsg
+            raise SqlmapSyntaxException(errMsg)
 
         hostnamePort = _.netloc.split(":")
 
@@ -1148,7 +1149,7 @@ def _setHTTPHandlers():
             raise SqlmapSyntaxException(errMsg)
 
         if conf.proxyCred:
-            _ = re.search("^(.*?):(.*?)$", conf.proxyCred)
+            _ = re.search(r"\A(.*?):(.*?)\Z", conf.proxyCred)
             if not _:
                 errMsg = "proxy authentication credentials "
                 errMsg += "value must be in format username:password"
@@ -1254,9 +1255,9 @@ def _setSafeVisit():
                 kb.safeReq.post = None
         else:
             errMsg = "invalid format of a safe request file"
-            raise SqlmapSyntaxException, errMsg
+            raise SqlmapSyntaxException(errMsg)
     else:
-        if not re.search("^http[s]*://", conf.safeUrl):
+        if not re.search(r"\Ahttp[s]*://", conf.safeUrl):
             if ":443/" in conf.safeUrl:
                 conf.safeUrl = "https://" + conf.safeUrl
             else:
@@ -1376,7 +1377,7 @@ def _setHTTPAuthentication():
             except ImportError:
                 errMsg = "sqlmap requires Python NTLM third-party library "
                 errMsg += "in order to authenticate via NTLM, "
-                errMsg += "http://code.google.com/p/python-ntlm/"
+                errMsg += "https://github.com/mullender/python-ntlm"
                 raise SqlmapMissingDependence(errMsg)
 
             authHandler = HTTPNtlmAuthHandler.HTTPNtlmAuthHandler(kb.passwordMgr)
@@ -1488,8 +1489,8 @@ def _setHTTPUserAgent():
 
         userAgent = random.sample(kb.userAgents or [_defaultHTTPUserAgent()], 1)[0]
 
-        infoMsg = "fetched random HTTP User-Agent header from "
-        infoMsg += "file '%s': '%s'" % (paths.USER_AGENTS, userAgent)
+        infoMsg = "fetched random HTTP User-Agent header value '%s' from " % userAgent
+        infoMsg += "file '%s'" % paths.USER_AGENTS
         logger.info(infoMsg)
 
         conf.httpHeaders.append((HTTP_HEADER.USER_AGENT, userAgent))
@@ -1579,7 +1580,7 @@ def _createTemporaryDirectory():
         except (OSError, IOError), ex:
             errMsg = "there has been a problem while accessing "
             errMsg += "temporary directory location(s) ('%s')" % getSafeExString(ex)
-            raise SqlmapSystemException, errMsg
+            raise SqlmapSystemException(errMsg)
     else:
         try:
             if not os.path.isdir(tempfile.gettempdir()):
@@ -1606,7 +1607,7 @@ def _createTemporaryDirectory():
         except (OSError, IOError, WindowsError), ex:
             errMsg = "there has been a problem while setting "
             errMsg += "temporary directory location ('%s')" % getSafeExString(ex)
-            raise SqlmapSystemException, errMsg
+            raise SqlmapSystemException(errMsg)
 
 def _cleanupOptions():
     """
@@ -1647,7 +1648,10 @@ def _cleanupOptions():
         conf.rParam = []
 
     if conf.paramDel and '\\' in conf.paramDel:
-        conf.paramDel = conf.paramDel.decode("string_escape")
+        try:
+            conf.paramDel = conf.paramDel.decode("string_escape")
+        except ValueError:
+            pass
 
     if conf.skip:
         conf.skip = conf.skip.replace(" ", "")
@@ -1661,6 +1665,9 @@ def _cleanupOptions():
     if conf.delay:
         conf.delay = float(conf.delay)
 
+    if conf.url:
+        conf.url = conf.url.strip()
+
     if conf.rFile:
         conf.rFile = ntToPosixSlashes(normalizePath(conf.rFile))
 
@@ -1766,7 +1773,7 @@ def _cleanupOptions():
                 conf.string = conf.string.replace(_.encode("string_escape"), _)
 
     if conf.getAll:
-        map(lambda x: conf.__setitem__(x, True), WIZARD.ALL)
+        map(lambda _: conf.__setitem__(_, True), WIZARD.ALL)
 
     if conf.noCast:
         for _ in DUMP_REPLACEMENTS.keys():
@@ -1781,8 +1788,8 @@ def _cleanupOptions():
     if conf.col:
         conf.col = re.sub(r"\s*,\s*", ',', conf.col)
 
-    if conf.excludeCol:
-        conf.excludeCol = re.sub(r"\s*,\s*", ',', conf.excludeCol)
+    if conf.exclude:
+        conf.exclude = re.sub(r"\s*,\s*", ',', conf.exclude)
 
     if conf.binaryFields:
         conf.binaryFields = re.sub(r"\s*,\s*", ',', conf.binaryFields)
@@ -1886,6 +1893,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.cache.addrinfo = {}
     kb.cache.content = {}
     kb.cache.encoding = {}
+    kb.cache.alphaBoundaries = None
     kb.cache.intBoundaries = None
     kb.cache.parsedDbms = {}
     kb.cache.regex = {}
@@ -1937,6 +1945,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.forcePartialUnion = False
     kb.forceWhere = None
     kb.futileUnion = None
+    kb.heavilyDynamic = False
     kb.headersFp = {}
     kb.heuristicDbms = None
     kb.heuristicExtendedDbms = None
@@ -2030,6 +2039,7 @@ def _setKnowledgeBaseAttributes(flushAll=True):
     kb.uChar = NULL
     kb.unionDuplicates = False
     kb.wafSpecificResponse = None
+    kb.wizardMode = False
     kb.xpCmdshellAvailable = False
 
     if flushAll:
@@ -2097,11 +2107,11 @@ def _useWizardInterface():
             choice = readInput(message, default='1')
 
             if choice == '2':
-                map(lambda x: conf.__setitem__(x, True), WIZARD.INTERMEDIATE)
+                map(lambda _: conf.__setitem__(_, True), WIZARD.INTERMEDIATE)
             elif choice == '3':
-                map(lambda x: conf.__setitem__(x, True), WIZARD.ALL)
+                map(lambda _: conf.__setitem__(_, True), WIZARD.ALL)
             else:
-                map(lambda x: conf.__setitem__(x, True), WIZARD.BASIC)
+                map(lambda _: conf.__setitem__(_, True), WIZARD.BASIC)
 
     logger.debug("muting sqlmap.. it will do the magic for you")
     conf.verbose = 0
@@ -2111,6 +2121,8 @@ def _useWizardInterface():
 
     dataToStdout("\nsqlmap is running, please wait..\n\n")
 
+    kb.wizardMode = True
+
 def _saveConfig():
     """
     Saves the command line options to a sqlmap configuration INI file
@@ -2305,7 +2317,6 @@ def _setTorHttpProxySettings():
         errMsg = "can't establish connection with the Tor HTTP proxy. "
         errMsg += "Please make sure that you have Tor (bundle) installed and setup "
         errMsg += "so you could be able to successfully use switch '--tor' "
-
         raise SqlmapConnectionException(errMsg)
 
     if not conf.checkTor:
@@ -2326,7 +2337,6 @@ def _setTorSocksProxySettings():
         errMsg = "can't establish connection with the Tor SOCKS proxy. "
         errMsg += "Please make sure that you have Tor service installed and setup "
         errMsg += "so you could be able to successfully use switch '--tor' "
-
         raise SqlmapConnectionException(errMsg)
 
     # SOCKS5 to prevent DNS leaks (http://en.wikipedia.org/wiki/Tor_%28anonymity_network%29)
@@ -2339,7 +2349,7 @@ def _checkWebSocket():
             from websocket import ABNF
         except ImportError:
             errMsg = "sqlmap requires third-party module 'websocket-client' "
-            errMsg += "in order to use WebSocket funcionality"
+            errMsg += "in order to use WebSocket functionality"
             raise SqlmapMissingDependence(errMsg)
 
 def _checkTor():
@@ -2531,11 +2541,11 @@ def _basicOptionValidation():
         raise SqlmapSyntaxException(errMsg)
 
     if conf.checkTor and not any((conf.tor, conf.proxy)):
-        errMsg = "switch '--check-tor' requires usage of switch '--tor' (or option '--proxy' with HTTP proxy address using Tor)"
+        errMsg = "switch '--check-tor' requires usage of switch '--tor' (or option '--proxy' with HTTP proxy address of Tor service)"
         raise SqlmapSyntaxException(errMsg)
 
     if conf.torPort is not None and not (isinstance(conf.torPort, int) and conf.torPort >= 0 and conf.torPort <= 65535):
-        errMsg = "value for option '--tor-port' must be in range 0-65535"
+        errMsg = "value for option '--tor-port' must be in range [0, 65535]"
         raise SqlmapSyntaxException(errMsg)
 
     if conf.torType not in getPublicTypeMembers(PROXY_TYPE, True):
@@ -2580,9 +2590,9 @@ def _basicOptionValidation():
     if conf.encoding:
         _ = checkCharEncoding(conf.encoding, False)
         if _ is None:
-            errMsg = "unknown charset '%s'. Please visit " % conf.encoding
+            errMsg = "unknown encoding '%s'. Please visit " % conf.encoding
             errMsg += "'%s' to get the full list of " % CODECS_LIST_PAGE
-            errMsg += "supported charsets"
+            errMsg += "supported encodings"
             raise SqlmapSyntaxException(errMsg)
         else:
             conf.encoding = _

lib/core/optiondict.py

@@ -1,255 +1,257 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 optDict = {
-            # Format:
-            # Family:        { "parameter name":    "parameter datatype" },
-            # Or:
-            # Family:        { "parameter name":    ("parameter datatype", "category name used for common outputs feature") },
-            "Target":        {
-                               "direct":            "string",
-                               "url":               "string",
-                               "logFile":           "string",
-                               "bulkFile":          "string",
-                               "requestFile":       "string",
-                               "sessionFile":       "string",
-                               "googleDork":        "string",
-                               "configFile":        "string",
-                               "sitemapUrl":        "string",
-                             },
+    # Family: {"parameter name": "parameter datatype"},
+    # --OR--
+    # Family: {"parameter name": ("parameter datatype", "category name used for common outputs feature")},
 
-            "Request":       {
-                               "method":            "string",
-                               "data":              "string",
-                               "paramDel":          "string",
-                               "cookie":            "string",
-                               "cookieDel":         "string",
-                               "loadCookies":       "string",
-                               "dropSetCookie":     "boolean",
-                               "agent":             "string",
-                               "randomAgent":       "boolean",
-                               "host":              "string",
-                               "referer":           "string",
-                               "headers":           "string",
-                               "authType":          "string",
-                               "authCred":          "string",
-                               "authFile":          "string",
-                               "ignoreCode":        "integer",
-                               "ignoreProxy":       "boolean",
-                               "ignoreRedirects":   "boolean",
-                               "ignoreTimeouts":    "boolean",
-                               "proxy":             "string",
-                               "proxyCred":         "string",
-                               "proxyFile":         "string",
-                               "tor":               "boolean",
-                               "torPort":           "integer",
-                               "torType":           "string",
-                               "checkTor":          "boolean",
-                               "delay":             "float",
-                               "timeout":           "float",
-                               "retries":           "integer",
-                               "rParam":            "string",
-                               "safeUrl":           "string",
-                               "safePost":          "string",
-                               "safeReqFile":       "string",
-                               "safeFreq":          "integer",
-                               "skipUrlEncode":     "boolean",
-                               "csrfToken":         "string",
-                               "csrfUrl":           "string",
-                               "forceSSL":          "boolean",
-                               "hpp":               "boolean",
-                               "evalCode":          "string",
-                             },
+    "Target": {
+        "direct": "string",
+        "url": "string",
+        "logFile": "string",
+        "bulkFile": "string",
+        "requestFile": "string",
+        "sessionFile": "string",
+        "googleDork": "string",
+        "configFile": "string",
+        "sitemapUrl": "string",
+    },
 
-            "Optimization":  {
-                               "optimize":          "boolean",
-                               "predictOutput":     "boolean",
-                               "keepAlive":         "boolean",
-                               "nullConnection":    "boolean",
-                               "threads":           "integer",
-                             },
+    "Request": {
+        "method": "string",
+        "data": "string",
+        "paramDel": "string",
+        "cookie": "string",
+        "cookieDel": "string",
+        "loadCookies": "string",
+        "dropSetCookie": "boolean",
+        "agent": "string",
+        "randomAgent": "boolean",
+        "host": "string",
+        "referer": "string",
+        "headers": "string",
+        "authType": "string",
+        "authCred": "string",
+        "authFile": "string",
+        "ignoreCode": "integer",
+        "ignoreProxy": "boolean",
+        "ignoreRedirects": "boolean",
+        "ignoreTimeouts": "boolean",
+        "proxy": "string",
+        "proxyCred": "string",
+        "proxyFile": "string",
+        "tor": "boolean",
+        "torPort": "integer",
+        "torType": "string",
+        "checkTor": "boolean",
+        "delay": "float",
+        "timeout": "float",
+        "retries": "integer",
+        "rParam": "string",
+        "safeUrl": "string",
+        "safePost": "string",
+        "safeReqFile": "string",
+        "safeFreq": "integer",
+        "skipUrlEncode": "boolean",
+        "csrfToken": "string",
+        "csrfUrl": "string",
+        "forceSSL": "boolean",
+        "hpp": "boolean",
+        "evalCode": "string",
+    },
 
-            "Injection":     {
-                               "testParameter":     "string",
-                               "skip":              "string",
-                               "skipStatic":        "boolean",
-                               "paramExclude":      "string",
-                               "dbms":              "string",
-                               "dbmsCred":          "string",
-                               "os":                "string",
-                               "invalidBignum":     "boolean",
-                               "invalidLogical":    "boolean",
-                               "invalidString":     "boolean",
-                               "noCast":            "boolean",
-                               "noEscape":          "boolean",
-                               "prefix":            "string",
-                               "suffix":            "string",
-                               "tamper":            "string",
-                             },
+    "Optimization": {
+        "optimize": "boolean",
+        "predictOutput": "boolean",
+        "keepAlive": "boolean",
+        "nullConnection": "boolean",
+        "threads": "integer",
+    },
 
-            "Detection":     {
-                               "level":             "integer",
-                               "risk":              "integer",
-                               "string":            "string",
-                               "notString":         "string",
-                               "regexp":            "string",
-                               "code":              "integer",
-                               "textOnly":          "boolean",
-                               "titles":            "boolean",
-                             },
+    "Injection": {
+        "testParameter": "string",
+        "skip": "string",
+        "skipStatic": "boolean",
+        "paramExclude": "string",
+        "dbms": "string",
+        "dbmsCred": "string",
+        "os": "string",
+        "invalidBignum": "boolean",
+        "invalidLogical": "boolean",
+        "invalidString": "boolean",
+        "noCast": "boolean",
+        "noEscape": "boolean",
+        "prefix": "string",
+        "suffix": "string",
+        "tamper": "string",
+    },
 
-            "Techniques":    {
-                               "tech":              "string",
-                               "timeSec":           "integer",
-                               "uCols":             "string",
-                               "uChar":             "string",
-                               "uFrom":             "string",
-                               "dnsDomain":         "string",
-                               "secondOrder":       "string",
-                             },
+    "Detection": {
+        "level": "integer",
+        "risk": "integer",
+        "string": "string",
+        "notString": "string",
+        "regexp": "string",
+        "code": "integer",
+        "textOnly": "boolean",
+        "titles": "boolean",
+    },
 
-            "Fingerprint":   {
-                               "extensiveFp":       "boolean",
-                             },
+    "Techniques": {
+        "tech": "string",
+        "timeSec": "integer",
+        "uCols": "string",
+        "uChar": "string",
+        "uFrom": "string",
+        "dnsDomain": "string",
+        "secondOrder": "string",
+    },
 
-            "Enumeration":   {
-                               "getAll":            "boolean",
-                               "getBanner":         ("boolean", "Banners"),
-                               "getCurrentUser":    ("boolean", "Users"),
-                               "getCurrentDb":      ("boolean", "Databases"),
-                               "getHostname":       "boolean",
-                               "isDba":             "boolean",
-                               "getUsers":          ("boolean", "Users"),
-                               "getPasswordHashes": ("boolean", "Passwords"),
-                               "getPrivileges":     ("boolean", "Privileges"),
-                               "getRoles":          ("boolean", "Roles"),
-                               "getDbs":            ("boolean", "Databases"),
-                               "getTables":         ("boolean", "Tables"),
-                               "getColumns":        ("boolean", "Columns"),
-                               "getSchema":         "boolean",
-                               "getCount":          "boolean",
-                               "dumpTable":         "boolean",
-                               "dumpAll":           "boolean",
-                               "search":            "boolean",
-                               "getComments":       "boolean",
-                               "db":                "string",
-                               "tbl":               "string",
-                               "col":               "string",
-                               "excludeCol":        "string",
-                               "pivotColumn":       "string",
-                               "dumpWhere":         "string",
-                               "user":              "string",
-                               "excludeSysDbs":     "boolean",
-                               "limitStart":        "integer",
-                               "limitStop":         "integer",
-                               "firstChar":         "integer",
-                               "lastChar":          "integer",
-                               "query":             "string",
-                               "sqlShell":          "boolean",
-                               "sqlFile":           "string",
-                             },
+    "Fingerprint": {
+        "extensiveFp": "boolean",
+    },
 
-            "Brute":         {
-                               "commonTables":       "boolean",
-                               "commonColumns":      "boolean",
-                             },
+    "Enumeration": {
+        "getAll": "boolean",
+        "getBanner": ("boolean", "Banners"),
+        "getCurrentUser": ("boolean", "Users"),
+        "getCurrentDb": ("boolean", "Databases"),
+        "getHostname": "boolean",
+        "isDba": "boolean",
+        "getUsers": ("boolean", "Users"),
+        "getPasswordHashes": ("boolean", "Passwords"),
+        "getPrivileges": ("boolean", "Privileges"),
+        "getRoles": ("boolean", "Roles"),
+        "getDbs": ("boolean", "Databases"),
+        "getTables": ("boolean", "Tables"),
+        "getColumns": ("boolean", "Columns"),
+        "getSchema": "boolean",
+        "getCount": "boolean",
+        "dumpTable": "boolean",
+        "dumpAll": "boolean",
+        "search": "boolean",
+        "getComments": "boolean",
+        "db": "string",
+        "tbl": "string",
+        "col": "string",
+        "exclude": "string",
+        "pivotColumn": "string",
+        "dumpWhere": "string",
+        "user": "string",
+        "excludeSysDbs": "boolean",
+        "limitStart": "integer",
+        "limitStop": "integer",
+        "firstChar": "integer",
+        "lastChar": "integer",
+        "query": "string",
+        "sqlShell": "boolean",
+        "sqlFile": "string",
+    },
 
-            "User-defined function": {
-                               "udfInject":         "boolean",
-                               "shLib":             "string",
-                             },
+    "Brute": {
+        "commonTables": "boolean",
+        "commonColumns": "boolean",
+    },
 
-            "File system":   {
-                               "rFile":             "string",
-                               "wFile":             "string",
-                               "dFile":             "string",
-                             },
+    "User-defined function": {
+        "udfInject": "boolean",
+        "shLib": "string",
+    },
 
-            "Takeover":      {
-                               "osCmd":             "string",
-                               "osShell":           "boolean",
-                               "osPwn":             "boolean",
-                               "osSmb":             "boolean",
-                               "osBof":             "boolean",
-                               "privEsc":           "boolean",
-                               "msfPath":           "string",
-                               "tmpPath":           "string",
-                             },
+    "File system": {
+        "rFile": "string",
+        "wFile": "string",
+        "dFile": "string",
+    },
 
-            "Windows":       {
-                               "regRead":           "boolean",
-                               "regAdd":            "boolean",
-                               "regDel":            "boolean",
-                               "regKey":            "string",
-                               "regVal":            "string",
-                               "regData":           "string",
-                               "regType":           "string",
-                             },
+    "Takeover": {
+        "osCmd": "string",
+        "osShell": "boolean",
+        "osPwn": "boolean",
+        "osSmb": "boolean",
+        "osBof": "boolean",
+        "privEsc": "boolean",
+        "msfPath": "string",
+        "tmpPath": "string",
+    },
 
-            "General":       {
-                               #"xmlFile":           "string",
-                               "trafficFile":       "string",
-                               "batch":             "boolean",
-                               "binaryFields":      "string",
-                               "charset":           "string",
-                               "checkInternet":     "boolean",
-                               "crawlDepth":        "integer",
-                               "crawlExclude":      "string",
-                               "csvDel":            "string",
-                               "dumpFormat":        "string",
-                               "encoding":          "string",
-                               "eta":               "boolean",
-                               "flushSession":      "boolean",
-                               "forms":             "boolean",
-                               "freshQueries":      "boolean",
-                               "harFile":           "string",
-                               "hexConvert":        "boolean",
-                               "outputDir":         "string",
-                               "parseErrors":       "boolean",
-                               "saveConfig":        "string",
-                               "scope":             "string",
-                               "testFilter":        "string",
-                               "testSkip":          "string",
-                               "updateAll":         "boolean",
-                             },
+    "Windows": {
+        "regRead": "boolean",
+        "regAdd": "boolean",
+        "regDel": "boolean",
+        "regKey": "string",
+        "regVal": "string",
+        "regData": "string",
+        "regType": "string",
+    },
 
-            "Miscellaneous": {
-                               "alert":             "string",
-                               "answers":           "string",
-                               "beep":              "boolean",
-                               "cleanup":           "boolean",
-                               "dependencies":      "boolean",
-                               "disableColoring":   "boolean",
-                               "googlePage":        "integer",
-                               "identifyWaf":       "boolean",
-                               "mobile":            "boolean",
-                               "offline":           "boolean",
-                               "purgeOutput":       "boolean",
-                               "skipWaf":           "boolean",
-                               "smart":             "boolean",
-                               "tmpDir":            "string",
-                               "webRoot":           "string",
-                               "wizard":            "boolean",
-                               "verbose":           "integer",
-                             },
-            "Hidden": {
-                               "dummy":             "boolean",
-                               "disablePrecon":     "boolean",
-                               "profile":           "boolean",
-                               "forceDns":          "boolean",
-                               "murphyRate":        "integer",
-                               "smokeTest":         "boolean",
-                               "liveTest":          "boolean",
-                               "stopFail":          "boolean",
-                               "runCase":           "string",
-                      },
-            "API": {
-                               "api":               "boolean",
-                               "taskid":            "string",
-                               "database":          "string",
-                      }
-          }
+    "General": {
+        # "xmlFile": "string",
+        "trafficFile": "string",
+        "batch": "boolean",
+        "binaryFields": "string",
+        "charset": "string",
+        "checkInternet": "boolean",
+        "crawlDepth": "integer",
+        "crawlExclude": "string",
+        "csvDel": "string",
+        "dumpFormat": "string",
+        "encoding": "string",
+        "eta": "boolean",
+        "flushSession": "boolean",
+        "forms": "boolean",
+        "freshQueries": "boolean",
+        "harFile": "string",
+        "hexConvert": "boolean",
+        "outputDir": "string",
+        "parseErrors": "boolean",
+        "saveConfig": "string",
+        "scope": "string",
+        "testFilter": "string",
+        "testSkip": "string",
+        "updateAll": "boolean",
+    },
+
+    "Miscellaneous": {
+        "alert": "string",
+        "answers": "string",
+        "beep": "boolean",
+        "cleanup": "boolean",
+        "dependencies": "boolean",
+        "disableColoring": "boolean",
+        "googlePage": "integer",
+        "identifyWaf": "boolean",
+        "mobile": "boolean",
+        "offline": "boolean",
+        "purgeOutput": "boolean",
+        "skipWaf": "boolean",
+        "smart": "boolean",
+        "tmpDir": "string",
+        "webRoot": "string",
+        "wizard": "boolean",
+        "verbose": "integer",
+    },
+
+    "Hidden": {
+        "dummy": "boolean",
+        "disablePrecon": "boolean",
+        "profile": "boolean",
+        "forceDns": "boolean",
+        "murphyRate": "integer",
+        "smokeTest": "boolean",
+        "liveTest": "boolean",
+        "stopFail": "boolean",
+        "runCase": "string",
+    },
+
+    "API": {
+        "api": "boolean",
+        "taskid": "string",
+        "database": "string",
+    }
+}

lib/core/profiling.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import codecs
@@ -20,9 +20,9 @@ def profile(profileOutputFile=None, dotOutputFile=None, imageOutputFile=None):
     """
 
     try:
+        __import__("gobject")
         from thirdparty.gprof2dot import gprof2dot
         from thirdparty.xdot import xdot
-        import gobject
         import gtk
         import pydot
     except ImportError, e:

lib/core/readlineng.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 from lib.core.data import logger

lib/core/replication.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import sqlite3

lib/core/revision.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import os

lib/core/session.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import re

lib/core/settings.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import os
@@ -19,15 +19,17 @@ from lib.core.enums import DBMS_DIRECTORY_NAME
 from lib.core.enums import OS
 
 # sqlmap version (<major>.<minor>.<month>.<monthly commit>)
-VERSION = "1.1.10.0"
+VERSION = "1.2.4.0"
 TYPE = "dev" if VERSION.count('.') > 2 and VERSION.split('.')[-1] != '0' else "stable"
 TYPE_COLORS = {"dev": 33, "stable": 90, "pip": 34}
 VERSION_STRING = "sqlmap/%s#%s" % ('.'.join(VERSION.split('.')[:-1]) if VERSION.count('.') > 2 and VERSION.split('.')[-1] == '0' else VERSION, TYPE)
 DESCRIPTION = "automatic SQL injection and database takeover tool"
 SITE = "http://sqlmap.org"
+DEV_EMAIL_ADDRESS = "dev@sqlmap.org"
 ISSUES_PAGE = "https://github.com/sqlmapproject/sqlmap/issues/new"
-GIT_REPOSITORY = "git://github.com/sqlmapproject/sqlmap.git"
+GIT_REPOSITORY = "https://github.com/sqlmapproject/sqlmap.git"
 GIT_PAGE = "https://github.com/sqlmapproject/sqlmap"
+ZIPBALL_PAGE = "https://github.com/sqlmapproject/sqlmap/zipball/master"
 
 # colorful banner
 BANNER = """\033[01;33m\
@@ -63,6 +65,7 @@ URI_QUESTION_MARKER = "__QUESTION_MARK__"
 ASTERISK_MARKER = "__ASTERISK_MARK__"
 REPLACEMENT_MARKER = "__REPLACEMENT_MARK__"
 BOUNDED_INJECTION_MARKER = "__BOUNDED_INJECTION_MARK__"
+SAFE_VARIABLE_MARKER = "__SAFE__"
 
 RANDOM_INTEGER_MARKER = "[RANDINT]"
 RANDOM_STRING_MARKER = "[RANDSTR]"
@@ -74,16 +77,19 @@ CHAR_INFERENCE_MARK = "%c"
 PRINTABLE_CHAR_REGEX = r"[^\x00-\x1f\x7f-\xff]"
 
 # Regular expression used for extraction of table names (useful for (e.g.) MsAccess)
-SELECT_FROM_TABLE_REGEX = r"\bSELECT .+? FROM (?P<result>([\w.]|`[^`<>]+`)+)"
+SELECT_FROM_TABLE_REGEX = r"\bSELECT\b.+?\bFROM\s+(?P<result>([\w.]|`[^`<>]+`)+)"
 
 # Regular expression used for recognition of textual content-type
 TEXT_CONTENT_TYPE_REGEX = r"(?i)(text|form|message|xml|javascript|ecmascript|json)"
 
 # Regular expression used for recognition of generic permission messages
-PERMISSION_DENIED_REGEX = r"(command|permission|access)\s*(was|is)?\s*denied"
+PERMISSION_DENIED_REGEX = r"(?P<result>(command|permission|access)\s*(was|is)?\s*denied)"
+
+# Regular expression used in recognition of generic protection mechanisms
+GENERIC_PROTECTION_REGEX = r"(?i)\b(rejected|blocked|protection|incident|denied|detected|dangerous|firewall)\b"
 
 # Regular expression used for recognition of generic maximum connection messages
-MAX_CONNECTIONS_REGEX = r"max.+connections"
+MAX_CONNECTIONS_REGEX = r"\bmax.+?\bconnection"
 
 # Maximum consecutive connection errors before asking the user if he wants to continue
 MAX_CONSECUTIVE_CONNECTION_ERRORS = 15
@@ -100,8 +106,8 @@ GOOGLE_REGEX = r"webcache\.googleusercontent\.com/search\?q=cache:[^:]+:([^+]+)\
 # Regular expression used for extracting results from DuckDuckGo search
 DUCKDUCKGO_REGEX = r'"u":"([^"]+)'
 
-# Regular expression used for extracting results from Disconnect Search
-DISCONNECT_SEARCH_REGEX = r'<p class="url wrapword">([^<]+)</p>'
+# Regular expression used for extracting results from Bing search
+BING_REGEX = r'<h2><a href="([^"]+)" h='
 
 # Dummy user agent for search (if default one returns different results)
 DUMMY_SEARCH_USER_AGENT = "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:49.0) Gecko/20100101 Firefox/49.0"
@@ -191,8 +197,8 @@ UNKNOWN_DBMS = "Unknown"
 # String used for representation of unknown DBMS version
 UNKNOWN_DBMS_VERSION = "Unknown"
 
-# Dynamicity mark length used in dynamicity removal engine
-DYNAMICITY_MARK_LENGTH = 32
+# Dynamicity boundary length used in dynamicity removal engine
+DYNAMICITY_BOUNDARY_LENGTH = 20
 
 # Dummy user prefix used in dictionary attack
 DUMMY_USER_PREFIX = "__dummy__"
@@ -200,6 +206,11 @@ DUMMY_USER_PREFIX = "__dummy__"
 # Reference: http://en.wikipedia.org/wiki/ISO/IEC_8859-1
 DEFAULT_PAGE_ENCODING = "iso-8859-1"
 
+try:
+    unicode(DEFAULT_PAGE_ENCODING, DEFAULT_PAGE_ENCODING)
+except LookupError:
+    DEFAULT_PAGE_ENCODING = "utf8"
+
 # URL used in dummy runs
 DUMMY_URL = "http://foo/bar?id=1"
 
@@ -214,7 +225,7 @@ PYVERSION = sys.version.split()[0]
 MSSQL_SYSTEM_DBS = ("Northwind", "master", "model", "msdb", "pubs", "tempdb")
 MYSQL_SYSTEM_DBS = ("information_schema", "mysql", "performance_schema")
 PGSQL_SYSTEM_DBS = ("information_schema", "pg_catalog", "pg_toast", "pgagent")
-ORACLE_SYSTEM_DBS = ("ANONYMOUS", "APEX_PUBLIC_USER", "CTXSYS", "DBSNMP", "DIP", "EXFSYS", "FLOWS_%", "FLOWS_FILES", "LBACSYS", "MDDATA", "MDSYS", "MGMT_VIEW", "OLAPSYS", "ORACLE_OCM", "ORDDATA", "ORDPLUGINS", "ORDSYS", "OUTLN", "OWBSYS", "SI_INFORMTN_SCHEMA", "SPATIAL_CSW_ADMIN_USR", "SPATIAL_WFS_ADMIN_USR", "SYS", "SYSMAN", "SYSTEM", "WKPROXY", "WKSYS", "WK_TEST", "WMSYS", "XDB", "XS$NULL")  # Reference: https://blog.vishalgupta.com/2011/06/19/predefined-oracle-system-schemas/ 
+ORACLE_SYSTEM_DBS = ('ANONYMOUS', 'APEX_030200', 'APEX_PUBLIC_USER', 'APPQOSSYS', 'BI', 'CTXSYS', 'DBSNMP', 'DIP', 'EXFSYS', 'FLOWS_%', 'FLOWS_FILES', 'HR', 'IX', 'LBACSYS', 'MDDATA', 'MDSYS', 'MGMT_VIEW', 'OC', 'OE', 'OLAPSYS', 'ORACLE_OCM', 'ORDDATA', 'ORDPLUGINS', 'ORDSYS', 'OUTLN', 'OWBSYS', 'PM', 'SCOTT', 'SH', 'SI_INFORMTN_SCHEMA', 'SPATIAL_CSW_ADMIN_USR', 'SPATIAL_WFS_ADMIN_USR', 'SYS', 'SYSMAN', 'SYSTEM', 'WKPROXY', 'WKSYS', 'WK_TEST', 'WMSYS', 'XDB', 'XS$NULL')
 SQLITE_SYSTEM_DBS = ("sqlite_master", "sqlite_temp_master")
 ACCESS_SYSTEM_DBS = ("MSysAccessObjects", "MSysACEs", "MSysObjects", "MSysQueries", "MSysRelationships", "MSysAccessStorage", "MSysAccessXML", "MSysModules", "MSysModules2")
 FIREBIRD_SYSTEM_DBS = ("RDB$BACKUP_HISTORY", "RDB$CHARACTER_SETS", "RDB$CHECK_CONSTRAINTS", "RDB$COLLATIONS", "RDB$DATABASE", "RDB$DEPENDENCIES", "RDB$EXCEPTIONS", "RDB$FIELDS", "RDB$FIELD_DIMENSIONS", " RDB$FILES", "RDB$FILTERS", "RDB$FORMATS", "RDB$FUNCTIONS", "RDB$FUNCTION_ARGUMENTS", "RDB$GENERATORS", "RDB$INDEX_SEGMENTS", "RDB$INDICES", "RDB$LOG_FILES", "RDB$PAGES", "RDB$PROCEDURES", "RDB$PROCEDURE_PARAMETERS", "RDB$REF_CONSTRAINTS", "RDB$RELATIONS", "RDB$RELATION_CONSTRAINTS", "RDB$RELATION_FIELDS", "RDB$ROLES", "RDB$SECURITY_CLASSES", "RDB$TRANSACTIONS", "RDB$TRIGGERS", "RDB$TRIGGER_MESSAGES", "RDB$TYPES", "RDB$USER_PRIVILEGES", "RDB$VIEW_RELATIONS")
@@ -289,6 +300,10 @@ BASIC_HELP_ITEMS = (
     "wizard",
 )
 
+# Tags used for value replacements inside shell scripts
+SHELL_WRITABLE_DIR_TAG = "%WRITABLE_DIR%"
+SHELL_RUNCMD_EXE_TAG = "%RUNCMD_EXE%"
+
 # String representation for NULL value
 NULL = "NULL"
 
@@ -298,13 +313,16 @@ BLANK = "<blank>"
 # String representation for current database
 CURRENT_DB = "CD"
 
+# Name of SQLite file used for storing session data
+SESSION_SQLITE_FILE = "session.sqlite"
+
 # Regular expressions used for finding file paths in error messages
-FILE_PATH_REGEXES = (r"<b>(?P<result>[^<>]+?)</b> on line \d+", r"(?P<result>[^<>'\"]+?)['\"]? on line \d+", r"(?:[>(\[\s])(?P<result>[A-Za-z]:[\\/][\w. \\/-]*)", r"(?:[>(\[\s])(?P<result>/\w[/\w.-]+)", r"href=['\"]file://(?P<result>/[^'\"]+)")
+FILE_PATH_REGEXES = (r"<b>(?P<result>[^<>]+?)</b> on line \d+", r"in (?P<result>[^<>'\"]+?)['\"]? on line \d+", r"(?:[>(\[\s])(?P<result>[A-Za-z]:[\\/][\w. \\/-]*)", r"(?:[>(\[\s])(?P<result>/\w[/\w.~-]+)", r"href=['\"]file://(?P<result>/[^'\"]+)")
 
 # Regular expressions used for parsing error messages (--parse-errors)
 ERROR_PARSING_REGEXES = (
     r"<b>[^<]*(fatal|error|warning|exception)[^<]*</b>:?\s*(?P<result>.+?)<br\s*/?\s*>",
-    r"(?m)^(fatal|error|warning|exception):?\s*(?P<result>[^\n]+?)$",
+    r"(?m)^\s*(fatal|error|warning|exception):?\s*(?P<result>[^\n]+?)$",
     r"(?P<result>[^\n>]*SQL Syntax[^\n<]+)",
     r"<li>Error Type:<br>(?P<result>.+?)</li>",
     r"CDbCommand (?P<result>[^<>\n]*SQL[^<>\n]+)",
@@ -411,6 +429,9 @@ HASH_MOD_ITEM_DISPLAY = 11
 # Maximum integer value
 MAX_INT = sys.maxint
 
+# Replacement for unsafe characters in dump table filenames
+UNSAFE_DUMP_FILEPATH_REPLACEMENT = '_'
+
 # Options that need to be restored in multiple targets run mode
 RESTORE_MERGED_OPTIONS = ("col", "db", "dnsDomain", "privEsc", "tbl", "regexp", "string", "textOnly", "threads", "timeSec", "tmpPath", "uChar", "user")
 
@@ -477,7 +498,7 @@ PICKLE_REDUCE_WHITELIST = (types.BooleanType, types.DictType, types.FloatType, t
 DUMMY_SQL_INJECTION_CHARS = ";()'"
 
 # Simple check against dummy users
-DUMMY_USER_INJECTION = r"(?i)[^\w](AND|OR)\s+[^\s]+[=><]|\bUNION\b.+\bSELECT\b|\bSELECT\b.+\bFROM\b|\b(CONCAT|information_schema|SLEEP|DELAY)\b"
+DUMMY_USER_INJECTION = r"(?i)[^\w](AND|OR)\s+[^\s]+[=><]|\bUNION\b.+\bSELECT\b|\bSELECT\b.+\bFROM\b|\b(CONCAT|information_schema|SLEEP|DELAY|FLOOR\(RAND)\b"
 
 # Extensions skipped by crawler
 CRAWL_EXCLUDE_EXTENSIONS = ("3ds", "3g2", "3gp", "7z", "DS_Store", "a", "aac", "adp", "ai", "aif", "aiff", "apk", "ar", "asf", "au", "avi", "bak", "bin", "bk", "bmp", "btif", "bz2", "cab", "caf", "cgm", "cmx", "cpio", "cr2", "dat", "deb", "djvu", "dll", "dmg", "dmp", "dng", "doc", "docx", "dot", "dotx", "dra", "dsk", "dts", "dtshd", "dvb", "dwg", "dxf", "ear", "ecelp4800", "ecelp7470", "ecelp9600", "egg", "eol", "eot", "epub", "exe", "f4v", "fbs", "fh", "fla", "flac", "fli", "flv", "fpx", "fst", "fvt", "g3", "gif", "gz", "h261", "h263", "h264", "ico", "ief", "image", "img", "ipa", "iso", "jar", "jpeg", "jpg", "jpgv", "jpm", "jxr", "ktx", "lvp", "lz", "lzma", "lzo", "m3u", "m4a", "m4v", "mar", "mdi", "mid", "mj2", "mka", "mkv", "mmr", "mng", "mov", "movie", "mp3", "mp4", "mp4a", "mpeg", "mpg", "mpga", "mxu", "nef", "npx", "o", "oga", "ogg", "ogv", "otf", "pbm", "pcx", "pdf", "pea", "pgm", "pic", "png", "pnm", "ppm", "pps", "ppt", "pptx", "ps", "psd", "pya", "pyc", "pyo", "pyv", "qt", "rar", "ras", "raw", "rgb", "rip", "rlc", "rz", "s3m", "s7z", "scm", "scpt", "sgi", "shar", "sil", "smv", "so", "sub", "swf", "tar", "tbz2", "tga", "tgz", "tif", "tiff", "tlz", "ts", "ttf", "uvh", "uvi", "uvm", "uvp", "uvs", "uvu", "viv", "vob", "war", "wav", "wax", "wbmp", "wdp", "weba", "webm", "webp", "whl", "wm", "wma", "wmv", "wmx", "woff", "woff2", "wvx", "xbm", "xif", "xls", "xlsx", "xlt", "xm", "xpi", "xpm", "xwd", "xz", "z", "zip", "zipx")
@@ -498,7 +519,7 @@ IDS_WAF_CHECK_PAYLOAD = "AND 1=1 UNION ALL SELECT 1,NULL,'<script>alert(\"XSS\")
 SHELLCODEEXEC_RANDOM_STRING_MARKER = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
 
 # Generic address for checking the Internet connection while using switch --check-internet
-CHECK_INTERNET_ADDRESS = "http://ipinfo.io/"
+CHECK_INTERNET_ADDRESS = "https://ipinfo.io/"
 
 # Value to look for in response to CHECK_INTERNET_ADDRESS
 CHECK_INTERNET_VALUE = "IP Address Details"
@@ -518,6 +539,9 @@ ROTATING_CHARS = ('\\', '|', '|', '/', '-')
 # Approximate chunk length (in bytes) used by BigArray objects (only last chunk and cached one are held in memory)
 BIGARRAY_CHUNK_SIZE = 1024 * 1024
 
+# Compress level used for storing BigArray chunks to disk (0-9)
+BIGARRAY_COMPRESS_LEVEL = 9
+
 # Maximum number of socket pre-connects
 SOCKET_PRE_CONNECT_QUEUE_SIZE = 3
 
@@ -595,7 +619,7 @@ MAX_TOTAL_REDIRECTIONS = 10
 MAX_DNS_LABEL = 63
 
 # Alphabet used for prefix and suffix strings of name resolution requests in DNS technique (excluding hexadecimal chars for not mixing with inner content)
-DNS_BOUNDARIES_ALPHABET = re.sub("[a-fA-F]", "", string.ascii_letters)
+DNS_BOUNDARIES_ALPHABET = re.sub(r"[a-fA-F]", "", string.ascii_letters)
 
 # Alphabet used for heuristic checks
 HEURISTIC_CHECK_ALPHABET = ('"', '\'', ')', '(', ',', '.')
@@ -616,7 +640,7 @@ NON_SQLI_CHECK_PREFIX_SUFFIX_LENGTH = 6
 MAX_CONNECTION_CHUNK_SIZE = 10 * 1024 * 1024
 
 # Maximum response total page size (trimmed if larger)
-MAX_CONNECTION_TOTAL_SIZE = 50 * 1024 * 1024
+MAX_CONNECTION_TOTAL_SIZE = 100 * 1024 * 1024
 
 # For preventing MemoryError exceptions (caused when using large sequences in difflib.SequenceMatcher)
 MAX_DIFFLIB_SEQUENCE_LENGTH = 10 * 1024 * 1024
@@ -637,7 +661,7 @@ VALID_TIME_CHARS_RUN_THRESHOLD = 100
 CHECK_ZERO_COLUMNS_THRESHOLD = 10
 
 # Boldify all logger messages containing these "patterns"
-BOLD_PATTERNS = ("' injectable", "provided empty", "leftover chars", "might be injectable", "' is vulnerable", "is not injectable", "does not seem to be", "test failed", "test passed", "live test final result", "test shows that", "the back-end DBMS is", "created Github", "blocked by the target server", "protection is involved", "CAPTCHA", "specific response")
+BOLD_PATTERNS = ("' injectable", "provided empty", "leftover chars", "might be injectable", "' is vulnerable", "is not injectable", "does not seem to be", "test failed", "test passed", "live test final result", "test shows that", "the back-end DBMS is", "created Github", "blocked by the target server", "protection is involved", "CAPTCHA", "specific response", "NULL connection is supported")
 
 # Generic www root directory names
 GENERIC_DOC_ROOT_DIRECTORY_NAMES = ("htdocs", "httpdocs", "public", "wwwroot", "www")

lib/core/shell.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import atexit
@@ -104,20 +104,20 @@ def autoCompletion(completion=None, os=None, commands=None):
         if os == OS.WINDOWS:
             # Reference: http://en.wikipedia.org/wiki/List_of_DOS_commands
             completer = CompleterNG({
-                                      "copy": None, "del": None, "dir": None,
-                                      "echo": None, "md": None, "mem": None,
-                                      "move": None, "net": None, "netstat -na": None,
-                                      "ver": None, "xcopy": None, "whoami": None,
-                                    })
+                "copy": None, "del": None, "dir": None,
+                "echo": None, "md": None, "mem": None,
+                "move": None, "net": None, "netstat -na": None,
+                "ver": None, "xcopy": None, "whoami": None,
+            })
 
         else:
             # Reference: http://en.wikipedia.org/wiki/List_of_Unix_commands
             completer = CompleterNG({
-                                      "cp": None, "rm": None, "ls": None,
-                                      "echo": None, "mkdir": None, "free": None,
-                                      "mv": None, "ifconfig": None, "netstat -natu": None,
-                                      "pwd": None, "uname": None, "id": None,
-                                    })
+                "cp": None, "rm": None, "ls": None,
+                "echo": None, "mkdir": None, "free": None,
+                "mv": None, "ifconfig": None, "netstat -natu": None,
+                "pwd": None, "uname": None, "id": None,
+            })
 
         readline.set_completer(completer.complete)
         readline.parse_and_bind("tab: complete")

lib/core/subprocessng.py

@@ -1,14 +1,13 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import errno
 import os
 import subprocess
-import sys
 import time
 
 from lib.core.settings import IS_WIN
@@ -24,11 +23,6 @@ else:
     import select
     import fcntl
 
-    if (sys.hexversion >> 16) >= 0x202:
-        FCNTL = fcntl
-    else:
-        import FCNTL
-
 def blockingReadFromFD(fd):
     # Quick twist around original Twisted function
     # Blocking read from a non-blocking file descriptor

lib/core/target.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import codecs
@@ -62,6 +62,7 @@ from lib.core.settings import PROBLEMATIC_CUSTOM_INJECTION_PATTERNS
 from lib.core.settings import REFERER_ALIASES
 from lib.core.settings import RESTORE_MERGED_OPTIONS
 from lib.core.settings import RESULTS_FILE_FORMAT
+from lib.core.settings import SESSION_SQLITE_FILE
 from lib.core.settings import SUPPORTED_DBMS
 from lib.core.settings import UNENCODED_ORIGINAL_VALUE
 from lib.core.settings import UNICODE_ENCODING
@@ -141,7 +142,7 @@ def _setRequestParams():
                 if not (kb.processUserMarks and kb.customInjectionMark in conf.data):
                     conf.data = getattr(conf.data, UNENCODED_ORIGINAL_VALUE, conf.data)
                     conf.data = conf.data.replace(kb.customInjectionMark, ASTERISK_MARKER)
-                    conf.data = re.sub(r'("(?P<name>[^"]+)"\s*:\s*"[^"]*)"', functools.partial(process, repl=r'\g<1>%s"' % kb.customInjectionMark), conf.data)
+                    conf.data = re.sub(r'("(?P<name>[^"]+)"\s*:\s*".+?)"(?<!\\")', functools.partial(process, repl=r'\g<1>%s"' % kb.customInjectionMark), conf.data)
                     conf.data = re.sub(r'("(?P<name>[^"]+)"\s*:\s*)(-?\d[\d\.]*)\b', functools.partial(process, repl=r'\g<1>\g<3>%s' % kb.customInjectionMark), conf.data)
                     conf.data = re.sub(r'("(?P<name>[^"]+)"\s*:\s*)((true|false|null))\b', functools.partial(process, repl=r'\g<1>\g<3>%s' % kb.customInjectionMark), conf.data)
                     match = re.search(r'(?P<name>[^"]+)"\s*:\s*\[([^\]]+)\]', conf.data)
@@ -231,7 +232,7 @@ def _setRequestParams():
 
     kb.processUserMarks = True if (kb.postHint and kb.customInjectionMark in conf.data) else kb.processUserMarks
 
-    if re.search(URI_INJECTABLE_REGEX, conf.url, re.I) and not any(place in conf.parameters for place in (PLACE.GET, PLACE.POST)) and not kb.postHint and not kb.customInjectionMark in (conf.data or "") and conf.url.startswith("http"):
+    if re.search(URI_INJECTABLE_REGEX, conf.url, re.I) and not any(place in conf.parameters for place in (PLACE.GET, PLACE.POST)) and not kb.postHint and kb.customInjectionMark not in (conf.data or "") and conf.url.startswith("http"):
         warnMsg = "you've provided target URL without any GET "
         warnMsg += "parameters (e.g. 'http://www.site.com/article.php?id=1') "
         warnMsg += "and without providing any POST parameters "
@@ -376,7 +377,7 @@ def _setRequestParams():
                 if condition:
                     conf.parameters[PLACE.CUSTOM_HEADER] = str(conf.httpHeaders)
                     conf.paramDict[PLACE.CUSTOM_HEADER] = {httpHeader: "%s,%s%s" % (httpHeader, headerValue, kb.customInjectionMark)}
-                    conf.httpHeaders = [(header, value.replace(kb.customInjectionMark, "")) for header, value in conf.httpHeaders]
+                    conf.httpHeaders = [(_[0], _[1].replace(kb.customInjectionMark, "")) for _ in conf.httpHeaders]
                     testableParameters = True
 
     if not conf.parameters:
@@ -390,12 +391,15 @@ def _setRequestParams():
         raise SqlmapGenericException(errMsg)
 
     if conf.csrfToken:
-        if not any(conf.csrfToken in _ for _ in (conf.paramDict.get(PLACE.GET, {}), conf.paramDict.get(PLACE.POST, {}))) and not re.search(r"\b%s\b" % re.escape(conf.csrfToken), conf.data or "") and not conf.csrfToken in set(_[0].lower() for _ in conf.httpHeaders) and not conf.csrfToken in conf.paramDict.get(PLACE.COOKIE, {}):
+        if not any(conf.csrfToken in _ for _ in (conf.paramDict.get(PLACE.GET, {}), conf.paramDict.get(PLACE.POST, {}))) and not re.search(r"\b%s\b" % re.escape(conf.csrfToken), conf.data or "") and conf.csrfToken not in set(_[0].lower() for _ in conf.httpHeaders) and conf.csrfToken not in conf.paramDict.get(PLACE.COOKIE, {}):
             errMsg = "anti-CSRF token parameter '%s' not " % conf.csrfToken
             errMsg += "found in provided GET, POST, Cookie or header values"
             raise SqlmapGenericException(errMsg)
     else:
         for place in (PLACE.GET, PLACE.POST, PLACE.COOKIE):
+            if conf.csrfToken:
+                break
+
             for parameter in conf.paramDict.get(place, {}):
                 if any(parameter.lower().count(_) for _ in CSRF_TOKEN_PARAMETER_INFIXES):
                     message = "%s parameter '%s' appears to hold anti-CSRF token. " % (place, parameter)
@@ -403,7 +407,7 @@ def _setRequestParams():
 
                     if readInput(message, default='N', boolean=True):
                         conf.csrfToken = getUnicode(parameter)
-                    break
+                        break
 
 def _setHashDB():
     """
@@ -411,7 +415,7 @@ def _setHashDB():
     """
 
     if not conf.hashDBFile:
-        conf.hashDBFile = conf.sessionFile or os.path.join(conf.outputPath, "session.sqlite")
+        conf.hashDBFile = conf.sessionFile or os.path.join(conf.outputPath, SESSION_SQLITE_FILE)
 
     if os.path.exists(conf.hashDBFile):
         if conf.flushSession:
@@ -445,13 +449,10 @@ def _resumeHashDBValues():
     conf.tmpPath = conf.tmpPath or hashDBRetrieve(HASHDB_KEYS.CONF_TMP_PATH)
 
     for injection in hashDBRetrieve(HASHDB_KEYS.KB_INJECTIONS, True) or []:
-        if isinstance(injection, InjectionDict) and injection.place in conf.paramDict and \
-            injection.parameter in conf.paramDict[injection.place]:
-
+        if isinstance(injection, InjectionDict) and injection.place in conf.paramDict and injection.parameter in conf.paramDict[injection.place]:
             if not conf.tech or intersect(conf.tech, injection.data.keys()):
                 if intersect(conf.tech, injection.data.keys()):
                     injection.data = dict(_ for _ in injection.data.items() if _[0] in conf.tech)
-
                 if injection not in kb.injections:
                     kb.injections.append(injection)
 
@@ -545,7 +546,7 @@ def _setResultsFile():
     if not conf.resultsFP:
         conf.resultsFilename = os.path.join(paths.SQLMAP_OUTPUT_PATH, time.strftime(RESULTS_FILE_FORMAT).lower())
         try:
-            conf.resultsFP = openFile(conf.resultsFilename, "w+", UNICODE_ENCODING, buffering=0)
+            conf.resultsFP = openFile(conf.resultsFilename, "a", UNICODE_ENCODING, buffering=0)
         except (OSError, IOError), ex:
             try:
                 warnMsg = "unable to create results file '%s' ('%s'). " % (conf.resultsFilename, getUnicode(ex))
@@ -577,7 +578,7 @@ def _createFilesDir():
 
     if not os.path.isdir(conf.filePath):
         try:
-            os.makedirs(conf.filePath, 0755)
+            os.makedirs(conf.filePath)
         except OSError, ex:
             tempDir = tempfile.mkdtemp(prefix="sqlmapfiles")
             warnMsg = "unable to create files directory "
@@ -599,7 +600,7 @@ def _createDumpDir():
 
     if not os.path.isdir(conf.dumpPath):
         try:
-            os.makedirs(conf.dumpPath, 0755)
+            os.makedirs(conf.dumpPath)
         except OSError, ex:
             tempDir = tempfile.mkdtemp(prefix="sqlmapdump")
             warnMsg = "unable to create dump directory "
@@ -620,7 +621,7 @@ def _createTargetDirs():
 
     try:
         if not os.path.isdir(paths.SQLMAP_OUTPUT_PATH):
-            os.makedirs(paths.SQLMAP_OUTPUT_PATH, 0755)
+            os.makedirs(paths.SQLMAP_OUTPUT_PATH)
 
         _ = os.path.join(paths.SQLMAP_OUTPUT_PATH, randomStr())
         open(_, "w+b").close()
@@ -650,7 +651,7 @@ def _createTargetDirs():
 
     try:
         if not os.path.isdir(conf.outputPath):
-            os.makedirs(conf.outputPath, 0755)
+            os.makedirs(conf.outputPath)
     except (OSError, IOError, TypeError), ex:
         try:
             tempDir = tempfile.mkdtemp(prefix="sqlmapoutput")

lib/core/testing.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import codecs

lib/core/threads.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import difflib
@@ -64,9 +64,6 @@ class _ThreadData(threading.local):
 
 ThreadData = _ThreadData()
 
-def getCurrentThreadUID():
-    return hash(threading.currentThread())
-
 def readInput(message, default=None, checkBatch=True, boolean=False):
     # It will be overwritten by original from lib.core.common
     pass
@@ -98,6 +95,9 @@ def exceptionHandledFunction(threadFunction, silent=False):
         if not silent:
             logger.error("thread %s: %s" % (threading.currentThread().getName(), ex.message))
 
+            if conf.verbose > 1:
+                traceback.print_exc()
+
 def setDaemon(thread):
     # Reference: http://stackoverflow.com/questions/190010/daemon-threads-explanation
     if PYVERSION >= "2.6":
@@ -188,6 +188,9 @@ def runThreads(numThreads, threadFunction, cleanupFunction=None, forwardExceptio
         kb.threadException = True
         logger.error("thread %s: %s" % (threading.currentThread().getName(), ex.message))
 
+        if conf.verbose > 1:
+            traceback.print_exc()
+
     except:
         from lib.core.common import unhandledExceptionMessage
 

lib/core/unescaper.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 from lib.core.common import Backend

lib/core/update.py

@@ -1,25 +1,32 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
-import locale
+import glob
 import os
 import re
+import shutil
 import subprocess
+import sys
 import time
+import urllib
+import zipfile
 
 from lib.core.common import dataToStdout
 from lib.core.common import getSafeExString
 from lib.core.common import pollProcess
+from lib.core.common import readInput
 from lib.core.data import conf
 from lib.core.data import logger
 from lib.core.data import paths
 from lib.core.revision import getRevisionNumber
 from lib.core.settings import GIT_REPOSITORY
 from lib.core.settings import IS_WIN
+from lib.core.settings import ZIPBALL_PAGE
+from lib.core.settings import UNICODE_ENCODING
 
 def update():
     if not conf.updateAll:
@@ -28,11 +35,56 @@ def update():
     success = False
 
     if not os.path.exists(os.path.join(paths.SQLMAP_ROOT_PATH, ".git")):
-        errMsg = "not a git repository. Please checkout the 'sqlmapproject/sqlmap' repository "
-        errMsg += "from GitHub (e.g. 'git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap')"
-        logger.error(errMsg)
+        warnMsg = "not a git repository. It is recommended to clone the 'sqlmapproject/sqlmap' repository "
+        warnMsg += "from GitHub (e.g. 'git clone --depth 1 %s sqlmap')" % GIT_REPOSITORY
+        logger.warn(warnMsg)
+
+        message = "do you want to try to fetch the latest 'zipball' from repository and extract it (experimental) ? [y/N]"
+        if readInput(message, default='N', boolean=True):
+            directory = os.path.abspath(paths.SQLMAP_ROOT_PATH)
+
+            try:
+                open(os.path.join(directory, "sqlmap.py"), "w+b")
+            except Exception, ex:
+                errMsg = "unable to update content of directory '%s' ('%s')" % (directory, getSafeExString(ex))
+                logger.error(errMsg)
+            else:
+                for wildcard in ('*', ".*"):
+                    for _ in glob.glob(os.path.join(directory, wildcard)):
+                        try:
+                            if os.path.isdir(_):
+                                shutil.rmtree(_)
+                            else:
+                                os.remove(_)
+                        except:
+                            pass
+
+                if glob.glob(os.path.join(directory, '*')):
+                    errMsg = "unable to clear the content of directory '%s'" % directory
+                    logger.error(errMsg)
+                else:
+                    try:
+                        archive = urllib.urlretrieve(ZIPBALL_PAGE)[0]
+
+                        with zipfile.ZipFile(archive) as f:
+                            for info in f.infolist():
+                                info.filename = re.sub(r"\Asqlmap[^/]+", "", info.filename)
+                                if info.filename:
+                                    f.extract(info, directory)
+
+                        filepath = os.path.join(paths.SQLMAP_ROOT_PATH, "lib", "core", "settings.py")
+                        if os.path.isfile(filepath):
+                            with open(filepath, "rb") as f:
+                                version = re.search(r"(?m)^VERSION\s*=\s*['\"]([^'\"]+)", f.read()).group(1)
+                                logger.info("updated to the latest version '%s#dev'" % version)
+                                success = True
+                    except Exception, ex:
+                        logger.error("update could not be completed ('%s')" % getSafeExString(ex))
+                    else:
+                        if not success:
+                            logger.error("update could not be completed")
     else:
-        infoMsg = "updating sqlmap to the latest development version from the "
+        infoMsg = "updating sqlmap to the latest development revision from the "
         infoMsg += "GitHub repository"
         logger.info(infoMsg)
 
@@ -42,7 +94,7 @@ def update():
         dataToStdout("\r[%s] [INFO] update in progress " % time.strftime("%X"))
 
         try:
-            process = subprocess.Popen("git checkout . && git pull %s HEAD" % GIT_REPOSITORY, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=paths.SQLMAP_ROOT_PATH.encode(locale.getpreferredencoding()))  # Reference: http://blog.stastnarodina.com/honza-en/spot/python-unicodeencodeerror/
+            process = subprocess.Popen("git checkout . && git pull %s HEAD" % GIT_REPOSITORY, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=paths.SQLMAP_ROOT_PATH.encode(sys.getfilesystemencoding() or UNICODE_ENCODING))
             pollProcess(process, True)
             stdout, stderr = process.communicate()
             success = not process.returncode
@@ -55,7 +107,7 @@ def update():
         else:
             if "Not a git repository" in stderr:
                 errMsg = "not a valid git repository. Please checkout the 'sqlmapproject/sqlmap' repository "
-                errMsg += "from GitHub (e.g. 'git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap')"
+                errMsg += "from GitHub (e.g. 'git clone --depth 1 %s sqlmap')" % GIT_REPOSITORY
                 logger.error(errMsg)
             else:
                 logger.error("update could not be completed ('%s')" % re.sub(r"\W+", " ", stderr).strip())
@@ -68,7 +120,7 @@ def update():
             infoMsg += "download the latest snapshot from "
             infoMsg += "https://github.com/sqlmapproject/sqlmap/downloads"
         else:
-            infoMsg = "for Linux platform it's required "
+            infoMsg = "for Linux platform it's recommended "
             infoMsg += "to install a standard 'git' package (e.g.: 'sudo apt-get install git')"
 
         logger.info(infoMsg)

lib/core/wordlist.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import os
@@ -47,7 +47,7 @@ class Wordlist(object):
                     errMsg = "something appears to be wrong with "
                     errMsg += "the file '%s' ('%s'). Please make " % (self.current, getSafeExString(ex))
                     errMsg += "sure that you haven't made any changes to it"
-                    raise SqlmapInstallationException, errMsg
+                    raise SqlmapInstallationException(errMsg)
                 if len(_.namelist()) == 0:
                     errMsg = "no file(s) inside '%s'" % self.current
                     raise SqlmapDataException(errMsg)
@@ -73,7 +73,7 @@ class Wordlist(object):
                 errMsg = "something appears to be wrong with "
                 errMsg += "the file '%s' ('%s'). Please make " % (self.current, getSafeExString(ex))
                 errMsg += "sure that you haven't made any changes to it"
-                raise SqlmapInstallationException, errMsg
+                raise SqlmapInstallationException(errMsg)
             except StopIteration:
                 self.adjust()
                 retVal = self.iter.next().rstrip()

lib/parse/__init__.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 pass

lib/parse/banner.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import re

lib/parse/cmdline.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import os
@@ -50,9 +50,7 @@ def cmdLineParser(argv=None):
     # Reference: https://stackoverflow.com/a/4012683 (Note: previously used "...sys.getfilesystemencoding() or UNICODE_ENCODING")
     _ = getUnicode(os.path.basename(argv[0]), encoding=sys.stdin.encoding)
 
-    usage = "%s%s [options]" % ("python " if not IS_WIN else "", \
-            "\"%s\"" % _ if " " in _ else _)
-
+    usage = "%s%s [options]" % ("python " if not IS_WIN else "", "\"%s\"" % _ if " " in _ else _)
     parser = OptionParser(usage=usage)
 
     try:
@@ -115,15 +113,13 @@ def cmdLineParser(argv=None):
         request.add_option("--load-cookies", dest="loadCookies",
                            help="File containing cookies in Netscape/wget format")
 
-        request.add_option("--drop-set-cookie", dest="dropSetCookie",
-                           action="store_true",
+        request.add_option("--drop-set-cookie", dest="dropSetCookie", action="store_true",
                            help="Ignore Set-Cookie header from response")
 
         request.add_option("--user-agent", dest="agent",
                            help="HTTP User-Agent header value")
 
-        request.add_option("--random-agent", dest="randomAgent",
-                           action="store_true",
+        request.add_option("--random-agent", dest="randomAgent", action="store_true",
                            help="Use randomly selected HTTP User-Agent header value")
 
         request.add_option("--host", dest="host",
@@ -139,62 +135,55 @@ def cmdLineParser(argv=None):
                            help="Extra headers (e.g. \"Accept-Language: fr\\nETag: 123\")")
 
         request.add_option("--auth-type", dest="authType",
-                           help="HTTP authentication type "
-                                "(Basic, Digest, NTLM or PKI)")
+                           help="HTTP authentication type (Basic, Digest, NTLM or PKI)")
 
         request.add_option("--auth-cred", dest="authCred",
-                           help="HTTP authentication credentials "
-                                "(name:password)")
+                           help="HTTP authentication credentials (name:password)")
 
         request.add_option("--auth-file", dest="authFile",
                            help="HTTP authentication PEM cert/private key file")
 
         request.add_option("--ignore-code", dest="ignoreCode", type="int",
-                          help="Ignore HTTP error code (e.g. 401)")
+                           help="Ignore HTTP error code (e.g. 401)")
 
         request.add_option("--ignore-proxy", dest="ignoreProxy", action="store_true",
                            help="Ignore system default proxy settings")
 
         request.add_option("--ignore-redirects", dest="ignoreRedirects", action="store_true",
-                          help="Ignore redirection attempts")
+                           help="Ignore redirection attempts")
 
         request.add_option("--ignore-timeouts", dest="ignoreTimeouts", action="store_true",
-                          help="Ignore connection timeouts")
+                           help="Ignore connection timeouts")
 
         request.add_option("--proxy", dest="proxy",
                            help="Use a proxy to connect to the target URL")
 
         request.add_option("--proxy-cred", dest="proxyCred",
-                           help="Proxy authentication credentials "
-                                "(name:password)")
+                           help="Proxy authentication credentials (name:password)")
 
         request.add_option("--proxy-file", dest="proxyFile",
                            help="Load proxy list from a file")
 
-        request.add_option("--tor", dest="tor",
-                                  action="store_true",
-                                  help="Use Tor anonymity network")
+        request.add_option("--tor", dest="tor", action="store_true",
+                           help="Use Tor anonymity network")
 
         request.add_option("--tor-port", dest="torPort",
-                                  help="Set Tor proxy port other than default")
+                           help="Set Tor proxy port other than default")
 
         request.add_option("--tor-type", dest="torType",
-                                  help="Set Tor proxy type (HTTP, SOCKS4 or SOCKS5 (default))")
+                           help="Set Tor proxy type (HTTP, SOCKS4 or SOCKS5 (default))")
 
-        request.add_option("--check-tor", dest="checkTor",
-                                  action="store_true",
-                                  help="Check to see if Tor is used properly")
+        request.add_option("--check-tor", dest="checkTor", action="store_true",
+                           help="Check to see if Tor is used properly")
 
         request.add_option("--delay", dest="delay", type="float",
                            help="Delay in seconds between each HTTP request")
 
         request.add_option("--timeout", dest="timeout", type="float",
-                           help="Seconds to wait before timeout connection "
-                                "(default %d)" % defaults.timeout)
+                           help="Seconds to wait before timeout connection (default %d)" % defaults.timeout)
 
         request.add_option("--retries", dest="retries", type="int",
-                           help="Retries when the connection timeouts "
-                                "(default %d)" % defaults.retries)
+                           help="Retries when the connection timeouts (default %d)" % defaults.retries)
 
         request.add_option("--randomize", dest="rParam",
                            help="Randomly change value for given parameter(s)")
@@ -211,8 +200,7 @@ def cmdLineParser(argv=None):
         request.add_option("--safe-freq", dest="safeFreq", type="int",
                            help="Test requests between two visits to a given safe URL")
 
-        request.add_option("--skip-urlencode", dest="skipUrlEncode",
-                           action="store_true",
+        request.add_option("--skip-urlencode", dest="skipUrlEncode", action="store_true",
                            help="Skip URL encoding of payload data")
 
         request.add_option("--csrf-token", dest="csrfToken",
@@ -221,44 +209,36 @@ def cmdLineParser(argv=None):
         request.add_option("--csrf-url", dest="csrfUrl",
                            help="URL address to visit to extract anti-CSRF token")
 
-        request.add_option("--force-ssl", dest="forceSSL",
-                           action="store_true",
+        request.add_option("--force-ssl", dest="forceSSL", action="store_true",
                            help="Force usage of SSL/HTTPS")
 
-        request.add_option("--hpp", dest="hpp",
-                                  action="store_true",
-                                  help="Use HTTP parameter pollution method")
+        request.add_option("--hpp", dest="hpp", action="store_true",
+                           help="Use HTTP parameter pollution method")
 
         request.add_option("--eval", dest="evalCode",
                            help="Evaluate provided Python code before the request (e.g. \"import hashlib;id2=hashlib.md5(id).hexdigest()\")")
 
         # Optimization options
-        optimization = OptionGroup(parser, "Optimization", "These "
-                               "options can be used to optimize the "
-                               "performance of sqlmap")
+        optimization = OptionGroup(parser, "Optimization", "These options can be used to optimize the performance of sqlmap")
 
-        optimization.add_option("-o", dest="optimize",
-                                 action="store_true",
-                                 help="Turn on all optimization switches")
+        optimization.add_option("-o", dest="optimize", action="store_true",
+                                help="Turn on all optimization switches")
 
         optimization.add_option("--predict-output", dest="predictOutput", action="store_true",
-                          help="Predict common queries output")
+                                help="Predict common queries output")
 
         optimization.add_option("--keep-alive", dest="keepAlive", action="store_true",
-                           help="Use persistent HTTP(s) connections")
+                                help="Use persistent HTTP(s) connections")
 
         optimization.add_option("--null-connection", dest="nullConnection", action="store_true",
-                          help="Retrieve page length without actual HTTP response body")
+                                help="Retrieve page length without actual HTTP response body")
 
         optimization.add_option("--threads", dest="threads", type="int",
-                           help="Max number of concurrent HTTP(s) "
+                                help="Max number of concurrent HTTP(s) "
                                 "requests (default %d)" % defaults.threads)
 
         # Injection options
-        injection = OptionGroup(parser, "Injection", "These options can be "
-                                "used to specify which parameters to test "
-                                "for, provide custom injection payloads and "
-                                "optional tampering scripts")
+        injection = OptionGroup(parser, "Injection", "These options can be used to specify which parameters to test for, provide custom injection payloads and optional tampering scripts")
 
         injection.add_option("-p", dest="testParameter",
                              help="Testable parameter(s)")
@@ -270,36 +250,30 @@ def cmdLineParser(argv=None):
                              help="Skip testing parameters that not appear to be dynamic")
 
         injection.add_option("--param-exclude", dest="paramExclude",
-                           help="Regexp to exclude parameters from testing (e.g. \"ses\")")
+                             help="Regexp to exclude parameters from testing (e.g. \"ses\")")
 
         injection.add_option("--dbms", dest="dbms",
                              help="Force back-end DBMS to this value")
 
         injection.add_option("--dbms-cred", dest="dbmsCred",
-                            help="DBMS authentication credentials (user:password)")
+                             help="DBMS authentication credentials (user:password)")
 
         injection.add_option("--os", dest="os",
-                             help="Force back-end DBMS operating system "
-                                  "to this value")
+                             help="Force back-end DBMS operating system to this value")
 
-        injection.add_option("--invalid-bignum", dest="invalidBignum",
-                             action="store_true",
+        injection.add_option("--invalid-bignum", dest="invalidBignum", action="store_true",
                              help="Use big numbers for invalidating values")
 
-        injection.add_option("--invalid-logical", dest="invalidLogical",
-                             action="store_true",
+        injection.add_option("--invalid-logical", dest="invalidLogical", action="store_true",
                              help="Use logical operations for invalidating values")
 
-        injection.add_option("--invalid-string", dest="invalidString",
-                             action="store_true",
+        injection.add_option("--invalid-string", dest="invalidString", action="store_true",
                              help="Use random strings for invalidating values")
 
-        injection.add_option("--no-cast", dest="noCast",
-                             action="store_true",
+        injection.add_option("--no-cast", dest="noCast", action="store_true",
                              help="Turn off payload casting mechanism")
 
-        injection.add_option("--no-escape", dest="noEscape",
-                             action="store_true",
+        injection.add_option("--no-escape", dest="noEscape", action="store_true",
                              help="Turn off string escaping mechanism")
 
         injection.add_option("--prefix", dest="prefix",
@@ -312,54 +286,40 @@ def cmdLineParser(argv=None):
                              help="Use given script(s) for tampering injection data")
 
         # Detection options
-        detection = OptionGroup(parser, "Detection", "These options can be "
-                                "used to customize the detection phase")
+        detection = OptionGroup(parser, "Detection", "These options can be used to customize the detection phase")
 
         detection.add_option("--level", dest="level", type="int",
-                             help="Level of tests to perform (1-5, "
-                                  "default %d)" % defaults.level)
+                             help="Level of tests to perform (1-5, default %d)" % defaults.level)
 
         detection.add_option("--risk", dest="risk", type="int",
-                             help="Risk of tests to perform (1-3, "
-                                  "default %d)" % defaults.risk)
+                             help="Risk of tests to perform (1-3, default %d)" % defaults.risk)
 
         detection.add_option("--string", dest="string",
-                             help="String to match when "
-                                  "query is evaluated to True")
+                             help="String to match when query is evaluated to True")
 
         detection.add_option("--not-string", dest="notString",
-                             help="String to match when "
-                                  "query is evaluated to False")
+                             help="String to match when query is evaluated to False")
 
         detection.add_option("--regexp", dest="regexp",
-                             help="Regexp to match when "
-                                  "query is evaluated to True")
+                             help="Regexp to match when query is evaluated to True")
 
         detection.add_option("--code", dest="code", type="int",
-                             help="HTTP code to match when "
-                                  "query is evaluated to True")
+                             help="HTTP code to match when query is evaluated to True")
 
-        detection.add_option("--text-only", dest="textOnly",
-                             action="store_true",
+        detection.add_option("--text-only", dest="textOnly", action="store_true",
                              help="Compare pages based only on the textual content")
 
-        detection.add_option("--titles", dest="titles",
-                             action="store_true",
+        detection.add_option("--titles", dest="titles", action="store_true",
                              help="Compare pages based only on their titles")
 
         # Techniques options
-        techniques = OptionGroup(parser, "Techniques", "These options can be "
-                                 "used to tweak testing of specific SQL "
-                                 "injection techniques")
+        techniques = OptionGroup(parser, "Techniques", "These options can be used to tweak testing of specific SQL injection techniques")
 
         techniques.add_option("--technique", dest="tech",
-                              help="SQL injection techniques to use "
-                                   "(default \"%s\")" % defaults.tech)
+                              help="SQL injection techniques to use (default \"%s\")" % defaults.tech)
 
-        techniques.add_option("--time-sec", dest="timeSec",
-                              type="int",
-                              help="Seconds to delay the DBMS response "
-                                   "(default %d)" % defaults.timeSec)
+        techniques.add_option("--time-sec", dest="timeSec", type="int",
+                              help="Seconds to delay the DBMS response (default %d)" % defaults.timeSec)
 
         techniques.add_option("--union-cols", dest="uCols",
                               help="Range of columns to test for UNION query SQL injection")
@@ -374,58 +334,45 @@ def cmdLineParser(argv=None):
                               help="Domain name used for DNS exfiltration attack")
 
         techniques.add_option("--second-order", dest="secondOrder",
-                             help="Resulting page URL searched for second-order "
-                                  "response")
+                              help="Resulting page URL searched for second-order response")
 
         # Fingerprint options
         fingerprint = OptionGroup(parser, "Fingerprint")
 
-        fingerprint.add_option("-f", "--fingerprint", dest="extensiveFp",
-                               action="store_true",
+        fingerprint.add_option("-f", "--fingerprint", dest="extensiveFp", action="store_true",
                                help="Perform an extensive DBMS version fingerprint")
 
         # Enumeration options
-        enumeration = OptionGroup(parser, "Enumeration", "These options can "
-                                  "be used to enumerate the back-end database "
-                                  "management system information, structure "
-                                  "and data contained in the tables. Moreover "
-                                  "you can run your own SQL statements")
+        enumeration = OptionGroup(parser, "Enumeration", "These options can be used to enumerate the back-end database management system information, structure and data contained in the tables. Moreover you can run your own SQL statements")
 
-        enumeration.add_option("-a", "--all", dest="getAll",
-                               action="store_true", help="Retrieve everything")
+        enumeration.add_option("-a", "--all", dest="getAll", action="store_true",
+                               help="Retrieve everything")
 
-        enumeration.add_option("-b", "--banner", dest="getBanner",
-                               action="store_true", help="Retrieve DBMS banner")
+        enumeration.add_option("-b", "--banner", dest="getBanner", action="store_true",
+                               help="Retrieve DBMS banner")
 
-        enumeration.add_option("--current-user", dest="getCurrentUser",
-                               action="store_true",
+        enumeration.add_option("--current-user", dest="getCurrentUser", action="store_true",
                                help="Retrieve DBMS current user")
 
-        enumeration.add_option("--current-db", dest="getCurrentDb",
-                               action="store_true",
+        enumeration.add_option("--current-db", dest="getCurrentDb", action="store_true",
                                help="Retrieve DBMS current database")
 
-        enumeration.add_option("--hostname", dest="getHostname",
-                               action="store_true",
+        enumeration.add_option("--hostname", dest="getHostname", action="store_true",
                                help="Retrieve DBMS server hostname")
 
-        enumeration.add_option("--is-dba", dest="isDba",
-                               action="store_true",
+        enumeration.add_option("--is-dba", dest="isDba", action="store_true",
                                help="Detect if the DBMS current user is DBA")
 
         enumeration.add_option("--users", dest="getUsers", action="store_true",
                                help="Enumerate DBMS users")
 
-        enumeration.add_option("--passwords", dest="getPasswordHashes",
-                               action="store_true",
+        enumeration.add_option("--passwords", dest="getPasswordHashes", action="store_true",
                                help="Enumerate DBMS users password hashes")
 
-        enumeration.add_option("--privileges", dest="getPrivileges",
-                               action="store_true",
+        enumeration.add_option("--privileges", dest="getPrivileges", action="store_true",
                                help="Enumerate DBMS users privileges")
 
-        enumeration.add_option("--roles", dest="getRoles",
-                               action="store_true",
+        enumeration.add_option("--roles", dest="getRoles", action="store_true",
                                help="Enumerate DBMS users roles")
 
         enumeration.add_option("--dbs", dest="getDbs", action="store_true",
@@ -464,16 +411,14 @@ def cmdLineParser(argv=None):
         enumeration.add_option("-C", dest="col",
                                help="DBMS database table column(s) to enumerate")
 
-        enumeration.add_option("-X", dest="excludeCol",
-                               help="DBMS database table column(s) to not enumerate")
+        enumeration.add_option("-X", dest="exclude",
+                               help="DBMS database identifier(s) to not enumerate")
 
         enumeration.add_option("-U", dest="user",
                                help="DBMS user to enumerate")
 
-        enumeration.add_option("--exclude-sysdbs", dest="excludeSysDbs",
-                               action="store_true",
-                               help="Exclude DBMS system databases when "
-                                    "enumerating tables")
+        enumeration.add_option("--exclude-sysdbs", dest="excludeSysDbs", action="store_true",
+                               help="Exclude DBMS system databases when enumerating tables")
 
         enumeration.add_option("--pivot-column", dest="pivotColumn",
                                help="Pivot column name")
@@ -496,28 +441,23 @@ def cmdLineParser(argv=None):
         enumeration.add_option("--sql-query", dest="query",
                                help="SQL statement to be executed")
 
-        enumeration.add_option("--sql-shell", dest="sqlShell",
-                               action="store_true",
+        enumeration.add_option("--sql-shell", dest="sqlShell", action="store_true",
                                help="Prompt for an interactive SQL shell")
 
         enumeration.add_option("--sql-file", dest="sqlFile",
                                help="Execute SQL statements from given file(s)")
 
         # Brute force options
-        brute = OptionGroup(parser, "Brute force", "These "
-                          "options can be used to run brute force "
-                          "checks")
+        brute = OptionGroup(parser, "Brute force", "These options can be used to run brute force checks")
 
         brute.add_option("--common-tables", dest="commonTables", action="store_true",
-                               help="Check existence of common tables")
+                         help="Check existence of common tables")
 
         brute.add_option("--common-columns", dest="commonColumns", action="store_true",
-                               help="Check existence of common columns")
+                         help="Check existence of common columns")
 
         # User-defined function options
-        udf = OptionGroup(parser, "User-defined function injection", "These "
-                          "options can be used to create custom user-defined "
-                          "functions")
+        udf = OptionGroup(parser, "User-defined function injection", "These options can be used to create custom user-defined functions")
 
         udf.add_option("--udf-inject", dest="udfInject", action="store_true",
                        help="Inject custom user-defined functions")
@@ -526,167 +466,131 @@ def cmdLineParser(argv=None):
                        help="Local path of the shared library")
 
         # File system options
-        filesystem = OptionGroup(parser, "File system access", "These options "
-                                 "can be used to access the back-end database "
-                                 "management system underlying file system")
+        filesystem = OptionGroup(parser, "File system access", "These options can be used to access the back-end database management system underlying file system")
 
         filesystem.add_option("--file-read", dest="rFile",
-                              help="Read a file from the back-end DBMS "
-                                   "file system")
+                              help="Read a file from the back-end DBMS file system")
 
         filesystem.add_option("--file-write", dest="wFile",
-                              help="Write a local file on the back-end "
-                                   "DBMS file system")
+                              help="Write a local file on the back-end DBMS file system")
 
         filesystem.add_option("--file-dest", dest="dFile",
-                              help="Back-end DBMS absolute filepath to "
-                                   "write to")
+                              help="Back-end DBMS absolute filepath to write to")
 
         # Takeover options
-        takeover = OptionGroup(parser, "Operating system access", "These "
-                               "options can be used to access the back-end "
-                               "database management system underlying "
-                               "operating system")
+        takeover = OptionGroup(parser, "Operating system access", "These options can be used to access the back-end database management system underlying operating system")
 
         takeover.add_option("--os-cmd", dest="osCmd",
                             help="Execute an operating system command")
 
-        takeover.add_option("--os-shell", dest="osShell",
-                            action="store_true",
-                            help="Prompt for an interactive operating "
-                                 "system shell")
+        takeover.add_option("--os-shell", dest="osShell", action="store_true",
+                            help="Prompt for an interactive operating system shell")
 
-        takeover.add_option("--os-pwn", dest="osPwn",
-                            action="store_true",
-                            help="Prompt for an OOB shell, "
-                                 "Meterpreter or VNC")
+        takeover.add_option("--os-pwn", dest="osPwn", action="store_true",
+                            help="Prompt for an OOB shell, Meterpreter or VNC")
 
-        takeover.add_option("--os-smbrelay", dest="osSmb",
-                            action="store_true",
-                            help="One click prompt for an OOB shell, "
-                                 "Meterpreter or VNC")
+        takeover.add_option("--os-smbrelay", dest="osSmb", action="store_true",
+                            help="One click prompt for an OOB shell, Meterpreter or VNC")
 
-        takeover.add_option("--os-bof", dest="osBof",
-                            action="store_true",
+        takeover.add_option("--os-bof", dest="osBof", action="store_true",
                             help="Stored procedure buffer overflow "
                                  "exploitation")
 
-        takeover.add_option("--priv-esc", dest="privEsc",
-                            action="store_true",
+        takeover.add_option("--priv-esc", dest="privEsc", action="store_true",
                             help="Database process user privilege escalation")
 
         takeover.add_option("--msf-path", dest="msfPath",
-                            help="Local path where Metasploit Framework "
-                                 "is installed")
+                            help="Local path where Metasploit Framework is installed")
 
         takeover.add_option("--tmp-path", dest="tmpPath",
-                            help="Remote absolute path of temporary files "
-                                 "directory")
+                            help="Remote absolute path of temporary files directory")
 
         # Windows registry options
-        windows = OptionGroup(parser, "Windows registry access", "These "
-                               "options can be used to access the back-end "
-                               "database management system Windows "
-                               "registry")
+        windows = OptionGroup(parser, "Windows registry access", "These options can be used to access the back-end database management system Windows registry")
 
-        windows.add_option("--reg-read", dest="regRead",
-                            action="store_true",
-                            help="Read a Windows registry key value")
+        windows.add_option("--reg-read", dest="regRead", action="store_true",
+                           help="Read a Windows registry key value")
 
-        windows.add_option("--reg-add", dest="regAdd",
-                            action="store_true",
-                            help="Write a Windows registry key value data")
+        windows.add_option("--reg-add", dest="regAdd", action="store_true",
+                           help="Write a Windows registry key value data")
 
-        windows.add_option("--reg-del", dest="regDel",
-                            action="store_true",
-                            help="Delete a Windows registry key value")
+        windows.add_option("--reg-del", dest="regDel", action="store_true",
+                           help="Delete a Windows registry key value")
 
         windows.add_option("--reg-key", dest="regKey",
-                            help="Windows registry key")
+                           help="Windows registry key")
 
         windows.add_option("--reg-value", dest="regVal",
-                            help="Windows registry key value")
+                           help="Windows registry key value")
 
         windows.add_option("--reg-data", dest="regData",
-                            help="Windows registry key value data")
+                           help="Windows registry key value data")
 
         windows.add_option("--reg-type", dest="regType",
-                            help="Windows registry key value type")
+                           help="Windows registry key value type")
 
         # General options
-        general = OptionGroup(parser, "General", "These options can be used "
-                             "to set some general working parameters")
+        general = OptionGroup(parser, "General", "These options can be used to set some general working parameters")
 
         general.add_option("-s", dest="sessionFile",
-                            help="Load session from a stored (.sqlite) file")
+                           help="Load session from a stored (.sqlite) file")
 
         general.add_option("-t", dest="trafficFile",
-                            help="Log all HTTP traffic into a "
-                            "textual file")
+                           help="Log all HTTP traffic into a textual file")
 
-        general.add_option("--batch", dest="batch",
-                            action="store_true",
-                            help="Never ask for user input, use the default behaviour")
+        general.add_option("--batch", dest="batch", action="store_true",
+                           help="Never ask for user input, use the default behavior")
 
         general.add_option("--binary-fields", dest="binaryFields",
-                          help="Result fields having binary values (e.g. \"digest\")")
+                           help="Result fields having binary values (e.g. \"digest\")")
 
-        general.add_option("--check-internet", dest="checkInternet",
-                            action="store_true",
-                            help="Check Internet connection before assessing the target")
+        general.add_option("--check-internet", dest="checkInternet", action="store_true",
+                           help="Check Internet connection before assessing the target")
 
         general.add_option("--crawl", dest="crawlDepth", type="int",
-                            help="Crawl the website starting from the target URL")
+                           help="Crawl the website starting from the target URL")
 
         general.add_option("--crawl-exclude", dest="crawlExclude",
                            help="Regexp to exclude pages from crawling (e.g. \"logout\")")
 
         general.add_option("--csv-del", dest="csvDel",
-                                  help="Delimiting character used in CSV output "
-                                  "(default \"%s\")" % defaults.csvDel)
+                           help="Delimiting character used in CSV output (default \"%s\")" % defaults.csvDel)
 
         general.add_option("--charset", dest="charset",
                            help="Blind SQL injection charset (e.g. \"0123456789abcdef\")")
 
         general.add_option("--dump-format", dest="dumpFormat",
-                                  help="Format of dumped data (CSV (default), HTML or SQLITE)")
+                           help="Format of dumped data (CSV (default), HTML or SQLITE)")
 
         general.add_option("--encoding", dest="encoding",
-                            help="Character encoding used for data retrieval (e.g. GBK)")
+                           help="Character encoding used for data retrieval (e.g. GBK)")
 
-        general.add_option("--eta", dest="eta",
-                            action="store_true",
-                            help="Display for each output the estimated time of arrival")
+        general.add_option("--eta", dest="eta", action="store_true",
+                           help="Display for each output the estimated time of arrival")
 
-        general.add_option("--flush-session", dest="flushSession",
-                            action="store_true",
-                            help="Flush session files for current target")
+        general.add_option("--flush-session", dest="flushSession", action="store_true",
+                           help="Flush session files for current target")
 
-        general.add_option("--forms", dest="forms",
-                                  action="store_true",
-                                  help="Parse and test forms on target URL")
+        general.add_option("--forms", dest="forms", action="store_true",
+                           help="Parse and test forms on target URL")
 
-        general.add_option("--fresh-queries", dest="freshQueries",
-                            action="store_true",
-                            help="Ignore query results stored in session file")
+        general.add_option("--fresh-queries", dest="freshQueries", action="store_true",
+                           help="Ignore query results stored in session file")
 
         general.add_option("--har", dest="harFile",
                            help="Log all HTTP traffic into a HAR file")
 
-        general.add_option("--hex", dest="hexConvert",
-                            action="store_true",
-                            help="Use DBMS hex function(s) for data retrieval")
+        general.add_option("--hex", dest="hexConvert", action="store_true",
+                           help="Use DBMS hex function(s) for data retrieval")
 
-        general.add_option("--output-dir", dest="outputDir",
-                            action="store",
-                            help="Custom output directory path")
+        general.add_option("--output-dir", dest="outputDir", action="store",
+                           help="Custom output directory path")
 
-        general.add_option("--parse-errors", dest="parseErrors",
-                                  action="store_true",
-                                  help="Parse and display DBMS error messages from responses")
+        general.add_option("--parse-errors", dest="parseErrors", action="store_true",
+                           help="Parse and display DBMS error messages from responses")
 
         general.add_option("--save", dest="saveConfig",
-                            help="Save options to a configuration INI file")
+                           help="Save options to a configuration INI file")
 
         general.add_option("--scope", dest="scope",
                            help="Regexp to filter targets from provided proxy log")
@@ -697,77 +601,65 @@ def cmdLineParser(argv=None):
         general.add_option("--test-skip", dest="testSkip",
                            help="Skip tests by payloads and/or titles (e.g. BENCHMARK)")
 
-        general.add_option("--update", dest="updateAll",
-                            action="store_true",
-                            help="Update sqlmap")
+        general.add_option("--update", dest="updateAll", action="store_true",
+                           help="Update sqlmap")
 
         # Miscellaneous options
         miscellaneous = OptionGroup(parser, "Miscellaneous")
 
         miscellaneous.add_option("-z", dest="mnemonics",
-                               help="Use short mnemonics (e.g. \"flu,bat,ban,tec=EU\")")
+                                 help="Use short mnemonics (e.g. \"flu,bat,ban,tec=EU\")")
 
         miscellaneous.add_option("--alert", dest="alert",
-                                  help="Run host OS command(s) when SQL injection is found")
+                                 help="Run host OS command(s) when SQL injection is found")
 
         miscellaneous.add_option("--answers", dest="answers",
-                                  help="Set question answers (e.g. \"quit=N,follow=N\")")
+                                 help="Set question answers (e.g. \"quit=N,follow=N\")")
 
         miscellaneous.add_option("--beep", dest="beep", action="store_true",
-                                  help="Beep on question and/or when SQL injection is found")
+                                 help="Beep on question and/or when SQL injection is found")
 
-        miscellaneous.add_option("--cleanup", dest="cleanup",
-                                  action="store_true",
-                                  help="Clean up the DBMS from sqlmap specific "
-                                  "UDF and tables")
+        miscellaneous.add_option("--cleanup", dest="cleanup", action="store_true",
+                                 help="Clean up the DBMS from sqlmap specific UDF and tables")
 
-        miscellaneous.add_option("--dependencies", dest="dependencies",
-                                  action="store_true",
-                                  help="Check for missing (non-core) sqlmap dependencies")
+        miscellaneous.add_option("--dependencies", dest="dependencies", action="store_true",
+                                 help="Check for missing (non-core) sqlmap dependencies")
 
-        miscellaneous.add_option("--disable-coloring", dest="disableColoring",
-                                  action="store_true",
-                                  help="Disable console output coloring")
+        miscellaneous.add_option("--disable-coloring", dest="disableColoring", action="store_true",
+                                 help="Disable console output coloring")
 
         miscellaneous.add_option("--gpage", dest="googlePage", type="int",
-                                  help="Use Google dork results from specified page number")
+                                 help="Use Google dork results from specified page number")
 
-        miscellaneous.add_option("--identify-waf", dest="identifyWaf",
-                                  action="store_true",
-                                  help="Make a thorough testing for a WAF/IPS/IDS protection")
+        miscellaneous.add_option("--identify-waf", dest="identifyWaf", action="store_true",
+                                 help="Make a thorough testing for a WAF/IPS/IDS protection")
 
-        miscellaneous.add_option("--mobile", dest="mobile",
-                                  action="store_true",
-                                  help="Imitate smartphone through HTTP User-Agent header")
+        miscellaneous.add_option("--mobile", dest="mobile", action="store_true",
+                                 help="Imitate smartphone through HTTP User-Agent header")
 
-        miscellaneous.add_option("--offline", dest="offline",
-                                  action="store_true",
-                                  help="Work in offline mode (only use session data)")
+        miscellaneous.add_option("--offline", dest="offline", action="store_true",
+                                 help="Work in offline mode (only use session data)")
 
-        miscellaneous.add_option("--purge-output", dest="purgeOutput",
-                                  action="store_true",
-                                  help="Safely remove all content from output directory")
+        miscellaneous.add_option("--purge-output", dest="purgeOutput", action="store_true",
+                                 help="Safely remove all content from output directory")
 
-        miscellaneous.add_option("--skip-waf", dest="skipWaf",
-                                  action="store_true",
-                                  help="Skip heuristic detection of WAF/IPS/IDS protection")
+        miscellaneous.add_option("--skip-waf", dest="skipWaf", action="store_true",
+                                 help="Skip heuristic detection of WAF/IPS/IDS protection")
 
-        miscellaneous.add_option("--smart", dest="smart",
-                                  action="store_true",
-                                  help="Conduct thorough tests only if positive heuristic(s)")
+        miscellaneous.add_option("--smart", dest="smart", action="store_true",
+                                 help="Conduct thorough tests only if positive heuristic(s)")
 
         miscellaneous.add_option("--sqlmap-shell", dest="sqlmapShell", action="store_true",
-                                  help="Prompt for an interactive sqlmap shell")
+                                 help="Prompt for an interactive sqlmap shell")
 
         miscellaneous.add_option("--tmp-dir", dest="tmpDir",
-                                  help="Local directory for storing temporary files")
+                                 help="Local directory for storing temporary files")
 
         miscellaneous.add_option("--web-root", dest="webRoot",
-                                  help="Web server document root directory (e.g. \"/var/www\")")
+                                 help="Web server document root directory (e.g. \"/var/www\")")
 
-        miscellaneous.add_option("--wizard", dest="wizard",
-                                  action="store_true",
-                                  help="Simple wizard interface for beginner users")
+        miscellaneous.add_option("--wizard", dest="wizard", action="store_true",
+                                 help="Simple wizard interface for beginner users")
 
         # Hidden and/or experimental options
         parser.add_option("--dummy", dest="dummy", action="store_true",
@@ -909,7 +801,7 @@ def cmdLineParser(argv=None):
                 for arg in shlex.split(command):
                     argv.append(getUnicode(arg, encoding=sys.stdin.encoding))
             except ValueError, ex:
-                raise SqlmapSyntaxException, "something went wrong during command line parsing ('%s')" % ex.message
+                raise SqlmapSyntaxException("something went wrong during command line parsing ('%s')" % ex.message)
 
         for i in xrange(len(argv)):
             if argv[i] == "-hh":
@@ -976,9 +868,7 @@ def cmdLineParser(argv=None):
         if args.dummy:
             args.url = args.url or DUMMY_URL
 
-        if not any((args.direct, args.url, args.logFile, args.bulkFile, args.googleDork, args.configFile, \
-            args.requestFile, args.updateAll, args.smokeTest, args.liveTest, args.wizard, args.dependencies, \
-            args.purgeOutput, args.sitemapUrl)):
+        if not any((args.direct, args.url, args.logFile, args.bulkFile, args.googleDork, args.configFile, args.requestFile, args.updateAll, args.smokeTest, args.liveTest, args.wizard, args.dependencies, args.purgeOutput, args.sitemapUrl)):
             errMsg = "missing a mandatory option (-d, -u, -l, -m, -r, -g, -c, -x, --wizard, --update, --purge-output or --dependencies), "
             errMsg += "use -h for basic or -hh for advanced help\n"
             parser.error(errMsg)

lib/parse/configfile.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 from lib.core.common import checkFile

lib/parse/handler.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import re
@@ -44,7 +44,7 @@ class FingerprintHandler(ContentHandler):
     def startElement(self, name, attrs):
         if name == "regexp":
             self._regexp = sanitizeStr(attrs.get("value"))
-            _ = re.match("\A[A-Za-z0-9]+", self._regexp)  # minor trick avoiding compiling of large amount of regexes
+            _ = re.match(r"\A[A-Za-z0-9]+", self._regexp)  # minor trick avoiding compiling of large amount of regexes
 
             if _ and _.group(0).lower() in self._banner.lower() or not _:
                 self._match = re.search(self._regexp, self._banner, re.I | re.M)

lib/parse/headers.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import itertools
@@ -24,18 +24,16 @@ def headersParser(headers):
     if not kb.headerPaths:
         kb.headerPaths = {
             "microsoftsharepointteamservices": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "sharepoint.xml"),
-            "server":                          os.path.join(paths.SQLMAP_XML_BANNER_PATH, "server.xml"),
-            "servlet-engine":                  os.path.join(paths.SQLMAP_XML_BANNER_PATH, "servlet-engine.xml"),
-            "set-cookie":                      os.path.join(paths.SQLMAP_XML_BANNER_PATH, "set-cookie.xml"),
-            "x-aspnet-version":                os.path.join(paths.SQLMAP_XML_BANNER_PATH, "x-aspnet-version.xml"),
-            "x-powered-by":                    os.path.join(paths.SQLMAP_XML_BANNER_PATH, "x-powered-by.xml"),
+            "server": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "server.xml"),
+            "servlet-engine": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "servlet-engine.xml"),
+            "set-cookie": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "set-cookie.xml"),
+            "x-aspnet-version": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "x-aspnet-version.xml"),
+            "x-powered-by": os.path.join(paths.SQLMAP_XML_BANNER_PATH, "x-powered-by.xml"),
         }
 
     for header in itertools.ifilter(lambda x: x in kb.headerPaths, headers):
         value = headers[header]
         xmlfile = kb.headerPaths[header]
-
         handler = FingerprintHandler(value, kb.headersFp)
-
         parseXmlFile(xmlfile, handler)
         parseXmlFile(paths.GENERIC_XML, handler)

lib/parse/html.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import re
@@ -43,7 +43,7 @@ class HTMLHandler(ContentHandler):
         elif name == "error":
             regexp = attrs.get("regexp")
             if regexp not in kb.cache.regex:
-                keywords = re.findall("\w+", re.sub(r"\\.", " ", regexp))
+                keywords = re.findall(r"\w+", re.sub(r"\\.", " ", regexp))
                 keywords = sorted(keywords, key=len)
                 kb.cache.regex[regexp] = keywords[-1].lower()
 

lib/parse/payloads.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import os
@@ -78,7 +78,7 @@ def loadBoundaries():
         errMsg = "something appears to be wrong with "
         errMsg += "the file '%s' ('%s'). Please make " % (paths.BOUNDARIES_XML, getSafeExString(ex))
         errMsg += "sure that you haven't made any changes to it"
-        raise SqlmapInstallationException, errMsg
+        raise SqlmapInstallationException(errMsg)
 
     root = doc.getroot()
     parseXmlNode(root)
@@ -93,7 +93,7 @@ def loadPayloads():
             errMsg = "something appears to be wrong with "
             errMsg += "the file '%s' ('%s'). Please make " % (payloadFilePath, getSafeExString(ex))
             errMsg += "sure that you haven't made any changes to it"
-            raise SqlmapInstallationException, errMsg
+            raise SqlmapInstallationException(errMsg)
 
         root = doc.getroot()
         parseXmlNode(root)

lib/parse/sitemap.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import httplib
@@ -32,7 +32,7 @@ def parseSitemap(url, retVal=None):
             content = Request.getPage(url=url, raise404=True)[0] if not abortedFlag else ""
         except httplib.InvalidURL:
             errMsg = "invalid URL given for sitemap ('%s')" % url
-            raise SqlmapSyntaxException, errMsg
+            raise SqlmapSyntaxException(errMsg)
 
         for match in re.finditer(r"<loc>\s*([^<]+)", content or ""):
             if abortedFlag:

lib/request/__init__.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 pass

lib/request/basic.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import codecs
@@ -18,11 +18,13 @@ from lib.core.common import extractErrorMessage
 from lib.core.common import extractRegexResult
 from lib.core.common import getPublicTypeMembers
 from lib.core.common import getUnicode
+from lib.core.common import isListLike
 from lib.core.common import randomStr
 from lib.core.common import readInput
 from lib.core.common import resetCookieJar
 from lib.core.common import singleTimeLogMessage
 from lib.core.common import singleTimeWarnMessage
+from lib.core.common import unArrayizeValue
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
@@ -33,6 +35,7 @@ from lib.core.enums import PLACE
 from lib.core.exception import SqlmapCompressionException
 from lib.core.settings import BLOCKED_IP_REGEX
 from lib.core.settings import DEFAULT_COOKIE_DELIMITER
+from lib.core.settings import DEV_EMAIL_ADDRESS
 from lib.core.settings import EVENTVALIDATION_REGEX
 from lib.core.settings import MAX_CONNECTION_TOTAL_SIZE
 from lib.core.settings import META_CHARSET_REGEX
@@ -58,7 +61,7 @@ def forgeHeaders(items=None, base=None):
         if items[_] is None:
             del items[_]
 
-    headers = OrderedDict(base or conf.httpHeaders)
+    headers = OrderedDict(conf.httpHeaders if base is None else base)
     headers.update(items.items())
 
     class _str(str):
@@ -92,7 +95,7 @@ def forgeHeaders(items=None, base=None):
     if conf.cj:
         if HTTP_HEADER.COOKIE in headers:
             for cookie in conf.cj:
-                if cookie.domain_specified and not conf.hostname.endswith(cookie.domain):
+                if cookie.domain_specified and not (conf.hostname or "").endswith(cookie.domain):
                     continue
 
                 if ("%s=" % getUnicode(cookie.name)) in getUnicode(headers[HTTP_HEADER.COOKIE]):
@@ -107,7 +110,9 @@ def forgeHeaders(items=None, base=None):
                         kb.mergeCookies = readInput(message, default='Y', boolean=True)
 
                     if kb.mergeCookies and kb.injection.place != PLACE.COOKIE:
-                        _ = lambda x: re.sub(r"(?i)\b%s=[^%s]+" % (re.escape(getUnicode(cookie.name)), conf.cookieDel or DEFAULT_COOKIE_DELIMITER), ("%s=%s" % (getUnicode(cookie.name), getUnicode(cookie.value))).replace('\\', r'\\'), x)
+                        def _(value):
+                            return re.sub(r"(?i)\b%s=[^%s]+" % (re.escape(getUnicode(cookie.name)), conf.cookieDel or DEFAULT_COOKIE_DELIMITER), ("%s=%s" % (getUnicode(cookie.name), getUnicode(cookie.value))).replace('\\', r'\\'), value)
+
                         headers[HTTP_HEADER.COOKIE] = _(headers[HTTP_HEADER.COOKIE])
 
                         if PLACE.COOKIE in conf.parameters:
@@ -149,13 +154,16 @@ def checkCharEncoding(encoding, warn=True):
     'utf8'
     """
 
+    if isListLike(encoding):
+        encoding = unArrayizeValue(encoding)
+
     if encoding:
         encoding = encoding.lower()
     else:
         return encoding
 
     # Reference: http://www.destructor.de/charsets/index.htm
-    translate = {"windows-874": "iso-8859-11", "utf-8859-1": "utf8", "en_us": "utf8", "macintosh": "iso-8859-1", "euc_tw": "big5_tw", "th": "tis-620", "unicode": "utf8",  "utc8": "utf8", "ebcdic": "ebcdic-cp-be", "iso-8859": "iso8859-1", "iso-8859-0": "iso8859-1", "ansi": "ascii", "gbk2312": "gbk", "windows-31j": "cp932", "en": "us"}
+    translate = {"windows-874": "iso-8859-11", "utf-8859-1": "utf8", "en_us": "utf8", "macintosh": "iso-8859-1", "euc_tw": "big5_tw", "th": "tis-620", "unicode": "utf8", "utc8": "utf8", "ebcdic": "ebcdic-cp-be", "iso-8859": "iso8859-1", "iso-8859-0": "iso8859-1", "ansi": "ascii", "gbk2312": "gbk", "windows-31j": "cp932", "en": "us"}
 
     for delimiter in (';', ',', '('):
         if delimiter in encoding:
@@ -212,9 +220,9 @@ def checkCharEncoding(encoding, warn=True):
     try:
         codecs.lookup(encoding.encode(UNICODE_ENCODING) if isinstance(encoding, unicode) else encoding)
     except (LookupError, ValueError):
-        if warn:
+        if warn and ' ' not in encoding:
             warnMsg = "unknown web page charset '%s'. " % encoding
-            warnMsg += "Please report by e-mail to 'dev@sqlmap.org'"
+            warnMsg += "Please report by e-mail to '%s'" % DEV_EMAIL_ADDRESS
             singleTimeLogMessage(warnMsg, logging.WARN, encoding)
         encoding = None
 
@@ -253,12 +261,22 @@ def decodePage(page, contentEncoding, contentType):
     if not page or (conf.nullConnection and len(page) < 2):
         return getUnicode(page)
 
-    if isinstance(contentEncoding, basestring) and contentEncoding.lower() in ("gzip", "x-gzip", "deflate"):
+    if isinstance(contentEncoding, basestring) and contentEncoding:
+        contentEncoding = contentEncoding.lower()
+    else:
+        contentEncoding = ""
+
+    if isinstance(contentType, basestring) and contentType:
+        contentType = contentType.lower()
+    else:
+        contentType = ""
+
+    if contentEncoding in ("gzip", "x-gzip", "deflate"):
         if not kb.pageCompress:
             return None
 
         try:
-            if contentEncoding.lower() == "deflate":
+            if contentEncoding == "deflate":
                 data = StringIO.StringIO(zlib.decompress(page, -15))  # Reference: http://stackoverflow.com/questions/1089662/python-inflate-and-deflate-implementations
             else:
                 data = gzip.GzipFile("", "rb", 9, StringIO.StringIO(page))
@@ -283,13 +301,12 @@ def decodePage(page, contentEncoding, contentType):
         httpCharset, metaCharset = None, None
 
         # Reference: http://stackoverflow.com/questions/1020892/python-urllib2-read-to-unicode
-        if contentType and (contentType.find("charset=") != -1):
+        if contentType.find("charset=") != -1:
             httpCharset = checkCharEncoding(contentType.split("charset=")[-1])
 
         metaCharset = checkCharEncoding(extractRegexResult(META_CHARSET_REGEX, page))
 
-        if (any((httpCharset, metaCharset)) and not all((httpCharset, metaCharset)))\
-            or (httpCharset == metaCharset and all((httpCharset, metaCharset))):
+        if (any((httpCharset, metaCharset)) and not all((httpCharset, metaCharset))) or (httpCharset == metaCharset and all((httpCharset, metaCharset))):
             kb.pageEncoding = httpCharset or metaCharset  # Reference: http://bytes.com/topic/html-css/answers/154758-http-equiv-vs-true-header-has-precedence
             debugMsg = "declared web page charset '%s'" % kb.pageEncoding
             singleTimeLogMessage(debugMsg, logging.DEBUG, debugMsg)
@@ -299,7 +316,7 @@ def decodePage(page, contentEncoding, contentType):
         kb.pageEncoding = conf.encoding
 
     # can't do for all responses because we need to support binary files too
-    if contentType and not isinstance(page, unicode) and "text/" in contentType.lower():
+    if not isinstance(page, unicode) and "text/" in contentType:
         if kb.heuristicMode:
             kb.pageEncoding = kb.pageEncoding or checkCharEncoding(getHeuristicCharEncoding(page))
             page = getUnicode(page, kb.pageEncoding)
@@ -373,7 +390,7 @@ def processResponse(page, responseHeaders, status=None):
                             continue
 
                         conf.paramDict[PLACE.POST][name] = value
-                conf.parameters[PLACE.POST] = re.sub("(?i)(%s=)[^&]+" % re.escape(name), r"\g<1>%s" % re.escape(value), conf.parameters[PLACE.POST])
+                conf.parameters[PLACE.POST] = re.sub(r"(?i)(%s=)[^&]+" % re.escape(name), r"\g<1>%s" % re.escape(value), conf.parameters[PLACE.POST])
 
     if not kb.browserVerification and re.search(r"(?i)browser.?verification", page or ""):
         kb.browserVerification = True

lib/request/basicauthhandler.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import urllib2
@@ -30,10 +30,8 @@ class SmartHTTPBasicAuthHandler(urllib2.HTTPBasicAuthHandler):
             self.retried_count = 0
         else:
             if self.retried_count > 5:
-                raise urllib2.HTTPError(req.get_full_url(), 401, "basic auth failed",
-                                headers, None)
+                raise urllib2.HTTPError(req.get_full_url(), 401, "basic auth failed", headers, None)
             else:
                 self.retried_count += 1
 
-        return urllib2.HTTPBasicAuthHandler.http_error_auth_reqed(
-                        self, auth_header, host, req, headers)
+        return urllib2.HTTPBasicAuthHandler.http_error_auth_reqed(self, auth_header, host, req, headers)

lib/request/comparison.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import re
@@ -49,7 +49,7 @@ def _comparison(page, headers, code, getRatioValue, pageLength):
     threadData = getCurrentThreadData()
 
     if kb.testMode:
-        threadData.lastComparisonHeaders = listToStrValue([_ for _ in headers.headers if not _.startswith("%s:" % URI_HTTP_HEADER)]) if headers else ""
+        threadData.lastComparisonHeaders = listToStrValue(_ for _ in headers.headers if not _.startswith("%s:" % URI_HTTP_HEADER)) if headers else ""
         threadData.lastComparisonPage = page
         threadData.lastComparisonCode = code
 
@@ -57,7 +57,7 @@ def _comparison(page, headers, code, getRatioValue, pageLength):
         return None
 
     if any((conf.string, conf.notString, conf.regexp)):
-        rawResponse = "%s%s" % (listToStrValue([_ for _ in headers.headers if not _.startswith("%s:" % URI_HTTP_HEADER)]) if headers else "", page)
+        rawResponse = "%s%s" % (listToStrValue(_ for _ in headers.headers if not _.startswith("%s:" % URI_HTTP_HEADER)) if headers else "", page)
 
         # String to match in page when the query is True and/or valid
         if conf.string:
@@ -106,16 +106,21 @@ def _comparison(page, headers, code, getRatioValue, pageLength):
         # Preventing "Unicode equal comparison failed to convert both arguments to Unicode"
         # (e.g. if one page is PDF and the other is HTML)
         if isinstance(seqMatcher.a, str) and isinstance(page, unicode):
-            page = page.encode(kb.pageEncoding or DEFAULT_PAGE_ENCODING, 'ignore')
+            page = page.encode(kb.pageEncoding or DEFAULT_PAGE_ENCODING, "ignore")
         elif isinstance(seqMatcher.a, unicode) and isinstance(page, str):
-            seqMatcher.a = seqMatcher.a.encode(kb.pageEncoding or DEFAULT_PAGE_ENCODING, 'ignore')
+            seqMatcher.a = seqMatcher.a.encode(kb.pageEncoding or DEFAULT_PAGE_ENCODING, "ignore")
 
-        if seqMatcher.a and page and seqMatcher.a == page:
-            ratio = 1
+        if any(_ is None for _ in (page, seqMatcher.a)):
+            return None
+        elif seqMatcher.a and page and seqMatcher.a == page:
+            ratio = 1.
         elif kb.skipSeqMatcher or seqMatcher.a and page and any(len(_) > MAX_DIFFLIB_SEQUENCE_LENGTH for _ in (seqMatcher.a, page)):
-            ratio = 1.0 * len(seqMatcher.a) / len(page)
-            if ratio > 1:
-                ratio = 1. / ratio
+            if not page or not seqMatcher.a:
+                return float(seqMatcher.a == page)
+            else:
+                ratio = 1. * len(seqMatcher.a) / len(page)
+                if ratio > 1:
+                    ratio = 1. / ratio
         else:
             seq1, seq2 = None, None
 

lib/request/connect.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import binascii
@@ -51,11 +51,13 @@ from lib.core.common import randomInt
 from lib.core.common import randomStr
 from lib.core.common import readInput
 from lib.core.common import removeReflectiveValues
+from lib.core.common import safeVariableNaming
 from lib.core.common import singleTimeLogMessage
 from lib.core.common import singleTimeWarnMessage
 from lib.core.common import stdev
 from lib.core.common import wasLastResponseDelayed
 from lib.core.common import unicodeencode
+from lib.core.common import unsafeVariableNaming
 from lib.core.common import urldecode
 from lib.core.common import urlencode
 from lib.core.data import conf
@@ -185,13 +187,13 @@ class Connect(object):
 
         if not kb.dnsMode and conn:
             headers = conn.info()
-            if headers and hasattr(headers, "getheader") and (headers.getheader(HTTP_HEADER.CONTENT_ENCODING, "").lower() in ("gzip", "deflate")\
-              or "text" not in headers.getheader(HTTP_HEADER.CONTENT_TYPE, "").lower()):
+            if kb.pageCompress and headers and hasattr(headers, "getheader") and (headers.getheader(HTTP_HEADER.CONTENT_ENCODING, "").lower() in ("gzip", "deflate") or "text" not in headers.getheader(HTTP_HEADER.CONTENT_TYPE, "").lower()):
                 retVal = conn.read(MAX_CONNECTION_TOTAL_SIZE)
                 if len(retVal) == MAX_CONNECTION_TOTAL_SIZE:
                     warnMsg = "large compressed response detected. Disabling compression"
                     singleTimeWarnMessage(warnMsg)
                     kb.pageCompress = False
+                    raise SqlmapCompressionException
             else:
                 while True:
                     if not conn:
@@ -239,27 +241,27 @@ class Connect(object):
             kb.requestCounter += 1
             threadData.lastRequestUID = kb.requestCounter
 
-        url = kwargs.get("url",                     None) or conf.url
-        get = kwargs.get("get",                     None)
-        post = kwargs.get("post",                   None)
-        method = kwargs.get("method",               None)
-        cookie = kwargs.get("cookie",               None)
-        ua = kwargs.get("ua",                       None) or conf.agent
-        referer = kwargs.get("referer",             None) or conf.referer
-        host = kwargs.get("host",                   None) or conf.host
-        direct_ = kwargs.get("direct",              False)
-        multipart = kwargs.get("multipart",         None)
-        silent = kwargs.get("silent",               False)
-        raise404 = kwargs.get("raise404",           True)
-        timeout = kwargs.get("timeout",             None) or conf.timeout
-        auxHeaders = kwargs.get("auxHeaders",       None)
-        response = kwargs.get("response",           False)
+        url = kwargs.get("url", None) or conf.url
+        get = kwargs.get("get", None)
+        post = kwargs.get("post", None)
+        method = kwargs.get("method", None)
+        cookie = kwargs.get("cookie", None)
+        ua = kwargs.get("ua", None) or conf.agent
+        referer = kwargs.get("referer", None) or conf.referer
+        host = kwargs.get("host", None) or conf.host
+        direct_ = kwargs.get("direct", False)
+        multipart = kwargs.get("multipart", None)
+        silent = kwargs.get("silent", False)
+        raise404 = kwargs.get("raise404", True)
+        timeout = kwargs.get("timeout", None) or conf.timeout
+        auxHeaders = kwargs.get("auxHeaders", None)
+        response = kwargs.get("response", False)
         ignoreTimeout = kwargs.get("ignoreTimeout", False) or kb.ignoreTimeout or conf.ignoreTimeouts
-        refreshing = kwargs.get("refreshing",       False)
-        retrying = kwargs.get("retrying",           False)
-        crawling = kwargs.get("crawling",           False)
-        checking = kwargs.get("checking",           False)
-        skipRead = kwargs.get("skipRead",           False)
+        refreshing = kwargs.get("refreshing", False)
+        retrying = kwargs.get("retrying", False)
+        crawling = kwargs.get("crawling", False)
+        checking = kwargs.get("checking", False)
+        skipRead = kwargs.get("skipRead", False)
 
         if multipart:
             post = multipart
@@ -317,8 +319,8 @@ class Connect(object):
 
             elif target:
                 if conf.forceSSL and urlparse.urlparse(url).scheme != "https":
-                    url = re.sub("(?i)\Ahttp:", "https:", url)
-                    url = re.sub("(?i):80/", ":443/", url)
+                    url = re.sub(r"(?i)\Ahttp:", "https:", url)
+                    url = re.sub(r"(?i):80/", ":443/", url)
 
                 if PLACE.GET in conf.parameters and not get:
                     get = conf.parameters[PLACE.GET]
@@ -344,7 +346,7 @@ class Connect(object):
             requestMsg += " %s" % httplib.HTTPConnection._http_vsn_str
 
             # Prepare HTTP headers
-            headers = forgeHeaders({HTTP_HEADER.COOKIE: cookie, HTTP_HEADER.USER_AGENT: ua, HTTP_HEADER.REFERER: referer, HTTP_HEADER.HOST: host})
+            headers = forgeHeaders({HTTP_HEADER.COOKIE: cookie, HTTP_HEADER.USER_AGENT: ua, HTTP_HEADER.REFERER: referer, HTTP_HEADER.HOST: host}, base=None if target else {})
 
             if HTTP_HEADER.COOKIE in headers:
                 cookie = headers[HTTP_HEADER.COOKIE]
@@ -477,8 +479,7 @@ class Connect(object):
 
                 # Get HTTP response
                 if hasattr(conn, "redurl"):
-                    page = (threadData.lastRedirectMsg[1] if kb.redirectChoice == REDIRECTION.NO\
-                    else Connect._connReadProxy(conn)) if not skipRead else None
+                    page = (threadData.lastRedirectMsg[1] if kb.redirectChoice == REDIRECTION.NO else Connect._connReadProxy(conn)) if not skipRead else None
                     skipLogTraffic = kb.redirectChoice == REDIRECTION.NO
                     code = conn.redcode
                 else:
@@ -679,9 +680,12 @@ class Connect(object):
                 warnMsg = "there was an incomplete read error while retrieving data "
                 warnMsg += "from the target URL"
             elif "Handshake status" in tbMsg:
-                status = re.search("Handshake status ([\d]{3})", tbMsg)
+                status = re.search(r"Handshake status ([\d]{3})", tbMsg)
                 errMsg = "websocket handshake status %s" % status.group(1) if status else "unknown"
                 raise SqlmapConnectionException(errMsg)
+            elif "SqlmapCompressionException" in tbMsg:
+                warnMsg = "problems with response (de)compression"
+                retrying = True
             else:
                 warnMsg = "unable to connect to the target URL"
 
@@ -717,7 +721,7 @@ class Connect(object):
                 else:
                     logger.debug(warnMsg)
                 return Connect._retryProxy(**kwargs)
-            elif kb.testMode:
+            elif kb.testMode or kb.multiThreadMode:
                 logger.critical(warnMsg)
                 return None, None, None
             else:
@@ -736,12 +740,12 @@ class Connect(object):
         if conn and getattr(conn, "redurl", None):
             _ = urlparse.urlsplit(conn.redurl)
             _ = ("%s%s" % (_.path or "/", ("?%s" % _.query) if _.query else ""))
-            requestMsg = re.sub("(\n[A-Z]+ ).+?( HTTP/\d)", "\g<1>%s\g<2>" % getUnicode(_).replace("\\", "\\\\"), requestMsg, 1)
+            requestMsg = re.sub(r"(\n[A-Z]+ ).+?( HTTP/\d)", "\g<1>%s\g<2>" % getUnicode(_).replace("\\", "\\\\"), requestMsg, 1)
 
             if kb.resendPostOnRedirect is False:
-                requestMsg = re.sub("(\[#\d+\]:\n)POST ", "\g<1>GET ", requestMsg)
-                requestMsg = re.sub("(?i)Content-length: \d+\n", "", requestMsg)
-                requestMsg = re.sub("(?s)\n\n.+", "\n", requestMsg)
+                requestMsg = re.sub(r"(\[#\d+\]:\n)POST ", "\g<1>GET ", requestMsg)
+                requestMsg = re.sub(r"(?i)Content-length: \d+\n", "", requestMsg)
+                requestMsg = re.sub(r"(?s)\n\n.+", "\n", requestMsg)
 
             responseMsg += "[#%d] (%d %s):\r\n" % (threadData.lastRequestUID, conn.code, status)
         else:
@@ -794,6 +798,8 @@ class Connect(object):
         raise404 = place != PLACE.URI if raise404 is None else raise404
         method = method or conf.method
 
+        postUrlEncode = kb.postUrlEncode
+
         value = agent.adjustLateValues(value)
         payload = agent.extractPayload(value)
         threadData = getCurrentThreadData()
@@ -802,8 +808,8 @@ class Connect(object):
             headers = OrderedDict(conf.httpHeaders)
             contentType = max(headers[_] if _.upper() == HTTP_HEADER.CONTENT_TYPE.upper() else None for _ in headers.keys())
 
-            if (kb.postHint or conf.skipUrlEncode) and kb.postUrlEncode:
-                kb.postUrlEncode = False
+            if (kb.postHint or conf.skipUrlEncode) and postUrlEncode:
+                postUrlEncode = False
                 conf.httpHeaders = [_ for _ in conf.httpHeaders if _[1] != contentType]
                 contentType = POST_HINT_CONTENT_TYPES.get(kb.postHint, PLAIN_TEXT_CONTENT_TYPE)
                 conf.httpHeaders.append((HTTP_HEADER.CONTENT_TYPE, contentType))
@@ -847,7 +853,7 @@ class Connect(object):
                 value = agent.replacePayload(value, payload)
             else:
                 # GET, POST, URI and Cookie payload needs to be thoroughly URL encoded
-                if (place in (PLACE.GET, PLACE.URI, PLACE.COOKIE) or place == PLACE.CUSTOM_HEADER and value.split(',')[0] == HTTP_HEADER.COOKIE) and not conf.skipUrlEncode or place in (PLACE.POST, PLACE.CUSTOM_POST) and kb.postUrlEncode:
+                if (place in (PLACE.GET, PLACE.URI, PLACE.COOKIE) or place == PLACE.CUSTOM_HEADER and value.split(',')[0] == HTTP_HEADER.COOKIE) and not conf.skipUrlEncode or place in (PLACE.POST, PLACE.CUSTOM_POST) and postUrlEncode:
                     skip = False
 
                     if place == PLACE.COOKIE or place == PLACE.CUSTOM_HEADER and value.split(',')[0] == HTTP_HEADER.COOKIE:
@@ -858,8 +864,11 @@ class Connect(object):
                             skip = True
 
                     if not skip:
-                        payload = urlencode(payload, '%', False, place != PLACE.URI)  # spaceplus is handled down below
+                        if place in (PLACE.POST, PLACE.CUSTOM_POST):  # potential problems in other cases (e.g. URL encoding of whole URI - including path)
+                            value = urlencode(value, spaceplus=kb.postSpaceToPlus)
+                        payload = urlencode(payload, safe='%', spaceplus=kb.postSpaceToPlus)
                         value = agent.replacePayload(value, payload)
+                        postUrlEncode = False
 
             if conf.hpp:
                 if not any(conf.url.lower().endswith(_.lower()) for _ in (WEB_API.ASP, WEB_API.ASPX)):
@@ -868,7 +877,7 @@ class Connect(object):
                     singleTimeWarnMessage(warnMsg)
                 if place in (PLACE.GET, PLACE.POST):
                     _ = re.escape(PAYLOAD_DELIMITER)
-                    match = re.search("(?P<name>\w+)=%s(?P<value>.+?)%s" % (_, _), value)
+                    match = re.search(r"(?P<name>\w+)=%s(?P<value>.+?)%s" % (_, _), value)
                     if match:
                         payload = match.group("value")
 
@@ -927,18 +936,18 @@ class Connect(object):
 
         if value and place == PLACE.CUSTOM_HEADER:
             if value.split(',')[0].capitalize() == PLACE.COOKIE:
-                cookie = value.split(',', 1)[1]
+                cookie = value.split(',', 1)[-1]
             else:
-                auxHeaders[value.split(',')[0]] = value.split(',', 1)[1]
+                auxHeaders[value.split(',')[0]] = value.split(',', 1)[-1]
 
         if conf.csrfToken:
             def _adjustParameter(paramString, parameter, newValue):
                 retVal = paramString
-                match = re.search("%s=[^&]*" % re.escape(parameter), paramString)
+                match = re.search(r"%s=[^&]*" % re.escape(parameter), paramString)
                 if match:
                     retVal = re.sub(re.escape(match.group(0)), "%s=%s" % (parameter, newValue), paramString)
                 else:
-                    match = re.search("(%s[\"']:[\"'])([^\"']+)" % re.escape(parameter), paramString)
+                    match = re.search(r"(%s[\"']:[\"'])([^\"']+)" % re.escape(parameter), paramString)
                     if match:
                         retVal = re.sub(re.escape(match.group(0)), "%s%s" % (match.group(1), newValue), paramString)
                 return retVal
@@ -976,7 +985,7 @@ class Connect(object):
                     if not conf.csrfUrl:
                         errMsg += ". You can try to rerun by providing "
                         errMsg += "a valid value for option '--csrf-url'"
-                    raise SqlmapTokenException, errMsg
+                    raise SqlmapTokenException(errMsg)
 
             if token:
                 token = token.strip("'\"")
@@ -1028,18 +1037,24 @@ class Connect(object):
                 for part in item.split(delimiter):
                     if '=' in part:
                         name, value = part.split('=', 1)
-                        name = re.sub(r"[^\w]", "", name.strip())
-                        if name in keywords:
+                        name = name.strip()
+                        if safeVariableNaming(name) != name:
+                            conf.evalCode = re.sub(r"\b%s\b" % re.escape(name), safeVariableNaming(name), conf.evalCode)
+                            name = safeVariableNaming(name)
+                        elif name in keywords:
                             name = "%s%s" % (name, EVALCODE_KEYWORD_SUFFIX)
-                        value = urldecode(value, convall=True, plusspace=(item==post and kb.postSpaceToPlus))
+                        value = urldecode(value, convall=True, spaceplus=(item == post and kb.postSpaceToPlus))
                         variables[name] = value
 
             if cookie:
                 for part in cookie.split(conf.cookieDel or DEFAULT_COOKIE_DELIMITER):
                     if '=' in part:
                         name, value = part.split('=', 1)
-                        name = re.sub(r"[^\w]", "", name.strip())
-                        if name in keywords:
+                        name = name.strip()
+                        if safeVariableNaming(name) != name:
+                            conf.evalCode = re.sub(r"\b%s\b" % re.escape(name), safeVariableNaming(name), conf.evalCode)
+                            name = safeVariableNaming(name)
+                        elif name in keywords:
                             name = "%s%s" % (name, EVALCODE_KEYWORD_SUFFIX)
                         value = urldecode(value, convall=True)
                         variables[name] = value
@@ -1050,10 +1065,18 @@ class Connect(object):
                 except SyntaxError, ex:
                     if ex.text:
                         original = replacement = ex.text.strip()
-                        for _ in re.findall(r"[A-Za-z_]+", original)[::-1]:
-                            if _ in keywords:
-                                replacement = replacement.replace(_, "%s%s" % (_, EVALCODE_KEYWORD_SUFFIX))
-                                break
+                        if '=' in original:
+                            name, value = original.split('=', 1)
+                            name = name.strip()
+                            if safeVariableNaming(name) != name:
+                                replacement = re.sub(r"\b%s\b" % re.escape(name), safeVariableNaming(name), replacement)
+                            elif name in keywords:
+                                replacement = re.sub(r"\b%s\b" % re.escape(name), "%s%s" % (name, EVALCODE_KEYWORD_SUFFIX), replacement)
+                        else:
+                            for _ in re.findall(r"[A-Za-z_]+", original)[::-1]:
+                                if _ in keywords:
+                                    replacement = replacement.replace(_, "%s%s" % (_, EVALCODE_KEYWORD_SUFFIX))
+                                    break
                         if original == replacement:
                             conf.evalCode = conf.evalCode.replace(EVALCODE_KEYWORD_SUFFIX, "")
                             break
@@ -1073,6 +1096,11 @@ class Connect(object):
                     del variables[variable]
                     variables[variable.replace(EVALCODE_KEYWORD_SUFFIX, "")] = value
 
+                if unsafeVariableNaming(variable) != variable:
+                    value = variables[variable]
+                    del variables[variable]
+                    variables[unsafeVariableNaming(variable)] = value
+
             uri = variables["uri"]
 
             for name, value in variables.items():
@@ -1108,7 +1136,7 @@ class Connect(object):
                             found = True
                             uri = re.sub(regex.replace(r"\A", r"\?"), "\g<1>%s\g<3>" % value.replace('\\', r'\\'), uri)
 
-                        regex = r"((\A|%s)%s=).+?(%s|\Z)" % (re.escape(conf.cookieDel or DEFAULT_COOKIE_DELIMITER), name, re.escape(conf.cookieDel or DEFAULT_COOKIE_DELIMITER))
+                        regex = r"((\A|%s)%s=).+?(%s|\Z)" % (re.escape(conf.cookieDel or DEFAULT_COOKIE_DELIMITER), re.escape(name), re.escape(conf.cookieDel or DEFAULT_COOKIE_DELIMITER))
                         if re.search(regex, (cookie or "")):
                             found = True
                             cookie = re.sub(regex, "\g<1>%s\g<3>" % value.replace('\\', r'\\'), cookie)
@@ -1127,7 +1155,7 @@ class Connect(object):
         if post is not None:
             if place not in (PLACE.POST, PLACE.CUSTOM_POST) and hasattr(post, UNENCODED_ORIGINAL_VALUE):
                 post = getattr(post, UNENCODED_ORIGINAL_VALUE)
-            elif kb.postUrlEncode:
+            elif postUrlEncode:
                 post = urlencode(post, spaceplus=kb.postSpaceToPlus)
 
         if timeBasedCompare and not conf.disableStats:
@@ -1138,7 +1166,7 @@ class Connect(object):
 
                 if conf.tor:
                     warnMsg = "it's highly recommended to avoid usage of switch '--tor' for "
-                    warnMsg += "time-based injections because of its high latency time"
+                    warnMsg += "time-based injections because of inherent high latency time"
                     singleTimeWarnMessage(warnMsg)
 
                 warnMsg = "[%s] [WARNING] %stime-based comparison requires " % (time.strftime("%X"), "(case) " if kb.responseTimeMode else "")
@@ -1237,7 +1265,11 @@ class Connect(object):
             page = removeReflectiveValues(page, payload)
 
         kb.maxConnectionsFlag = re.search(MAX_CONNECTIONS_REGEX, page or "", re.I) is not None
-        kb.permissionFlag = re.search(PERMISSION_DENIED_REGEX, page or "", re.I) is not None
+
+        message = extractRegexResult(PERMISSION_DENIED_REGEX, page or "", re.I)
+        if message:
+            kb.permissionFlag = True
+            singleTimeWarnMessage("potential permission problems detected ('%s')" % message)
 
         if content or response:
             return page, headers, code
@@ -1247,5 +1279,5 @@ class Connect(object):
         else:
             return comparison(page, headers, code, getRatioValue, pageLength)
 
-def setHTTPHandlers():  # Cross-linked function
+def setHTTPHandlers():  # Cross-referenced function
     raise NotImplementedError

lib/request/direct.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import time

lib/request/dns.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import os
@@ -94,7 +94,7 @@ class DNSServer(object):
 
         with self._lock:
             for _ in self._requests:
-                if prefix is None and suffix is None or re.search("%s\..+\.%s" % (prefix, suffix), _, re.I):
+                if prefix is None and suffix is None or re.search(r"%s\..+\.%s" % (prefix, suffix), _, re.I):
                     retVal = _
                     self._requests.remove(_)
                     break

lib/request/httpshandler.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import distutils.version
@@ -48,7 +48,7 @@ class HTTPSConnection(httplib.HTTPSConnection):
 
         # Reference(s): https://docs.python.org/2/library/ssl.html#ssl.SSLContext
         #               https://www.mnot.net/blog/2014/12/27/python_2_and_tls_sni
-        if re.search(r"\A[\d.]+\Z", self.host) is None and kb.tlsSNI.get(self.host) != False and hasattr(ssl, "SSLContext"):
+        if re.search(r"\A[\d.]+\Z", self.host) is None and kb.tlsSNI.get(self.host) is not False and hasattr(ssl, "SSLContext"):
             for protocol in filter(lambda _: _ >= ssl.PROTOCOL_TLSv1, _protocols):
                 try:
                     sock = create_sock()

lib/request/inject.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import re
@@ -43,8 +43,10 @@ from lib.core.exception import SqlmapDataException
 from lib.core.exception import SqlmapNotVulnerableException
 from lib.core.exception import SqlmapUserQuitException
 from lib.core.settings import GET_VALUE_UPPERCASE_KEYWORDS
+from lib.core.settings import INFERENCE_MARKER
 from lib.core.settings import MAX_TECHNIQUES_PER_VALUE
 from lib.core.settings import SQL_SCALAR_REGEX
+from lib.core.settings import UNICODE_ENCODING
 from lib.core.threads import getCurrentThreadData
 from lib.request.connect import Connect as Request
 from lib.request.direct import direct
@@ -80,9 +82,9 @@ def _goInference(payload, expression, charsetType=None, firstChar=None, lastChar
     timeBasedCompare = (kb.technique in (PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED))
 
     if not (timeBasedCompare and kb.dnsTest):
-        if (conf.eta or conf.threads > 1) and Backend.getIdentifiedDbms() and not re.search("(COUNT|LTRIM)\(", expression, re.I) and not (timeBasedCompare and not conf.forceThreads):
+        if (conf.eta or conf.threads > 1) and Backend.getIdentifiedDbms() and not re.search(r"(COUNT|LTRIM)\(", expression, re.I) and not (timeBasedCompare and not conf.forceThreads):
 
-            if field and re.search("\ASELECT\s+DISTINCT\((.+?)\)\s+FROM", expression, re.I):
+            if field and re.search(r"\ASELECT\s+DISTINCT\((.+?)\)\s+FROM", expression, re.I):
                 expression = "SELECT %s FROM (%s)" % (field, expression)
 
                 if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL):
@@ -157,7 +159,7 @@ def _goInferenceProxy(expression, fromUser=False, batch=False, unpack=True, char
 
     _, _, _, _, _, expressionFieldsList, expressionFields, _ = agent.getFields(expression)
 
-    rdbRegExp = re.search("RDB\$GET_CONTEXT\([^)]+\)", expression, re.I)
+    rdbRegExp = re.search(r"RDB\$GET_CONTEXT\([^)]+\)", expression, re.I)
     if rdbRegExp and Backend.isDbms(DBMS.FIREBIRD):
         expressionFieldsList = [expressionFields]
 
@@ -173,10 +175,7 @@ def _goInferenceProxy(expression, fromUser=False, batch=False, unpack=True, char
     # forge the SQL limiting the query output one entry at a time
     # NOTE: we assume that only queries that get data from a table
     # can return multiple entries
-    if fromUser and " FROM " in expression.upper() and ((Backend.getIdentifiedDbms() \
-      not in FROM_DUMMY_TABLE) or (Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE and not \
-      expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]))) \
-      and not re.search(SQL_SCALAR_REGEX, expression, re.I):
+    if fromUser and " FROM " in expression.upper() and ((Backend.getIdentifiedDbms() not in FROM_DUMMY_TABLE) or (Backend.getIdentifiedDbms() in FROM_DUMMY_TABLE and not expression.upper().endswith(FROM_DUMMY_TABLE[Backend.getIdentifiedDbms()]))) and not re.search(SQL_SCALAR_REGEX, expression, re.I):
         expression, limitCond, topLimit, startLimit, stopLimit = agent.limitCondition(expression)
 
         if limitCond:
@@ -304,7 +303,7 @@ def _goBooleanProxy(expression):
             return output
 
     vector = kb.injection.data[kb.technique].vector
-    vector = vector.replace("[INFERENCE]", expression)
+    vector = vector.replace(INFERENCE_MARKER, expression)
     query = agent.prefixQuery(vector)
     query = agent.suffixQuery(query)
     payload = agent.payload(newValue=query)
@@ -347,7 +346,7 @@ def getValue(expression, blind=True, union=True, error=True, time=True, fromUser
     kb.resumeValues = resumeValue
 
     for keyword in GET_VALUE_UPPERCASE_KEYWORDS:
-        expression = re.sub("(?i)(\A|\(|\)|\s)%s(\Z|\(|\)|\s)" % keyword, r"\g<1>%s\g<2>" % keyword, expression)
+        expression = re.sub(r"(?i)(\A|\(|\)|\s)%s(\Z|\(|\)|\s)" % keyword, r"\g<1>%s\g<2>" % keyword, expression)
 
     if suppressOutput is not None:
         pushValue(getCurrentThreadData().disableStdOut)
@@ -470,6 +469,15 @@ def getValue(expression, blind=True, union=True, error=True, time=True, fromUser
         warnMsg += "or switch '--hex'" if Backend.getIdentifiedDbms() not in (DBMS.ACCESS, DBMS.FIREBIRD) else ""
         singleTimeWarnMessage(warnMsg)
 
+    # Dirty patch (safe-encoded unicode characters)
+    if isinstance(value, unicode) and "\\x" in value:
+        try:
+            candidate = eval(repr(value).replace("\\\\x", "\\x").replace("u'", "'", 1)).decode(conf.encoding or UNICODE_ENCODING)
+            if "\\x" not in candidate:
+                value = candidate
+        except:
+            pass
+
     return extractExpectedValue(value, expected)
 
 def goStacked(expression, silent=False):

lib/request/methodrequest.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import urllib2

lib/request/pkihandler.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import httplib

lib/request/rangehandler.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import urllib

lib/request/redirecthandler.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
-See the file 'doc/COPYING' for copying permission
+Copyright (c) 2006-2018 sqlmap developers (http://sqlmap.org/)
+See the file 'LICENSE' for copying permission
 """
 
 import re
@@ -129,7 +129,7 @@ class SmartRedirectHandler(urllib2.HTTPRedirectHandler):
                 if HTTP_HEADER.COOKIE not in req.headers:
                     req.headers[HTTP_HEADER.COOKIE] = _
                 else:
-                    req.headers[HTTP_HEADER.COOKIE] = re.sub("%s{2,}" % delimiter, delimiter, ("%s%s%s" % (re.sub(r"\b%s=[^%s]*%s?" % (re.escape(_.split('=')[0]), delimiter, delimiter), "", req.headers[HTTP_HEADER.COOKIE]), delimiter, _)).strip(delimiter))
+                    req.headers[HTTP_HEADER.COOKIE] = re.sub(r"%s{2,}" % delimiter, delimiter, ("%s%s%s" % (re.sub(r"\b%s=[^%s]*%s?" % (re.escape(_.split('=')[0]), delimiter, delimiter), "", req.headers[HTTP_HEADER.COOKIE]), delimiter, _)).strip(delimiter))
             try:
                 result = urllib2.HTTPRedirectHandler.http_error_302(self, req, fp, code, msg, headers)
             except urllib2.HTTPError, e: