mirror of
https://github.com/nmap/nmap.git
synced 2026-02-04 20:46:33 +00:00
get rid of silly top-level trunk dir
This commit is contained in:
fyodor
357
libpcre/README
Normal file
357
libpcre/README
Normal file
@@ -0,0 +1,357 @@
|
||||
README file for PCRE (Perl-compatible regular expression library)
|
||||
-----------------------------------------------------------------
|
||||
|
||||
The latest release of PCRE is always available from
|
||||
|
||||
ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.gz
|
||||
|
||||
Please read the NEWS file if you are upgrading from a previous release.
|
||||
|
||||
PCRE has its own native API, but a set of "wrapper" functions that are based on
|
||||
the POSIX API are also supplied in the library libpcreposix. Note that this
|
||||
just provides a POSIX calling interface to PCRE: the regular expressions
|
||||
themselves still follow Perl syntax and semantics. The header file
|
||||
for the POSIX-style functions is called pcreposix.h. The official POSIX name is
|
||||
regex.h, but I didn't want to risk possible problems with existing files of
|
||||
that name by distributing it that way. To use it with an existing program that
|
||||
uses the POSIX API, it will have to be renamed or pointed at by a link.
|
||||
|
||||
|
||||
Contributions by users of PCRE
|
||||
------------------------------
|
||||
|
||||
You can find contributions from PCRE users in the directory
|
||||
|
||||
ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/Contrib
|
||||
|
||||
where there is also a README file giving brief descriptions of what they are.
|
||||
Several of them provide support for compiling PCRE on various flavours of
|
||||
Windows systems (I myself do not use Windows). Some are complete in themselves;
|
||||
others are pointers to URLs containing relevant files.
|
||||
|
||||
|
||||
Building PCRE on a Unix-like system
|
||||
-----------------------------------
|
||||
|
||||
To build PCRE on a Unix-like system, first run the "configure" command from the
|
||||
PCRE distribution directory, with your current directory set to the directory
|
||||
where you want the files to be created. This command is a standard GNU
|
||||
"autoconf" configuration script, for which generic instructions are supplied in
|
||||
INSTALL.
|
||||
|
||||
Most commonly, people build PCRE within its own distribution directory, and in
|
||||
this case, on many systems, just running "./configure" is sufficient, but the
|
||||
usual methods of changing standard defaults are available. For example,
|
||||
|
||||
CFLAGS='-O2 -Wall' ./configure --prefix=/opt/local
|
||||
|
||||
specifies that the C compiler should be run with the flags '-O2 -Wall' instead
|
||||
of the default, and that "make install" should install PCRE under /opt/local
|
||||
instead of the default /usr/local.
|
||||
|
||||
If you want to build in a different directory, just run "configure" with that
|
||||
directory as current. For example, suppose you have unpacked the PCRE source
|
||||
into /source/pcre/pcre-xxx, but you want to build it in /build/pcre/pcre-xxx:
|
||||
|
||||
cd /build/pcre/pcre-xxx
|
||||
/source/pcre/pcre-xxx/configure
|
||||
|
||||
There are some optional features that can be included or omitted from the PCRE
|
||||
library. You can read more about them in the pcrebuild man page.
|
||||
|
||||
. If you want to make use of the support for UTF-8 character strings in PCRE,
|
||||
you must add --enable-utf8 to the "configure" command. Without it, the code
|
||||
for handling UTF-8 is not included in the library. (Even when included, it
|
||||
still has to be enabled by an option at run time.)
|
||||
|
||||
. You can build PCRE to recognized CR or NL as the newline character, instead
|
||||
of whatever your compiler uses for "\n", by adding --newline-is-cr or
|
||||
--newline-is-nl to the "configure" command, respectively. Only do this if you
|
||||
really understand what you are doing. On traditional Unix-like systems, the
|
||||
newline character is NL.
|
||||
|
||||
. When called via the POSIX interface, PCRE uses malloc() to get additional
|
||||
storage for processing capturing parentheses if there are more than 10 of
|
||||
them. You can increase this threshold by setting, for example,
|
||||
|
||||
--with-posix-malloc-threshold=20
|
||||
|
||||
on the "configure" command.
|
||||
|
||||
. PCRE has a counter which can be set to limit the amount of resources it uses.
|
||||
If the limit is exceeded during a match, the match fails. The default is ten
|
||||
million. You can change the default by setting, for example,
|
||||
|
||||
--with-match-limit=500000
|
||||
|
||||
on the "configure" command. This is just the default; individual calls to
|
||||
pcre_exec() can supply their own value. There is discussion on the pcreapi
|
||||
man page.
|
||||
|
||||
. The default maximum compiled pattern size is around 64K. You can increase
|
||||
this by adding --with-link-size=3 to the "configure" command. You can
|
||||
increase it even more by setting --with-link-size=4, but this is unlikely
|
||||
ever to be necessary. If you build PCRE with an increased link size, test 2
|
||||
(and 5 if you are using UTF-8) will fail. Part of the output of these tests
|
||||
is a representation of the compiled pattern, and this changes with the link
|
||||
size.
|
||||
|
||||
The "configure" script builds five files:
|
||||
|
||||
. libtool is a script that builds shared and/or static libraries
|
||||
. Makefile is built by copying Makefile.in and making substitutions.
|
||||
. config.h is built by copying config.in and making substitutions.
|
||||
. pcre-config is built by copying pcre-config.in and making substitutions.
|
||||
. RunTest is a script for running tests
|
||||
|
||||
Once "configure" has run, you can run "make". It builds two libraries called
|
||||
libpcre and libpcreposix, a test program called pcretest, and the pcregrep
|
||||
command. You can use "make install" to copy these, the public header files
|
||||
pcre.h and pcreposix.h, and the man pages to appropriate live directories on
|
||||
your system, in the normal way.
|
||||
|
||||
Running "make install" also installs the command pcre-config, which can be used
|
||||
to recall information about the PCRE configuration and installation. For
|
||||
example,
|
||||
|
||||
pcre-config --version
|
||||
|
||||
prints the version number, and
|
||||
|
||||
pcre-config --libs
|
||||
|
||||
outputs information about where the library is installed. This command can be
|
||||
included in makefiles for programs that use PCRE, saving the programmer from
|
||||
having to remember too many details.
|
||||
|
||||
|
||||
Cross-compiling PCRE on a Unix-like system
|
||||
------------------------------------------
|
||||
|
||||
PCRE needs to compile and run an auxiliary program as part of the building
|
||||
process. Obviously, if the real compilation is for some other system, it can't
|
||||
use the same CC and CFLAGS values when it is doing this. For cross compilation,
|
||||
therefore, you must set CC_FOR_BUILD to the local host's compiler, and you can
|
||||
set flags in CFLAGS_FOR_BUILD if you need to.
|
||||
|
||||
|
||||
Shared libraries on Unix-like systems
|
||||
-------------------------------------
|
||||
|
||||
The default distribution builds PCRE as two shared libraries and two static
|
||||
libraries, as long as the operating system supports shared libraries. Shared
|
||||
library support relies on the "libtool" script which is built as part of the
|
||||
"configure" process.
|
||||
|
||||
The libtool script is used to compile and link both shared and static
|
||||
libraries. They are placed in a subdirectory called .libs when they are newly
|
||||
built. The programs pcretest and pcregrep are built to use these uninstalled
|
||||
libraries (by means of wrapper scripts in the case of shared libraries). When
|
||||
you use "make install" to install shared libraries, pcregrep and pcretest are
|
||||
automatically re-built to use the newly installed shared libraries before being
|
||||
installed themselves. However, the versions left in the source directory still
|
||||
use the uninstalled libraries.
|
||||
|
||||
To build PCRE using static libraries only you must use --disable-shared when
|
||||
configuring it. For example
|
||||
|
||||
./configure --prefix=/usr/gnu --disable-shared
|
||||
|
||||
Then run "make" in the usual way. Similarly, you can use --disable-static to
|
||||
build only shared libraries.
|
||||
|
||||
|
||||
Cross-compiling on a Unix-like system
|
||||
-------------------------------------
|
||||
|
||||
You can specify CC and CFLAGS in the normal way to the "configure" command, in
|
||||
order to cross-compile PCRE for some other host. However, during the building
|
||||
process, the dftables.c source file is compiled *and run* on the local host, in
|
||||
order to generate the default character tables (the chartables.c file). It
|
||||
therefore needs to be compiled with the local compiler, not the cross compiler.
|
||||
You can do this by specifying HOST_CC (and if necessary HOST_CFLAGS) when
|
||||
calling the "configure" command. If they are not specified, they default to the
|
||||
values of CC and CFLAGS.
|
||||
|
||||
|
||||
Building on non-Unix systems
|
||||
----------------------------
|
||||
|
||||
For a non-Unix system, read the comments in the file NON-UNIX-USE. PCRE has
|
||||
been compiled on Windows systems and on Macintoshes, but I don't know the
|
||||
details because I don't use those systems. It should be straightforward to
|
||||
build PCRE on any system that has a Standard C compiler, because it uses only
|
||||
Standard C functions.
|
||||
|
||||
|
||||
Testing PCRE
|
||||
------------
|
||||
|
||||
To test PCRE on a Unix system, run the RunTest script that is created by the
|
||||
configuring process. (This can also be run by "make runtest", "make check", or
|
||||
"make test".) For other systems, see the instruction in NON-UNIX-USE.
|
||||
|
||||
The script runs the pcretest test program (which is documented in its own man
|
||||
page) on each of the testinput files (in the testdata directory) in turn,
|
||||
and compares the output with the contents of the corresponding testoutput file.
|
||||
A file called testtry is used to hold the output from pcretest. To run pcretest
|
||||
on just one of the test files, give its number as an argument to RunTest, for
|
||||
example:
|
||||
|
||||
RunTest 2
|
||||
|
||||
The first file can also be fed directly into the perltest script to check that
|
||||
Perl gives the same results. The only difference you should see is in the first
|
||||
few lines, where the Perl version is given instead of the PCRE version.
|
||||
|
||||
The second set of tests check pcre_fullinfo(), pcre_info(), pcre_study(),
|
||||
pcre_copy_substring(), pcre_get_substring(), pcre_get_substring_list(), error
|
||||
detection, and run-time flags that are specific to PCRE, as well as the POSIX
|
||||
wrapper API. It also uses the debugging flag to check some of the internals of
|
||||
pcre_compile().
|
||||
|
||||
If you build PCRE with a locale setting that is not the standard C locale, the
|
||||
character tables may be different (see next paragraph). In some cases, this may
|
||||
cause failures in the second set of tests. For example, in a locale where the
|
||||
isprint() function yields TRUE for characters in the range 128-255, the use of
|
||||
[:isascii:] inside a character class defines a different set of characters, and
|
||||
this shows up in this test as a difference in the compiled code, which is being
|
||||
listed for checking. Where the comparison test output contains [\x00-\x7f] the
|
||||
test will contain [\x00-\xff], and similarly in some other cases. This is not a
|
||||
bug in PCRE.
|
||||
|
||||
The third set of tests checks pcre_maketables(), the facility for building a
|
||||
set of character tables for a specific locale and using them instead of the
|
||||
default tables. The tests make use of the "fr" (French) locale. Before running
|
||||
the test, the script checks for the presence of this locale by running the
|
||||
"locale" command. If that command fails, or if it doesn't include "fr" in the
|
||||
list of available locales, the third test cannot be run, and a comment is
|
||||
output to say why. If running this test produces instances of the error
|
||||
|
||||
** Failed to set locale "fr"
|
||||
|
||||
in the comparison output, it means that locale is not available on your system,
|
||||
despite being listed by "locale". This does not mean that PCRE is broken.
|
||||
|
||||
The fourth test checks the UTF-8 support. It is not run automatically unless
|
||||
PCRE is built with UTF-8 support. To do this you must set --enable-utf8 when
|
||||
running "configure". This file can be also fed directly to the perltest script,
|
||||
provided you are running Perl 5.8 or higher. (For Perl 5.6, a small patch,
|
||||
commented in the script, can be be used.)
|
||||
|
||||
The fifth and final file tests error handling with UTF-8 encoding, and internal
|
||||
UTF-8 features of PCRE that are not relevant to Perl.
|
||||
|
||||
|
||||
Character tables
|
||||
----------------
|
||||
|
||||
PCRE uses four tables for manipulating and identifying characters. The final
|
||||
argument of the pcre_compile() function is a pointer to a block of memory
|
||||
containing the concatenated tables. A call to pcre_maketables() can be used to
|
||||
generate a set of tables in the current locale. If the final argument for
|
||||
pcre_compile() is passed as NULL, a set of default tables that is built into
|
||||
the binary is used.
|
||||
|
||||
The source file called chartables.c contains the default set of tables. This is
|
||||
not supplied in the distribution, but is built by the program dftables
|
||||
(compiled from dftables.c), which uses the ANSI C character handling functions
|
||||
such as isalnum(), isalpha(), isupper(), islower(), etc. to build the table
|
||||
sources. This means that the default C locale which is set for your system will
|
||||
control the contents of these default tables. You can change the default tables
|
||||
by editing chartables.c and then re-building PCRE. If you do this, you should
|
||||
probably also edit Makefile to ensure that the file doesn't ever get
|
||||
re-generated.
|
||||
|
||||
The first two 256-byte tables provide lower casing and case flipping functions,
|
||||
respectively. The next table consists of three 32-byte bit maps which identify
|
||||
digits, "word" characters, and white space, respectively. These are used when
|
||||
building 32-byte bit maps that represent character classes.
|
||||
|
||||
The final 256-byte table has bits indicating various character types, as
|
||||
follows:
|
||||
|
||||
1 white space character
|
||||
2 letter
|
||||
4 decimal digit
|
||||
8 hexadecimal digit
|
||||
16 alphanumeric or '_'
|
||||
128 regular expression metacharacter or binary zero
|
||||
|
||||
You should not alter the set of characters that contain the 128 bit, as that
|
||||
will cause PCRE to malfunction.
|
||||
|
||||
|
||||
Manifest
|
||||
--------
|
||||
|
||||
The distribution should contain the following files:
|
||||
|
||||
(A) The actual source files of the PCRE library functions and their
|
||||
headers:
|
||||
|
||||
dftables.c auxiliary program for building chartables.c
|
||||
get.c )
|
||||
maketables.c )
|
||||
study.c ) source of
|
||||
pcre.c ) the functions
|
||||
pcreposix.c )
|
||||
printint.c )
|
||||
pcre.in "source" for the header for the external API; pcre.h
|
||||
is built from this by "configure"
|
||||
pcreposix.h header for the external POSIX wrapper API
|
||||
internal.h header for internal use
|
||||
config.in template for config.h, which is built by configure
|
||||
|
||||
(B) Auxiliary files:
|
||||
|
||||
AUTHORS information about the author of PCRE
|
||||
ChangeLog log of changes to the code
|
||||
INSTALL generic installation instructions
|
||||
LICENCE conditions for the use of PCRE
|
||||
COPYING the same, using GNU's standard name
|
||||
Makefile.in template for Unix Makefile, which is built by configure
|
||||
NEWS important changes in this release
|
||||
NON-UNIX-USE notes on building PCRE on non-Unix systems
|
||||
README this file
|
||||
RunTest.in template for a Unix shell script for running tests
|
||||
config.guess ) files used by libtool,
|
||||
config.sub ) used only when building a shared library
|
||||
configure a configuring shell script (built by autoconf)
|
||||
configure.in the autoconf input used to build configure
|
||||
doc/Tech.Notes notes on the encoding
|
||||
doc/*.3 man page sources for the PCRE functions
|
||||
doc/*.1 man page sources for pcregrep and pcretest
|
||||
doc/html/* HTML documentation
|
||||
doc/pcre.txt plain text version of the man pages
|
||||
doc/pcretest.txt plain text documentation of test program
|
||||
doc/perltest.txt plain text documentation of Perl test program
|
||||
install-sh a shell script for installing files
|
||||
ltmain.sh file used to build a libtool script
|
||||
pcretest.c comprehensive test program
|
||||
pcredemo.c simple demonstration of coding calls to PCRE
|
||||
perltest Perl test program
|
||||
pcregrep.c source of a grep utility that uses PCRE
|
||||
pcre-config.in source of script which retains PCRE information
|
||||
testdata/testinput1 test data, compatible with Perl
|
||||
testdata/testinput2 test data for error messages and non-Perl things
|
||||
testdata/testinput3 test data for locale-specific tests
|
||||
testdata/testinput4 test data for UTF-8 tests compatible with Perl
|
||||
testdata/testinput5 test data for other UTF-8 tests
|
||||
testdata/testoutput1 test results corresponding to testinput1
|
||||
testdata/testoutput2 test results corresponding to testinput2
|
||||
testdata/testoutput3 test results corresponding to testinput3
|
||||
testdata/testoutput4 test results corresponding to testinput4
|
||||
testdata/testoutput5 test results corresponding to testinput5
|
||||
|
||||
(C) Auxiliary files for Win32 DLL
|
||||
|
||||
dll.mk
|
||||
pcre.def
|
||||
|
||||
(D) Auxiliary file for VPASCAL
|
||||
|
||||
makevp.bat
|
||||
|
||||
Philip Hazel <ph10@cam.ac.uk>
|
||||
February 2003
|
||||
Reference in New Issue
Block a user