Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 57454e5

Browse files
author
Skip Montanaro
committed
This introduces stricter library/header file checking for the Berkeley DB
library. Since multiple versions can be installed simultaneously, it's crucial that you only select libraries and header files which are compatible with each other. Version checking is done from highest version to lowest. Building using version 1 of Berkeley DB is disabled by default because of the hash file bugs people keep rediscovering. It can be enabled by uncommenting a few lines in setup.py. Closes patch 553108.
1 parent a0c5e9f commit 57454e5

7 files changed

Lines changed: 162 additions & 60 deletions

File tree

‎Misc/NEWS‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -276,6 +276,11 @@ Build
276276
- The tools used to build the documentation now work under Cygwin as
277277
well as Unix.
278278

279+
- The bsddb and dbm module builds have been changed to try and avoid version
280+
skew problems and disable linkage with Berkeley DB 1.85 unless the
281+
installer knows what s/he's doing. See the section on building these
282+
modules in the README file for details.
283+
279284
C API
280285

281286
- Added new macro PySequence_ITEM(o, i) that directly calls

‎Modules/dbmmodule.c‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,6 @@ static char *which_dbm = "ndbm";
1818
#else
1919
static char *which_dbm = "GNU gdbm"; /* EMX port of GDBM */
2020
#endif
21-
#elif defined(HAVE_DB1_NDBM_H)
22-
#include <db1/ndbm.h>
23-
static char *which_dbm = "BSD db";
2421
#elif defined(HAVE_GDBM_NDBM_H)
2522
#include <gdbm/ndbm.h>
2623
static char *which_dbm = "GNU gdbm";

‎README‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -503,6 +503,46 @@ Cygwin: With recent (relative to the time of writing, 2001-12-19)
503503
News regarding these platforms with more recent Cygwin
504504
versions would be appreciated!
505505

506+
Configuring the bsddb and dbm modules
507+
-------------------------------------
508+
509+
Configuring the bsddb module can sometimes be a bit tricky. This module
510+
provides a Python interface to the Berkeley DB library. As of this writing
511+
several versions of the underlying library are in common use (versions 1.85,
512+
2.x, 3.x, and 4.x). The file formats across the various versions tend to be
513+
incompatible. Some Linux distributions install multiple versions by
514+
default. It is important that compatible versions of header files and
515+
libraries are used when building bsddb. To make matters worse, version 1.85
516+
of Berkeley DB has known bugs in its hash file implementation, but is still
517+
the most widely available version of the library. Many people build bsddb
518+
with version 1.85 but aren't aware of the bugs. This affects people using
519+
the anydbm and dbhash modules because they are both use Berkeley DB's hash
520+
file format as a side effect of calling bsddb.hashopen.
521+
522+
To try and remedy this problem, beginning with Python version 2.3 a number
523+
of changes to the bsddb build process were made. First, and most important,
524+
the bsddb module will not be built with version 1.85 unless the relevant
525+
lines in setup.py are uncommented first and no other higher-numbered
526+
versions are found. Second, matching versions of the library and include
527+
files must be found. Third, searching is performed in order, starting from
528+
version 4 and proceeding to version 2 (or version 1 if it is enabled).
529+
Version-independent libraries and header files (e.g. /usr/lib/libdb.a and
530+
/usr/include/db.h) are never considered. They must be in version-specific
531+
directories or have version-specific filenames (e.g. /usr/lib/libdb-3.2.so
532+
and /usr/include/db3/db_185.h).
533+
534+
Since the bsddb module is programmed using the Berkeley DB version 1 API,
535+
the underlying library must be configured with the --enable-compat185 flag.
536+
Most vendor-provided distributions are so-configured. This is generally
537+
only an issue if you build Berkeley DB from source.
538+
539+
All this affects the dbm module as well. There are several dbm-compliant
540+
APIs provided by different libraries, including ndbm, gdbm and Berkeley DB.
541+
The build process for dbm would previously use the version 1.85 library,
542+
thus extending the potential hash file corruption to the dbm module as well.
543+
The dbm module will use the library and include files found for the bsddb
544+
module if neither ndbm nor gdbm libraries are found.
545+
506546
Configuring threads
507547
-------------------
508548

‎configure‎

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
#! /bin/sh
2-
# From configure.in Revision: 1.322 .
2+
# From configure.in Revision: 1.323 .
33
# Guess values for system-dependent variables and create Makefiles.
44
# Generated by GNU Autoconf 2.53.
55
#
@@ -3861,20 +3861,17 @@ fi
38613861
38623862
38633863
3864-
3865-
3866-
38673864
38683865
38693866
38703867
38713868
for ac_header in dlfcn.h fcntl.h grp.h limits.h langinfo.h \
38723869
libintl.h locale.h ncurses.h poll.h pthread.h \
38733870
signal.h stdarg.h stddef.h stdlib.h thread.h unistd.h utime.h termios.h \
3874-
sys/audioio.h sys/file.h sys/lock.h sys/mkdev.h sys/modem.h db_185.h db.h \
3871+
sys/audioio.h sys/file.h sys/lock.h sys/mkdev.h sys/modem.h \
38753872
sys/param.h sys/poll.h sys/select.h sys/socket.h sys/time.h sys/times.h \
38763873
sys/un.h sys/utsname.h sys/wait.h pty.h term.h libutil.h \
3877-
ndbm.h db1/ndbm.h gdbm/ndbm.h sys/resource.h netpacket/packet.h
3874+
ndbm.h gdbm/ndbm.h sys/resource.h netpacket/packet.h
38783875
do
38793876
as_ac_Header=`echo "ac_cv_header_$ac_header" | $as_tr_sh`
38803877
if eval "test \"\${$as_ac_Header+set}\" = set"; then

‎configure.in‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -596,10 +596,10 @@ AC_HEADER_STDC
596596
AC_CHECK_HEADERS(dlfcn.h fcntl.h grp.h limits.h langinfo.h \
597597
libintl.h locale.h ncurses.h poll.h pthread.h \
598598
signal.h stdarg.h stddef.h stdlib.h thread.h unistd.h utime.h termios.h \
599-
sys/audioio.h sys/file.h sys/lock.h sys/mkdev.h sys/modem.h db_185.h db.h \
599+
sys/audioio.h sys/file.h sys/lock.h sys/mkdev.h sys/modem.h \
600600
sys/param.h sys/poll.h sys/select.h sys/socket.h sys/time.h sys/times.h \
601601
sys/un.h sys/utsname.h sys/wait.h pty.h term.h libutil.h \
602-
ndbm.h db1/ndbm.h gdbm/ndbm.h sys/resource.h netpacket/packet.h)
602+
ndbm.h gdbm/ndbm.h sys/resource.h netpacket/packet.h)
603603
AC_HEADER_DIRENT
604604

605605
# checks for typedefs

‎pyconfig.h.in‎

Lines changed: 0 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -61,15 +61,6 @@
6161
/* Define to 1 if you have the `ctermid_r' function. */
6262
#undef HAVE_CTERMID_R
6363

64-
/* Define to 1 if you have the <db1/ndbm.h> header file. */
65-
#undef HAVE_DB1_NDBM_H
66-
67-
/* Define to 1 if you have the <db_185.h> header file. */
68-
#undef HAVE_DB_185_H
69-
70-
/* Define to 1 if you have the <db.h> header file. */
71-
#undef HAVE_DB_H
72-
7364
/* Define to 1 if you have the <dirent.h> header file, and it defines `DIR'.
7465
*/
7566
#undef HAVE_DIRENT_H

‎setup.py‎

Lines changed: 112 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -418,14 +418,124 @@ def detect_modules(self):
418418
# implementation independent wrapper for these; dumbdbm.py provides
419419
# similar functionality (but slower of course) implemented in Python.
420420

421+
# Berkeley DB interface.
422+
#
423+
# This requires the Berkeley DB code, see
424+
# ftp://ftp.cs.berkeley.edu/pub/4bsd/db.1.85.tar.gz
425+
#
426+
# (See http://pybsddb.sourceforge.net/ for an interface to
427+
# Berkeley DB 3.x.)
428+
429+
# when sorted in reverse order, keys for this dict must appear in the
430+
# order you wish to search - e.g., search for db3 before db2, db2
431+
# before db1
432+
db_try_this = {
433+
'db4': {'libs': ('db-4.3', 'db-4.2', 'db-4.1', 'db-4.0'),
434+
'libdirs': ('/usr/local/BerkeleyDB.4.3/lib',
435+
'/usr/local/BerkeleyDB.4.2/lib',
436+
'/usr/local/BerkeleyDB.4.1/lib',
437+
'/usr/local/BerkeleyDB.4.0/lib',
438+
'/usr/lib',
439+
'/opt/sfw',
440+
'/sw/lib',
441+
'/lib',
442+
),
443+
'incdirs': ('/usr/local/BerkeleyDB.4.3/include',
444+
'/usr/local/BerkeleyDB.4.2/include',
445+
'/usr/local/BerkeleyDB.4.1/include',
446+
'/usr/local/BerkeleyDB.4.0/include',
447+
'/usr/include/db3',
448+
'/opt/sfw/include/db3',
449+
'/sw/include/db3',
450+
'/usr/local/include/db3',
451+
),
452+
'incs': ('db_185.h',)},
453+
'db3': {'libs': ('db-3.3', 'db-3.2', 'db-3.1', 'db-3.0'),
454+
'libdirs': ('/usr/local/BerkeleyDB.3.3/lib',
455+
'/usr/local/BerkeleyDB.3.2/lib',
456+
'/usr/local/BerkeleyDB.3.1/lib',
457+
'/usr/local/BerkeleyDB.3.0/lib',
458+
'/usr/lib',
459+
'/opt/sfw',
460+
'/sw/lib',
461+
'/lib',
462+
),
463+
'incdirs': ('/usr/local/BerkeleyDB.3.3/include',
464+
'/usr/local/BerkeleyDB.3.2/include',
465+
'/usr/local/BerkeleyDB.3.1/include',
466+
'/usr/local/BerkeleyDB.3.0/include',
467+
'/usr/include/db3',
468+
'/opt/sfw/include/db3',
469+
'/sw/include/db3',
470+
'/usr/local/include/db3',
471+
),
472+
'incs': ('db_185.h',)},
473+
'db2': {'libs': ('db2',),
474+
'libdirs': ('/usr/lib', '/sw/lib', '/lib'),
475+
'incdirs': ('/usr/include/db2',
476+
'/usr/local/include/db2', '/sw/include/db2'),
477+
'incs': ('db_185.h',)},
478+
# if you are willing to risk hash db file corruption you can
479+
# uncomment the lines below for db1. Note that this will affect
480+
# not only the bsddb module, but the dbhash and anydbm modules
481+
# as well. you have been warned!!!
482+
##'db1': {'libs': ('db1', 'db'),
483+
## 'libdirs': ('/usr/lib', '/sw/lib', '/lib'),
484+
## 'incdirs': ('/usr/include/db1', '/usr/local/include/db1',
485+
## '/usr/include', '/usr/local/include'),
486+
## 'incs': ('db.h',)},
487+
}
488+
489+
# override this list to affect the library version search order
490+
# for example, if you want to force version 2 to be used:
491+
# db_search_order = ["db2"]
492+
db_search_order = db_try_this.keys()
493+
db_search_order.sort()
494+
db_search_order.reverse()
495+
496+
find_lib_file = self.compiler.find_library_file
497+
class found(Exception): pass
498+
try:
499+
for dbkey in db_search_order:
500+
dbd = db_try_this[dbkey]
501+
for dblib in dbd['libs']:
502+
for dbinc in dbd['incs']:
503+
db_incs = find_file(dbinc, [], dbd['incdirs'])
504+
dblib_dir = find_lib_file(dbd['libdirs'], dblib)
505+
if db_incs and dblib_dir:
506+
dblib_dir = os.path.dirname(dblib_dir)
507+
dblibs = [dblib]
508+
raise found
509+
except found:
510+
if dbinc == 'db_185.h':
511+
exts.append(Extension('bsddb', ['bsddbmodule.c'],
512+
library_dirs=[dblib_dir],
513+
include_dirs=db_incs,
514+
define_macros=[('HAVE_DB_185_H',1)],
515+
libraries=[dblib]))
516+
else:
517+
exts.append(Extension('bsddb', ['bsddbmodule.c'],
518+
library_dirs=[dblib_dir],
519+
include_dirs=db_incs,
520+
libraries=[dblib]))
521+
else:
522+
db_incs = None
523+
dblibs = []
524+
dblib_dir = None
525+
421526
# The standard Unix dbm module:
422527
if platform not in ['cygwin']:
423528
if (self.compiler.find_library_file(lib_dirs, 'ndbm')):
424529
exts.append( Extension('dbm', ['dbmmodule.c'],
425530
libraries = ['ndbm'] ) )
426-
elif self.compiler.find_library_file(lib_dirs, 'db1'):
531+
elif self.compiler.find_library_file(lib_dirs, 'gdbm'):
427532
exts.append( Extension('dbm', ['dbmmodule.c'],
428-
libraries = ['db1'] ) )
533+
libraries = ['gdbm'] ) )
534+
elif db_incs is not None:
535+
exts.append( Extension('dbm', ['dbmmodule.c'],
536+
library_dirs=dblib_dir,
537+
include_dirs=db_incs,
538+
libraries=dblibs))
429539
else:
430540
exts.append( Extension('dbm', ['dbmmodule.c']) )
431541

@@ -434,44 +544,6 @@ def detect_modules(self):
434544
exts.append( Extension('gdbm', ['gdbmmodule.c'],
435545
libraries = ['gdbm'] ) )
436546

437-
# Berkeley DB interface.
438-
#
439-
# This requires the Berkeley DB code, see
440-
# ftp://ftp.cs.berkeley.edu/pub/4bsd/db.1.85.tar.gz
441-
#
442-
# Edit the variables DB and DBPORT to point to the db top directory
443-
# and the subdirectory of PORT where you built it.
444-
#
445-
# (See http://pybsddb.sourceforge.net/ for an interface to
446-
# Berkeley DB 3.x.)
447-
448-
dblib = []
449-
if self.compiler.find_library_file(lib_dirs, 'db-3.2'):
450-
dblib = ['db-3.2']
451-
elif self.compiler.find_library_file(lib_dirs, 'db-3.1'):
452-
dblib = ['db-3.1']
453-
elif self.compiler.find_library_file(lib_dirs, 'db3'):
454-
dblib = ['db3']
455-
elif self.compiler.find_library_file(lib_dirs, 'db2'):
456-
dblib = ['db2']
457-
elif self.compiler.find_library_file(lib_dirs, 'db1'):
458-
dblib = ['db1']
459-
elif self.compiler.find_library_file(lib_dirs, 'db'):
460-
dblib = ['db']
461-
462-
db185_incs = find_file('db_185.h', inc_dirs,
463-
['/usr/include/db3', '/usr/include/db2'])
464-
db_inc = find_file('db.h', inc_dirs, ['/usr/include/db1'])
465-
if db185_incs is not None:
466-
exts.append( Extension('bsddb', ['bsddbmodule.c'],
467-
include_dirs = db185_incs,
468-
define_macros=[('HAVE_DB_185_H',1)],
469-
libraries = dblib ) )
470-
elif db_inc is not None:
471-
exts.append( Extension('bsddb', ['bsddbmodule.c'],
472-
include_dirs = db_inc,
473-
libraries = dblib) )
474-
475547
# The mpz module interfaces to the GNU Multiple Precision library.
476548
# You need to ftp the GNU MP library.
477549
# This was originally written and tested against GMP 1.2 and 1.3.2.

0 commit comments

Comments
 (0)