Introduction

This API has been defined to encourage similarity between the Python modules that are used to access databases. By doing this, we hope to achieve a consistency leading to more easily understood modules, code that is generally more portable across databases, and a broader reach of database connectivity from Python.

Comments and questions about this specification may be directed to the SIG for Database Interfacing with Python.

For more information on database interfacing with Python and available packages see the Database Topic Guide.

This document describes the Python Database API Specification 2.0 and a set of common optional extensions. The previous version 1.0 version is still available as reference, in PEP 248. Package writers are encouraged to use this version of the specification as basis for new interfaces.

Module Interface

StandardError
|__Warning
|__Error
   |__InterfaceError
   |__DatabaseError
      |__DataError
      |__OperationalError
      |__IntegrityError
      |__InternalError
      |__ProgrammingError
      |__NotSupportedError

Connection Objects

Connection objects should respond to the following methods.

Cursor Objects

These objects represent a database cursor, which is used to manage the context of a fetch operation. Cursors created from the same connection are not isolated, i.e., any changes done to the database by a cursor are immediately visible by the other cursors. Cursors created from different connections can or can not be isolated, depending on how the transaction support is implemented (see also the connection’s .rollback() and .commit() methods).

Cursor Objects should respond to the following methods and attributes.

Type Objects and Constructors

Many databases need to have the input in a particular format for binding to an operation’s input parameters. For example, if an input is destined for a DATE column, then it must be bound to the database in a particular string format. Similar problems exist for “Row ID” columns or large binary items (e.g. blobs or RAW columns). This presents problems for Python since the parameters to the .execute*() method are untyped. When the database module sees a Python string object, it doesn’t know if it should be bound as a simple CHAR column, as a raw BINARY item, or as a DATE.

To overcome this problem, a module must provide the constructors defined below to create objects that can hold special values. When passed to the cursor methods, the module can then detect the proper type of the input parameter and bind it accordingly.

A Cursor Object’s description attribute returns information about each of the result columns of a query. The type_code must compare equal to one of Type Objects defined below. Type Objects may be equal to more than one type code (e.g. DATETIME could be equal to the type codes for date, time and timestamp columns; see the Implementation Hints below for details).

The module exports the following constructors and singletons:

Date(year, month, day)

This function constructs an object holding a date value.

Time(hour, minute, second)

This function constructs an object holding a time value.

Timestamp(year, month, day, hour, minute, second)

This function constructs an object holding a time stamp value.

DateFromTicks(ticks)

This function constructs an object holding a date value from the given ticks value (number of seconds since the epoch; see the documentation of the standard Python time module for details).

TimeFromTicks(ticks)

This function constructs an object holding a time value from the given ticks value (number of seconds since the epoch; see the documentation of the standard Python time module for details).

TimestampFromTicks(ticks)

This function constructs an object holding a time stamp value from the given ticks value (number of seconds since the epoch; see the documentation of the standard Python time module for details).

Binary(string)

This function constructs an object capable of holding a binary (long) string value.

STRING type

This type object is used to describe columns in a database that are string-based (e.g. CHAR).

BINARY type

This type object is used to describe (long) binary columns in a database (e.g. LONG, RAW, BLOBs).

NUMBER type

This type object is used to describe numeric columns in a database.

DATETIME type

This type object is used to describe date/time columns in a database.

ROWID type

This type object is used to describe the “Row ID” column in a database.

SQL NULL values are represented by the Python None singleton on input and output.

Implementation Hints for Module Authors

• Date/time objects can be implemented as Python datetime module objects (available since Python 2.3, with a C API since 2.4) or using the mxDateTime package (available for all Python versions since 1.5.2). They both provide all necessary constructors and methods at Python and C level.
• Here is a sample implementation of the Unix ticks based constructors for date/time delegating work to the generic constructors:

  import time
  
  def DateFromTicks(ticks):
      return Date(*time.localtime(ticks)[:3])
  
  def TimeFromTicks(ticks):
      return Time(*time.localtime(ticks)[3:6])
  
  def TimestampFromTicks(ticks):
      return Timestamp(*time.localtime(ticks)[:6])
  

• The preferred object type for Binary objects are the buffer types available in standard Python starting with version 1.5.2. Please see the Python documentation for details. For information about the C interface have a look at Include/bufferobject.h and Objects/bufferobject.c in the Python source distribution.
• This Python class allows implementing the above type objects even though the description type code field yields multiple values for on type object:

  class DBAPITypeObject:
      def __init__(self,*values):
          self.values = values
      def __cmp__(self,other):
          if other in self.values:
              return 0
          if other < self.values:
              return 1
          else:
              return -1
  

  The resulting type object compares equal to all values passed to the constructor.

• Here is a snippet of Python code that implements the exception hierarchy defined above:

  import exceptions
  
  class Error(exceptions.StandardError):
      pass
  
  class Warning(exceptions.StandardError):
      pass
  
  class InterfaceError(Error):
      pass
  
  class DatabaseError(Error):
      pass
  
  class InternalError(DatabaseError):
      pass
  
  class OperationalError(DatabaseError):
      pass
  
  class ProgrammingError(DatabaseError):
      pass
  
  class IntegrityError(DatabaseError):
      pass
  
  class DataError(DatabaseError):
      pass
  
  class NotSupportedError(DatabaseError):
      pass
  

  In C you can use the PyErr_NewException(fullname, base, NULL) API to create the exception objects.

Optional DB API Extensions

During the lifetime of DB API 2.0, module authors have often extended their implementations beyond what is required by this DB API specification. To enhance compatibility and to provide a clean upgrade path to possible future versions of the specification, this section defines a set of common extensions to the core DB API 2.0 specification.

As with all DB API optional features, the database module authors are free to not implement these additional attributes and methods (using them will then result in an AttributeError) or to raise a NotSupportedError in case the availability can only be checked at run-time.

It has been proposed to make usage of these extensions optionally visible to the programmer by issuing Python warnings through the Python warning framework. To make this feature useful, the warning messages must be standardized in order to be able to mask them. These standard messages are referred to below as Warning Message.

Cursor.rownumber

This read-only attribute should provide the current 0-based index of the cursor in the result set or None if the index cannot be determined.

The index can be seen as index of the cursor in a sequence (the result set). The next fetch operation will fetch the row indexed by .rownumber in that sequence.

Warning Message: “DB-API extension cursor.rownumber used”

Connection.Error, Connection.ProgrammingError, etc.

All exception classes defined by the DB API standard should be exposed on the Connection objects as attributes (in addition to being available at module scope).

These attributes simplify error handling in multi-connection environments.

Warning Message: “DB-API extension connection.<exception> used”

Cursor.connection

This read-only attribute return a reference to the Connection object on which the cursor was created.

The attribute simplifies writing polymorph code in multi-connection environments.

Warning Message: “DB-API extension cursor.connection used”

Cursor.scroll(value [, mode=’relative’ ])

Scroll the cursor in the result set to a new position according to mode.

If mode is relative (default), value is taken as offset to the current position in the result set, if set to absolute, value states an absolute target position.

An IndexError should be raised in case a scroll operation would leave the result set. In this case, the cursor position is left undefined (ideal would be to not move the cursor at all).

Note

This method should use native scrollable cursors, if available, or revert to an emulation for forward-only scrollable cursors. The method may raise NotSupportedError to signal that a specific operation is not supported by the database (e.g. backward scrolling).

Warning Message: “DB-API extension cursor.scroll() used”

Cursor.messages

This is a Python list object to which the interface appends tuples (exception class, exception value) for all messages which the interfaces receives from the underlying database for this cursor.

The list is cleared by all standard cursor methods calls (prior to executing the call) except for the .fetch*() calls automatically to avoid excessive memory usage and can also be cleared by executing del cursor.messages[:].

All error and warning messages generated by the database are placed into this list, so checking the list allows the user to verify correct operation of the method calls.

The aim of this attribute is to eliminate the need for a Warning exception which often causes problems (some warnings really only have informational character).

Warning Message: “DB-API extension cursor.messages used”

Connection.messages

Same as Cursor.messages except that the messages in the list are connection oriented.

The list is cleared automatically by all standard connection methods calls (prior to executing the call) to avoid excessive memory usage and can also be cleared by executing del connection.messages[:].

Warning Message: “DB-API extension connection.messages used”

Cursor.next()

Return the next row from the currently executing SQL statement using the same semantics as .fetchone(). A StopIteration exception is raised when the result set is exhausted for Python versions 2.2 and later. Previous versions don’t have the StopIteration exception and so the method should raise an IndexError instead.

Warning Message: “DB-API extension cursor.next() used”

Cursor.__iter__()

Return self to make cursors compatible to the iteration protocol [8].

Warning Message: “DB-API extension cursor.__iter__() used”

Cursor.lastrowid

This read-only attribute provides the rowid of the last modified row (most databases return a rowid only when a single INSERT operation is performed). If the operation does not set a rowid or if the database does not support rowids, this attribute should be set to None.

The semantics of .lastrowid are undefined in case the last executed statement modified more than one row, e.g. when using INSERT with .executemany().

Warning Message: “DB-API extension cursor.lastrowid used”

Optional Error Handling Extensions

The core DB API specification only introduces a set of exceptions which can be raised to report errors to the user. In some cases, exceptions may be too disruptive for the flow of a program or even render execution impossible.

For these cases and in order to simplify error handling when dealing with databases, database module authors may choose to implement user definable error handlers. This section describes a standard way of defining these error handlers.

Connection.errorhandler, Cursor.errorhandler

Read/write attribute which references an error handler to call in case an error condition is met.

The handler must be a Python callable taking the following arguments:

errorhandler(connection, cursor, errorclass, errorvalue)

where connection is a reference to the connection on which the cursor operates, cursor a reference to the cursor (or None in case the error does not apply to a cursor), errorclass is an error class which to instantiate using errorvalue as construction argument.

The standard error handler should add the error information to the appropriate .messages attribute (Connection.messages or Cursor.messages) and raise the exception defined by the given errorclass and errorvalue parameters.

If no .errorhandler is set (the attribute is None), the standard error handling scheme as outlined above, should be applied.

Warning Message: “DB-API extension .errorhandler used”

Cursors should inherit the .errorhandler setting from their connection objects at cursor creation time.

Optional Two-Phase Commit Extensions

Many databases have support for two-phase commit (TPC) which allows managing transactions across multiple database connections and other resources.

If a database backend provides support for two-phase commit and the database module author wishes to expose this support, the following API should be implemented. NotSupportedError should be raised, if the database backend support for two-phase commit can only be checked at run-time.

Frequently Asked Questions

The database SIG often sees reoccurring questions about the DB API specification. This section covers some of the issues people sometimes have with the specification.

Question:

How can I construct a dictionary out of the tuples returned by .fetch*():

Answer:

There are several existing tools available which provide helpers for this task. Most of them use the approach of using the column names defined in the cursor attribute .description as basis for the keys in the row dictionary.

Note that the reason for not extending the DB API specification to also support dictionary return values for the .fetch*() methods is that this approach has several drawbacks:

• Some databases don’t support case-sensitive column names or auto-convert them to all lowercase or all uppercase characters.
• Columns in the result set which are generated by the query (e.g. using SQL functions) don’t map to table column names and databases usually generate names for these columns in a very database specific way.

As a result, accessing the columns through dictionary keys varies between databases and makes writing portable code impossible.

Major Changes from Version 1.0 to Version 2.0

The Python Database API 2.0 introduces a few major changes compared to the 1.0 version. Because some of these changes will cause existing DB API 1.0 based scripts to break, the major version number was adjusted to reflect this change.

These are the most important changes from 1.0 to 2.0:

• The need for a separate dbi module was dropped and the functionality merged into the module interface itself.
• New constructors and Type Objects were added for date/time values, the RAW Type Object was renamed to BINARY. The resulting set should cover all basic data types commonly found in modern SQL databases.
• New constants (apilevel, threadsafety, paramstyle) and methods (.executemany(), .nextset()) were added to provide better database bindings.
• The semantics of .callproc() needed to call stored procedures are now clearly defined.
• The definition of the .execute() return value changed. Previously, the return value was based on the SQL statement type (which was hard to implement right) — it is undefined now; use the more flexible .rowcount attribute instead. Modules are free to return the old style return values, but these are no longer mandated by the specification and should be considered database interface dependent.
• Class based exceptions were incorporated into the specification. Module implementors are free to extend the exception layout defined in this specification by subclassing the defined exception classes.

Post-publishing additions to the DB API 2.0 specification:

• Additional optional DB API extensions to the set of core functionality were specified.

Open Issues

Although the version 2.0 specification clarifies a lot of questions that were left open in the 1.0 version, there are still some remaining issues which should be addressed in future versions:

• Define a useful return value for .nextset() for the case where a new result set is available.
• Integrate the decimal module Decimal object for use as loss-less monetary and decimal interchange format.

Footnotes

[1]

As a guideline the connection constructor parameters should be implemented as keyword parameters for more intuitive use and follow this order of parameters:

Parameter	Meaning
dsn	Data source name as string
user	User name as string (optional)
password	Password as string (optional)
host	Hostname (optional)
database	Database name (optional)

E.g. a connect could look like this:

connect(dsn='myhost:MYDB', user='guido', password='234$')

[2]

Module implementors should prefer numeric, named or pyformat over the other formats because these offer more clarity and flexibility.

[3] (1, 2, 3)

If the database does not support the functionality required by the method, the interface should throw an exception in case the method is used.

The preferred approach is to not implement the method and thus have Python generate an AttributeError in case the method is requested. This allows the programmer to check for database capabilities using the standard hasattr() function.

For some dynamically configured interfaces it may not be appropriate to require dynamically making the method available. These interfaces should then raise a NotSupportedError to indicate the non-ability to perform the roll back when the method is invoked.

[4]

A database interface may choose to support named cursors by allowing a string argument to the method. This feature is not part of the specification, since it complicates semantics of the .fetch*() methods.

[5]

The module will use the __getitem__ method of the parameters object to map either positions (integers) or names (strings) to parameter values. This allows for both sequences and mappings to be used as input.

The term bound refers to the process of binding an input value to a database execution buffer. In practical terms, this means that the input value is directly used as a value in the operation. The client should not be required to “escape” the value so that it can be used — the value should be equal to the actual database value.

[6]

Note that the interface may implement row fetching using arrays and other optimizations. It is not guaranteed that a call to this method will only move the associated cursor forward by one row.

[7]

The rowcount attribute may be coded in a way that updates its value dynamically. This can be useful for databases that return usable rowcount values only after the first call to a .fetch*() method.

[8]

Implementation Note: Python C extensions will have to implement the tp_iter slot on the cursor object instead of the .__iter__() method.

[9]

The term number of affected rows generally refers to the number of rows deleted, updated or inserted by the last statement run on the database cursor. Most databases will return the total number of rows that were found by the corresponding WHERE clause of the statement. Some databases use a different interpretation for UPDATEs and only return the number of rows that were changed by the UPDATE, even though the WHERE clause of the statement may have found more matching rows. Database module authors should try to implement the more common interpretation of returning the total number of rows found by the WHERE clause, or clearly document a different interpretation of the .rowcount attribute.

Acknowledgements

Many thanks go to Andrew Kuchling who converted the Python Database API Specification 2.0 from the original HTML format into the PEP format.

Many thanks to James Henstridge for leading the discussion which led to the standardization of the two-phase commit API extensions.

Many thanks to Daniele Varrazzo for converting the specification from text PEP format to ReST PEP format, which allows linking to various parts.

Copyright

This document has been placed in the Public Domain.