章节简介

There are two aspects to preserving data for long-term use: converting the data back and forth between the object in-memory and the storage format, and working with the storage of the converted data. The standard library includes a variety of modules that handle both aspects in different situations.

Two modules convert objects into a format that can be transmitted or stored (a process known as serializing). It is most common to use pickle for persistence, since it is integrated with some of the other standard library modules that actually store the serialized data, such as shelve. json is more frequently used for web-based applications, however, since it integrates better with existing web service storage tools.

Once the in-memory object is converted to a format that can be saved, the next step is to decide how to store the data. A simple flat-file with serialized objects written one after the other works for data that does not need to be indexed in any way. Python includes a collection of modules for storing key-value pairs in a simple database using one of the DBM format variants when an indexed lookup is needed.

The most straightforward way to take advantage of the DBM format is shelve. Open the shelve file, and access it through a dictionary-like API. Objects saved to the database are automatically pickled and saved without any extra work by the caller.

One drawback of shelve, though, is that when using the default interface there is no way to predict which DBM format will be used, since it selects one based on the libraries available on the system where the database is created. The format does not matter if an application will not need to share the database files between hosts with different libraries, but if portability is a requirement, use one of the classes in the module to ensure a specific format is selected.

For web applications that work with data in JSON already, using json and dbm provides another persistence mechanism. Using dbm directly is a little more work than shelve because the DBM database keys and values must be strings, and the objects will not be re-created automatically when the value is accessed in the database.

The sqlite3 in-process relational database is available with most Python distributions for storing data in more complex arrangements than key/value pairs. It stores its database in memory or in a local file, and all access is from within the same process so there is no network communication lag. The compact nature of sqlite3 makes it especially well suited for embedding in desktop applications or development versions of web apps.

There are also modules for parsing more formally defined formats, useful for exchanging data between Python programs and applications written in other languages. xml.etree.ElementTree can parse XML documents, and provides several operating modes for different applications. Besides the parsing tools, ElementTree includes an interface for creating well-formed XML documents from objects in memory. The csv module can read and write tabular data in formats produced by spreadsheets or database applications, making it useful for bulk loading data, or converting the data from one format to another.