populse_db.engine package¶
This Python package contains engines that contains the minimum API to connect an external database engine to a Populse database.
- class populse_db.engine.Engine¶
Bases:
object
Base class for database engines. All methods of this base class raise NotImplementedError. This class exists to list all the methods that must be implemented by an engine.
Parameters passed to __init__ may differ between engine classes. It is up to engine_factory to extract appropriate parameter(s) from URL.
- add_collection(collection, primary_key)¶
Create a new collection given the name of a field that will be created with a text type. This field is the primary key of each document. It means that, in a single collection, a value must uniquely identify a document. In other word, two documents cannot have the same value for this field.
- Parameters
collection – collection name (str)
primary_key – name of the primary key field (str)
- add_field(collection, field, type, description, index)¶
Adds a new field in a collection.
- Parameters
collection – collection name (str, must be existing)
field – new field name (str, must not be existing)
type – field type, in (‘string’, ‘int’, ‘float’, ‘boolean’, ‘date’, ‘datetime’, ‘time’, ‘json’, ‘list_string’, ‘list_int’, ‘list_float’, ‘list_boolean’, ‘list_date’, ‘list_datetime’, ‘list_time’, ‘list_json’)
description – field description (str or None)
index – boolean indicating if a database index must be created for this field to speed-ud queries involving values for this field.
- clear()¶
Erase database data and schema.
- collection(collection)¶
Returns a Row Object with at least the following items: collection_name: name of the collection primary_key: name of the primary key field More engine specific items may be present but they must not be used in the general API. Returns None if the collection does not exist.
- Parameters
collection – collection name (str)
- collections()¶
Return an iterable (e.g. list, generator, etc.) browsing the collections presents in the database. See collection() method for the format of each collection object.
- commit()¶
Store in the database all modifications done since the last call to either __enter__, commit or rollback.
- document(collection, document_id, fields=None, as_list=False)¶
Returns a Row Object corresponding to a document in the collection. The object has one item per selected fields. If fields is not given, fields returned by the fields() method are used. If as_list is True, a list of values is returned (one value per selected field). Returns None if the collection or document does not exist.
- Parameters
collection – collection name (str)
document_id – document identifier: the value of the primary key field (str)
fields – list of fields to get values from (other fields are ignored) (list of str or None)
as_list – if True, return a list of values insted of a Row Object (str)
- field(collection, field)¶
Returns a Row Object corresponding to a collection field with at least the following items: collection_name: name of the collection field_name: field name field_type: type of values for this field description: text describing the field usage has_index: boolean indicating if an index was created for this field More engine specific items may be present but they must not be used in the general API. Returns None if the collection or the field does not exist.
- Parameters
collection – collection name (str)
field – field name (str)
- fields(collection=None)¶
Return an iterable (e.g. list, generator, etc.) browsing the fields presents in the given collection (or all collections if collection parameter is None). See field() method for the format of each field object.
- Parameters
collection – collection name (str)
- filter_documents(parsed_filter, fields=None, as_list=False)¶
Iterate over document selected by a filter. See document() method for the format of a document object.
- Parameters
parsed_filter – internal object representing a filter on a collection (filter object returned by parse_filter())
fields – list of fields to get values from (other fields are ignored) (list of str or None)
as_list – if True, return a list of values insted of a Row Object (str)
- has_collection(collection)¶
Checks existence of a collection. May be called often, must be fast.
- Parameters
collection – collection name (str)
- has_document(collection, document_id)¶
Checks existence of a document in a collection.
- Parameters
collection – collection name (str)
document_id – document identifier: the value of the primary key field (str)
- has_field(collection, field)¶
Checks existence of a field in a collection. May be called often, must be fast. Returns False if collection does not exist.
- Parameters
collection – collection name (str)
field – field name (str)
- has_value(collection, document_id, field)¶
Check if a document has a not null value for a given field.
- parse_filter(collection, filter)¶
Given a filter string, return a internal query representation that can be used with filter_documents() to select documents
- Parameters
collection – the collection for which the filter is intended (str, must be existing)
filter – the selection string using the populse_db selection language.
- primary_key(collection)¶
Return the name of the primary key of a collection. Returns None if the collection does not exist.
- Parameters
collection – collection name (str)
- remove_collection(collection)¶
Delete a collection and its data.
- Parameters
collection – collection name (str)
- remove_document(collection, document_id)¶
Remove a document from a collection.
- Parameters
collection – collection name (str)
document_id – document identifier: the value of the primary key field (str)
- remove_fields(collection, fields)¶
Remove given fields from a collection as well as all corresponding data.
- Parameters
collection – collection name (str)
fields – field name (str)
- remove_value(collection, document_id, field)¶
Remove a value from a document (setting its value to None).
- Parameters
collection – collection name (str)
document_id – document identifier: the value of the primary key field (str)
fields – field name (str)
- rollback()¶
Store discards all database modifications done since the last call to either __enter__, commit or rollback.
- set_values(collection, document_id, values)¶
Change some values in an existing document.
- Parameters
collection – collection name (str)
document_id – document identifier: the value of the primary key field (str)
values – dictionary with field/value pairs (dict)
- populse_db.engine.engine_factory(database_url)¶
Method that interprets an URL and creates the corresponding database engine.
Submodules¶
populse_db.engine.sqlite module¶
- class populse_db.engine.sqlite.FilterToSqliteQuery(engine, collection)¶
Bases:
populse_db.filter.FilterToQuery
Implements required methods to produce a SQLite query given a document selection filter. This class returns either None (all documents are selected) or an SQL WHERE clause (without the WHERE keyword) as a list of string (that must be joined with spaces). This WHERE clause is useable with a SELECT from the table containing the collection documents. Using a list for combining strings is supposed to be more efficient (especially for long queries).
Create a parser for a givent engine and collection
- build_condition_all()¶
Return a selection query that select all documents. This query is directly given to the engine and never combined with other queries.
- build_condition_combine_conditions(left_condition, operator_str, right_condition)¶
Builds a condition that combines two conditions with an operator.
- Parameters
left_condition – condition object returned by one of the build_condition_*() method (except build_condition_all)
operator_str – string containing one of the BOOLEAN_OPERATOR defined in the grammar (in lowercase)
right_condition – condition object returned by one of the build_condition_*() method (except build_condition_all)
- build_condition_field_in_list(field, list_value)¶
Builds a condition checking if a field value is a constant list value
- Parameters
field – field object as returned by Database.get_field
list_value – Pyton list containing literals
- build_condition_field_in_list_field(field, list_field)¶
Builds a condition checking if a field value is in another list field value
- Parameters
field – field object as returned by Database.get_field
list_field – field object as returned by Database.get_field
- build_condition_field_op_field(left_field, operator_str, right_field)¶
Builds a condition comparing the content of two fields with an operator.
- Parameters
left_field – field object as returned by Database.get_field
operator – string containing one of the CONDITION_OPERATOR defined in the grammar (in lowercase)
right_field – field object as returned by Database.get_field
- build_condition_field_op_value(field, operator_str, value)¶
Builds a condition comparing the content of a field with a constant value using an operator.
- Parameters
field – field object as returned by Database.get_field
operator_str – string containing one of the CONDITION_OPERATOR defined in the grammar (in lowercase)
value – Python value (None, string number, boolean or date/time)
- build_condition_literal_in_list_field(value, list_field)¶
Builds a condition checking if a constant value is in a list field
- Parameters
value – Python literal
list_field – field object as returned by Database.get_field
- build_condition_negation(condition)¶
Builds a condition inverting another condition.
- Parameters
condition – condition object returned by one of the build_condition_*() method (except build_condition_all)
- build_condition_value_op_field(value, operator_str, field)¶
Builds a condition comparing a constant value with the content of a field withusing an operator.
- Parameters
value – Python value (None, string number, boolean or date/time)
operator_str – string containing one of the CONDITION_OPERATOR defined in the grammar (in lowercase)
field – field object as returned by Database.get_field
- get_column(field)¶
- Returns
The SQL representation of a field object.
- get_column_value(python_value)¶
Converts a Python value to a value suitable to put in a database column
- no_list_operator = {'<', '<=', '>', '>=', 'ilike', 'like'}¶
- sql_operators = {'!=': 'IS NOT', '==': 'IS', 'ilike': 'LIKE'}¶
- class populse_db.engine.sqlite.SQLiteEngine(database)¶
Bases:
populse_db.engine.Engine
Parameters passed to __init__ may differ between engine classes. It is up to engine_factory to extract appropriate parameter(s) from URL.
- add_collection(collection, primary_key)¶
Create a new collection given the name of a field that will be created with a text type. This field is the primary key of each document. It means that, in a single collection, a value must uniquely identify a document. In other word, two documents cannot have the same value for this field.
- Parameters
collection – collection name (str)
primary_key – name of the primary key field (str)
- add_document(collection, document, create_missing_fields)¶
- add_field(collection, field, type, description, index)¶
Adds a new field in a collection.
- Parameters
collection – collection name (str, must be existing)
field – new field name (str, must not be existing)
type – field type, in (‘string’, ‘int’, ‘float’, ‘boolean’, ‘date’, ‘datetime’, ‘time’, ‘json’, ‘list_string’, ‘list_int’, ‘list_float’, ‘list_boolean’, ‘list_date’, ‘list_datetime’, ‘list_time’, ‘list_json’)
description – field description (str or None)
index – boolean indicating if a database index must be created for this field to speed-ud queries involving values for this field.
- clear()¶
Erase database data and schema.
- collection(collection)¶
Returns a Row Object with at least the following items: collection_name: name of the collection primary_key: name of the primary key field More engine specific items may be present but they must not be used in the general API. Returns None if the collection does not exist.
- Parameters
collection – collection name (str)
- collections()¶
Return an iterable (e.g. list, generator, etc.) browsing the collections presents in the database. See collection() method for the format of each collection object.
- static column_to_python(field_type, value)¶
- commit()¶
Store in the database all modifications done since the last call to either __enter__, commit or rollback.
- document(collection, document_id, fields=None, as_list=False)¶
Returns a Row Object corresponding to a document in the collection. The object has one item per selected fields. If fields is not given, fields returned by the fields() method are used. If as_list is True, a list of values is returned (one value per selected field). Returns None if the collection or document does not exist.
- Parameters
collection – collection name (str)
document_id – document identifier: the value of the primary key field (str)
fields – list of fields to get values from (other fields are ignored) (list of str or None)
as_list – if True, return a list of values insted of a Row Object (str)
- field(collection, field)¶
Returns a Row Object corresponding to a collection field with at least the following items: collection_name: name of the collection field_name: field name field_type: type of values for this field description: text describing the field usage has_index: boolean indicating if an index was created for this field More engine specific items may be present but they must not be used in the general API. Returns None if the collection or the field does not exist.
- Parameters
collection – collection name (str)
field – field name (str)
- fields(collection=None)¶
Return an iterable (e.g. list, generator, etc.) browsing the fields presents in the given collection (or all collections if collection parameter is None). See field() method for the format of each field object.
- Parameters
collection – collection name (str)
- filter_documents(parsed_filter, fields=None, as_list=False)¶
Iterate over document selected by a filter. See document() method for the format of a document object.
- Parameters
parsed_filter – internal object representing a filter on a collection (filter object returned by parse_filter())
fields – list of fields to get values from (other fields are ignored) (list of str or None)
as_list – if True, return a list of values insted of a Row Object (str)
- has_collection(collection)¶
Checks existence of a collection. May be called often, must be fast.
- Parameters
collection – collection name (str)
- has_document(collection, document_id)¶
Checks existence of a document in a collection.
- Parameters
collection – collection name (str)
document_id – document identifier: the value of the primary key field (str)
- has_field(collection, field)¶
Checks existence of a field in a collection. May be called often, must be fast. Returns False if collection does not exist.
- Parameters
collection – collection name (str)
field – field name (str)
- has_table(table)¶
- has_value(collection, document_id, field)¶
Check if a document has a not null value for a given field.
- static list_hash(list)¶
- name_to_sql(name)¶
Transforms the name into a valid and unique SQLite table/column name. Since all names are quoted in SQL with ‘[]’, there is no restriction on character that can be used. However, case is insignificant in SQLite. Therefore, all upper case characters are prefixed with ‘!’.
- Parameters
name – Name (str)
- Returns
Valid and unique table/column name
- parse_filter(collection, filter)¶
Given a filter string, return a internal query representation that can be used with filter_documents() to select documents
- Parameters
collection – the collection for which the filter is intended (str, must be existing)
filter – the selection string using the populse_db selection language.
- primary_key(collection)¶
Return the name of the primary key of a collection. Returns None if the collection does not exist.
- Parameters
collection – collection name (str)
- static python_to_column(field_type, value)¶
Converts a python value into a suitable value to put in a database column.
- remove_collection(collection)¶
Delete a collection and its data.
- Parameters
collection – collection name (str)
- remove_document(collection, document_id)¶
Remove a document from a collection.
- Parameters
collection – collection name (str)
document_id – document identifier: the value of the primary key field (str)
- remove_fields(collection, fields)¶
Remove given fields from a collection as well as all corresponding data.
- Parameters
collection – collection name (str)
fields – field name (str)
- remove_value(collection, document_id, field)¶
Remove a value from a document (setting its value to None).
- Parameters
collection – collection name (str)
document_id – document identifier: the value of the primary key field (str)
fields – field name (str)
- rollback()¶
Store discards all database modifications done since the last call to either __enter__, commit or rollback.
- set_values(collection, document_id, values)¶
Change some values in an existing document.
- Parameters
collection – collection name (str)
document_id – document identifier: the value of the primary key field (str)
values – dictionary with field/value pairs (dict)
- sql_type(type)¶
- type_to_sql = {'boolean': 'BOOLEAN', 'date': 'TEXT', 'datetime': 'TEXT', 'float': 'REAL', 'int': 'INT', 'json': 'STRING', 'string': 'TEXT', 'time': 'TEXT'}¶