graphene-elastic¶
Elasticsearch (DSL) integration for Graphene.
Note
Project status is alpha.
Prerequisites¶
- Graphene 2.x. Support for Graphene 1.x is not planned, but might be considered.
- Python 3.6, 3.7. Support for Python 2 is not intended.
- Elasticsearch 6.x, 7.x. Support for Elasticsearch 5.x is not intended.
Main features and highlights¶
- Implemented
ElasticsearchConnectionField
andElasticsearchObjectType
are the core classes to work withgraphene
. - Pluggable backends for searching, filtering, ordering, etc. Don’t like existing ones? Override, extend or write your own.
- Implemented search backend.
- Implemented filter backend.
- Implemented ordering backend.
- Implemented pagination.
See the Road-map for what’s yet planned to implemented.
Documentation¶
Documentation is available on Read the Docs.
Installation¶
Install latest stable version from PyPI:
pip install graphene-elastic
Or latest development version from GitHub:
pip install https://github.com/barseghyanartur/graphene-elastic/archive/master.zip
Examples¶
Install requirements¶
pip install -r requirements.txt
Populate sample data¶
The following command will create indexes for User
and Post
documents
and populate them with sample data:
./scripts/populate_elasticsearch_data.sh
Sample document definition¶
search_index/documents/post.py
See examples/search_index/documents/post.py for full example.
import datetime
from elasticsearch_dsl import (
Boolean,
Date,
Document,
InnerDoc,
Keyword,
Nested,
Text,
Integer,
)
class Comment(InnerDoc):
author = Text(fields={'raw': Keyword()})
content = Text(analyzer='snowball')
created_at = Date()
def age(self):
return datetime.datetime.now() - self.created_at
class Post(Document):
title = Text(
fields={'raw': Keyword()}
)
content = Text()
created_at = Date()
published = Boolean()
category = Text(
fields={'raw': Keyword()}
)
comments = Nested(Comment)
tags = Text(
analyzer=html_strip,
fields={'raw': Keyword(multi=True)},
multi=True
)
num_views = Integer()
class Index:
name = 'blog_post'
settings = {
'number_of_shards': 1,
'number_of_replicas': 1,
'blocks': {'read_only_allow_delete': None},
}
Sample apps¶
Sample Flask app¶
Run the sample Flask app:
./scripts/run_flask.sh
Open Flask graphiql client
http://127.0.0.1:8001/graphql
Sample Django app¶
Run the sample Django app:
./scripts/run_django.sh runserver
Open Django graphiql client
http://127.0.0.1:8000/graphql
ConnectionField example¶
ConnectionField is the most flexible and feature rich solution you have. It uses filter backends which you can tie to your needs the way you want in a declarative manner.
Sample schema definition
import graphene
from graphene_elastic import (
ElasticsearchObjectType,
ElasticsearchConnectionField,
)
from graphene_elastic.filter_backends import (
FilteringFilterBackend,
SearchFilterBackend,
OrderingFilterBackend,
DefaultOrderingFilterBackend,
)
from graphene_elastic.constants import (
LOOKUP_FILTER_PREFIX,
LOOKUP_FILTER_TERM,
LOOKUP_FILTER_TERMS,
LOOKUP_FILTER_WILDCARD,
LOOKUP_QUERY_EXCLUDE,
LOOKUP_QUERY_IN,
)
# Object type definition
class Post(ElasticsearchObjectType):
class Meta(object):
document = PostDocument
interfaces = (Node,)
filter_backends = [
FilteringFilterBackend,
SearchFilterBackend,
OrderingFilterBackend,
DefaultOrderingFilterBackend,
]
# For `FilteringFilterBackend` backend
filter_fields = {
# The dictionary key (in this case `title`) is the name of
# the corresponding GraphQL query argument. The dictionary
# value could be simple or complex structure (in this case
# complex). The `field` key points to the `title.raw`, which
# is the field name in the Elasticsearch document
# (`PostDocument`). Since `lookups` key is provided, number
# of lookups is limited to the given set, while term is the
# default lookup (as specified in `default_lookup`).
'title': {
'field': 'title.raw',
# Available lookups
'lookups': [
LOOKUP_FILTER_TERM,
LOOKUP_FILTER_TERMS,
LOOKUP_FILTER_PREFIX,
LOOKUP_FILTER_WILDCARD,
LOOKUP_QUERY_IN,
LOOKUP_QUERY_EXCLUDE,
],
# Default lookup
'default_lookup': LOOKUP_FILTER_TERM,
},
# The dictionary key (in this case `category`) is the name of
# the corresponding GraphQL query argument. Since no lookups
# or default_lookup is provided, defaults are used (all lookups
# available, term is the default lookup). The dictionary value
# (in this case `category.raw`) is the field name in the
# Elasticsearch document (`PostDocument`).
'category': 'category.raw',
# The dictionary key (in this case `tags`) is the name of
# the corresponding GraphQL query argument. Since no lookups
# or default_lookup is provided, defaults are used (all lookups
# available, term is the default lookup). The dictionary value
# (in this case `tags.raw`) is the field name in the
# Elasticsearch document (`PostDocument`).
'tags': 'tags.raw',
# The dictionary key (in this case `num_views`) is the name of
# the corresponding GraphQL query argument. Since no lookups
# or default_lookup is provided, defaults are used (all lookups
# available, term is the default lookup). The dictionary value
# (in this case `num_views`) is the field name in the
# Elasticsearch document (`PostDocument`).
'num_views': 'num_views',
}
# For `SearchFilterBackend` backend
search_fields = {
'title': {'boost': 4},
'content': {'boost': 2},
'category': None,
}
# For `OrderingFilterBackend` backend
ordering_fields = {
# The dictionary key (in this case `tags`) is the name of
# the corresponding GraphQL query argument. The dictionary
# value (in this case `tags.raw`) is the field name in the
# Elasticsearch document (`PostDocument`).
'title': 'title.raw',
# The dictionary key (in this case `created_at`) is the name of
# the corresponding GraphQL query argument. The dictionary
# value (in this case `created_at`) is the field name in the
# Elasticsearch document (`PostDocument`).
'created_at': 'created_at',
# The dictionary key (in this case `num_views`) is the name of
# the corresponding GraphQL query argument. The dictionary
# value (in this case `num_views`) is the field name in the
# Elasticsearch document (`PostDocument`).
'num_views': 'num_views',
}
# For `DefaultOrderingFilterBackend` backend
ordering_defaults = (
'-num_views', # Field name in the Elasticsearch document
'title.raw', # Field name in the Elasticsearch document
)
# Query definition
class Query(graphene.ObjectType):
all_post_documents = ElasticsearchConnectionField(Post)
# Schema definition
schema = graphene.Schema(query=Query)
Filter¶
Sample queries¶
Since we didn’t specify any lookups on category, by default all lookups
are available and the default lookup would be term
. Note, that in the
{value:"Elastic"}
part, the value
stands for default lookup, whatever
it has been set to.
query PostsQuery {
allPostDocuments(filter:{category:{value:"Elastic"}}) {
edges {
node {
id
title
category
content
createdAt
comments
}
}
}
}
But, we could use another lookup (in example below - terms
). Note, that
in the {terms:["Elastic", "Python"]}
part, the terms
is the lookup
name.
query PostsQuery {
allPostDocuments(
filter:{category:{terms:["Elastic", "Python"]}}
) {
edges {
node {
id
title
category
content
createdAt
comments
}
}
}
}
Or apply a gt
(range
) query in addition to filtering:
{
allPostDocuments(filter:{
category:{term:"Python"},
numViews:{gt:"700"}
}) {
edges {
node {
category
title
comments
numViews
}
}
}
}
Implemented filter lookups¶
The following lookups are available:
contains
ends_with
(orendsWith
for camelCase)exclude
exists
gt
gte
in
is_null
(orisNull
for camelCase)lt
lte
prefix
range
starts_with
(orstartsWith
for camelCase)term
terms
wildcard
See dedicated documentation on filter lookups for more information.
Search¶
Search in all fields:
query {
allPostDocuments(
search:{query:"Release Box"}
) {
edges {
node {
category
title
content
}
}
}
}
Search in specific fields:
query {
allPostDocuments(
search:{
title:{value:"Release", boost:2},
content:{value:"Box"}
}
) {
edges {
node {
category
title
content
}
}
}
}
Ordering¶
Possible choices are ASC
and DESC
.
query {
allPostDocuments(
filter:{category:{term:"Photography"}},
ordering:{title:ASC}
) {
edges {
node {
category
title
content
numViews
tags
}
}
}
}
Pagination¶
The first
, last
, before
and after
arguments are supported.
By default number of results is limited to 100.
query {
allPostDocuments(first:12) {
pageInfo {
startCursor
endCursor
hasNextPage
hasPreviousPage
}
edges {
cursor
node {
category
title
content
numViews
}
}
}
}
Road-map¶
Road-map and development plans.
Lots of features are planned to be released in the upcoming Beta releases:
- Geo-spatial backend
- Filter lookup
geo_bounding_box
(orgeoBoundingBox
for camelCase) - Filter lookup
geo_distance
(orgeoDistance
for camelCase) - Filter lookup
geo_polygon
(orgeoPolygon
for camelCase) - Aggregations (faceted search) backend
- Post-filter backend
- Nested backend
- Highlight backend
- Suggester backend
- Global aggregations backend
- More-like-this backend
- Complex search backends, such as Simple query search
- Source filter backend
Stay tuned or reach out if you want to help.
Testing¶
Project is covered with tests.
By defaults tests are executed against the Elasticsearch 7.x.
Running tests¶
Make sure you have the test requirements installed:
pip install -r requirements/test.txt
To test with all supported Python versions type:
tox
To test against specific environment, type:
tox -e py37
To test just your working environment type:
./runtests.py
To run a single test module in your working environment type:
./runtests.py src/graphene_elastic/tests/test_filter_backend.py
To run a single test class in a given test module in your working environment type:
./runtests.py src/graphene_elastic/tests/test_filter_backend.py::FilterBackendElasticTestCase
Debugging¶
For development purposes, you could use the flask app (easy to debug). Standard
pdb
works (import pdb; pdb.set_trace()
). If ipdb
does not work
well for you, use ptpdb
.
Writing documentation¶
Keep the following hierarchy.
=====
title
=====
header
======
sub-header
----------
sub-sub-header
~~~~~~~~~~~~~~
sub-sub-sub-header
^^^^^^^^^^^^^^^^^^
sub-sub-sub-sub-header
++++++++++++++++++++++
sub-sub-sub-sub-sub-header
**************************
Author¶
Artur Barseghyan <artur.barseghyan@gmail.com>
Project documentation¶
Contents:
Table of Contents
- graphene-elastic
- Prerequisites
- Main features and highlights
- Documentation
- Installation
- Examples
- Road-map
- Testing
- Debugging
- Writing documentation
- License
- Support
- Author
- Project documentation
- Concepts
- Quick start
- Search
- Filtering
- Filter lookups
- Filter lookup
contains
- Filter lookup
ends_with
- Filter lookup
exclude
- Filter lookup
exists
- Filter lookup
gt
- Filter lookup
gte
- Filter lookup
in
- Filter lookup
lt
- Filter lookup
lte
- Filter lookup
prefix
- Filter lookup
range
- Filter lookup
starts_with
- Filter lookup
term
- Filter lookup
terms
- Filter lookup
wildcard
- Filter lookup
- Filter lookups
- Ordering
- Pagination
- Settings
- Running Elasticsearch
- FAQ
- Debugging
- Release history and notes
- graphene_elastic package
- Subpackages
- Submodules
- graphene_elastic.advanced_types module
- graphene_elastic.constants module
- graphene_elastic.converter module
- graphene_elastic.enums module
- graphene_elastic.fields module
- graphene_elastic.helpers module
- graphene_elastic.registry module
- graphene_elastic.settings module
- graphene_elastic.utils module
- Module contents
- Indices and tables
- Concepts
- Quick start
- Search
- Filtering
- Filter lookups
- Filter lookup
contains
- Filter lookup
ends_with
- Filter lookup
exclude
- Filter lookup
exists
- Filter lookup
gt
- Filter lookup
gte
- Filter lookup
in
- Filter lookup
lt
- Filter lookup
lte
- Filter lookup
prefix
- Filter lookup
range
- Filter lookup
starts_with
- Filter lookup
term
- Filter lookup
terms
- Filter lookup
wildcard
- Filter lookup
- Filter lookups
- Ordering
- Pagination
- Settings
- Running Elasticsearch
- FAQ
- Debugging
- Release history and notes
- graphene_elastic package
- Subpackages
- Submodules
- graphene_elastic.advanced_types module
- graphene_elastic.constants module
- graphene_elastic.converter module
- graphene_elastic.enums module
- graphene_elastic.fields module
- graphene_elastic.helpers module
- graphene_elastic.registry module
- graphene_elastic.settings module
- graphene_elastic.utils module
- Module contents